Nick Sellen

For a recent project I was wondering what the best way to share node.js code across projects without publishing modules publically to npm.

The node.js ecosystem is all about sharing code publically and it’s super easy to do this entirely from the comfort of the command line with npm publish and friends. It’s quite a joy actually.

However, sometimes the code must stay private, but you still want nice modular npm modules. This scenario is less well supported by npm - npm themselves says:

Lots of companies using Node.js love the “many small modules” pattern that is part of the Node culture. However, splitting internal applications and private code up into small modules has been inconvenient, requiring git dependencies or other workarounds to avoid publishing sensitive code to the public registry.

Difficulties related this topic are well discussed:

• https://gist.github.com/branneman/8048520#comment-972519 - this is about 1.5 years worth of discussion on the topic, including some of the key node.js community
• http://injoin.io/2015/01/31/nodejs-require-problem-es6.html
• https://github.com/npm/npm/issues/7426

I actually found a solution pretty quickly that I’m happy with but I wanted to dig a bit deeper into the alternatives.

Terminology key:

• repo = e.g. a git project
• module = an npm module / npm package (what are these?)
• library = module that contains reusable library code
• application = module that contains an application

Some of the factors to consider:

• what goes require('<here>')
• what goes in package.json dependency configuration {"here":"and here"}
• what goes npm install <here>
• how does updating work
• does it require creating a lot of seperate npm modules
• implications for filesystem layout
• are your libraries stored clearly and seperately from external libraries
• will your editor try and find across files in external libraries
• will somebody else be able to simply npm install it
• will it be easy to change which technique you use in the future
• will it work cross platform

The options

As with many things in node.js/npm world, there are many ways to skin this cat:

1. have a lib folder insider the application and require('../../lib/foo') (or however deep you are)
2. put them in local node_modules and add a .gitignore exclusion - as recommended by Isaac Schluter in 2012
3. put sharable libs in a node_modules directory higher up the filesystem - like in this Stack Overflow answer
4. npm install from a relative path - like in this Stack Overflow answer
5. require() from a relative path
6. use/abuse npm link - like in this tutorial from justjs.com or this Stack Overflow answer
7. use a private git repo via git+ssh protocol
8. pay $7/month for npm private modules (unlimited number of modules)
9. pay from $20/month for npm Enterprise
10. host your own private npm repository - guide from clock
11. use a third party private module system - such as private module manager
12. use a third party private npm repositry - such as gemfury
13. hack something together with glue and sellotape (well, scripts, postinstall, etc)

This is a lot of methods to evaluate, but two stand out to me right now:

put sharable libs in a node_modules directory higher up the filesystem

This builds on node.js’s default inclination to go up the filesystem hierarchy looking for node_modules directories.

Advantages:

• can use just one repo (containing multiple libraries/applications)
• maintains seperation of external libraries and your libraries
• simple to use the library in an application, just require() it by its name - it does not need to be declared as a dependency in package.json
• very low cost to writing lots of small libraries - no special process after making change in library
• if you evolve a library into a seperately managed repo/module - you’ll be able to just npm install --save it at it’s new home and all the existing code will carry on working
• free and uses only features built-in to node.js that aren’t going away anytime soon

Disadvantages:

• putting many applications/libraries in one repo is less modular
• deployment/build is less neat, either have to checkout all applications/libraries for each deployment/build, or have scripts to pull out only the applcation you need and include all the libraries (npm can’t help you here as they weren’t declared in package.json)

use a private git repo via git+ssh protocol

Advantages:

• library can be managed in it’s own repo
• access control via ssh keys
• can self host any number of applications/libraries with nothing more than an ssh server
• can pin dependency to a specific commit/tag
• npm update-able
• supported by npm itself, nothing third party involved

Disadvatages:

• might have to maintain a git server - and it might be gitlab or something complicated
• have to pay if using private github repo
• access control via ssh keys (need to create/distribute keys)
• more moving parts to co-ordinate

I’d welcome any thoughts, corrections, additions, and particularly any good/bad experiences people have had in real-world usage.