Using local NPM package on open source Travis CI

When creating software with multiple contributors I find it utterly important to have a continuous integration server running at all times. It makes the health of a project very visible if combined with badges. As an example check the below badges from some popular open source projects on GitHub.




In a recent project I hit an issue when settings this up. We have a dependency to a package that is distributed as a .tgz file. Further more it is a closed source packages which made things a bit complicated to use on a CI server.

Container based CI

We are using the excellent Travis CI service that is a free open source continuous integration service. It is container based which is great and fast, but actually is also part of the problem. On every build a new container is started, the repository is cloned and the build starts. This way it is not possible to just put the file on the CI and be happy.

Problem

The problem comes down to this. I needed a way for the Travis CI to get a hold of the package file before running npm install. It had to be done in a way where the package is kept protected from the public but available to Travis CI on every build.

Solution

I did a bit of thinking and came up with the following solution. If I could place the package on a web server and protect it with some credentials, the Travis CI server could fetch it from here.

I created a private repository fore the sole purpose to store the closed source package and make it available over the internet. This could have been any kind of web server, but I decided this was the easiest solution.

Then added a script to my open source repository that is run before the installation step on Travis. The script fetches the package and places it in the root of the working folder on the container and it uses environment variables for the credentials to the private repository. This way the source code do not give away any information except the location of the package.

The script is a simple curl command as seen below. The initial if condition is to stop the build if the user is not set on the server.

#!/bin/bash
if [ "$PRIVATE_USER" = "" ];
then
  echo "ERROR: Can't get package. User not set";
  exit;
fi
 
curl --trace /dev/null -s -O -u $PRIVATE_USER {URL_TO_RESOURCE}

The environment variable in use is PRIVATE_USER and is formatted as USER:PASSWORD.

The {URL_TO_RESOURCE} should be replaced with the URL of the package to get, e.g. for GitHub this could be https://raw.githubusercontent.com/user/repo/master/my-package.tgz.

Only thing left is to enable the script on the Travis CI before_install hook.

before_install:
  - sh get-local-package.sh

Posted in CI/CD, JavaScript
Write a comment