The fastest way to start sharing code between projects

May 18, 2020

Today let’s talk about simple and effective sharing code between repositories which Snake CI brings in the 0.4.0 version.

Extracting shared code into a separate repository

As a software product grows, the codebase grows too, which inevitably lead to a code duplication of a shared code used in different parts of the project.

A code duplication leads to bugs, makes project structure less organized, which eventually results in a much slower release cycle and overall quality drop.

The common approach to handle shared code duplication is to extract it into a separate repository as a library with its own release cycle.

The extracted library can be linked to the project using either Git submodules or package managers like npm.

Managing dependencies using Git submodules

Git submodules allow you to put a Git repository as a subdirectory of another Git repository. It lets you include another repository into your project and keep your development history separate.

Git submodules are language agnostic and do not require any other tools except git.

To link library code back to the main project using submodules use git submodule add command:

git submodule add git+ssh://git@bitbucket.company:7999/libraries/our-lib.git \
    deps/our-lib

git commit -m "integrate our-lib"

In this example, we use the previously extracted shared library called our-lib and add it back to the main project.

Submodules are the most straightforward approach which works with Snake CI out of the box without any additional configuration.

Submodules code appears in the CI environment right after the clone step.

Managing dependencies using a package manager

It is also possible to use your preferred package manager to manage dependencies.

Let’s see how to add extracted libraries using npm as an example package manager.

First, we need to add our-lib into the main project by executing npm install and providing it with the path to the repository hosting shared library:

npm install -S 'git+ssh://git@bitbucket.company:7999/libraries/our-lib.git'

However, to fetch this dependency later in the CI pipeline, an additional step is required.

Snake CI automatically manages access keys to the repositories, and the CI pipeline has the same access as a user that triggered the pipeline, you don't need to add any SSH keys on your own. However, to successfully fetch the dependencies from the Bitbucket Server, we need to make sure that SSH in the build container is properly configured.

By default, SSH requires to add any remote server to the trusted list manually. Since we’re going to fetch dependencies from the Bitbucket instance and this instance is trusted, we can skip remote server verification step by specifying the global GIT_SSH_COMMAND environment variable in the snake-ci.yaml:

variables:
    GIT_SSH_COMMAND: ssh
        -o UserKnownHostsFile=/dev/null
        -o StrictHostKeyChecking=no

When the GIT_SSH_COMMAND environment variable is set, npm install can be used as usual to fetch all dependencies.

A complete snake-ci.yaml may look like this:

variables:
    GIT_SSH_COMMAND: ssh
        -o UserKnownHostsFile=/dev/null
        -o StrictHostKeyChecking=no

stages:
    # other stages listed here
    - deps

# other jobs defined here

fetch npm dependencies:
    stage: deps
    image: node
    commands:
        - npm install

This is all that is required to configure Snake CI to fetch dependencies from the Bitbucket instance.

Check out full tutorial in our docs.

Version control for shared libraries

Code deduplication for shared libraries works well by itself, but without proper version management it may lead to a new set of problems, such as implicit library updates that cause breaking changes in the product.

To avoid these problems altogether, you need to version and publish shared libraries accordingly.

One of the popular versioning approaches is called semantic versioning, which suggests using the following version schema MAJOR.MINOR.PATCH:

  • MAJOR is incremented when you make incompatible changes,
  • MINOR is incremented when you add functionality in a backward-compatible manner, and
  • PATCH (optional) is incremented when you make backward-compatible bug fixes.

Using this approach, we can tag individual commits with specific versions
using git tag command:

git tag v1.2.3  # will tag current master with v1.2.3 tag
git push --tags # will push tags to the remote server (note --tags)

Snake CI can be configured to automatically publish a tagged version to a private package repository (like Sinopia) with the use of the only section that now supports the regular expressions syntax.

For example, you can use the regular expression syntax in the condition to include into a pipeline the job which publishes stable release only when the tag like v1.2.3 is specified.

We need to define a job that publishes stable release. In this example,
we use npm publish in the publish stable release job definition.

stages:
    # other stages listed here
    - deploy

# other jobs defined here

publish stable release:
    stage: deploy
    only:
        tags:
            - /^v\d+/
    commands:
        - npm publish # publishes package to the private or public registry

The tags parameter means that job will be executed only if the listed tags are changed.

Note the /^v\d+/ expression in the only: tags section.

The surrounding / character means that the tag name will be matched against the given regular expression. If you're not familiar with regular expressions, ^v\d+ means, that an incoming tag name must be started with the letter v that is followed by at least one number (e.g., v5):

  • ^ requires match from the beginning of the tag name,
  • v is just plain letter,
  • \d means any number in range 0–9,
  • + means that previous subpattern (in our case it's 0–9) may be repeated any number of times, but at least once.

If the tag name doesn’t conform to the specified pattern, then the regular expression doesn’t match, and the publish stable release job is skipped.

Therefore, to trigger the described job, we need to create a tag in the Git repository using git tag command, like this: git tag v1.2.3; and then push it to Bitbucket using git push --tags.

Learn more about the regular expressions support.

Conclusions

To accommodate with needs of growing software projects and to open possibilities for tackling an increasing complexity, we’ve introduced the new version
of Snake CI — 0.4.0, that includes:

The full release notes are available here.

If you need assistance or want to say hi, contact us:

Start building faster. Today.

Free 30-days trial.