Using private Docker registries

Pull from and push to private Docker registries.

Snake Runner supports pulling from private Docker registries since version 0.8.1.

Introduction

Two common use cases include:

  • Pulling a build image from a private registry.

    The path to a private image is specified in the image parameter in the snake-ci.yaml file, for example, when using Google Cloud Container Registry:

    build code:
        stage: build
        image: gcr.io/<project-name>/<build-image>:latest
    
  • Pushing a final product image to the private registry.

    To set a target private registry image should be tagged with the full path to the registry, for example, by using the --tag parameter for the docker build command. You can also use the docker tag command to tag the image.

    build and push image:
        stage: image
        image: docker
        commands:
            - # login to the docker registry
            - docker build --tag gcr.io/<project-name>/<product-image>:latest …
            - docker push gcr.io/<project-name>/<product-image>:latest
    

In this tutorial, we’re going to discuss how to configure Runner and your pipeline to pull and push from a private Docker registry.

To be able to pull from the private registry, Runner should be aware of access credentials.

Runner uses two special environment variables named DOCKER_AUTH_CONFIG and SNAKE_DOCKER_AUTH_CONFIG which should contain entire contents of the .docker/config.json file:

  • DOCKER_AUTH_CONFIG can be specified as usual environment variable at project, repository, pipeline, or job levels.

  • SNAKE_DOCKER_AUTH_CONFIG may be specified only on the runner start.
    Use this variable to declare global access to the private registries for all projects and repositories.
    Check out runner installation instructions for more details.

Runner merges authentication parameters from both variables, and values which are specified in the DOCKER_AUTH_CONFIG take precedence.

The easiest way to obtain the correct value for those environment variables is to use docker login on the local machine and then copy the contents of ~/.docker/config.json.

Preparation

First, authenticate to the private registry from the local machine using the docker login command.

To avoid changes in your local .docker/config.json file, pass --config flag to the docker login with a directory name which will contain config.json with only required credentials.

  • For user/password authentication use docker login with your registry address, user and password:

    docker --config snake-ci-docker login <registry-address> -u <registry-username>
    
  • For Google Cloud Container Registry (gcr.io), use the JSON key based authentication.

    Follow the official instructions to download the JSON key with GCR credentials.

    Then, use docker login with the special username _json_key:

    cat <json-key-file>.json | docker --config snake-ci-docker login -u _json_key --password-stdin https://gcr.io
    

    NOTE: instead of https://gcr.io, you may need to specify https://us.gcr.io or other hostname depending on your region.

To validate that credentials are correct, run docker pull with some image from the private registry:

docker --config snake-ci-docker pull <registry-address>/<image-name>

Repeat this process for each private registry you wish to use in your pipelines.

Finally, copy the entire contents of the snake-ci-docker/config.json file to use in the following steps.

Configure pulling from the private registries

For all projects

To allow Runner to pull private images in all projects and repositories in the Bitbucket instance, specify the SNAKE_DOCKER_AUTH_CONFIG environment variable at the Runner start. If you do not wish to allow all projects to access the private registries just skip this step.

For example, if you’re using Runner in a docker container, pass additional -e argument to the docker run command from the Admin panel:

docker run \
   --name snake-runner \
   …
   -e 'SNAKE_DOCKER_AUTH_CONFIG=<value-from-preparation-step>' \
   …
   reconquest/snake-runner:latest

NOTE: it will only enable pulling build images from the private registries. See the next section to learn how to push to private registries as well.

For specific projects, repositories, pipelines or jobs

To allow only specific projects, repositories, pipelines or jobs to access the private registry, use the DOCKER_AUTH_CONFIG environment variable.

Project and repository access

Navigate to the project or repository settings → Snake CIVariables and add an environment variable named DOCKER_AUTH_CONFIG.

Paste the Docker config content copied from the preparation step and mark variable as Secret.

screenshot

It is the most secure way since authentication credentials will not be stored in the Git repository and will not be visible in the Job Logs.

Pipelines and jobs access

As with all other environment variables, the DOCKER_AUTH_CONFIG variable can be specified directly in the snake-ci.yaml file.

For example, you may allow only specific job to access the private registry by using the variables configuration parameter:

stages:
    - build
    …

build project:
    stage: build
    image: <private-registry>/<image-name>
    variables: # paste value from the preparation step here ↓
        DOCKER_AUTH_CONFIG: |
            {
                "auths": {
                    "gcr.io": {
                        "auth": "X2pzb25…"
                        }
                    }
            }
    …

NOTE: this is not a secure way to specify credentials, because they will be visible to anyone with read access to the repository with the snake-ci.yaml file.

Configure pushing to the private registries

Pushing to the private registries is supported only when the DOCKER_AUTH_CONFIG environment variable is specified as described in the steps above because SNAKE_DOCKER_AUTH_CONFIG is not accessible in the pipelines.

To enable pushing to the private registry, you need to put the value from the DOCKER_AUTH_CONFIG variable in the .docker/config.json file inside build container, as shown in the example below:

stages:
    …
    - push

push image:
    stage: push
    image: docker
    commands:
        - mkdir ~/.docker
        - echo "$DOCKER_AUTH_CONFIG" > ~/.docker/config.json
        - docker push …

Last modified September 17, 2020