Creating a docker image for use in a later step/pipeline without uploading it or being a trusted build

Drone Support
1

Hi!

This may be asking a lot. I have a test process that is made easier by creating a custom mysql docker image and using that as a service which the tests connect to. I’d like to create this image as part of the build/test process without uploading the built image to a docker repo. Also, I’d like to not have to make the build trusted.

I started on the trusted route with this:

kind: pipeline
name: build

steps:
  - name: build test image
    image: docker:latest
    volumes:
      - name: docker_sock
        path: /var/run/docker.sock
    commands:
      - docker build --no-cache -t test-db:mysql .

volumes:
  - name: docker_sock
    host:
      path: /var/run/docker.sock

---

kind: pipeline
name: test

steps:
  - name: test
    image: python:3.7
    commands:
      - make test

services:
  - name: mydb
    image: test-db:mysql
    pull: if-not-exists
    environment:
      MYSQL_ROOT_PASSWORD: mysql
      MYSQL_DATABASE: mydb

This is creating a docker image using the host’s docker socket which makes the image available to the host for the next pipeline to pick-up…hence the need for trusted with this solution. Oddly enough, the image wasn’t available on the first run of the build…but was for each time afterwards. It sounds like there’s a timing issue or something else where the built image isn’t available right after it’s built.

So then I learned about detached steps and switched to this:

kind: pipeline
name: test

steps:
  - name: build test image
    image: docker:latest
    volumes:
      - name: docker_sock
        path: /var/run/docker.sock
    commands:
      - docker build --no-cache -t test-db:mysql .

  - name: mydb
    image: test-db:mysql
    pull: if-not-exists
    environment:
      MYSQL_ROOT_PASSWORD: mysql
      MYSQL_DATABASE: mydb
    detach: true

  - name: test
    image: python:3.7
    commands:
      - make test

volumes:
  - name: docker_sock
    host:
      path: /var/run/docker.sock

This worked even on the first run, but it’s not a great solution to use the host’s docker service in this way for my builds. So I tried using the docker plugin:

kind: pipeline
name: test

steps:
  - name: build test db image
    image: plugins/docker
    settings:
      repo: test-db
      tags: mysql
      dry_run: true

  - name: mydb
    image: test-db:mysql
    pull: if-not-exists
    environment:
      MYSQL_ROOT_PASSWORD: mysql
      MYSQL_DATABASE: mydb
    detach: true

  - name: test
    image: python:3.7
    commands:
      - make test

However, that doesn’t make the image available in the next step.

So, for now, I’ve settled on uploading the image in one step and downloading it in the next:

steps:
  - name: build test db image
    image: plugins/docker
    settings:
      registry: myregistry
      repo: myregistry/test-db
      tags: mysql
      username:
        from_secret: DOCKER_USERNAME
      password:
        from_secret: DOCKER_PASSWORD

  - name: mydb
    image: test-db:mysql
    pull: if-not-exists
    environment:
      MYSQL_ROOT_PASSWORD: mysql
      MYSQL_DATABASE: mydb
    detach: true

  - name: test
    image: python:3.7
    commands:
      - make test

Is there a way to build a docker image without uploading it and use it in a following step or pipeline?

Thanks!
-Chris

2

Hi Chris, unfortunately this is not going to be possible with Docker pipelines. You either need privileged mode to run docker-in-docker, or you need to mount the host machine docker socket, both of which required trusted mode.

If you cannot use trusted mode, the best option is to use a VM for your pipeline. We provide a digital ocean runner that creates a dedicated VM for you pipeline. Inside the VM you can do whatever you want. https://blog.drone.io/drone-digitoalocean-runner/

1 Like
3

Thanks Brad for the confirmation that this is a limitation and the alternate solution.

4

I’m hitting the same issue, looking for a secure way to build a docker image to then be used for subsequent steps. Not mentioned here, Kaniko appears to provide a way to build docker images without privileges and a plugin exists, but it’s not clear to me how or if it’s possible to build a local image that can then be retrieved for the next steps. Anyone found a secure solution that doesn’t require running with a trusted repo?