Building a new image from another image hosted on a private docker registry

Drone Support
1

I am brand new on Drone, which I find great. Despite being familiar with docker, I find the learning curve here very steep – maybe it is the documentation…

So, here is what I want to do:

  • I have a private docker registry hosted on our server, at 192.168.1.46:5000
  • I first build a base image from a dockerhub image, which I saved to the local registry > OK
  • Now, using a separate Gitea repo, I want to add some R stuff to the image I just saved to the local registry and save it to another image to the local registry, with another name

[dockerhub img] —> [img#1, local registry] —> [img#2, local registry]

I have struggled quite a bit with the docker plugin because it was sending errors about:

  1. the registry expecting it to be secure > found the solution with the insecure setting
  2. the docker plugin missing the Dockerfile > created a Dockerfile in the initial git repo and delegated most of the build to the Dockerfile
  3. lastly the docker plugin losing access to the dockear daemon > found the solution by adding registry: {registry IP and port} to the settings

This just to show you I actually did my homework and how complicated it can be to learn Drone, and mostly the docker plugin, which, IMO, does too much and is not easy to understand as it relies heavily on the Dockerfile, with boundaries hard to comprehend

Now, I am struggling to:

  • copy files that are in the project’s Gitea repo (so, by default copied by Drone to the default workspace at /drone/src/) into a folder of the image loaded by the Dockerfile
  • mount a volume from the host to the “inside” of the docker plugin: working on R, I want to mount the renv plugin cache so that the plugin installation can rely on already cached files.

The project’s Gitea repo is called vsr-bas. It is a trusted project.
Here is the .drone.yml file:

kind: pipeline
type: docker
name: vsr_bas

steps:
  - name: publish_vsr-bas
    image: plugins/docker
    settings:
      repo: 192.168.1.46:5000/vsr-bas
      registry: 192.168.1.46:5000
      pull: never
      insecure: true
      auto_tag: true

Here is the Dockerfile:

FROM 192.168.1.46:5000/vsr-000:latest
# NOTE: Copy Rprofile.site for the correct URL of the CRAN directory
RUN mkdir -p /usr/lib/R/etc/
RUN cp /drone/src/Renviron /usr/lib/R/etc/Renviron
RUN cp /drone/src/Renviron.site /usr/lib/R/etc/Renviron.site
RUN cp /drone/src/Rprofile.site /usr/lib/R/etc/Rprofile.site

# NOTE: Base plugin installation
ENV RENV_VERSION 0.14.0
RUN R -e "install.packages('config')"
RUN R -e "install.packages('renv')"
  • vsr-000 is the low-level image currently saved on the local registry (called img#1 above)
  • vsr-bas is the base image with all R plugins installed I want to save, in turn, to the local registry (called img#2 above)

Currently I get an error when trying to copy from /drone/src within the Dockerfile
I also tried to mount the renv cache as a volume, but then I get an error with the docker daemon not being reachable.

What am I doing wrong?
Thanks for your help.
Stephen.

2

Also, in one of his responses to a similar topic, Brad Rydzewski said that one could bypass the docker plugin altogether.
I used the docker plugin because:

  1. I could not execute a docker pull or a docker push to my local registry without the registry server complaining about receiving an insecure connection.
  2. and because I could not change, on the docker runner the daemon.json file contents – the runner is built using Alpine Linux and I could not locate the file
3

does too much and is not easy to understand as it relies heavily on the Dockerfile, with boundaries hard to comprehend

I am confused by this statement because you cannot build a Docker image without a Dockerfile. The Dockerfile is required by Docker. If you do not want to rely on the Dockerfile to build your Docker images, that is something you’ll have to take up with the Docker maintainers.

Currently I get an error when trying to copy from /drone/src within the Dockerfile

What is the error message? I recommend providing full steps to reproduce (perhaps a sample, public github repository) along with the full output for your docker step (from the user interface) so that members of the Drone community have enough information to provide you with support.

4

Hi Brad, thanks for responding so fast. I appreciate.
Well, my idea was:

  1. to load an image using Drone (which is the first step of any docker runner pipeline, if I understood well). I guess at this point it is a docker container living in the docker runner
  2. add some stuff in the docker container, for example install R plugins
  3. then transfer back the docker container to an image using:
docker commit {the_container_id} {the_image_name}
docker tag {the_image_name} {the_registry_ip}/{the_image_name}:{the_tag}
docker push {the_registry_ip}/{the_image_name}:{the_tag}

This is what I usually do when I work on containers and want to convert them as images, for further use.
Frankly, that was my first (and preferred) idea when learning Drone.
But then I hit the usual http: server gave HTTP response to HTTPS client error.
I had already solved it several times, but when docker was installed in a Debian environment with services.
Here I could not figure out on how to allow insecure-registries because the operating system of the docker runner is Alpine.

5

As per the error I encounter while trying to copy files from the default Drone workspace, here is some additional info.

Here is the Dockerfile with the copy instructions (the 3 files are present in the initial git repo: Renviron, Renviron.site, Rprofile.site):

FROM 192.168.1.46:5000/vsr-000:latest
# NOTE: Copy Rprofile.site for the correct URL of the CRAN directory
RUN mkdir -p /usr/lib/R/etc/
RUN cp /drone/src/Renviron /usr/lib/R/etc/Renviron
RUN cp /drone/src/Renviron.site /usr/lib/R/etc/Renviron.site
RUN cp /drone/src/Rprofile.site /usr/lib/R/etc/Rprofile.site

And here is the error I get, once the image vsr-000:latest has fully loaded:

Digest: sha256:a1e49975153a89ea805fdaa75d240d07f43dea56f1d5109f458d278ac1b26a44
Status: Downloaded newer image for 192.168.1.46:5000/vsr-000:latest
 ---> 8de92997331d
Step 2/12 : RUN mkdir -p /usr/lib/R/etc/
 ---> Running in a6b7147c781b
Removing intermediate container a6b7147c781b
 ---> 47e458179794
Step 3/12 : RUN cp /drone/src/Renviron /usr/lib/R/etc/Renviron
 ---> Running in 43df649ebfe6
cp: cannot stat '/drone/src/Renviron': No such file or directory
The command '/bin/sh -c cp /drone/src/Renviron /usr/lib/R/etc/Renviron' returned a non-zero code: 1
exit status 1
6

Just for the record, here is the content of the initiating Gitea repo:

image

7

If you want to copy files into your image you need to use the COPY or ADD directive.

Instead of this:

RUN mkdir -p /usr/lib/R/etc/
RUN cp /drone/src/Renviron /usr/lib/R/etc/Renviron
RUN cp /drone/src/Renviron.site /usr/lib/R/etc/Renviron.site
RUN cp /drone/src/Rprofile.site /usr/lib/R/etc/Rprofile.site

You need to do something like this:

RUN mkdir -p /usr/lib/R/etc/
ADD Renviron /usr/lib/R/etc
ADD Renviron.site /usr/lib/R/etc/Renviron.site
ADD Rprofile.site /usr/lib/R/etc/Rprofile.site

I also recommend running docker build locally, from your terminal, to test your Dockefile and troubleshoot issues. The error message you provided can be reproduced without using Drone. So once you have a working Dockerfile and you can run docker build locally, it should work just fine when using the Docker plugin.

8

My bad, you are perfectly right, and this was the case in the initial Dockerfile that has been working for months now, outside of Drone.

9

Just one precision though: do I have to write:

ADD /drone/src/Renviron /usr/lib/R/etc

with /drone/src/ as a prefix since those files come from the initiating Gitea repo?

10

nope, because /drone/src is the current working directory for all pipeline steps (including plugin steps) and the current working directory is the default docker context when building images.

11

Yup, I just tried with the /drone/src/ and it failed.
I do not want to be obnoxious on a Saturday evening, but you see, this is the very kind of things that is not thoroughly explained in the documentation and may become frustrating for people less stubborn than I am…

12

/drone/src is the current working directory for all pipeline steps

this is documented here:
https://docs.drone.io/pipeline/docker/syntax/workspace/

and the current working directory is the default docker context when building images

this is documented here:
http://plugins.drone.io/drone-plugins/drone-docker/ the context path […] defaults to root of the git repo

however, with that being said, we always welcome pull requests to improve the docs. github.com/drone/docs

13

Does your documentation accept Mermaid graphs? I think it could be vastly improved by using schemata.