Cloning a public Github Repo to build a docker image

Drone Support
1

Is it possible to have a Gitea repo with only one file, .drone.yml, that clones a public Github repo and then builds the docker image? If I mirror the public Github repo to Gitea so all the files are there, the docker image builds fine.

I can’t figure out how to clone the public Github repo during the build process to get the latest code and build the docker image.

Ultimately want a pipeline that builds a new docker image with the latest security updates and pulls in any new changes from Github. The maintainer of the Github repo does not update the docker container very frequently.

Right now I am mirroring the public Github repo to Gitea. I then fork it another Gitea repo so I can do a new pull request when I am notified by RSS that the Github repo is updated. It rebuilds every day so it is up to date from a security perspective. Trying to eliminate the manual step of the pull request and get it fully automated.

I tried the below based on the docs and other posts I have seen, but does not seem to work.

kind: pipeline
name: myapp
steps:
-name: clone
 image: drone/git
 environment:
  - DRONE_WORKSPACE=/data
  - DRONE_REMOTE_URL https://github.com/publicrepo/app.git
- name: build
  image: plugins/docker
  settings:
    repo: local.registry:5000/myrepo/app
    registry: local.registry:5000
    settings:
    insecure: true
    tags: latest
  trigger:
    branch:
    - master
2

Hello @samcro1967

Have you followed this documentation for overriding the default clone step? Cloning | Drone

Thanks

3

@jimsheldon Thank you. That is one of the variations I tried, but with docker/git instead of alpine/git. That got me a little father.

When it kicks off the build I get the following in the drone UI:
image

I see the following in the logs even though I have the DRONE_GITEA_CLIENT_ID, DRONE_GITEA_CLIENT_SECRET, and DRONE_RPC_SECRET set as environment variables for the drone container.

{"level":"debug","msg":"api: authentication required for write access","name":"myapp","namespace":"myrepo","request-id":"26vYHFC3XLWB9nkozCYjGH7PDbo","time":"2022-03-26T09:37:16-05:00"}

Here is the most recent .drone.yml.

kind: pipeline
type: docker
name: myapp
clone:
  disable: true
steps:
- name: clone
 image: alpine/git
 commands:
 - git clone https://github.com/publicrepo/app.git .
 - git checkout $DRONE_COMMIT
- name: build
  image: plugins/docker
  settings:
    repo: local.registry:5000/myrepo/app
    registry: local.registry:5000
    settings:
    insecure: true
    tags: latest
  trigger:
    branch:
    - master
4

Restarting the gitea, drone, and runner containers got me past the auth issue. Removing “git checkout $DRONE_COMMIT”, adding a volume and an explicit location to Dockerfile seems to have gotten me further.

Now I get the following:

Unable to reach Docker Daemon after 15 attempts.
Registry credentials or Docker config not provided. Guest mode enabled.
+ /usr/local/bin/docker version
Client:
 Version:           20.10.9
 API version:       1.41
 Go version:        go1.16.8
 Git commit:        c2ea9bc
 Built:             Mon Oct  4 16:03:22 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
exit status 1

/var/run/docker.sock:/var/run/docker.sock is mounted to the runner container and docker builds work other gitea repos where the files are already present in the repo. My local docker registry does not have any auth on it and I have insecure set to true.

Current .drone.yml

kind: pipeline
type: docker
name: myapp
settings:
  insecure: true
clone:
  disable: true
trigger:
  branch:
  - master
steps:
- name: clone
  image: alpine/git
  volumes:
  - name: cache
    path: /cache
  commands:
  - git clone https://github.com/myrepo/myapp.git /cache
#  - git checkout $DRONE_COMMIT
- name: build
  image: plugins/docker
  volumes:
  - name: cache
    path: /cache
  settings:
    repo: local.registry:5000/myrepo/myapp
    registry: local.registry:5000
    dockerfile: /data/myapp/Dockerfile
    insecure: true
    tags: latest
5

Unable to reach Docker Daemon after 15 attempts.

@samcro1967 this error is unrelated to mounting the docker socket into the runner. The docker plugin in your yaml runs in its own separate container, which uses docker in docker, which means it starts its own docker daemon inside the container. The reason you are having issues with the Docker plugin is described at https://docs.drone.io/plugins/popular/docker/#using-volumes

6

SOLVED

Had to change the volume from /cache to /drone/src/. plugins/docker would not see a volume unless it was in that path. I am sure this in the documentation somewhere I either missed or did not pay attention to closely enough as /cache worked for alpine/git. I also had to make the repo trusted in Drone after making the user an admin, add /drone/src as destination path to the git clone command, and elevate the job to privileged. Lastly, I had to specify the path and name Dockerfile as it is named dockerfile on the public GitHub repo.


kind: pipeline
type: docker
name: myapp
settings:
  insecure: true
clone:
  disable: true
trigger:
  branch:
  - master
volumes:
- name: cache
  path: /drone/src/
steps:
- name: clone
  image: alpine/git
  commands:
  - git clone https://github.com/publicrepo/app.git /drone/src/
  - ls /drone/src/
- name: build
  privileged: true
  image: plugins/docker
  volumes:
  - name: cache
    path: /drone/src/
#  commands:
#  - ls /drone/src/
  settings:
    repo: local.registry:5000/myrepo/myapp
    registry: local.registry:5000
    dockerfile: /drone/src/dockerfile
    settings:
    insecure: true
    tags: latest
7

@samcro1967 the /drone/src directory is the pipeline workspace volume and is automatically mounted into every pipeline step by default, which means it is redundant to include in your yaml.

Here is a real world pipeline to demonstrate:
https://github.com/drone/hello-world/commit/3f7acd27ee6070b96bbba082dd428f3e71060e04

And you can see the pipeline execution logs to verify it works as expected:
https://cloud.drone.io/drone/hello-world/298/1/3

From the docs:

Drone automatically creates a temporary volume, known as your workspace, where it clones your repository. The workspace is the current working directory for each step in your pipeline.

Because the workspace is a volume, filesystem changes are persisted between pipeline steps. In other words, individual steps can communicate and share state using the filesystem.