Angular Dockerfile build stopped working

Drone Support
1

Hi everyone,

I have been using Drone from helm charts for a few months now, and this is the first time I experience this issue.

The .drone.yml file contains a regular build step via docker/plugin like the following:

- name: build-beta
  image: plugins/docker
  pull: always
  settings:
    repo: xxx
    tags:
    - beta
  dockerfile: Dockerfile
  registry: xxx
  username: xxx
  password: xxx
  build_args:
    - ENV=beta
  when:
    event: [push, tag]
    branch: beta

Without changing either the .drone.yml or the Dockerfile, build stops now at this step without any warnings or error messages:

---> 7386dc0d3761 Step 3/16 : RUN npm i && mkdir /ng-app && mv ./node_modules ./ng-app 
---> Running in 586cb8e1d285 time="2020-04-22T11:06:45Z" level=info msg="shim containerd-shim started" address="/containerd shim/moby/586cb8e1d285dc84838a3b858b95aceb4a0fb054a766e9dfcad356ae68512d86/shim.sock" debug=false pid=294
npm WARN deprecated popper.js@1.15.0: You can find the new Popper v2 at @popperjs/core, this 
package is dedicated to the legacy v1
npm WARN deprecated request@2.88.2: request has been deprecated, see 
https://github.com/request/request/issues/3142
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 
with 15x less dependencies.
npm WARN deprecated fsevents@1.2.12: fsevents 1 will break on node v14+. Upgrade to fsevents 
2 with massive improvements.

And from there it timeouts (whatever that timeout may be 15 minutes - 1 hour). Of course nothing is actually built and pushed to the docker registry.

Drone debug variable is set to true but does not produce any output.

Anyone has experienced something similar?

Kind regards,
Marios.

2

One thing that jumps out is a spacing problem in your yaml. You have plugin settings that are not defined in the settings section of your pipeline. Make sure you have the correct indentation as show below:

  name: build-beta
  image: plugins/docker
  pull: always
  settings:
    repo: xxx
    tags:
    - beta
- dockerfile: Dockerfile
- registry: xxx
- username: xxx
- password: xxx
- build_args:
-   - ENV=beta
+   dockerfile: Dockerfile
+   registry: xxx
+   username: xxx
+   password: xxx
+   build_args:
+     - ENV=beta
3

Sorry this only due to copy-pasting it without checking thoroughly.
The spacing is fine in the actual yaml file.

Thanks for the reply, though.

4

also note that a timeout generally indicates a network timeout. If docker push running inside the official docker-in-docker container hangs, that is probably a question for the docker folks. There is nothing inherent to Drone that would cause a docker push to hang.

5

Yes, I agree with you, but as I already mentioned (maybe it was not clear enough) this build is the first time that it does not output all the steps but rather stops at step 3 as in the logs above.

Typically I could see the whole Dockerfile execution and after a successful built it would be pushed to the registry.

6

We could probably rephrase this question as:

why does docker build hang when run inside my Pod on Kubernetes?

My guess is that if we showed the output to the Docker maintainers, they would also point out that network calls (to npm) appear to be hanging and this could be a network issue, which would need to be triaged at the kubernetes layer.

Pipeline steps are plain old containers running inside regular Kubernetes Pods. There is nothing inherent to how Drone creates Pods that would cause programs running inside containers to hang.

Probably won’t help much, but you can review the Docker plugin code here to see how it works: https://github.com/drone-plugins/drone-docker/blob/master/docker.go

1 Like
7

Thanks for the elaborate answer.

Apparently, the project builds and is pushed to the docker registry.

However, the logs are still missing (it stops at step 3), and the build exhausts the available time!
So if I set timeout to 60minutes it will keep up until 59:59 and then stop.

Why that happens? No clue. Nothing is different.

Edit: This only happens with the angular build, if I try to build something different the logs are there as used to.

8

hmmm, this definitely sounds odd. I would recommend you enable trace logging for the runner to see if there are any relevant logs that reveal what is happening. I also recommend debugging the code responsible for streaming logs and waiting for the container to exit at drone-runner-kube/engine_impl.go at master · drone-runners/drone-runner-kube · GitHub

Lastly you may want to make sure you are running the latest version of the runner. We are frequently accepting pull requests that improve the runner that may not be available in older builds.

9

Thanks for the replies, I have not figured this out yet.

If anything changes, I will let you know.