Drone stuck in "pending" state

vimes · November 7, 2020, 11:58pm

All builds are stuck in status pending, I have tried following the triage guide but have been unable to resolve the issue.
Server config: Raspberry Pi running Ubuntu Server 20.04, drone runner is running on the same machine as drone
Agent config: Running in docker using docker-compose

Server logs:

Recreating drone ... done
Recreating drone-runner-docker ... done
Attaching to drone, drone-runner-docker
drone           | license: ""
drone           | authn:
drone           |   endpoint: ""
drone           |   secret: ""
drone           |   skipverify: false
drone           | agent:
drone           |   disabled: false
drone           | azureblob:
drone           |   containername: ""
drone           |   storageaccountname: ""
drone           |   storageaccesskey: ""
drone           | convert:
drone           |   extension: ""
drone           |   endpoint: ""
drone           |   secret: ""
drone           |   skipverify: false
drone           |   timeout: 1m0s
drone           | cleanup:
drone           |   disabled: false
drone           |   interval: 24h0m0s
drone           |   running: 24h0m0s
drone           |   pending: 24h0m0s
drone           | cron:
drone           |   disabled: false
drone           |   interval: 30m0s
drone           | cloning:
drone           |   alwaysauth: false
drone           |   username: ""
drone           |   password: ""
drone           |   image: ""
drone           |   pull: IfNotExists
drone           | database:
drone           |   driver: sqlite3
drone           |   datasource: /data/database.sqlite
drone           |   secret: ""
drone           |   legacybatch: false
drone           | datadog:
drone           |   enabled: true
drone           |   endpoint: https://stats.drone.ci/api/v1/series
drone           |   token: ""
drone           | docker:
drone           |   config: ""
drone           | http:
drone           |   allowedhosts: []
drone           |   hostsproxyheaders: []
drone           |   sslredirect: false
drone           |   ssltemporaryredirect: false
drone           |   sslhost: ""
drone           |   sslproxyheaders: {}
drone           |   stsseconds: 0
drone           |   stsincludesubdomains: false
drone           |   stspreload: false
drone           |   forcestsheader: false
drone           |   browserxssfilter: true
drone           |   framedeny: true
drone           |   contenttypenosniff: false
drone           |   contentsecuritypolicy: ""
drone           |   referrerpolicy: ""
drone           | jsonnet:
drone           |   enabled: false
drone           | logging:
drone           |   debug: false
drone           |   trace: true
drone           |   color: false
drone           |   pretty: false
drone           |   text: false
drone           | prometheus:
drone           |   enableanonymousaccess: false
drone           | proxy:
drone           |   addr: https://drone.feitdue.no
drone           |   host: drone.feitdue.no
drone           |   proto: https
drone           | registration:
drone           |   closed: false
drone           | registries:
drone           |   endpoint: ""
drone           |   password: ""
drone           |   skipverify: false
drone           | repository:
drone           |   filter: []
drone           |   visibility: ""
drone           |   trusted: false
drone           |   ignore: []
drone           | runner:
drone           |   local: false
drone           |   image: drone/controller:1
drone           |   platform: linux/amd64
drone           |   os: linux
drone           |   arch: arm
drone           |   kernel: ""
drone           |   variant: ""
drone           |   machine: 3507e80cd446
drone           |   capacity: 2
drone           |   labels: {}
drone           |   volumes: []
drone           |   networks: []
drone           |   devices: []
drone           |   privileged: []
drone           |   environ: {}
drone           |   limits:
drone           |     memswaplimit: 0
drone           |     memlimit: 0
drone           |     shmsize: 0
drone           |     cpuquota: 0
drone           |     cpushares: 0
drone           |     cpuset: ""
drone           | nomad:
drone           |   enabled: false
drone           |   datacenters:
drone           |   - dc1
drone           |   namespace: ""
drone           |   region: ""
drone           |   prefix: drone-job-
drone           |   image: ""
drone           |   imagepull: false
drone           |   memory: 1024
drone           |   labels: {}
drone           |   cpu: 500
drone           | kube:
drone           |   enabled: false
drone           |   namespace: ""
drone           |   path: ""
drone           |   url: ""
drone           |   ttl: 300
drone           |   serviceaccountname: ""
drone           |   pullpolicy: Always
drone           |   image: ""
drone           | rpc:
drone           |   server: ""
drone           |   secret: <REDACTED>
drone           |   debug: false
drone           |   host: drone.feitdue.no
drone           |   proto: https
drone           | s3:
drone           |   bucket: ""
drone           |   prefix: ""
drone           |   endpoint: ""
drone           |   pathstyle: false
drone           | secrets:
drone           |   endpoint: ""
drone           |   password: ""
drone           |   skipverify: false
drone           | server:
drone           |   addr: https://drone.feitdue.no
drone           |   host: drone.feitdue.no
drone           |   port: :80
drone           |   proto: https
drone           |   pprof: false
drone           |   acme: false
drone           |   email: ""
drone           |   cert: ""
drone           |   key: ""
drone           | session:
drone           |   timeout: 720h0m0s
drone           |   secret: <REDACTED>
drone           |   secure: false
drone           |   mappingfile: ""
drone           | status:
drone           |   disabled: false
drone           |   name: ""
drone           | users:
drone           |   create:
drone           |     username: andreas
drone           |     machine: false
drone           |     admin: true
drone           |     token: ""
drone           |   filter:
drone           |   - AndreasJacobsen
drone           |   minage: 0s
drone           | validate:
drone           |   endpoint: ""
drone           |   secret: ""
drone           |   skipverify: false
drone           |   timeout: 1m0s
drone           | webhook:
drone           |   events: []
drone           |   endpoint: []
drone           |   secret: ""
drone           |   skipverify: false
drone           | yaml:
drone           |   endpoint: ""
drone           |   secret: ""
drone           |   skipverify: false
drone           |   timeout: 1m0s
drone           | bitbucket:
drone           |   clientid: ""
drone           |   clientsecret: ""
drone           |   skipverify: false
drone           |   debug: false
drone           | gitea:
drone           |   server: ""
drone           |   clientid: ""
drone           |   clientsecret: ""
drone           |   skipverify: false
drone           |   scope:
drone           |   - repo
drone           |   - repo:status
drone           |   - user:email
drone           |   - read:org
drone           |   debug: false
drone           | github:
drone           |   server: https://github.com
drone           |   apiserver: https://api.github.com
drone           |   clientid:<REDACTED>
drone           |   clientsecret: <REDACTED>
drone           |   skipverify: false
drone           |   scope:
drone           |   - repo
drone           |   - repo:status
drone           |   - user:email
drone           |   - read:org
drone           |   ratelimit: 0
drone           |   debug: false
drone           | gitlab:
drone           |   server: https://gitlab.com
drone           |   clientid: ""
drone           |   clientsecret: ""
drone           |   skipverify: false
drone           |   debug: false
drone           | gogs:
drone           |   server: ""
drone           |   skipverify: false
drone           |   debug: false
drone           | stash:
drone           |   server: ""
drone           |   consumerkey: ""
drone           |   consumersecret: ""
drone           |   privatekey: ""
drone           |   skipverify: false
drone           |   debug: false
drone           |
drone-runner-docker | time="2020-11-07T23:49:44Z" level=debug msg="successfully pinged the docker daemon"
drone-runner-docker | time="2020-11-07T23:49:44Z" level=info msg="starting the server" addr=":3000"
drone           | {"level":"info","msg":"main: internal scheduler enabled","time":"2020-11-07T23:49:42Z"}
drone           | {"build.limit":5000,"expires":"0001-01-01T00:00:00Z","kind":"trial","level":"debug","msg":"main: license loaded","repo.limit":0,"time":"2020-11-07T23:49:42Z","user.limit":0}
drone           | {"admin":true,"level":"debug","login":"andreas","machine":false,"msg":"bootstrap: create account","time":"2020-11-07T23:49:42Z","token":""}
drone           | {"admin":true,"level":"debug","login":"andreas","machine":false,"msg":"bootstrap: updating account","time":"2020-11-07T23:49:42Z","token":""}
drone           | {"admin":true,"level":"debug","login":"andreas","machine":false,"msg":"bootstrap: account already up-to-date","time":"2020-11-07T23:49:42Z","token":""}
drone           | {"acme":false,"host":"drone.feitdue.no","level":"info","msg":"starting the http server","port":":80","proto":"https","time":"2020-11-07T23:49:42Z","url":"https://drone.feitdue.no"}
drone           | {"interval":"30m0s","level":"info","msg":"starting the cron scheduler","time":"2020-11-07T23:49:42Z"}
drone           | {"interval":"24h0m0s","level":"info","msg":"starting the zombie build reaper","time":"2020-11-07T23:49:42Z"}
drone-runner-docker | time="2020-11-07T23:49:44Z" level=info msg="successfully pinged the remote server"
drone-runner-docker | time="2020-11-07T23:49:44Z" level=info msg="polling the remote server" arch=arm capacity=1 endpoint="https://drone.feitdue.no" kind=pipeline os=linux type=docker
drone-runner-docker | time="2020-11-07T23:49:44Z" level=debug msg="poller: request stage from remote server" thread=1
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:49:44Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:50:14Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:50:14Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:50:24Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:50:54Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:50:54Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:51:04Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:51:34Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:51:34Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:51:44Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:52:14Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:52:14Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:52:24Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:52:54Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:52:54Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:53:04Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:53:34Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:53:34Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:53:44Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:54:14Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:54:14Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:54:24Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:54:54Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:54:54Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:55:04Z","type":"docker","variant":""}
drone           | {"level":"debug","msg":"api: authentication required","name":"feitdue","namespace":"AndreasJacobsen","request-id":"1jz3tnNsCC7gN1YQk0N7ElKx2V2","time":"2020-11-07T23:55:09Z","visibility":"private"}
drone           | {"level":"debug","msg":"api: guest access","request-id":"1jz3tnNsCC7gN1YQk0N7ElKx2V2","time":"2020-11-07T23:55:09Z"}
drone           | {"fields.time":"2020-11-07T23:55:09Z","latency":1575863,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43090","request":"/api/repos/AndreasJacobsen/feitdue","request-id":"1jz3tnNsCC7gN1YQk0N7ElKx2V2","time":"2020-11-07T23:55:09Z"}
drone           | {"level":"debug","msg":"api: authentication required","name":"feitdue","namespace":"AndreasJacobsen","request-id":"1jz3tkE4CjrnK1lhSfdvoc5K7kW","time":"2020-11-07T23:55:09Z","visibility":"private"}
drone           | {"level":"debug","msg":"api: guest access","request-id":"1jz3tkE4CjrnK1lhSfdvoc5K7kW","time":"2020-11-07T23:55:09Z"}
drone           | {"fields.time":"2020-11-07T23:55:09Z","latency":1253479,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43094","request":"/api/repos/AndreasJacobsen/feitdue/builds?page=1","request-id":"1jz3tkE4CjrnK1lhSfdvoc5K7kW","time":"2020-11-07T23:55:09Z"}
drone           | {"fields.time":"2020-11-07T23:55:11Z","latency":844097,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43098","request":"/AndreasJacobsen/feitdue/","request-id":"1jz3u2JfdqcITTxCSK9T3mhiXHy","time":"2020-11-07T23:55:11Z"}
drone           | {"level":"debug","msg":"api: authentication required","request-id":"1jz3u9m23lP7PDPaTRqQnkeZidl","time":"2020-11-07T23:55:12Z"}
drone           | {"level":"debug","msg":"api: guest access","request-id":"1jz3u9m23lP7PDPaTRqQnkeZidl","time":"2020-11-07T23:55:12Z"}
drone           | {"fields.time":"2020-11-07T23:55:12Z","latency":668860,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43102","request":"/api/user","request-id":"1jz3u9m23lP7PDPaTRqQnkeZidl","time":"2020-11-07T23:55:12Z"}
drone           | {"level":"debug","msg":"api: authentication required","name":"feitdue","namespace":"AndreasJacobsen","request-id":"1jz3u8u7YCRzEzEigupDzuA1bqK","time":"2020-11-07T23:55:12Z","visibility":"private"}
drone           | {"level":"debug","msg":"api: guest access","request-id":"1jz3u8u7YCRzEzEigupDzuA1bqK","time":"2020-11-07T23:55:12Z"}
drone           | {"fields.time":"2020-11-07T23:55:12Z","latency":1870414,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43106","request":"/api/repos/AndreasJacobsen/feitdue","request-id":"1jz3u8u7YCRzEzEigupDzuA1bqK","time":"2020-11-07T23:55:12Z"}
drone           | {"level":"debug","msg":"api: authentication required","name":"feitdue","namespace":"AndreasJacobsen","request-id":"1jz3uAOfKQJEumOD8WfaSo6N6Vo","time":"2020-11-07T23:55:12Z","visibility":"private"}
drone           | {"level":"debug","msg":"api: guest access","request-id":"1jz3uAOfKQJEumOD8WfaSo6N6Vo","time":"2020-11-07T23:55:12Z"}
drone           | {"fields.time":"2020-11-07T23:55:12Z","latency":1898394,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43110","request":"/api/repos/AndreasJacobsen/feitdue/builds?page=1","request-id":"1jz3uAOfKQJEumOD8WfaSo6N6Vo","time":"2020-11-07T23:55:12Z"}
drone           | {"level":"debug","msg":"events: stream opened","request-id":"1jz3u7WTaNa2EcGaaPxQrCFftXm","time":"2020-11-07T23:55:12Z"}
drone           | {"fields.time":"2020-11-07T23:55:12Z","latency":918096,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43118","request":"/favicon.png","request-id":"1jz3uCKMHYwj1JCtaqtDahPVUMu","time":"2020-11-07T23:55:12Z"}
drone           | {"fields.time":"2020-11-07T23:55:16Z","latency":274810,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43122","request":"/login","request-id":"1jz3uf1IMUoGroei0EmOJKd4YTc","time":"2020-11-07T23:55:16Z"}
drone           | {"level":"debug","login":"AndreasJacobsen","msg":"attempting authentication","time":"2020-11-07T23:55:17Z"}
drone           | {"level":"debug","login":"AndreasJacobsen","msg":"authentication successful","time":"2020-11-07T23:55:17Z"}
drone           | {"fields.time":"2020-11-07T23:55:17Z","latency":878109093,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43126","request":"/login?code=b030211db130092fd8fb\u0026state=4d65822107fcfd52","request-id":"1jz3uhRwUmI2NL6NUmghXFrIgIQ","time":"2020-11-07T23:55:17Z"}
drone           | {"fields.time":"2020-11-07T23:55:17Z","latency":1249646,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43134","request":"/","request-id":"1jz3unUNwDcrIMQkkN5e6cGi0Az","time":"2020-11-07T23:55:17Z"}
drone           | {"level":"debug","msg":"events: stream cancelled","request-id":"1jz3u7WTaNa2EcGaaPxQrCFftXm","time":"2020-11-07T23:55:17Z"}
drone           | {"level":"debug","msg":"events: stream closed","request-id":"1jz3u7WTaNa2EcGaaPxQrCFftXm","time":"2020-11-07T23:55:17Z"}
drone           | {"level":"debug","msg":"api: guest access","request-id":"1jz3u7WTaNa2EcGaaPxQrCFftXm","time":"2020-11-07T23:55:17Z"}
drone           | {"fields.time":"2020-11-07T23:55:17Z","latency":5177699622,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43114","request":"/api/stream","request-id":"1jz3u7WTaNa2EcGaaPxQrCFftXm","time":"2020-11-07T23:55:17Z"}
drone           | {"fields.time":"2020-11-07T23:55:17Z","latency":1583974,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43138","request":"/api/user","request-id":"1jz3umxXXblUkX4Y9dviqjU2xc2","time":"2020-11-07T23:55:17Z"}
drone           | {"fields.time":"2020-11-07T23:55:17Z","latency":31936143,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43142","request":"/api/user/repos?latest=true","request-id":"1jz3uk4Mekq1qNwGhJPIPWemWv5","time":"2020-11-07T23:55:17Z"}
drone           | {"fields.time":"2020-11-07T23:55:17Z","latency":15380396,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43146","request":"/api/user/builds/recent","request-id":"1jz3uizEd7u3Kbya8xxyXir0cCC","time":"2020-11-07T23:55:17Z"}
drone           | {"level":"debug","msg":"events: stream opened","request-id":"1jz3uj6w2WrpCRGPyRp7Se8kAFg","time":"2020-11-07T23:55:17Z","user.login":"AndreasJacobsen"}
drone           | {"fields.time":"2020-11-07T23:55:17Z","latency":15664447,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43154","request":"/api/user/repos?latest=true","request-id":"1jz3umoV10eUA9UJ27qCvKuxcQq","time":"2020-11-07T23:55:17Z"}
drone           | {"admin":true,"level":"debug","msg":"api: access granted","name":"feitdue","namespace":"AndreasJacobsen","read":true,"request-id":"1jz3vCbtvyezqGrYaHGSP8RfUKe","time":"2020-11-07T23:55:20Z","user.login":"AndreasJacobsen","visibility":"private","write":true}
drone           | {"fields.time":"2020-11-07T23:55:20Z","latency":3357907,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43158","request":"/api/repos/AndreasJacobsen/feitdue","request-id":"1jz3vCbtvyezqGrYaHGSP8RfUKe","time":"2020-11-07T23:55:20Z"}
drone           | {"admin":true,"level":"debug","msg":"api: access granted","name":"feitdue","namespace":"AndreasJacobsen","read":true,"request-id":"1jz3v86hqadHqHeKxybtvKb3uKi","time":"2020-11-07T23:55:20Z","user.login":"AndreasJacobsen","visibility":"private","write":true}
drone           | {"fields.time":"2020-11-07T23:55:20Z","latency":7258233,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43162","request":"/api/repos/AndreasJacobsen/feitdue/builds?page=1","request-id":"1jz3v86hqadHqHeKxybtvKb3uKi","time":"2020-11-07T23:55:20Z"}
drone           | {"admin":true,"level":"debug","msg":"api: access granted","name":"feitdue","namespace":"AndreasJacobsen","read":true,"request-id":"1jz3vQsa05xmInIVD7HBFgreInC","time":"2020-11-07T23:55:22Z","user.login":"AndreasJacobsen","visibility":"private","write":true}
drone           | {"fields.time":"2020-11-07T23:55:22Z","latency":3404518,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43166","request":"/api/repos/AndreasJacobsen/feitdue","request-id":"1jz3vQsa05xmInIVD7HBFgreInC","time":"2020-11-07T23:55:22Z"}
drone           | {"admin":true,"level":"debug","msg":"api: access granted","name":"feitdue","namespace":"AndreasJacobsen","read":true,"request-id":"1jz3vQARaC5Dfou6EDqyjYe0h8c","time":"2020-11-07T23:55:22Z","user.login":"AndreasJacobsen","visibility":"private","write":true}
drone           | {"fields.time":"2020-11-07T23:55:22Z","latency":6101363,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43170","request":"/api/repos/AndreasJacobsen/feitdue/builds/12","request-id":"1jz3vQARaC5Dfou6EDqyjYe0h8c","time":"2020-11-07T23:55:22Z"}
drone           | {"admin":true,"level":"debug","msg":"api: access granted","name":"feitdue","namespace":"AndreasJacobsen","read":true,"request-id":"1jz3w6k6dQwHqpSIDBEYixQdfHK","time":"2020-11-07T23:55:28Z","user.login":"AndreasJacobsen","visibility":"private","write":true}
drone           | {"fields.time":"2020-11-07T23:55:28Z","latency":6935108,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43174","request":"/api/repos/AndreasJacobsen/feitdue/builds/12","request-id":"1jz3w6k6dQwHqpSIDBEYixQdfHK","time":"2020-11-07T23:55:28Z"}
drone           | {"level":"debug","msg":"events: stream cancelled","request-id":"1jz3uj6w2WrpCRGPyRp7Se8kAFg","time":"2020-11-07T23:55:28Z","user.login":"AndreasJacobsen"}
drone           | {"level":"debug","msg":"events: stream closed","request-id":"1jz3uj6w2WrpCRGPyRp7Se8kAFg","time":"2020-11-07T23:55:28Z","user.login":"AndreasJacobsen"}
drone           | {"fields.time":"2020-11-07T23:55:28Z","latency":10666483842,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43150","request":"/api/stream","request-id":"1jz3uj6w2WrpCRGPyRp7Se8kAFg","time":"2020-11-07T23:55:28Z"}
drone           | {"fields.time":"2020-11-07T23:55:28Z","latency":1499494,"level":"debug","method":"GET","msg":"","remote":"172.20.0.1:43178","request":"/favicon.ico","request-id":"1jz3wC6DDi8EGeqFkj16eFQwwtQ","time":"2020-11-07T23:55:28Z"}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:55:34Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:55:34Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:55:44Z","type":"docker","variant":""}
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2020-11-07T23:56:14Z","type":"docker","variant":""}
drone-runner-docker | time="2020-11-07T23:56:14Z" level=trace msg="http: no content returned: re-connect and re-try"
drone           | {"arch":"arm","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2020-11-07T23:56:24Z","type":"docker","variant":""}

docker-compose

version: '3.3'
services:
    drone:
        volumes:
            - /var/lib/drone:/data
        environment:
            - DRONE_GITHUB_CLIENT_ID=<REDACTED>
            - DRONE_GITHUB_CLIENT_SECRET=<REDACTED>
            - DRONE_RPC_SECRET=<REDACTED>
            - DRONE_SERVER_HOST=drone.feitdue.no
            - DRONE_SERVER_PROTO=https
            - DRONE_LOGS_TRACE=true
            - DRONE_TLS_AUTOCERT=false
            - DRONE_USER_FILTER=AndreasJacobsen
            - DRONE_USER_CREATE=username:andreas,admin:true
            - DRONE_AGENTS_ENABLED=true
        ports:
            - '8000:80'
            - '9000:443'
        restart: always
        networks:
            - drone
        container_name: drone
        image: 'drone/drone:1.9.2-linux-arm' 


    drone-runner:
        container_name: drone-runner-docker
        image: drone/drone-runner-docker:linux-arm
        restart: always
        links:
            - drone
        environment:
            - DRONE_RPC_PROTO=https
            - DRONE_RPC_HOST=drone.feitdue.no
            - DRONE_RPC_SECRET=<REDACTED>
            - DRONE_LOGS_TRACE=true
            - DRONE_RUNNER_CAPACITY=1
            - DRONE_RUNNER_LABELS=runner:rpi
            - DRONE_RUNNER_NAME=rpi
        networks:
            - drone
        ports:
            - '3000:3000'
        depends_on:
            - drone
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
networks:
  drone:

drone.yml

kind: pipeline
type: docker
name: feitdue

platform:
    os: linux
    arch: arm64

node:
  runnner: rpi  
steps:
  - name: react
    image: node
    commands:
      - npm install
      - npm build
    volumes:
     - name: react
       path: ./feit-due:feit-due 
  
  - name: node
    image: node

    commands:
        - npm install
        - npm start
    volumes:
     - name: node
       path: ./api:api


volumes:
- name: react
- name: node
temp: {}

API log from task stuck in pending mode

{"id":12,"repo_id":34,"trigger":"@hook","number":12,"status":"pending","event":"push","action":"","link":"https://github.com/AndreasJacobsen/feitdue/compare/f58576d33de1...88a3562d5677","timestamp":0,"message":"removed workspace for testing","before":"f58576d33de18803b6c4c59a5fc22ad0f48dc141","after":"88a3562d5677aff3a8ffa5f2ad007ac624af3a45","ref":"refs/heads/master","source_repo":"","source":"master","target":"master","author_login":"AndreasJacobsen","author_name":"Andreas Jacobsen","author_email":"andreas1300@gmail.com","author_avatar":"https://avatars1.githubusercontent.com/u/8527173?v=4","sender":"AndreasJacobsen","started":0,"finished":0,"created":1604791754,"updated":1604791754,"version":1,"stages":[{"id":11,"repo_id":34,"build_id":12,"number":1,"name":"feitdue","kind":"pipeline","type":"docker","status":"pending","errignore":false,"exit_code":0,"os":"linux","arch":"arm64","started":0,"stopped":0,"created":1604791754,"updated":1604791754,"version":1,"on_success":true,"on_failure":false,"labels":{"runnner":"rpi"}}]}

bradrydzewski · November 8, 2020, 1:42pm

I can see from the runner output that it is polling for the arm architecture, however, the platform section in your yaml specifies the arm64 architecture. Since the architectures do not match, the pipeline does not get picked up by the runner and sits in a pending state.

kind: pipeline
type: docker
name: feitdue

platform:
    os: linux
-   arch: arm64
+   arch: arm

vimes · November 8, 2020, 2:05pm

Thank you for the reply! It had correctly set platform tags previously but I had tried every arm platform tags in the drone docs before writing this post, it is now set back to just “arch: arm”

But the issue remains, I cannot see any new information in docker compose log output

Last few lines docker compose log
drone | {“arch”:“arm”,“kernel”:"",“kind”:“pipeline”,“level”:“debug”,“msg”:“manager: context canceled”,“os”:“linux”,“time”:“2020-11-08T14:03:45Z”,“type”:“docker”,“variant”:""}
drone-runner-docker | time=“2020-11-08T14:03:45Z” level=trace msg=“http: no content returned: re-connect and re-try”
drone | {“arch”:“arm”,“kernel”:"",“kind”:“pipeline”,“level”:“debug”,“msg”:“manager: request queue item”,“os”:“linux”,“time”:“2020-11-08T14:03:55Z”,“type”:“docker”,“variant”:""}
drone | {“arch”:“arm”,“kernel”:"",“kind”:“pipeline”,“level”:“debug”,“msg”:“manager: context canceled”,“os”:“linux”,“time”:“2020-11-08T14:04:25Z”,“type”:“docker”,“variant”:""}
drone-runner-docker | time=“2020-11-08T14:04:25Z” level=trace msg=“http: no content returned: re-connect and re-try”
drone | {“arch”:“arm”,“kernel”:"",“kind”:“pipeline”,“level”:“debug”,“msg”:“manager: request queue item”,“os”:“linux”,“time”:“2020-11-08T14:04:35Z”,“type”:“docker”,“variant”:""}

kind: pipeline
type: docker
name: feitdue

platform:
    os: linux
    arch: arm


steps:
  - name: react
    image: node
    commands:
      - npm install
      - npm build
    volumes:
     - name: react
       path: ./feit-due:feit-due 
  
  - name: node
    image: node

    commands:
        - npm install
        - npm start
    volumes:
     - name: node
       path: ./api:api


volumes:
- name: react
- name: node
temp: {}
node:
    runnner: rpi

bradrydzewski · November 8, 2020, 4:44pm

can we see your latest runner and server configuration, in addition to the yaml and logs you already provided? The only root cause for a build to get stuck in pending is a problem with configuration (like the incorrect arch which you just corrected). It sounds like you are making various changes and trying different combinations of configurations, so we need to make sure we see the latest state of things.

I also recommend taking a look at this guide. This discusses common problems like incorrect platform, so maybe this can help you triage other potential problems with your configuration.

It would be helpful if you could enable DRONE_RPC_DUMP_HTTP=true and DRONE_RPC_DUMP_HTTP_BODY=true which will dump additional information to the logs which can be helpful for troubleshooting.

vimes · November 8, 2020, 8:37pm

The logs were too long for discours so I’ve added a gist. I just pulled the drone-runner-docker and drone images without any architecture tags, in the beginning; to me it seems like that should work, but I don’t mind specifying architecture, I’ve gone through the “Builds are Stuck in Pending Status” guide.

Gist at https://gist.github.com/AndreasJacobsen/c106e95c51d5c19c3bd418432897f4b7