How to gracefully stop agents?

Drone Support
1

Hi,

We’re wanting to scale agents based on the build queue; scale up with demand and down over time when the queue is empty. Scaling up is no problem but scaling down is causing a few issues such as agents/builds slowing down. Currently, an agent will wait a set time and then check if a build is running by checking if there is 1 Docker container (drone/agent) running, if there is then send ‘docker kill --signal=SIGTERM’ to the agent container

I was wondering:

  • How to gracefully stop agents; so that the agent doesn’t pull any builds from the master while shutting down and be removed from the Drone worker pool? Will ‘docker kill --signal=SIGTERM’ gracefully stop the agent? I’ve read online if you want to initiate a graceful shutdown of an nginx server, you should send a SIGQUIT and Apache uses SIGWINCH. Although I see there’s support here https://github.com/drone/signal/blob/master/signal.go#L26 I just wanted to check it’s what I think it is.
  • What’s the best way to determine if a build is running? Watching the Docker logs of the agent container for “message”:“pipeline done”?

Thanks,
Harry

2 Likes
2

For anyone interested

How to gracefully stop agents?
Use ‘docker kill < container > --signal=SIGTERM’ once on the drone/agent container.

What’s the best way to determine if a build is running?
We assumed if there are >1 containers then a build is running

1 Like
3

You may also want to consider https://github.com/drone/autoscaler , which handles agent autoscaling based on build queue volume

4

searching for something like: how to drain an agent.
‘docker kill < container > --signal=SIGTERM’ once did seem to work. Great advice.