Drone verbose output slows builds

Hello, I don’t know if this is new or not but this happening since I started using drone. I started using drone since version ~1.0.
Currently I am running version: 1.4.0 both server and agents.
This is getting me very upset because the way it affects the build speed.
Let’me explain, I have several projects on my Drone setup, currently I have:
2 build servers (agents) running Linux with docker. They have a 100/100 mb internet connection and they are located in Portugal. My drone server is located in France with a 1GB/400 mb internet connection, the connection between the agent and the server is stable and fast however when I am running a job and watching through the Drone webui it gets slower depending on the amount of logs/records the job sends. However if I configure a drone step with something like " 1 > /dev/null " which omits the stdout the step finishes in half of the time. I should say that if I reproduce the same step in my local machine it finishes at the same time with or without stdout.
This happens with every step that I have that throws more than 200, 300… lines of logs. I even had a step that trowed 7000 lines of logs at it took 17 minutes instead of the regular 4/5 minutes.

In this example I am running a unit testing on a application. On the first image it is with the default settings, showing the regular logs. (finished in 06:20)

In this second image the tests step was tweaked to output less verbosity on the logs (finished in 02:20)

This is slowing down my builds, any help is appreciated. Thanks!

I am also having the same problem. Maybe the output should be compressed? for now i’m forced to minimize output of commands by forwarding to /dev/null

From what I’ve been checking Drone on version 1.0 removed GRPC support and replaced with HTTP Post request’s to send the build output from the agent to server. From my point of view GRPC would be best suited for this. Using stream capabilities from GRPC probably would benefit a lot on this use case.

@bradrydzewski can you have a word on this?

Most of the runners use a buffered log streamer https://github.com/drone/runner-go/blob/master/livelog/livelog.go.

The exception is the docker runner, which has not been ported to the buffered log streamer yet. This effort is in-progress. The buffered log streamer performs well; there are no noticeable differences between the buffered log streamer and grcp.

We removed grpc because many Drone installations run on corporate networks, and many corporate networks had difficulty with grpc. It frequently errored, had problems with load balancers, was not compatible with reverse proxies, and required a significant amount of end-user support and networking expertise. We could no longer justify the support cost. We have no plans (or desire) to re-introduce grpc support.

Same problem here with Drone server 1.9 and agent 1.6. We have to aggressively suppress any output in order to keep build times bearable.

Any progress porting the docker runner?

drone/agent was deprecated in favor of drone/droner-runner-docker, which supports buffered log streams.

1 Like