Multiple repos: better/fairer scheduling

Drone Support
1

Hi,
I hope it is the right place for this feature request, as on github it is said to write to discourse.

Imagine you have one drone with lots of workers and multiple repositories to check.
Currently the order of jobs is defined by creation time of the corresponding PR.

If on repoA are 20 pending jobs and you open up one job on repoB you will have to wait for the 20 jobs to finish.

Given that each repository is equally it is better (at least in my opinion) to try to serve each repository equally.
In above scenario the job on repoB would start as next job and then the pending jobs on repoA.

A pseudocode for the scheduler on free worker:
loop through repositories
on each repo:
- take the first pending job
- if no pending job is there continue

The current position of the “loop through repositories” has to be remembered (at least on each worker) as otherwise if you always start from the beginning the first repos would benefit.

What do you think?

2

There are no plans for drone core to directly solve this problem. In general, we strive to ship drone with simple defaults that can be customized through plugins. We will therefore work toward making more of drone pluggable so that teams can replace default implementations with custom implementations that meet their specific usage requirements. So in this case we would make the FIFO queue pluggable, with the ability to swap with your own queue implementation.

3

Support via plugin is a good idea and sufficient enough :+1:
Looking forward to have this :slight_smile:

Overall what do you think about this queue idea?

4

Any news regarding the plugin system?
I really would like to implement this scheduler :slight_smile:

5

@bradrydzewski are there any news on this?

Thanky you and merry christmas! :christmas_tree: