Unable to sync large amount of repos from a private GitHub

Hi there,

we have a large private GitHub environment (more than 12500 repos across 200 orgs) and we have recently started running into sync issues. Running the sync three times it seems that gets stuck at later pages

time=“2020-03-16T16:13:01Z” level=warning msg=“syncer: cannot get remote repository list” error=“Get private-github.example.com/api/v3/user/repos?page=45&per_page=100: context canceled” login=my-user
time=“2020-03-16T16:13:01Z” level=warning msg=“api: cannot synchronize account” error=“Get private-github.example.com/api/v3/user/repos?page=45&per_page=100: context canceled” request-id=da93e406693bcc3437794f861c28bfcf user.login=my-user
time=“2020-03-16T16:22:42Z” level=warning msg=“syncer: cannot get remote repository list” error=“Get https://private-github.example.com/api/v3/user/repos?page=51&per_page=100: context canceled” login=my-user
time=“2020-03-16T16:22:42Z” level=warning msg=“api: cannot synchronize account” error=“Get private-github.example.com/api/v3/user/repos?page=51&per_page=100: context canceled” request-id=6fc990453fda08161303df22aa588d2c user.login=my-user
time=“2020-03-16T16:31:43Z” level=warning msg=“syncer: cannot get remote repository list” error=“Get private-github.example.com/api/v3/user/repos?page=41&per_page=100: context canceled” login=my-user
time=“2020-03-16T16:31:43Z” level=warning msg=“api: cannot synchronize account” error=“Get private-github.example.com/api/v3/user/repos?page=41&per_page=100: context canceled” request-id=628d0bc90935ec34383bbef935d61ef1 user.login=my-user

We’re running Drone on Kubernetes (GKE) and it has been installed with latest Helm chart (1.6.5).

Questions:

  1. Does drone support per-organization scanning (our team has 50 repos in our org and that’s the only one we care about)
  2. Can this be fixed?

Thank you!

This was identified and fixed by this patch and will be available in our next release. It is available today if you use the drone/drone:latest image.

Hi Brad,

thanks - we’ve updated the image and I confirm this is no longer an issue! Thanks again! :slight_smile: