I’m running into an issue trying to setup an automated process to get our bot account for drone to sync the repos from github upon repository creation.
I’ve got a probot app setup to listen for the repository.created event from github which then in turn performs a sync via the drone api (i.e. POST /api/user/repos). I can see in the logs that it performs a sync. However, in the response to that request, the new repository is not added to the list. If we later perform a sync it looks to pick it up just fine. Any idea what’s going on here?
I recommend enabling trace logging [1] which will write every repository to the logs that is returned from the GitHub API call [2]. If you do not see an entry for the repository in the logs it means it was not returned from GitHub.
I do not want to get too far ahead of ourselves without seeing the trace logs, but we have seen evidence that read-only API calls to GitHub may be eventually consistent. If the API call reads data from a replica this is not updated yet, it would return stale data.
In addition to the trace logs, you could also test this hypothesis by waiting 30 seconds after receiving the repository creation hook before attempting to sync the repository list.
We can confirm that the new repo is NOT coming back from the remote (i.e. github), so it must be a replica thing. However, we also tried updating the script to retry the sync call every 5s for 30s and it still never found it. Is there a cache mechanism at play here and maybe over that 30s we are keeping that cache alive by retrying so frequently?
When you sync the repository Drone invokes the GitHub API directly. Drone does not implement any caching or set any cache headers or ETag headers that might impact the results. There could be caching or consistency delays within GitHub, but we can only speculate since we don’t have much visibility into how they are responding to requests.
Somewhat related, we have a 60 second backoff and retry in our codebase for using the content API to get the yaml file [1] because in some cases it would incorrectly return 404 Not Found. I recall submitting a support request to GitHub and them indicating it was due to eventual consistency.
Looks like we were able to get it working by waiting 30s first. We’ve also got it doing additional retries after that, but it looks like it picks it up after that initial 30s. That should be sufficient for our needs . Thanks for your help!
. Thanks for your help!