Hello, we are seeing intermittent outages with the drone UI (showing as Nginx timeout and are having some trouble debugging the root cause. The drone agent containers look OK. I have provided the below log. “[Recovery] panic recovered” appears multiple times. Any feedback is appreciated
Log:
error: cannot find parent proc with id 77317: dial tcp 172.18.4.16:5432: getsockopt: connection timed out
error: rpc.update: cannot find pproc with id 77317: dial tcp 172.18.4.16:5432: getsockopt: connection timed out
error: cannot find proc with id 77317: dial tcp 172.18.4.16:5432: getsockopt: connection timed out
error: rpc.update: cannot find pproc with id 77332: dial tcp 172.18.4.16:5432: getsockopt: connection timed out
error: cannot find parent proc with id 77332: dial tcp 172.18.4.16:5432: getsockopt: connection timed out
error: rpc.update: cannot find pproc with id 77332: dial tcp 172.18.4.16:5432: getsockopt: connection timed out
time=“2019-11-20T19:19:24Z” level=error msg=“Error #01: sql: no rows in result set\n” ip=10.254.0.59 latency=3.099827ms method=GET path="/api/repos/XXXlogs/34/2" status=404 time=“2019-11-20T19:19:24Z” user-agent=“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.87 Safari/537.36”
time=“2019-11-20T19:19:25Z” level=error msg=“Error #01: sql: no rows in result set\n” ip=10.254.0.59 latency=3.750217ms method=GET path="/api/repos/XXX/logs/34/2" status=404 time=“2019-11-20T19:19:25Z” user-agent=“Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:71.0) Gecko/20100101 Firefox/71.0”
time=“2019-11-20T19:33:18Z” level=error msg=“error setting commit status for XXX/34: Post https://XXX/rest/build-status/1.0/commits/e515b5dca56558d4fbede922f880a7f588d02af8: Post https://XXX/rest/build-status/1.0/commits/e515b5dca56558d4fbede922f880a7f588d02af8: read tcp 192.168.199.11:45060->172.18.3.9:443: read: connection timed out”
2019/11/20 19:53:11 [Recovery] panic recovered:
POST /hook?access_token=XXX
Host: XXX
Connection: close
Accept: application/json
Accept-Encoding: gzip,deflate
Connection: close
Content-Length: 555
Content-Type: application/json