Drone agent unable to get secrets from aws


we have drone setup on aws ecs cluster ( one server and 4 agents).
Due to some reason, from past one day, agents are unable to fetch aws secrets which are fetched by invoking an api-gw calling linked with a lambda function.

From drone-agent i can see the environment variable “DRONE_SECRET_ENDPOINT” is correctly defined. “https://abcfdef-api.eu-west-1.amazonaws.com/prod”. However now if i check api-gateway or from lambda function, they are not called/invoked.

Agent-logs give following message when build fails:


{“error”:"{“message”:“Forbidden”}",“kind”:“secret”,“level”:“trace”,“msg”:“secret: external: cannot get secret”,“name”:“clientsecret”,“time”:“2021-08-02T09:25:25Z”}

Anyone knows how can i debug the agent, so it is able to call this “DRONE_SECRET_ENDPOINT”.


The error message would tell me that Drone is invoking the endpoint as expected, but the endpoint is returning a 403 Forbidden error message. You can see in the logs you posted here:


If Drone was working fine, and then all of a sudden stopped working, have you considered the possibility that something changed in Amazon that caused this to stop working? For example, a 403 Forbidden error could indicate something changed with your security settings, credentials, etc in Amazon.

Thanks @bradrydzewski yes it looks some issue when drone calls aws api-gateway which then calls lambda function. Need to check but we didn’t change any aws resource or setting

did you change anything in Drone?

No we use the same configuration.

hmm … well we can see that Drone is making the API call to the endpoint and it is returning a 403 Forbidden. If nothing changed in Drone, and the AWS gateway started returning 403 Forbidden errors, it would stand to reason that something in AWS changed.

1 Like

yes you are right. we can check the api-gw then. Thanks for looking into this.

@bradrydzewski we enabled access logs on our api-gateway to which drone-agent is calling. From access logs we don’t see any invocation from drone-agent or 403 errors. I guess the above error message “{“message”:“Forbidden”}” comes from drone-agent.

@farooqte96 I can confirm that Forbidden does not come from Drone, it comes from the http response when invoking the endpoint.

1 Like

Thanks alot for confirming this.