Drone: Error response from daemon: No such image:

I have build my own drone ‘plugin’ containers and it seems sometimes drone fails to find them with this message
“Error response from daemon: No such image: grails_build:2.3.11”

I know the machine with the runner has the container on it already I also have this in the step

  - name: grails_war
    image: grails_build:2.3.11
    pull: never

When I run docker ps

If I docker build the image again it works.
I am not sure if this a bug or something I am doing wrong.

“Error response from daemon: No such image: grails_build:2.3.11”

This error comes from the docker daemon, indicating that it cannot find your image in its cache. If the docker image exists in the cache, but docker is returning an error, you may need to raise an issue with the docker team; they are going to have the expertise needed to triage unexpected docker daemon issues and determine whether or not there is a bug.

Oh ok that makes sense. Sorry.

I have been unable to clean up the docker server as it’s a production server and I think I will need to clear out all images and rebuild everything (maybe just a reboot will be enough) but I have noticed if I restart the runner this error disappears for 1 build. I thought this may be of interest is anyone has a similar issue.

Is it possible you have a process that is removing the image? perhaps another pipeline step that mounts the host machine docker socket and removes images? We have seen folks mount the host machine docker socket when using the docker plugin (which is not recommended since the docker plugin is designed to use its own docker-in-docker daemon). Since the docker plugin executes docker system prune -f, if you mount the host machine docker socket, it will effectively remove all unused images on your host machine. And since you set pull: never , once the image is gone, Drone will never pull the image which would explain your error. This is of course just one very specific example … I don’t have any insight into whats happening in your pipeline or on your host machine but I do know that Drone cannot cause Docker to throw an image not found error; that error can only come from Docker when an image does not exist in the Docker cache.

Brad, Thanks for trying to help. No I don’t think i am doing that. The container is build on the host, the same as a few other containers.
When I run docker images on the host it is there. I think there is a corruption of my docker (as you suggested) server since an ubuntu upgrade. But as I said I cannot restart (and/or) rebuild it all as it’s running and I can’t have the down time right now to rebuild if I need too.

I just thought it was ‘interesting’ that restarting the runner caused the docker API to find it. I thought I would document it here JUST in case someone else has a similar issue to me :slight_smile:

Thanks for a great project by the way. I love the use for Docker for this.