Failing to run dind in a Kubernetes pipeline

Kash · May 5, 2020, 4:48pm

I’m currently testing out “drone-runner-kube” (v1.0.0-beta.3) and have certain images that I would like to always run privileged, regardless of whether a repository is marked as trusted. I’ve set the following environment variable in the deployment:

        - name: DRONE_RUNNER_PRIVILEGED_IMAGES
          value: "docker:dind"

A snippet of my pipeline is (following the Drone doc examples):

- name: build
  image: docker:dind
  commands:
  - docker build -t build .
  volumes:
  - name: dockersock
    path: /var/run

services:
- name: docker
  image: docker:dind
  volumes:
  - name: dockersock
    path: /var/run

volumes:
- name: dockersock
  temp: {}

When the service starts, the following logs are output:

...
mount: permission denied (are you root?)
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied (are you root?)

Now if I include “privileged: true” and mark the repository as trusted, this works fine. So I suspect there is an issue or bug with the DRONE_RUNNER_PRIVILEGED_IMAGES values being parsed. Is anyone able to confirm whether this is a bug, or spot any glaring issues with what I’ve mentioned above?

bradrydzewski · May 5, 2020, 5:08pm

The DRONE_RUNNER_PRIVILEGED_IMAGES parameter is only used for privileged plugin images (plugins/docker, etc) for security reasons. A plugin is defined as a pipeline step that does not have any user-defined commands, command or entrypoint. You can see the logic here:

github.com

drone-runners/drone-runner-kube/blob/master/engine/compiler/compiler.go#L536:L559


			v.Resources.Limits.CPU = c.Resources.Limits.CPU
		}
		if v.Resources.Limits.Memory == 0 {
			v.Resources.Limits.Memory = c.Resources.Limits.Memory
		}
	}


	return spec
}


func (c *Compiler) isPrivileged(step *resource.Step) bool {
	// privileged-by-default containers are only
	// enabled for plugins steps that do not define
	// commands, command, or entrypoint.
	if len(step.Commands) > 0 {
		return false
	}
	if len(step.Command) > 0 {
		return false
	}
	if len(step.Entrypoint) > 0 {

We should probably rename this variable to DRONE_RUNNER_PRIVILEGED_PLUGINS to more accurately describe its behavior.

Kash · May 5, 2020, 5:31pm

I see, thanks for the clarification. I came across the above code but missed that it only applied to the official plugin images.

Would there be scope to extend this and allow users for self-hosted installs specify their own privileged images (still retaining the check for no custom commands or entrypoint), or alternatively allow this functionality via a different env var?

In my use case I have a multi tenanted environment and don’t intend to be marking any user repositories as trusted.

bradrydzewski · May 5, 2020, 5:37pm

There are a number of self-hosted installations that are public facing (e.g. open source repositories) and we use the same code at cloud.drone.io so we need to be cautious. I agree that adding a new variable is the best approach. Here is my recommendation:

rename the existing variable to DRONE_PRIVILEGED_PLUGINS and retain the same logic we have in place today. This removes any confusion.
map the old variable name to the new variable name for compatibility here.
create a new variable named DRONE_PRIVILEGED_IMAGES that toggles the privileged flag without any restrictions

bradrydzewski · May 5, 2020, 5:40pm

I will take care of 1 and 2. any chance you could send a pr for 3?

Kash · May 5, 2020, 5:41pm

Sure I’ll try pick it up this week, thanks!

Kash · May 6, 2020, 3:00pm

Created a PR here with a bit more discovery: https://github.com/drone-runners/drone-runner-kube/pull/31#issuecomment-624584398