Issues with multiline secrets concealing and being passed to environment

Drone Support
1

Conceal not working

Given a .drone.yml of

pipeline:
  test:
    image: alpine:3.4
    commands:
      - 'echo "${SECRET}"'

Having a secret of

{
  "test": "value",
  "multi": "multi\nline\nstring"
}

And adding that secret with

drone secret add --conceal jmccann/test SECRET @file.json

I would expect the logs to conceal the secrets, but they do not seem to:

{"proc":"test","pos":0,"out":"+ echo \"\"{\\n  \\\"test\\\": \\\"value\\\",\\n  \\\"multi\\\": \\\"multi\\\\nline\\\\nstring\\\"\\n}\\n\"\""},
{"proc":"test","pos":1,"out":"{n \"test\": \"value\",n \"multi\": \"multi\\nline\\nstring\"n}n"},
{"proc":"test","type":2,"pos":0,"out":"0"},
{}]

With a “simple” string of abc123 secret I do see expected masking:

{"proc":"test","pos":0,"out":"+ echo \"*****\""},
{"proc":"test","pos":1,"out":"*****"},
{"proc":"test","type":2,"pos":0,"out":"0"},
{}]

Passing multiline secret to environment not working

Given:

pipeline:
  test:
    image: alpine:3.4
    environment:
      - DPASS=${SECRET}
    commands:
      - 'echo "$DPASS"'

With multiline secret contents and method of adding described in previous section it tries to build but provides error:

ERROR: yaml: line 4: mapping values are not allowed in this context

With a “simple” secret of abc123 it works fine.

Current Workaround

Given the multiline secret above we are doing the following to workaround the issue.

pipeline:
  test:
    image: alpine:3.4
    commands:
      - 'echo "$SECRET"'

This works because (for now?) secrets are still injected as shell envvars.

Questions

  • Should conceal be expected to work in the above use case? If so I can open an issue.
  • Should setting an environment variable from a multiline secrets be expected to work? If so I can open an issue.
  • Are secrets going to ever stop being injected into the containers now that interpolation is deployed or will it continue to stick around? We are trying to come up with a “future proof” solution as best we can.

How we ran into all of this is we are using a multiline secret as described above to echo the contents to a file without exposing the secret in my logs.

2

The technical reason concealing doesn’t work with multi-line secrets is that the stdout buffer is flushed after every newline (this is controlled by the OS). Each time the buffer is flushed, drone receives a new chunk of logs which it scans for secrets, and then streams to the user interface. Multi-line secrets are split across multiple chunks and are therefore not detected.

Should conceal be expected to work in the above use case

I’m inclined to say no, given the technical limitations.

I’m not sure there is a fool proof way to prevent this, but am definitely open to suggestions. We could buffer log lines before streaming to the user interface, so that we could scan multiple lines for secrets. The challenge is that we would need to do this without visibly delaying streaming. Also how we know when we’ve buffered enough lines? How do we make sure a secret isn’t split between two different sets of buffered lines?

Should setting an environment variable from a multiline secrets be expected to work?

This is working for me. EDIT see my follow-up comment in this thread

Are secrets going to ever stop being injected into the containers now that interpolation is deployed or will it continue to stick around? We are trying to come up with a “future proof” solution as best we can.

Yes, injecting secrets as environment variables will be removed in favor of interpolation. Kubernetes and Docker are adding the ability to mount secrets as volumes which we will definitely consider going forward as well. But we’ll pretty much try to mirror existing solutions instead of having anything drone-specific.

3

EDIT one minor difference is that I declared my environment variables in map format instead of array format. See the below example:

pipeline:
  test:
    image: golang
    environment:
-     - TOKEN=${GOOGLE_TOKEN}
+     TOKEN: ${GOOGLE_TOKEN}
    commands:
      - echo $TOKEN

If I want to declare my environment variable in array format (below) I need to quote it. This is because my secret was a json file and had some characters that were tripping up the yaml parser. I think it was the colons …

pipeline:
  test:
    image: golang
    environment:
-     - TOKEN=${GOOGLE_TOKEN}
+     - 'TOKEN=${GOOGLE_TOKEN}'
    commands:
      - echo $TOKEN
4

I was also facing the same issue with multiline secrets i.e secret was printed in console output even after concealing. However, for me it worked by setting it as environment variable.

Given a multiline secret file app.secret :

###########Secrets##########\nUSER=someuser\nPWD=somepwd

and adding the secret using

drone secret add --conceal user/repo SECRET @app.secret

below .drone.yml does not print multiline secrets in console output

build:
    image: java:openjdk-8
    environment:
      - ENV_SECRET=${SECRET}
    commands:
      - echo $ENV_SECRET
5

Was the conceal feature removed in the meantime? I get the error below when trying to use “–conceal”:

Incorrect Usage: flag provided but not defined: -conceal

6

Is the conceal command available?