[Proposal] Add support for variable indirection in drone configuration file

Drone Support
1

I have several projects that are built, shipped and deployed by drone. I also manage multiple AWS accounts (dev, stage, prod etc) and Drone runs in a separate account (CI). Drone pushes to ECR and invokes Lambda functions in all other accounts using cross account IAM roles.
At the moment the pipeline configuration for all projects contain three copies of every step, for example, pushing the container to each account is repeated three times with conditionals.

To help keep dron files maintainable I propose to add variable indirection support while substituting variables in drone file.

Lets say I have two variabls

  • VAR_NAME=Gufran
  • USER_Gufran=enabled

and an expression:

${USER_${VAR_NAME}}

where value of VAR_NAME can change across builds (deployment parameter for example) and USER_Gufran is a build secret.

The expression should first resolve the value of VAR_NAME and then ${USER_Gufran}. Finally, enabled should be substituted for the expression.

If this expression is available then I can write each step in the pipeline without conditionals:

pipeline:
  push:
    image: plugins/ecr
    repo: ${ACCOUNT_ID_${DRONE_DEPLOY_TO}}.dkr.ecr.us-east-1.amazonaws.com/app

and put ACCOUNT_ID_PRODUCTION, ACCOUNT_ID_STAGING in build secret.

Bash has support for indirect expansion (${!var}) and variable indirection (\$$var) but those are quite powerful and complex features. To keep things simple and consistent drone should utilize existing syntax for variable substitution and perform only one pass for secondary substitution.

If this is something acceptable then I can submit a pull request to https://github.com/drone/envsubst

2

The envsubst library is meant to emulate bash. If I execute the example syntax in my bash prompt I get an error message:

export VAR_NAME=Gufran
export USER_Gufran=enabled
echo ${USER_${VAR_NAME}}
-bash: ${USER_${VAR_NAME}}: bad substitution

So while I do appreciate the offer, I would not accept a PR that implements logic that does not attempt to accurately emulate bash behavior.

Perhaps something like anchors might help simplify your YAML file?

docker_config: &docker_config
  image: plugins/docker
  secrets: [ docker_username, docker_password ]
  repo: foo/bar

pipeline:
  staging:
    tags: latest
    <<: *docker_config
  production:
    tags: 1.0
    <<: *docker_config
3

@bradrydzewski I would’ve gone for indirect expansion syntax ${!varname} but it will be of no use in drone file. Bash syntax uses the value of the variable as the name of variable to be expanded. It works for bash because variables can be allocated and their value can be changed at runtime, it allows a program to create the name of variable before expanding it. that is not the case with YAML file.
If we want to closely emulate bash and use ${!varname} syntax then do you think introducing a new variables block is possible? If yes, then those two combined can solve all the problems:

pipeline:
  push:
    image: plugins/ecr
    repo: ${!account_id}.dkr.ecr.us-east-1.amazonaws.com/app

variables:
  account_id: account_id_${DRONE_DEPLOY_TO}

Do you think this is doable? and of course I am going to work on the feature if we come to an agreement.