Hello Everybody !!
Introduction
Harness provides a vast array of Variables and expressions, which can used to parameterise settings in your Harness components, create environment variables, creating Workflow variables, and it also allows you to create your own variable expressions to create Golden components which can be used a base to build Applications on or pass information through your Workflow & Pipelines.
We also provide built-in expressions to reference a majority of the settings a Component is using and information from a Deployment. Having said that, We have noticed recently some customers using the process of Nesting these custom and built-in Variables to achieve their use-cases.
In this Article I will share with you the various ways we can use the Variables Harness provides which are built-in and a few Custom ones created by us and see the process of Nesting them and achieving the required information during Deployments.
Lets Get Started.
Setup:
In order to test these scenarios and use-cases we would need a simple K8s Application in Harness that runs a Rolling Workflow.
The Workflow would look something like this :
For this test we will not be including the Rollout Step in the Workflow as we do not plan on Deploying anything to the Infrastructure but rather to run a Shell script to simulate the Nesting of the Variables.
Common Use-cases :
Nesting using Built-In Variables
Lets consider a Use-case where the requirement is to fetch a particular secret from the Harness Secret Manager to use in the Deployment Steps. However, the catch here is that the secret that is to be picked up depends on the Infra Definition we are using at the time of Deployment.
In this scenario we are taking a look at nesting 2 inbuilt variables provided by Harness. As an example we have chosen to use the In-Built variable ${infra.name} which fetches the Infrastructure name we are Deploying with and nesting this in the Get secrets Variable.
echo "${infra.name}"
echo "${secrets.getValue("${infra.name}-deploy")}"
We will be using the above statement in a shell-script step to fetch the Secret Value from a secret with a name that depends on the resolution of the Nested Variable ${infra.name}
The Infrastructure Definition name variable is being used here as in this case the Infra Defs are as follows:
Depending on the use-case when we select a certain Infra-Def it will only pickup the Secret which is required to be used by that Infra with the help of Nesting.
Taking the above mentioned echo statement into account the variables would resolve to the following:-
-
${infra.name} would resolve to Prod as this is the Infra Definition we have selected at the time of Deployment.
-
The above variable resolution would result in the Get Secret variable
${secrets.getValue(“Prod-deploy”)} looking for a secret with the name *Prod-deploy
Based on this nesting if we were to deploy to the Infra Dev or Stage the resolution would take place appropriately and only those secrets would be picked up.
We can use any of the Built-in Variables Harness provides for its components to support this use case.
Nesting using Custom Variables
Lets consider the same use-case as above but in this scenario we will be leveraging the power of the Custom variables we can create in order to nest them within the same Get Secrets Variable
In this case lets create a Workflow Variable called env_type and provide it a value called “Dev”
In this example the reason why we are not leveraging the same Infra.Name variable to fetch the secrets like in the previous case is because in most cases users usually do not name their Infrastructure Definitions as Dev, Stage & Prod but rather use different names based on their requirement.
In these cases what we can leverage, is using Custom Variables to resolve to the appropriate secret required as below :
echo "${workflow.variables.env_type}"
echo "${secrets.getValue("${workflow.variables.env_type}-num")}"
We will be using the above statement in a shell-script step and the deployment would look like this:
Taking the above mentioned echo statements into account the variables would resolve to the following:-
-
${workflow.variables.env_type} would resolve to Dev as this is the value we have selected at the time of Deployment. This was also one of the Allowed values we gave this Custom Variable when creating it.
-
The above variable resolution would result in the Get Secret variable becoming
${secrets.getValue(“Dev-num”)} looking for a secret with the name Dev-num
Nesting using a Combination of Built-in & Custom Variables
In this use-case while again considering the above scenario we will be using a combination of Built-in & Custom Variables to fetch the appropriate secret Value.
echo "${workflow.variables.NON_PROD}"
echo "${env.environmentType}"
echo "${secrets.getValue("${workflow.variables.NON_PROD}-${env.environmentType}")}"
In this example we will be using the the above statement in the Shell script step and the deployment would look like this:
Taking the above mentioned echo statements into account the variables would resolve to the following:-
-
${workflow.variables.NON_PROD} would resolve to harness as this is the value we have selected at the time of Deployment. This was also one of the Allowed values we gave this Custom Variable when creating it.
-
${env.environmentType} is the built in variable used to fetch the type of Environment being used in the Deployment. This would resolve to either PROD or NON_PROD. In this case our Environment was of type NON_PROD
-
The above variable resolution would result in the Get Secret variable becoming
${secrets.getValue(“harness-NON_PROD”)} looking for a secret with the name harness-NON_PROD
We can use other Built in Variables in this case as well.
Limitations
There are certain limitations to how we can nest variables when using them in a Workflow, consider the below example:
echo "${secrets.getValue("${workflow.variables."${env.environmentType}"}-zee")}"
The above echo Statement would not resolve to what is expected due to one main reason which is the change we are making to a Buil-In Variable.
This is the same condition with other built in variables where we cannot change their structure to include another nested built-in or Customer Variable.