This article is going to discuss the process of setting up Triggers to execute pipelines based on Artifacts and the conditions and Actions used to set it up. We will also cover common issues users run into with Trigger Actions. Let’s get Started.
Let’s create two different services on Harness. I’ve chosen ECS for the exercise.
Next I’ll be using one Environment with 2 Infra Definitions for Pre-prod and Prod
Now let’s get the Workflows created for our Pipeline.
Once that is complete let’s setup our Pipeline.
Now let’s discuss the Trigger Cases.
Deployment getting Triggered with the Old Artifact
In this example we will discuss the setup of a Deployment Pipeline with 3 Stages which is executed by a Trigger :-
- First Stage is executed with the new Artifact pushed into the Source and is deployed on the Stage Infrastructure.
- Second Stage is an approval step to verify.
- Third Stage is used to install the “Last Successfully Deployed” artifact on the Prod Infrastructure.
When we specify the Action “Last Successfully Deployed” in the Trigger the context on how this works is that like the name suggests, the trigger will use the Artifact which was run the last deployment for the pipeline which was successful.
This means that when we are using the trigger to execute a Pipeline, it is required for us to have a pre-existing successful deployment for the pipeline from which it would pick up the artifact.
The common occurrence of an issue we see here is the 3rd Stage deploying the old Artifact from the source. The artifact to be deployed was “nginx-0.1.1.tgz” but it deployed “nginx-0.1.0.tgz” this is due to the conditions on the Trigger end.
We see here that the Action to be performed by the Trigger for the 3rd Prod Workflow using the Service-2 of the pipeline is to use the “Last Successfully Deployed” Artifact.
In this event there are a few cases why this can occur:-
- This is the first Deployment for the Pipeline and there is no previous deployment where we can pick up the artifact from
- The previous run for the Pipeline was not Successful
A pipeline execution is considered to be successful only if all the Stages in it have been completed. To add to this if we were to take an example of the below Pipeline which ran all its stages successfully and deployed the artifact “nginx-0.1.1.tgz” but the last step which is the Approval Step was rejected.
This causes the overall execution to be marked as a failure and so in-turn this run of pipeline is not considered by the Action “Last Successfully Deployed”.
So when we do trigger our Deployment the Artifact in the most recent run which was marked as successful was “nginx-0.1.0.tgz” which will be used, this is why we see the old Artifact being used instead.
We can avoid these issues by making sure all the steps of the Pipeline are complete as below in order for the Trigger actions to work as expected.
Here we can see after letting the Pipeline step complete with all its stages when the Trigger is executed it deploys with the required Artifact.
Pipeline Workflow step deploying with the wrong Artifact
Consider the following Pipeline with 2 Workflows, each with its own Service & Artifact Source. This pipeline is executed by a Trigger when a new Artifact is pushed to the Artifact source of a Service.
The Actions set on the Trigger end for the Services of the Workflows look something like this where the Pipeline is Triggered when a new Artifact is pushed to Service-1 while Service-2 uses the action “Last Collected”.
This means that Service-2 will be using the Last Collected artifact from its Source.
In order to Trigger the pipeline we will now push the Artifacts to the S3 buckets for both Service-1 and Service-2. The pipeline is triggered as the Artifacts are pushed to the Sources based on the above Trigger conditions.
We see here that although the new Artifact versions (mainline-perl) were pushed to both the artifact sources the Stage WF uses the new artifact while the Prod Workflow was deployed with an older artifact in the source queue (stable-perl).
The cause for this to occur is related to the significance on time and when these Artifacts showed up on the Harness end. We see that the Pipeline would trigger once a new Artifact was pushed for Service-1 and then it would wait at the Approval step before proceeding on to run the Prod Workflow which is to use the “Last Collected” Artifact.
From what we see on the Service-2 Artifact Source history the Artifact to be deployed was (mainline-perl) according the to the Trigger action “Last Collected”. Something important to note here is the time at which the new Artifact for Service-2 was pushed to the source, we see that the Artifact was pushed at 12:35pm to Service-2. But the pipeline was triggered at 12:32pm when the new Service-1 artifact was pushed.
The reason an old Artifact (stable-perl) was deployed in the Prod WF was because at the time the Trigger was executed at 12:32pm the only artifact in the queue for Service-2 which matches the “Last Collected” Action was “stable-perl”. We can see from the Timestamp that the new artifact for Service-2 only showed up at 12:35pm. By this time the pipeline has already initialised and resolved all the required artifacts it would need to run all the stages of the Pipeline.
It is important to keep in mind the context of these Actions on the Trigger end as they depend on various factors and a common thats missed out is the Timestamp. To avoid the above issue we can push all the artifacts to the sources with the exception of the Artifact which would execute the Trigger. This way the Trigger would execute all the stages of the pipeline with the appropriate artifacts.