Hello Gang 
As software applications grow and become more complex, managing the resources they require can become a significant challenge. In particular, managing the resources used by multiple concurrent processes can be a delicate balancing act, as too much demand can overwhelm available resources and cause performance problems, including crashes and downtime.
If you’re using Harness for continuous delivery, you may have encountered this issue, especially when running multiple Terraform pipelines simultaneously. When your Harness delegate runs out of memory, it can cause serious issues, including crashes and errors. Fortunately, there is a solution to this problem: using queue steps to control resource usage.
Harness provides a feature called “queue steps” that allows you to control the level of concurrent execution at the delegate level. This feature enables you to define a specific number of steps that can be executed at the same time, effectively limiting the amount of resources used by the concurrent processes.
Here’s how you can use queue steps to control resource usage in your Harness pipelines:
- Open the pipeline you want to modify in Harness.
- Click on the stage where you want to add the queue step.
- Click the “Add Step” button, and select “Queue” from the list of available steps.
- Configure the queue step by specifying the number of steps that can be executed at the same time. You can also set a timeout value, which determines how long the step will wait for a free slot before timing out.
- Add the steps you want to run within the queue step. These steps will be executed one at a time, up to the maximum number specified in the queue step.
- Save your changes and run the pipeline.
By using queue steps in this way, you can limit the number of concurrent steps running at any given time, thereby controlling the amount of resources used by the pipeline. This can help prevent issues like crashes and errors due to memory exhaustion.
In conclusion, if you’re experiencing issues with Harness delegates crashing due to memory exhaustion, consider using queue steps to control resource usage. With this feature, you can limit the amount of concurrent execution and prevent performance issues from occurring. For more information about queue steps and how to use them in your pipelines, check out the Harness documentation on “Control Resource Usage with Queue Steps”.