How do I Run 1 CI Job at a Time | How to Use "queue" step in CI Stage/Pipeline

Harness Experts
, , , ,
1

16d12d54b3b4ae7aabc8a93417570bce0984e3c9

How do I Run 1 CI Job at a Time | How to Use the “queue” step in CI Stage.

The CI stages currently do not support barriers. This is, however, part of our current roadmap and will be added as the Platform continues to evolve. We plan to add support for the “queue” step to CI.

As part of this article, we will provide a simple workaround to have a “queue” step in CI Stage.

  • To achieve this, you can use the Custom stage as part of your CI Pipeline.

  • I’ve created a simple pipeline to demonstrate it.

  1. YAML :
pipeline:
  name: queue test
  identifier: queue_test
  projectIdentifier: playground
  orgIdentifier: default
  tags: {}
  stages:
    - stage:
        name: stage with queue step 2
        identifier: ggfgddd
        description: ""
        type: Custom
        spec:
          execution:
            steps:
              - step:
                  type: ShellScript
                  name: sleep 2
                  identifier: sleep
                  spec:
                    shell: Bash
                    onDelegate: true
                    source:
                      type: Inline
                      spec:
                        script: sleep 2
                    environmentVariables: []
                    outputVariables: []
                  timeout: 10m
              - step:
                  type: Queue
                  name: queue
                  identifier: ddd
                  spec:
                    key: myKey
                    scope: Pipeline
                  timeout: 10m
              - step:
                  type: ShellScript
                  name: sleep 10
                  identifier: sleep_10
                  spec:
                    shell: Bash
                    onDelegate: true
                    source:
                      type: Inline
                      spec:
                        script: sleep 10
                    environmentVariables: []
                    outputVariables: []
                  timeout: 10m
        tags: {}
  flowControl:
    barriers:
      - name: key
        identifier: key
  1. VISUAL:
    image
    image2976×1222 239 KB

  • You can see that I have a queue step that is locking "myKey" and its scope is a pipeline.

  • This means that if you trigger multiple pipelines, The first one will run and make the lock, and the following pipelines will not start until the first one is finished.

  • If your second stage is the CI stage, and the first stage is a Custom stage that does the lock, it should give you the desired behaviour.

If you are planning to use Controlling Resource Usage with Barriers, Resource Constraints, and Queue Steps in your Harness CD Stage or Pipeline, then please go through our documentation here, which talks about this in detail: Controlling Resource Usage with Barriers, Resource Constraints, and Queue Steps - Harness.io Docs.

We need to follow these steps to Use the “queue” step in CI Stage/Pipeline in Harness NextGen.

3 Likes