Only build after matrix

Drone Support
1

Hi,
Apologies if asked before, is there a way to build a step before or after the matrix? I’d like to (for example) only publish, deploy and notify (once) if all versions of the matrix pass. Likewise I’d imagine there are steps that could be done before the matrix that you might only want done once.
M.

2

I came here with a very similar question. Basically: I want a matrix to only be tied to a specific build step (or steps), while all other steps are matrix-agnostic and only execute once.

I think one way to hack it would be to use a conditional step with a specific matrix permutation, but it’s a little awkward.

3

In this case, matrix might not be the best option since they are treated by Drone as separate entities. I would suggest using the parallel step execution, as an alternative to matrix builds http://docs.drone.io/pipelines/#parallel-execution

To help illustrate the difference between matrix builds and parallel execution, let’s look at two different examples:

Example 1: Matrix Execution

In the below example we have a matrix build that will send two Slack notifications. Ideally we would only send one Slack notification once both permutations of the build step are complete.

pipeline:
  build:
    image: golang:$version
    commands:
      - go build
      - go test
  notify:
    image: plugins/slack

matrix:
  version: [ 1.9, 1.8 ]

Example 2: Parallel Execution

You can use the group attribute to group pipeline steps and fan-out and execute pipeline steps in parallel, and then fan-in to execute a single Slack notification:

pipeline:
  build18:
    image: golang:1.8
    group: build
    commands:
      - go build
      - go test
  build19:
    image: golang:1.9
    group: build
    commands:
      - go build
      - go test
  notify:
    image: plugins/slack
4

Makes sense, thanks @bradrydzewski.

The problem with the Parallel Execution example is the amount of duplication required. If you have for example 10 variations, it starts to get a bit unwieldy. I know I can merge yaml from an anchor, but I haven’t figured out how to parameterize it yet. Was trying this example below but $BUILD_NAME was empty:

build: &build
  group: build
  commands:
    - echo Building ${BUILD_NAME}
    - # do a bunch
    - # of other common stuff here

pipeline:
  build1
    <<: *build
    environment:
      - BUILD_NAME="foo"

  build2
    <<: *build
    environment:
      - BUILD_NAME="bar"

Populating BUILD_NAME values was the main reason for using a matrix.

5

@devth this won’t help in the short term, but I’m planning on adding support for a .drone.js file as an alternative to yaml. Using an actual programming language for configuration should really help with this use case. I have a working proof of concept that I demoed recently, but there is still a lot of work to be done.

In the meantime, have you considered generating the yaml file programmatically from a template? I know this is how the majority of teams using GitLab CI work around its lack of matrix support.

6

Yep, was starting to think about using templates. Might look into jinja2.

.drone.js sounds cool long term.

7

I would also accept a pull request to drone-cli to generate yaml files using Go template syntax, if you are interested in having a more “official” option.

8

Cool. I will give it some thought. The immediate downside is we then need to generate the config as a pre commit hook (I think). Unless Drone automatically generated the template after clone.