Shared Storage/Caching in Drone CI Runs

Drone Support
1

I’m trying to dynamically generate a client library off an OpenAPI specification as a build artifact in my Drone CI/CD pipeline and publish it to an Artifactory registry. For numerous reasons this has to happen in the service repository that contains the API specification - I can’t pop it in another repo that “listens” for changes in the API spec, for example.

I can successfully generate the client and all that jazz, but the issue I have is: My CI pipeline will generate & publish a “new” version of the client on every merge to master.

This is suboptimal, because the service API might not have really changed.

My build step that does the publishing is specified as follows:

  - name: publish-ts-library
    image: node
    environment:
      ARTIFACTORY_USERNAME:
        from_secret: ARTIFACTORY_USERNAME
      ARTIFACTORY_PASSWORD:
        from_secret: ARTIFACTORY_PASSWORD
      RC_RELEASE: false
    commands:
      - bash ./publish-ts-library.sh
    depends_on:
      - build-read-gateway-ts-client
    when:
      branch: master
      event:
        exclude:
        - pull_request

The shell script executed is:

#!/usr/bin/env bash

touch .npmrc
curl -u$ARTIFACTORY_USERNAME:$ARTIFACTORY_PASSWORD https://mycompany.jfrog.io/mycompany/api/npm/auth > .npmrc
echo "registry=https://mycompany.jfrog.io/mycompany/api/npm/npm" >> .npmrc
if [ $RC_RELEASE ]
then
    sed -i '/version/s/[^.]*$/'"0-dev${DRONE_BUILD_NUMBER}\",/" package.json
else
    sed -i '/version/s/[^.]*$/'"0.${DRONE_BUILD_NUMBER}\",/" package.json
fi
npm publish

I know it’s pretty gross, apologies.

What’s happening is pretty simple: we grab some auth credentials from the secrets in Drone, and pop them in a .npmrc file. Then, if it’s an RC release we add dev to the version, else we just bump the version to whatever the drone build number is.

So, my initial thinking was to take a sha1sum of the OpenAPI spec yaml file that I use to generate the library, and compare it to the sha of the “last” generated client - if it matches then no-op instead of npm publish , but the kicker is I have no idea where I can store this SHA, and how to configure drone to have access to this storage.

Crucially, this OpenAPI spec yaml file only becomes available as part of the build pipeline - it’s something that’s generated at compile time.

I’m aware of plugins like Volume Cache | Drone - but they seem decidedly ephemeral, I need to be assured that this SHA will be persisted across builds consistently, and not suddenly “vanish” because a cache gets cleared, thereby forcing a republish + bump of an otherwise identical generated client.

Any ideas?
Thanks!

2

What’s wrong with publishing the same artifact multiple times? If your workflow is designed to release a version on every build (e.g. using semantic release), it’s not only totally fine to release identical artifacts several times even under different versions, but I would even expect it to do so. Otherwise there would be released versions, missing their artifacts and silently assume, you are gonna go for older ones.

3

So, the issue with that is the API might not have necessarily changed. E.g. someone might have updated a comment in code somewhere, which doesn’t necessitate a whole new client library be released by the CI pipeline.

Doing so on every release would mean downstream consumers of the library are perpetually updating their lib version for no reason.

4

Because Drone is ephemeral, Drone does not have any native mechanism to support storing values between builds. The good news is that anyone can extend how Drone works through plugins … the sky is the limit when it comes to plugins, but you would be required to design and implement your solution (assuming you have looked at existing plugin and determined none satisfy your usecase).

For example, you could read and write the shasum to an external storage like S3, you could read and write the shasum to an orphan branch in your repository, you could read and write to a gist, etc.