[Private Helm Charts] How to use Github Pages (from a Private Repo) as an Helm Chart HTTP Repo

gabriel.cerioni · May 5, 2022, 11:48pm

Howdy, gang!

Introduction

I’m proud to say that now Harness supports a Private GitHub Project Repo working as an HTTP Helm Repo, via GitHub Pages.

We are going to use GH Actions to make sure we package the Helm Chart, store it, and update the index.yaml file.

That’s a slightly different process than when you have a Public Repo, but it’s as easy as that other approach. At the end of the process, we’ll be using the http://raw.githubusercontent.com path to configure the Harness Connector, because harness will just append index.yaml at the end of the URL (+ Token).

Buckle up!

Tutorial

Requirement

An Enterprise GH Account, so you can make Private GH Pages.

First Step

Let’s create a super simple PRIVATE GitHub Repo/Project:

Second Step

Now let’s create a branch called gh-pages.
I’m already at the UI, so I won’t clone/checkout right now.
I like to do this now because it does not have many files that might get you confused later.

Here:

Third Step

You’ll want to make sure your gh-pages branch is set as GitHub Pages, on / (root).
Click on your repo Settings and scroll down to GitHub pages section and set as per below:

Alright, it looks like we are in the right direction. This is the main difference from Public Repos: that’s a random URL with an Auth layer.

Fourth Step

Now, it’s time to give this repo superpowers, by adding a GitHub Action on the MAIN branch
This means that you need to just add a file here, on the main branch: /.github/workflows/release.yml

Basically, we want an Action that will be triggered by pushes to the main branch.
Then it will read all your Chart that will be under the /charts folder at the root of your main branch, and do all the hard work for us.

It will create the .tar.gz, store the Artifact on the Project Repo itself, and then update the index.yaml file that your GH Pages will be serving. Nice, right?

It actually runs helm package, helm repo index, etc. for us. But I really like this user-friendly approach.

Important: you just need to edit the helm repo index line so it gets the pertinent raw content from your repo. Example:

helm repo index . --url https://raw.githubusercontent.com/GabsLabs/demo-private-nginx-helm/gh-pages/

You can go to your gh-branch and click on the index.yaml file. Then, you just go to the raw option to get your URL:

This is pretty much it. So, this will be the content of the release.yml file. I’ll store it in a public S3 Bucket, because this website can mess with the indentation for real.

release.yml

Fifth Step

It’s time to add the helm chart on the main branch too! We are almost done!
So, let’s do it! It will be under the /charts path, like we discussed before.

Let’s clone the repo - you do the same to yours:

➜  Harness_Labs git clone https://gabrielcerioni@github.com/GabsLabs/demo-private-nginx-helm.git

Then, I’ll just enter my repo and create the folder on the main branch:

cd demo-private-nginx-helm
mkdir charts
cd charts

Nice. Now, I’ll just create a super dummy nginx Chart:

gacerioni@Gabriel Cerioni charts % helm create community-nginx-gabs
Creating community-nginx-gabs

I won’t do much here, because I know you already have a Helm Chart ready to go.
I’ll just delete some extra entities that are on the templates folder, and call it a day.

And, in the values.yaml, I’ll just add some tricks that work pretty well with Harness:

# Default values for community-nginx-gabs.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

image:
  repository: ${artifact.metadata.image}
  pullPolicy: Always

createNamespace: true
namespace: ${infra.kubernetes.namespace}

So, I’ll just create an extra template file containing my namespace creator trick too:

vim templates/namespace.yaml

{{- if .Values.createNamespace}}
apiVersion: v1
kind: Namespace
metadata:
  name: {{.Values.namespace}}
{{- end}}

And a quick fix on the Deployment entity too:

<...>
      containers:
        - name: {{ .Chart.Name }}
          securityContext:
            {{- toYaml .Values.securityContext | nindent 12 }}
          image: "{{ .Values.image.repository }}"
<...>

Sixth Step

Nice! Time to push that to our main origin!

git add .
git commit -m "initial attempt - hail Tibia!"
git push origin main

This would be what my main branch looks like:

My chart is inside that charts folder. I’ll share the tree view, so it helps you to compare:

It looks promising! This is my Action that has just executed:

Seventh Step - This is important!

Let’s just make sure this is working.

We can see that the Action has created everything we need, even the index file.

So, it’s time to add this to Harness, using the required credentials.

IMPORTANT: The detail here is that we are not using the GH Pages URL, since this is not compatible with our strategy. We’ll use the raw.githubusercontent.com path instead.

For example, this is my Repo URL, to fit this Private use case:

https://raw.githubusercontent.com/GabsLabs/demo-private-nginx-helm/gh-pages

You can go to your gh-branch and click on the index.yaml file. Then, you just go to the raw option to get your URL:

Ok, let’s move on.
I’ll use my username and Personal Access Token (PAT).

And that’s pretty much it:

Harness will run normal helm repo add, and etc. against this repo. It has no clue this is a GH trick.

Big Last Step - Use it in Harness:

I could add the Helm Chart as the Service Artifact. So it will display all the versions on the UI.
But I’m trying to take control over the docker image under the hood, so I’m trying this different approach now. It’s just a lab.

With that said, I’ll add the Public Docker Hub nginx as the Artifact Source:

Let’s add that Repo as the Service Remote Helm Manifest!

Super nice! Let’s run it in a template Rolling Update Workflow!

No one can stop us!

Tags:

<cloud: aws, gcp, azure>
<function: ci,cd>
<role: swdev,devops,secops,itexec>
<type: howto, experts>
<category: gitops, github, kubernetes, k8s, deployment, repo, helmcharts, charts, helm, http>

Jon-Call · May 18, 2022, 2:44pm

Awesome writeup!

Also if you don’t want to use github pages you can just have a dedicated chart branch and a custom github action that does something like:

name: Release Helm Charts

# Do not change this
concurrency: release-helm

on:
  workflow_dispatch:
  push:
    branches:
      - master

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          path: 'src'
          fetch-depth: 0
      - name: Checkout
        uses: actions/checkout@v2
        with:
          path: 'dest'
          ref: 'charts'
          fetch-depth: 0

      - name: Install Helm
        uses: azure/setup-helm@v1

      - name: Package Helm Charts
        shell: bash
        run: |
          find src/charts/ -type f -name 'Chart.yaml' | sed -r 's|/[^/]+$||' | sort | uniq | xargs -L 1 helm dep up
          mkdir dest/tmp
          mkdir dest/tmp/packages
          for d in src/charts/*/ ; do
              echo "$d"
              helm package "$d" -u -d dest/tmp/packages
          done
      - name: Push New Files
        shell: bash
        working-directory: dest
        run: |
          brew install yq
          set -x
          cd tmp
          helm repo index . --url {{Insert GITHUB raw url to chart branch}}
          yq eval-all '. as $item ireduce ({}; . *+ $item )' index.yaml ../index.yaml > new_index.yaml
          mv new_index.yaml ../index.yaml
          rm index.yaml
          mv packages/* ../packages
          cd ..
          rm -rf tmp
          git config user.name "Helm Updater"
          git config user.email "actions@users.noreply.github.com"
          git add $(git ls-files -o --exclude-standard)
          git add index.yaml
          git commit -m "Updated from ref: $GITHUB_SHA"
          git push

This solution packages all the charts found on master branch in /charts in to a tmp directory, creates a new index.yaml, merges that with existing index.yaml on your “chart” branch, then copies the new chart bundles over before committing and pushing that back to the chart branch.