Howdy, gang!
Introduction
Harness is about to release a new option to integrate with Vault, to serve it as a Secrets Manager.
We are talking about a safe and reliable method called Vault Agent.
I had some difficulties setting up a good integration, that reduced the toil to 0.
I mean, I was not sure how to handle the underlying Auth method options provided by Vault. There’s TTL everywhere, RoleIDs, Secrets that expire, etc.
So, this is my humble contribution to anyone suffering from the same.
Buckle up!
Tutorial
Requirement
A Vault Server, good credentials, and some patience.
Tasks on Vault
I’ll create a ‘kv’ Secrets Engine called Harness
:
Then, I’ll create a policy named harness_v2_engine_gabs
:
path "harness/*" {
capabilities = ["create", "list", "read", "update", "delete"]
}
path "harness" {
capabilities = ["create", "list", "read", "update", "delete"]
}
path "auth/token/renew-self"
{
capabilities = ["read", "update"]
}
# List existing secrets engines.
path "sys/mounts"
{
capabilities = ["read"]
}
Ok, I guess we’re ready to roll.
First Step
I’ll enable the AppRole
method via the Vault UI console, then I’ll switch to the API (using a powerful token) to perform the following operations
First, we run that to enable the AppRole
method:
vault auth enable approle
Then, we can create our Role:
curl \
--header "X-Vault-Token: s.<...>" \
--request POST \
--data '{"token_ttl": "60m", "token_max_ttl": "120m", "token_policies": ["harness_v2_engine_gabs"], "bind_secret_id": true}' \
http://15.228.43.18:8200/v1/auth/approle/role/harness_gabs_agent
I guess it worked!
Second Step
Alright, time to configure our Vault Agent
We first start by retrieving the very important RoleID
:
curl \
--header "X-Vault-Token: s.pasnBqCp46cuJov6COJ7uTw3" \
http://15.228.43.18:8200/v1/auth/approle/role/harness_gabs_agent/role-id
{"request_id":"<...>","lease_id":"","renewable":false,"lease_duration":0,"data":{"role_id":"4ecb33<..>"},"wrap_info":null,"warnings":null,"auth":null}
And now, our “not so loved” SecretID:
curl \
--header "X-Vault-Token: s.pasnBqCp46cuJov6COJ7uTw3" \
--request POST \
http://15.228.43.18:8200/v1/auth/approle/role/harness_gabs_agent/secret-id
{"request_id":"<...>","lease_id":"","renewable":false,"lease_duration":0,"data":{"secret_id":"e72d<...>","secret_id_accessor":"3aa<...>","secret_id_ttl":0},"wrap_info":null,"warnings":null,"auth":null}
Ok, keep those two IDs in your mental Vault, a.k.a. your brain (or Notepad++, since this is a Lab).
Third Step
Time to get the Vault Agent!
It’s super important to mention that Harness only needs access to the sink
file that the agent will write the tokens into.
I’ll run the Vault Agent at the same Server that is hosting my Harness Delegate.
But, depending on your SecOps team, maybe they’ll come up with a shared volume or anything like that.
The only required thing is to make sure that the Harness Delegate is capable of reaching the sink file path.
The first thing you do is to get the vault on that Delegate. We’re not going to start the Server, ok? It’s the agent, and I guess it is in the same bundle (at least, the easier way I could find):
And this is my Vault Agent Home, at /etc/vault
:
I’ll explain each file in the next step!
Fourth Step
The first thing I’d like to do is to write a good config.hcl
This might be different, depending on your prefered Auth Method and Security restrictions, but I’m using the AppRole Method in this lab. So this is a good starting config file:
pid_file = "./pidfile"
vault {
address = "http://15.228.43.18:8200"
tls_skip_verify = true
retry {
num_retries = 5
}
}
auto_auth {
method "approle" {
config = {
role_id_file_path = "./roleid"
secret_id_file_path = "./secretid"
remove_secret_id_file_after_reading = false
}
}
sink "file" {
config = {
path = "./sink"
}
}
}
cache {
use_auto_auth_token = true
}
listener "tcp" {
address = "127.0.0.1:8100"
tls_disable = true
}
You can use this documentation to enhance or even change it:
So, my current config asks me to provide the RoleID + AppID. And this is what I did here:
Fifth Step
Ok. Time to run and test the Agent Credentials!
Let’s run the agent. If you are confident, you can use it with nohup
!
vault agent -config=/etc/vault/config.hcl
<or>
nohup vault agent -config=/etc/vault/config.hcl &
There’s no official way to run and enable the Vault Agent, and that’s why I used nohup in this brief tutorial. But, if I’m responsible for deploying it in Prod, and considering that I will run this in a good old Linux env, I’d create a Systemd Service.
You can follow this tutorial, in case you want to change the way the Vault Agent behaves:
So, in our case, this is what I’d come up with:
vim /etc/systemd/system/vault-agent.service
Adding this very smart content:
[Unit]
Description="Vault Agent to serve Tokens"
[Service]
User=root
WorkingDirectory=/etc/vault
ExecStart=/usr/bin/vault agent -config=/etc/vault/config.hcl
Restart=always
[Install]
WantedBy=multi-user.target
Some cool commands:
sudo systemctl daemon-reload
sudo systemctl start vault-agent
sudo systemctl status vault-agent
sudo systemctl enable vault-agent
Voilá:
Hopefully, you will see something like this:
Sixth Step
Nice! Time to test it with Harness!
Important: Currently, this capability for the Secrets Manager is behind a Feature Flag.
You can ask your CSM or any contact to enable that (I can do, just open a ticket).
But, since this is going to be GA in the next week, I don’t think you need to do that.
Alright, let’s get that Secrets Manager working!
Access your Harness UI -> Secrets Manager -> Configure Secrets Manager (blue icon) -> Add Secrets Manager
.
Please pay attention to the new fields, such as Delegate Selector
and Sink Path
.
But, it looks pretty easy to configure, right? Take a look:
Big Last Step - Test that Vault!
Let’s create a very nice secret via Harness UI
And it’s working!!!
Outcome
I guess this is a safer and reliable approach to make Vault your Harness Secrets Manager.
I mean, if you need TTL in secretID, this might add a little Toil to make the Agent keep the good credentials, but this is eligible to automation, IMHO.
Further reading:
Tags:
<cloud: aws, gcp, azure>
<function: ci,cd>
<role: swdev,devops,secops,itexec>
<type: howto, experts>
<category: gitops, vault, secrets>