Configmap Kubernetes updates from file without Pod restart

Harness Support
1

Hi community,
I’m using Harness to deploy my services to Kube.
So there is a Helm charts and when I trigger Deploy the corresponding service is rolled over.
There is a volume mounted to the pod from config maps in order to have this volume updated.
I’m using it to update logging configuration (it is very environment specific and it is a whole file, not just variables aka DEBUG vs INFO)
So in my configmaps I have a configuration like:

apiVersion: v1
kind: ConfigMap
...
data:
  log4j.xml:
{{ if .Values.logingConfigContent }}
    {{ .Values.log4jFileContent | quote }}
{{ else }}
    {{ (.Files.Get "defaults/log4j.xml") .| quote }}
{{ end }}

So if nothing explicitly specified the file will be mounted to the pod from /defaults/log4j.xml that is zipped together with the helm chart package.
Now, I’d like to make it environment specific
Harness provides the ability doing with “Service Configuration Overrides”
and I’m using Values YAML override option to change the default values.yaml with environment specific file from repository - not sure if this will work well for XML file, haven’t tried it yet, because I already have one Values YAML override for environment specific aka prod-values.yaml and Harness prevents me from adding 2nd.
File Override option doesn’t look appealing because it allows to load a file from local and I’d like to point it to the repository (git).
Having log4jFileContent inside the values.yaml doesn’t look maintainable (having big xml or yaml file inside another file is error prone)
Having prod-log4j.xml, dev-log4j.xml files inside helm chart doesn’t work for me because I’m not allowed to have env specific settings inside helm chart.
So the 1st question I have:

  1. How to supply environment specific plain file (not /production/values.yaml , but /production/log4j.xml) using Harness?
    With plane kubectl it is possible to do like that:
    updates configmap from the file

kubectl -n mynamespace create configmap mynamespace-configmap --from-file=./production/new/config/log4j.xml -o yaml --dry-run | kubectl apply -f -

with helm it is possible to do like that (saying helm to load variable content from the file):

helm upgrade myrelease mychart-1.0.0 --set-file=logingConfigContent=./production/new/config/log4j.xml

I just don’t see whether it is possible with Harness and what would be the way to do so?
The second question I have:
2. How can I make sure, when the configmap is updated pods are not restarted - this is must have requirement.
kubernetes doesn’t restart pods when configmap is updated (example with kubectl above)
helm applies logic to compare current with new state and doesn’t restart pods unless deployment configuration changed (you have to play some tricks to force pods redeployment upon configmap change https://v3.helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments for reference)
How Harness manages state and How I can configure it not to restart pods if only configmap is changed?

Hope someone could help and I provide enough details.
Please let me know if there are any thoughts,
Thanks

2

Hey Oleksii,

Q1 - Harness lets you add multiple values.yaml overrides (just comma separate the file paths). You can alternatively add one values.yaml remote link in the Service page (service configuration override), and the 2nd log4j.xml override in the Service Overrides in the Environment page. Useful if the two files are in different repos, or if you want to override per Environment, which it sounds like you plan to do. Here’s a link to the docs for this feature - Override Values YAML Files - Harness.io Docs

image
image798×458 28.4 KB

Q2 - Still checking with the team on how we handle this, stay tuned!

1 Like
3

log4j.xml is specific to the environment, kind of /production/log4j.xml, /staging/log4j.xml
It is not specific to the service over all. When service is deployed /production/log4j.xml and /production/log4j.xml should go together.
I’m afraid if I specify log4j.xml at the service layer it would be specific for all the environments, not the specific one.
Is it the case?

4

Not a problem, you can add multiple values.yaml (or log4j.xml) remote overrides to an environment override, which would only apply to the service(s) being deployed into that specific environment and not others. Here’s a doc reference with more info - https://docs.harness.io/article/4m2kst307m-override-service-files-and-variables-in-environments#override-a-service-configuration-in-an-environment

5

Great, it looks like what I’m looking for. Let me check and I’ll reply with the feedback!

6

Hi @Arunav,
I have tried this approach pointed to the files on my git aka
environment.yaml, log4j.xml and got the following error
Invalid values file. Object is not a map…
Looks like Harness requires explicit yaml format for the file,
Any ideas how to make it work with other formats?

Thanks,

7

As a sanity check, could you please try renaming the file to .yaml format, and seeing if the above step works? Would be good to isolate if it’s filtering on filetype, or whether the formatting of the contents is the issue.

8

So I’ve got

INFO   ...    Successfully fetched following files:
...
INFO   ...   - path/to/file/staging/log4j.yaml

And got the same error
Invalid values file. Object is not a map…

9

Hi @Arunav,
Any ideas?