Helm Charts: How We Manage 12 Services Across 3 Clusters

By the fifth service on our healthcare platform, the Kubernetes YAML situation was unsustainable. Each service needed a Deployment, Service, ConfigMap, Ingress, HPA, and ServiceAccount. Multiply by 3 environments (dev, staging, prod), and we were maintaining 90+ YAML files with nearly identical structures and three values changed per environment.

We copy-pasted a ConfigMap, forgot to change the namespace, and pushed staging database credentials to production. That was the last time we managed raw manifests.

The problem

Raw Kubernetes manifests are verbose, repetitive, and environment-specific by necessity. The YAML for a production deployment differs from staging in 4-5 values: image tag, replica count, resource limits, ingress hostname, and maybe a feature flag. Everything else is identical.

Without templating, you maintain N copies of the same file. Without packaging, you deploy resources one at a time and hope you didn't miss one. Without versioning, rollback means "find the old YAML in Git history and kubectl apply it."

Our principles

1. One chart per service, not per resource. A service's Deployment, Service, Ingress, HPA, and ConfigMap all belong in one chart. Never separate them.
2. Values files are the only thing that varies. Templates are generic. values.yaml is the default. values-staging.yaml and values-prod.yaml override what differs.
3. _helpers.tpl eliminates duplication. Shared labels, selectors, annotations, and naming conventions live in helpers. Copy-paste between templates is a bug.
4. Chart.lock is non-negotiable. Like package-lock.json, it ensures reproducible deployments. We've been burned by a sub-chart updating between deploys.

Chart structure

services/
  patient-portal/
    Chart.yaml
    Chart.lock
    values.yaml
    values-staging.yaml
    values-prod.yaml
    templates/
      deployment.yaml
      service.yaml
      ingress.yaml
      hpa.yaml
      configmap.yaml
      _helpers.tpl
  appointment-api/
    Chart.yaml
    ...

Each service lives under services/ in our infra repo. ArgoCD watches each directory and syncs changes automatically.

What the templates look like

Our _helpers.tpl defines every label and selector once:

{{- define "app.labels" -}}
app.kubernetes.io/name: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Values.image.tag | default .Chart.AppVersion }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

The deployment template uses these helpers and pulls everything else from values:

spec:
  replicas: {{ .Values.replicaCount }}
  template:
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          resources:
            limits:
              cpu: {{ .Values.resources.limits.cpu }}
              memory: {{ .Values.resources.limits.memory }}

The values file for production differs from staging in exactly the places that matter:

# values-prod.yaml
replicaCount: 3
image:
  tag: "v2.4.1"
resources:
  limits:
    cpu: "500m"
    memory: "512Mi"
ingress:
  host: portal.client.health

Helm vs Kustomize

We use both. They're not competing tools -- they solve different problems.

Concern	Our choice
Application packaging and deployment	Helm
Cluster-level base config (namespaces, RBAC, network policies)	Kustomize
Templating with complex logic	Helm (Go templates)
Simple per-environment overrides	Kustomize (patches)

Helm handles the application layer. Kustomize handles the platform layer. Mixing them in the same layer creates confusion.

What we learned

• Pin sub-chart versions obsessively. We use Bitnami's PostgreSQL chart as a dependency. An unnoticed minor version bump changed the default authentication method and broke connections for an hour. Chart.lock prevents this. Run helm dependency update deliberately, not accidentally.
• helm template in CI, always. Every PR that touches a chart runs helm template to render the manifests. This catches template errors before they hit the cluster. It also lets reviewers see the actual YAML that will be applied.
• Keep values flat. Deeply nested values objects are hard to override and hard to read. replicaCount: 3 is better than deployment.scaling.replicas: 3.
• Never use helm install against production. We deploy exclusively through ArgoCD, which uses Helm to render templates but manages the lifecycle itself. No Tiller, no Helm release state in the cluster.

The tradeoffs

• Go template syntax is painful. Whitespace control, nested if blocks, and the include function have sharp edges. Budget time for template debugging.
• Chart testing is immature. helm unittest works but it's testing YAML output, not behavior. You still need integration tests that deploy to a real cluster.
• Upgrade friction. Helm chart upgrades (especially for third-party charts like cert-manager or ingress-nginx) require careful diffing. We run helm diff upgrade before every major version bump.

Our recommendation

If you're deploying more than 3 services to Kubernetes across multiple environments, use Helm. Structure your charts per service, use values files for environment differences, and lint everything in CI. Pair it with ArgoCD for GitOps deployment, and you'll have a deployment system that's audited, reproducible, and boring -- exactly what production infrastructure should be.