Reference

Secrets and ESO bridges

CrowdSec requires the bouncer API key to be readable in two namespaces simultaneously: crowdsec (for the engine) and traefik (for the plugin). Kubernetes does not permit cross-namespace secret mounts, so we replicate the source secret into both target namespaces using External Secrets Operator.

Source of truth: application-secrets namespace

The original secret is created manually once during cluster setup.

KEY=$(openssl rand -hex 32)
kubectl -n application-secrets create secret generic crowdsec-bouncer-key \
  --from-literal=key="${KEY}"

The key value must be preserved in the password manager. ESO uses this secret as the source for both bridges.

Bridge 1: into the crowdsec namespace

Construct: dp-infra/crowdsec/charts/constructs/secret-store.ts

The construct creates:

  • ClusterSecretStore crowdsec-app-secrets-store, restricted to the crowdsec namespace

  • ExternalSecret crowdsec-bouncer-key in the crowdsec namespace

Purpose: the engine’s docker-start-custom.sh reads the BOUNCER_KEY_traefik_bouncer environment variable on startup and auto-registers the bouncer in LAPI via cscli bouncers add.

Bridge 2: into the traefik namespace

Construct: dp-infra/crowdsec/charts/constructs/traefik-secret-bridge.ts

The construct creates:

  • ClusterSecretStore crowdsec-bouncer-traefik-store, restricted to the traefik namespace

  • ExternalSecret crowdsec-bouncer-key in the traefik namespace

Purpose: Traefik pods mount the secret as a volume at /etc/traefik/crowdsec/.

Volume mount in Traefik

The Traefik helm values in kube-hetzner/clusters/kup6s/traefik_overrides.yaml reference the secret:

volumes:
  - name: crowdsec-bouncer-key
    mountPath: /etc/traefik/crowdsec
    type: secret

The resulting in-pod file is /etc/traefik/crowdsec/key, matching the plugin’s CrowdsecLapiKeyFile setting.

Key rotation

Rotate the bouncer API key in four steps.

  1. Generate a new key and store it in the password manager:

    NEW_KEY=$(openssl rand -hex 32)

  2. Update the source secret in application-secrets:

    kubectl -n application-secrets create secret generic crowdsec-bouncer-key \
  --from-literal=key="${NEW_KEY}" --dry-run=client -o yaml | kubectl apply -f -

  3. Force ESO sync in both target namespaces:

    kubectl annotate externalsecret -n crowdsec crowdsec-bouncer-key force-sync=$(date +%s) --overwrite
kubectl annotate externalsecret -n traefik crowdsec-bouncer-key force-sync=$(date +%s) --overwrite

  4. Delete the existing LAPI registration and restart the engine to re-register with the new key:

    kubectl exec -n crowdsec deploy/crowdsec-lapi -- cscli bouncers delete traefik_bouncer
kubectl rollout restart deploy/crowdsec-lapi -n crowdsec

Total wait time: about 2 minutes when ESO sync is forced.