How-To Guide

Work as a Capsule Tenant¶

Type: How-To (Task-oriented)

Goal: Set up your environment as a Capsule tenant owner to create and manage namespaces, deploy applications, and use External Secrets Operator for secret management.

Audience: Tenant owners (e.g., Phil with the bd tenant)

Prerequisites¶

Kubeconfig file from cluster administrator
kubectl installed
Basic understanding of Kubernetes namespaces and deployments

Step 1: Configure kubectl Access¶

Save the kubeconfig provided by the cluster administrator:

# Option A: Separate kubeconfig file
export KUBECONFIG=~/.kube/kup6s-tenant.yaml

# Option B: Merge into existing kubeconfig
KUBECONFIG=~/.kube/config:~/Downloads/tenant-kubeconfig.yaml kubectl config view --flatten > ~/.kube/config.new
mv ~/.kube/config.new ~/.kube/config

Verify access:

kubectl auth can-i create namespaces
# Expected: yes

kubectl get tenants
# Expected: Error (tenants are cluster-scoped, you can't list them)

kubectl auth can-i get namespaces
# Expected: yes

Step 2: Create Your First Namespace¶

As a tenant owner, you can create namespaces that automatically belong to your tenant:

# Replace 'bd' with your tenant prefix
kubectl create namespace bd-myapp

Verify the namespace belongs to your tenant:

kubectl get namespace bd-myapp -o jsonpath='{.metadata.labels}'
# Should include: capsule.clastix.io/tenant: bd

List all your namespaces:

kubectl get namespaces -l capsule.clastix.io/tenant=bd

Step 3: Deploy Applications¶

Deploy applications to your namespaces using standard Kubernetes resources:

# Example: Deploy nginx
kubectl create deployment nginx --image=nginx -n bd-myapp
kubectl expose deployment nginx --port=80 -n bd-myapp

Using Helm¶

helm install my-release bitnami/postgresql -n bd-myapp

Using Kustomize¶

kubectl apply -k ./my-app -n bd-myapp

Step 4: Set Up Secret Management with ESO¶

For applications with secrets, set up a dedicated secrets namespace and use External Secrets Operator.

4.1 Create Secrets Namespace¶

kubectl create namespace bd-secrets

4.2 Create Source Secrets¶

Store your application secrets in the secrets namespace:

# Database credentials
kubectl create secret generic myapp-db \
  --from-literal=username=myapp \
  --from-literal=password=$(openssl rand -base64 32) \
  -n bd-secrets

# API keys
kubectl create secret generic myapp-api \
  --from-literal=api-key=$(openssl rand -hex 32) \
  -n bd-secrets

4.3 Request ClusterSecretStore from Administrator¶

Ask the cluster administrator to create a ClusterSecretStore for your tenant. Provide:

Your tenant name (e.g., bd)
Your secrets namespace (e.g., bd-secrets)

The administrator will create:

# Created by administrator
apiVersion: external-secrets.io/v1
kind: ClusterSecretStore
metadata:
  name: bd-secrets-store
spec:
  conditions:
    - namespaceRegex: "^bd-.*"
  provider:
    kubernetes:
      remoteNamespace: bd-secrets
      auth:
        serviceAccount:
          name: eso-bd-reader
          namespace: bd-secrets

Note

The ClusterSecretStore is cluster-scoped and requires administrator privileges. Once created, you can use it from any of your bd-* namespaces.

4.4 Create ExternalSecrets in Application Namespaces¶

Once the ClusterSecretStore exists, create ExternalSecrets in your application namespaces:

kubectl apply -f - <<EOF
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: bd-myapp
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: bd-secrets-store
    kind: ClusterSecretStore
  target:
    name: db-credentials
    creationPolicy: Owner
  dataFrom:
    - extract:
        key: myapp-db
EOF

Verify synchronization:

kubectl get externalsecret -n bd-myapp
# Expected: STATUS=SecretSynced, READY=True

kubectl get secret db-credentials -n bd-myapp
# Secret should exist with synced data

4.5 Use Secrets in Deployments¶

Reference the synced secrets in your deployments:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: bd-myapp
spec:
  template:
    spec:
      containers:
      - name: myapp
        image: myapp:latest
        env:
        - name: DB_USERNAME
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: username
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: password

Step 5: Monitor Resource Usage¶

Check your tenant’s resource consumption:

# List all pods across your namespaces
kubectl get pods -l capsule.clastix.io/tenant=bd -A

# Check resource usage per namespace
kubectl top pods -n bd-myapp

Warning

Your tenant has resource quotas. If deployments fail with quota errors, check your total resource usage across all namespaces.

Common Tasks¶

Create Ingress for External Access¶

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  namespace: bd-myapp
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  ingressClassName: traefik
  tls:
  - hosts:
    - myapp.sites.kup6s.com
    secretName: myapp-tls
  rules:
  - host: myapp.sites.kup6s.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: myapp
            port:
              number: 80

Note

DNS records for *.sites.kup6s.com must be configured by the cluster administrator.

Use Persistent Storage¶

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: myapp-data
  namespace: bd-myapp
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: longhorn  # or longhorn-backup for backups
  resources:
    requests:
      storage: 10Gi

Rotate Secrets¶

Update the source secret in your secrets namespace:

kubectl patch secret myapp-db -n bd-secrets \
  --type='json' \
  -p='[{"op": "replace", "path": "/data/password", "value": "'$(echo -n 'new-password' | base64)'"}]'

Wait for ESO to sync (up to refreshInterval) or force sync:

kubectl delete secret db-credentials -n bd-myapp
# ESO will recreate it with new values

Restart your application:

kubectl rollout restart deployment/myapp -n bd-myapp

Delete a Namespace¶

kubectl delete namespace bd-myapp

Warning

This permanently deletes all resources in the namespace including PersistentVolumeClaims and their data.

Troubleshooting¶

Cannot create namespace¶

Error: namespaces is forbidden: User "system:serviceaccount:capsule-system:bd-sa" cannot create resource "namespaces"

Cause: Your ServiceAccount token may have expired or RBAC is misconfigured. Solution: Contact the cluster administrator to verify your tenant configuration.

Namespace not associated with tenant¶

kubectl get namespace bd-newapp -o jsonpath='{.metadata.labels}'
# Missing capsule.clastix.io/tenant label

Cause: Namespace was created before Capsule was installed or by a different user. Solution: Ask administrator to label the namespace or delete and recreate it.

ExternalSecret shows SecretSyncedError¶

kubectl describe externalsecret db-credentials -n bd-myapp

Common causes:

ClusterSecretStore doesn’t exist yet (ask administrator)
Source secret doesn’t exist in bd-secrets
Namespace not matching the namespaceRegex pattern

Resource quota exceeded¶

Error: exceeded quota: tenant-bd, requested: limits.cpu=4, used: limits.cpu=30, limited: limits.cpu=32

Solution: Reduce resource requests in other namespaces or ask administrator to increase quota.

What You Cannot Do¶

As a tenant owner, these operations are restricted:

Action	Reason
Create ClusterRoles	Cluster-scoped resource
Create ClusterSecretStores	Cluster-scoped resource
Access other tenants’ namespaces	Tenant isolation
Modify tenant configuration	Administrator only
Create namespaces without tenant prefix	Automatic association only

Next Steps¶

Capsule Multi-Tenancy - Understand the architecture
Bootstrap Application Secrets - Detailed ESO setup guide
Choose Storage Class - Storage options for your applications