Fix ArgoCD Application OutOfSync/Degraded

This guide explains how to diagnose and resolve ArgoCD Applications stuck in “OutOfSync” or “Degraded” health status.

Problem

Symptom: ArgoCD Application shows “OutOfSync” or “Degraded” health status in UI or CLI.

Check application status:

kubectl get applications -n argocd

# Output shows:
NAME          SYNC STATUS   HEALTH STATUS
monitoring    OutOfSync     Degraded

In ArgoCD UI:

  • Application tile shows red/yellow status

  • Sync status shows “OutOfSync”

  • Health status shows “Degraded” or “Missing”

Common Causes

Cause 1: Storage Size Cannot Shrink

Symptom: Application with PersistentVolumeClaim or CNPG Cluster shows OutOfSync after reducing storage size.

Logs show:

kubectl describe application <app-name> -n argocd

# Events show:
error validating data: spec.storage.size: Invalid value: "10Gi": field is immutable

Root Cause: Kubernetes forbids shrinking PVC storage (data loss risk).

Solution:

Option 1: Increase storage back (recommended):

# Restore original size or increase
spec:
  storage:
    size: 20Gi  # Increase from original 15Gi (cannot decrease to 10Gi)

Option 2: Delete and recreate (data loss!):

# DANGER: This deletes all data!

# 1. Backup data first
kubectl exec -it <pod> -n <namespace> -- pg_dump ... > backup.sql

# 2. Delete cluster/PVC
kubectl delete cluster <cluster-name> -n <namespace>
kubectl delete pvc <pvc-name> -n <namespace>

# 3. Recreate with new size
kubectl apply -f cluster.yaml

Cause 2: Missing Repository Credentials

Symptom: Application references private git repository without credentials configured.

Logs show:

kubectl describe application <app-name> -n argocd

# Status shows:
ComparisonError: failed to fetch repository: authentication required

Root Cause: ArgoCD cannot access private repository (missing deploy token/credentials).

Solution:

Add repository credentials:

# Create secret with repository credentials
kubectl create secret generic myapp-repo-creds \
  -n argocd \
  --from-literal=type=git \
  --from-literal=url=https://git.example.com/org/repo.git \
  --from-literal=username=deploy-token-username \
  --from-literal=password=deploy-token \
  --dry-run=client -o yaml | kubectl apply -f -

# Label secret for ArgoCD discovery
kubectl label secret myapp-repo-creds \
  -n argocd \
  argocd.argoproj.io/secret-type=repository \
  --overwrite

Verify:

# Check repository connection in ArgoCD UI:
# Settings → Repositories → Should show "Successful" connection

Refresh application:

argocd app refresh <app-name>

Cause 3: CRD Not Installed

Symptom: Application uses Custom Resource (e.g., Cluster, Pooler) but CRD not installed.

Logs show:

kubectl describe application <app-name> -n argocd

# Status shows:
unable to recognize "": no matches for kind "Cluster" in version "postgresql.cnpg.io/v1"

Root Cause: Operator not installed or CRD deleted.

Solution:

Install operator (provides CRDs):

# Example: Install CNPG operator
argocd app sync cnpg-app-operator

# Or for other operators, apply CRD manually:
kubectl apply -f https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/main/config/crd/bases/postgresql.cnpg.io_clusters.yaml

Verify CRD installed:

kubectl get crd | grep <crd-name>

# Example:
kubectl get crd | grep cnpg

Sync application again:

argocd app sync <app-name>

Cause 4: Resource Conflict

Symptom: Resource exists in cluster but not in git (manual kubectl apply).

Logs show:

kubectl describe application <app-name> -n argocd

# Status shows:
resource <resource> is not managed by ArgoCD

Root Cause: Resource created manually (not via ArgoCD).

Solution:

Option 1: Add resource to git (recommended):

# Add resource manifest to git repository
kubectl get <resource> -n <namespace> -o yaml > resource.yaml
# Edit, commit to git, push

Option 2: Delete manual resource:

# Delete resource (ArgoCD will recreate from git)
kubectl delete <resource> <name> -n <namespace>

# Sync application
argocd app sync <app-name>

Option 3: Ignore resource (if intentionally manual):

# In ArgoCD Application spec
spec:
  ignoreDifferences:
  - group: apps
    kind: Deployment
    name: manual-deployment
    jsonPointers:
    - /spec/replicas

Cause 5: ExternalSecrets Show OutOfSync

Symptom: ExternalSecret resources permanently show OutOfSync even after successful sync.

Check which resources are OutOfSync:

kubectl get application <app-name> -n argocd \
  -o jsonpath='{range .status.resources[?(@.status=="OutOfSync")]}{.kind}/{.name}{"\n"}{end}'

# Output shows:
ExternalSecret/my-external-secret

Root Cause: Two issues cause this:

  1. ESO controller adds default fields not in your manifest:

    • spec.dataFrom[].extract.conversionStrategy: Default

    • spec.dataFrom[].extract.decodingStrategy: None

    • spec.dataFrom[].extract.metadataPolicy: None

    • spec.target.deletionPolicy: Retain

  2. Duration normalization: Kubernetes normalizes 1h to 1h0m0s

Solution: Add ignoreDifferences for ExternalSecrets:

# In ArgoCD Application spec
spec:
  ignoreDifferences:
    # ESO-managed secrets (ignore data changes)
    - group: ''
      kind: Secret
      name: my-secret
      jsonPointers:
        - /data
    # ExternalSecrets have defaults added by ESO controller
    - group: external-secrets.io
      kind: ExternalSecret
      jqPathExpressions:
        - '.spec.dataFrom[].extract.conversionStrategy'
        - '.spec.dataFrom[].extract.decodingStrategy'
        - '.spec.dataFrom[].extract.metadataPolicy'
        - '.spec.target.deletionPolicy'
        - '.spec.refreshInterval'

CDK8S Example (TypeScript):

ignoreDifferences: [
  // ExternalSecrets have defaults added by ESO controller
  {
    group: 'external-secrets.io',
    kind: 'ExternalSecret',
    jqPathExpressions: [
      '.spec.dataFrom[].extract.conversionStrategy',
      '.spec.dataFrom[].extract.decodingStrategy',
      '.spec.dataFrom[].extract.metadataPolicy',
      '.spec.target.deletionPolicy',
      '.spec.refreshInterval',
    ],
  },
],

Verify fix:

# Refresh application
kubectl annotate application <app-name> -n argocd \
  argocd.argoproj.io/refresh=hard --overwrite

# Check status
kubectl get application <app-name> -n argocd \
  -o jsonpath='Sync: {.status.sync.status}{"\n"}'

Cause 6: Namespace Not Created

Symptom: Application targets namespace that doesn’t exist.

Logs show:

kubectl describe application <app-name> -n argocd

# Status shows:
namespaces "<namespace>" not found

Root Cause: Namespace not created, auto-create disabled.

Solution:

Option 1: Enable auto-create:

# In ArgoCD Application spec
spec:
  syncPolicy:
    syncOptions:
    - CreateNamespace=true  # ArgoCD creates namespace automatically

Option 2: Create namespace manually:

kubectl create namespace <namespace>

# Then sync
argocd app sync <app-name>

Debugging Process

Step 1: Check Application Status

# List all applications
kubectl get applications -n argocd

# Get detailed status
kubectl describe application <app-name> -n argocd

# Check sync errors
kubectl get application <app-name> -n argocd -o jsonpath='{.status.conditions}'

Step 2: Check Resource Status

# Check individual resources
kubectl get application <app-name> -n argocd -o jsonpath='{.status.resources}' | jq

# Check specific resource health
kubectl get application <app-name> -n argocd -o jsonpath='{.status.resources[?(@.kind=="Deployment")].health}'

Step 3: Check Application Logs

Via CLI:

argocd app get <app-name>
argocd app diff <app-name>  # Show differences between git and cluster
argocd app logs <app-name>

Via UI:

  1. Navigate to ArgoCD UI: https://argocd.ops.kup6s.net

  2. Click on application

  3. View resource tree and error messages

Step 4: Manual Sync

# Refresh (re-check git)
argocd app refresh <app-name>

# Sync (apply changes)
argocd app sync <app-name>

# Sync with prune (delete resources not in git)
argocd app sync <app-name> --prune

# Force sync (override conflicts)
argocd app sync <app-name> --force

Prevention

Auto-Sync

Enable auto-sync to automatically deploy changes from git:

spec:
  syncPolicy:
    automated:
      prune: true      # Delete resources removed from git
      selfHeal: true   # Revert manual changes to match git
      allowEmpty: false  # Prevent syncing empty commits

When to use:

  • ✅ Development/staging environments

  • ✅ Infrastructure applications (monitoring, operators)

  • ⚠️ Production (consider manual approval for critical apps)

Sync Windows

Restrict sync times for production:

spec:
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
    automated:
      prune: true
      selfHeal: true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

Health Checks

Custom health checks for application-specific resources:

# In argocd-cm ConfigMap
data:
  resource.customizations.health.postgresql.cnpg.io_Cluster: |
    hs = {}
    if obj.status ~= nil and obj.status.phase ~= nil then
      if obj.status.phase == "Cluster in healthy state" then
        hs.status = "Healthy"
        hs.message = obj.status.phase
        return hs
      end
    end
    hs.status = "Progressing"
    hs.message = "Waiting for cluster"
    return hs

Troubleshooting Checklist

When application is OutOfSync:

  • [ ] Check git repository accessible (credentials configured)

  • [ ] Check target namespace exists

  • [ ] Check CRDs installed (operators deployed)

  • [ ] Check resource conflicts (manual vs ArgoCD-managed)

  • [ ] Check immutable fields (storage size, etc.)

  • [ ] Check ExternalSecrets (ESO adds defaults causing diff)

  • [ ] Review application logs and events

  • [ ] Try manual sync with argocd app sync

When application is Degraded:

  • [ ] Check pod status (kubectl get pods)

  • [ ] Check pod logs (kubectl logs)

  • [ ] Check resource health (kubectl describe)

  • [ ] Check dependencies (databases, services)

  • [ ] Review application-specific health checks

Further Reading