How-to · Troubleshooting

Handle a false-positive block

This guide shows you how to investigate and resolve false-positive blocks reported by users or detected via monitoring. For your own lockout, see self-rescue-lockout.

When this happens

A legitimate user, customer, or webhook source receives HTTP 403 from a kup6s site. Triggers:

  • User complaint via email or chat

  • Grafana shows a sudden 4xx spike

  • Loki logs show repeated 403 responses with crowdsec tags

Step 1: Confirm the block

Determine the affected source IP. For a user complaint, ask the user — curl ifconfig.me is the simplest method. For a monitoring spike, query Loki for the top 403-returning IPs in the time window.

Check whether the IP is in the decision list:

kubectl exec -n crowdsec deploy/crowdsec-lapi -- cscli decisions list --ip <user-ip>

If the command returns an entry, note the origin and scenario fields:

  • origin=CAPI — generic community feed

  • origin=lists, scenario=crowdsecurity/<name> — premium blocklist

  • origin=cscli — manually added

Step 2: Decide false-positive or real

Use these criteria.

If the IP belongs to a known customer or partner — false-positive. If the IP belongs to an internal monitoring or webhook source — false-positive. If the source is from a too-aggressive blocklist for our use case — false-positive for our context. If the IP shows recognizable botnet patterns (multiple scenarios, repeated scan attempts) — not a false-positive.

Inspect alert details for context:

kubectl exec -n crowdsec deploy/crowdsec-lapi -- cscli alerts list --ip <ip>

Step 3: Ad-hoc unblock

Remove the decision immediately:

kubectl exec -n crowdsec deploy/crowdsec-lapi -- cscli decisions delete --ip <user-ip>

The unblock takes effect within 60 seconds.

This is temporary — the decision can return when the source refreshes.

Step 4: No permanent allowlist on v1.6.4

CrowdSec v1.6.4 has no cscli allowlists subcommand. That feature ships with CrowdSec v2 and is not available on the kup6s engine.

For known customer IPs and CIDRs, use one of these workarounds:

  • Re-run cscli decisions delete --ip <ip> (or --range <cidr>) when the block returns; pair with monitoring so recurring blocks surface quickly.

  • Identify the offending blocklist (see step 5) and disable it in the CrowdSec web console.

  • Add the customer IP/CIDR to the Traefik ClientTrustedIPs plugin parameter — the plugin then skips enforcement for those clients regardless of CrowdSec decisions. Edit middlewares.ts in dp-infra/ and merge an MR.

  • Plan a CrowdSec v2 upgrade for proper allowlist support — out of scope for Phase 2.

The Traefik-side trust list is the most durable v1.6 option for stable customer ranges.

Step 5: Disable the noisy source if false-positives recur

If a single blocklist produces multiple false-positives, disable it via the CrowdSec web console at https://app.crowdsec.net:

  1. Log in to app.crowdsec.net.

  2. Select your engine (kup6s-lapi) in the sidebar.

  3. Open the blocklists tab.

  4. Toggle the offending blocklist off.

The local engine pulls the new configuration on the next sync (within a few minutes), and the plugin stops applying decisions from this blocklist.

Existing decisions remain in the cache until their TTL expires. To force-clear them via cscli, use --origin lists together with --scenario:

kubectl exec -n crowdsec deploy/crowdsec-lapi -- cscli decisions delete \
  --origin lists --scenario crowdsecurity/<name>

Step 6: When CAPI itself produces false-positives

The CAPI feed cannot be disabled — it is the default. Address recurring CAPI false-positives by either:

  • Adding the legitimate IP or CIDR to the Traefik ClientTrustedIPs plugin parameter (see step 4)

  • Per-endpoint allowlists in middlewares.ts (future work, not yet implemented in kup6s)

  • Temporarily reverting to detect-only via self-rescue-lockout Action 3 Option A while investigating

Audit trail

CrowdSec does not maintain an automatic audit log of manual decision deletes, console blocklist toggles, or Traefik trust-list changes. Document each manual action in a team channel (Slack, email) with the reason and date. For stricter compliance, snapshot cscli decisions list -o json into Loki on a schedule.