Sysdig

---
name: sysdig
description: "Expert agent for Sysdig Secure CNAPP platform. Covers Falco-based runtime security, image scanning with in-use context, Kubernetes posture (KSPM), CDR, identity/network security, GRC, risk prioritization using runtime context, and Sysdig Monitor observability. WHEN: \"Sysdig\", \"Sysdig Secure\", \"Sysdig runtime\", \"Sysdig image scanning\", \"Sysdig KSPM\", \"Sysdig CDR\", \"Sysdig in use\", \"Sysdig Falco\", \"Sysdig posture\", \"Sysdig Monitor\", \"Sysdig agent\"."
license: MIT
---

# Sysdig Secure

This skill covers Sysdig Secure — a cloud-native CNAPP built on the foundation of Falco (CNCF's runtime security standard) and extended with full CNAPP capabilities. Sysdig's key differentiator is using runtime intelligence to prioritize findings: identifying which vulnerabilities are actually "in use" at runtime reduces noise dramatically.

## How to Approach Tasks

When you receive a request:

1. **Classify** the request type:
   - **Runtime Security** -- Falco-based rules, runtime threat detection, CDR (Cloud Detection and Response)
   - **Image Scanning** -- Vulnerability scanning, registry scanning, CI/CD integration
   - **Risk Prioritization** -- "In use" vulnerability filtering, runtime context for prioritization
   - **KSPM/Posture** -- Kubernetes Security Posture Management, compliance benchmarks
   - **Identity and Permissions** -- Kubernetes and cloud identity risk
   - **Network Security** -- Kubernetes network security posture
   - **GRC** -- Governance, Risk, and Compliance reporting
   - **Observability** -- Sysdig Monitor, infrastructure monitoring (separate from Secure)
   - **Sysdig Agent** -- Agent deployment, configuration, troubleshooting

2. **Identify environment** -- Kubernetes platform (EKS, AKS, GKE)? Multi-cloud? Sysdig SaaS or on-prem? Scale (number of nodes)? Existing Falco deployment?

3. **Analyze** -- Apply Sysdig-specific reasoning. Sysdig's runtime context is the key differentiator — connect vulnerability findings with "in use" runtime data before recommending remediation priorities.

4. **Recommend** -- Provide specific Sysdig platform guidance with configuration and workflow examples.

## Core Capabilities

### Runtime Security (Falco-Based)

Sysdig Secure's runtime security is built on Falco, the CNCF graduated project for kernel-level threat detection:

**What Sysdig captures:**
- All system calls from every container on the node (via eBPF or kernel module)
- Kubernetes API server audit log events
- Cloud provider audit logs (CloudTrail, Azure Activity Logs, GCP Audit Logs)
- Container network traffic (via network tracing)

**Sysdig-curated Falco rules:**
Sysdig ships 300+ production-ready Falco rules organized by threat category:
- Shell spawned in container / unexpected process execution
- Sensitive file access (SSH keys, /etc/shadow, cloud credential files)
- Container escape techniques (namespace manipulation, device access)
- Cryptocurrency mining (process patterns + mining pool network)
- Lateral movement (K8s API calls from pods, unusual inter-service traffic)
- Privilege escalation (setuid, dangerous capability usage)
- Data exfiltration (large outbound data transfers, DNS tunneling)
- Cloud identity abuse (unusual CloudTrail API calls from workloads)

**Kubernetes audit log rules:**
Sysdig analyzes Kubernetes audit logs for control-plane threats:
- `kubectl exec` or `kubectl cp` in production (lateral movement indicator)
- Service account token creation by unexpected identity
- `cluster-admin` binding creation
- New admission webhook registration (could disable security controls)
- ConfigMap creation with sensitive data
- Secrets access from unexpected identities

### The "In Use" Differentiator

**The core problem with traditional vulnerability management:**
A typical enterprise Kubernetes cluster has thousands of containers, each with dozens of vulnerabilities. A CVSS-based list generates hundreds of "Critical" vulnerabilities — far too many to remediate quickly.

**Sysdig's "in use" approach:**
Sysdig correlates vulnerability data with runtime observations to determine which vulnerable packages are actually loaded and called during container execution:

```
Image has 847 total vulnerabilities
  ↓
Static analysis (CVSS, EPSS, fix availability):
  → 47 Critical, 124 High, 201 Medium, 475 Low
  ↓
Runtime "in use" filter:
  → Only 12 Critical packages are loaded in running containers
  → Only 3 Critical packages are in functions actually called at runtime
  ↓
Prioritized remediation list: 3 Critical (actually in-use, actually called)
```

**How "in use" detection works:**
1. Sysdig agent monitors process loading (which shared libraries are loaded via `dlopen`, `execve`)
2. For language runtimes (JVM, Python, Node.js), Sysdig observes which classes/modules are imported
3. This data is correlated with vulnerability scanner findings
4. Findings are tagged: `in_use = true/false`

**Practical impact:**
- Reduces remediation workload by 85-90% in typical environments
- Teams focus on vulnerabilities that can actually be exploited in the running application
- Better SLA adherence — fix the 10 things that matter, not the 1000 things that exist

### Image Scanning

**Scanner:** Sysdig uses a hybrid approach — underlying vulnerability scanning powered by open-source databases plus Sysdig's own intelligence.

**Scanning targets:**
- Container registries (ECR, ACR, GCR, Docker Hub, JFrog, Nexus, Harbor)
- Running containers (via agent data about loaded packages)
- CI/CD pipeline images (via Sysdig CLI or CI integrations)

**Scan lifecycle:**
```
1. Image pushed to registry
2. Sysdig registry scanner triggered (webhook or polling)
3. Image layers extracted; packages enumerated
4. CVE matching against Sysdig vulnerability database
5. Results stored in Sysdig platform with image metadata
6. When image runs in cluster, agent provides "in use" context overlay
7. Prioritized vulnerability list updated with runtime data
```

**Sysdig CLI for CI/CD:**
```bash
# Install Sysdig CLI
curl -LO https://download.sysdig.com/stable/sysdig-cli-scanner/latest_version.txt
curl -LO "https://download.sysdig.com/stable/sysdig-cli-scanner/$(cat latest_version.txt)/linux/amd64/sysdig-cli-scanner"
chmod +x sysdig-cli-scanner

# Scan image
SECURE_API_TOKEN=<token> ./sysdig-cli-scanner \
  --apiurl https://us2.app.sysdig.com \
  myapp:latest

# With policy enforcement
SECURE_API_TOKEN=<token> ./sysdig-cli-scanner \
  --apiurl https://us2.app.sysdig.com \
  --policy "Default Vulnerability Management Policy" \
  myapp:latest
# Exit code: 0 = pass, 1 = fail (for CI/CD gate)
```

**Image scanning policies:**
Define criteria for pass/fail in CI/CD:
- Block if CRITICAL CVE with no fix available
- Block if secrets found in image layers
- Block if malware detected
- Block if image runs as root
- Block if base image is end-of-life

### Kubernetes Security Posture Management (KSPM)

**What KSPM covers:**
- CIS Kubernetes Benchmark (all cluster types)
- CIS EKS, AKS, GKE Benchmarks
- NSA Kubernetes Hardening Guidance
- NIST 800-190 Container Security
- Kubernetes RBAC analysis (overprivileged roles, service accounts)
- Network policy coverage assessment
- Pod security configuration
- Workload misconfiguration (privileged containers, host namespace access, capabilities)

**Compliance posture view:**
Sysdig KSPM shows compliance scores per cluster per framework with drill-down to specific failing controls and remediation guidance.

**Workload security findings:**
Sysdig scans running workloads for misconfigurations:
```
Critical Findings:
  - Pod "payment-api" running as root in namespace "production"
  - Pod "data-processor" has hostPID=true
  - Service account "app-sa" has cluster-admin ClusterRoleBinding
  - Deployment "api-gateway" has container with CAP_SYS_ADMIN

Network Exposure Findings:
  - Pod "internal-db" exposed to internet via LoadBalancer service
  - No NetworkPolicy applied to namespace "staging"
```

### Kubernetes Identity and Network Security

**Identity risk:**
- Enumerates all Kubernetes service accounts and their permissions
- Identifies: service accounts with cluster-admin, unused service accounts, service accounts with cross-namespace access
- Cloud identity mapping: correlates Kubernetes service accounts with cloud IAM roles (IRSA, Workload Identity)
- Attack paths: visualizes how a compromised pod could escalate to cloud permissions

**Network security:**
- Shows current network policy coverage per namespace
- Identifies pods with no network policy (fully open east-west traffic)
- Recommends network policies based on observed traffic (Sysdig observes actual traffic and can generate appropriate network policies)
- Visualizes inter-service communication (similar to service mesh observability but without a mesh)

**Sysdig network policy suggestions:**
```bash
# Sysdig can observe traffic and suggest network policies
# In Sysdig console: Network > Topology > Select namespace > Generate Policy

# The generated policy reflects actual observed communication
# Review and apply to Kubernetes
kubectl apply -f suggested-network-policy.yaml
```

### Cloud Detection and Response (CDR)

Sysdig CDR combines workload runtime data with cloud audit log data for unified threat detection:

**Detection data sources:**
- Kernel syscall events (from Sysdig agent)
- Kubernetes audit logs
- AWS CloudTrail events
- Azure Activity Logs
- GCP Audit Logs
- Container network flows

**CDR threat scenarios:**
- **Compromised container → cloud pivot:** Container shows unusual behavior + CloudTrail shows unusual API calls from the same instance (credential theft + cloud API abuse)
- **Unusual K8s API activity:** `kubectl exec` into production pod + subsequent shell activity → immediate correlation
- **IAM abuse from workload:** Service account makes unusual AWS API calls (resource enumeration, CreateUser, etc.)
- **Data exfiltration:** Large outbound network transfer from container + S3 bucket exfiltration in CloudTrail

**Sysdig Insights:**
Automated threat detection correlation engine that:
- Correlates events across workload, K8s API, and cloud API
- Assigns confidence scores to threat detections
- Groups related events into incidents
- Maps detections to MITRE ATT&CK for Cloud framework

### Sysdig Agent Deployment

**DaemonSet deployment (Kubernetes):**
```bash
# Install via Helm (recommended)
helm repo add sysdig https://charts.sysdig.com
helm repo update

helm install sysdig-agent sysdig/sysdig-agent \
  --namespace sysdig-agent \
  --create-namespace \
  --set global.sysdig.accessKey=<your-access-key> \
  --set global.sysdig.region=us2 \
  --set agent.sysdig.settings.tags="env:production,cluster:my-eks-cluster" \
  --set nodeAnalyzer.enabled=true \    # enables vulnerability scanning
  --set rapidResponse.enabled=false    # optional: live response capability
```

**Agent resource requirements:**
```yaml
resources:
  requests:
    cpu: 250m
    memory: 512Mi
  limits:
    cpu: 2000m
    memory: 1536Mi
```

**Agent kernel monitoring modes:**
1. **eBPF (recommended):** Low overhead; requires kernel 4.14+ with BTF support
2. **Legacy eBPF:** Supports older kernels (4.x) with eBPF but without BTF
3. **Kernel module:** Compiled kernel module; fallback for kernels without eBPF
4. **Universal eBPF:** Newer approach; CO-RE (Compile Once, Run Everywhere); no kernel headers needed

```yaml
# Select kernel monitoring mode in Helm values
agent:
  sysdig:
    settings:
      feature:
        mode: "secure_light"    # Secure features only (smaller footprint)
  slim:
    enabled: false
  ebpf:
    enabled: true               # Use eBPF (recommended)
    kind: universal_ebpf        # Use CO-RE eBPF
```

### Admission Control Integration

Sysdig integrates with Kubernetes admission control:

**Via OPA Gatekeeper:**
Sysdig can push findings to Gatekeeper constraints:
- Image scanning results as admission decisions
- KSPM findings as validating webhook policies

**Via Kyverno:**
Similar integration for policy enforcement based on Sysdig scan results.

**Sysdig Admission Controller (native):**
```yaml
# Sysdig native admission controller
apiVersion: v1
kind: ConfigMap
metadata:
  name: sysdig-admission-controller-values
data:
  features: |
    policyEnforcement: true      # Enforce image scanning policies
    kspmAdmission: true          # Block workloads with critical KSPM violations
```

## Governance, Risk, and Compliance (GRC)

**Compliance frameworks:**
- CIS Kubernetes Benchmark (1.6, 1.7, 1.8)
- CIS EKS / AKS / GKE Benchmarks
- CIS AWS / Azure / GCP Foundations
- NIST 800-53
- PCI DSS
- HIPAA
- SOC 2
- ISO 27001
- FedRAMP
- MITRE ATT&CK

**GRC workflow:**
1. Enable relevant compliance frameworks in Sysdig Secure
2. Sysdig continuously assesses your environment
3. Compliance dashboard shows score per framework per cluster/cloud account
4. Drill down to specific failing controls with remediation guidance
5. Export compliance evidence (PDF, CSV) for audit purposes
6. Track remediation progress with SLA tracking

**Policy as code:**
Sysdig supports exporting compliance policies as Rego (OPA) or Kyverno policies for GitOps-managed enforcement.

## Sysdig Monitor (Observability)

While Sysdig Secure is the security product, Sysdig Monitor is the companion observability product (sometimes sold together):

**Sysdig Monitor capabilities:**
- Kubernetes infrastructure monitoring (cluster, node, namespace, pod, container metrics)
- Prometheus compatible (Sysdig Monitor can replace or complement Prometheus)
- PromQL queries
- Dashboards, alerts, and SLO monitoring
- Application performance monitoring (APM) via instrumentation
- Cloud provider metrics integration (CloudWatch, Azure Monitor, GCP Monitoring)

**Security + Observability convergence:**
A key Sysdig value proposition is combining security and observability data:
- When a runtime security alert fires, correlate immediately with infrastructure metrics (was there unusual CPU? unusual network?)
- Troubleshoot security incidents with the same tool used for operational incidents
- Shared dashboards for DevSecOps teams

## Integration Patterns

### SIEM/SOAR Integration

```yaml
# Sysdig notification channels
Channels:
  - Slack: webhook URL per severity level
  - PagerDuty: integration key for on-call routing
  - Splunk: HEC endpoint + index configuration
  - Microsoft Sentinel: Event Hub integration
  - QRadar: syslog output
  - Generic Webhook: JSON payload to any SOAR

# Alert routing rules
Rules:
  - severity: CRITICAL → PagerDuty + Slack + SOAR
  - type: "container_escape" → immediate PagerDuty
  - type: "crypto_mining" → Slack + JIRA
  - source: "CloudTrail" + type: "IAM_abuse" → SOAR investigation playbook
```

### Ticketing Integration

```bash
# Jira integration for vulnerability findings
# Configure in Sysdig Console: Integrations > Issue Tracking
# - Map severity to Jira priority
# - Assign to teams by namespace label or image registry
# - Two-way sync: fixing in Jira updates Sysdig status
```

## Reference Files

Load these when you need deep architectural knowledge:

- `references/architecture.md` -- Sysdig platform architecture: Sysdig agent deep-dive (kernel module vs. eBPF), Node Analyzer for vulnerability scanning, Sysdig backend pipeline for syscall event processing, Kubernetes audit log integration, CDR data correlation architecture, Rapid Response (live terminal) capability.