Container Security: Hardening Docker and Kubernetes in 4 Stages

🎯 Benefits in Numbers

⏱️ Reading time: 18 min | 💡 Level: Intermediate to Expert

📋 Why This Guide?

Challenge: Container breaches affect 89% of organizations running containerized workloads in production, with an average remediation cost of $4.2M. Most security incidents stem from unpatched images, insecure configurations, and inadequate runtime monitoring—yet most teams lack a systematic hardening methodology.

Measured Impact

🗓️ 4-Stage Container Security Framework

📝 Stage 1: Assessment & Audit

🎯 Measurable Objectives

⚠️ Pitfalls vs Solutions

✅ Vulnerability Baseline

💡 Calyo Tip: Use multi-stage Dockerfile builds to reduce image size by 70-90% and eliminate build dependencies—fewer packages mean fewer vulnerabilities to patch.

📝 Stage 2: Hardening & Configuration

🎯 Security Hardening Objectives

🛠️ Docker Image Hardening Best Practices

📊 Hardening Impact by Category

Essential Dockerfile Hardening Template

# Multi-stage build for minimal final image
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

# Final minimal image
FROM node:20-alpine
RUN addgroup -g 1001 -S appgroup && \
    adduser -S appuser -u 1001 -G appgroup
WORKDIR /app

# Copy from builder
COPY --from=builder --chown=appuser:appgroup /app/node_modules ./node_modules
COPY --chown=appuser:appgroup . .

# Security configurations
USER appuser
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD node healthcheck.js

# Non-root user
CMD ["node", "server.js"]

💡 Calyo Tip: Distroless images reduce attack surface by 95%—a 200MB traditional image becomes 20MB, containing only your application and runtime, zero package manager or shell.

📝 Stage 3: Kubernetes Security Configuration

🎯 Kubernetes Hardening Objectives

🔐 Pod Security & Configuration

Pod Security Standards Configuration

apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-sa
  namespace: production
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: app-role
  namespace: production
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list"]
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["app-secret"]
  verbs: ["get"]
---
apiVersion: v1
kind: Pod
metadata:
  name: secure-app
  namespace: production
spec:
  serviceAccountName: app-sa
  securityContext:
    runAsNonRoot: true
    runAsUser: 1001
    fsGroup: 1001
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    image: myapp:1.2.3
    imagePullPolicy: Always
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      runAsNonRoot: true
      capabilities:
        drop:
        - ALL
    resources:
      requests:
        memory: "256Mi"
        cpu: "250m"
      limits:
        memory: "512Mi"
        cpu: "500m"
    volumeMounts:
    - name: tmp
      mountPath: /tmp
    - name: cache
      mountPath: /app/cache
  volumes:
  - name: tmp
    emptyDir: {}
  - name: cache
    emptyDir: {}

Network Policy for Segmentation

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: production
spec:
  podSelector: {}
  policyTypes:
  - Ingress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-api-ingress
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: ingress
    ports:
    - protocol: TCP
      port: 8080
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080

📊 Security Implementation Approaches

📝 Stage 4: Runtime Protection & Monitoring

🎯 Runtime Security Objectives

🛠️ Runtime Security Monitoring Stack

Falco Runtime Monitoring Configuration

# Falco installation in Kubernetes
apiVersion: v1
kind: Namespace
metadata:
  name: falco
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: falco-config
  namespace: falco
data:
  falco.yaml: |
    rules_file:
      - /etc/falco/rules.yaml
      - /etc/falco/rules.d

    engine:
      kind: modern_ebpf

    outputs:
      - file_output:
          enabled: false
          keep_alive: false
      - stdout:
          enabled: true
      - syslog:
          enabled: true
          facility: LOG_USER

    priority: warning

    buffered_outputs: false

    syscall_event_timeouts:
      warn: 1000
      crit: 10000

    output_timeout: 2000

    metrics:
      enabled: true
      interval: 60
      output_rule: true
      resource_utilization_enabled: true

    base64_enc: true

  rules.yaml: |
    - rule: Unauthorized Container Escape Attempt
      desc: Detect attempts to escape container
      condition: >
        spawned_process and container
        and (proc.name = "nsenter" or proc.name = "unshare")
      output: >
        Potential container escape (user=%user.name
        command=%proc.cmdline container_id=%container.id)
      priority: CRITICAL
      tags: [container, escape]

    - rule: Privilege Escalation via Setuid
      desc: Detect setuid binary execution
      condition: >
        spawned_process and container and
        proc.cap.effective contains CAP_SETUID
      output: >
        Setuid execution in container (user=%user.name
        exe=%proc.name container_id=%container.id)
      priority: HIGH
      tags: [privilege, escalation]

    - rule: Suspicious Package Manager Execution
      desc: Package manager in production container
      condition: >
        spawned_process and container and
        (proc.name in (apt, yum, apk, pip, npm))
      output: >
        Package manager in container (user=%user.name
        command=%proc.cmdline container_id=%container.id)
      priority: MEDIUM
      tags: [container, suspicious]
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: falco
  namespace: falco
spec:
  selector:
    matchLabels:
      app: falco
  template:
    metadata:
      labels:
        app: falco
    spec:
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      containers:
      - name: falco
        image: falcosecurity/falco:latest
        securityContext:
          privileged: true
        volumeMounts:
        - name: docker
          mountPath: /var/run/docker.sock
        - name: containerd
          mountPath: /run/containerd
        - name: cgroup
          mountPath: /host/sys/fs/cgroup
        - name: boot
          mountPath: /host/boot
        - name: lib
          mountPath: /host/lib
        - name: usr
          mountPath: /host/usr
        - name: etc
          mountPath: /host/etc
        - name: falco-config
          mountPath: /etc/falco
      volumes:
      - name: docker
        hostPath:
          path: /var/run/docker.sock
      - name: containerd
        hostPath:
          path: /run/containerd
      - name: cgroup
        hostPath:
          path: /sys/fs/cgroup
      - name: boot
        hostPath:
          path: /boot
      - name: lib
        hostPath:
          path: /lib
      - name: usr
        hostPath:
          path: /usr
      - name: etc
        hostPath:
          path: /etc
      - name: falco-config
        configMap:
          name: falco-config

📊 Security Event Categories

💡 Calyo Tip: Enable eBPF-based detection instead of syscall-based monitoring—you’ll cut CPU overhead by 60% while maintaining threat detection capability.

📊 Container Security Maturity Model

📈 Success Measurement

Essential KPIs

• Mean Time to Detect (MTTD): Industry average 6.2 hours → Target: <5 minutes
• Vulnerability Remediation Rate: Track % of critical CVEs patched within 7 days (Target: 95%+)
• Image Compliance Score: CIS Benchmark compliance percentage (Target: >85%)
• Runtime Incident Response Time: Time from alert to remediation (Target: <15 minutes)
• Container Escape Attempts: Zero tolerance metric (Target: 0 successful escapes)

Monitoring Dashboard Elements

Real-time indicators to track:

• Vulnerability Dashboard: CVE database integration, image scan results, age of base images
• Network Traffic Alerts: Suspicious connections, lateral movement detection, egress anomalies
• Runtime Threat Detection: Syscall anomalies, privilege escalation attempts, container escape attempts
• Compliance Status: Pod security standard violations, RBAC policy changes, audit log completeness

💡 Expert Tips

Quick Wins (Week 1)

1. Enable container image scanning in CI/CD (2-4 hours) - Prevent 80% of vulnerabilities from reaching production
2. Implement non-root USER in Dockerfiles (1 day) - Eliminate root execution vector used in 67% of breaches
3. Enable Pod Security Standards enforcement (1 day) - Prevent 45+ known Kubernetes misconfigurations
4. Add resource limits to pods (2-4 hours) - Prevent resource exhaustion attacks and improve scheduling

Long-term Investments

• Deploy service mesh (Istio/Linkerd) - 8-12 weeks for complete implementation, provides encrypted mutual TLS between all services
• Implement zero-trust network policies - 6-8 weeks, requires network topology mapping before deployment
• Build automated compliance pipeline - Continuous policy enforcement, reduces manual audit overhead by 70%
• Establish container supply chain security - Image signing, SCA integration, SBOM generation for complete transparency

🔍 Common Vulnerabilities Deep Dive

1. Base Image Vulnerabilities (47% of incidents)

Problem: Ubuntu 20.04 base image contains 89 vulnerabilities on day 1. Calyo Solution:

• Switch to Alpine (14 base vulnerabilities) or Distroless (zero OS packages)
• Automate base image updates via renovate-bot
• Implement image freshness checks (>3 months old = fail build)

2. Secrets in Images (12% of breaches)

Problem: AWS keys found in 89% of public Docker images analyzed. Calyo Solution:

• Use Docker BuildKit with --secret flag for build-time secrets
• Scan builds with TruffleHog before pushing
• Implement HashiCorp Vault for runtime secrets
• Never commit credentials to source control

3. Privileged Containers (8% of compromises)

Problem: One container with —privileged flag can compromise entire host. Calyo Solution:

• Default deny: capabilities.drop = [“ALL”]
• Add only required capabilities (CAP_NET_BIND_SERVICE, etc)
• Use seccomp profiles to restrict syscalls
• Monitor with Falco for privilege escalation attempts

🚀 Going Further

Complementary Resources

• 📥 [Container Hardening Checklist]: 127-item verification matrix for Docker and Kubernetes
• 📊 [Security Audit Template]: Pre-built vulnerability assessment framework
• 🎓 [Advanced Threat Modeling]: Container threat landscape analysis and remediation planning

Advanced Use Cases

1. Multi-tenant Kubernetes: Namespace isolation, resource quotas, network policies for 200+ tenants
2. Supply Chain Security: Container image signing, SBOM generation, vulnerability tracking across builds
3. Compliance Automation: Automated CIS Benchmark scanning, SOC 2/PCI-DSS compliance reporting, audit trail maintenance
4. Zero-Trust Networking: Microsegmentation with Cilium eBPF, mutual TLS encryption, identity-based access control

❓ FAQ

Q: How often should we scan container images for vulnerabilities? A: Scan immediately after build (fail on critical), daily for running containers (identify new CVEs in deployed images), and rescan all images in registry weekly. Most organizations miss new vulnerabilities in already-deployed images—establish a continuous scanning program.

Q: What’s the difference between Pod Security Policies and Pod Security Standards? A: PSP (deprecated in K8s 1.25, removed in 1.29) was a complex policy engine. PSS (simpler replacement) uses three hardened profiles: unrestricted, baseline, and restricted. For new clusters, implement restricted profile in enforce mode + Kyverno for advanced policies.

Q: How do we handle secrets rotation in containers without redeployment? A: Use external secret management (HashiCorp Vault, AWS Secrets Manager) with automatic sync operators like External Secrets Operator or Vault Agent. Implement sub-minute rotation at the orchestration layer without container restarts.

Q: What’s the overhead of enabling runtime security monitoring with Falco? A: eBPF-based Falco adds 3-5% CPU overhead on worker nodes. Syscall-based detection adds 8-12%. For 100-node clusters, budget 5-8 additional CPU cores. Cost-benefit: prevent $2.2M average breach versus $5K additional infrastructure.

Q: How do we approach legacy applications that require root privileges? A: Containerize with --user=root only if unavoidable, but isolate in dedicated namespace with restrictive network policies, deploy minimal base image, monitor with Falco syscall rules, rotate credentials weekly, and commit to root-less refactoring within 6 months.

Q: What should we prioritize: image security or runtime security? A: Both are essential, but image security provides the foundation. Prioritize in order: (1) image scanning + hardening (prevents 60% of attacks), (2) pod security standards (prevents misconfiguration), (3) network policies (prevents lateral movement), (4) runtime monitoring (detects advanced threats). This layered approach follows defense-in-depth principles.

Written by

Azzeddine AMIAR

Founder & CEO

Calyo Consulting

Connect