CONTAINER & KUBERNETES

Container Security

Comprehensive security for Docker containers and Kubernetes clusters - from build time through deployment to runtime protection.

What is Container Security?

Container Security encompasses all practices, tools, and processes for securing containerized applications from build time through deployment to runtime. Containers share the host system's kernel, which introduces new security challenges compared to traditional virtual machines. Effective container security requires a multi-layered approach covering image scanning, runtime protection, network isolation, secrets management, and compliance enforcement.

Why is Container Security Critical?

Containers have transformed how we develop and deploy applications, but they have also introduced new security risks:

  • Shared kernel: All containers on a host share the same OS kernel - a kernel exploit can compromise the entire host
  • Image vulnerabilities: 80%+ of Docker images contain known CVE vulnerabilities
  • Misconfigurations: Privileged containers, mounted Docker socket, exposed APIs
  • Supply chain risks: Untrusted base images, malicious layers, backdoored dependencies
  • Runtime threats: Container breakout, lateral movement, crypto mining
  • Secrets exposure: Hardcoded credentials, env variables, mounted secrets
  • Network attacks: Pod-to-pod lateral movement without network policies

Notable Container Security Incidents

  • Tesla Kubernetes cryptojacking (2018): Unprotected Kubernetes dashboard used for mining
  • Docker Hub malicious images: 17 malicious images with 5M+ pulls contained cryptominers
  • RunC vulnerability (CVE-2019-5736): Container breakout enabling root access on the host
  • Kinsing malware: Exploits misconfigured Kubernetes for cryptomining and botnet operations

Container Security Layers

Defense in Depth Approach

1. Build Time Security

Security during the image build process

  • Base image selection (minimal, trusted sources)
  • Dockerfile best practices (non-root user, no secrets)
  • Image scanning for CVEs (Trivy, Snyk, Clair)
  • Multi-stage builds for a smaller attack surface
  • Image signing and verification (Cosign, Notary)

2. Registry Security

Securing the container registry

  • Private registries with authentication
  • Image vulnerability scanning in the registry
  • Access control (RBAC, IAM)
  • Image retention policies
  • Webhook notifications for new vulnerabilities

3. Orchestration Security (Kubernetes)

Securing the Kubernetes cluster

  • Pod Security Standards (Restricted, Baseline)
  • Network Policies (CNI-based isolation)
  • RBAC for users and service accounts
  • Admission Controllers (OPA, Kyverno)
  • Secrets management (Vault, External Secrets)

4. Runtime Security

Monitoring and protection of running containers

  • Runtime behavior monitoring (Falco, Tetragon)
  • Anomaly detection
  • Process/file/network monitoring
  • Intrusion prevention
  • Audit logging

5. Host Security

Securing the underlying infrastructure

  • Minimal host OS (COS, Bottlerocket)
  • Kernel hardening (AppArmor, SELinux)
  • Host patching and updates
  • CIS Benchmarks compliance
  • Node isolation

Docker Security Best Practices

Secure Dockerfile

# BAD - Security issues FROM ubuntu:latest RUN apt-get update && apt-get install -y curl COPY secret.key /app/ ENV DB_PASSWORD="hardcoded123" USER root EXPOSE 22 # GOOD - Secure Dockerfile # 1. Use specific version tag (not latest) FROM node:18.17.1-alpine3.18 # 2. Install security updates RUN apk update && apk upgrade && \ apk add --no-cache dumb-init && \ rm -rf /var/cache/apk/* # 3. Create non-root user RUN addgroup -g 1001 -S nodejs && \ adduser -S nodejs -u 1001 # 4. Set working directory WORKDIR /app # 5. Copy only necessary files COPY --chown=nodejs:nodejs package*.json ./ RUN npm ci --only=production && npm cache clean --force COPY --chown=nodejs:nodejs . . # 6. Switch to non-root user USER nodejs # 7. Expose only necessary ports EXPOSE 3000 # 8. Use exec form for CMD (signal handling) ENTRYPOINT ["/usr/bin/dumb-init", "--"] CMD ["node", "server.js"] # 9. Health check HEALTHCHECK --interval=30s --timeout=3s \ CMD node healthcheck.js || exit 1

Docker Compose Security

# docker-compose.yml version: '3.8' services: app: image: myapp:1.2.3 # Read-only root filesystem read_only: true tmpfs: - /tmp - /var/run # Drop all capabilities, add only needed cap_drop: - ALL cap_add: - NET_BIND_SERVICE # Non-root user user: "1001:1001" # Security options security_opt: - no-new-privileges:true # Resource limits deploy: resources: limits: cpus: '0.5' memory: 512M reservations: cpus: '0.25' memory: 256M # Secrets from file (not environment) secrets: - db_password environment: - NODE_ENV=production - DB_HOST=postgres # NO hardcoded secrets! networks: - app_network secrets: db_password: file: ./secrets/db_password.txt networks: app_network: driver: bridge

Image Scanning Tools

Open Source

Trivy

The most popular open-source vulnerability scanner for container images, filesystems, and git repos.

  • Detects CVEs in OS packages and application dependencies
  • Supports Docker, containerd, Kubernetes
  • Fast (seconds) with offline mode
  • IaC scanning (Terraform, CloudFormation)
  • Secret detection
  • SBOM generation
Open Source

Clair

Static analysis tool for vulnerabilities in container images by CoreOS.

  • Layer-by-layer analysis
  • Multiple vulnerability databases
  • API-first design
  • Integration with Quay registry
Commercial

Snyk Container

Developer-first container security with fix recommendations.

  • Base image recommendations
  • Prioritized fix guidance
  • CI/CD integration
  • Kubernetes deployment scanning
Cloud Native

AWS ECR Scanning

Native image scanning in Amazon ECR registries.

  • Basic (Clair-based) scanning
  • Enhanced (Inspector-based)
  • Continuous scan on push
  • EventBridge notifications

Practical Example: Trivy in CI/CD

# GitHub Actions - Trivy scan name: Container Security Scan on: push: branches: [ main ] pull_request: jobs: trivy_scan: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v3 - name: Build Docker image run: docker build -t myapp:${{ github.sha }} . - name: Run Trivy vulnerability scanner uses: aquasecurity/trivy-action@master with: image-ref: 'myapp:${{ github.sha }}' format: 'sarif' output: 'trivy-results.sarif' severity: 'CRITICAL,HIGH' exit-code: '1' # Fail build on HIGH/CRITICAL - name: Upload Trivy results to GitHub Security uses: github/codeql-action/upload-sarif@v2 if: always() with: sarif_file: 'trivy-results.sarif' - name: Trivy config scan (Dockerfile) uses: aquasecurity/trivy-action@master with: scan-type: 'config' scan-ref: '.' format: 'table'

Kubernetes Security

Pod Security Standards

Kubernetes defines three levels of pod security policies:

# Privileged - No restrictions (dev only) # Baseline - Minimal restrictions (prevents known privilege escalations) # Restricted - Heavily restricted (hardened, follows pod hardening best practices) # Namespace labeling for enforcement apiVersion: v1 kind: Namespace metadata: name: production labels: pod-security.kubernetes.io/enforce: restricted pod-security.kubernetes.io/audit: restricted pod-security.kubernetes.io/warn: restricted --- # Restricted Pod example apiVersion: v1 kind: Pod metadata: name: secure-pod namespace: production spec: securityContext: # Run as non-root runAsNonRoot: true runAsUser: 1000 fsGroup: 2000 # Drop all capabilities seccompProfile: type: RuntimeDefault containers: - name: app image: myapp:1.0.0 securityContext: # Read-only root filesystem readOnlyRootFilesystem: true # Drop all capabilities capabilities: drop: - ALL # No privilege escalation allowPrivilegeEscalation: false # Non-root runAsNonRoot: true runAsUser: 1000 # Resource limits resources: limits: memory: "256Mi" cpu: "500m" requests: memory: "128Mi" cpu: "250m" # Writable volumes only where necessary volumeMounts: - name: tmp mountPath: /tmp - name: cache mountPath: /app/cache volumes: - name: tmp emptyDir: {} - name: cache emptyDir: {}

Network Policies

# Default Deny All (best practice baseline) apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: production spec: podSelector: {} policyTypes: - Ingress - Egress --- # Allow specific ingress/egress apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: api-network-policy namespace: production spec: podSelector: matchLabels: app: api policyTypes: - Ingress - Egress # Ingress rules ingress: - from: # Only from frontend pods - podSelector: matchLabels: app: frontend # Only from same namespace - namespaceSelector: matchLabels: name: production ports: - protocol: TCP port: 8080 # Egress rules egress: # Allow DNS - to: - namespaceSelector: matchLabels: name: kube-system - podSelector: matchLabels: k8s-app: kube-dns ports: - protocol: UDP port: 53 # Allow database - to: - podSelector: matchLabels: app: postgres ports: - protocol: TCP port: 5432

Runtime Security

Open Source (CNCF)

Falco

Runtime security tool using eBPF for syscall monitoring and anomaly detection.

  • Detects abnormal behavior (shell spawn, file access)
  • Custom rules (Falco rules syntax)
  • Kubernetes-aware
  • Real-time alerting
  • Integration with SIEM
Open Source

Tetragon

eBPF-based security observability and runtime enforcement by Cilium.

  • Process execution monitoring
  • File access tracking
  • Network connections
  • Syscall filtering
  • Policy enforcement

Falco Rules Example

# /etc/falco/falco_rules.local.yaml # Detect shell spawned in a container - rule: Shell Spawned in Container desc: Detect shell spawned in container condition: > spawned_process and container and proc.name in (bash, sh, zsh) and not proc.pname in (sh, bash, docker, containerd) output: > Shell spawned in container (user=%user.name container_id=%container.id image=%container.image.repository proc=%proc.cmdline parent=%proc.pname) priority: WARNING tags: [container, shell, mitre_execution] # Detect sensitive file reads - rule: Read Sensitive File desc: Detect reads of sensitive files condition: > open_read and container and fd.name in (/etc/shadow, /etc/sudoers, /root/.ssh/id_rsa) output: > Sensitive file opened for reading (user=%user.name file=%fd.name container=%container.name image=%container.image) priority: CRITICAL tags: [filesystem, mitre_credential_access] # Detect crypto mining - rule: Crypto Miner Detected desc: Detect known crypto mining processes condition: > spawned_process and proc.name in (xmrig, ethminer, cpuminer) output: > Crypto miner detected! (process=%proc.name cmdline=%proc.cmdline) priority: CRITICAL tags: [malware, cryptomining]

Container Security Best Practices - Summary

Build Time

  • Use minimal base images (alpine, distroless, scratch)
  • Specific version tags (not latest)
  • Multi-stage builds for smaller images
  • Scan images before pushing to registry (Trivy, Snyk)
  • Non-root user
  • No secrets in images or Dockerfile
  • Sign images (Cosign, Notary)

Kubernetes Deployment

  • Pod Security Standards (Restricted for production)
  • Network Policies (default deny all)
  • Resource limits (CPU, memory)
  • RBAC (least privilege)
  • Secrets from external store (Vault, AWS Secrets Manager)
  • Admission controllers (OPA, Kyverno)
  • Service mesh for mTLS (Istio, Linkerd)

Runtime

  • Runtime monitoring (Falco, Tetragon)
  • Anomaly detection and alerting
  • Audit logging
  • Incident response playbooks
  • Regular security audits

What NOT to Do

  • Use the latest tag in production
  • Run containers as root
  • Mount Docker socket into a container
  • Privileged containers without justification
  • Hardcoded secrets in images
  • No network policies
  • Disable security features for "faster debugging"
  • Ignore scan results