Skip to content

Operations

Scope

Production deployment, eBPF datapath management, network policy operations, Hubble observability, and troubleshooting.

Deployment Patterns

Installation Methods

Method Use Case Notes
Helm Production Most flexible, recommended for prod
cilium CLI Dev/quick setup cilium install
Managed (GKE, EKS) Cloud-native Uses cloud CNI integration
Standalone Non-K8s eBPF Runtime enforcement without K8s 
# Production Helm install
helm install cilium cilium/cilium --version 1.17.x \
  --namespace kube-system \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=${API_SERVER_IP} \
  --set k8sServicePort=6443 \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true \
  --set bpf.masquerade=true \
  --set ipam.mode=kubernetes

kube-proxy Replacement

Cilium can fully replace kube-proxy using eBPF for service load balancing:

# Verify kube-proxy replacement
cilium status | grep KubeProxyReplacement
# Should show: KubeProxyReplacement: True

Network Policy

CiliumNetworkPolicy (L3/L4/L7)

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-http-frontend
spec:
  endpointSelector:
    matchLabels:
      app: backend
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP
      rules:
        http:
        - method: GET
          path: "/api/.*"

Hubble Observability

# Enable Hubble
cilium hubble enable --ui

# Observe live flows
hubble observe --namespace default --follow
hubble observe --verdict DROPPED --follow

# Service map
hubble observe --output json | hubble map

Troubleshooting

Symptom Diagnosis Fix
Pod connectivity fails cilium status, cilium connectivity test Check BPF maps, restart agent
Policy not enforced cilium endpoint list, check labels Verify label selectors match
High CPU on agent cilium metrics Tune BPF map sizes, check conntrack
Hubble flows missing hubble status Enable Hubble relay, check port 4245
DNS resolution issues cilium monitor --type l7 Check DNS proxy, CoreDNS connectivity

Upgrade Procedure

# Pre-flight check
cilium connectivity test

# Upgrade via Helm
helm upgrade cilium cilium/cilium --version 1.17.x --namespace kube-system --reuse-values

# Post-upgrade validation
cilium status --wait
cilium connectivity test

Commands & Recipes

Installation

# Install Cilium CLI
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
curl -L --fail --remote-name-all \
  https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-amd64.tar.gz
sudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin

# Install Cilium on K8s (Helm)
helm repo add cilium https://helm.cilium.io/
helm install cilium cilium/cilium --version 1.19.2 \
  --namespace kube-system \
  --set hubble.enabled=true \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true

# Validate installation
cilium status --wait
cilium connectivity test

Hubble Observability

# Install Hubble CLI
export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
curl -L --fail --remote-name-all \
  https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-amd64.tar.gz
sudo tar xzvfC hubble-linux-amd64.tar.gz /usr/local/bin

# Port-forward Hubble Relay
cilium hubble port-forward &

# Observe flows
hubble observe --namespace default
hubble observe --pod myapp --protocol HTTP --verdict DROPPED
hubble observe --to-fqdn "*.amazonaws.com"

# Service map (requires Hubble UI)
kubectl port-forward -n kube-system svc/hubble-ui 12000:80

Network Policies

# L7 HTTP policy — allow only GET /api/*
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-api-get
spec:
  endpointSelector:
    matchLabels:
      app: backend
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: frontend
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              - method: GET
                path: "/api/.*"

# DNS-based egress policy
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-external-dns
spec:
  endpointSelector:
    matchLabels:
      app: myapp
  egress:
    - toFQDNs:
        - matchPattern: "*.googleapis.com"
      toPorts:
        - ports:
            - port: "443"

Tetragon Runtime Security

# Detect privilege escalation
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: detect-priv-escalation
spec:
  kprobes:
    - call: __x64_sys_setuid
      syscall: true
      args:
        - index: 0
          type: int
      selectors:
        - matchArgs:
            - index: 0
              operator: Equal
              values: ["0"]
          matchActions:
            - action: Sigkill

Troubleshooting

# Check Cilium agent status
cilium status
kubectl -n kube-system exec ds/cilium -- cilium-dbg status --verbose

# Check endpoint state
kubectl -n kube-system exec ds/cilium -- cilium-dbg endpoint list

# Check BPF maps
kubectl -n kube-system exec ds/cilium -- cilium-dbg bpf ct list global

# Monitor dropped packets
cilium monitor --type drop

# Debug specific pod
cilium-dbg endpoint get <endpoint-id>

Sources