Skip to content

Operations

Deployment, configuration, scaling, and day-2 operations for Apache SkyWalking.

Deployment

Kubernetes (Helm) — with BanyanDB

SkyWalking uses OCI-based Helm charts:

# Login to Docker Hub OCI registry (required for OCI charts)
helm registry login registry-1.docker.io

# Install SkyWalking with BanyanDB
helm install skywalking \
  oci://registry-1.docker.io/apache/skywalking-helm \
  --version <VERSION> \
  -n skywalking --create-namespace \
  --set oap.storageType=banyandb \
  --set banyandb.enabled=true

Docker Compose

git clone https://github.com/apache/skywalking.git
cd skywalking/docker
docker compose up -d

Configuration

OAP Server Environment Variables

Variable Default Description
SW_STORAGE banyandb Storage backend (banyandb, elasticsearch, h2, mysql, postgresql)
SW_STORAGE_BANYANDB_TARGETS banyandb:17912 BanyanDB gRPC endpoint
SW_CORE_RECORD_DATA_TTL 3 Trace/log retention (days)
SW_CORE_METRICS_DATA_TTL 7 Metrics retention (days)
SW_VIRTUAL_THREADS_ENABLED true (JDK 25+) Enable virtual threads
SW_RECEIVER_GRPC_PORT 11800 gRPC receiver port
SW_RECEIVER_HTTP_PORT 12800 HTTP receiver port

BanyanDB Configuration

Variable Default Description
BYDB_PORT 17912 gRPC listening port
BYDB_HTTP_PORT 17913 HTTP/UI port

BanyanDB Lifecycle Stages (Helm 0.5.0+)

banyandb:
  lifecycle:
    hot:
      retention: 3d
    warm:
      retention: 14d
    cold:
      retention: 90d
      storage:
        type: s3
        bucket: skywalking-cold

Scaling

OAP Cluster

OAP instances are stateless — scale horizontally:

oap:
  replicas: 3
  resources:
    requests:
      cpu: "2"
      memory: "4Gi"
    limits:
      cpu: "4"
      memory: "8Gi"

BanyanDB Cluster

banyandb:
  liaison:
    replicas: 2
  data:
    replicas: 3
    persistentVolume:
      size: 100Gi

Monitoring

Health Checks

# OAP health
curl http://oap:12800/internal/l7check

# BanyanDB health
curl http://banyandb:17913/api/healthz

# Kubernetes pods
kubectl get pods -n skywalking

Key Metrics

Metric Description
oap_server_instance_count Number of OAP instances
banyandb_write_latency BanyanDB write performance
banyandb_disk_usage Storage consumption

Upgrades

helm upgrade skywalking \
  oci://registry-1.docker.io/apache/skywalking-helm \
  --version <NEW_VERSION> \
  -n skywalking \
  -f custom-values.yaml

v10.4.0 Migration Notes

  • LAL breaking change: slowSql {} and sampledTrace {} sub-DSLs removed — migrate to outputType mechanism
  • LALOutputBuilder.init() signature changed — custom implementations must update
  • Default JDK: Docker base image now JDK 25; JDK 11 available as -java11 variant

Sources


Commands & Recipes

Runnable commands, configuration snippets, and troubleshooting recipes for Apache SkyWalking.

Installation

Helm with BanyanDB

helm install skywalking \
  oci://registry-1.docker.io/apache/skywalking-helm \
  --version <VERSION> \
  -n skywalking --create-namespace \
  --set oap.storageType=banyandb \
  --set banyandb.enabled=true

Helm with Elasticsearch

helm install skywalking \
  oci://registry-1.docker.io/apache/skywalking-helm \
  --version <VERSION> \
  -n skywalking --create-namespace \
  --set oap.storageType=elasticsearch \
  --set oap.env.SW_STORAGE_ES_CLUSTER_NODES="es-master:9200"

BanyanDB CLI (bydbctl)

# List all groups
bydbctl group list

# Query a measure
bydbctl measure query --group sw_metric --name service_cpm

# List streams in a group
bydbctl stream list --group sw_record

# Create an index rule
bydbctl indexrule create -f my-index-rule.yaml

# Delete old data from a group
bydbctl group update --name sw_metric --ttl 7d

BanyanDB Embedded UI

# Access BanyanDB UI (port 17913)
kubectl port-forward -n skywalking svc/banyandb 17913:17913
# Open http://localhost:17913

OAP Server

Health & Status

# OAP L7 health check
curl http://oap:12800/internal/l7check

# GraphQL query — list services
curl -X POST http://oap:12800/graphql \
  -H "Content-Type: application/json" \
  -d '{"query": "{ listServices(duration: {start: \"2026-04-11\", end: \"2026-04-11\", step: DAY}) { key: id label: name } }"}'

Environment Variable Overrides

# Common OAP overrides in Helm values
oap:
  env:
    SW_STORAGE: banyandb
    SW_STORAGE_BANYANDB_TARGETS: "banyandb:17912"
    SW_CORE_RECORD_DATA_TTL: "7"
    SW_CORE_METRICS_DATA_TTL: "30"
    SW_VIRTUAL_THREADS_ENABLED: "true"
    SW_RECEIVER_GRPC_PORT: "11800"

Java Agent

# Attach Java agent
java -javaagent:/path/to/skywalking-agent.jar \
  -Dskywalking.agent.service_name=my-service \
  -Dskywalking.collector.backend_service=oap:11800 \
  -jar my-application.jar

Troubleshooting

OAP Not Receiving Data

# Check receiver ports
kubectl get svc -n skywalking | grep oap

# Check OAP logs for receiver errors
kubectl logs -n skywalking -l app=oap --tail=100 | grep -i "error\|receiver"

# Verify agent connectivity from app pod
kubectl exec -it <app-pod> -- nc -zv oap 11800

BanyanDB Disk Full

# Check disk usage
kubectl exec -n skywalking $(kubectl get pod -n skywalking -l app=banyandb -o name | head -1) -- \
  df -h /data

# Reduce retention
kubectl set env deployment/oap -n skywalking \
  SW_CORE_RECORD_DATA_TTL=3 \
  SW_CORE_METRICS_DATA_TTL=7

Upgrade

helm upgrade skywalking \
  oci://registry-1.docker.io/apache/skywalking-helm \
  --version <NEW_VERSION> \
  -n skywalking -f values.yaml

# Verify
kubectl get pods -n skywalking

Sources