Security

Ceph provides multiple layers of security: CephX authentication for daemon and client identity, a capability-based authorization system, encryption at rest, and TLS for network transport. When deployed via Rook on Kubernetes, additional pod security context and RBAC controls apply.

See also: index, architecture, operations

CephX Authentication

Protocol Overview

CephX is Ceph's native authentication protocol, enabled by default. It uses shared secret keys to provide mutual authentication between clients and the cluster, similar in design to Kerberos:

1. A client contacts a Monitor and presents its user ID.
2. The Monitor generates a session key encrypted with the user's permanent secret key and returns it to the client.
3. The client decrypts the session key (proving possession of the secret) and requests a ticket for OSD and MDS access.
4. The Monitor returns a ticket encrypted with the session key.
5. The client presents the ticket to OSDs and MDS daemons to authenticate ongoing operations.

Every message between client and daemon is signed with a ticket verifiable by all cluster daemons, since Monitors, OSDs, and MDS share a secret. Tickets expire based on a configurable TTL (auth_service_ticket_ttl, default 1 hour).

CephX Scope

CephX authenticates connections between Ceph clients and Ceph daemons. It does not encrypt data in transit or at rest by itself. For transport encryption, use the msgr2 protocol with TLS. For data at rest, use dmcrypt on OSDs.

Configuration

CephX is controlled by three configuration options (all default to cephx):

Setting	Scope	Default
auth_cluster_required	Daemon-to-daemon authentication	cephx
auth_service_required	Client-to-daemon authentication	cephx
auth_client_required	Daemon-to-client authentication	cephx

Disabling any of these (setting to none) is strongly discouraged because it exposes the cluster to man-in-the-middle attacks.

User and Capability System

Users

Ceph users are identified as client.<name> (for example, client.admin, client.glance, client.rbd). Each user has a secret key stored in a keyring file and a set of capabilities (caps) that define what the user can do.

Capability Syntax

Capabilities are expressed per daemon type with specific keywords:

Keyword	Meaning
allow	Precedes access settings; implies rw for MDS
r	Read access
w	Write access
x	Execute (call class methods, conduct auth operations)
class-read	Subset of x; call class read methods
class-write	Subset of x; call class write methods
* / all	Full read, write, execute, and admin permissions
profile rbd	Predefined profile for RBD access
profile osd	Predefined profile for OSD-to-Monitor communication
profile mgr	Predefined profile for Manager daemon
profile mds	Predefined profile for MDS daemon

Capabilities can be scoped to specific pools:

# Grant read/write access to a single pool
ceph auth get-or-create client.app1 mon 'allow r' osd 'allow rw pool=myapp'

# Grant RBD profile access to an OpenStack Glance pool
ceph auth get-or-create client.glance mon 'profile rbd' osd 'profile rbd pool=images' mgr 'profile rbd pool=images'

Daemon Keyrings

Each daemon type has a default keyring location and minimal required capabilities:

Daemon	Keyring Location	Required Capabilities
ceph-mon	$mon_data/keyring	mon 'allow *'
ceph-osd	$osd_data/keyring	mgr 'allow profile osd' mon 'allow profile osd' osd 'allow *'
ceph-mds	$mds_data/keyring	mds 'allow' mgr 'allow profile mds' mon 'allow profile mds' osd 'allow rwx'
ceph-mgr	$mgr_data/keyring	mon 'allow profile mgr' mds 'allow *' osd 'allow *'
radosgw	$rgw_data/keyring	mon 'allow rwx' osd 'allow rwx'

User Management Commands

# List all users
ceph auth ls

# Create a user with specific caps
ceph auth get-or-create client.app1 mon 'allow r' osd 'allow rw pool=data'

# Modify capabilities
ceph auth caps client.app1 mon 'allow rw' osd 'allow rwx pool=data'

# View user details
ceph auth get client.app1

# Delete a user
ceph auth del client.app1

Encryption at Rest

OSD Data Encryption (dmcrypt)

Ceph supports full-disk encryption on OSDs using Linux dm-crypt (LUKS). When enabled via ceph-volume:

• Each OSD's data device is encrypted with a unique dmcrypt_key.
• The dmcrypt_key is stored in a "lockbox" -- a special OSD monitored by the Ceph Monitors, encrypted with a cephx_lockbox_secret.
• On OSD startup, the Monitor provides the lockbox secret, which decrypts the dmcrypt_key, which in turn unlocks the data device.

The lockbox key payload is structured as:

{
    "cephx_secret": "<CEPHX_SECRET>",
    "dmcrypt_key": "<DMCRYPT_KEY>",
    "cephx_lockbox_secret": "<LOCKBOX_SECRET>"
}

BlueStore Encryption

BlueStore also supports internal encryption of the WAL and DB devices via its own bluefs_buffered_io and key management, but the primary at-rest encryption mechanism remains dm-crypt at the block device level.

Network Transport Security

Messenger v2 (msgr2)

Ceph's Messenger v2 protocol supports two modes:

• crc mode: Provides CRC32C integrity checking and CephX authentication. Messages are authenticated but not encrypted.
• secure mode: Adds full TLS-like encryption using the cephx session keys for key exchange, encrypting all message payloads.

Configuration:

# Require secure mode for all connections
ms_cluster_mode = secure
ms_service_mode = secure
ms_client_mode = secure

RGW TLS

The RADOS Gateway supports TLS termination natively:

rgw_frontends = beast ssl_port=443 ssl_certificate=/etc/ceph/rgw.crt ssl_private_key=/etc/ceph/rgw.key

For deployments behind a load balancer (HAProxy, NGINX), TLS can be terminated at the load balancer with backend connections over plain HTTP or over secure msgr2.

RGW Bucket Policies and ACLs

The RADOS Gateway implements S3-compatible access control:

• Bucket policies: JSON policy documents attached to buckets, supporting principal-based, IP-based, and VPC-endpoint-based conditions.
• ACLs: Legacy S3 ACLs for backward compatibility (canned ACLs like private, public-read, authenticated-read).
• User-level permissions: RGW maintains its own user database separate from CephX. RGW users have access keys and secret keys for S3 authentication (AWS Signature V4).

Rook Security Context

When Ceph is deployed via Rook on Kubernetes, the operator applies security contexts:

• Pod Security Standards: Rook requires privileged or baseline Pod Security Standards for Ceph OSD pods (needed for block device access and dm-crypt).
• RBAC: The Rook operator uses ClusterRoles with specific permissions for CRD management. Ceph CSI drivers have separate ServiceAccounts scoped to provisioning and publishing operations.
• SecurityContextConstraints (OpenShift): Rook ships with SCC definitions for OSD, MGR, and MON pods.
• Network Policies: Rook does not ship default NetworkPolicies; administrators should create them to restrict inter-namespace traffic to only Ceph daemon communication ports (6789, 3300, 6800-7300).

Threat Model Summary

Threat	Mitigation
Unauthorized client access	CephX mutual authentication, capability scoping
Man-in-the-middle on cluster network	msgr2 secure mode, CephX message signatures
Data exposure from stolen disks	dm-crypt OSD encryption
Unauthorized S3 access	RGW access keys, bucket policies, IAM
Privilege escalation in Kubernetes	Rook RBAC, Pod Security Standards, dedicated ServiceAccounts
Replay attacks	CephX ticket TTL, session key rotation
Silent data corruption	BlueStore checksumming, scrubbing

Sources

• Ceph Architecture - Authentication
• CephX Config Reference
• Ceph User Management
• Ceph Volume Encryption
• Messenger v2