Failure domain
A group of components that can fail together. This is a placement boundary, not an Internet domain.
PLANNING & FUNDAMENTALS
Ceph capacity calculator & resilience
Plan capacity, replication and erasure coding with worked examples, custom inputs and explanations of host and rack failures.
PLAN YOUR CLUSTER
Your hardware. Your capacity.
Calculate capacity from your node and drive configuration. Choose a worked example or enter your own values, then compare replication and erasure coding.
Examples illustrate capacity arithmetic; they are not hardware or deployment recommendations. All inputs remain editable.
Capacity budget after reserve
— TB- Raw capacity
- —
- Replication / parity overhead
- —
- After protection, before reserve
- —
- Reserved usable capacity
- —
| Profile | Efficiency | Before reserve | After reserve | Host placement |
|---|
Planning estimate, not a guaranteed usable capacity or availability assessment. TB is decimal; TiB is binary. Actual capacity is lower due to metadata, small-object overhead, placement imbalance and pool limits. The reserve is deducted from capacity after protection. Placement checks assume distinct hosts; rack-level rules need a separate assessment. Minimum host counts provide no extra host for rebuilding. Validate recovery space, quorum, min_size, workload and the supported EC profile before deployment.
Plan metadata and DB/WAL capacity separately →
CAPACITY IS ONLY ONE PART
What can fail—and why more hosts help
Host
One physical server, including its drives. A power supply, mainboard or network failure can make all its OSDs unavailable. With host placement, chunks of the same object go to different servers.
Rack
Several servers sharing a rack, often also power or a switch. Rack placement separates chunks of the same object across racks. For a standard 4+1 rule with one chunk per rack, five suitable racks are needed—not just five hosts.
Worked example: 4+1 with host placement
Four data chunks plus one parity chunk form a set of five. Any four intact chunks are enough to reconstruct it. The limit applies per object / chunk set, not as a global count of broken disks.
Host 1D1Host 2D2Host 3D3Host 4D4Host 5PHost 6↻Recovery target
Illustration of one chunk set. The sixth host can already store other data; it need not be an idle spare.
One drive / OSD
One affected chunk can be reconstructed from the other four.
One complete host
With correct host placement, at most one chunk per object is missing—even if every drive on that host fails.
Two hosts before recovery completes
Not covered by 4+1. If both hold chunks of the same object, it is unavailable; permanently losing both chunks makes that object unrecoverable.
A switch, PDU or entire rack
Host placement alone offers no rack guarantee: several chunks may be affected together. With one chunk per rack, 4+1 can reconstruct the data after losing one rack.
Network partition, gateway or quorum loss
Data may remain intact while applications lose access. Redundant paths, services and monitor placement need their own design.
Reconstructable does not mean continuously writable
With 4+1 and min_size=5, losing one chunk leaves four: reconstruction is possible, but normal client I/O to the affected placement groups is blocked until enough chunks are available again. Do not lower min_size just to make a failure look safe. Deletion, ransomware and site-wide loss still need independent backups.
Five hosts versus six or more
With exactly five hosts and host-level 4+1, one failed host leaves only four placement targets. Free space on those four does not create a fifth failure domain. A replacement or returning host is needed to restore all five chunks. With six hosts, five remain after one fails, so reconstruction onto another host becomes possible—if eligible space, network bandwidth and operational conditions permit it.
More balanced hosts can spread recovery work and reduce the share of capacity lost with one server. They do not turn 4+1 into 4+2: before protection is fully restored, a second overlapping chunk loss remains dangerous. More hardware also adds cost and operational work. For rack placement, extra hosts in the same racks do not replace an additional eligible rack.
Technical references: Ceph — EC recovery & min_size ↗ · Ceph — CRUSH failure domains ↗
Have a workload in mind?
Start with your capacity, applications and recovery goals. Then choose the architecture.