Designing distributed systems means balancing reliability, scalability, and fault tolerance. A key principle here is the CAP Theorem, introduced by Eric Brewer (2000) and proved by Gilbert & Lynch (2002).

What is the CAP Theorem?

In the event of a network partition, a distributed system can only guarantee two out of three properties:

Consistency (C): Every read gets the latest write.
Availability (A): Every request receives a response (may not be the latest).
Partition Tolerance (P): The system keeps working despite network failures.

👉 In practice, P is unavoidable, so during a partition you must choose Consistency (CP) or Availability (AP).

CP vs AP Systems

CP (Consistency + Partition Tolerance): Prioritizes correctness over availability.
- 📌 Example: MongoDB with majority reads/writes ensures data is consistent, but may reject requests during partitions.
AP (Availability + Partition Tolerance): Stays responsive but may serve stale data.
- 📌 Example: Amazon DynamoDB accepts reads/writes on all nodes, resolving conflicts later.

CAP vs ACID Consistency

ACID Consistency: Ensures database validity (constraints, rules).
CAP Consistency: Ensures all nodes return the same latest value.

Beyond CAP: PACELC

CAP applies during partitions, but PACELC extends it:

If Partition (P): Choose Availability or Consistency.
Else (E): Trade between Latency and Consistency.

Key Takeaway

Failures are inevitable. CAP isn’t about choosing “2 of 3” but about deciding between consistency or availability when partitions occur. Designing distributed systems means making informed trade-offs, not avoiding them.