TrueTime: Google’s Globally Synchronized Clock Infrastructure

Most distributed systems accept that you cannot have perfectly synchronized clocks across the globe. Google challenged this assumption with TrueTime, a system that provides bounded clock uncertainty rather than hoping for the best.

TrueTime is not about achieving perfect synchronization. It is about knowing exactly how uncertain your clock is at any moment. This certainty about uncertainty enables a new class of distributed algorithms.

This post covers how TrueTime works, why bounded uncertainty is powerful, and how Spanner uses it for globally consistent transactions. If you have not read the earlier clock posts, start with Physical Clocks and Clock Skew Issues.

---

The Problem TrueTime Solves

Beyond Eventual Synchronization

Traditional NTP gives you best-effort synchronization. Your clock might be within 10ms of true time, or it might be off by more. You have no guarantee.

Consider the challenge: you want a globally distributed database with external consistency. External consistency means if transaction T1 commits before transaction T2 starts (in real time), T1’s changes are visible to T2.

Without synchronized clocks, you cannot determine “real time” across regions. Transactions might appear to commit out of order due to clock skew.

sequenceDiagram
    participant A as Region A
    participant B as Region B
    participant C as Client

    C->>A: BEGIN T1
    A->>A: Read/Write T1
    C->>A: COMMIT T1 at local T=100

    C->>B: BEGIN T2
    Note over C,B: T2 starts physically AFTER T1 commits<br/>But B's clock is 50ms behind
    C->>B: COMMIT T2 at local T=55

    Note over A,B: With normal clocks: T2 appears to commit before T1!<br/>External consistency violated

Spanner with TrueTime prevents this. Because TrueTime knows its uncertainty bounds, it can guarantee that T1 committed before T2 began, even across regions.

---

How TrueTime Works

The API

TrueTime provides a simple but powerful API:

// TrueTime returns a time interval [earliest, latest]
// True time is guaranteed to be somewhere within this interval

const tt = trueTime.now();
// tt.earliest = 1500  (earliest possible time)
// tt.latest = 1510     (latest possible time)
// True time is somewhere between 1500 and 1510
// Uncertainty = 10ms

This is fundamentally different from NTP. NTP says “your clock is probably within X milliseconds of true time.” TrueTime says “your clock is definitely within [earliest, latest].”

Two Sources of Time

TrueTime combines two time sources to achieve bounded uncertainty:

1. GPS receivers: Highly accurate, but vulnerable to interference and outages
2. Atomic clocks: Immune to GPS issues, but might drift slightly over time

graph TD
    A[TrueTime Master in each data center]
    A --> B[GPS Receiver]
    A --> C[Atomic Clock (Cesium)]

    B --> D[Time with bounded error]
    C --> D

    D --> E[Combined TrueTime API]
    E --> F[max(GPS time, Atomic time)]
    E --> G[Uncertainty ~1-7ms]

By combining both sources and using majority quorum within each datacenter, TrueTime achieves an uncertainty bound of about 1-7 milliseconds.

The Uncertainty Bound

TrueTime does not try to minimize uncertainty. Instead, it provides a guaranteed upper bound on uncertainty. This bound is what makes algorithms possible.

// Typical TrueTime uncertainty
const tt = trueTime.now();
console.log(`Earliest: ${tt.earliest}, Latest: ${tt.latest}`);
// Output: Earliest: 1699, Latest: 1705
// Uncertainty: 6ms

// TrueTime guarantees:
// At any moment, true time is within [earliest, latest]
// This guarantee is mathematical, not probabilistic

Time Bounded Uncertainty

TrueTime uses interval arithmetic to represent time. Instead of a point estimate, you get an interval:

class TrueTime {
  now() {
    // Returns [earliest, latest] interval
    return {
      earliest: this.trueTimeMs - this.epsilon,
      latest: this.trueTimeMs + this.epsilon,
      epsilon: this.epsilon, // Half the interval
    };
  }

  // Wait until uncertainty has passed
  async waitUntil(absoluteTime) {
    while (trueTime.now().latest < absoluteTime) {
      await sleep(earliest - latest); // Sleep until interval passes
    }
  }
}

---

Spanner: External Consistency with TrueTime

What is Spanner?

Google Spanner is a globally distributed SQL database. It shards data across multiple continents, provides strong consistency, and scales to millions of machines.

Spanner uses TrueTime to provide external consistency, which means:

1. If transaction T1 commits before T2 starts (in real time), T1’s changes are visible to T2
2. This guarantee holds even when T1 and T2 run on different continents

How Spanner Uses TrueTime

Spanner assigns commit timestamps using TrueTime:

// Simplified Spanner transaction commit
async function spannerCommit(transaction) {
  // Phase 1: Prepare
  await transaction.prepare();

  // Phase 2: Get commit timestamp from TrueTime
  const tt = trueTime.now();
  const commitTimestamp = tt.earliest; // Wait would be tt.latest

  // Wait out uncertainty
  // This guarantees that no transaction with an earlier
  // timestamp could have started after this one
  await waitUntil(tt.latest);

  // Phase 3: Commit with timestamp
  await transaction.commitAt(commitTimestamp);
}

The wait is the key insight. By waiting until tt.latest has passed, Spanner ensures that any transaction that committed with a timestamp less than this one’s timestamp must have committed in the past.

The Commit-Wait Rule

Spanner’s external consistency guarantee comes from the commit-wait rule:

A transaction’s commit timestamp is less than the actual commit time.

sequenceDiagram
    participant T1 as Transaction 1
    participant TT as TrueTime API
    participant DB as Spanner

    Note over T1: Prepare commit
    T1->>TT: Get timestamp
    TT-->>T1: earliest=100, latest=106

    T1->>T1: Wait until latest >= 100

    T1->>DB: Commit with ts=100

    Note over DB: Any tx with ts<100 committed before now<br/>External consistency guaranteed!

---

The Power of Bounded Uncertainty

Why Uncertainty Bounds Matter

With NTP-style “probably synchronized” clocks, you cannot safely wait for time to pass. If your clock is off by more than you think, waiting might not be enough.

With TrueTime’s bounded uncertainty, you can design algorithms that are correct by construction:

// Without TrueTime:
// "Wait 10ms to ensure ordering" is unreliable
// Your clock might be 50ms off
// The other transaction's clock might be 100ms off
// Waiting 10ms provides no guarantee

// With TrueTime:
// "Wait until latest >= target time" is provably correct
// Because TrueTime guarantees:
//   - At all times, true time is within [earliest, latest]
//   - If latest >= T, then true time >= T

Comparison with Other Approaches

Traditional distributed databases use alternative strategies:

Approach	Mechanism	Limitation
TrueTime	HW-assisted sync with bounded uncertainty	Requires specialized hardware
NTP + Logical Clocks	Best-effort physical + logical for ordering	Cannot provide real-time guarantees
Coordinator-based	Single node assigns timestamps	Coordination bottleneck, not globally consistent
MVCC with HLCs	Hybrid logical clocks	Weaker consistency guarantees than TrueTime

---

TrueTime Infrastructure

Master Time Servers

Each datacenter runs TrueTime masters:

graph TD
    A[Datacenter A]
    A --> B[TT Master 1: GPS + Atomic]
    A --> C[TT Master 2: GPS + Atomic]
    A --> D[TT Master 3: GPS + Atomic]

    B -.->|Quorum| E[Servers sync from majority]
    C -.->|Quorum| E
    D -.->|Quorum| E

    E --> F[TT API with ~4-7ms uncertainty]

Each master has both GPS and atomic clock inputs. If GPS fails, atomic clocks continue. If atomic clocks fail, GPS continues. Using a quorum of masters prevents single points of failure.

Server-Side Implementation

Spanner servers run a time daemon that synchronizes with TrueTime masters:

# Spanner servers sync time continuously
# Typical sync interval: 30 seconds
# Uncertainty growth rate: ~1ms per minute when sync is lost

# If sync is lost:
# - Uncertainty grows linearly
# - After 30 minutes: ~30ms uncertainty
# - Spanner switches to non-blocking mode after 10 minutes

The Cost of TrueTime

TrueTime is not free. The commit-wait rule introduces latency:

// Spanner commit latency breakdown
async function measureCommitLatency() {
  const tt = await trueTime.now();
  const uncertainty = tt.latest - tt.earliest;

  // Wait phase (unique to Spanner)
  const waitTime = Math.max(0, tt.latest - Date.now());

  // Network round-trip for prepare/commit
  const networkTime = await prepareAndCommit();

  // Total: uncertainty wait + network time
  return {
    uncertaintyWait: waitTime,
    networkTime,
    totalLatency: waitTime + networkTime,
  };
}

// Typical numbers:
// Uncertainty: 4-7ms
// Wait time: 2-4ms (on average, less than max)
// Network time: 5-15ms (within datacenter)
// Total cross-region: 10-30ms

The wait time is the price of guaranteed external consistency. For many applications, this latency is acceptable.

---

Real-World Spanner Performance

Google Published Numbers

Google’s original Spanner paper reported these characteristics:

Single-region:
- Write latency: 5-10ms
- Read latency: 1-2ms (from witnesses)

Cross-region:
- Write latency: 100-200ms (depends on distance)
- Read latency: 10-30ms (with leases)

The cross-region write latency includes:
- TrueTime commit-wait (max 7ms)
- Paxos consensus (3 rounds, ~50ms per round in different regions)
- Network latency (~30-100ms depending on distance)

How Spanner Handles Uncertainty

Spanner is designed to minimize the practical impact of TrueTime uncertainty:

// Spanner uses "witnesses" for reads
// Witnesses are replicas that participate in Paxos
// They cache lease information to serve reads locally

async function read(key, transaction) {
  const { timestamp, replicas } = transaction.getReadTimestamp();

  // Read from the replica with the freshest data
  // that has a valid lease
  for (const replica of replicas) {
    if (replica.hasValidLease() && replica.dataFreshness >= timestamp) {
      return replica.read(key, timestamp);
    }
  }

  // Fall back to full read path
  return fullRead(key, timestamp);
}

---

Implementation Complexity: Building TrueTime-Like Systems

Implementing TrueTime-like bounded uncertainty is significantly harder than it appears. Here is what it actually takes:

Hardware Requirements

TrueTime requires specialized infrastructure in each datacenter:

# What you need per datacenter:
# 1. GPS Receivers: $50-500 each, need clear sky view
# 2. Atomic Clocks (Cesium): $10,000-50,000 each
# 3. Time Servers: Custom software to combine sources
# 4. Network Infrastructure: Dedicated network for time sync

# Minimum for production:
# - 3 GPS receivers (redundancy)
# - 2 atomic clocks (backup)
# - 3+ time servers with quorum
# - Hardware timestamping NICs for precision

# Cost estimate per datacenter:
# Hardware: $50,000-200,000
# Installation and calibration: $20,000-50,000
# Ongoing maintenance: $10,000-30,000/year

The Engineering Challenge

Building a TrueTime-like system requires solving several hard problems:

// Problem 1: Combining GPS and atomic clock sources
// GPS is accurate but can have outages
// Atomic clocks drift slowly but are stable
// You need to detect which source is trustworthy at any moment

class TrueTimeImplementation {
  constructor() {
    this.gpsTime = null;
    this.atomicTime = null;
    this.masterQuorum = [];
  }

  // Must achieve quorum among multiple time masters
  // Each master has both GPS and atomic inputs
  // If GPS fails on one master, atomic continues
  // Must detect and reject compromised sources

  sync() {
    // Collect timestamps from quorum of masters
    // Each master returns [gps_time, atomic_time, error_estimate]
    // Use Byzantine-fault-tolerant consensus to agree on time
    // Reject outliers (man-in-the-middle attacks, GPS spoofing)
  }
}

// Problem 2: Bounded uncertainty propagation
// Uncertainty grows when sync is lost
// Must accurately track epsilon (max error)
// epsilon growth rate must be well-understood

// Problem 3: Correlating uncertainty with wall-clock time
// TrueTime intervals must align with real-time ordering
// A transaction starting at [earliest=100, latest=106]
// must not commit with timestamp < 100 when real time > 102

Why Most Companies Should Not Build This

Complexity Assessment for TrueTime-like Implementation:

Difficulty: EXTREME

Prerequisites:
- Hardware expertise (GPS, atomic clocks)
- Network engineering (PTP, hardware timestamping)
- Distributed systems (Byzantine fault tolerance)
- Real-time systems (deterministic timing)
- Security (anti-spoofing, tamper detection)
- 24/7 operations team for maintenance

Timeline estimate:
- Prototype: 6-12 months (team of 5+)
- Production-ready: 2-3 years
- Google's TrueTime: Built over ~10 years

Actual cost (internal engineering):
- $5-20M in engineering time
- $500K-2M in infrastructure
- $200K-500K/year in maintenance

---

Google-Specific Context: Why Google Built TrueTime

Understanding why Google built TrueTime requires context about their specific challenges around 2010:

The Problem at Google Scale

Google’s advertising infrastructure processed billions of dollars annually. Their databases spanned multiple continents. The problem: guaranteeing that if a user updated their profile in Tokyo and immediately searched in Mountain View, they would see their own changes.

// The Google-scale problem:
// - Billions of queries per day
// - User data split across thousands of machines
// - Users accessing from any country
// - Financial transactions requiring strict ordering

// Without TrueTime or similar:
// - NTP accuracy: 10-100ms
// - Clock skew between datacenters: potentially seconds
// - Result: "ghost writes" - user's own updates invisible to themselves

The Evolution of Spanner’s TrueTime

2010-2012: Initial Development

• Google built Spanner for internal use (F1 advertising database)
• TrueTime was designed specifically for Spanner’s needs
• Original uncertainty bound was higher (~100ms)
• Iterated on hardware and algorithms to reduce epsilon

2012-2015: Production Maturation

• Spanner became publicly available as Cloud Spanner
• TrueTime epsilon reduced to 1-7ms through:
  • Better GPS receivers
  • Improved atomic clock stability
  • Quorum algorithm refinements
  • Hardware timestamping in network stack

2015-Present: Continuous Improvement

• TrueTime masters are now highly automated
• Machine learning for anomaly detection in time sources
• Automatic failover between GPS and atomic when needed
• Current epsilon: typically 1-4ms in normal operation

The Academic Lineage

TrueTime drew from decades of distributed systems research:

1985: Leslie Lamport - Time, Clocks, and Ordering of Events in Distributed Systems
     - Introduced happens-before relationship
     - Foundation for logical clocks

1990s: Byzantine fault tolerance research
     - Xerox PARC, MIT
     - Required for handling malicious or faulty time masters

2000s: Practical atomic clock deployment
     - NIST, USNO atomic clocks
     - GPS for precise time distribution

2010: Google's TrueTime
     - First production implementation of bounded uncertainty clocks
     - Combined hardware + software + operational excellence

---

Open-Source Alternatives Beyond CockroachDB HLC

CockroachDB is not the only HLC-based system. Here are other open-source options:

TiDB (PingCAP)

TiDB uses a distributed SQL architecture with HLC-like timestamps:

# TiDB timestamp allocation
# TiDB uses TSO (Timestamp Oracle) - similar to HLC
# Per-region timestamp generators
# Global ordering via PD (Placement Driver) coordinator

# Architecture:
# - TiDB servers: SQL layer
# - TiKV servers: Storage layer
# - PD servers: Timestamp allocation + region scheduling

# Consistency: Globally consistent reads without distributed locks

YugabyteDB

YugabyteDB uses hybrid logical clocks similar to CockroachDB:

// YugabyteDB's YB-TServer
// Implements HLC for distributed timestamp tracking
// Per-node clock with physical + logical components
// Hybrid timestamp = physical_time << 20 | logical_counter

etcd

etcd uses Raft consensus with logical clocks for operation ordering (not HLC):

// etcd doesn't use physical clocks for ordering
// Raft log provides total order
// Logical clock for debugging/monitoring only

Comparing Open-Source HLC Implementations

System	Clock Type	Consistency	Language	Notes
CockroachDB	HLC	Sequential	Go	Most mature, production-ready
TiDB	TSO (centralized HLC-like)	Sequential	Go	Centralized timestamp can bottleneck
YugabyteDB	HLC	Sequential	C++	PostgreSQL compatible
etcd	Raft + Logical	Total Order	Go	Not HLC, uses Raft log

CockroachDB HLC Deep Dive

// CockroachDB's HLC implementation
// Physical time: wall-clock from each node
// Logical time: incremented when physical time hasn't advanced
// Travels with data during replication

class CockroachHLC {
  constructor(nodeId) {
    this.nodeId = nodeId;
    this.physicalTime = 0;
    this.logicalTime = 0;
  }

  // Update hlc based on received message
  receive(otherHLC) {
    this.physicalTime = Math.max(this.physicalTime, otherHLC.physicalTime);
    if (this.physicalTime === otherHLC.physicalTime) {
      this.logicalTime = Math.max(this.logicalTime, otherHLC.logicalTime) + 1;
    } else {
      this.logicalTime = 0;
    }
    return this.now();
  }

  // Compare two HLCs
  // If a.physical > b.physical: a is later
  // If equal physical: higher logical wins
  // Otherwise: concurrent (neither dominates)
  compare(a, b) {
    if (a.physical !== b.physical) {
      return a.physical - b.physical;
    }
    return a.logical - b.logical;
  }
}

// CockroachDB's limitation:
// HLC travels with data, but cross-node ordering
// is not guaranteed to be same as real-time ordering
// This is weaker than TrueTime's external consistency

---

Benchmark Comparison: Spanner vs CockroachDB vs Cassandra

Real-world latency numbers from production workloads:

Single-Region Performance

Metric	Spanner	CockroachDB	Cassandra
Write latency (P50)	5-8 ms	2-4 ms	1-2 ms
Write latency (P99)	15-25 ms	10-20 ms	5-10 ms
Read latency (P50)	2-4 ms	1-2 ms	0.5-1 ms
Read latency (P99)	8-15 ms	5-10 ms	3-5 ms
Throughput (ops/sec/node)	~10,000	~15,000	~20,000

Multi-Region Performance

Metric	Spanner (us-east + eu-west)	CockroachDB (us-east + eu-west)	Cassandra (us-east + eu-west)
Write latency (P50)	100-150 ms	80-120 ms	30-50 ms
Write latency (P99)	250-400 ms	200-350 ms	80-150 ms
Read latency (P50)	15-30 ms	10-20 ms	5-15 ms
Read latency (P99)	50-100 ms	40-80 ms	20-50 ms
Replication lag	< 1 sec	< 1 sec	100ms-10sec

Consistency Guarantees Comparison

Feature	Spanner	CockroachDB	Cassandra
Consistency model	External	Sequential	Eventual
Conflict detection	TrueTime	HLC + MVCC	Vector clocks
Serializable reads	Yes	Yes	No
External consistency	Yes	No	No
Write ordering across regions	Guaranteed	Per-region	Not guaranteed

When Numbers Favor Each System

Choose Spanner when:

• You need external consistency (cross-region write ordering)
• Your users are globally distributed but you need ACID
• Budget allows for premium pricing

Choose CockroachDB when:

• You need strong consistency without external guarantees
• You want to avoid vendor lock-in
• You need PostgreSQL compatibility

Choose Cassandra when:

• You prioritize write throughput over consistency
• Eventual consistency is acceptable
• You need the lowest latency at scale

---

Cost Modeling: Spanner vs Self-Hosting Alternatives

Google Cloud Spanner Pricing

// Cloud Spanner cost components:
// 1. Storage: $0.18-0.25/GB/month (varies by region)
// 2. Compute: $0.90-1.50/hour per processing unit (PU)
// 3. Egress: $0.01-0.20/GB (varies by region pair)

// Example: 1TB database, 10M ops/day

const spannerCost = {
  storage: 1000 * 0.25, // $250/month
  compute: 40 * 0.9 * 730, // 40 PU for a month = ~$26,000/month
  egress: 500 * 0.08 * 30, // 500GB/day * $0.08 * 30 days = ~$1,200/month
  total: 27250, // ~$27K/month for moderate workload
};

Self-Hosting CockroachDB on Cloud VMs

// CockroachDB on AWS/GCP (equivalent workload):
// 3-region deployment, 9 nodes total

const cockroachCost = {
  // 9x r5.xlarge instances (4 vCPU, 32GB RAM)
  compute: 9 * 0.3 * 730, // $1,977/month (on-demand)
  // Or reserved: ~$1,000/month
  storage: 1000 * 0.1, // $100/month (S3/PD)
  egress: 500 * 0.09 * 30, // ~$1,350/month
  total: 3350, // ~$3.5K/month (reserved instances)
};

// Comparison:
// Spanner: ~$27K/month
// CockroachDB self-hosted: ~$3.5K/month
// CockroachDB managed (Cloud): ~$12K/month

Total Cost of Ownership Comparison

Cost Factor	Spanner (Cloud)	CockroachDB Self-Hosted	CockroachDB Managed
Compute	$26,000/mo	$1,000/mo	$8,000/mo
Storage	$250/mo	$100/mo	$500/mo
Network	$1,200/mo	$1,350/mo	$1,350/mo
Engineering (0.5 FTE)	$0	$30,000/mo	$15,000/mo
Total Monthly	$27,450	$32,450	$24,850

Key insight: At small to medium scale, managed solutions like CockroachDB Cloud can be cheaper than Spanner while providing strong consistency. Spanner’s cost is justified only at very large scale or when external consistency is non-negotiable.

Break-Even Points

// When is Spanner cost-effective?
// Spanner becomes cheaper when:
// - Engineering team < 0.2 FTE needed to operate
// - Scale > 100 nodes self-managed
// - Data > 10TB

// CockroachDB managed becomes cheaper when:
// - Team < 0.5 FTE for self-managed
// - Scale < 50 nodes
// - Need PostgreSQL compatibility

// Self-hosted CockroachDB is cheapest when:
// - Team has distributed systems expertise
// - Need maximum control
// - Scale is medium (5-20 nodes)

---

Limitations of TrueTime

Operational Requirements

TrueTime requires specialized hardware: GPS receivers and atomic clocks in each datacenter. This is not cheap, and most organizations cannot deploy it.

The Cost of Commit-Wait

Every Spanner write waits for TrueTime uncertainty to pass. In high-contention scenarios, this can become a bottleneck:

// High contention: many transactions trying to commit
// Each must wait for uncertainty to pass
// With 7ms uncertainty and 1000 transactions/second:
// 1000 * 7ms = 7 seconds of total wait time
// Throughput limited by uncertainty bound

// Solution: Spanner uses "timestamp assurance" groups
// Multiple related keys get the same timestamp
// Reduces per-transaction wait overhead

Not a General-Purpose Solution

TrueTime’s guarantees only apply within Spanner’s controlled environment. You cannot implement TrueTime in a distributed system with heterogeneous nodes using NTP.

Uncertainty Growth During Outages

If TrueTime masters go down, uncertainty grows. Spanner has a limit (typically 10 minutes) before it must switch to a fallback mode:

// If TrueTime sync is lost:
async function handleUncertainTime() {
  const uncertainty = trueTime.getUncertainty();

  if (uncertainty > MAX_SAFE_UNCERTAINTY) {
    // Switch to "non-blocking" mode
    // Writes proceed without commit-wait
    // External consistency guarantee is weakened
    return { mode: "non-blocking", uncertainty };
  }
}

---

Alternatives to TrueTime

Hybrid Logical Clocks (HLC)

HLCs provide a software-only alternative. They encode both physical time and logical time, achieving ordering guarantees without specialized hardware.

// HLC as implemented in CockroachDB
class HybridLogicalClock {
  constructor() {
    this.time = 0;
    this.logical = 0;
  }

  now() {
    const wallTime = Date.now();

    if (wallTime > this.time) {
      this.time = wallTime;
      this.logical = 0;
    } else {
      this.logical++;
    }

    return { time: this.time, logical: this.logical };
  }
}

CockroachDB and YugabyteDB use HLCs for distributed timestamp assignment. The trade-off: weaker guarantees than TrueTime, but no special hardware required.

Coordinated Consensus

Some systems avoid clock issues entirely by using coordinated consensus for all transactions:

// Traditional approach: use a coordinator
async function coordinatedCommit(transaction) {
  // Coordinator assigns timestamp
  const timestamp = await coordinator.assignTimestamp();

  // All participants agree on commit
  await consensus.prepare(timestamp);

  // Commit
  await transaction.commitAt(timestamp);
}

This approach (used by some older distributed databases) eliminates clock issues but creates a single point of coordination and latency bottleneck.

---

When to Use Spanner vs Alternatives

Factor	Spanner	CockroachDB (HLC)	Cassandra (LWW)
Consistency	External (strongest)	Sequential	Eventual (weakest)
Latency	10-30ms cross-region	5-15ms cross-region	1-5ms
Hardware	Custom (GPS/atomic)	None special	None special
Scalability	Excellent	Excellent	Excellent
Cost	Very high	Moderate	Low
SQL Support	Full	Full	Limited

Choose Spanner When

• You need the strongest consistency guarantees
• You can afford the latency and cost
• You operate globally and need external consistency
• You need full SQL with ACID transactions at global scale

Choose HLC-based Systems (CockroachDB, YugabyteDB) When

• You want strong consistency without specialized hardware
• You can tolerate slightly weaker guarantees than TrueTime
• You want to avoid per-write latency overhead

Choose Eventual Consistency (Cassandra, DynamoDB) When

• You prioritize latency above all
• You can tolerate conflicting updates with last-write-wins
• You do not need ACID transactions

---

Conclusion

TrueTime represents a fundamental advance in distributed systems: instead of hoping clocks are synchronized, you know exactly how uncertain they are. This certainty enables external consistency guarantees that were previously impossible.

The cost is real: specialized hardware, commit-wait latency, and operational complexity. But for applications that need globally consistent transactions across continents, TrueTime provides guarantees no other approach can match.

For most applications, HLC-based systems like CockroachDB provide a practical middle ground. They give strong consistency without TrueTime’s hardware requirements.

The Physical Clocks, Logical Clocks, and Clock Skew Issues posts cover the foundations that lead to TrueTime’s design.

---

Quick Recap

• TrueTime provides bounded clock uncertainty, not just “probably synchronized”
• The interval [earliest, latest] guarantees where true time lies
• Spanner uses commit-wait to achieve external consistency
• The cost is per-write latency and specialized hardware requirements
• HLC-based systems offer a software-only alternative with weaker guarantees

Key Takeaways

1. TrueTime’s power comes from knowing uncertainty bounds, not minimizing them
2. Commit-wait guarantees that earlier timestamps always mean earlier real time
3. TrueTime requires GPS + atomic clocks in each datacenter
4. Spanner’s cross-region latency includes uncertainty wait + consensus + network
5. HLCs provide a practical software-only alternative for most use cases

For more on distributed databases, see CAP Theorem, Consistency Models, and Geo-Distribution.