NoSQL Databases: Document, Key-Value, Column-Family, and Graph Databases

Introduction

NoSQL is a broad label for data stores built around models such as documents, key-value pairs, column families, and graphs. These systems can fit workloads where access patterns, scale, or data shape make a relational design awkward, but each model brings different query and consistency trade-offs.

This guide compares common NoSQL types and the problems they solve, including partitioning, consistency choices, and operational risks. It also explains when a relational database remains the simpler and stronger fit.

Topic-Specific Deep Dives

Sharding Fundamentals

Sharding distributes data across multiple nodes so no single machine handles everything. Write throughput and storage grow as you add nodes, which sounds simple because it is. The catch is that your application now needs to figure out which node has the data it wants, and queries that span multiple shards get expensive fast. You trade operational complexity for horizontal scalability.

The distribution strategies each make different tradeoffs. Hash-based sharding spreads writes evenly across nodes, but range queries scatter across every shard. Range-based sharding keeps related data together for efficient range scans, but you can end up with hot spots if your access patterns match your partition key. Consistent hashing reduces data movement when nodes join or leave, but the implementation is more involved. Picking a shard key is one of those decisions that is hard to change later, so it pays to think about access patterns before you commit.

The next few sections break down each strategy, starting with hash-based approaches and how most databases handle those automatically, then moving into range-based sharding and where time-series workloads fit.

Sharding Patterns Deep Dive

Sharding strategies differ more than the marketing suggests. Here is how they actually work.

Hash-Based Sharding

The basic formula: shard_key = hash(primary_key) % num_shards. This distributes data evenly and avoids hotspots, but range queries scatter across every shard — expensive at scale.

# Application-level hash sharding
def get_shard(key, num_shards=16):
    return abs(hash(key)) % num_shards

shards = [connect(f"shard_{i}.db") for i in range(num_shards)]
key = "user:12345"
shard = shards[get_shard(key)]

MongoDB and Cassandra handle hash-based sharding automatically. MongoDB uses a variant called “consistent hashing” with chunk-based reassignment. Cassandra uses the partitioner’s hash output to assign tokens to nodes.

Consistent Hashing

The idea is to minimize shuffling when nodes join or leave. Instead of modulo on the hash, data goes to the first node clockwise on the ring.

graph LR
    subgraph "Hash Ring"
        H1["hash(A)"];
        H2["hash(B)"];
        H3["hash(C)"];
        H4["hash(D)"];
    end
    A["Node A<br/>Range: H1→H2"] --> B["Node B<br/>Range: H2→H3"];
    B --> C["Node C<br/>Range: H3→H4"];
    C --> D["Node D<br/>Range: H4→H1"];
    D --> A;

When a node fails, only its range moves to the next node. Adding a node takes ranges from neighbors rather than rehashing everything.

Advanced Sharding Strategies

Range-Based Sharding

Partition by key ranges instead of hash. This makes range queries within a partition efficient, but you risk hotspots if your access patterns correlate with the shard key.

Cassandra excels with range-based sharding using partition keys. A time-series table might shard by month, enabling efficient queries for a given time window:

-- Cassandra: range sharding for time-series
CREATE TABLE sensor_readings (
    device_id uuid,
    month text,  -- '2026-03'
    reading_time timestamp,
    temperature float,
    PRIMARY KEY ((device_id, month), reading_time)
);

Application-Level Sharding

Some systems push sharding entirely to the application. Redis Cluster and memcached use client-side routing — the application figures out which node owns a key and talks to it directly.

# Application-level sharding with Redis Cluster-like routing
class ShardedRedis:
    def __init__(self, nodes):
        self.nodes = nodes
        self.slot_map = self._build_slot_map(16384, nodes)

    def _slot_for_key(self, key):
        return crc16(key) % 16384

    def get(self, key):
        slot = self._slot_for_key(key)
        node = self._node_for_slot(slot)
        return node.get(key)

Sharding Anti-Patterns

Hot partition keys: If you shard by something low-cardinality — timestamps, status codes, a popular entity — one partition absorbs all the traffic. A celebrity user’s ID as partition key will saturate a single shard even in a 100-node cluster.

Unbounded partitions: Time-based or counter-based keys without bucketing grow without limit. In Cassandra, a single conversation with no end date eventually exceeds memory limits. Always bucket your time series.

Cross-partition transactions: When an operation touches multiple shards, you need distributed coordination — slower, more failure-prone. Design queries to keep related data on the same partition.

Trade-off Analysis

When Relational Still Wins

NoSQL is not always the answer — and honestly, most of the time it probably is not.

If your data fits a tabular model with clear relationships, relational databases offer more features and better tooling. ACID transactions across multiple tables are hard to match. SQL provides powerful querying for analytical workloads.

Operational simplicity matters. Most developers know SQL. Most teams can debug a MySQL or PostgreSQL issue quickly. The ecosystem around relational databases is mature.

NoSQL databases often require more operational expertise. You need to understand the CAP trade-offs, figure out capacity planning for sharding, and learn operational procedures specific to whichever system you chose.

Start with relational unless you have specific reasons not to. The flexibility argument for NoSQL is often overstated.

When to Use / When Not to Use

When to Use NoSQL Databases:

• Document databases: Your data is document-structured, varies in schema, or benefits from embedding related data
• Key-value stores: You need simple lookups at extreme speed for caching or sessions
• Column-family databases: You have write-heavy workloads with predictable query patterns
• Graph databases: Relationship traversal is the core of your workload
• You need horizontal write scaling beyond what relational databases offer
• Your access patterns are simple and predictable at scale

When Not to Use NoSQL Databases:

• You need ACID transactions across multiple documents or entities
• Your data has complex relationships requiring multi-table joins
• You need ad-hoc querying capabilities or powerful aggregation
• Your team lacks operational expertise for the specific NoSQL system
• You are using the “flexibility” argument without specific requirements

Production Failure Scenarios

Actual production incidents teach you more than any architecture diagram. Here is how real systems break.

DynamoDB Thundering Herd

DynamoDB adaptive capacity handles uneven access patterns, but only up to a point. If a popular item suddenly goes viral — a flash sale, viral post, or celebrity endorsement — the partition serving that item receives 100x normal traffic. Adaptive capacity splits that partition, but splitting takes time. During the split window, requests throttle. DynamoDB returns ProvisionedThroughputExceededException. Your application retries with exponential backoff, which queues more requests, which makes the overload worse.

For thundering herd, scatter-gather helps: cache popular items at the application layer, distribute reads across multiple keys that all map to the same underlying item, or add a random suffix to partition keys to spread load. Do not rely on DynamoDB adaptive capacity as your primary strategy — it is a backstop.

What this looks like in practice: A product page goes viral and receives 50,000 requests per second against a single partition key. DynamoDB begins splitting the partition, but the split takes 30-60 seconds to complete. During that window, every request to that partition key returns ProvisionedThroughputExceededException. Your application retries with exponential backoff starting at 100ms, doubling each retry. By the fifth retry, you are looking at 100ms × 2^5 = 3,200ms of queued delays. The retry queue grows faster than DynamoDB can drain it. The viral content eventually cools off before the split finishes, so the overload is self-limiting, but users experience degraded latency the entire time.

How to detect it early: Watch ConsumedCapacityUnits and ThrottledRequests per partition in CloudWatch. Set an alert when any single partition exceeds 80% of its consumed capacity for more than 30 seconds. Once you hit 100%, you are already in the throttling window. You can also monitor GetItemThrottledRequests and QueryScannedCount per partition to identify which partition key is the culprit before it cascades.

Architectural fixes: The real fix is redesigning your access patterns so popular items do not map to a single partition key. Add a random suffix, for example product:12345:A or product:12345:B, to spread reads across 10-100 partition keys. Your application handles the fan-out, but each partition now receives 1/10th to 1/100th of the load. Alternatively, cache aggressively at the application layer or API gateway. Serve the viral item from CloudFront, Cloudflare, or an in-memory cache for 30-60 seconds, and DynamoDB never sees the spike. For flash sales with known start times, pre-warm the cache before the sale begins.

Cassandra Compaction Storms

Cassandra compacts SSTables periodically to remove tombstones and overwrite dead data. Under heavy write workloads, compactions can fall behind. When compaction finally runs, it reads and rewrites large amounts of data, consuming disk I/O and CPU. This creates a feedback loop: writes accumulate faster, compaction falls further behind, and read performance degrades because the database is scanning past dead data.

The usual fixes: size your cluster for 50% headroom above peak write throughput. Use Size-Tiered Compaction Strategy (STCS) for read-heavy workloads or Time Window Compaction Strategy (TWCS) for time-series. Watch compaction queue depth and alert before it grows unbounded. When a compaction storm hits, reduce write throughput temporarily or add nodes to spread the I/O load.

What triggers compaction storms: Compaction kicks in when the number of SSTables on disk crosses a threshold. With STCS, that means 4 or more SSTables of similar size. Under heavy write load, Cassandra can produce SSTables faster than compaction can drain them. The tipping point sneaks up on you: if your write throughput stays above what compaction can handle, the queue grows until reads start scanning past dead data on every query. TWCS avoids this by only compacting within time windows, so SSTable counts stay bounded for time-series data.

How to detect it early: Watch CompactionTaskCount and PendingCompactions in nodetool cfstats. A growing pending compactions count is the first warning. DiskSpaceUsed anomalies help too — a sudden jump means compaction has started rewriting large amounts of data. Alert when pending compactions exceeds 10 for any table under write load. During the storm itself, nodetool tpstats shows the compaction thread pool queue depth growing without bound.

Why reducing writes helps: The fastest way to break the feedback loop is to cut write throughput by 30-50%. This gives compaction room to catch up. You can do this at the application layer (batch writes), at the Cassandra level (reduce concurrent_writes), or by adding nodes to spread the I/O across more disks. Adding nodes mid-storm is slower than cutting writes — the new node needs to bootstrap before it contributes.

MongoDB Shard Imbalance After Chunk Migration

MongoDB’s chunk balancer moves data between shards to maintain even distribution. During a migration, the source shard has reduced capacity for regular operations. If the balancer migrates too aggressively during peak traffic, client queries time out because the source shard is overloaded. After migration completes, the balancer may decide the distribution is still uneven and trigger another migration immediately.

Use sh.balancerWindow to schedule chunk migrations during off-peak windows. Set concurrentMergeThreads to limit compaction impact. Monitor mongos latency and shardChunkDistribution to catch imbalanced distributions before they degrade user-facing latency.

Why chunk migrations cause latency spikes: During a chunk migration, MongoDB streams documents from source to destination shard, then deletes from source. The source shard handles both regular traffic and the migration drain at the same time. With default settings, MongoDB allows up to 2 concurrent migrations per shard. Under load, each migration soaks up roughly 10-20 MB/s of network bandwidth between shards. If your shards are already near capacity, migration bandwidth fights with client query bandwidth for the same pipe, and source shard latency climbs.

How to detect an imbalanced cluster before it degrades users: Run sh.status() in the MongoDB shell to see chunk distribution across shards. Any shard with more than 30% of total chunks is the threshold worth investigating. The balancer logs its decisions in the config server’s log with entries like balance is already even, skipping. Repeated balancing rounds with no net improvement point to something blocking proper distribution: a missing index on the shard key, very large documents, or a hot spot that keeps chunking on one shard.

The migration latency window: A single 64 MB chunk migration takes 30-60 seconds on a healthy cluster with enough network bandwidth. The cutover from source to destination is atomic, but the final delete from source runs after the cutover, so there is a brief window where the same data lives on both shards. Under normal load this is invisible. Under high concurrency, a query landing on the destination during this window may not find the data yet.

The immediate fix: If you are already in a storm, sh.stopBalancer() halts all migrations immediately. It does not roll back in-progress migrations, but it stops new ones from starting. Then cut application load to give the overloaded shard breathing room. Once things stabilize, db.currentOp() shows long-running migrations and db.adminCommand({_flushChunkForSharding: ...}) lets you manually force distribution if needed.

Redis Dataset Eviction Under Memory Pressure

Redis is memory-resident by design. When Redis approaches its maxmemory limit, it evicts keys using the configured policy (LRU, LFU, TTL, or random). The application suddenly starts missing cache entries it expected to exist. If the application assumes data is always in Redis — not designed to handle cache misses — every miss hits the database, potentially causing a thundering herd on the backend.

Set maxmemory to 70-80% of available RAM to leave headroom for Redis internal operations. Watch mem_fragmentation_ratio — a value above 1.5 indicates memory fragmentation eating into available RAM. Build your application to handle cache misses gracefully with database fallbacks, not as a hard dependency.

Why eviction feels sudden: Redis does not evict until it actually hits maxmemory. But the tipping point is abrupt because the database uses memory in two places: user data and the allocator’s internal overhead. On a system with 100 GB RAM and maxmemory set to 80 GB, you might sit at 78 GB for weeks, then a batch job or traffic spike pushes you to 82 GB in minutes. When Redis crosses the limit, it evicts keys aggressively on every write — sometimes thousands per second — until usage drops below the threshold. LRU and LFU policies scan the entire keyspace to find the best eviction candidate, and that full scan itself causes a brief latency spike on the way down.

The four eviction policies and when each applies: noeviction rejects writes and is rarely useful in production. allkeys-lru is the most common choice for cache workloads — it removes the least recently used keys regardless of TTL. allkeys-lfu removes the least frequently used keys, which works better when you have popular items that should stay and unpopular items that should go. volatile-lru and volatile-lfu restrict eviction to keys with a TTL set, useful if you want to preserve keys without expiration. allkeys-random and volatile-random are for uniform sampling without tracking access patterns. The wrong policy for your workload can cause unnecessary cache misses — LRU when your hits are on recently written data makes sense; LFU when access frequencies follow a power law distribution is better.

Early warning signs: Watch evicted_keys and expired_keys in the Redis INFO output. Any eviction above zero means you are operating at or near the memory limit. Alert when used_memory exceeds 75% of maxmemory. Also watch mem_fragmentation_ratio — above 1.5 means your allocator has fragmented that 80 GB reservation into something consuming 120 GB of physical RAM, so your actual available headroom is much smaller than configured. MEMORY DEFRAG helps temporarily, but restarting Redis before fragmentation gets extreme is the reliable fix.

Network Partitions and Split-Brain Scenarios

When a network partition divides a Cassandra cluster, nodes in one partition can still talk to each other but not to the other partition. If your consistency level is ONE or LOCAL_ONE, both sides accept writes independently. When the partition heals, you have divergent data with no automatic way to determine which writes should win. This is not hypothetical — it has caused data loss in production systems that believed their configured consistency level was sufficient.

Use QUORUM consistency for critical data. Test network partition scenarios explicitly in staging. Run nodetool repair regularly to reconcile divergent data. For truly critical data, accept that NoSQL consistency guarantees have limits and design your application accordingly.

Common Pitfalls / Anti-Patterns

1. Using NoSQL without clear access patterns: NoSQL requires understanding your query patterns upfront. Without this, you either over-engineer or end up with hot spots and inefficient access.

2. Ignoring eventual consistency implications: Reading from replicas before writes propagate means you might get stale data. Consider your consistency requirements per operation.

3. Unbounded partition key growth: Time-based or counter-based partition keys create unbounded partitions that will haunt you. Use natural keys with fixed cardinality.

4. Over-reliance on secondary indexes: Secondary indexes often cause full cluster scans. Design your primary access patterns around partition keys instead.

5. Ignoring data modeling for access patterns: In relational DBs you model data. In NoSQL you model queries. Design tables for your query patterns, not your entity structure.

6. Skipping backup and recovery testing: Many NoSQL systems have complex backup mechanisms. Test recovery procedures regularly or you will lose data eventually.

7. Underestimating operational complexity: Each NoSQL system has its own operational quirks. Make sure your team has training and runbooks for your specific system.

8. Using wide partitions for “flexibility”: Stuffing too much data into single partitions kills performance. Keep partition size bounded.

9. Assuming linear scalability: Adding nodes does not instantly double your capacity. Rebalancing takes time and causes temporary load spikes you did not plan for.

Real-World Case Studies

Netflix runs one of the largest Cassandra deployments in the world. At peak they handle over 14 million concurrent streams, each backed by metadata queries to their Cassandra clusters. Their architecture uses Cassandra for its linear scalability — adding nodes directly increases throughput without redesigning queries or rebalancing clusters.

Their specific pattern: they partition data by user ID, so each user’s viewing history, preferences, and recommendations live on a small set of partitions. This gives them predictable latency per user and avoids the scatter-gather problem that kills performance when one query touches half the cluster. The lesson is that Cassandra rewards thoughtful partition design upfront. A hot partition — say, a popular show — can saturate a single node even in a 100-node cluster. Netflix mitigates this with virtual nodes and by spreading popular content across many partition keys.

Amazon DynamoDB was designed with a different constraint: partition the problem or lose availability. Their 2012 paper described how they partition by key range, with each partition responsible for a slice of the key space. If a partition receives more traffic than it can handle, DynamoDB splits it automatically. This adaptive splitting is what makes DynamoDB feel infinite — you never hit a ceiling, the service redistributes before you notice.

The tradeoff is that DynamoDB requires upfront access pattern planning. In DynamoDB you define your primary key structure and secondary indexes, and queries are efficient only within those structures. Ad-hoc queries that would be trivial in PostgreSQL require either scanning the entire table (expensive) or redesigning your indexes. This is the right tradeoff for Amazon’s use case — their services have well-defined access patterns. It is the wrong tradeoff for a startup that does not yet know how users will query their data.

Replication Topologies

How you replicate across regions determines what happens when partitions occur, how consistent your data is, and how much latency your writes tolerate.

Multi-Region Replication Topologies

Single-Leader Replication

One region is primary — all writes go there and replicate asynchronously to secondaries. Reads can come from any replica.

graph LR
    Primary["Primary Region<br/>US-East"] --> Replica1["Replica<br/>US-West"];
    Primary --> Replica2["Replica<br/>EU-West"];
    Client1["Client<br/>US-East"] --> Primary;
    Client2["Client<br/>US-West"] --> Replica1;

Simple model, strong write consistency, reads are fast locally. The downside: writes from other regions are slow since everything goes to the primary, and failover requires manual intervention. MongoDB replica sets and PostgreSQL streaming replication use this by default.

Multi-Leader Replication

Any region can accept writes and propagates them to all others. Write latency drops significantly for distributed users, but you now have conflict resolution problems — the same write can succeed differently in each region.

graph LR
    L1["Leader 1<br/>US-East"] <--> L2["Leader 2<br/>US-West"];
    L2 <--> L3["Leader 3<br/>EU-West"];
    L1 <--> L3;

CouchDB and Cassandra both support multi-datacenter replication. DynamoDB Global Tables use multi-leader replication.

Leaderless Replication

No primary at all — any replica accepts reads and writes. Quorum-based coordination keeps things consistent (or at least tunable).

graph LR
    Client --> R1["Replica 1<br/>US-East"];
    Client --> R2["Replica 2<br/>US-West"];
    Client --> R3["Replica 3<br/>EU-West"];
    R1 <--> R2;
    R2 <--> R3;
    R3 <--> R1;

No single point of failure, which sounds great until you try to tune quorum without hurting either latency or consistency. DynamoDB and Cassandra both use variants of this.

Consistency and Failover

Consistency vs Latency Trade-offs

Cassandra’s CONSISTENCY ONE returns immediately after local write. CONSISTENCY QUORUM waits for replicas across datacenters — adding cross-region latency.

Conflict Resolution Strategies

When writes can happen in multiple places, something has to decide which one wins:

Last-writer-wins (LWW): DynamoDB and Cassandra use timestamps — simple, but you can lose updates. Application-defined: Your code decides based on business logic — take the larger counter value, keep the more recent order, whatever makes sense for your domain. CRDTs: Data structures designed to merge without conflicts — sets, counters, registers that converge automatically regardless of order. Manual resolution: Flag conflicts for a human to sort out, which does not scale.

# Example: LWW conflict resolution
def resolve_conflict(local_value, remote_value, local_ts, remote_ts):
    return remote_value if remote_ts > local_ts else local_value

Cross-Region Failover Considerations

When an entire region disappears, what happens next depends on your topology:

Single-leader: Promote a replica in another region. Update DNS to point at the new primary. Multi-leader: Other regions keep accepting writes. When the failed region comes back, conflict resolution merges whatever diverged. Leaderless: Everything keeps working — reads and writes go to available regions. The failed region rejoins via hinted handoff or repair when it recovers.

Consistency Models and Data Modeling

Eventual Consistency vs Strong Consistency

NoSQL systems expose consistency as a dial, not a binary switch. The choice you make per operation determines latency, availability, and whether you read stale data.

flowchart TD
    subgraph "Consistency Spectrum"
        E["Eventual Consistency<br/>Writes propagate async<br/>Reads may return stale data"]
        S["Strong Consistency<br/>Writes sync to quorum<br/>Reads return latest write"]
    end
    E -->|Quorum reads/writes| S

Eventual consistency means reads may not immediately reflect the latest write. The window of inconsistency is typically milliseconds. DynamoDB, Cassandra, and CouchDB default to eventual consistency.

Strong consistency guarantees reads see all acknowledged writes. MongoDB replica sets with majority read concern, and Cassandra with quorum consistency, provide strong consistency.

The CAP theorem: during a partition, you choose between consistency (CP systems) or availability (AP systems). Most NoSQL systems are AP by default, tunable to CP via consistency levels.

The consistency spectrum in practice: Most NoSQL databases do not actually offer just two modes. Cassandra has six consistency levels (ONE, LOCAL_ONE, QUORUM, LOCAL_QUORUM, EACH_QUORUM, ALL). DynamoDB has three (Strong, Conditional, Eventual). MongoDB has five read concern levels. Each level sits somewhere on the latency-consistency curve. Writing with CONSISTENCY ONE in Cassandra is effectively eventual — the write succeeds as soon as the local node acknowledges it. Writing with CONSISTENCY QUORUM waits for a majority of replicas, adding cross-region latency if your cluster spans datacenters. Most reads can tolerate eventual consistency; critical writes like charging a credit card or decrementing inventory usually need quorum.

Stale reads in eventual consistency: With eventual consistency, a read can return data older than what you just wrote. Here is the scenario: you write key K with value “processed” to node N1. Before N1 replicates to N2, a client reads from N2 and gets the old value “pending”. The stale window is your replication lag — milliseconds normally, seconds during network congestion or node failures. Applications that read-then-write without re-reading after the write end up with read-before-write bugs. The fix is to read from a quorum of replicas, or to use a read repair mechanism that validates freshness on read.

Tunable consistency in DynamoDB: DynamoDB offers three consistency options per read. Eventual reads go to whichever replica responds first — fastest but potentially stale. Strong reads use a quorum-like mechanism that waits for writes to propagate before reading — adds roughly 10-20ms latency. Consistent reads in DynamoDB are per-item and only available when you specify a partition key and sort key together. The strong read path requires coordination across partitions, which is why DynamoDB does not default to it. For most use cases, eventual reads are fine. For financial or inventory operations, strong reads are worth the latency cost.

NoSQL Data Modeling Patterns

Materialized Path Pattern

Store the full path to enable efficient tree traversal.

# MongoDB: Materialized Path
{
  "_id": "electronics/tvs/large-tvs",
  "name": "65 inch TV",
  "path": "/electronics/tvs/large-tvs/",
  "ancestors": ["electronics", "electronics/tvs"]
}

# Query all descendants of electronics
db.categories.find({ path: /^electronics\// })

Tree Structure with Nested Sets

Alternative for read-heavy tree operations.

// Nested set model - efficient subtree reads
{
  "_id": "electronics",
  "left": 1,
  "right": 14
}
{
  "_id": "tvs",
  "left": 2,
  "right": 13
}

Tradeoff: updates require rebalancing the entire tree. Use materialized path for write-heavy workloads.

Document Versioning Pattern

Track change history without schema migrations.

// Current version
{ "_id": "doc-123", "version": 3, "data": {...} }

// Version history (separate collection)
{ "doc_id": "doc-123", "version": 1, "data": {...}, "updated_at": ... }
{ "doc_id": "doc-123", "version": 2, "data": {...}, "updated_at": ... }

Quick Recap Checklist

• Choose a NoSQL model that fits the application’s access patterns and query needs.
• Design partition keys to distribute load and avoid hot partitions.
• Decide how much consistency and replication lag the application can tolerate.
• Plan sharding, capacity, and recovery around expected growth and failure scenarios.
• Monitor latency, throttling, replication, storage, and compaction; secure data and credentials.

Observability Checklist

Metrics to Monitor:

Track request latency (p50, p95, p99) by operation type and request throughput (reads/writes per second). Watch node availability and cluster health, disk usage per node and cluster-wide, and network I/O between nodes. Replication lag and pending writes matter for consistency. Compaction progress and queue depth affect performance unpredictably. If you use caching, track cache hit ratios. Connection pool utilization catches exhaustion before it becomes an outage.

Logs to Capture:

Log node failures and restarts, compaction events with I/O patterns, authentication failures and denied permissions. Slow query logs (set a threshold that makes sense for your workload) are essential. For JVM-based systems like Cassandra, GC pauses and heap pressure show up at the worst times. Network partition events are worth capturing for post-mortems.

Alerts to Set:

Alert on node down or cluster minority partition immediately. Set thresholds for latency spikes that exceed your SLA, disk usage above 80% on any node, and replication lag beyond your consistency requirements. Watch for failed request rate increases and compaction queue depth spiking. Connection pool exhaustion will take down your app before you notice anything else.

# Example: Cassandra nodetool commands for monitoring
# nodetool status           # Cluster health and node status
# nodetool tpstats          # Thread pool statistics
# nodetool cfstats          # Keyspace and table statistics
# nodetool proxyhistograms  # Client request latency

Security Checklist

• Enable authentication (internal authentication or external like Kerberos)
• Implement authorization with role-based access control
• Encrypt client-to-node and node-to-node traffic
• Encrypt data at rest (filesystem or application-level encryption)
• Use TLS for all inter-node communication
• Restrict network access to database nodes (firewall rules)
• Implement keyspace-level or table-level permissions
• Audit log sensitive operations and access patterns
• Sanitize all user inputs to prevent injection attacks
• Use secure secret management for database credentials
• Regularly rotate credentials and encryption keys
• Test security configuration with penetration testing

Interview Questions

Further Reading

Official Documentation

• MongoDB Documentation — Comprehensive guides on aggregation pipelines, sharding, and replication
• Cassandra Documentation — CQL reference, architecture docs, and operational guides
• DynamoDB Developer Guide — Access patterns, global tables, and performance tuning
• Redis Documentation — Data types, persistence, and cluster mode
• CockroachDB Documentation — Distributed SQL concepts and deployment
• TiDB Documentation — Architecture and MySQL compatibility

Books

• “Designing Data-Intensive Applications” by Martin Kleppmann — The definitive text on distributed systems and data storage trade-offs
• “Cassandra: The Definitive Guide” by Jeff Carpenter and Eben Hewitt — Practical Cassandra operations and modeling
• “MongoDB Applied Design Patterns” by Rick Copeland — Document modeling patterns for common use cases

Papers and References

• Dynamo: Amazon’s Highly Available Key-value Store — The foundational paper behind DynamoDB
• Bigtable: A Distributed Storage System for Structured Data — Google’s approach to column-family storage
• CAP Twelve Years Later: How the “Rules” Have Changed — Eric Brewer’s follow-up on CAP theorem nuances

Related Articles

• Relational databases compares SQL’s transaction and query model with NoSQL trade-offs.
• Database replication covers replica topologies, lag, and failover.
• Capacity planning explains how to estimate storage and throughput needs as workloads grow.
• Column-family databases explores wide-column modeling and Cassandra-style access patterns.
• Graph databases covers relationship-first models and graph traversal.
• Horizontal sharding goes deeper on shard keys, routing, and partition trade-offs.

Conclusion

Key takeaways:

• NoSQL databases are purpose-built for specific access patterns and scale requirements
• CAP theorem trade-offs force you to choose between consistency and availability during partitions
• Document databases work well for flexible schemas and embedded data patterns
• Key-value stores give you extreme speed for simple lookups
• Column-family databases handle write-heavy workloads with predictable queries
• Graph databases are built for relationship traversal workloads

Copy/Paste Checklist:

# MongoDB connection with monitoring
from pymongo import MongoClient
client = MongoClient("mongodb://user:pass@host:27017/?replicaSet=rs0")
db = client.admin
# Check replica set status
print(db.command('replSetGetStatus'))

# Cassandra connection with monitoring
from cassandra.cluster import Cluster
cluster = Cluster(['host1', 'host2', 'host3'])
session = cluster.connect('mykeyspace')
# Check cluster health
session.execute("SELECT * FROM system.local")
session.execute("SELECT * FROM system.peers")

# Redis security and monitoring
redis-cli INFO clients  # Client connections
redis-cli INFO stats     # Command statistics
redis-cli CONFIG GET *   # Configuration audit