Setting Up MongoDB Sharding for Horizontal Scaling

MongoDB sharding distributes data across multiple servers (shards), enabling horizontal scaling for databases that exceed the capacity of a single machine. This guide covers planning, deploying, and managing a sharded MongoDB cluster for production workloads.

Sharding Architecture

A sharded MongoDB cluster has three component types:

• Shards — each shard is a replica set holding a subset of the data
• Config servers — a replica set storing cluster metadata and chunk distribution
• mongos routers — query routers that direct operations to the correct shards

Applications connect to mongos routers, which transparently route queries to the appropriate shards based on the shard key.

Choosing a Shard Key

The shard key determines how data is distributed across shards. A poor shard key choice can lead to unbalanced data distribution (hotspots) and cannot be easily changed. Consider these factors:

• High cardinality — many distinct values enable fine-grained distribution
• Low frequency — no single value should dominate (avoid status fields)
• Non-monotonic — monotonically increasing keys (like ObjectId) cause all writes to go to one shard
• Query isolation — queries that include the shard key can be routed to a single shard

// Good shard keys
{ tenant_id: 1 }                          // Multi-tenant SaaS
{ tenant_id: 1, created_at: 1 }           // Compound key
{ _id: "hashed" }                         // Hashed for even distribution

// Poor shard keys
{ created_at: 1 }                         // Monotonic — creates hotspot
{ status: 1 }                             // Low cardinality
{ country: 1 }                            // Uneven distribution

Deploying the Config Server Replica Set

# mongod.conf for config servers
sharding:
  clusterRole: configsvr
replication:
  replSetName: configRS
net:
  port: 27019
  bindIp: 0.0.0.0
storage:
  dbPath: /data/configdb
security:
  keyFile: /etc/mongod-keyfile

# Start config servers and initiate replica set
mongosh --port 27019

rs.initiate({
  _id: "configRS",
  configsvr: true,
  members: [
    { _id: 0, host: "cfg1:27019" },
    { _id: 1, host: "cfg2:27019" },
    { _id: 2, host: "cfg3:27019" }
  ]
})

Deploying Shard Replica Sets

# mongod.conf for shard members
sharding:
  clusterRole: shardsvr
replication:
  replSetName: shard1RS
net:
  port: 27018
  bindIp: 0.0.0.0
storage:
  dbPath: /data/shard1
security:
  keyFile: /etc/mongod-keyfile

# Initialize each shard replica set
mongosh --port 27018

rs.initiate({
  _id: "shard1RS",
  members: [
    { _id: 0, host: "shard1a:27018" },
    { _id: 1, host: "shard1b:27018" },
    { _id: 2, host: "shard1c:27018" }
  ]
})

// Repeat for shard2RS, shard3RS, etc.

Deploying mongos Router

# mongos.conf
sharding:
  configDB: configRS/cfg1:27019,cfg2:27019,cfg3:27019
net:
  port: 27017
  bindIp: 0.0.0.0
security:
  keyFile: /etc/mongod-keyfile

# Start mongos
mongos --config /etc/mongos.conf

# Connect and add shards
mongosh --port 27017

sh.addShard("shard1RS/shard1a:27018,shard1b:27018,shard1c:27018")
sh.addShard("shard2RS/shard2a:27018,shard2b:27018,shard2c:27018")

Enable Sharding on a Collection

// Enable sharding for the database
sh.enableSharding("myapp")

// Shard the orders collection with a compound key
sh.shardCollection("myapp.orders", { tenant_id: 1, order_date: 1 })

// Or use hashed sharding for even distribution
sh.shardCollection("myapp.events", { _id: "hashed" })

Managing Chunks and Balancing

// Check shard distribution
db.orders.getShardDistribution()

// View chunk distribution
use config
db.chunks.aggregate([
  { $group: { _id: "$shard", count: { $sum: 1 } } }
])

// Check balancer status
sh.getBalancerState()
sh.isBalancerRunning()

// Set balancer window (to avoid peak hours)
db.settings.updateOne(
  { _id: "balancer" },
  { $set: {
    activeWindow: { start: "02:00", stop: "06:00" }
  }},
  { upsert: true }
)

Monitoring a Sharded Cluster

// Cluster overview
sh.status()

// Check for jumbo chunks (cannot be moved/split)
use config
db.chunks.find({ jumbo: true })

// Monitor operation routing
db.orders.find({ tenant_id: 42 }).explain("executionStats")
// Look for "SINGLE_SHARD" (good) vs "SHARD_MERGE" (scatter-gather)

Adding and Removing Shards

// Add a new shard
sh.addShard("shard3RS/shard3a:27018,shard3b:27018,shard3c:27018")
// The balancer will automatically migrate chunks to the new shard

// Remove a shard (drains data first)
db.adminCommand({ removeShard: "shard2RS" })
// Monitor drain progress
db.adminCommand({ removeShard: "shard2RS" })  // Run again to check status

Production Best Practices

• Deploy at least 2 mongos routers behind a load balancer for high availability
• Use 3-member replica sets for each shard and config servers
• Choose your shard key very carefully — it cannot be changed after sharding (you would need to recreate the collection)
• Avoid scatter-gather queries by always including the shard key in query filters
• Monitor chunk distribution and balancer activity regularly
• Set a balancer window to avoid migrations during peak traffic
• Plan for at least 3 shards to avoid rebalancing overhead with just 2
• Use zones (tag-aware sharding) for data locality requirements like GDPR compliance