Data & Databases
MongoDB and the Document Model: Schema Design, Aggregation and Sharding
Intermediate8 min readUpdated
MongoDB is not a relational database with the constraints removed; it rewards a different modelling instinct, where the unit of storage is the thing you read together. This guide covers the document model and BSON, schema design patterns, the aggregation pipeline, index selection, replica-set durability, shard-key choice and where multi-document transactions genuinely fit.
What you will learn
- Explain the document model, BSON types and the limits that shape a design
- Choose between embedding and referencing, and apply the extended reference and bucket patterns
- Write aggregation pipelines that filter early and shape data on the server
- Design indexes using the equality-sort-range rule and verify them with explain
- Reason about replica-set elections, read and write concerns, sharding and transactions
MongoDB is the database most often adopted for the wrong reason and abandoned for the right one. The wrong reason is "we do not want to define a schema"; the right reason to leave is discovering, two years later, that the data was relational all along. Used deliberately, though, the document model solves a real problem: when the thing you store and the thing you read are the same self-contained object, keeping it as one document removes joins, round trips and mapping code.
This guide covers MongoDB 8 and assumes you already know relational modelling. The habits that transfer are the ones about access patterns; the habits about normal forms mostly do not, and knowing which is which is the whole skill.
Documents, BSON and what they imply
A document is a set of field-value pairs stored in BSON, a binary encoding of JSON with extra types: ObjectId (a 12-byte identifier embedding a timestamp), Date, Decimal128 for exact money, Binary, 32- and 64-bit integers, and regular expressions. Documents live in collections, collections live in a database. Every document has a unique _id, indexed automatically.
{
"_id": { "$oid": "665f1a2b7c9e4f0012ab34cd" },
"sku": "BK-1042",
"title": "Field Guide to Estuaries",
"price": { "$numberDecimal": "24.90" },
"tags": ["nature", "reference"],
"dimensions": { "widthMm": 130, "heightMm": 198 },
"stock": [
{ "warehouse": "LYON", "quantity": 12 },
{ "warehouse": "PORTO", "quantity": 3 }
]
}
Three properties of that structure drive everything else. Fields can nest arbitrarily, so a whole aggregate fits in one record. Arrays are first-class and indexable, so a one-to-many relationship needs no second collection. And a single document is limited to 16 MB, which is a design constraint disguised as a technical limit: any array that can grow without bound is a modelling mistake, not a capacity problem.
A single-document write is atomic no matter how deeply it nests. That guarantee is the reason document modelling works: if the data that must change together lives in one document, you get transactional behaviour for free.
Documents are not schemaless in any useful sense — your code depends on fields being present and typed. Declare that dependency:
db.createCollection("product", {
validator: { $jsonSchema: {
bsonType: "object",
required: ["sku", "title", "price"],
properties: {
sku: { bsonType: "string", pattern: "^[A-Z]{2}-[0-9]{4}$" },
price: { bsonType: "decimal", description: "list price, excl. tax" }
}
}},
validationLevel: "strict"
});
Reading and writing
mongosh is the shell, and the driver APIs mirror it closely.
db.product.insertOne({ sku: "BK-1042", title: "Field Guide to Estuaries", price: NumberDecimal("24.90") });
db.product.find({ tags: "nature", "price": { $lt: NumberDecimal("30") } })
.sort({ title: 1 }).limit(20);
// Match a whole array element, not two independent conditions
db.product.find({ stock: { $elemMatch: { warehouse: "LYON", quantity: { $gt: 0 } } } });
db.product.updateOne(
{ sku: "BK-1042", "stock.warehouse": "LYON" },
{ $inc: { "stock.$.quantity": -1 }, $set: { updatedAt: new Date() } }
);
Two details catch newcomers. find({ "stock.warehouse": "LYON", "stock.quantity": { $gt: 0 } }) matches a document where some element has that warehouse and some element has stock — not necessarily the same one; $elemMatch is what you meant. And update operators ($set, $inc, $push, $pull, $addToSet) modify fields in place, whereas passing a bare document replaces it entirely, silently deleting everything you left out. When a nested update misbehaves, pasting the document into a JSON formatter to see its real shape is faster than squinting at shell output.
Modelling: embed or reference
The relational instinct is to normalise until each fact lives once. The document instinct is to store together what is read together, and to accept duplication where the duplicate is either immutable or cheap to refresh. Neither is universally right; the question is always what the application asks for.
Embed when the child has no independent life, is bounded in number, and is read with its parent — order lines inside an order, an address inside a customer, per-warehouse stock inside a product. One read, one atomic write, no join.
Reference when the child is large, unbounded, shared between parents, or accessed on its own — an author referenced by thousands of books, or an audit trail that grows forever.
Four patterns cover most of the middle ground:
- Extended reference. Store the reference plus the two or three fields you always display:
{ "authorId": ..., "authorName": "..." }. This removes the lookup from the read path. It is denormalisation, so decide up front how a name change propagates — often a background job, sometimes nothing at all because the value is historical. - Bucket. For high-volume time series, group many readings into one document per device per hour instead of one document per reading. Fewer documents, smaller index, dramatically less overhead. MongoDB's native time-series collections implement this internally.
- Computed. Store a running total or a count alongside the raw data and maintain it with
$incon write, rather than aggregating on every read. - Subset. Keep the ten most recent or most relevant array elements in the parent document and the full history in a separate collection, so the hot document stays small.
The anti-pattern to name explicitly is the unbounded array: comments embedded in a post, events embedded in a user. It grows past 16 MB, and long before that every update rewrites a document that keeps getting bigger. The relational reasoning about keys and dependencies covered in database design and normalization still helps here — you are choosing which dependencies to physically co-locate, not pretending they do not exist.
The aggregation pipeline
Aggregation is a sequence of stages, each transforming the stream of documents it receives. It is where MongoDB does the work that SQL does with GROUP BY, joins and window functions.
db.order.aggregate([
{ $match: { placedAt: { $gte: ISODate("2026-01-01") }, status: "SHIPPED" } },
{ $unwind: "$lines" },
{ $group: {
_id: "$lines.sku",
units: { $sum: "$lines.quantity" },
revenue: { $sum: { $multiply: ["$lines.quantity", "$lines.unitPrice"] } }
}},
{ $sort: { revenue: -1 } },
{ $limit: 10 },
{ $lookup: { from: "product", localField: "_id", foreignField: "sku", as: "product" } },
{ $project: { _id: 0, sku: "$_id", units: 1, revenue: 1, title: { $first: "$product.title" } } }
]);
The single most important rule is put $match first, and $sort immediately after if an index can serve both — those are the only stages that can use an index, and only while they remain at the head of the pipeline. Everything after $unwind or $group operates on materialised intermediate documents.
Other stages worth knowing: $facet runs several sub-pipelines over the same input, which is how you compute results and their aggregate counts in one round trip; $bucket and $bucketAuto build histograms; $merge and $out write results into a collection, turning a pipeline into a scheduled materialised view. $lookup performs a left outer join, and it works, but a pipeline that needs three of them is telling you the model is relational.
Indexes
Index rules rhyme with the relational ones from the execution plan guide, with MongoDB-specific twists:
- Compound indexes follow the ESR rule: Equality fields first, then Sort fields, then Range fields. Getting sort before range is what avoids an in-memory sort, and MongoDB aborts sorts exceeding 100 MB unless you allow disk use.
- Multikey indexes are what you get automatically when you index an array field; each element becomes an entry. You cannot compound two array fields in one index.
- Partial indexes (
partialFilterExpression) index only matching documents — ideal for a small "pending" subset of a large collection. Sparse is the older, blunter version. - TTL indexes delete documents a set time after a date field, which is how you expire sessions or logs without a purge job.
- Wildcard indexes cover unknown field names in user-defined attributes. Convenient, and much larger than a targeted index.
- Text and vector indexes support keyword and similarity search; for serious full-text relevance a dedicated engine is still better, as the Solr guide explains.
Verify with explain, reading it the same way you read a relational plan:
db.order.find({ status: "SHIPPED", placedAt: { $gte: ISODate("2026-01-01") } })
.sort({ placedAt: -1 })
.explain("executionStats");
Compare totalKeysExamined, totalDocsExamined and nReturned. Documents examined far exceeding documents returned means the index is filtering poorly; a COLLSCAN winning stage means it is not being used at all.
Replica sets, sharding and consistency
A replica set is a group of mongod processes holding the same data: one primary that accepts writes, and secondaries that replicate its oplog. If the primary becomes unreachable, the remaining members hold an election and a majority elects a new primary, typically within a few seconds. Always run an odd number of voting members across separate failure domains, or use an arbiter only as a last resort.
Durability and freshness are per-operation choices, not server-wide settings:
- Write concern
{ w: "majority" }acknowledges only once a majority has the write, making it durable across a failover.w: 1acknowledges at the primary and can be rolled back if that primary fails. Addj: trueto require the journal on disk. - Read concern
"local"returns whatever the node has;"majority"returns only data that cannot be rolled back;"snapshot"gives a consistent point-in-time view inside a transaction. - Read preference decides which member serves the read.
secondaryPreferredspreads load at the cost of possible staleness — a fine trade for reporting, a bad one for read-your-own-write flows.
Sharding partitions a collection across shards by a shard key. Each shard is itself a replica set; mongos routers direct queries, and config servers hold the metadata. A query that includes the shard key is routed to one shard; one that does not is broadcast to all, which is the difference between a millisecond and a cluster-wide fan-out.
Choose the shard key on three criteria: high cardinality, even write distribution, and presence in your common query filters. A timestamp fails the second test — all inserts land on the shard owning the highest range. Hashing the key spreads writes evenly but destroys range queries. In practice a compound key such as { tenantId: 1, orderId: "hashed" } often satisfies all three. Modern versions can refine a shard key and even reshard a live collection, but that is a heavy operation; get it approximately right first. And do not shard prematurely — a well-indexed replica set on decent hardware handles far more than most teams assume, and sharding adds real operational weight, a theme the big data architecture guide returns to.
Transactions span multiple documents and collections with snapshot isolation, on replica sets and sharded clusters alike. They are the right tool for a transfer between two accounts held in two documents. They are the wrong tool for something a better model would have made a single-document update, and they carry a real cost in latency and lock contention.
Key takeaways
- A document is atomic on write, so store together what changes together; the 16 MB limit is a modelling constraint that forbids unbounded arrays.
- Embed bounded, dependent, co-read data; reference shared or unbounded data; use extended reference, bucket, computed and subset patterns for the cases in between.
- Aggregation pipelines must start with
$match(and a well-placed$sort) to use an index;$facet,$mergeand$bucketcover reporting needs without extra round trips. - Build compound indexes in equality-sort-range order and confirm with
explain("executionStats")that documents examined stays close to documents returned. - Write concern, read concern and read preference are per-operation decisions;
w: "majority"is what makes a write survive a failover. - Shard only when you must, on a key with high cardinality, even write spread and presence in your common filters — and validate the schema you already depend on with
$jsonSchema.
Sources and further reading
Frequently asked questions
- Does MongoDB really have no schema?
- It has no schema enforced at write time by default, which is not the same as having none. Your application code, your indexes and every query you write encode a schema whether you declare it or not. Collections support JSON Schema validation rules, and using them for the fields you depend on turns silent corruption into an immediate error, which is usually what you want in production.
- Are MongoDB transactions safe to use in normal code?
- Yes, since version 4.0 on replica sets and 4.2 on sharded clusters, and they provide snapshot isolation with all-or-nothing commit. They are also more expensive than a single-document write, which is already atomic. Treat them as the exception for genuinely cross-document operations rather than the default wrapper, and design so that most updates touch one document.
- How do I choose a shard key?
- You need high cardinality, even write distribution and, ideally, alignment with your most common query filter so reads target one shard. A monotonically increasing key such as a timestamp sends every insert to the same shard; hashing it fixes distribution but breaks range queries. A compound key combining a tenant or category with a hashed component is often the best compromise.
- When should I use MongoDB instead of PostgreSQL?
- When your data genuinely arrives as self-contained, variably shaped documents that you read as a whole, and when horizontal write scaling matters more than joins across many entities. If your access patterns are relational — many-to-many links, ad-hoc joins, strong reporting needs — a relational engine with a jsonb column usually serves you better.