Choosing by evidence
Lesson 6 of 6
6Choosing by evidence
205 times faster to read, at the price of 12 stale documents from one rename. This part measures the trade-off instead of philosophising about it, then hands you a five-question decision frame.
What the picture shows: Start from the most frequent question, measure it on real data, weigh the cost, then decide and write down why.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
Choosing a storage model is like choosing a car: the sports car is faster but will not carry the family's luggage. No car is "best" in general, only best for your trip. This lesson teaches you to try and measure before deciding, not to pick what is popular.
Deciding by evidence means timing the same query on the same data in each model, then weighing speed against maintenance cost and consistency (such as duplicated data in documents). The decision frame asks about read and write patterns, the need for transactions, growth, team experience and the cost of change.
Required fundamentals
Part five
Four models, and the lesson that an index gave the same time as a graph traversal.
Readiness to read a number
Every judgement in this part is backed by a measurement on our own data. And a number without context means nothing, so we always state the size and the question.
Accepting the trade
No model wins at everything. Whoever looks for “the best” will not find it; whoever asks “best for which question, at what price” always finds an answer.
Core concepts
Not “fast versus slow” but read speed versus update cost. Normalisation puts the truth in one place (cheap updates, costlier reads); duplication puts it in many (cheap reads, costlier and riskier updates).
The first thing to write before any choice: your ten most frequent questions and how often each occurs. A system read a thousand times per write is not the same system as one written to every second.
Duplicated data goes stale the instant the original changes. And the engineering question is not “does it go stale?” but how many seconds of staleness are acceptable, and who pays if it lasts longer? A name in a comment: minutes are fine. An account balance: zero.
Make one model the source of truth and every other shape derived and rebuildable from it. Whoever makes the specialised shape the source of truth becomes unable to answer a new question.
In a distributed system you cannot have immediate consistency and full availability at once when the network between nodes breaks, you must choose. And the decision is not technical but commercial: do we answer with stale data, or answer with an error?
“I do not know my questions yet” is a common and honest answer. In that case the right choice is relational: it is the best at answering a question you did not anticipate, and migrating away from it later is cheaper than migrating to it.
How it works
Diagram 1, The trade-off with real numbers from the measurement below
Diagram 2, The decision frame: five questions before naming any product
| Read : write ratio | The natural inclination | Why |
|---|---|---|
| 1000 : 1 | Duplication and derived shapes | Updates are rare, so their cost is acceptable |
| 10 : 1 | Relational with good indexes | No extreme justifies itself |
| 1 : 1 | Normalisation | Every duplicate becomes a burden on every write |
| 1 : 10 (write-heavy) | Normalisation + append-only writes | Derived shapes are built later |
The exercise: 205× faster reads, and 12 stale documents
We measured the trade on one screen every system has: the member page, their name and city, their posts, and each post's comments with their authors' names. We built it twice over the same data: once by joining four tables, and once as a single pre-stored document.
def profile_joined(u):
row = users[u]
out = {"name": row["name"], "city": cities[row["city_id"]]["name"], "posts": []}
for p_id in posts_by_user[u]: # join 1
p = posts[p_id]
cs = []
for c_id in comments_by_post.get(p_id, []): # join 2
c = comments[c_id]
cs.append({"text": c["text"],
"author": users[c["author_id"]]["name"]}) # join 3
out["posts"].append({"text": p["text"], "comments": cs})
return out
docs = {u: profile_joined(u) for u in range(N)} # the same screen, stored$ python3 profile.py tables: users=50000 cities=300 posts=200000 comments=600000 documents: 50000 one document is 657 bytes of JSON 3 joins per profile 26.64 ms for 2000 profiles 1 document read per profile 0.13 ms for 2000 profiles
--- the cost of the document shape --- before rename: comment author in doc 0 = user29647 renamed user 29647 in the users table joined view now says : renamed_user document still says : user29647 <-- stale documents that must be rewritten to fix one rename: 12
And note the number 12 is specific to our data: a popular user who comments a lot could appear in thousands of documents. Which is why the practical question is not “how many documents?” but “what is the worst case, and who bears it?”
| Normalised · tables | Duplicated · documents | |
|---|---|---|
| Reading 2,000 pages | 26.64 ms | 0.13 ms |
| Renaming one user | One update | 12 documents (in our data) |
| Immediately after the change | Always correct | Stale until rewritten |
| A new, unanticipated question | A new query | Usually a rebuild of every document |
| Space | Less | More, 657 bytes × 50,000 |
| Suits | Frequent writes, varied questions | Read-dominated, a fixed screen |
Myth vs. reality
| The myth | The reality |
|---|---|
| “Duplication is always bad.” | It gave us reads 205 times faster. What is bad is duplication with no plan for rebuilding. |
| “Normalisation slows the system down.” | Normalisation slows reads and speeds up updates. A write-heavy system gains from it. |
| “Documents remove the need for normalisation.” | They move its cost to write time: one rename became 12 documents in our data, and could become thousands. |
| “Pick one model for the whole system.” | Mature systems use more than one shape with clear roles: one is the source of truth and the rest are derived. |
| “Staleness is a bug.” | Staleness is a decision. The bug is not knowing its duration or who bears it. |
| “I don't know my questions, so I'll pick the newest thing.” | You don't know your questions → pick relational. It is the best at a question you did not anticipate, and leaving it is cheaper. |
Try it yourself
A page read 1000 times per write, with a fixed shape. What is the right inclination?
A 1000:1 ratio makes the update cost acceptable, so you take the 205× read win. But keep the source of truth normalised so you can answer a new question and rebuild the derived shape. The third option is dangerous because it makes the specialised shape the source of truth.
A user's wallet balance. How many seconds of staleness are acceptable?
Because whoever reads the number will make a financial decision on it immediately. Acceptable staleness follows the cost of being wrong, not our wish for speed: a name in a comment tolerates minutes; a balance tolerates not one second.
Quick review flashcards
Check your understanding
1. In one sentence: why is 205× not enough evidence to choose documents?
Because it measures reads alone, and the second measurement showed one change costing 12 documents, so the decision needs both numbers plus your system's read-to-write ratio.
2. Explain in detail where the 26.64 ms came from.
Each page needs three matches: the member's posts, each post's comments, and each comment author's name. So one page with four posts and three comments per post means 4 post reads, 12 comment reads and 12 author-name reads, about 28 reads per page, or roughly 56,000 reads for two thousand pages. The document turns those 28 into one read, because the work was done at write time rather than read time. Duplication does not remove work, it moves it to another moment.
3. (Spot the bug) A team stored the username inside every comment “for speed”, then names changed. What are they missing?
Not the decision but a rebuild plan. Duplication is a sound decision for read-dominated work, but it requires four things: knowing every place the value was copied to, a process that updates those places when the original changes, a declared staleness bound, and monitoring that alerts when it is exceeded. Without those four the system is correct the day it is built and silently wrong a month later.
4. You have a 1:1 read-to-write system. Which model do you lean toward, and why?
Normalisation. Because every duplicate will be paid for on almost every write, so the read gain is matched or exceeded by the loss, and it adds staleness risk and rebuild complexity for nothing. Duplication justifies itself when reads exceed writes by orders of magnitude, not by one.
5. (Decision) The team wants to rebuild the whole system on a document model because “reads are 200× faster”. Your answer?
That the number is correct and insufficient. I ask for three things before any decision: our system's actual read-to-write ratio, a list of duplicated values with the worst case for how many places go stale, and a staleness bound that each screen's owners accept. Then I propose the cheapest reversible path: keep the tables as the source of truth, build the document shape for the single heaviest screen, and measure. Rebuilding the whole system makes the decision irreversible before we hold any evidence, and worse than a wrong model is a wrong model you cannot come back from.
Resources for this part
Sorted by kind: start with the official docs when you want the precise reference, or with an article or video when you want the explanation first.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Official docs1
Articles2
Done with this lesson?
Sign in to save your progress.