Large Language Models
Lesson 3 of 3
3Large Language Models (LLMs): Capabilities and Trade-offs
Understanding what these models actually do "under the hood" lets you use them wisely instead of blind trust or total rejection.
What the picture shows: The model sees "The cat sat on the", gives every next word a probability, picks "mat", and does the same for the word after.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
A language model has read a huge amount of text and learned one thing: what word most likely comes next? Like a friend who memorised thousands of books and can finish any sentence you start. But it does not "know" whether its sentence is true, nor anything that happened after the last book it read.
A LLM is a transformer network that predicts the next token. It is strong at wording, summarising, translation and code, and weak at exact facts, arithmetic and hallucination. Bigger models are more accurate, slower and costlier, and their knowledge stops at a knowledge cutoff unless you connect them to fresh sources.
What You Need Before This Part
✍️Understanding prompting basics
The four principles from Part Two (context, format, decomposition, iteration), because understanding how the model works "under the hood" directly explains why these principles work.
🔢A basic concept of probability
An intuitive understanding that a probability is a number between 0 and 1 expressing how likely something is, you don't need advanced math, just knowing that the model "chooses" the next word based on which is most likely.
⚙️A general idea of machine training
A simplified picture that "training" a model means exposing it to enormous amounts of text and gradually adjusting its internal weights until it gets better at predicting the next word, the details come in this part.
The Core Idea: Statistical Prediction, Not Human Understanding
At its core, a large language model predicts the most likely next word (or word fragment) based on everything before it, learning this probability from enormous amounts of text. It doesn't "understand" in the human sense, but it captures linguistic and logical patterns deep enough that answers often seem very intelligent.
In practice this happens in two stages: during training, the model is exposed to enormous amounts of text broken into small units called tokens (a whole word, or just a fragment of one), and millions or billions of its internal weights are gradually adjusted until its next-token predictions become as accurate as possible. During actual use (inference), the model generates text one token at a time: it computes a probability distribution over all plausible next tokens, samples one from it (not always the single highest-probability option, with a calculated degree of randomness), appends it to the text, then repeats the whole process again on the new text, which is why the same question can get you two slightly different answers on two separate runs.
Figure 1, how a large language model generates its answer: one prediction step repeated token by token until the answer is complete.
1Strengths
Summarizing long texts, explaining concepts multiple ways, generating repetitive boilerplate code, translation, and spotting patterns in given text.
2Weaknesses
Precise, rare facts (it may fabricate them), complex multi-step math, knowledge of events after a certain date (Knowledge Cutoff), and very long logical reasoning chains.
3Size vs. speed trade-off
A bigger model is usually more accurate but slower and more expensive to run; a smaller model is faster and cheaper but less accurate on complex tasks, there's no "absolute best" model for every task.
4Knowledge Cutoff
Every model is trained on data up to a certain date, any event after that date is unknown to it unless equipped with an external search tool.
These four points aren't separate, unrelated traits, they're natural consequences of the next-token-prediction mechanism described above. The strengths (summarizing, explaining in multiple ways, translation) are strong precisely because these linguistic patterns are repeated with enormous density in the training data, millions of examples of summarizing or translating text genuinely exist in the text the model learned from, so its probability distribution over these tasks is rich and very stable. The weakness around rare facts comes from the same cause we discussed in Part One about hallucination: if a particular piece of information doesn't repeat often enough in training, there's no strong statistical basis for the model to build an accurate prediction on.
Complex math is weak for a completely different, more specific reason: the model generates numbers one token at a time through the same mechanism shown in Diagram 8 above, not through a precise computational engine like a calculator. Numbers themselves get split into tokens in a way that doesn't always preserve place value (the same number can be split differently depending on its surrounding context), so if the model slips on a single digit-carry step in the middle of a long multiplication, that error becomes part of the text fed back in as input for the next step, recall Diagram 8: the current output becomes the next step's input, and the error compounds across the following steps until a completely wrong final answer comes out looking exactly as confident and polished as a correct one.
The size-versus-speed trade-off has a direct computational cause, not a marketing one: every parameter inside the model is a number actually used in a calculation when generating each single token; if a model has a larger number of parameters (potentially billions), generating each token requires more arithmetic operations, measured in FLOPs. More operations mean longer processing time and higher running cost (electricity and server hardware), which is exactly why most companies offer "small" and "large" versions of the same model family: the small version is faster and cheaper for simple everyday tasks, and the large one is more accurate for tasks needing deeper reasoning.
Finally, Knowledge Cutoff is an unavoidable consequence of how these models are built in the first place: assembling a training corpus of hundreds of billions of words and training the model on it genuinely takes months, so by the moment a model is released to the public, its data is necessarily a "frozen snapshot" of some past moment. The practical fix isn't training a brand-new model every day (completely impractical), it's equipping the model with an external search tool that fetches fresh text and inserts it into that same context window described in Part Two, so the model uses it as fresh, immediately-processed context, not as knowledge stored permanently in its weights.
When to Trust the Model and When to Verify Manually
| Type of request | Recommended trust level |
|---|---|
| Explaining a well-known general programming concept (loops, pointers) | High, this is stable knowledge repeated heavily in training data |
| A specific historical fact or precise statistic | Low, always verify with an independent source |
| A very recent event or one after the model's knowledge date | Very low, the model may not know it at all or may mix up dates |
| Writing code for a common, well-known task | Medium to high, but actually test the code, don't assume it's correct |
Notice that this table isn't a list of arbitrary rules, it's a direct application of what was explained above about pattern density in training data: the more a request rests on a heavily repeated, common pattern (like a basic programming concept), the stronger and more reliable the statistical foundation behind the model's prediction. And the closer a request gets to a rare fact or a very recent event, the weaker or more absent that statistical foundation becomes, which is exactly what raises the chance of the hallucination described in Part One.
This table can be simplified into one quick decision rule you can apply to any question you ask the model:
Figure 2, a quick decision rule for when to trust a model's answer and when to verify it.
Worked Example: How the Model Actually Generates a Sentence
Let's trace how the model generates a simple sentence step by step, imagine you typed: "The capital of Saudi Arabia is"
| Step | Current text | Top probabilities for next word | Word chosen |
|---|---|---|---|
| 1 | "The capital of Saudi Arabia is" | Riyadh (92%) • a (4%) • Jeddah (1%) | Riyadh |
| 2 | "The capital of Saudi Arabia is Riyadh" | . (68%) • , (18%) • and (7%) | . |
| 3 | "The capital of Saudi Arabia is Riyadh." | [end] (85%) • It (8%) | [end] |
Notice three important things in this example:
- Each step is independent: The model recomputes probabilities from scratch at each step based on all previous text, it doesn't plan the full sentence in advance.
- The choice isn't deterministic: "Riyadh" at 92% doesn't mean it will always be chosen, at a temperature above zero, there's a small chance another word gets selected, which is why the same question can produce slightly different answers.
- Common information is more accurate: "Riyadh is Saudi Arabia's capital" is repeated enormously in training data (92%), so the model is very confident about it. But a question about a rare fact might produce close probabilities (say 30%, 28%, 25%), it picks any of them with equal confidence, even if wrong.
Myth vs. Reality
| Myth | Reality |
|---|---|
| Large language models "think" like humans and understand what they read | The model processes massive statistical patterns, it captures correlations between words deep enough to seem like understanding, but it has no consciousness or mental model of the world like a human does |
| The bigger model is always the best for every task | A bigger model is more accurate on complex tasks but slower and more expensive, for a simple task like summarizing a short paragraph, a small, fast model does the same job at a fraction of the cost |
| The model does math calculations as accurately as a calculator | The model generates digits token by token using the same linguistic prediction mechanism, it has no actual computational engine, so one mistake in a digit midway through compounds across subsequent steps and produces a confidently wrong final answer |
Real-World Applications
💻Code generation and review
Language models excel at writing repetitive boilerplate code and converting between languages, but reviewing code logic and testing manually remains essential because a logical error looks syntactically fine.
🌍Translation and summarization
Among the strongest applications of these models, because these tasks rely on linguistic patterns repeated with enormous density in training data, making the probability distribution rich and stable.
🎓AI as a tutor
The model can explain the same concept in multiple different ways until you find the approach that suits you, an application that directly leverages its ability to rephrase linguistic patterns in diverse forms.
🔍Retrieval-Augmented Generation (RAG)
Providing the model with real documents inside the context window lets it read them on the fly and produce answers grounded in up-to-date information, a practical solution to the Knowledge Cutoff problem.
Deep Dive: Model, Platform and Trade-offs
The point of comparing models is not to crown an absolute "best", but to build judgment about which model suits a specific task and context. We start by separating the model from the platform.
Model vs. platform
A model is a trained system that turns an input into an output. A platform (such as ChatGPT, Gemini, Claude or Grok) is a product wrapped around one or more models: a user interface, safety controls, memory, tools and default settings. So answers can differ because the model differs, because the platform’s behavior differs, or both.
Figure 3, Platform = model + extra layers. Changing any layer can change the answer.
Types of generative models
| Type | Input → output | Example use |
|---|---|---|
| LLM | text → text | explanations, summaries, coding help |
| VLM | image + text → text | describing a screenshot or explaining a chart |
| text → image | text description → image | generating illustrations |
| speech ↔ text | audio ↔ text | transcription and text-to-speech |
| text → video | description → clip | short generated clips |
Why models differ
📚Training data and size
What the model saw and how many parameters it has shape its knowledge and reasoning ability.
🛡️Alignment tuning
Extra training that makes the model more cautious, cooperative, or concise.
🎙️Default style
Some models are verbose, some direct, some overly cautious.
🧠Context window and reasoning limits
How much text can be seen at once, and how deep the available reasoning is.
How to compare fairly
- Use the exact same prompt with every model.
- Record the differences: structure, assumptions, tone, length, level of caution.
- Focus on trade-offs, not on "who was right".
- If possible, compare blind (without knowing the model’s name) to reduce bias toward famous names.
Failure modes to look for
| Failure mode | How it shows up | How to detect it |
|---|---|---|
| Hallucination | A non-existent source or number with high confidence | Ask for the source and open it |
| Knowledge cutoff | Outdated info about recent events | Ask about a recent event whose result you know |
| Arithmetic | Wrong result for a big calculation | Check with a calculator |
| Sycophancy | Agrees with a false premise you supply | Supply a deliberately false premise and watch |
Local or remote?
Open-source models can run on your own machine with tools like Ollama, so your data is not sent to an outside service, but performance depends on your hardware (memory and graphics card), and local models are usually smaller and less capable than huge remote ones. A rough memory rule: parameters × bytes per parameter. A 7-billion-parameter model at 16-bit needs about 14 GB, and after 4-bit quantization about 3.5–4 GB, plus headroom to run.
| Criterion | Local | Remote |
|---|---|---|
| Privacy | Data stays with you | Sent to the provider |
| Capability | Usually lower | Usually higher |
| Requirements | Strong memory and processor | Just an internet connection |
| Cost | One-time hardware cost | Subscription or usage limits |
| Offline use | Possible | Not possible |
Figure 4, Choosing a model: a practical decision balancing privacy, quality and hardware.
Try it yourself
Why can the same question give two slightly different answers?
At inference the model computes a distribution over the next tokens and samples one (not always the top), then repeats.
Why does a model sometimes get long multiplication wrong?
It is a text predictor, not a calculator. One slipped carry in the middle spreads into the rest, so use a calculation tool for numbers.
You ask about an event after the model's training date and give it no search tool. What should you expect?
The knowledge cutoff means anything after it is missing unless an outside source is supplied, and the gap can produce a made-up answer.
Quick Review Flashcards
Check Your Understanding
Why doesn't a language model "know" events that happened after its training date, despite seeming very intelligent?
Because its entire knowledge is built from the data it trained on up to a certain moment; it has no direct connection to the current world unless explicitly equipped with an external search tool that fetches new information.
Why does the model make mistakes in complex math calculations despite being able to explain mathematics clearly?
Because it generates digits token by token using the same linguistic prediction mechanism, not a precise computational engine. The linguistic explanation of math is a heavily repeated pattern in training data, so it excels at that; but executing the actual computation relies on each numerical token being accurate, and one mistake compounds across subsequent steps.
What's the difference between pre-training, fine-tuning, and RLHF?
Pre-training teaches the model to predict the next word from massive undirected text, producing a skilled "text completer" but not an assistant. Fine-tuning trains it on human-written prompt/response pairs so it follows instructions. RLHF further improves it through human preference rankings that train a reward model to steer outputs toward more helpful answers.
Spot the two mistakes: your classmate says, "I asked the model about a match played yesterday and it answered confidently, and I asked it to multiply two large numbers and I don't need to double-check since big models calculate like a calculator."
Mistake one: Knowledge Cutoff, the model doesn't know events after its training unless it is given a search tool, and it may mix things up or fabricate. Mistake two: the model generates digits token by token with no calculation engine, so any slip in a middle step compounds; verify with an actual calculator.
Your machine has 8 GB of memory; does a 13-billion-parameter 16-bit model fit?
No: it needs about 26 GB. After 4-bit quantization it approaches 6.5 GB and might run tightly, but a smaller model is more sensible.
Resources for this part
Sorted by kind: start with the official docs when you want the precise reference, or with an article or video when you want the explanation first.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Video2
Done with this lesson?
Sign in to save your progress.