Framing the problem
Lesson 1 of 8
1Framing the problem
The big pictureBefore any model, write one sentence: what am I predicting, from which inputs, and how will I know I succeeded. Most failed machine learning projects fail here, not in the choice of algorithm.
What the picture shows: A solution that learns nothing and says "every order is fine" scores 94% accuracy and catches none of the 60 fraud cases. That is the number any model must clearly beat, and it also shows that accuracy may be the wrong metric.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
Before picking a tool, know the kind of question. "Is this message fraud?" is yes or no. "What will this house sell for?" asks for a number. "Which customers are alike?" has no ready answer. Each kind has its method. Then you ask: how will I know the answer is good?
classification predicts a category and regression a continuous value, both supervised learning from labelled examples. clustering is unsupervised. Before any model you fix the target, the features, the metric and a naive baseline you must beat.
Before You Start
🧮Data and tables
You finished the data track, so you can read a table with pandas and know a row is a sample and a column a feature.
⚖️The base rate
You know from the probability lesson why 99% accuracy on a rare class can fool you.
The Core Idea
Before any model you need three things written down: the type of output, the score of a solution that learns nothing, and the cost of each error. In this lesson you build them in order, then see when a simple rule beats a model.
1The output decides the problem
The first question is what comes out of the solution. A category means classification, a number means regression, and no known right answer means clustering.
Look at the picture: two questions take you to the type of problem. "Is this order fraud?" follows the lit path to classification.
"Is this order fraud?" outputs yes or no, so it is classification. "How much will the customer pay?" outputs a number, so it is regression. "What natural groups exist among our customers?" has no known right answer, so it is clustering. The type of output decides everything after it: the algorithm, the metric, the shape of the data.
Even star ratings from 1 to 5 may be treated as categories. In clustering the result is a hypothesis a human must check, not a fact.
2The baseline for classification
The naive baseline is the score of a solution that learns nothing. In classification, that solution always predicts the most frequent class.
We have 1000 orders, 60 of them fraud. Compute what the "every order is fine" solution scores.
Before any model, score a solution that learns nothing. For classification: always predict the most frequent class.
from collections import Counter
labels = ["ok"] * 940 + ["fraud"] * 60
counts = Counter(labels)
majority = counts.most_common(1)[0][0]
accuracy = counts[majority] / len(labels)
print("majority class:", majority)
print(f"baseline accuracy: {accuracy:.1%}")
A solution that says "every order is fine" scores 94%. Any model you build must clearly beat that number, or you are paying for a model in exchange for nothing. Notice too that this solution caught not a single fraud case, which tells you accuracy itself may be the wrong metric here.
3The baseline for regression
In regression the naive solution predicts the mean for every row. Its average error is the number any model must beat.
In regression, the naive solution predicts the mean for every row. Compute how far off it is on average; that is the number to beat.
import numpy as np
prices = np.array([310, 450, 280, 520, 390, 610, 330, 470], dtype=float)
predict_mean = np.full_like(prices, prices.mean())
mae = np.abs(prices - predict_mean).mean()
print(f"mean price: {prices.mean():.1f}")
print(f"baseline MAE: {mae:.1f}")
A solution that knows nothing about the flats misses by 92.5 thousand on average. A model off by 85 thousand barely helps; one off by 30 thousand is a real improvement. Without this number, "an error of 85 thousand" tells you nothing about good or bad.
Look at the picture: a model off by 85 thousand is very close to a solution that knows nothing about the flats. A model off by 30 thousand is a real improvement.
4The cost of an error picks the metric
Errors are not equal. That is why you write the metric before the code: after seeing results you will be tempted by whichever metric flatters them.
Missing a fraud costs money; blocking a good order costs an angry customer. The two costs are not equal, and that is what will later decide which metric you pick and where you put the threshold.
In fraud, missing a case usually costs more, so you ask for high recall at an acceptable precision. It is a business decision before it is a technical one.
5When a rule is enough
You do not need machine learning when an expert can write a clear rule in a few lines, or when there is no past data with the right answer.
Look at the picture: a message passes two checks, and whatever matches neither goes to the general team.
Some problems pitched as machine learning projects are solved by simple rules. Try the rule first: if it is accurate enough, it is cheaper, clearer and easier to fix than any model.
def route(message):
text = message.lower()
if "refund" in text or "استرجاع" in text:
return "billing"
if "password" in text or "كلمة المرور" in text:
return "account"
return "general"
for m in ["I want a refund", "نسيت كلمة المرور", "متى تفتحون؟"]:
print(route(m), "<-", m)
Three conditionals route the messages. If they get 90% of real tickets right, you saved weeks of data collection and training. Machine learning is worth it when there are too many cases for rules to describe, and when you have past examples with the right answer.
How It Works, Step by Step: how to frame a vague idea as a machine learning problem
Define the sample and the output: what a row stands for, and whether the output is a category, a number, or has no known right answer.
Pick only the inputs available at prediction time, and exclude any column you learn only after the event.
Compute the naive baseline, such as the majority class or the mean, and write down its number.
Write the metric down before the code, choosing it from the cost of each kind of error, not from what flatters the results.
Going deeper
| Business question | Problem type | Naive baseline | A fitting metric |
|---|---|---|---|
| Is the order fraud? | Binary classification | "Everything is fine" | Recall at acceptable precision |
| How many minutes will delivery take? | Regression | Mean of past times | MAE in minutes |
| Which team gets this ticket? | Multiclass classification | The busiest team | Per-class accuracy |
| What segments do our customers form? | Clustering | One segment | Interpretability and human review |
Worked example
Three questions turn a vague idea like "we want to use AI on our orders" into a problem with a measurable answer. Try them on a small example before we name anything.
Try it without code
Myth vs Reality
| Myth | Reality |
|---|---|
| Machine learning improves any problem. | If a rule an expert can write in a few lines is good enough, it is cheaper, clearer and easier to fix than any model. |
| 94% accuracy is an excellent result. | On data where 94% of orders are fine, a solution that learns nothing gets exactly that. A number without a baseline means nothing. |
| Every column in the table is a valid input. | A column you only learn after the event, like the date of last support contact in a cancellation problem, makes the model brilliant in testing and useless in reality. |
Try it yourself
A team wants to predict "customer satisfaction" from 1 to 5 from the review text. Which framing is the best start?
A known correct output in historical data (the stars) makes this supervised learning. Stars can be treated as classes or as a number; the choice depends on whether confusing 1 with 5 is worse than confusing 4 with 5. Clustering is for when no right answer exists.
Your fraud model scores 95% accuracy, and fraud is 5% of the data. What do you conclude?
A solution saying "never fraud" scores exactly 95%. The number alone cannot tell a useful model from one that does nothing. This is why you compute the baseline first, and why the metrics lesson starts with this very case.
Quick review cards
Check Your Understanding
A manager asks for "a model that predicts sales". Which three questions do you ask before opening any data?
First: sales of what, over which period, at which granularity (daily per branch? monthly for the company?), which fixes the sample and the output. Second: what will be done with the prediction and when, since that fixes which inputs exist at prediction time and what error is acceptable. Third: how is it predicted today, because the current method, like "same week last year", is the baseline you must beat.
What is the naive baseline for a regression problem like flat prices?
Predicting the mean for every row, then computing the mean absolute error, which was 92.5 thousand in the lesson's example. Every model is judged by how far it beats that number.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Done with this lesson?
Sign in to save your progress.