An image as a tensor
Lesson 1 of 7
1An image as a tensor
The big pictureTo a model an image is not a picture but three stacked grids of numbers: red, green and blue. Before any network you need to know their dimension order, their value range and how to normalise them, because a small mistake here makes the best model in the world see noise.
What the picture shows: The image passes four stops: it is read with channels last, cropped to 224, divided by 255 and reordered to (C, H, W) by ToTensor, then normalised with the ImageNet numbers. The marked stop is normalisation, because a mistake there raises no message at all.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
A digital image is a grid of small squares, and each square is three numbers: how much red, green and blue it holds. For a model to understand it, we turn it into a box of numbers with three layers and scale the numbers to roughly between zero and one, so no number drowns out the others.
An image is a tensor shaped (C, H, W) in PyTorch, with C = 3 colours; pixel values 0..255 are divided by 255 then normalised by each channel's mean and standard deviation. A batch adds a fourth dimension (N, C, H, W), and watch for the HWC order in libraries like PIL.
Before You Start
🧊Tensors
You read a shape as a sentence and know the difference between view and permute from the deep learning track.
📊Arrays with NumPy
You know the mean and spread along a chosen axis, like axis=(0, 2, 3) in this lesson.
The Core Idea
To a model, an image is only numbers. In this lesson you see those numbers channel by channel, put their dimensions in the order PyTorch wants, normalise them, then join it all in one pipeline.
1Channels: three grids of numbers
Every pixel in a colour image is three numbers from 0 to 255: red, green and blue. Each colour is a grid the size of the image, called a channel.
Look at the picture: the image is three layers stacked. A greyscale image is one layer, and some images carry a fourth alpha channel you must drop before the model.
scikit-learn ships handwritten digit images, 8 by 8 pixels with values from 0 to 16. Draw one in characters to see that a digit is just a shape in a grid.
from sklearn.datasets import load_digits
digits = load_digits()
img = digits.images[0]
print("label:", digits.target[0], "shape:", img.shape)
for row in img.astype(int):
print("".join(" .:-=+*#%@"[min(v * 10 // 17, 9)] for v in row))
That is a zero. The model will see exactly this grid, 64 numbers, and nothing else.
BGR, not RGB, so the model sees an orange sky and blue faces and its accuracy drops with no error message.2Dimension order: (C, H, W)
Image libraries give you the shape (H, W, C), with channels last. PyTorch wants channels first: (C, H, W).
Look at the picture: the same three dimensions in a new order. The numbers are the same; only which comes first changed.
import numpy as np
image = np.zeros((4, 6, 3), dtype=np.uint8)
image[:, :3] = [200, 30, 30]
image[:, 3:] = [30, 30, 200]
print("shape (H, W, C):", image.shape, image.dtype)
print("top-left pixel:", image[0, 0])
print("red channel row 0:", image[0, :, 0])
chw = image.transpose(2, 0, 1)
print("shape (C, H, W):", chw.shape)
The left half is red [200, 30, 30] and the right half blue. Image libraries (Pillow, OpenCV, matplotlib) give you (H, W, C), channels last, while PyTorch expects (C, H, W). transpose(2, 0, 1) is the bridge, and you will write it or its equivalent in every vision project.
During training a fourth dimension comes first, the number of images, so the shape becomes (N, C, H, W). Always print the shape after loading the first image.
3Normalising per channel
Networks learn faster and more stably when inputs sit near zero with a spread near one. So we normalise in two steps.
Look at the picture: divide by 255 so values fall between 0 and 1, then subtract each channel's mean and divide by its spread.
A batch of 16 random 32 by 32 images. axis=(0, 2, 3) means: compute across images, height and width, keeping channels separate.
import numpy as np
rng = np.random.default_rng(0)
batch = rng.integers(0, 256, size=(16, 3, 32, 32)).astype(np.float32)
scaled = batch / 255.0
mean = scaled.mean(axis=(0, 2, 3), keepdims=True)
std = scaled.std(axis=(0, 2, 3), keepdims=True)
normalised = (scaled - mean) / std
print("raw range:", batch.min(), batch.max())
print("scaled range:", scaled.min(), round(float(scaled.max()), 3))
print("per-channel mean after:", normalised.mean(axis=(0, 2, 3)).round(3) + 0.0)
print("per-channel std after: ", normalised.std(axis=(0, 2, 3)).round(3))
After normalising, each channel has mean zero and spread one. That is exactly what transforms.Normalize in torchvision does, except it takes the mean and std as given numbers instead of computing them from the batch.
4ImageNet numbers and the full pipeline
With a pretrained model you do not compute the mean and std from your data. Use its original data's numbers exactly: mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225] for ImageNet models.
Models expect a fixed size, often 224 by 224. So we resize and crop first, then normalise.
This is what you will write in a real project. It needs torchvision and an image on your machine, so it is here for reading.
from PIL import Image
from torchvision import transforms
preprocess = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
image = Image.open("cat.jpg").convert("RGB")
x = preprocess(image).unsqueeze(0)
print(x.shape) # torch.Size([1, 3, 224, 224])
convert("RGB") drops alpha and expands greyscale to three channels. ToTensor divides by 255 and turns (H, W, C) into (C, H, W) in one step. unsqueeze(0) adds the batch dimension, since the model expects a batch even for a single image.
How It Works, Step by Step: how an image becomes a model input
Open the image with Pillow and convert("RGB"), which drops the alpha channel and expands greyscale to three channels.
Resize it to 256, then crop the centre to 224, the size the model expects.
ToTensor divides by 255 and turns (H, W, C) into (C, H, W) in one step.
Normalize subtracts each channel's mean and divides by its spread, then unsqueeze(0) adds the batch dimension.
Worked example
Every pixel in a colour image is three numbers from 0 to 255. A 224 by 224 image means 150,528 numbers. Start with a tiny image you build yourself so you can see every number.
x.min(), x.max() and x.mean() for the first batch.
Try it without code
Myth vs Reality
| Myth | Reality |
|---|---|
| Every library reads colours in the same order. | OpenCV reads channels as BGR, not RGB, so the model sees an orange sky and its accuracy drops with no error message. |
| To turn (480, 640, 3) into (3, 480, 640), view is enough. | view changes the shape without moving data, so it scrambles pixels. Reordering dimensions needs permute or ToTensor. |
| With my own data I always compute a new mean and std. | With a pretrained model, start with its original data's numbers, because its first layers learned on inputs normalised with them. |
Try it yourself
You loaded an image with shape (480, 640, 3). What do you do before feeding it to a PyTorch model?
Channels are last, so you need to reorder with permute or ToTensor, not view, which scrambles pixels. Then the size and normalisation the model was trained with, then the batch dimension.
You use an ImageNet-trained model. Which normalisation numbers do you use for your medical images?
The model's first layers learned their weights on inputs normalised with ImageNet's numbers, so changing them changes what those layers "see". In practice both often work when fine-tuning the whole model, but the safe starting point is the original numbers.
Quick review cards
(N, C, H, W): number of images, channels, height, width. Other libraries put channels last, so you need a conversion.Check Your Understanding
You have single-channel greyscale images and a pretrained model expecting three channels. The simple fix?
Repeat the single channel three times, with convert("RGB") in Pillow or x.repeat(3, 1, 1) on the tensor. The image still looks grey, but the shape is now (3, H, W) as the model expects, and each channel gets its own normalisation. The alternative, changing the first layer to accept one channel, is harder and loses part of the trained weights.
A 224 by 224 colour image. How many numbers does the model see?
224 × 224 × 3 = 150,528 numbers, three per pixel: red, green and blue.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Official docs2
Done with this lesson?
Sign in to save your progress.