Language and transformers

Attention step by step, until you build a small transformer and train it on Arabic text.

TokenisationBPE and SentencePiece, and why an Arabic sentence costs more tokens.
EmbeddingsMeaning as a vector and cosine similarity by hand, and where it misleads.
AttentionQuery, key and value computed on paper for a four word sentence, then in code.
The transformer blockMulti-head attention, layer norm, residuals and the feed-forward, together.