The foundations
How models learn the language game
Follow the journey from rule-based systems to generative AI, then unpack the pieces inside modern language models: tokens, embeddings, attention, prediction, and alignment.
- Rules → machine learning → deep learning → generative AI
- Tokenization, embeddings, and self-attention
- BERT vs. GPT and decoding with temperature, top-k, and top-p
- Human preference alignment with RLHF and DPO