Observing the Probabilities
What I observed while training a tiny language model: the loss stopped falling, rug's embedding never moved, and cat and dog were not as close as I expected.
Llms
Viewing Language Through a Probabilistic Lens
How next-word prediction, distributed representations, and a neural probability model let useful structure emerge from language.
Working With Text Data
How raw text becomes something an LLM can actually train on: tokenizing, building a vocabulary, byte pair encoding, sliding windows, and turning token IDs into token and positional embeddings.
Understanding LLMs
A foundational look at large language models, recurrent and convolutional architectures, transformers, and the difference between BERT and GPT.