Train Your Own LLM from Scratch on a Laptop

🧠 Train an LLM from Scratch: A Hands-On Workshop
#

Inspired by Andrej Karpathy’s nanoGPT, this project guides you to build a complete GPT model from first principles — no black boxes. 🎯

📌 What You’ll Build
#

A ~10M parameter GPT model that trains in under an hour on your laptop, capable of generating Shakespeare-like text.

You’ll write from scratch:

🔤 Tokenizer — converts text into numbers the model can process
🏗️ Model architecture — transformer with attention and feed-forward layers
🔄 Training loop — forward pass, backprop, optimizer, learning rate scheduling
✍️ Text generation — sampling from your trained model

⚡ No Black-Box Libraries
#

No AutoModel.from_pretrained(). Everything built from first principles.

💡 Explanation in a nutshell
#

An LLM (Large Language Model) like GPT is a neural network that learns to predict the next word in a text. It’s built on the Transformer architecture. This workshop shows you how every internal piece works — from how text becomes numbers, to how the model “learns” language patterns — in under 1000 lines of Python code.