Curriculum

Vectors, matrices, tensors, decompositions, and the operations that make neural networks possible. The complete working vocabulary — taught the way a practitioner uses it.

Derivatives, the chain rule, the gradient, the Jacobian, and the Hessian. Plus the optimizers built on top: SGD, momentum, RMSProp, Adam.

Distributions, expectations, maximum likelihood, KL divergence, cross-entropy, the bias-variance trade-off, and the information-theoretic view of loss functions.

The Tensor object: device, dtype, strides, views. The autograd engine: graph construction, backward, gradient accumulation, the leaf-vs-non-leaf distinction, and the operations that silently break it.

How models are structured: subclassing nn.Module, the layer zoo (Linear, Conv, Norm, Embedding, Dropout, activations), parameter registration, initialization, and the residual/normalization combo that makes deep nets trainable.

Losses, optimizers, schedulers, dataloaders. Mixed precision, gradient accumulation, gradient clipping, checkpointing, EMA, distributed training (DDP, FSDP). The complete production training script, line by line.

Convolution arithmetic, pooling, receptive fields, the classic architectures (LeNet → AlexNet → VGG → ResNet → EfficientNet → ConvNeXt), and the transfer-learning workflow that powers production vision.

The pre-transformer way to model sequences. Why vanilla RNNs fail, how LSTM gates solve it, when GRU is the right cheap alternative, and where recurrence still wins in 2026 (streaming, very long context, RNN-T speech).

Scaled dot-product attention. Multi-head attention. Position encodings (sinusoidal, learned, RoPE, ALiBi). The encoder block, the decoder block, and the architectural families (encoder-only BERT, decoder-only GPT, encoder-decoder T5). Built up from numpy to nn.Module.

Three families of generative models, the math behind each, and where they win. VAE for structured latent spaces, GAN for photorealism with adversarial training, diffusion for the state of the art in images, audio, and video.

MDPs, value functions, policy gradients, PPO. Then RLHF: reward modeling, PPO for LLMs, and the modern alternatives (DPO, ORPO, KTO). The math behind how ChatGPT learned to be helpful.

The 2026 startup playbook. Full fine-tune vs. LoRA vs. QLoRA. Unsloth for 2× speed / 70% less memory. Datasets, chat templates, evaluation, and a complete end-to-end recipe to fine-tune Llama-3 / Mistral / Gemma on a single 24GB GPU.

Quantization (GPTQ, AWQ, GGUF). Inference engines (vLLM, TGI, Llama.cpp, TensorRT-LLM, Ollama). KV-cache, paged attention, speculative decoding. Latency vs. throughput vs. cost. A real serving recipe.