Q1
Why is the dot product central to deep learning?
Q2
You have `A.shape == (4, 3)` and want `A @ B` to produce shape `(4, 5)`. What must `B.shape` be?
Q3
Why do we step *against* the gradient during training?
Q4
What is backpropagation, in one sentence?
Q5
Why is cross-entropy the standard loss for classification?
Q6
You see training loss decreasing but validation loss starting to rise. What's happening?
Q7
Why does PyTorch require `optimizer.zero_grad()` at the top of every step?
Q8
What does `tensor.detach()` return?
Q9
Why call `model.eval()` before inference?
Q10
Why are residual connections (`out = x + F(x)`) crucial in deep nets?
Q11
Your loss is `nan` after a few hundred steps. Most likely cause?
Q12
What does `CrossEntropyLoss` expect as input?
Q13
Why are convolutions parameter-efficient compared to fully-connected layers for images?
Q14
What's the receptive field of two stacked 3×3 convs (stride 1)?
Q15
Why is the startup pattern in 2026 to fine-tune pretrained models, not train from scratch?
Q16
What's the typical learning rate for fine-tuning a BERT-family model?
Q17
Why does an LSTM solve vanishing gradients while a vanilla RNN doesn't?
Q18
Why can't you use a bidirectional RNN for autoregressive language modeling?
Q19
Why divide attention scores by √d_k before softmax?
Q20
What does Grouped-Query Attention (GQA) optimize?
Q21
You need a fixed-size embedding for retrieval. Which architecture family?
Q22
Why does Stable Diffusion operate in VAE latent space instead of pixel space?
Q23
What does the KL term in the VAE objective do?
Q24
What does the diffusion training loss optimize?
Q25
Why does PPO clip the policy ratio?
Q26
Why does RLHF add a KL penalty to the reward?
Q27
What's DPO's main advantage over PPO-based RLHF?
Q28
Why does LoRA train only ~1% of params and still achieve ~95% of full-FT quality?
Q29
Why does QLoRA lose almost no quality despite a 4-bit base?
Q30
Your fine-tune nails your task but the model now hallucinates basic facts. Most likely cause?
Q31
Why is the LR for DPO ~10× smaller than for SFT?
Q32
Why does PagedAttention (vLLM) give ~24× throughput vs naive batching?
Q33
You want to ship a fine-tuned 8B model to laptop users with one click. Best engine?
Q34
What does speculative decoding optimize?