The video is an interview in which Jeremy Berman, a researcher at Anthropic, gives an intuitive overview of how large language models are built. He says he is self-taught and that the field is approachable for anyone who follows their curiosity. The structure runs from neural networks and training, through language models, pre-training and post-training, to thinking and reinforcement learning.

He begins with a toy model that predicts New York apartment prices from location, size and bathrooms. Each connection is a numerical weight, which he calls a knob, and training means nudging the knobs to reduce the loss, the gap between prediction and reality. He explains that PyTorch, an autograd engine, tracks how each parameter affects the loss, and he runs a short PyTorch script that trains the toy model over a thousand steps.

He then shows the same data as plain text, where a language model predicts the next token, with a probability assigned to every possible token. Using the Qwen 0.5B model in Google Colab on a free T4, he shows a base model acting as an autocomplete engine. It produced incoherent output and made up a quote attributed to Matthew. Wrapping a prompt in the chat format makes it behave like an assistant, and Jeremy says post-training on many example conversations with a stop token makes this reliable.

The final section covers reasoning. Jeremy says models that blurt out answers get arithmetic wrong, and that prompting step-by-step thinking helps. He credits reinforcement learning with producing models like o1, and describes how DeepSeek's R1 paper used GRPO: sample many answers, score format and correctness, and reinforce answers above the group average. The transcript supplied ends partway through this explanation, so the closing sections on discovery are not covered here.

Key takeaways

  • Jeremy says a language model is essentially a file of numbers plus a short program describing how they are wired together.
  • Training adjusts parameters to reduce loss, with PyTorch calculating each parameter's effect automatically via backpropagation.
  • A base model only autocompletes text; the video shows it can be coaxed into acting as an assistant by formatting the prompt.
  • Post-training on example conversations, including a stop token, turns the base model into an instruct model.
  • According to Jeremy, GRPO improves on simple rejection sampling by learning from poor answers as well as good ones.
  • The demos are free to reproduce, using the open-weights Qwen model on Google Colab.

Our take

This is a teaching video rather than news or a benchmark. It is a useful grounding for enthusiasts who want the mechanics of training without the maths, and the toy demos are simple enough to repeat at home. The explanations are one researcher's simplified account, and he says himself that he is glossing over a lot.

The transcript we worked from stops partway through the GRPO explanation, so we cannot say how the video handles the final topic of whether models can discover new things. We have no relevant ByteOrbital coverage to compare it against.