hackertubescom

HACKER TUBES / Playlists

From neural networks to GPT

Build an intuition for learning, implement backpropagation, and follow attention into a working language model.

7 videos · 8 hours 9 min

Start watching

Suggested viewing order

Select any video to watch. Use the previous and next links on its watch page to continue through this playlist.

  1. 01

    But what is a neural network? | Deep learning chapter 1

    3Blue1Brown · 16 HN recommendations

    3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.

  2. 02

    Gradient descent, how neural networks learn | Deep Learning Chapter 2

    3Blue1Brown · 4 HN recommendations

    3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

  3. 03

    The spelled-out intro to neural networks and backpropagation: building micrograd

    Andrej Karpathy · 17 HN recommendations

    Andrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.

  4. 04

    Transformers, the tech behind LLMs | Deep Learning Chapter 5

    3Blue1Brown · 13 HN recommendations

    3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

  5. 05

    Attention in transformers, step-by-step | Deep Learning Chapter 6

    3Blue1Brown · 5 HN recommendations

    3Blue1Brown's visual walkthrough of the attention mechanism inside transformers, covering self-attention, multiple heads and cross-attention. It is chapter 6 of his deep learning series.

  6. 06

    Let's build the GPT Tokenizer

    Andrej Karpathy · 5 HN recommendations

    Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

  7. 07

    Let's build GPT: from scratch, in code, spelled out.

    Andrej Karpathy · 14 HN recommendations

    Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.