hackertubescom

HACKER TUBES / 재생목록

From neural networks to GPT

Build an intuition for learning, implement backpropagation, and follow attention into a working language model.

7 개 영상 · 8 시간 9 분

시청 시작

추천 시청 순서

동영상을 선택하세요. 시청 페이지에서 이전 및 다음 링크로 계속 볼 수 있습니다.

  1. 01

    But what is a neural network? | Deep learning chapter 1

    3Blue1Brown · 16 개의 HN 추천

    3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.

  2. 02

    Gradient descent, how neural networks learn | Deep Learning Chapter 2

    3Blue1Brown · 4 개의 HN 추천

    3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

  3. 03

    The spelled-out intro to neural networks and backpropagation: building micrograd

    Andrej Karpathy · 17 개의 HN 추천

    Andrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.

  4. 04

    Transformers, the tech behind LLMs | Deep Learning Chapter 5

    3Blue1Brown · 13 개의 HN 추천

    3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

  5. 05

    Attention in transformers, step-by-step | Deep Learning Chapter 6

    3Blue1Brown · 5 개의 HN 추천

    3Blue1Brown's visual walkthrough of the attention mechanism inside transformers, covering self-attention, multiple heads and cross-attention. It is chapter 6 of his deep learning series.

  6. 06

    Let's build the GPT Tokenizer

    Andrej Karpathy · 5 개의 HN 추천

    Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

  7. 07

    Let's build GPT: from scratch, in code, spelled out.

    Andrej Karpathy · 14 개의 HN 추천

    Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.