The spelled-out intro to language modeling: building makemore

Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.

HN 推薦的原因： HN recommends it as a long but worthwhile way to demystify how language models work technically.

“It's long (2h), but well worth the time if anyone is curious about the technical workings of LLMs.” 閱讀 HN 討論 ↗