upul/Build-GPT-2

GPT-2 124M trained from scratch in PyTorch — explicit data sharding, DDP, resumable checkpoints, HellaSwag evaluation and W&B logging, in ~1,600 lines.

1 best points · 0 comments · 1 Hacker News post · 2 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: GPT-2 124M trained from scratch – 2.48B tokens, 2.8h on 4 A40s (1 points, 0 comments, by upulbandara, 2026-10-15)