AmanPriyanshu/Stratified-LLM-Subsets-100K-1M-Scale

Stratified LLM Subsets delivers diverse training data at 100K-1M scales across pre-training (FineWeb-Edu, Proof-Pile-2), instruction-following (Tulu-3, Orca AgentInstruct), and reasoning distillation (Llama-Nemotron). Embedding-based k-means clustering ensures maximum diversity across 5 high-quality open datasets.

2 best points · 1 comments · 1 Hacker News post · 1 GitHub stars · View on GitHub

Hacker News discussions

• Diverse LLM subsets via k-means (100K-1M) [Pretraining, IF, Reasoning] (2 points, 1 comments, by radii-llm, 2025-10-04)