openai/evals

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

123 best points · 22 comments · 5 Hacker News posts · 19,710 GitHub stars · View on GitHub

Hacker News discussions

• Evals: a framework for evaluating OpenAI models and a registry of benchmarks (123 points, 16 comments, by tosh, 2023-03-14)
• Example PR to OpenAI evals to get GPT-4 early API access (3 points, 0 comments, by harrisonjackson, 2023-03-14)
• GPT-4 doesn't pay close attention to detail in some cases (3 points, 2 comments, by mcaledonensis, 2023-03-18)
• OpenAI crowd sources LLM benchmarking datasets by offering advanced GPT-4 access (1 points, 2 comments, by teaearlgraycold, 2023-03-15)
• Tell HN: OpenAI charges you even if you're helping them on their Eval plarform (1 points, 2 comments, by behnamoh, 2023-03-19)