Danial-ui6/REYTHUN-IAA-SAMPLE-

RYTHUN — Sample work: quality report for a 200-item RLHF harmlessness preference dataset. 3 annotators, 84% observed agreement, Fleiss' Kappa 0.68, documented adjudication. Methodology and sanitized samples only; full dataset under NDA.

2 best points · 1 comments · 1 Hacker News post · 0 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: A 200-item RLHF quality audit with Fleiss' Kappa (0.68) (2 points, 1 comments, by cipernex, 2026-10-23)