sweepai/bpe-qwen

A blazing-fast BPE tokenizer for Qwen models built with Rust

5 best points · 0 comments · 1 Hacker News post · 29 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Rust BPE tokenizer for Qwen models that's 12x faster than HuggingFace (5 points, 0 comments, by williamzeng0, 2025-09-30)