NLPOptimize/flash-tokenizer

EFFICIENT AND OPTIMIZED TOKENIZER ENGINE FOR LLM INFERENCE SERVING

5 best points · 0 comments · 1 Hacker News post · 457 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: FlashTokenizer – 10x faster C++ tokenizer for Python (5 points, 0 comments, by springkim, 2025-04-03)