belladoreai/llama-tokenizer-js

JS tokenizer for LLaMA 1 and 2

115 best points · 23 comments · 1 Hacker News post · 364 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: LLaMA tokenizer that runs in browser (115 points, 23 comments, by belladoreai, 2023-06-13)