google/sentencepiece

Unsupervised text tokenizer for Neural Network-based text generation.

2 best points · 0 comments · 1 Hacker News post · 12,168 GitHub stars · View on GitHub

Hacker News discussions