clipperhouse/uax29

A tokenizer based on Unicode text segmentation (UAX #29), for Go. Split graphemes, words, sentences.

3 best points · 0 comments · 3 Hacker News posts · 125 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Unicode Text Segmentation for Go (3 points, 0 comments, by mwsherman, 2020-04-22)
• Unicode Text Segmentation for Go (2 points, 0 comments, by mwsherman, 2020-05-21)
• Fast Unicode text segmentation for Go (2 points, 0 comments, by mwsherman, 2020-06-29)