Open source real-time translation app for Android that runs locally

Fast and customizable text tokenization library with BPE and SentencePiece support

Minimal example of using a traced huggingface transformers model with libtorch

R package for Byte Pair Encoding / Unigram modelling based on Sentencepiece

sentencepiece C# wrapper

Sentencepiece Dart is a wrapper for Google's Sentencepiece C++ library modified

Temp fork to provide Python 3.13 macOS wheels ahead of official project releases

Unsupervised text tokenizer for Neural Network-based text generation.