Agent skill · davila7
sentencepiece
Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deter
Reference it in any coding agent with:
@skills davila7/sentencepiece