Agent skill · zechenzhangagi
sentencepiece
Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deter
Reference it in any coding agent with:
@skills zechenzhangagi/sentencepiece