Tanuki-8x8B-dpo-v1.0 / tokenizer.json
kanhatakeyama's picture
トークナイザーに全角記号・数字のNKFC正規化を追加 (#2)
88992a1 verified
raw
history contribute delete
4.26 MB
File too large to display, you can check the raw version instead.