Apppublic

Mnttnt21/tokenizer-bench

sourceHugging Faceupdated 4mo agoView on Hugging Face

0likes

composer.json17 linesDownload Raw Back to root

1{2    "name": "asolytics/tokenizer-bench",3    "description": "Side-by-side comparison bench for CJK/Thai/Korean PHP tokenizers (Asolytics meta editor)",4    "type": "project",5    "require": {6        "php": ">=8.1",7        "ext-intl": "*",8        "ext-mbstring": "*",9        "fukuball/jieba-php": "*"10    },11    "autoload": {12        "psr-4": {13            "Bench\\": "src/"14        }15    }16}17