cjk
Datasets
All datasets matching “cjk”wiki_cjk_ocrcode-romance-cjk-wiki
Dataset Card for "code-romance-cjk-wiki"
More Information needed
daily-paper-2026-08-15-cjk-skill-router-blind-spots
Script-Blind Retrieval: Quantifying CJK Trigger Blind Spots in BM25-Style Skill Routers
TL;DR — BM25-style skill routers with ASCII-only tokenizers fail completely for CJK-script queries, but a one-line tokenizer extension recovers 95% accuracy — revealing that principled score normalization can paradoxically collapse gated recall from 95% to 19% by rescaling scores below the injection threshold.
ThakiCloud AI Research · 2026-08-15 · 📝 Tech blog (KO)
Problem… See the full description on the dataset page: https://huggingface.co/datasets/thaki-AI/daily-paper-2026-08-15-cjk-skill-router-blind-spots.cjk-tokenizer-from-scratch
CJK Tokenizer From Scratch
Offline research dump for building Chinese / Japanese tokenizers from scratch.
Download tokenizer.zip (≈464 MB), then:
unzip tokenizer.zip
Contents:
papers/ — 41 free arXiv / ACL PDFs (BPE, WordPiece, Unigram, SentencePiece, Chinese SubChar, Japanese MeCab/Sudachi bake-off, …)
github/ — 23 shallow-cloned repos (minbpe, sentencepiece, tokenizers, jieba, mecab, Sudachi, …)
docs/ — Hugging Face NLP course chapter 6 (train BPE / WordPiece / Unigram)… See the full description on the dataset page: https://huggingface.co/datasets/aicrazy124/cjk-tokenizer-from-scratch.huggingface-cjk-metadata
Dataset Card for HuggingFace-CJK-Metadata
Dataset Summary
This dataset provides structured metadata and documentation extracted from the top 700 most downloaded datasets per language on the Hugging Face Hub for Chinese (zh), Japanese (ja), Korean (ko), and English (en, as a reference). The collection includes both high-level metadata (e.g., size, license, task type) and raw dataset card contents, enabling large-scale, cross-linguistic analysis of data curation… See the full description on the dataset page: https://huggingface.co/datasets/Dasool/huggingface-cjk-metadata.cjk_glyph