Modelpublic

zymonody/chinese-babylm-v3

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes1.6kdownloads

Model Card

Chinese BabyLM 2026 submission (v3)

A Chinese masked language model for the NLPCC 2026 Chinese BabyLM Challenge, pretrained within the ≤102M Jieba-word data budget.

—Architecture: DeBERTa-v2 base (12 layers, hidden 768, 12 heads), ~102M params
—Tokenizer: character-level
—Objective: masked language modeling

Methodology details will be released in the technical report after the evaluation period.

Usage

python

from transformers import AutoTokenizer, AutoModelForMaskedLM
tok = AutoTokenizer.from_pretrained("zymonody/chinese-babylm-v3")
model = AutoModelForMaskedLM.from_pretrained("zymonody/chinese-babylm-v3")