Modelpublic

dubito-inc/gan-cao-v3.2-qwen

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes11downloads

Model Card

🌿 Gan Cao v3.2

A 229M Parameter Bilingual TCM Model | MASSIF Accelerator Class

![MASSIF Class](https://github.com/Ergo-sum-AGI/massif-llm) ![τ_eff](https://github.com/Ergo-sum-AGI/massif-llm) ![License](https://www.apache.org/licenses/LICENSE-2.0) ![Model Size](https://huggingface.co/dubito-inc/gan-cao-v3.2-qwen) ![Tokenizer](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)

Overview

Gan Cao v3.2 is a 229M parameter bilingual language model fine-tuned for Traditional Chinese Medicine (TCM) and Eastern philosophy. It is built on the MASSIF architecture (Pre-LN + RMSNorm + SwiGLU + RoPE) with a Qwen tokenizer for proper Chinese character handling.

Unlike earlier versions that used GPT-2 tokenizer (which fragmented Chinese characters), v3.2 uses Qwen's tokenizer for accurate Chinese text representation.

Key Improvements from v3

Aspect	v3	v3.2
Tokenizer	GPT-2 (fragments Chinese)	Qwen (proper Chinese tokens)
Parameters	63.5M	229M
d_model	512	768
n_layers	12	16
n_heads	8	12
τ_eff	0.58 (Neutral)	0.68 (Accelerator)
Training Data	85 bilingual pairs	6 streaming datasets (10k steps)

MASSIF Classification

Under the MASSIF taxonomy (Solis, 2026), Gan Cao v3.2 achieves Accelerator Class:

Metric	Value	Threshold
τ_eff	0.68	> 0.55 (Accelerator)
Loss Drop	109 → 38	-65%
Architecture	Pre-LN + RMSNorm + SwiGLU + RoPE	Validated

Evolution Across Versions

Model	Params	τ_eff	Class
Gan Cao v1	63.5M	0.00	Neutral
Gan Cao v2	63.5M	0.58	Neutral
Gan Cao v3	63.5M	0.93*	Accelerator*
Gan Cao v3.2	229M	0.68	Accelerator

*Note: v3 measurement used conversational prompts; v3.2 uses N=50 stress protocol for comparability with taxonomy.

Architecture

Component	Value
d_model	768
n_layers	16
n_heads	12
d_head	64
Vocab Size	151,643 (Qwen)
Context Length	512
Total Parameters	229M
Architecture	Pre-LN + RMSNorm + SwiGLU + RoPE
Position Encoding	RoPE (Rotary Position Embedding)
Activation	SwiGLU

Training Data

Gan Cao v3.2 was trained on 6 streaming datasets (10,000 steps, 8-hour training on NVIDIA T4):

Dataset	Weight	Content
TCM Pretrain (ShizhenGPT)	25%	5B tokens TCM medical texts
Classical Chinese (ctext)	20%	Pre-Qin philosophy, classical grammar
Buddhist Canon	15%	Chinese Buddhist scriptures
Ancient Documents (ByteDance)	15%	Classical → Modern translations
TCM Chat	15%	Clinical TCM conversations
Chinese Poetry	10%	Tang poems (Daoist/Buddhist themes)

Total training composition: 90% medical corpora, 10% philosophical context

Usage

Installation

bash

pip install transformers torch