yhhugging/DualMLC-eurlex-4k

DualMLC — EUR-Lex-4K

Dataset-specific checkpoint for LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification by Hui Ye, Jing Zhang, Xiulong Yang, and Rajshekhar Sunderraman.

• —Paper: https://arxiv.org/abs/2609.12915
• —Code: https://github.com/huiyegit/DualMLC
• —Paper page: https://huggingface.co/papers/2609.12915

DualMLC processes a document with a Qwen2.5-7B branch adapted using LoRA and a fully fine-tuned BERT branch. Each branch has its own classifier head. The final label ranking uses late logit fusion; the architecture, label count, and fusion weight are recorded in config.json.

Checkpoint contents

config.json
qwen_lora_adapter/      # Qwen LoRA adapter; not the full 7B base model
bert_encoder/          # full fine-tuned BERT encoder
qwen_tokenizer/
bert_tokenizer/
head_qwen.pt
head_bert.pt
best_metrics.json      # if included in the original release
labels.txt             # optional label names, in training label-index order

The Qwen base model is downloaded separately from Qwen/Qwen2.5-7B. Use the DualMLC loader below to reconstruct both branches and classifier heads.

Download and predict

Create the environment following the GitHub setup instructions, then run from the source repository:

git clone https://github.com/huiyegit/DualMLC.git
cd DualMLC
git checkout de43c6b11a85b6d641ae8dc468930df40dac96bd
pip install huggingface_hub

Download the checkpoint:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="yhhugging/DualMLC-eurlex-4k",
    local_dir="models/dualmlc-eurlex-4k",
)

Predict on a document:

python test.py \
    --ckpt models/dualmlc-eurlex-4k \
    --qwen-name Qwen/Qwen2.5-7B \
    --no-eval \
    --text "A document to classify." \
    --predict-topk 5

Predictions are label IDs in this checkpoint's training label order. If you have the corresponding label vocabulary, pass --label-file path/to/labels.txt, with one label name per line. Label IDs are not interchangeable across datasets or dataset preprocessing variants.

To use the existing Python loader:

from model import load_checkpoint

model, qwen_tokenizer, bert_tokenizer, arch = load_checkpoint(
    "models/dualmlc-eurlex-4k",
    device="cuda",
    qwen_name="Qwen/Qwen2.5-7B",
)

Evaluation

These values are recorded in the released checkpoint's best_metrics.json at step 7000, not a new evaluation. The GitHub Model Zoo reports P@1=88.82, P@3=75.80 and P@5=62.74; its P@3 and P@5 differ from this checkpoint's recorded values. The reported training time is 0.45 hours on 8 × RTX 4090 with CUDA 11.8. Refer to the source repository for the dataset preparation and dataset-specific training command.

To evaluate, download the matching preprocessed dataset as described in the source repository, then run:

python test.py \
    --ckpt models/dualmlc-eurlex-4k \
    --qwen-name Qwen/Qwen2.5-7B \
    --data-dir xmc-base/eurlex-4k

Intended use and limitations

This checkpoint ranks labels from the fixed EUR-Lex-4K label vocabulary for multi-label text classification research. It does not generate text or accept an arbitrary new label vocabulary. Inputs are truncated to the maximum lengths in config.json; performance on other domains and languages has not been established by these reported results. A GPU with sufficient memory for the Qwen base model and both branches is recommended. The existing inference code reports sigmoid-transformed scores; these are not documented as calibrated probabilities.

License

The publisher declares the checkpoint license as apache-2.0. The source repository's Apache 2.0 license originally covers code only. The base models and benchmark datasets have their own licenses and terms; see Qwen2.5-7B and BERT for upstream information.

Citation

@article{ye2026dualmlc,
  title={LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification},
  author={Ye, Hui and Zhang, Jing and Yang, Xiulong and Sunderraman, Rajshekhar},
  journal={arXiv preprint arXiv:2609.12915},
  year={2026}
}