Modelpublic

cometadata/citation-parsing-lora-qwen3-0.6b

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

2likes

Model Card

Citation Parsing LoRA

This model is a fine-tuned version of Qwen/Qwen3-0.6B trained using Group Relative Policy Optimization (GRPO) for parsing and annotating bibliographic citations in JATS XML format.

Model Description

—Base Model: Qwen3-0.6B (601M parameters)
—Training Method: Group Relative Policy Optimization (GRPO) with LoRA
—Task: Bibliographic citation parsing and XML annotation
—Training Data: Citation parsing dataset with plaintext citations and corresponding JATS XML annotations

Training Details

Training Configuration

—Training Algorithm: GRPO with Direct Reward optimization (dr_grpo)
—Learning Rate: 1e-5 with cosine scheduler
—Training Epochs: 0.1 epochs completed
—Batch Size: 1 per device, 8 gradient accumulation steps
—LoRA Configuration:
—Rank (r): 8
—Alpha: 32
—Target modules: qproj, vproj, kproj, oproj, gateproj, upproj, down_proj

Training Metrics

—Total Training Steps: 435
—Total Tokens Processed: 41,835,117
—Training Loss: 0.007
—Mean Reward: 1.64
—Answer Reward: 0.68 ± 0.21
—Format Reward: 0.96 ± 0.15

Hardware

—GPUs: 8x NVIDIA H100 80GB HBM3
—Training Time: ~7.2 hours (26,008 seconds)
—Precision: bfloat16

Reward Functions

The model was trained with two reward functions:

1.Format Reward: Checks if the generated output contains proper thinking tags (<think>...</think>) and valid XML structure
2.Answer Reward: Evaluates the accuracy of extracted bibliographic fields using edit distance thresholds

Usage

The model expects input in a specific format with system and user messages. It generates responses with chain-of-thought reasoning followed by JATS XML annotations.

System Prompt

You are a helpful assistant with an expertise in annotating bibliographic references in JATS XML format.

You will be given a plaintext citation and you should respond with the annotated reference. Before providing the final XML annotation, give your step-by-step thinking. After your explanation, provide the final XML annotation.

# Examples

### Citation
Stone NJ, Robinson JG, Lichtenstein AH, et al. 2013 ACC/AHA Guideline on the Treatment of Blood Cholesterol to Reduce Atherosclerotic Cardiovascular Risk in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;129(25 Suppl 2):S1-S45. doi:10.1161/01.cir.0000437738.63853.7a.

### Annotation
<mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Stone</surname> <given-names>NJ</given-names></string-name>, <string-name><surname>Robinson</surname> <given-names>JG</given-names></string-name>, <string-name><surname>Lichtenstein</surname> <given-names>AH</given-names></string-name>, <etal>et al</etal></person-group>. <article-title>2013 ACC/AHA Guideline on the Treatment of Blood Cholesterol to Reduce Atherosclerotic Cardiovascular Risk in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines</article-title>. <source><italic>Circulation</italic></source> <year>2014</year>;<volume>129</volume>(<issue>25</issue> <supplement>Suppl 2</supplement>):<fpage>S1</fpage>-<lpage>S45</lpage>. <comment>doi</comment>:<pub-id pub-id-type="doi">10.1161/01.cir.0000437738.63853.7a</pub-id>.</mixed-citation>

### Citation
Sagel Z, Tutluer Mİ, Peskircioglu H, et al.: Determination of Effect of Chemical Mutagen EMS on TAEK A-3 and TAEK C-10 Mutant Soybean Varieties in M1 Generation. Ekin Journal of Crop Breeding and Genetics. 2017; 3(1): 19–24. Reference Source

### Annotation
<mixed-citation publication-type="journal"><person-group person-group-type="author"><name name-style="western"><surname>Sagel</surname> <given-names>Z</given-names></name>, <name name-style="western"><surname>Tutluer</surname> <given-names>Mİ</given-names></name>, <name name-style="western"><surname>Peskircioglu</surname>, <given-names>H</given-names></name>, <etal /></person-group>:<article-title>Determination of Effect of Chemical Mutagen EMS on TAEK A-3 and TAEK C-10 Mutant Soybean Varieties in M<sub>1</sub> Generation.</article-title>. <source><italic toggle="yes">Ekin Journal of Crop Breeding and Genetics</italic></source>. <year>2017</year>;<volume>3</volume>(<issue>1</issue>):<fpage>19</fpage>–<lpage>24</lpage>. <ext-link>Reference Source</ext-link></mixed-citation>

Expected Output Format

<think>
[reasoning about the citation parsing]
</think>

[JATS XML annotation]