manifesta/scientific-chart-qa-lora-qwen3.5-9b

scientific-chart-qa-lora-qwen3.5-9b

<p align="center"> <a href="https://huggingface.co/datasets/manifesta/scientific-chart-qa-17k"><img alt="dataset" src="https://img.shields.io/badge/dataset-scientific--chart--qa--17k-ffcc4d?logo=huggingface&logoColor=black"></a> <a href="https://github.com/A1VARA5/scientific-chart-qa-17k"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-pipeline%20%2B%20verifier-181717?logo=github"></a> <a href="https://github.com/A1VARA5/scientific-chart-qa-17k/actions/workflows/verify.yml"><img alt="verify" src="https://github.com/A1VARA5/scientific-chart-qa-17k/actions/workflows/verify.yml/badge.svg"></a> <img alt="base" src="https://img.shields.io/badge/base-Qwen%2FQwen3.5--9B-6b46c1"> <img alt="rows" src="https://img.shields.io/badge/trained%20on-6%2C976%20rows-0b0b0b"> <img alt="eval" src="https://img.shields.io/badge/win%20rate-51%20vs%2049-1baf7a"> </p>

A LoRA adapter for Qwen/Qwen3.5-9B, trained on scientific chart question answering. Produced by an Adaption Labs AutoScientist run for the AutoScientist Part 2 challenge.

This repository contains a LoRA adapter. It is not a merged model and it is not a standalone model. There are no base model weights here. The 15.7 MB adapter_model.safetensors file holds 3,932,160 LoRA parameters and nothing else. You must load the base model separately and apply this adapter on top of it with peft.

What you need to load it

The exact base model string is:

Qwen/Qwen3.5-9B

A note on the base model string, read this before you file an issue

Adaption's exporter wrote this into adapter_config.json, and it shipped that way until 2026-08-13:

"base_model_name_or_path": "togethercomputer/Qwen3.5-9B"

That repo id does not resolve on the Hugging Face Hub, anonymously or with a valid token, so peft could not auto-resolve a base model from it.

Corrected 2026-08-13. The file now reads:

"base_model_name_or_path": "Qwen/Qwen3.5-9B",
"original_base_model_name_or_path": "togethercomputer/Qwen3.5-9B"

I kept the original exported string verbatim in the same file rather than deleting it, so the export stays auditable and you can see exactly what changed. No weight was touched.

curl -s -o /dev/null -w "%{http_code}
" https://huggingface.co/api/models/Qwen/Qwen3.5-9B                # 200
curl -s -o /dev/null -w "%{http_code}
" https://huggingface.co/api/models/togethercomputer/Qwen3.5-9B     # 401

The base is Qwen/Qwen3.5-9B, and here is the evidence beyond the string itself. The evidence is the config.json shipped alongside the adapter in this repo. Flattening both configs and comparing all 66 keys, every architecture-defining value is identical:

The only differences are serialization and training-time fields (transformers_version, torch_dtype, use_cache, explicit token ids). The LoRA tensor shapes agree with this too: lora_A is [16, 4096] against hidden_size 4096, and q_proj.lora_B is [8192, 16], which is 16 heads times 256 head_dim times the 2x width of this model's gated attention output.

So: pass Qwen/Qwen3.5-9B explicitly when you load, and everything lines up.

How to load it

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

BASE = "Qwen/Qwen3.5-9B"
ADAPTER = "manifesta/scientific-chart-qa-lora-qwen3.5-9b"

model = AutoModelForImageTextToText.from_pretrained(
    BASE,
    dtype=torch.bfloat16,
    device_map="auto",
)

# Passing the already-loaded base model object is the safe route. The config now
# points at a base that resolves, but Qwen/Qwen3.5-9B is a large download either way.
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

# The processor is included in this repo, so you can load it from either place.
processor = AutoProcessor.from_pretrained(ADAPTER)

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "url": "https://upload.wikimedia.org/wikipedia/commons/2/26/Line_graph_example.png"},
        {"type": "text", "text": "What is the value on the y axis at x = 3?"},
    ],
}]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

out = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

If you want a single merged checkpoint instead of runtime adapter application:

merged = model.merge_and_unload()
merged.save_pretrained("./chart-qa-qwen3.5-9b-merged")

Merging bakes the adapter into a full-size copy of the base weights. That copy is not distributed here, you produce it yourself.

Tested against peft 0.15.1, which is the version that wrote this adapter. transformers must be new enough to know the qwen3_5 architecture (the export was written by transformers 5.13.0).

LoRA configuration

Taken verbatim from adapter_config.json.

What actually got adapted

Reading the tensor index of adapter_model.safetensors rather than trusting the config prose:

• —64 tensors, all float32, 3,932,160 parameters total.
• —Every tensor is under base_model.model.model.language_model.layers.*.self_attn.*.
• —Only 8 of the 32 decoder layers carry LoRA: layers 3, 7, 11, 15, 19, 23, 27, 31. Those are exactly the full_attention layers. This model interleaves linear attention and full attention with full_attention_interval 4, and the linear attention layers do not expose q_proj/k_proj/v_proj/o_proj, so they were skipped.
• —Zero vision tensors. The vision tower is completely untouched. adapter_config.json carries a long exclude_modules list covering all 27 model.visual.blocks.*. The chart reading behaviour that changed is language-side behaviour conditioned on vision features, not a retrained vision encoder.

Practical consequence: only 8 attention blocks out of 32 layers were adapted, on a 9B base. This is a small intervention, which is consistent with the small measured effect described below.

Training run

Facts below come from trainer_state.json, included in this repo so you can check the curve yourself.

Loss

Training loss, step 1 to step 21:

1.5476  1.4797  1.4729  1.5156  1.4297  1.5247  1.5907  1.5457  1.5095  1.3912  1.5430
0.7522  1.4543  1.5291  1.3928  1.5344  1.5432  1.4104  1.3699  1.4127  1.5413

Evaluation loss, every 4 steps:

Eval loss fell monotonically from 1.4545 to 1.3909 across the run, about 4.4 percent. Training loss did not visibly trend, it stayed in a 1.37 to 1.59 band with one outlier at step 12 (0.7522, a single batch at batch size 1, so it is one example and not a trend). With train_batch_size 1 and 21 steps the per-step training loss is essentially a per-example reading and is very noisy. The eval curve is the one worth reading.

21 optimizer steps at batch size 1 is 21 examples of gradient signal against a 6,976 row dataset. This is a short run. Read the results section with that in mind.

Results, stated plainly

Adaption's head to head evaluation compared the adapted model against the unmodified base model:

51 versus 49 out of 100. That is a two point margin. On 100 paired comparisons the one-sigma spread from coin flipping alone is about 5 points, so this result is well inside noise and should be read as no measurable win, not as a win. The eval loss improvement (1.4545 to 1.3909) is real and monotonic, but it did not convert into a preference margin that survives its own error bars.

I am not going to dress this up. What this adapter demonstrates is a clean, completed, fully documented adaptation run with a downward eval curve. It is not a demonstration of a large capability gain on chart QA.

The other two runs in this batch, for context

Three AutoScientist runs were completed. Only this one had its weights exported, so only this one is published. The other two are listed here for honesty about the full picture, not as a claim of anything. No weights are published for them, and no repository exists for them. Do not go looking for one.

The math and code run made the model worse. A peak gradient norm of 744.9 against 0.79 and 4.44 on the two chart runs is a roughly three orders of magnitude difference and is the obvious thing to look at first when explaining why. That run is reported here because suppressing a negative result would make the two positive-ish numbers meaningless.

Intended use and limits

Intended use: research on parameter efficient adaptation for chart and scientific figure question answering, and as a reproducible artifact for the AutoScientist Part 2 submission. Load it, read trainer_state.json, check the numbers above against it.

Limits, all of them real:

• —The measured preference win is inside noise. Do not deploy this expecting better chart reading than plain Qwen/Qwen3.5-9B.
• —21 optimizer steps at batch size 1. Under-trained by any normal standard.
• —Vision tower untouched, so nothing here improves figure perception itself.
• —Only 8 of 32 layers adapted, attention projections only, no MLP.
• —English only. Training data is English scientific chart QA.
• —Inherits every limitation, bias and failure mode of Qwen/Qwen3.5-9B. Chart QA models in general will confidently misread axis values, mislabel series and invent gridline numbers. Nothing in this adapter fixes that, and a 51 to 49 preference margin certainly does not.
• —No safety tuning of any kind was performed.

Files in this repository

Training data

manifesta/scientific-chart-qa-17k on the Hugging Face Hub, and the same dataset on Kaggle. The Adaption side ingested 6,976 rows into dataset 46b5c093-a9e5-45a0-9265-412b003c1d28 for this specific run.

Everything behind these weights is public

The dataset this adapter was trained on, the scripts that built it, and a verifier that rechecks every number claimed here against the live artifacts:

<https://github.com/A1VARA5/scientific-chart-qa-17k>

git clone https://github.com/A1VARA5/scientific-chart-qa-17k
cd scientific-chart-qa-17k
python verify.py

Standard library only, no install step and no account. 16 checks, and the same 16 run on a daily schedule in GitHub Actions, so the badge above goes red if any claim on this card stops being true. The tensor facts in this card are among the checks: layer coverage is re-derived from the published adapter_model.safetensors by name, not copied from the config.

Dataset: <https://huggingface.co/datasets/manifesta/scientific-chart-qa-17k>

License

The adapter weights are released under Apache 2.0. The base model Qwen/Qwen3.5-9B carries its own license, which applies to you separately whenever you load the base, and it is your responsibility to comply with it.

Citation

@misc{navardauskas2026chartqalora,
  title  = {scientific-chart-qa-lora-qwen3.5-9b: a LoRA adapter for scientific chart question answering},
  author = {Navardauskas, Aivaras},
  year   = {2026},
  note   = {Adaption Labs AutoScientist Part 2. Trained on Adaption dataset 46b5c093-a9e5-45a0-9265-412b003c1d28.},
  url    = {https://huggingface.co/manifesta/scientific-chart-qa-lora-qwen3.5-9b}
}