Papajams/orbura-dataviz-mistral-7b-autoscientist

Orbura AutoScientist Data Visualization Model (Mistral 7B)

Model Details

• —Base model: mistralai/Mistral-7B-Instruct-v0.2
• —Fine-tuning method: LoRA (r=16, alpha=32) via AutoScientist co-optimization
• —Dataset: Papajams/orbura-dataviz-augmented (5,000 rows with reasoning traces)
• —Task: Data visualization — matplotlib code generation, chart QA, code-to-description, style transfer, bug repair, chart-type selection
• —Competition: AutoScientist Challenge Part 2 — Data Visualization

Training

• —Train examples: 5,000 (augmented with reasoning traces via Adaption Adaptive Data)
• —Validation examples: 2,000 (deterministic, held-out)
• —Iterations: 3 (AutoScientist co-optimization loop)
• —Best win rate: 53.77%
• —LoRA rank: 16, alpha: 32
• —Learning rate: 1e-5, cosine schedule
• —Epochs: 1
• —Training type: LoRA
• —Batch size: max (platform-selected)
• —Compute: Free via Adaption voucher

Evaluation

Naive baseline: 17.6% on 2k validation set.

Run the eval harness:

cd showcases/autoscientist-dataviz
python3 eval_finetuned.py --baseline-only

Scoring metrics (per-task deterministic evaluation):

Usage

This is a LoRA adapter. Load it with PEFT:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
model = PeftModel.from_pretrained(base, "Papajams/orbura-dataviz-mistral-7b-autoscientist")
tokenizer = AutoTokenizer.from_pretrained("Papajams/orbura-dataviz-mistral-7b-autoscientist")

messages = [{"role": "user", "content": "Answer the question using the chart data.\n\n- A: 45\n- B: 78"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Limitations

• —Trained on synthetic chart data with 4 chart types (line, bar, scatter, pie). Performance on novel chart types (boxplot, heatmap, violin) is not validated.
• —The text-only dataset does not test visual chart reading. The multimodal pilot (100 rows) begins to address this but is too small for full training.
• —Always validate generated matplotlib code before execution.

Reproducibility

# 1. Generate the dataset
cd showcases/autoscientist-dataviz
python3 generate_full.py
python3 prepare_adaption.py

# 2. Run Adaption augmentation
python3 run_adaption.py --input data/adaption_train_5k.jsonl

# 3. Train via AutoScientist (adaptionlabs.ai/auto-scientist)
#    Upload the augmented dataset, select Mistral-7B, run co-optimization

# 4. Evaluate
python3 eval_finetuned.py --baseline-only

# 5. Publish
python3 publish_hf.py --model path/to/model

License

Apache 2.0 (base model license applies). Dataset and model generated for the AutoScientist Challenge by Adaption Labs.