AItonomy/PhAI-IDE-4B

1---2model_name: PhAI-IDE-4B3base_model: Qwen/Qwen3.5-4B4base_model_relation: finetune5library_name: transformers6pipeline_tag: image-text-to-text7license: apache-2.08tags:9- phai-ide10- science11- code12- tool-use13- sft14- lora15- safetensors16---17 18# PhAI-IDE19 20**PhAI-IDE** is a family of models for scientific coding and interaction with tools, available in **4B, 9B, and 72B** sizes. Each model is supervised fine-tuned with [ms-swift](https://github.com/modelscope/ms-swift) and released as full BF16 weights with the final LoRA adapter merged, together with its configuration and tokenizer.21 22The **training dataset is Codex trajectories**, sourced from [ScienceIDE](https://github.com/aitofound/ScienceIDE).23 24## Models25 26| Model | Base model | BF16 weights | License |27| --- | --- | ---: | --- |28| [PhAI-IDE-4B](https://huggingface.co/AItonomy/PhAI-IDE-4B) | [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) | 9.08 GB | Apache-2.0 |29| [PhAI-IDE-9B](https://huggingface.co/AItonomy/PhAI-IDE-9B) | [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) | 18.82 GB | Apache-2.0 |30| [PhAI-IDE-72B](https://huggingface.co/AItonomy/PhAI-IDE-72B) | [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) | 145.41 GB | [Qwen](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/495f39366efef23836d0cfae4fbe635880d2be31/LICENSE) |31 32Weight sizes are approximate; inference also requires memory for runtime allocations and the KV cache.33 34## ScienceAccelBench performance35 36Task-held-out, localized scientific-code repair on familiar codebases, with original numerical verification. Qwen3.5-4B and Qwen3.5-9B are compared with PhAI-IDE-4B and PhAI-IDE-9B, respectively, on identical tasks. Pass rates are percentages; gains are percentage points.37 38| Size | Environment | Tasks | Qwen3.5 | PhAI-IDE | Gain (pp) |39| :---: | --- | ---: | ---: | ---: | ---: |40| 4B | PLUTO-Particles-Dust | 3 | 0.00 | **33.33** | **+33.33** |41| 9B | LAPS | 16 | 31.25 | **50.00** | **+18.75** |42| 9B | MITgcm-biogeo | 8 | 0.00 | **12.50** | **+12.50** |43| 9B | PLUTO-RMHD | 7 | 0.00 | **28.57** | **+28.57** |44 45## Comparison with published models46 47Scores (%), grouped by benchmark and model size. Each reference entry gives its published score and the **PhAI-IDE score difference in percentage points**. Reference models are approximately the same size: 3–4B, 7–9B, and 67–72B, respectively.48 49| PhAI-IDE | Benchmark | Score | Reference models: score (difference) |50| --- | --- | ---: | --- |51| 4B | BBH multistep-arithmetic-two | **97.60** | [Llama-3.2-3B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (3.21B): 53.2 (**+44.40**); [Phi-3.5-mini-8k-instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (3.82B): 95.6 (**+2.00**) |52| 9B | BBH word-sorting | **60.40** | [Llama-3.1-8B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (8.03B): 51.2 (**+9.20**); [Qwen2.5-7B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (7.62B): 15.6 (**+44.80**) |53| 9B | MATH-500 | **92.20** | [InternLM3-8B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (8B): 83 (**+9.20**); [Qwen2.5-7B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (7B): 72.4 (**+19.80**); [Llama-3.1-8B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (8B): 48.4 (**+43.80**) |54| 72B | AQuA-RAT | **77.56** | [Llama-2-70B-Chat](https://openreview.net/pdf?id=FvfhHucpLd) (70B): 31.32 (**+46.24**) |55| 72B | ARC-Easy | **84.64** | [Llama-2-70B](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (70B): 76.5 (**+8.14**); [DeepSeek-LLM-67B-Chat](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (67B): 81.6 (**+3.04**) |56| 72B | ARC-Challenge | **64.42** | [Llama-2-70B](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (70B): 59.5 (**+4.92**); [DeepSeek-LLM-67B-Chat](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (67B): 64.1 (**+0.32**) |57 58Reference scores come from the linked publications, model cards, and independent evaluation reports; evaluation settings and sample counts vary by source. Differences describe reported scores across evaluations, rather than matched-protocol head-to-head gains. BBH entries refer to the named tasks.59 60## Quick start61 62Use Transformers 5.16.1, PyTorch and Accelerate. Set `model_id` to any model in the table above; the example selects the matching model class.63 64```python65from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForImageTextToText66 67model_id = "AItonomy/PhAI-IDE-4B"68loader = AutoModelForCausalLM if model_id.endswith("72B") else AutoModelForImageTextToText69tokenizer = AutoTokenizer.from_pretrained(model_id)70model = loader.from_pretrained(model_id, dtype="bfloat16", device_map="auto")71inputs = tokenizer.apply_chat_template(72    [{"role": "user", "content": "Explain how to verify a numerical simulation."}],73    add_generation_prompt=True, enable_thinking=False, return_dict=True, return_tensors="pt",74).to(model.device)75output = model.generate(**inputs, max_new_tokens=128, do_sample=False)76print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))77```78 79## Training procedure80 81ScienceIDE demonstrations were collected with **GPT-5.6-sol** and filtered using a **numerical-equivalence verifier**. They capture code inspection, tool use, and responses to execution feedback.82 83All three models use **ms-swift** supervised fine-tuning with **LoRA across trainable linear layers for three epochs**. The release merges each final checkpoint's adapter into its base model. Retained assistant targets provide the next-token training signal, while conversation history and tool observations provide context. The trajectories retain the native `exec` / `wait` interaction format. Heuristic target masking selects assistant actions for supervision while preserving the surrounding interaction history.84 85| Shared setting | Value |86| --- | --- |87| Training dataset | Codex trajectories |88| Training examples / tasks | 4,567 segments / 564 tasks |89| Validation examples / tasks | 544 segments / 81 tasks |90| Train/validation task overlap | 0 |91| Training epochs | 3 |92| LoRA rank / alpha / dropout | 32 / 64 / 0.05 |93| Released weights | LoRA merged into BF16 Safetensors |94 95Long trajectories are organized into segments. Source partition assignments are preserved, with no task identifiers shared between training and validation.96 97### Framework versions98 99The release was validated with the following environment.100 101| Component | Version |102| --- | --- |103| Python | 3.11 |104| ms-swift | 4.5.3 |105| Transformers | 5.16.1 |106| PyTorch | 2.6.0+cu124 |107| PEFT | 0.20.0 |108| Datasets | 4.8.4 |109| Tokenizers | 0.23.2 |110| Accelerate | 1.14.0 |111