lyrain2001/Auto-Fill-Qwen3-8B-Reasoning

Auto-Fill Reasoning Specialist (Qwen3-8B)

The reasoning specialist of Auto-Fill: Learning to Predict Missing Values Accurately with Specialist Language Models (PVLDB 19(11), 2026 — arXiv:2607.19847). Given a table with one cell marked [MISSING], this model reasons step by step before answering and reports a verbalized confidence; it targets cells that require inferring implicit patterns or relationships across rows and columns.

Auto-Fill runs three specialists in parallel — knowledge, reasoning and coding — calibrates their confidences with isotonic regression and returns the most confident answer (or abstains). Sibling specialists: Knowledge · Coding.

• —Code: https://github.com/lyrain2001/auto-fill
• —Benchmark: lyrain2001/Auto-Fill-Benchmark
• —Calibrators: calibrators.json in this repo (also in the code repository under checkpoints/)

Model details

Training tables come from public sources only (spreadsheets crawled from a search-engine index, public BI models, Wikipedia, nationalarchives.gov.uk, GitHub CSV/Parquet files); one cell per table is masked and its original value is the target.

Prompt and output format

The table is serialized as a Markdown pipe table (pandas.DataFrame.to_markdown(index=False, tablefmt="pipe")) with the cell to fill written as [MISSING]. The user message is exactly (see autofill/utils/prompts.py):

Please fill in the missing value in the input table and provide your confidence level as an integer between 0 (no confidence) and 100 (full confidence). The missing value is denoted by '[MISSING]'. Please return the value filled in JSON format: {"value": "filled_value", "confidence": confidence_level}.

Input Table:
<markdown table>

Expected output: <think> … </think> followed by {"value": "<filled value>", "confidence": <0-100>}.

Usage

With the code repository (recommended) — runs the full ensemble on one table:

python inference/run_specialists.py \
    --table          /path/to/table.csv \
    --knowledge_path lyrain2001/Auto-Fill-Qwen3-8B-Knowledge \
    --reasoning_path lyrain2001/Auto-Fill-Qwen3-8B-Reasoning \
    --coding_path    lyrain2001/Auto-Fill-Qwen3-8B-Coding \
    --calibrators    checkpoints/calibrators.json \
    --gpu_ids        0,1,2

or this specialist alone on the benchmark:

python inference/run_benchmark.py --mode reasoning --model_path lyrain2001/Auto-Fill-Qwen3-8B-Reasoning \
    --dataset Gov-CSV --benchmark Auto-Fill-Benchmark/sample200 --gpu_ids 0

Minimal vLLM example

import pandas as pd
from vllm import LLM, SamplingParams

llm = LLM(model="lyrain2001/Auto-Fill-Qwen3-8B-Reasoning", dtype="bfloat16", max_model_len=40960)
table = pd.read_csv("table.csv", dtype=str).to_markdown(index=False, tablefmt="pipe", disable_numparse=True)
prompt = PROMPT + table   # PROMPT = the user message above, up to and including "Input Table:\n"
text = llm.get_tokenizer().apply_chat_template(
    [{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)
out = llm.generate([text], SamplingParams(temperature=0.8, max_tokens=32768))
print(out[0].outputs[0].text)

Results

Recall@Precision=0.9 on the Auto-Fill benchmark (200 cases per dataset; from the paper's specialist ablation):

\* Ent-CSV / Ent-XLS are proprietary enterprise datasets that are not part of the public benchmark.

Limitations

• —Trained and evaluated on English-language tables with one missing cell per table; tables were serialized with at most 40,960 tokens.
• —The model can be wrong with high confidence on cells that require knowledge outside the table; use the calibrated confidence and abstain below a threshold, as in the paper.
• —Generated code (coding specialist) should be executed in a sandbox.

Citation

@article{liu2026autofill,
  title={Auto-Fill: Learning to Predict Missing Values Accurately with Specialist Language Models},
  author={Liu, Yurong and He, Yeye and Dong, Haoyu and Xing, Junjie and Han, Shi and Zhang, Dongmei and Chaudhuri, Surajit},
  journal={Proceedings of the VLDB Endowment},
  volume={19},
  number={11},
  pages={3160--3173},
  year={2026}
}