Modelpublic

merileijona/quantumgpt-124m-v3

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes94downloads

Model Card

QuantumGPT-124M-v3 (ctx512)

Third-generation QuantumGPT: a GPT-2 small (124M) model fine-tuned to generate OpenQASM 2.0 circuits from natural-language descriptions, with the context window extended from 256 to 512 tokens via position-embedding extension.

QSS-Core v0.1: 97.70 / 97.28 (two-seed replication) — up from 91.61 (v2) and 75.00 (v1 baseline) on the same 120-sample stratified development benchmark.

Model details


Architecture	GPT-2 small (12 layers, 12 heads, 768 embd), tied embeddings
Context window	512 tokens (extended from 256; positions 0–255 preserved, 256–511 initialized as continuation of learned embedding table)
Parameters	~124M
Lineage	`quantumgpt-124m-v2` → +dataset/format mutations (QEXP-100006) → +ctx512 extension & full-context training (this model)
Training data	merileijona/quantum-circuits-21k (validated OpenQASM 2.0 description→circuit pairs) with format-anchor oversampling ×2, long-circuit oversampling ×2, deterministic paraphrase augmentation
Tokenizer	GPT-2 BPE
Prompt format	`<	user	>{description}<	end	>\n<	assistant	>{qasm}<	end	>`

Benchmark results (QSemBench-dev-v0.1, n=120 stratified, greedy decode)

Scoring: qiskit qasm2 parse/compile + Aer statevector fidelity ≥ 0.99 vs reference circuit, entanglement check where reference entangles, prompt-perturbation robustness probes. Full methodology in the technical report below.

Metric	v1 baseline	v3 (seed 42)	v3 (seed 43)
qsyntax	95.00	99.17	99.17
qcompile	95.00	99.17	99.17
qsemantic	76.67	97.50	96.67
qexecute	76.67	97.50	96.67
qgeneralize	75.26	96.91	96.91
qrobust	90.22	97.44	98.28
qefficiency	100.00	100.00	100.00
QSS-Core v0.1	77.91*	97.70	97.28

*v1 baseline reported under pre-cap-lift conditions; see report for details.

Capabilities (qualitative evaluation)

Strong on: Bell/GHZ-style entanglement, teleportation structure, Deutsch-Jozsa architecture, three-qubit error-correction encoding, mid-circuit measurement with classical control (if(c==1)), parameterized rotations, novel state preparations adjacent to training distribution (e.g., plausible W-state synthesis with correct arccos(1/√3) angles).

Weak on: negation constraints ("WITHOUT any entangling gates" ignored), compound multi-part instructions (partial completion), physics-level self-checking (no-cloning request produces a circuit instead of refusal), casual/typo-heavy phrasing degrades semantic accuracy.

Limitations

—Development benchmark shares its generator with the training corpus; truly held-out evaluation requires newly generated circuits.
—Trained exclusively on synthetic Grok-generated circuits validated by qiskit parse checks.
—No alignment/refusal behavior: any prompt returns some QASM program, meaningful or not.
—512-token context is still limiting for large-scale algorithms (Shor-9 encoding sits at the edge).
—English prompts only.

Intended use

Research on small-scale code-generation models for quantum computing, education, and as a base for further scaling experiments. Not intended for production quantum-circuit synthesis without human verification of every generated circuit.

Reproducibility

Training and evaluation harness: deterministic orchestrator with sealed experiment packets. Key artifacts: champion lineage QEXP-100006 → QEXP-200042/200043; replication spread between seeds: 0.41 QSS points. See technical report in this repository for full experiment history.