Modelpublic

Yangfan78/3D_LLM_Diffusion-trimodal-qwen-v11

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes

Model Card

3D LLM Diffusion: Tri-Modal Qwen v11

This release contains the trained components required for the project's text/XRD/crystal shared representation and Qwen-controlled Crystalite runtime. Qwen owns both natural-language boundaries: it parses user requests into auditable crystal conditions and produces grounded descriptions from facts computed from the selected CIF. Crystalite generates structures; deterministic powder diffraction physics generates XRD. The Neutral Composition Planner and charge-balance gate are enabled for de novo generation by default. There is no Robocrystallographer or template-prose runtime path.

Runtime Contract

text

raw text -> Qwen parse + Qwen latent adapter -> v11 256-d shared hub
                                                   |
                                      conditioned Crystalite
                                                   |
                                             selected CIF
                                             /          \
                                  simulated XRD      facts -> Qwen text

The planner fixes a formally charge-feasible composition before Crystalite denoises coordinates and the lattice. This is a generation constraint, not a substitute for structure-aware validity, MLIP relaxation, or DFT validation.

End-to-End Qwen Runtime Validation

Gadi GPU job 174157319 exercised both production text routes with the pinned Qwen revision and exited successfully:

Route	Qwen parse	Crystal anchor	Text output
Retrieval	`formula=NaCl`, `crystal_system=cubic`	exact formula and crystal-system match, MP-20 `mp-22862`	Qwen grounded on saved CIF facts
De novo	`formula=NaCl`, `bandgap=1.1 eV`, `crystal_system=cubic`	two charge-balanced Planner + Crystalite candidates accepted	Qwen grounded on the selected generated CIF

The short 12-step de-novo integration sample is NaCl but has P1 symmetry, so it does not demonstrate cubic-condition accuracy. Parsed conditions, generated CIFs, charge audits, deterministic XRD, grounding facts, and Qwen descriptions are preserved under metrics/qwen_anyio_smoke/. This smoke test validates the runtime wiring; GenBench and property-control evaluations remain the quality evidence.

The Qwen weights are not redistributed here. Use `Qwen/Qwen3.6-35B-A3B` at exact revision 995ad96eacd98c81ed38be0c5b274b04031597b0 with transformers==5.14.1.

Released Components

Component	Purpose
`checkpoints/trimodal_v11/trimodal_best.pt`	final text/XRD/crystal shared encoder
`checkpoints/qwen_latent_adapter/qwen_latent_adapter_best.pt`	pooled Qwen hidden state to final 256-d hub
`checkpoints/generator_v12/best.pt`	multi-property conditioned Crystalite denoiser
`checkpoints/qwen_crystal_clip/clip_best.pt`	base Qwen/crystal alignment checkpoint
`checkpoints/property_encoder/encoder_prop_best.pt`	crystal property encoder
`checkpoints/qwen_property_projector/projector_step003000.pt`	crystal latent to Qwen soft-token projector
`checkpoints/mace_eform/mp20_eform_run-7.model`	energy guidance/evaluation model
`source/source_only_worktree_20260720.tar.gz`	exact CSCS source snapshot used for migration
`source/github_70a66c2e4ce1417cfef378e4c01045d753900d0b.tar.gz`	exact final Qwen runtime source at GitHub commit `70a66c2e`
`manifest/ARTIFACT_MANIFEST.json`	sizes and SHA-256 checksums

Large embedding tables and training/evaluation data are released separately in `Yangfan78/3D_LLM_Diffusion-trimodal-assets-v11`.

Qwen Latent Adapter Validation

The 2048-to-256 adapter was selected at epoch 99 on 9,047 held-out paired texts.

Metric	Value
Mean cosine to frozen v11 target	0.97135
Retrieval R@1	99.36%
Retrieval R@10	100.00%

These metrics measure recovery of the frozen v11 text-hub targets; they are not generated-crystal validity or property scores. The checkpoint includes the exact Qwen revision, environment versions, hyperparameters, row counts, and input/checkpoint SHA-256 hashes.

Tri-Modal Evaluation

The paired MP20 synthetic-XRD test split contains 9,046 examples.

Direction	Recall@10
Text to crystal	0.9667
Crystal to text	0.9423
XRD to crystal	0.9938
Crystal to XRD	0.9992
Text to XRD bridge	0.8737
XRD to text bridge	0.8332
Corrupted XRD to crystal	0.9000

These are retrieval results on simulated diffraction, not measured-XRD or generated-structure property accuracy. Generated candidates still require independent structural, MLIP, and preferably DFT validation.

Planner + ORB-v3 GenBench

The completed all-2,500 run uses the Neutral Composition Planner, Crystalite FK K8 sampling (lambda=4), independent ORB-v3 pre-relaxation, and the pinned LeMat-Bulk reference revision. It is a non-hybrid generated pool.

Metric	Result	Denominator
Official validity	96.88%	2,422 / 2,500
Uniqueness	80.14%	1,941 / 2,422
Novelty	79.15%	1,917 / 2,422
SUN	0.50%	12 / 2,422
SUN + MetaSUN	26.51%	642 / 2,422
Stable after GenBench three-MLIP relaxation	6.28%	152 / 2,422
Mean energy above hull after three-MLIP relaxation	0.1626 eV/atom	2,422
Mean relaxation displacement RMSE	0.1643 A	2,422

The run passes the project's validity, novelty, metastability, and displacement targets. It does not pass the uniqueness, stable-rate, or hull-energy targets. The full CIF pools, audit trail, benchmark pickle, source snapshot, hashes, and metric JSON are in the companion dataset under evaluation/planner_orb_genbench_all2500_20260719/.

Source

—Code: https://github.com/Richardyangfan78/3DLLMDiffusion/tree/70a66c2e4ce1417cfef378e4c01045d753900d0b
—FK generator release: https://huggingface.co/Yangfan78/3DLLMDiffusion-fk-k8-lam4