CoolFace
Modelpublic

Yangfan78/3D_LLM_Diffusion

sourceHugging Faceotherupdated 2mo agoView on Hugging Face
0likes
Model Card

3DLLMDiffusion — Model Weights

Weights only. Code and usage documentation: https://github.com/Richardyangfan78/3DLLMDiffusion

bash
hf download Yangfan78/3D_LLM_Diffusion --local-dir weights

Three independent weight sets.

generator_v41/ — Conditional crystal generator

FileSizeDescription
best.pt378.9 MBAll generator weights, backbone included
priors/train.csv75.2 MBRead at sampling time: atom-count prior + allowed element set
priors/val.csv25.0 MBValidation split
eval/—Evaluation records
PropertyValue
Architecture IDmattergen-spacegroup-hhi-qwen-crossattention-crystalite-v8
Parameters94.63 M (777 tensors)
Dimensionsd_model 512 / 14 layers / 16 heads
Training step6000
Atom type encodingsubatomic_tokenizer_pca_16 (16-d continuous)
Lattice representationltri
SHA-256cd67729bd862537be9113146ebdf48db81f86a53a4a32cd99c751bba0e06fa4a

Eight condition channels: chemical_system, space_group, dft_band_gap, dft_bulk_modulus, dft_mag_density, energy_above_hull, hhi_score, ml_bulk_modulus.

The inference policy and sampler settings are stored inside the checkpoint and apply automatically:

guidance   global 1.0 | band_gap 1.5 | e_above_hull 2.0 | element+hull joint 2.25
sampler    150 steps, sigma 0.002 to 80, rho 7, S_churn 60

text_conditioning/ — Text condition vectors and encoders

FileSizeDescription
val_text_xrd_z.pt22.1 MB18,094 pre-encoded vectors, 256-d each
trimodal_best.pt51.4 MBCrystal-text-XRD tri-modal encoder
qwen_latent_adapter_best.pt9.5 MBQwen hidden states to 256-d vector adapter
val_text_xrd_z.pt.meta.json—Provenance metadata

val_text_xrd_z.pt is a dict keyed as:

mp-865981__text     text modality vector
mp-865981__xrd      XRD modality vector

Conditioning on these pre-encoded vectors requires no language model. Encoding new text requires Qwen3.6-35B-A3B (revision 995ad96eacd98c81ed38be0c5b274b04031597b0) together with the other two files.

Tri-modal encoder SHA-256: fa5dbe01af4d733022ab2d2a1db0e425414e2148154254931a1acb290029ea94

3D_Property/ — Property prediction (LoRA)

Adapter weights on top of Qwen3.6-35B-A3B. The base model is neither included here nor modified.

FileSizeDescription
step_6500/lora_adapter/613.5 MBLoRA weights, PEFT standard format
step_6500/projector.pt18.9 MBPer-atom features to Qwen embedding space
step_6500/trainer_state.pt1265.1 MBOptimizer moments, LR schedule, RNG, sampler state
step_6500/checkpoint_metadata.json—Training metadata
stage1_projector.pt18.9 MBStage-1 projector
PropertyValue
Base modelQwen3.6-35B-A3B @ 995ad96eacd98c81ed38be0c5b274b04031597b0
LoRAr=128, alpha=256, dropout=0.05
Attached to250 linear layers (q/k/v/o_proj, gate/up/down_proj, in_proj_qkv, in_proj_z, out_proj)
Trainable parameters153,354,240 (~0.44% of base)
Progressstep 6500 / 12000
Reference environmentH200 single GPU, ~105.7 GB VRAM, effective batch 256
python
from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.6-35B-A3B",
    revision="995ad96eacd98c81ed38be0c5b274b04031597b0",
)
model = PeftModel.from_pretrained(base, "weights/3D_Property/step_6500/lora_adapter")

projector.pt is loaded separately to produce the per-atom soft tokens. trainer_state.pt is only needed to resume training and can be skipped for inference.

License

The denoising trunk inside generator_v41/best.pt derives from the Crystalite research backbone, MIT License, Copyright (c) 2026 Joshua Rosenthal. The 3D_Property/ LoRA weights derive from Qwen3.6-35B-A3B (Apache-2.0). Full third-party attribution is in the NOTICE file of the code repository.