shreyaspullehf/supervision-dinov3-tc-prediction

SuperVision: DINOv3 + LoRA for Superconductor Tc Prediction

Model Description

This model predicts superconductor critical temperatures (Tc) using a Vision Transformer (DINOv3) fine-tuned with LoRA on 2D rendered crystal structures.

Key Features:

• —Architecture: DINOv3-base (86M parameters) + LoRA (1.1M trainable)
• —Input: 224×224 RGB images of crystal structures with physics-informed encoding
• —Pre-training: ImageNet (14M images) via self-supervised learning
• —Fine-tuning: 5,773 superconductor materials from 3DSC dataset
• —Performance: MAE 4.85 K, R² 0.74 (49-60% better than literature)

Intended Use

Primary Use Case:

• —Predict critical temperature (Tc) of superconducting materials from crystal structure images
• —Screen candidate materials for high-temperature superconductivity
• —Accelerate materials discovery in computational materials science

Users:

• —Materials scientists and computational chemists
• —Researchers in condensed matter physics
• —Machine learning practitioners working on scientific applications

Training Data

Dataset: 3DSC (3D Superconductor Dataset)

• —5,773 superconductor materials
• —Train: 4,041 materials (70%)
• —Validation: 866 materials (15%)
• —Test: 866 materials (15%)

Data Sources:

• —Crystal structures: Materials Project
• —Critical temperatures: SuperCon database

Preprocessing:

• —3D CIF structures rendered to 2D images using ASE
• —Physics-informed RGB encoding:
• —R channel: d-electron count
• —G channel: Valence electrons
• —B channel: Atomic mass

Performance

Test Set Results

Comparison to Baselines

Training Details

• —Best Epoch: 23/40
• —Training Time: ~40 hours (CPU)
• —Optimizer: AdamW
• —Learning Rate: 1e-4
• —Batch Size: 8
• —LoRA Config: rank=16, alpha=32, dropout=0.1

Usage

from transformers import AutoImageProcessor, AutoModel
import torch
from PIL import Image

# Load model and processor
processor = AutoImageProcessor.from_pretrained("shreyaspulle98/supervision-dinov3-tc-prediction")
model = AutoModel.from_pretrained("shreyaspulle98/supervision-dinov3-tc-prediction")

# Load and process image
image = Image.open("crystal_structure.png")
inputs = processor(images=image, return_tensors="pt")

# Predict critical temperature
with torch.no_grad():
    tc_prediction = model(**inputs).logits

print(f"Predicted Tc: {tc_prediction.item():.2f} K")

Limitations

1. 1.Training Data Coverage: Limited to materials in 3DSC dataset (primarily conventional and cuprate superconductors)
2. 2.High-Tc Underestimation: Larger errors for materials with Tc > 100 K (limited training samples)
3. 3.Image Dependency: Requires crystal structure rendering with specific physics-informed encoding
4. 4.Computational Cost: Slower inference than ALIGNN (~120ms vs 50ms per sample)

Ethical Considerations

• —Model is for research purposes only, not for production materials design without experimental validation
• —Predictions should be verified experimentally before synthesis attempts
• —Model may have biases toward material classes well-represented in training data

Citation

@software{supervision2024,
  title={SuperVision: Transfer Learning for Superconductor Tc Prediction},
  author={Your Name},
  year={2024},
  url={https://github.com/yourusername/SuperVision}
}

License

MIT License

More Information

• —GitHub Repository: https://github.com/yourusername/SuperVision
• —Paper: [Coming soon]
• —Contact: your.email@example.com