Modelpublic

nexuslrf/LuxDiT

sourceHugging Faceotherupdated 6mo agoView on Hugging Face

0likes

Model Card

LuxDiT

This directory contains released model weights for LuxDiT: Lighting Estimation with Video Diffusion Transformer. It is finetuned on image data and includes a LoRA adapter for real scenes.

Model checkpoints	Description
`luxdit_image`	Image-finetuned checkpoint with LoRA adapter for real scenes
`luxdit_video`	Video-finetuned checkpoint with LoRA adapter for real scenes
`hdr_merge_mlp`	HDR merger used to produce `.exr` outputs from dual tonemapped predictions

Model description

LuxDiT is a generative lighting estimation model that predicts high-quality HDR environment maps from visual input. It preserves scene semantics while estimating realistic illumination for virtual object insertion.

—Architecture: Transformer (based on CogVideoX-5B)
—Parameters: 5B
—Runtime: Python + PyTorch
—Paper: LuxDiT: Lighting Estimation with Video Diffusion Transformer
—Project page: https://research.nvidia.com/labs/toronto-ai/LuxDiT/
—Github repository: https://github.com/nv-tlabs/LuxDiT

Detailed model card is available at model_card.md.

How to use

Please refer to our Github repository (https://github.com/nv-tlabs/LuxDiT) for usage instructions.

Ethical considerations

NVIDIA believes Trustworthy AI is a shared responsibility. Make sure your use case has proper rights and permissions for all input data, and implement appropriate safety and misuse mitigation before deployment. For concerns, see NVIDIA AI Concerns.

License

NVIDIA OneWay Noncommercial License. See LICENSE.md in the repository root.

Citation

bibtex

@article{liang2025luxdit,
  title={Luxdit: Lighting estimation with video diffusion transformer},
  author={Liang, Ruofan and He, Kai and Gojcic, Zan and Gilitschenski, Igor and Fidler, Sanja and Vijaykumar, Nandita and Wang, Zian},
  journal={arXiv preprint arXiv:2509.03680},
  year={2025}
}