nvidia/C-RADIOv4-1D-H
8301
1---2license: other3license_name: nvidia-open-model-license4license_link: https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf5pipeline_tag: image-feature-extraction6library_name: transformers7base_model: nvidia/C-RADIOv4-H8tags:9 - vision10 - image-embeddings11 - vision-transformer12 - foundation-model13 - knowledge-distillation14 - multi-teacher-distillation15 - elastic-tokens16 - token-compression17 - radio18 - icml-202619---20 21# Model Overview22 23## Description24 25This model performs visual feature extraction.26Unlike standard Vision Transformers that produce a fixed 2D grid of patch features, RADIO1D compresses an input image into a compact, **variable-length 1D sequence of tokens**.27The number of output tokens (from 1 up to 256) can be selected by the user at inference time, providing a continuous accuracy/efficiency trade-off.28For example, an image can be summarized into a single token for retrieval, or expanded to 256 tokens for fine-grained tasks such as OCR.29 30RADIO1D was produced by fine-tuning [C-RADIOv4-H](https://huggingface.co/nvidia/C-RADIOv4-H) using multi-teacher agglomerative distillation from:31* [SigLIP2-g](https://huggingface.co/google/siglip2-giant-opt-patch16-384)32* [DINOv3-7B](https://huggingface.co/facebook/dinov3-vit7b16-pretrain-lvd1689m)33* [SAM3](https://huggingface.co/facebook/sam3)34 35The encoder integrates a learnable Patch Merging block (4× sequence-length reduction, 2× channel expansion) part-way through the network for efficiency, and a lightweight Vision Transformer decoder is used only during training to project the 1D tokens back into a 2D-compatible grid for teacher alignment. At inference, only the encoder runs.36 37This model is ready for commercial or non-commercial use.38 39### License/Terms of Use40 41GOVERNING TERMS: Use of this model is governed by the [NVIDIA Open Model License Agreement](https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf).42 43## Deployment Geography44 45Global46 47## Use Case48 49The embeddings generated by this model are expected to be used by a downstream application.50The variable-length 1D token output makes RADIO1D especially well suited to:51 52* Integration into a Vision-Language Model with a user-tunable token budget per image (trade off accuracy vs. time-to-first-token).53* Image-level understanding (image classification, scene summarization, image curation).54* Composition-aware image-to-image retrieval (object presence and spatial arrangement).55* Dense processing (semantic segmentation) when a decoder is used.56 57## Release Date58 59Hugging Face: 07/01/2026 via [RADIO Collection of Models](https://huggingface.co/collections/nvidia/radio-669f77f1dd6b153f007dd1c6).60 61## References62 63* RADIO1D paper: G. Heinrich, M. Ranzinger, C. McCarthy et al., "RADIO1D: Elastic Representations for Condensed Vision Modeling," ICML 2026 ([arXiv:2607.03624](https://arxiv.org/pdf/2607.03624).64* AM-RADIO ([arXiv:2312.06709](https://arxiv.org/abs/2312.06709))65* RADIOv2.5 ([arXiv:2412.07679](https://arxiv.org/abs/2412.07679))66* PHI-S / RADIO scaling ([arXiv:2410.01680](https://arxiv.org/abs/2410.01680))67* C-RADIO ([arXiv:2502.16025](https://arxiv.org/abs/2502.16025))68* C-RADIOv4 ([arXiv:2601.17237](https://arxiv.org/abs/2601.17237))69 70## Model Architecture71 72**Architecture Type:** Neural Network <br>73**Network Architecture:** Vision Transformer with encoder-decoder for elastic 1D token generation <br>74**Number of model parameters:** ~1.14B (encoder, used at inference); ~314M additional decoder parameters used only during training <br>75 76The RADIO1D-H encoder is built from a ViT-H/16 backbone. Image patches (16×16 pixels) are flattened into a 1D sequence and processed by 24 transformer blocks at embedding dimension 1280, followed by a learnable Patch Merging block that groups 2×2 neighboring tokens (reducing sequence length by 4× and expanding the channel dimension by ρ=2 to 2560), followed by 8 further transformer blocks at the wider dimension. During training, a length ℓ is sampled stochastically from a triangular distribution p(x)=2−2x, and only the first ℓ encoder tokens are retained (a form of nested dropout). Earlier tokens are therefore encouraged to encode global, high-level semantics while later tokens specialize in finer details.77 78A small ViT decoder (~314M parameters, 6 blocks with one Patch Splitting upscale) is used **only during training**. It receives duplicated learnable query tokens equal in number to the original patch grid plus the encoder's ℓ tokens as additional register tokens, and cross-attention reconstructs a 2D-compatible feature grid that is aligned to the teacher representations.79 80At inference, only the encoder is used and the user specifies the desired number of output tokens.81 82## Input83 84**Input Type(s):** Image <br>85**Input Format(s):** Red, Green, Blue (RGB) <br>86**Input Parameters:** Two Dimensional (2D) <br>87**Other Properties Related to Input:** Image resolutions up to 2048×2048 in increments of 16 pixels. Training used a mix of low-resolution images (128, 192, 224, 256, 384, 432 px) and high-resolution images (512, 768, 1024, 1152 px). <br>88 89## Output90 91**Output Type(s):** Embeddings <br>92**Output Format:** Tensor <br>93**Output Parameters:** One Dimensional (1D) — variable-length sequence of tokens <br>94**Other Properties Related to Output:** The encoder returns a sequence of prefix tokens (CLS + register tokens) followed by `ℓ` global 1D tokens, where `ℓ` is selected by the caller at inference (typical values: 1, 8, 32, 64, 128, 192, 224, 256). Tokens are ordered hierarchically: token 0 encodes the strongest global summary (e.g., 85.0% k-NN Top-1 on ImageNet-1k with a single token), and later tokens add progressively finer detail. A downstream model is required to leverage the image features. Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g., GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>95 96## Software Integration97 98**Runtime Engine(s):**99* PyTorch <br>100 101 102**Supported Hardware Microarchitecture Compatibility:** <br>103* NVIDIA Ampere <br>104* NVIDIA Blackwell <br>105* NVIDIA Jetson <br>106* NVIDIA Hopper <br>107* NVIDIA Lovelace <br>108* NVIDIA Pascal <br>109* NVIDIA Turing <br>110* NVIDIA Volta <br>111 112**[Preferred/Supported] Operating System(s):** <br>113* Linux114* Linux 4 Tegra115* QNX116* Windows117 118The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.119 120This AI model can be embedded as an Application Programming Interface (API) call into the software environment described above.121 122## Model Version(s)123 124* C-RADIOv4-1D-H (~1.14B parameters; based on a ViT-H/16 backbone with learnable Patch Merging at block 24 and expansion factor ρ=2).125 126**Links:**127 128* https://huggingface.co/nvidia/C-RADIOv4-1D-H129 130# Training and Evaluation Datasets131 132 133## Training Dataset134 135**NV-CC-Img-Text-Dataset**136 137* **Data Modality:** Image138* **Image Training Data Size:** 1 Million to 1 Billion Images139* **Data Collection Method by dataset:** Automated140* **Labeling Method by dataset:** Not Applicable (no labels are needed; supervision comes from teacher models via multi-teacher distillation)141* **Properties:** ~172M total training samples processed over 300k optimizer steps (less than one epoch over the source dataset). Global batch size of 512 low-resolution images (sampled from 128, 192, 224, 256, 384, 432 px) plus 64 high-resolution images (from 512, 768, 1024, 1152 px).142 143## Evaluation Datasets144 145**ADE20K**146 147* **Link:** [ADE20K](https://ade20k.csail.mit.edu/)148* **Data Collection:** Manually-Collected149* **Labeling Method:** Manually-Collected150* **Training Images:** 25,574151* **Validation Images:** 2,000152 153**ImageNet**154 155* **Link:** [ImageNet](https://www.image-net.org/)156* **Data Collection:** Automated157* **Labeling Method:** Manually-Collected158* **Training Images:** 1,281,167159* **Validation Images:** 50,000160 161For downstream VLM evaluation, RADIO1D was paired with the Nemotron-Nano-9B-v2 LLM in the Nemotron VL framework and evaluated on TextVQA, DocVQA, InfoVQA, OCRBench, OCRBench v2 (EN/CN), AI2D, ChartQA, MMMU, SeedBench, and LongVideoBench.162 163### ADE20K linear-probe mIoU vs. number of 1D tokens164 165The decoder's reconstructed 2D feature grid is used with a frozen linear probe.166 167| Tokens | 1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 | 256 |168|--------|----|----|----|----|-----|-----|-----|------|------|169| mIoU | 40.23 | 46.84 | 51.14 | 54.11 | 55.22 | 55.83 | 55.60 | 55.63 | 55.68 |170 171For comparison, with a 1024-token feature grid C-RADIOv4-H reaches 55.20 mIoU and DINOv3-H+ reaches 54.80 mIoU on the same linear-probe protocol — RADIO1D matches or exceeds these scores using only 32 tokens.172 173### Vision-language modeling (Nemotron-Nano-9B-v2, 17M SFT samples)174 175Average accuracy across 11 multimodal benchmarks (TextVQA, DocVQA, InfoVQA, OCRBench, OCRBench v2 EN, OCRBench v2 CN, AI2D, ChartQA, MMMU, SeedBench, LongVideoBench), with H100 time-to-first-token (TTFT) measured in vLLM at 32 images and 128 LLM context tokens.176 177| Vision Encoder | Tokens/tile | TTFT (ms) | Avg. accuracy |178|----------------------|-------------|-----------|---------------|179| C-RADIOv4-H | 256 | 468.2 | 73.09 |180| SigLIP2-SO400m | 256 | 440.5 | 72.67 |181| SigLIP2-g | 256 | 517.6 | 72.81 |182| RADIO1D-H | 1 | 327.0 | 51.53 |183| RADIO1D-H | 8 | 329.7 | 63.88 |184| RADIO1D-H | 32 | 335.1 | 68.13 |185| RADIO1D-H | 64 | 346.1 | 70.07 |186| RADIO1D-H | 128 | 373.2 | 71.64 |187| RADIO1D-H | 192 | 380.4 | 72.36 |188| RADIO1D-H | 224 | 410.2 | 73.02 |189| RADIO1D-H | 256 | 452.8 | 73.29 |190 191## Inference192 193**Acceleration Engine:** TensorRT, TensorRT-LLM <br>194**Engine:** PyTorch <br>195**Test Hardware:** NVIDIA Hopper (H100) <br>196 197## Ethical Considerations198 199NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.200 201Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.202 203For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards below.204 205Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).206 207### Bias208 209Field | Response210:---------------------------------------------------------------------------------------------------|:---------------211Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing: | None212Measures taken to mitigate against unwanted bias: | None213Bias Metric (If Measured): | None214 215 216### Explainability217 218Field | Response219:------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------220Intended Task/Domain: | Visual Feature Extraction (variable-length 1D token sequence)221Model Type: | Vision Transformer with elastic 1D token bottleneck and (training-only) decoder222Intended Users: | Developers of downstream vision and vision-language applications223Output: | Variable-length sequence of 1D image embedding tokens (1–256 tokens, user-specified at inference)224Describe how the model works: | The model takes an image as input, processes the image through transformer blocks (with one learnable Patch Merging downscale), and outputs a hierarchical 1D sequence in which early tokens summarize global semantics and later tokens encode finer details. The user selects how many tokens to keep at inference.225Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of: | Not Applicable226Technical Limitations: | This model generates image embeddings that a downstream model must be trained to consume. The model is only tested on input resolutions ranging from 128 to 2048 pixels, in increments of 16 pixels. With very few output tokens (≤ 32) accuracy degrades on tasks requiring dense reading (e.g., DocVQA, InfoVQA, OCRBench), where the full 256-token budget is recommended. The model may fail to surface fine-grained orientation cues (e.g., whether a sign points left or right) and, like other vision foundation models, may not disambiguate visually similar concepts (e.g., different breeds of dog) without downstream fine-tuning.227Verified to have met prescribed NVIDIA quality standards: | Yes228Performance Metrics: | ADE20K linear-probe mIoU as a function of token count, and multimodal benchmark accuracy in a Nemotron VL pipeline.229Potential Known Risks: | This model may not perform well on visual domains that are not represented in the training data. The generated embeddings might fail to disambiguate differences that appear evident to humans. Domain-specific evaluation is required for the target application. Aggressive token compression (≤ 32 tokens) trades fine-grained spatial fidelity for efficiency and should be evaluated for the target task before deployment.230Licensing: | [NVIDIA Open Model License](https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf)231 232 233### Privacy234 235Field | Response236:----------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------237Generatable or reverse engineerable personal data? | No238Personal data used to create this model? | No239How often is dataset reviewed? | Before Every Release240Is there provenance for all datasets used in training? | Yes241Does data labeling (annotation, metadata) comply with privacy laws? | Yes242Is data compliant with data subject requests for data correction or removal, if such a request was made? | Yes243Was data from user interactions with the AI model (e.g. user input and prompts) used to train the model? | No244Applicable Privacy Policy | https://www.nvidia.com/en-us/about-nvidia/privacy-policy/245 246### Safety247 248Field | Response249:---------------------------------------------------|:----------------------------------250Model Application Field(s): | Generation of visual embeddings251Describe the life critical impact (if present). | Not Applicable252Use Case Restrictions: | Abide by NVIDIA Open Model License Agreement253Model and dataset restrictions: | The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to.254 255## Citation256 257If you use RADIO1D in your research, please cite:258 259```bibtex260@inproceedings{heinrich2026radio1d,261 title = {{RADIO1D}: Elastic Representations for Condensed Vision Modeling},262 author = {Heinrich, Greg and Ranzinger, Mike and McCarthy, Collin and Bagrov, Natan and Khvedchenya, Eugene and Catanzaro, Bryan and Kautz, Jan and Tao, Andrew and Molchanov, Pavlo},263 booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)},264 year = {2026},265 url = {https://arxiv.org/abs/2607.03624}266}267```268