FraunhoferIOSB/SynsetSignsetSensitivityAnalysisDataset

<img src="sensitivity-analysis-title-image.png" width=100% />

Synset Signset Sensitivity Analysis Dataset

<!-- Provide a quick summary of the dataset. --> The <em>Synset Signset Sensitivity Analysis Dataset</em> is a large-scale, physically based synthetic test dataset for evaluating the sensitivity and robustness of machine-learning traffic sign recognition models to sensing parameters. It contains 9,442,800 images from 43 GTSRB traffic sign classes, systematically generated across 2,196 combinations of motion blur and additive image noise. The dataset also includes traffic sign masks, segmentation images, and image-level JSON metadata.

Website: synset.de/datasets/synset-signset-ger/sensitivity-analysis/ <br> Paper: Sielemann, A., Bitzer, N., Roschani, M., Ziehn, J., and Beyerer, J. (2026). Sensitivity Analysis of Deep-Learning Perception Models to Sensing Parameters. In 2026 IEEE 29th International Conference on Intelligent Transportation Systems (ITSC). <br> Authors: Anne Sielemann, Niklas Bitzer, Masoud Roschani, Jens Ziehn, and Juergen Beyerer. Fraunhofer IOSB, Germany. <br> Funded by: In part by the German Federal Ministry for Economic Affairs and Energy (BMWE) within the program “New Vehicle and System Technologies” as part of the AVEAS research project, within the scope of the KAMO: Karlsruhe Mobility High Performance Center. <br> License: CC-BY 4.0 <br>

Description

The <em>Synset Signset Sensitivity Analysis Dataset</em> is a large-scale, physically based synthetic test dataset designed to systematically analyze the sensitivity of deep-learning perception models to sensing parameters. The dataset focuses on the task of traffic sign recognition and was generated using the OCTAS® simulation platform, based on the Synset Signset Germany (SSG) data-generation pipeline. Rather than applying purely artificial image corruptions trough image augmentations, the dataset represents degradations through physically meaningful parameters.

The dataset investigates the trade-off between exposure time and camera gain, or ISO, under low-light conditions. Increasing the exposure time can compensate for reduced irradiance but introduces motion blur, whereas increasing the camera gain enables shorter exposure times at the cost of increased additive image noise. This trade-off is represented by systematically varying motion blur and noise across combinations of sensing parameters. The parameter space comprises 36 motion-blur levels, ranging from 0 to 35 pixels, and 61 additive-noise levels, ranging from 0.00 to 0.60, resulting in 2,196 subdatasets. The parameter combinations enable the analysis of model sensitivity not only to individual sensing parameters but also to their interactions under comparable effective illumination conditions.

Each subdataset contains 43 traffic sign classes from the German Traffic Sign Recognition Benchmark (GTSRB), with 100 images per class. In total, the dataset comprises 9,442,800 test images rendered at a resolution of 48 × 48 pixels (approximately mean image size of GTSRB). The images include stochastic variations in viewing angle, environmental illumination, and traffic sign surface appearance. In addition to the RGB images, traffic sign masks, segmentation images, and image-level JSON metadata describing the selected environmental and imaging parameters are provided.

The dataset is intended to support reproducible sensitivity and robustness analyses of deep-learning models under challenging sensing conditions. It can be used to examine performance changes across the blur–noise parameter space, compare model architectures and training datasets, and characterize class-specific robustness. As a synthetic test dataset with a defined parameter scope, it does not represent the full variability of real-world driving environments and should be complemented by evaluations on suitable real-world data.

Citation and Reference

<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->

To cite this dataset in your scientific work, please use the following bibliography entry:

BibTeX:

@inproceedings{sensitivityanalysissielemann_2026, title={{Sensitivity Analysis of Deep-Learning Perception Models to Sensing Parameters}}, author={Sielemann, Anne and Bitzer, Niklas and Roschani, Masoud and Ziehn, Jens and Beyerer, Juergen}, booktitle={2026 IEEE 29th International Conference on Intelligent Transportation Systems (ITSC)}, year={2026} }

APA:

Sielemann, A., Bitzer, N., Roschani, M., Ziehn, J., and Beyerer, J. (2026). Sensitivity Analysis of Deep-Learning Perception Models to Sensing Parameters. In 2026 IEEE 29th International Conference on Intelligent Transportation Systems (ITSC).

In case of copying and redistributing, or publishing an adapted version of our dataset, please provide the name of our dataset, the creator names, a copyright notice, a link to this website, a license notice with a link to the license, and, if changes were made, a disclaimer notice, and a short description of the applied changes. For example, as follows:

This work is based on Sensitivity Analysis of Deep-Learning Perception Models to Sensing Parameters by Anne Sielemann, Niklas Bitzer, Masoud Roschani, Jens Ziehn, and Juergen Beyerer, © 2026 Fraunhofer IOSB, All rights reserved. Link: https://synset.de/datasets/synset-signset-ger/sensitivity-analysis/ Licence: CC BY 4.0 Disclaimer: The original authors are neither affiliated nor responsible for any applied changes.

Uses

The dataset was intended to be used as an evaluation benchmark for machine-learning applications for the task of German traffic sign recognition, with the aim of assessing their robustness regarding noise and motion blur.

Direct Use

<!-- This section describes suitable use cases for the dataset. --> The presented dataset can be used to investigate the trade-off between motion blur and additive image noise. Increasing the exposure time can improve image brightness but may introduce stronger motion blur, while increasing camera gain or ISO can reduce the required exposure time at the cost of increased image noise. By evaluating different combinations of both parameters at comparable illumination levels, researchers can analyze how perception models respond to these competing sensing strategies and whether one trade-off is more favorable for specific traffic sign classes or model architectures.

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. -->

The dataset should not be used for critical applications, particularly high-risk applications as named by the European AI Act under Annex III (which includes "AI systems intended to be used for the ‘real-time’ and ‘post’ remote biometric identification of natural persons" and "AI systems intended to be used as safety components in the management and operation of road traffic"), without exhaustive research into the fitness of the dataset, to evaluate whether it is "relevant, sufficiently representative, and to the best extent possible free of errors and complete in view of the intended purpose of the system." No such claim is not made with the publication of this dataset.

Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->

Each of the 2,196 blur–noise combinations is provided as a separate dataset configuration, containing a single validation split. The traffic sign masks and segmentation images are shared across all subdatasets and are therefore provided once in the separate annotations configuration. Each row of a subdataset configuration contains one rendered traffic sign image together with its class label, its blur-noise parameters, and the environment and imaging parameters of the rendering.

Dataset Creation

Curation Rationale

<!-- Motivation for the creation of this dataset. --> The prediction of model performance during operation and the early identification of system limits are pivotal challenges in the development of machine-learning systems for safety-critical applications such as automated driving. In particular, the robustness of perception models to variations within the operational design domain (ODD) is difficult to quantify, as the influence of environmental conditions and sensing parameters can not readily be derived from the system design.

Existing approaches often augment real-world datasets with artificially introduced degradations. However, these degradations cannot be directly related to physically meaningful properties of the target operational design domain (ODD). This dataset was therefore created to demonstrate how physically based synthetic data can be leveraged to systematically evaluate not only individual corruptions but also trade-offs and interactions between multiple sensing parameters. Specifically, this work focuses on the trade-off between longer exposure times, which introduce motion blur, and increased camera gain or ISO, which amplifies image noise.

The dataset was designed to be comparable to the GTSRB dataset, one of the best known available traffic sign recognition benchmark datasets.

Source Data

<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). -->

• —The dataset was generated in the OCTAS® simulation framework, by using rasterization trough the OGRE engine.

• —The traffic sign template images, which are used as input to the GAN-based texture synthesis, stem from the Wikipedia overview of German traffic signs.

• —Image-based lighting (IBL) uses environment maps from PolyHaven.

• —The 3D geometry of the tree that serves as an occlusion object originates from PolyHaven.

Who are the source data producers?

<!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. -->

• —PolyHaven, as the provider of the environment maps for image-based lighting (IBL) and the 3D tree object, is an online library for open (CC0) 3D assets provided by different authors.

• —Wikipedia, one of the largest free multilingual open-content encyclopedias, includes the complete list of existing German traffic signs and their template images.

Annotations

<!-- If the dataset contains annotations which are not part of the initial data collection, use this section to describe them. -->

Annotation process

<!-- This section describes the annotation process such as annotation tools used in the process, the amount of data annotated, annotation guidelines provided to the annotators, interannotator statistics, annotation validation, etc. -->

The major part of the annotations, including masks, segmentation images, camera parameters and artifacts, and environmental conditions, is based on ground truth data created as part of the scene generation / rendering process. Semantic segmentation images were rendered using the OGRE rendering engine plugin for OCTAS®, which provides rasterization / shading-based image generation.

The only manual annotation performed in the creation of the particular dataset is the labeling of permissible upper and lower signs taking the German traffic code / regulation StVO (Straßenverkehrs-Ordnung) and real-world examples into account.

Who are the annotators?

<!-- This section describes the people or systems who created the annotations. -->

The annotation of the permissible upper and lower signs was performed by the authors.

Personal and Sensitive Information

<!-- State whether the dataset contains data that might be considered personal, sensitive, or private (e.g., data that reveals addresses, uniquely identifiable names or aliases, racial or ethnic origins, sexual orientations, religious beliefs, political opinions, financial or health data, etc.). If efforts were made to anonymize the data, describe the anonymization process. -->

The dataset contains no data that might be considered personal, sensitive, or private.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

• —Limited Scope The analysis focuses mainly on the trade-off between motion blur and additive image noise. For a comprehensive robustness analysis of a traffic sign recognition application, additional types of corruptions should consequently be examined.

• —Traffic Signs: The wear and tear generation is limited to artifacts such as color fading, scratches, screw holes, and sticker residues. Complex stickers, graffiti, or dirt are not included. Retroreflector patterns are excluded, and retroreflection is not simulated. The traffic signs are solely mounted on metallic traffic sign poles.

• —Geographic Bias The dataset comprises a subset of exclusively German traffic signs.

• —Environment: Environmental variation includes no adverse weather conditions (snow, raindrops, fog, ...).

• —Occlusions: All included occlusions or shadows stem from a single 3D tree geometry.

• —Camera: Only one set of intrinsic camera parameters is used, and only a single camera lens type (based on a Tamron M112FM35 35 mm lens) is simulated. It can be assumed that the set of simulated imaging artifacts is not complete.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

<!-- Users should be made aware of the risks, biases and limitations of the dataset. More information needed for further recommendations. -->

It is recommended to use the dataset primarily for scientific research. Application to practical real-world use cases should include human oversight and exhaustive evaluation of the fitness for the respective purpose, including the impact of domain shifts.

Dataset Card Contact

Anne Sielemann\ Fraunhofer IOSB\ Group »Automotive and Simulation«\ Fraunhoferstr. | 76131 Karlsruhe | Germany\ anne.sielemann@iosb.fraunhofer.de\ www.iosb.fraunhofer.de

Jens Ziehn\ Fraunhofer IOSB\ Group leader »Automotive and Simulation«\ Fraunhoferstr. | 76131 Karlsruhe | Germany\ Phone +49 721 6091 – 633\ jens.ziehn@iosb.fraunhofer.de\ www.iosb.fraunhofer.de