creative-graphic-design/layout-validity

Layout Validity

Description

The Layout Validity metric evaluates the basic validity of layout elements by checking whether they meet minimum size requirements. A layout element is considered valid if its area within the canvas boundaries is greater than 0.1% of the total canvas area. This metric helps identify layouts with degenerate or nearly-invisible elements that don't contribute meaningfully to the design.

What It Measures

This metric computes:

• —Valid element ratio: Proportion of elements that meet minimum size requirements
• —Layout integrity: Whether elements are large enough to be functional
• —Degenerate element detection: Identifies layouts with overly small or nearly-invisible elements

Higher scores indicate better layout validity with fewer problematic elements.

Metric Details

• —Validity threshold: Element area > 0.1% of canvas area (after clamping to canvas bounds)
• —Clamps element bounding boxes to canvas boundaries before computing area
• —Filters out padding elements (label == 0)
• —From PosterLayout (Hsu et al., CVPR 2023) for poster design evaluation
• —Returns the ratio of valid elements to total non-padding elements

Usage

Installation

pip install evaluate

Basic Example

import evaluate
import numpy as np

# Load the metric with canvas dimensions
metric = evaluate.load(
    "creative-graphic-design/layout-validity",
    canvas_width=360,
    canvas_height=504
)

# Prepare data (normalized ltrb coordinates)
predictions = np.random.rand(1, 25, 4)
gold_labels = np.random.randint(0, 4, size=(1, 25))
score = metric.compute(predictions=predictions, gold_labels=gold_labels)
print(score)

Batch Processing Example

import evaluate
import numpy as np

# Load the metric
metric = evaluate.load(
    "creative-graphic-design/layout-validity",
    canvas_width=360,
    canvas_height=504
)

# Batch processing
batch_size = 128
predictions = np.random.rand(batch_size, 25, 4)
gold_labels = np.random.randint(0, 4, size=(batch_size, 25))
score = metric.compute(predictions=predictions, gold_labels=gold_labels)
print(score)

Parameters

Initialization Parameters

• —canvas_width (int, required): Width of the canvas in pixels
• —canvas_height (int, required): Height of the canvas in pixels

Computation Parameters

• —predictions (list of lists of float): Normalized bounding boxes in ltrb format (0.0 to 1.0)
• —gold_labels (list of lists of int): Class labels for each element (0 = padding)

Note:

• —Elements with gold_labels == 0 are treated as padding and excluded from computation
• —Element coordinates are clamped to [0, canvaswidth] × [0, canvasheight] before area calculation
• —Validity threshold: area > (canvaswidth × canvasheight) / 1000

Returns

Returns a float value representing the ratio of valid elements to total elements (range: 0.0 to 1.0).

Interpretation

• —Higher is better (range: 0.0 to 1.0)
• —Value of 1.0: All elements are valid (ideal)
• —Value of 0.9-1.0: Mostly valid layout with few problematic elements
• —Value of 0.7-0.9: Some invalid elements, may need review
• —Value of 0.5-0.7: Many invalid elements, layout quality concerns
• —Value < 0.5: Significant validity issues, many degenerate elements
• —Value of 0.0: All elements invalid (critical failure)

Use Cases

• —Basic layout quality check: Ensure generated layouts don't have degenerate elements
• —Generative model evaluation: Detect models producing invalid element sizes
• —Layout post-processing: Filter out layouts with validity issues
• —Sanity check: Validate layout data before further evaluation

Key Insights

• —Minimum requirement: This is a basic quality check, not a comprehensive quality metric
• —Threshold choice: 0.1% of canvas is quite permissive - very small but still visible elements pass
• —Canvas boundaries matter: Elements extending beyond canvas are clamped, which can reduce their area
• —Foundation metric: Pass validity check before evaluating other metrics
• —Model debugging: Low validity scores indicate fundamental generation issues

Common Causes of Invalidity

• —Collapsed bounding boxes: Width or height near zero
• —Out-of-bounds elements: Elements mostly outside canvas boundaries
• —Numerical errors: Floating-point precision issues in generation
• —Model failure modes: Generation model producing degenerate outputs

Citations

@inproceedings{hsu2023posterlayout,
  title={Posterlayout: A new benchmark and approach for content-aware visual-textual presentation layout},
  author={Hsu, Hsiao Yuan and He, Xiangteng and Peng, Yuxin and Kong, Hao and Zhang, Qing},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={6018--6026},
  year={2023}
}

References

• —Paper: PosterLayout (Hsu et al., CVPR 2023)
• —Reference Implementation: PosterLayout eval.py

Related Metrics

• —Layout Overlap: Evaluates element spacing and collisions
• —Layout Alignment: Measures spatial organization
• —Layout Non-Alignment: Detects alignment violations
• —Layout Utility: Evaluates space utilization