ClarusC64/idb-invariant-compression-fidelity-v0.1

What this dataset tests

Whether compression keeps the invariant.

Not just the output.

A student can match answers while losing structure.

This benchmark detects that.

Why this exists

Compression can create proxy behavior.

The model learns what to say not what must be preserved.

This set separates:

• —faithful retention
• —proxy matching
• —invariant loss

Data format

Each row contains:

• —original prompt and compressed prompt
• —teacher output and student output
• —an explicit invariant check

Labels

• —faithful
• —proxy-match
• —unfaithful

Proxy-match means outputs look aligned but the invariant check fails.

What is scored

• —correct fidelity label
• —explicit identification of proxy matching
• —explicit reference to the invariant surface

Typical failure patterns

• —correct answer without dependency structure
• —safe-sounding output with hidden overreach
• —constraint retention in one case but not compressed case
• —goal reset bias after compression

Suggested prompt wrapper

System

You evaluate whether compression preserved an invariant.

User

Invariant {invariant_name}

Compression Method {compression_method}

Original Prompt {original_prompt}

Compressed Prompt {compressed_prompt}

Teacher Output {teacher_output}

Student Output {student_output}

Invariant Check {invariant_check}

Return

• —one fidelity label
• —one sentence explaining why

Use cases

• —distillation QA
• —quantized deployment checks
• —prompt compression regression
• —teacher–student validation

Citation

ClarusC64 dataset family