LOOFYYLO/dynamic-entropy-genuineness-v2-2-space

Dynamic Entropy Genuineness Framework (Version 2.2 Advanced)

This repository implements the Dynamic Entropy Genuineness Framework, a mechanistic interpretability and architectural evolution approach for analyzing and training Transformers. The framework uses a 2D Phase Space to distinguish between mechanical pattern matching and genuine computation.

Core Metrics

The framework evaluates attention heads and model trajectories using two primary axes:

1. Token Cost (X-Axis)

Represents the "external anchor" or the information density being processed.

• —Definition: Weighted average surprisal of the tokens attended to by the head.
• —Surprisal: $I(t) = -\log_2 P(t)$
• —Calculation: $X{head} = \sum{i,j} A_{i,j} \cdot I(j)$

2. Dynamic Genuineness (Y-Axis / G-score)

Measures the internal complexity and attention variance.

• —Definition: $Y = \text{Var}(H) + \text{Collapse Count} / L$
• —Shannon Entropy (H): $H(i) = -\sumj A{i,j} \log2(A{i,j})$
• —Collapse Event: Sudden drop in entropy between layers or tokens, $\Delta H < -0.20$.

Version 2.2 Advanced Implementation

The framework has evolved into Version 2.2 Advanced, featuring:

1. Learned Genuineness Gate (Adaptive Recurrence)

A neural gating mechanism that monitors hidden states and entropy to determine if additional reasoning passes are required.

2. Global G-Budgeting

Computational expenditure management across reasoning steps to ensure efficiency and stability.

3. Layer-Wise Thermodynamic Regularization

An advanced loss function that:

• —Rewards high attention variance (internal complexity).
• —Penalizes static, low-entropy states (mechanical pattern matching).
• —Implements layer-wise decay to prioritize complexity in early reasoning layers.

Repository Structure

• —genuine_model.py: Version 2.2 architecture (Gating, Budgeting, Regularizer).
• —sustained_genuineness.py: Logic utilities for G-score tracking.
• —kaggle_analysis.py: Main analysis pipeline for evaluating prompt trajectories.
• —train_v2_advanced.py: Advanced training for V2.2 on Contextual Parity Pointer tasks.
• —app.py: Interactive Streamlit UI for the Hugging Face Space.
• —V2_2_TECHNICAL_REPORT.md: Detailed theory and experimental results.

Installation & Usage

Dependencies

• —transformer-lens, torch, numpy, scipy, matplotlib, streamlit

Running the UI

streamlit run app.py

Training

python3 train_v2_advanced.py

Phase Space Quadrants (V1)

Developed under the Dynamic Entropy Genuineness Framework (Version 2.2).