enfinity7B/torrential-attention-25m

🌊 Torrential Attention — torrential

Results

• —Eval Loss: 5.9273
• —Eval Perplexity: 375.13
• —Parameters: 15,414,800
• —Training Time: 1544s

Architecture

Torrential Attention Mechanism with five pillars:

1. 1.Sigmoid Flood Attention (unnormalized, pressure-driven)
2. 2.Position-Decay Budgets (cascading energy)
3. 3.Cross-Layer Flood Thresholds (dam-break phase transitions)
4. 4.Cross-Head Spillover (tributary merging)
5. 5.Differential Noise Cancellation