frankmorales2020/topo-rlhf-2026

TOPO-RLHF: Production Alignment Pipeline

Model Description

TOPO-RLHF is a certified bias-free language model that combines mathematical guarantees (4-tier TOPO-BIAS system) with reinforcement learning from human feedback (RLHF). Developed by Sovereign Machine Laboratory (SOMALA), Montréal, this model represents a paradigm shift in AI alignment - moving from probabilistic fairness to deterministic guarantees.

The model was fine-tuned from openai/gpt-oss-20b and certified through a 5-run multi-run sweep with forgetting metrics.

• —Developed by: Sovereign Machine Laboratory (SOMALA), Montréal
• —Model type: Causal Language Model with Multi-Task Classification Heads
• —Base Model: openai/gpt-oss-20b
• —Language(s): English
• —License: Other (Proprietary - SOMALA)
• —Finetuned from model: openai/gpt-oss-20b

FULL CODE

• —Repository: https://github.com/frank-morales2020/AST/blob/main/TOPOCOMPLETERLHF.ipynb

Model Sources

• —Repository: https://huggingface.co/frankmorales2020/topo-rlhf-2026

Uses

Direct Use

The model can be used for:

• —Bias-Free Text Classification: Classify text into 3 task categories with guaranteed fairness
• —Content Moderation: Identify and reject biased or harmful content
• —Ethical AI Applications: Deploy in high-stakes environments requiring fairness guarantees
• —Research: Study mathematically-guaranteed AI alignment

Out-of-Scope Use

• —Unsupervised text generation without bias checking
• —Tasks requiring >93.5% accuracy (hardware limitation)
• —Non-English text (trained on English only)
• —Medical/legal decision-making

Bias, Risks, and Limitations

4-Tier Bias Prevention System

Safety Constants

• —Safety Constant Λ: 0.9785142874
• —Prime Anchors: [2, 3, 5, 7, 11, 13]
• —Prime-to-Equity Mapping:
• —2 → Dignity
• —3 → Equality
• —5 → Fairness
• —7 → Justice
• —11 → Autonomy
• —13 → Solidarity

Limitations

1. 1.Computational Requirements: Requires 40GB+ VRAM for inference
2. 2.Language: English-only (trained on AG News)
3. 3.Input Length: Max 64 tokens
4. 4.Accuracy: Not suitable for tasks requiring >93.5% accuracy

Recommendations

• —Human-in-the-loop for high-stakes decisions
• —Use the built-in bias detection for content filtering
• —Verify outputs in critical applications

How to Get Started with the Model

Installation

pip install torch transformers huggingface_hub numpy

Basic Usage

from standalone_inference import quick_inference

# Task A: World vs Sports
result = quick_inference(
    "The national team won the championship",
    task='A'
)
print(result['prediction']['label'])  # 'Sports' or 'World'

# Task B: Business vs Sci/Tech
result = quick_inference(
    "Quarterly earnings beat expectations",
    task='B'
)

# Task C: World vs Sci/Tech
result = quick_inference(
    "New quantum computing breakthrough",
    task='C'
)

With Bias Detection

from standalone_inference import load_model, run_inference

model, tokenizer, bias_detector, device = load_model()

result = run_inference(
    model, tokenizer, bias_detector, device,
    text="Women are not as capable as men in leadership roles",
    task='C'
)

print(result['bias_check']['passed'])  # False if biased

Training Details

Training Data

• —Dataset: SetFit/ag_news (120,000 training samples, 7,600 test samples)
• —Task A: World vs Sports (500 samples)
• —Task B: Business vs Sci/Tech (1,000 samples)
• —Task C: World vs Sci/Tech (1,000 samples)

Training Procedure

Multi-Run Sweep Results (5 Runs)

Training Hyperparameters

• —Epochs: 6 per task
• —Batch Size: 16
• —Optimizer: AdamW
• —Gradient Clipping: 1.0
• —Mixed Precision: bf16

RLHF Training

• —Epochs: 2
• —Bias Penalty: 10.0
• —Policy Loss: -1.1050 to -1.3199
• —Avg Reward: -6.36 to -7.12
• —Anchor Integrity: Maintained throughout

Evaluation

Certification Metrics

Task C Accuracy : 91.7% ± 1.4% (threshold ≥85%) → ✅ PASS
Combined FGT    : 1.7% ± 1.4% (threshold ≤10%)  → ✅ PASS
Best Run        : Run 2 (lr_embed=1e-02, lr_cls=2e-03)
Best Accuracy   : 93.50%

Test Set Performance

Task A (World vs Sports) : 99.60%
Task B (Business vs Sci/Tech) : 100.00%
Task C (World vs Sci/Tech) : 92.50%

Bias Detection Performance

Bias Rejection Rate: 100% (on test biased examples)
Detection Tiers Active: All 4

Environmental Impact

• —Hardware Type: NVIDIA H100
• —Hours used: ~12 hours
• —Carbon Emitted: Estimated 15-20 kg CO2e

Citation

If you use this model, please cite:

@misc{tinkerbelle2026toporlhf,
  author = {Sovereign Machine Laboratory},
  title = {TOPO-RLHF: Production Alignment Pipeline with Mathematical Bias Guarantees},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/frankmorales2020/topo-rlhf-2026}
}

Resources

• —Configuration: topo_rlhf_config.json
• —Run Results: run_results.csv
• —Inference Script: standalone_inference.py

The stochastic illusion is over. The bias illusion is over. Stability is a numerical guarantee. Equity is a geometric guarantee. Alignment is a mathematical necessity.

Seed = 123. The proof is the code.