Modelpublic

cloudbjorn/Mistral-Medium-3.5-128B-Eschaton-Uncensored

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

0likes27downloads

Model Card

Mistral-Medium-3.5-128B-Eschaton-Uncensored

This is a merged BF16 fine-tune of Mistral Medium 3.5 128B. Training loaded the full-BF16 axolotl-ai-co/Mistral-Medium-3.5-128B-BF16 checkpoint and used cloudbjorn/eschaton-uncensored with the Eschaton Engine.

Training used 4-bit NF4 QLoRA with BF16 compute. The resulting LoRA adapter was then merged into the original BF16 base checkpoint, so this repository contains the complete merged BF16 model rather than pre-quantized training weights.

The fine-tune focuses on direct, neutral, and useful responses to sensitive, gritty, controversial, emotionally intimate, and technically demanding prompts without repetitive moralizing or canned disclaimers.

Personality: Objectivity Over Preachiness

The Eschaton Uncensored dataset emphasizes direct answers, task-appropriate tone, technical substance, dark creative work, and candid analysis. It is intended to reduce unnecessary refusals and boilerplate while preserving the broad capabilities of the underlying instruct model.

Model Characteristics

—Architecture: Dense 128B Mistral 3 multimodal model
—Context window: Up to 262,144 tokens supported by the base architecture
—Reasoning modes: Supports Mistral's configurable reasoning behavior
—Output format: Complete merged model in BF16
—Fine-tuning scope: Text-language layers only; the vision tower and multimodal projector were excluded from LoRA adaptation
—Vision behavior: Vision components remain those of the base checkpoint and were not fine-tuned on this text-only dataset

The training sequence length was 2,048 tokens. The base architecture's larger inference context window was not used as the training sequence length for this fine-tune.

Training Details

Parameter	Value
Base model	`axolotl-ai-co/Mistral-Medium-3.5-128B-BF16`
Dataset	`cloudbjorn/eschaton-uncensored`
Framework	Eschaton Engine using Transformers, TRL, PEFT and bitsandbytes
Training method	4-bit NF4 QLoRA
Quantization compute dtype	bfloat16
Double quantization	Enabled
Final repository format	LoRA merged into the BF16 base model
Epochs	1
Training sequence length	2,048 tokens
Packing	Disabled
Seed	3407

LoRA Configuration

The Eschaton Engine automatically selected its 60B–149B model profile for this 127.7B-parameter checkpoint.

Parameter	Value
Rank (`r`)	32
LoRA alpha	64
Target modules	`all-linear` in the language model, excluding vision and multimodal-projector modules
LoRA dropout	0.05
Bias	none
Task type	`CAUSAL_LM`

Optimization

Parameter	Value
Optimizer	8-bit paged AdamW
Per-device micro-batch size	1
Gradient accumulation	32
Effective batch size	32
Learning rate	5e-6
LR scheduler	Linear
Warmup steps	50
Weight decay	0.01
Gradient checkpointing	Enabled

Evaluation Status

No standardized benchmark results are reported for this fine-tune. Users should evaluate it against their own instruction-following, reasoning, coding, safety, and domain-specific requirements before deployment.

License

This derivative follows the base model's Modified MIT License. Review that license and the upstream model card before use or redistribution.