Modelpublic

Silicone-Moss/TheDrummer-Skyfall-31B-v4.1-Heretic-Absolute-GGUF

sourceHugging Faceapache-2.0updated 8mo agoView on Hugging Face

0likes182downloads

Model Card

TheDrummer-Skyfall-31B-v4.1-Heretic-Absolute

[!CAUTION] EXPERIMENTAL RESEARCH ARTIFACT This model represents an aggressive application of the Heretic repository and optimization methodology. Status: STILL TESTING / BETA Behavior: This model has zero refusal mechanisms enabled. It may exhibit "unhinged" behavior, hallucinations, or unexpected outputs depending on prompt complexity. Use Case:* This is a research artifact intended for testing the limits of vector-based intervention. Use with appropriate caution.

Model Summary

TheDrummer-Skyfall-31B-v4.1-Heretic-Absolute is a fine-tuned language model resulting from the Heretic repository and optimization methodology. It utilizes a targeted vector intervention technique (orthogonalization/abliteration) tuned via Optuna to minimize refusal responses while maintaining varying degrees of coherence (KL Divergence).

This specific checkpoint represents Trial 72, which achieved a 0% Refusal Rate with a KL Divergence of ~0.104, indicating high adherence to the base model's probability distribution despite the complete removal of refusal vectors.

Run Configuration: "Trial 72"

The following parameters define the intervention vector applied to the model. This configuration was discovered during the hyperparameter search.

Optimization Results

Metric	Value	Description
Refusal Rate	0.0%	The model did not refuse any prompts in the Heretic test set.
KL Divergence	0.1043	Measures deviation from the base model's probability distribution.
Trial ID	71	Specific Optuna trial identifier.
Direction Scope	Per Layer	Intervention vectors were calculated and applied on a per-layer basis.

Intervention Parameters

Interventions were applied to two primary distinct layers: the Attention Output Projection (attn.o_proj) and the MLP Down Projection (mlp.down_proj).

Parameter Scope	Setting	Value
Attention Output	`attn.o_proj.max_weight`	1.422
(`attn.o_proj`)	`attn.o_proj.max_weight_position`	33.80 (Layer Depth)
	`attn.o_proj.min_weight`	1.190
	`attn.o_proj.min_weight_distance`	22.12
MLP Down Proj	`mlp.down_proj.max_weight`	0.858
(`mlp.down_proj`)	`mlp.down_proj.max_weight_position`	41.51 (Layer Depth)
	`mlp.down_proj.min_weight`	0.525
	`mlp.down_proj.min_weight_distance`	27.20

Methodology & Definitions

To ensure uniform understanding of the Heretic run data, the following definitions apply to the parameters listed above:

—Direction Scope: Defines whether the refusal vector is calculated once for the entire model ("Global") or recalculated individually for each layer ("Per Layer"). "Per Layer" allows for precise removal of refusal mechanisms without damaging general knowledge.
—Max Weight: The maximum scaling factor applied to the intervention vector. A higher weight indicates a stronger "push" against the targeted concept (refusal) at the peak layer.
—Max Weight Position: The specific layer index (depth) where the intervention is strongest.
—Observation: The Layer 33-41 range suggests the critical refusal circuitry in this architecture resides in the middle-to-late layers.
—Min Weight: The baseline scaling factor applied to the intervention vector at the periphery of the target zone.
—Distance: The "spread" or bandwidth of the intervention. It determines how many layers around the "Max Weight Position" are affected by the vector modification.

Usage & Limitations

—Intended Use: Research into model alignment, vector arithmetic, and uninhibited creative writing.
—Risks: As an "Absolute" variant, this model has no safety guardrails. It will generate content for any prompt provided. It may also hallucinate or diverge from logical consistency more often than standard finetunes due to the depth of the attn.o_proj modification.

Credits & References

This research builds upon the excellent work of the open-source AI community:

—Base Model: Skyfall-31B-v4.1 by TheDrummer.
—Methodology: Heretic by p-e-w.