RichardErkhov/Mihaiii_-_Pallas-0.5-LASER-0.2-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Pallas-0.5-LASER-0.2 - GGUF

• —Model creator: https://huggingface.co/Mihaiii/
• —Original model: https://huggingface.co/Mihaiii/Pallas-0.5-LASER-0.2/

Original model description: --- basemodel: Mihaiii/Pallas-0.5-LASER-0.1 inference: false license: other licensename: yi-license license_link: https://huggingface.co/01-ai/Yi-34B/blob/main/LICENSE metrics:

• —accuracy ---

This model has a LASER intervention on Mihaiii/Pallas-0.5-LASER-0.1 .

Configs used:

• —lnum: 58
• —lnames: attn (meaning: ["selfattn.kproj.weight", "selfattn.qproj.weight", "selfattn.vproj.weight", "selfattn.oproj.weight"])
• —rate: 9.0
• —dataset: bigbench (subset: causal_judgement)
• —intervention type: rank-reduction

In order to replicate on a single A100, you can use my branch (the original code will throw OOM for 34b models).