AnishRacherla/aya-expanse-8b-compressed-final-int8
04
Aya Expanse 8B — Compressed (int8)
Self-contained. No other repo needed.
Pipeline: Pruning → LoRA fine-tune → Merge → int8 quantize
Load
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch
bnb = BitsAndBytesConfig(load_in_8bit=True)
tok = AutoTokenizer.from_pretrained('AnishRacherla/aya-expanse-8b-compressed-final-int8', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained('AnishRacherla/aya-expanse-8b-compressed-final-int8', quantization_config=bnb, device_map='auto', trust_remote_code=True)