Modelpublic

SADASDASDSASADSADSADSAD/goon-cave-7b-instruct-bnb-4bit

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes132downloads

Model Card

SADASDASDSASADSADSADSAD/goon-cave-7b-instruct-bnb-4bit

A compact Llama-architecture model in GGUF format, exported as I8 tensors for straightforward inspection and inference.

Model details

Property	Value
Architecture	Llama
Tensor precision	I8
Format	GGUF v3

Architecture parameters (context length, embedding length, block count, etc.) are declared in the GGUF metadata and rendered by the HF viewer.

Usage

llama.cpp

llama-cli -m goon-cave-7b-instruct-v1.0.gguf -p "Your prompt here" -n 256

Python (llama-cpp-python)

python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="SADASDASDSASADSADSADSAD/goon-cave-7b-instruct-bnb-4bit",
    filename="goon-cave-7b-instruct-v1.0.gguf",
)
output = llm("Q: What is the meaning of life? A:", max_tokens=64)
print(output["choices"][0]["text"])

Download

hf download SADASDASDSASADSADSADSAD/goon-cave-7b-instruct-bnb-4bit \
  --include "*.gguf" --local-dir ./