Modelpublic

psgokulkrishnan/Llama-3.2-1B-GGUF

sourceHugging Facellama3.2updated 3mo agoView on Hugging Face

0likes39downloads

Model Card

https://huggingface.co/meta-llama/Llama-3.2-1B

Usage: from llama_cpp import Llama

GGUFPATH = "./llama321bQ4KM.ggufQ4KM.gguf"

print("⏳ Initializing long-context GGUF engine directly into memory layout...") llm = Llama( modelpath=GGUFPATH, nctx=16384, flashattn=True, n_threads=4, verbose=False )