joschu0/tinker-llama-lora-test

joschu0/tinker-llama-lora-test

This model is a LoRA adapter fine-tuned with [Tinker](https://thinkingmachines.ai/tinker) from Thinking Machines Lab.

Model Details

Usage

With PEFT

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "joschu0/tinker-llama-lora-test")

With Tinker (for sampling/inference)

This checkpoint is also available on Tinker for high-throughput sampling:

import tinker

sc = tinker.ServiceClient()
sampling_client = sc.create_sampling_client("tinker://7f4705e7-551f-5133-b4bb-33444c0c405b:train:0/sampler_weights/test-push-to-hub")
result = sampling_client.sample(...)

Tinker path: tinker://7f4705e7-551f-5133-b4bb-33444c0c405b:train:0/sampler_weights/test-push-to-hub

Note: This is a sampler checkpoint and can only be used for inference, not for continued training.

Training

This model was trained using the Tinker API. For more information about training with Tinker, see the Tinker documentation.

<p align="center"> <em>Trained with <a href="https://thinkingmachines.ai/tinker">Tinker</a> by Thinking Machines Lab</em> </p>