saxon11/pricer-2025-02-25_22.36.49
0
Pricer — Llama 3.1 8B Fine-Tuned for Product Price Prediction
A QLoRA fine-tuned version of Meta-Llama-3.1-8B trained to predict the price of a product given its description. The model takes a product title, description, and attributes as input and generates a dollar-amount price estimate.
Intended Use
Given a prompt like:
How much does this cost to the nearest dollar? \[Product title and description\] Price is $
The model completes the text with a predicted price value.
Training Details
Base Model
- Model:
meta-llama/Meta-Llama-3.1-8B - Quantization: 4-bit (NF4) via BitsAndBytes with double quantization and bfloat16 compute dtype
Dataset
- Dataset: saxon11/pricer-data-small
- Train split: 20,000 rows
- Test split: 8,544 rows
- Max sequence length: 182 tokens
QLoRA Configuration
Training Hyperparameters
Training Results
Training loss decreased steadily from 1.97 at step 50 to 1.13 by the end of training, indicating the model learned the price prediction task effectively.
Data Collation
A completion-only data collator was used with the response template "Price is $", ensuring the model is only trained to predict the price tokens rather than the product description.
Framework & Libraries
- Transformers (HuggingFace)
- PEFT (Parameter-Efficient Fine-Tuning)
- TRL (Transformer Reinforcement Learning — SFTTrainer)
- BitsAndBytes (4-bit quantization)
- Weights & Biases (experiment tracking)
Limitations
- Trained on a relatively small dataset (20K examples) of product descriptions, primarily appliance parts and accessories.
- Prices in the training set are concentrated in the $1–$101 range (87.3% of data), so predictions outside this range may be less reliable.
- This is a LoRA adapter — it requires the base Llama 3.1 8B model to run inference.