flamiinngo/agronomy-llama-3.3-70b

Agronomy-Llama-3.3-70B — Agriculture Q&A Model

A LoRA fine-tune of Llama-3.3-70B-Instruct adapted for agriculture and agronomy question answering: crop management, planting, soil health, irrigation, pest and disease control, and livestock care.

Built with Adaption Labs' AutoScientist as an entry in the AutoScientist Challenge (Agriculture category).

Result

On Adaption Labs' held-out Agriculture evaluation, this model decisively outperforms its base in head-to-head win rate:

A confirmation run reproduced the direction and margin (adapted 71 vs base 29), indicating a stable improvement rather than a single lucky evaluation.

How the evaluation worked

Head-to-head win rate on a held-out agriculture question set that was not part of training. Each model answers the same question; the stronger answer wins. Scores are win counts, not accuracy percentages — so "82 vs 19" means the adapted model's answer was preferred on 82 items and the base model's on 19.

Both numbers come from Adaption Labs' evaluation harness, run twice on separate training runs. Ties are excluded, which is why the pairs do not sum identically.

Usage

The weights are distributed as a zstd-compressed tar archive. It extracts flat, so unpack it into a directory of its own:

mkdir -p agronomy-adapter
tar --zstd -xf agronomy-llama-3.3-70b-weights.tgz -C agronomy-adapter

Contents: adapter_config.json, adapter_model.safetensors (3.3 GB), plus the tokenizer and chat_template.jinja.

Then load the adapter on top of the base model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "meta-llama/Llama-3.3-70B-Instruct"
ADAPTER = "./agronomy-adapter"

# Load the tokenizer from the adapter directory - it ships the chat template
# this model was trained with.
tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForCausalLM.from_pretrained(
    BASE, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

messages = [
    {"role": "system", "content": "You are an agricultural advisor. Answer concisely and factually."},
    {"role": "user", "content": "How deep should maize seeds be planted?"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

out = model.generate(inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Hardware: the base model is 70B parameters — expect ~140 GB in bf16, or roughly 40 GB with 4-bit quantization (load_in_4bit=True). The LoRA adapter itself is small; the base model dominates the footprint.

Prompting: use the system prompt above, or one like it. The model was trained to answer briefly. Asking it to "explain in detail" works against what it was optimized for.

Note on `adapter_config.json`: its base_model_name_or_path records togethercomputer/Meta-Llama-3.3-70B-Instruct-Reference, the base as served during training. It is the same Llama-3.3-70B-Instruct architecture — loading against meta-llama/Llama-3.3-70B-Instruct as shown above works.

How it was built

• —Base model: meta-llama/Llama-3.3-70B-Instruct (80 layers, bf16)
• —Method: LoRA supervised fine-tuning via AutoScientist (recipe auto-optimized)
• —Training data: 1,914 concise, real agriculture Q&A pairs (see Dataset below)
• —Training time: roughly one hour on free-tier compute
• —Key insight: the held-out evaluation rewards concise, direct, factual answers. Training on short real answers rather than long generated essays was the decisive factor in beating the base model.

LoRA configuration

All attention and MLP projections are adapted, at rank 64 — which is why the adapter is 3.3 GB rather than the few hundred megabytes a low-rank attention-only LoRA would produce.

Training curve

Training loss fell from 1.549 to 0.700 over the run. Evaluation loss bottomed at epoch 2.41 and ticked up marginally by epoch 3 — the beginning of overfitting, though the change is small enough to be noise. A 2-epoch run would likely perform comparably.

An earlier phase of this project targeted HR question answering and lost to the baseline across eight training runs — base models are already strong there, and well-licensed HR data is scarce. Agriculture won because the base model is genuinely weak in the domain and real Apache-2.0 data exists.

Dataset

Trained on [flamiinngo/agronomy-qa-agriculture](https://huggingface.co/datasets/flamiinngo/agronomy-qa-agriculture) — 1,914 concise agriculture Q&A pairs (median answer 27 words), derived from KisanVaani/agriculture-qa-english-only (Apache 2.0): real agricultural Q&A curated from farming forums, extension resources, and FAQs.

Also mirrored on Kaggle: dataset · weights

Intended use

Answering practical agriculture and agronomy questions, and as a reference point for domain adaptation experiments. Best suited to concise, factual guidance.

Limitations

• —Not professional advice. Agricultural practice varies by region, crop, climate, and regulation. Always confirm with local agricultural extension services. Not a substitute for veterinary advice on livestock.
• —Source data skews toward Indian agriculture — the upstream KisanVaani corpus was curated largely from Indian farming forums and extension resources. Crop varieties, seasons, and pest pressures reflect that context and may not transfer to other regions.
• —Brevity is a design choice, not always the right one. The model is tuned to answer short. For questions that genuinely need nuance or caveats, it may under-explain.
• —Evaluated on one held-out set with an automated harness. Win rate over a base model is not a measure of factual accuracy in absolute terms.
• —English only.

License

This model is a derivative of Llama-3.3-70B-Instruct and is subject to the Llama 3.3 Community License. The training dataset is Apache 2.0.

Acknowledgements

• —Adaption Labs — AutoScientist platform and the AutoScientist Challenge
• —KisanVaani — source agriculture Q&A data
• —Meta — Llama 3.3 base model