beaunix/aegis-geo-mind-qwen2.5-7b-bnb-4bit

Aegis-Geo-Mind

A QLoRA fine-tune of Qwen2.5-7B-Instruct specialized in geology and petroleum geology, covering general geology, petroleum systems, sedimentary basin analysis, sequence stratigraphy, and well log interpretation.

This repository hosts the 4-bit (bitsandbytes, nf4) version of the model, optimized for inference on lower-VRAM hardware (e.g. a T4 GPU).

• —Dataset: beaunix/geo-mind-qa
• —Demo Space: beaunix/aegis-geo-mind-demo

Model description

Aegis-Geo-Mind is fine-tuned to explain and reason about geological and petroleum-geology concepts in fluent, domain-appropriate language, including correct technical terminology across:

• —Structural geology, stratigraphy, and sedimentology
• —Petroleum systems (source rock, migration, traps, seals, maturation)
• —Sequence stratigraphy and sedimentary basin analysis
• —Well log interpretation (gamma ray, resistivity, sonic, neutron/density, caliper, mud logs)

The model was built to address a gap in publicly available geoscience LLMs: none of the existing open geology-focused models specialize in petroleum geology.

This is a research preview. See Limitations below before relying on any specific factual or numeric claim.

Training data

• —Base filtered subset of daven3/geosignal (K2 corpus), reduced from 39,749 to 18,054 rows through five rounds of quality filtering (removal of generic instruction filler, noisy NER categories, bibliography-only answers, broken template artifacts, and empty-field placeholders).
• —818 original, hand-curated general-geology Q/A pairs.
• —125 original, hand-curated petroleum-geology Q/A pairs (sedimentology of petroliferous basins, reservoir stratigraphy, well logging, seismic interpretation).

The curated pairs were oversampled ×5 within the training split (never in validation) to offset their small share of the combined corpus. Final training set: 21,639 rows. Validation set: 940 rows, held disjoint from training at every merge and filtering step.

Full dataset available at beaunix/geo-mind-qa.

Training procedure

Batch configuration was selected via a dedicated throughput benchmark (examples/second, not steps/second) across multiple batch/accumulation combinations, rather than assumed.

Evaluation

Loss

Best validation loss: 1.560 (step 1,250 of 2,031).

A validation loss in this range should not, by itself, be read as evidence of a weak model. Much of the training data consists of open-ended explanatory answers where multiple correct phrasings exist — token-level loss penalizes deviation from one reference wording even when the model's own answer is equally correct. This produces an irreducible loss floor tied to the dataset's phrasing diversity, not a direct measure of geological competence.

Test set

Evaluated on 60 held-out Q/A pairs (disjoint from training). Test perplexity: 6.12.

Manual review of generated answers found:

• —Correct, well-structured reasoning on sequence stratigraphy, basin classification, trap types, hydrocarbon migration, and well-log interpretation, using accurate domain terminology.
• —No fabricated formation names or citations observed.
• —Confirmed factual errors on specific numeric/ratio claims (see Limitations).

How to use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model_id = "beaunix/aegis-geo-mind-qwen2.5-7b-bnb-4bit"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, quantization_config=bnb_config, device_map="auto"
)

messages = [
    {"role": "system", "content": "You are an expert geologist specializing in Earth Sciences."},
    {"role": "user", "content": "What are growth faults and why are they important in sedimentary basins?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.9)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limitations

This model is a supervised fine-tune without retrieval augmentation. It reliably reproduces the style and reasoning structure of expert geological explanation, but can state specific facts, numbers, and ratios incorrectly with high confidence. Confirmed weak areas from evaluation:

• —Kerogen type classification — H/C and O/C ratio relationships can be inverted between Type I and Type III kerogen.
• —Numeric ranges — has confused source-rock maturation temperature with burial depth in at least one generation.
• —Fabricated specific values — observed at least one invented, precise numeric claim (a halite ductile-brittle transition temperature) presented as fact.

For any factual, numeric, or operational use, verify claims against authoritative references (e.g. Tissot & Welte, Selley, Schlumberger logging references). A retrieval-augmented (RAG) version addressing these gaps is planned.

Not for operational decisions. This model must not be used as a sole basis for exploration, drilling, reservoir, or any other operational geological decision.

License

MIT.

Citation

If you use this model, please cite the repository:

@misc{aegis-geo-mind-2026,
  author = {beaunix},
  title = {Aegis-Geo-Mind: A QLoRA Fine-tune of Qwen2.5-7B for Geology and Petroleum Geology},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/beaunix/aegis-geo-mind-qwen2.5-7b-bnb-4bit}
}