joe-xhedi/Qwen-GRPO-geological-training

Qwen-GRPO-geological-training

GRPO-trained Qwen model specialized for geological questions and analysis

Model Details

• —Base Model: Qwen/Qwen2.5-0.5B-Instruct
• —Training Method: GRPO (Generalized Reward Preference Optimization)
• —Domain: Geology and Earth Sciences
• —Model Type: Causal Language Model
• —Architecture: Transformer-based

Training Details

This model was trained using GRPO (Generalized Reward Preference Optimization) on geological datasets. The training process included:

• —Reward Functions:
• —Geological accuracy reward
• —Format compliance reward
• —Reasoning steps reward
• —System Prompt: Specialized geological expert system prompt
• —Response Format: Structured thinking process with solution tags

Intended Use

This model is designed for:

• —Answering geological questions
• —Providing educational content about earth sciences
• —Assisting with mineral identification
• —Explaining geological processes
• —Rock and mineral analysis

Usage Example

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load the model
model_name = "joe-xhedi/Qwen-GRPO-geological-training"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)

# Geological system prompt
system_prompt = '''You are a geological expert assistant. When answering geological questions, follow this format: 
First, analyze the problem step by step in your thinking process within <|begin_of_thought|> and <|end_of_thought|> tags. 
Then provide your solution within <|begin_of_solution|> and <|end_of_solution|> tags. 
Your thinking process should include geological principles, data analysis, and reasoning. 
Your solution should be clear, accurate, and based on geological expertise.'''

# Example usage
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "What type of rock is formed by cooling magma?"}
]

# Generate response
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Response Format

The model is trained to respond in a structured format:

<|begin_of_thought|>
[Step-by-step geological reasoning and analysis]
<|end_of_thought|>

<|begin_of_solution|>
[Clear, accurate geological solution or explanation]
<|end_of_solution|>

Limitations

• —Specialized for geological topics
• —May not perform well on general conversational tasks
• —Responses are structured and may seem formal
• —Based on training data available up to the training cutoff

Training Data

The model was trained on geological datasets including:

• —Mineral identification questions
• —Rock formation processes
• —Geological principles and concepts
• —Earth science educational content

Ethical Considerations

• —This model is designed for educational and research purposes
• —Users should verify geological information for professional applications
• —The model may have biases present in the training data

Citation

If you use this model in your research, please cite:

@model{qwen-geological-expert,
  author = {joe-xhedi},
  title = {GRPO-trained Qwen Model for Geological Analysis},
  year = {2025},
  url = {https://huggingface.co/joe-xhedi/Qwen-GRPO-geological-training}
}

Model Card Contact

For questions about this model, please contact the model author through Hugging Face.