HoangHa/Pensez-v0.1-e1
<div align="center">
Pensez: Less Data, Better Reasoning – Rethinking French LLM
**About** | **How to Run Locally** | **Models and Datasets** | **Benchmarks** | **Training Details**
</div>
About
Pensez is a bilingual (French-English) reasoning model designed to maximize efficiency with significantly reduced training data. The model leverages a curated dataset focusing on daily reasoning tasks and scientific questions to enhance performance.
Key strategies for improved reasoning:
- Concise reasoning for simple tasks to prevent overthinking.
- Extended reasoning for complex domains like mathematics, coding, and science.
- Special tokens (`<think>...</think>`) to explicitly guide the model’s reasoning process.
These optimizations result in superior reasoning capabilities while maintaining robust general understanding compared to models like DeepSeek-R1-Distill-Qwen-7B.
Models and Datasets
Model Versions
Pensez is built upon Qwen 2.5 Instruct 7B and trained over five epochs.
Dataset
Pensez was trained on the hand-curated Pensez v0.1 dataset containing 2,000 samples (1,000 French, 1,000 English).
Benchmarks
Pensez was evaluated on French-specific benchmarks, demonstrating strong reasoning ability and improved task-specific performance:
Key Observations:
- Pensez outperforms Qwen2.5-7B-Instruct in reasoning tasks.
- Comparable to DeepSeek-R1-Distill-Qwen-7B in reasoning while maintaining strong understanding.
- Reduced degradation in knowledge-based tasks.
<details> <summary>Click for detailed benchmark results</summary>
</details>
Run Locally
You can run Pensez using Hugging Face’s transformers library:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_path = "HoangHa/Pensez-v0.1-e5"
# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
model_path, torch_dtype=torch.float16, device_map="auto"
)
# Example input
messages = [{"role": "user", "content": "Bonjour!"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors='pt').to("cuda")
generated_ids = model.generate(input_ids, max_new_tokens=2500, temperature=0.8, repetition_penalty=1.1, do_sample=True, eos_token_id=tokenizer.eos_token_id)
response = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_space=True)
print(f"Réponse: {response}")Training Details
Pensez was trained with:
- Packing Inputs Without Cross-Contamination Attention (Reference)
- Liger Kernel (Reference)
- DeepSpeed 3 (Reference)
- NEFTune Noise (Reference) for robustness.
More details: [Training Config]() | Loss curves: Wandb
Citation
@misc{dao2025alphamazeenhancinglargelanguage,
title={Pensez: Less Data, Better Reasoning – Rethinking French LLM},
author={Ha Huy Hoang},
year={2025},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={},
}