cs-552-2026-momy/general_knowledge_model

General Knowledge Model — CS-552 (MOMY)

A reasoning-focused model for general-knowledge question answering, post-trained from `Qwen/Qwen3-1.7B` using GRPO (Group Relative Policy Optimization) with verifiable rewards (RLVR). Developed for the EPFL CS-552 Modern NLP course project (Spring 2026).

The model reasons step-by-step inside <think>...</think> tags before producing a final answer enclosed in a \boxed{} environment, supporting automated answer extraction and verification.

Model Details

• —Base model: Qwen3-1.7B
• —Training method: GRPO (RLVR) with LoRA (r=16, α=32) adapters merged into base weights
• —Domain: General knowledge — science, history, geography, world affairs
• —Output format: <think> reasoning chain followed by \boxed{ANSWER}
• —Team: MOMY

Intended Use

The model answers both multiple-choice and short open-ended factual questions:

• —Multiple-choice: outputs the correct option letter, e.g. \boxed{B}
• —Open-ended: outputs a short factual answer, e.g. \boxed{Paris}

Training Data

A total of 8,100 training and 900 validation examples (90/10 split), each formatted with a chain-of-thought target terminating in Therefore, the final answer is \boxed{·}.

Training Configuration

Reward function: +1.0 for a correct boxed answer, +0.1 for a valid <think> block, -0.1 for a missing/malformed box, -0.1 for exceeding the token budget.

Evaluation

GRPO substantially improves over the base model on the held-out CI knowledge benchmark.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "cs-552-2026-momy/general_knowledge_model"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")

messages = [{"role": "user", "content": "What is the capital of Australia?"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95, top_k=20)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The chat template hardcodes enable_thinking=true, so the model always reasons before answering. A generation_config.json with the recommended inference parameters is included in the repo.

Limitations

• —As a 1.7B-parameter model, factual coverage is limited; performance on long-tail or highly specialized knowledge is unreliable.
• —The dominant failure mode is option-matching miscalibration: the model may derive a correct value in its reasoning but select a mismatched option when the computed answer is not listed among the choices.
• —Knowledge is skewed toward English-language and Western-centric sources (MMLU-Pro, TriviaQA).

Citation

If referencing this model, please cite the underlying methods:

• —GRPO: Shao et al., DeepSeekMath (2024)
• —MMLU-Pro: Wang et al. (2024)
• —TriviaQA: Joshi et al. (2017)
• —Base model: Qwen3-1.7B (Qwen Team, 2025)