cs-552-2026-databand/general_knowledge_model

General Knowledge Model

This is the final General Knowledge individual model for the CS-552 Modern NLP Spring 2026 standardized project.

The submitted model is the SFT-only merged model. A later DPO experiment was run on ARC/CommonsenseQA mistakes, but it reduced external benchmark accuracy, so it was not selected as the final model.

Model behavior

The model is specialized for multiple-choice general knowledge questions. It is prompted to output exactly one final boxed answer, for example:

\boxed{A}

The chat template enforces concise answer-only behavior and supports choices labeled from A through T.

Training setup

Starting point:

• —Baseline working model folder with the project chat template and generation config
• —LoRA SFT on top of the baseline model
• —Final model produced by merging the LoRA adapter into the baseline model

Training method:

• —LoRA supervised fine-tuning
• —Loss masked so that only the final assistant boxed answer contributes to training
• —Prompt, system message, question text, choices, chat markers, and template tokens are masked with -100
• —Assistant target format: \boxed{LETTER}

LoRA configuration:

• —r = 16
• —lora_alpha = 32
• —lora_dropout = 0.05
• —Target modules:
• —q_proj
• —k_proj
• —v_proj
• —o_proj
• —gate_proj
• —up_proj
• —down_proj

Main training hyperparameters:

• —Learning rate: 8e-5
• —Epochs: 1
• —Batch size per device: 1
• —Gradient accumulation steps: 8
• —Max sequence length: 8192
• —Precision: bf16
• —Scheduler: cosine
• —Warmup steps: 20

SFT datasets

The SFT training data was built from:

1. 1.Kaggle LLM Science
2. 2.EduQG
3. 3.EduAdapt, MCQ-only questions
4. 4.NCERT_MCQs
5. 5.SciQ train
6. 6.OpenBookQA train

Final SFT data sizes:

• —Train: 26,120
• —Validation: 2,000

The answer labels were balanced uniformly across A through T separately for train and validation.

Train answer distribution:

• —A through T: 1,306 examples each

Validation answer distribution:

• —A through T: 100 examples each

Evaluation

The final selected model is the SFT-only merged model.

The “SFT validation” set in the table is the held-out validation set created from the same six dataset families used for LoRA SFT training: Kaggle LLM Science, EduQG, EduAdapt MCQ, NCERT_MCQs, SciQ, and OpenBookQA. It contains 2,000 examples and is answer-balanced across A through T.

External benchmark sets:

• —MMLU Pro: 2,000 examples, uniformly sampled across categories
• —MMLU Redux: 2,000 examples, uniformly sampled across subjects
• —SuperGPQA: 2,000 examples, uniformly sampled across disciplines

The DPO experiment improved neither the selected SFT validation score nor the external benchmark scores. Therefore, the SFT-only merged model was selected as the final model.

SFT validation details

SFT-only evaluation on the held-out SFT validation set:

• —Total: 2,000
• —Extracted boxed answer: 2,000 / 2,000 = 100.00%
• —Accuracy: 1,706 / 2,000 = 85.30%

Accuracy by validation source:

SuperGPQA boxed-answer edge case

The SFT-only model produced boxed answers for 1,999 out of 2,000 SuperGPQA examples. The single unboxed example was a long, LaTeX-heavy numerical analysis question whose answer choices contained multi-line mathematical derivations. Instead of producing a boxed option, the model continued/copy-completed part of one answer choice, generating text beginning with:

mathrm{d} x^{2} = 2.1730$ and $| R_{1} | ...

Increasing maxnewtokens from 20 to 64 did not change this outcome. The reported SuperGPQA result therefore keeps the strict extraction score of 99.95%.

Expected input format

The model expects a multiple-choice question formatted like:

Question text here?

Choices: A. first option B. second option C. third option D. fourth option

It should answer with only:

\boxed{A}

Reproducibility notes

In the Github repo CS-552/standard-project-m3-databand/tree/main, you can find in the folder code/general_knowledge all scripts and files needed for reproducibility.

A README.md file is also present in code/general_knowledge folder, documenting all script functionalities and how to run them to reproduce both SFT and SFT+DPO experiments.

Important files from the code/general_knowledge folder:

• —SFT trainer: scripts/trainv3lorasftmasked.py
• —SFT data builder: scripts/buildmysftdatabalanced.py
• —DPO trainer used for the unselected experiment: scripts/trainv3loradpoboxed.py
• —Merge script: scripts/mergev3lora_adapter.py
• —Evaluation script: scripts/evaluatemcqaccuracy.py

Final selected model folder before upload:

outputs/lorasftv3boxedonly/mergedfullmodel

SFT LoRA adapter:

outputs/lorasftv3boxedonly/final_adapter

DPO adapter, experimental and not selected:

outputs/loradpoarccsqaonsft/finaladapter