Felixbrk/bert-base-dutch-cased-multi-score-tuned-positive

transformermultiheadbertupdated

A multi-head transformer regression model based on BERT (GroNLP/bert-base-dutch-cased), fine-tuned to predict four normalized delta scores for Dutch book reviews. The four output heads are:

1. 1.delta_cola_to_final
2. 2.delta_perplexity_to_final_large
3. 3.iter_to_final_simplified
4. 4.robbert_delta_blurb_to_final

⚠️ The order of these outputs is crucial and must be maintained exactly as above during inference. Changing the order will cause incorrect mapping of predicted values to their respective targets.

Additionally, a final aggregate score is provided (mean of the four heads).

📈 Training & Evaluation

• —Base model: GroNLP/bert-base-dutch-cased
• —Fine-tuning: 5 epochs on a proprietary dataset
• —Output heads: 4
• —Problem type: multi-head regression

Per-Epoch Validation Metrics

✅ Final Aggregate Performance (Test)

🗂️ Test Metrics (Per Target)

🏷️ Notes

• —Base model: GroNLP/bert-base-dutch-cased
• —Fine-tuned for multi-head regression on Dutch book reviews
• —Trained for 5 epochs on a proprietary dataset
• —Sigmoid activation built into each head
• —Re-aggregation: simple average of the four head outputs

🛠️ Training Arguments

• —num_train_epochs=5
• —per_device_train_batch_size=8
• —per_device_eval_batch_size=16
• —gradient_accumulation_steps=2
• —learning_rate=2e-5
• —weight_decay=0.01
• —eval_strategy="epoch"
• —save_strategy="epoch"
• —load_best_model_at_end=True
• —metric_for_best_model="mean_rmse"
• —greater_is_better=False
• —bf16 enabled if supported, else fp16 enabled
• —logging_strategy="epoch"
• —push_to_hub=True with model ID Felixbrk/bert-base-dutch-cased-multi-score-tuned-positive
• —hub_strategy="end"
• —Early stopping with patience 2 epochs

⚠️ Important:

• —Always load this model with trust_remote_code=True as it uses a custom multi-head regression architecture.
• —Maintain the output order exactly for correct interpretation of results.