vmcii/iclr-acceptance-predictor

ICLR Paper Acceptance — Conference Extension v2 (Final, Calibrated)

This Space ships v2, which is the final calibrated deployable model. v2 supersedes the earlier v1 (Phase 2) baseline: the calibrated v2 model has higher validation AUC, materially better probability calibration (Brier and ECE), and the same leakage-safe, pre-submission-only input contract. Final deployable bundle: best_conference_deployable_model_v2.joblib Model: S2v2 TFIDF+Meta+ScoreProxy LR (v2 score) — Platt-calibrated Decision threshold (validation-tuned): 0.240

A leakage-safe, two-stage pipeline that predicts ICLR paper acceptance from pre-review information only.

Stage 1 — Reviewer-score proxy (6-model ensemble)

Each base regressor is fit on training years 2017–2021; 5-fold OOF inside training years prevents within-train leakage. Validation (2022–2023) and test (2024) predictions come from full-train fits. The ensemble exposes predicted_score_mean / predicted_score_std / predicted_score_range to Stage 2.

1. 1.Ridge on TF-IDF (1–2-gram, sublinear).
2. 2.Ridge on SPECTER (allenai/specter, frozen, mean-pooled).
3. 3.ElasticNet on SPECTER.
4. 4.Ridge on MiniLM (sentence-transformers/all-MiniLM-L6-v2, frozen).
5. 5.ExtraTrees on TF-IDF-SVD + metadata.
6. 6.HistGradientBoosting on TF-IDF-SVD + metadata.

Stage 2 — Acceptance classifier (Platt-calibrated)

TF-IDF + metadata + Stage 1 score proxy → Logistic Regression. Calibration via Platt scaling (sigmoid) using internal 5-fold CV on the training set only — calibration is never fit on validation or test.

Inputs (deployable, all pre-review)

• —title
• —abstract
• —authors (comma-separated)
• —keywords (comma-separated)
• —year
• —arXiv flag
• —topic label

Outputs (Gradio app)

• —calibrated acceptance probability
• —likely accepted / likely rejected (relative to validation-tuned threshold 0.240)
• —decision threshold (shown for transparency)
• —predicted reviewer-score proxy (mean) and Stage 1 disagreement (std)
• —uncertainty level (Low / Medium / High) with explanation
• —calibration note (Brier + ECE on validation)
• —disclaimer that this is a decision-support research demo, not peer review

Headline metrics (validation 2022–2023 / test 2024)

For comparison, the v1 (Phase 2 baseline) two-stage model was 0.676 / 0.688 val/test AUC at ECE ≈ 0.08; the Oracle (real reviewer scores) was 0.967 / 0.952 val/test AUC.

Stage 1 v2 (validation MAE / test MAE, lower is better)

SPECTER alone does not beat Ridge TF-IDF, but the ensemble (which includes SPECTER as 1 of 6 base models) outperforms every single regressor.

Leakage rules (audited)

• —Real reviewer scores are used only as Stage 1 regression target and as an Oracle benchmark — never as deployable inputs.
• —Reviews, rebuttals, reviewer confidence, decision text, the accepted label, and reviewer identity are excluded from the deployable model.
• —All preprocessing (TF-IDF, SVD, scalers, encoders, calibration) is fit on training years (2017–2021) only. Validation (2022–2023) and test (2024) are only transformed.

Disclaimer

This is a decision-support research demo, not a replacement for peer review. Example inputs are synthetic and used only to demonstrate model behavior.

Files in this Space