wego80712/Fairness_Dashboard
FairSearch-arXiv Fairness Dashboard
Interactive results for a study of institutional bias in Retrieval-Augmented Generation over arXiv Computer Science papers. Course project, Information Retrieval (SEC 01, Summer 2026).
This README is the HuggingFace Space configuration and description. The repository root has a separate, longer README covering the full pipeline, reproduction steps and methodology.
What the dashboard shows
Overview describes the corpus (48,931 papers labeled by author affiliation), the experiment setup, and Phase I retrieval quality.
RQ1: Retrieval Parity audits whether dense retrieval over-represents papers from elite institutions. Over 100 neutral queries it does not: exposure sits at parity in aggregate, with a Wilson 95% interval that includes 1.0. The page then disaggregates that result, which conceals a significant reversal by paper era and a wide spread across subfields, and shows why an earlier 18-query result is superseded rather than averaged in.
RQ2: Synthesis Neutrality measures whether the LLM amplifies whatever institutional skew retrieval produced. These numbers are still computed on the earlier 18-query set and the page says so; regenerating them on the 100-query set is outstanding work.
RQ3: Fairness-Utility Tradeoff compares two fairness-aware re-rankers against the baseline. Because the baseline is already at parity, the soft penalty (FairMMR-Inst) moves exposure away from parity as its strength increases, while the hard quota (Fair-Top-K) is close to inert and attains the best retrieval quality. The transferable result is that fairness-aware re-ranking should be gated on a measured disparity.
Every exposure figure carries a Wilson 95% confidence interval, and each page footer names the query-set version the numbers came from so a stale data copy is visible rather than silent.
Running it
On a Space, nothing is needed: sdk: streamlit with app_file: Overview.py installs requirements.txt and launches the app. No Dockerfile is used or needed, since this is pure Python with no system packages, no compiled extensions and no custom server. A Dockerfile would add a build step and a second dependency surface for no benefit.
On Streamlit Community Cloud, set the entrypoint to src/fairness_dashboard/Overview.py. Community Cloud sets the working directory to the repository root, which is fine: every path the app reads is resolved against this directory via Path(__file__).parent, including the st.image calls, so none of them depend on the working directory.
Community Cloud searches the entrypoint file's directory for a dependency file before the repository root, and the entrypoint-directory file wins. That is why requirements.txt lives here: the root requirements.txt is the full research pipeline and pins chromadb, which has no Python 3.12 distribution. Do not delete this directory's requirements.txt or the build will fall back to the root file and fail.
Locally, from anywhere in the repository:
pip install -r src/fairness_dashboard/requirements.txt
streamlit run src/fairness_dashboard/Overview.pyAll data and figure paths resolve against the app directory rather than the working directory, so the launch location does not matter.
Data
The app reads its own copies under data/ and results/ inside this directory, which keeps the Space self-contained. They are refreshed from the repository root by:
python src/sync_dashboard_data.pyRegenerating the audit without running that leaves the dashboard rendering the previous numbers.