EmmaScharfmann/science-releases

HF Science Digest — POC

Fetches recently-released Hugging Face artifacts (models, datasets, blog posts) in strict domain-science areas (biology, chemistry, physics, medicine, climate, astronomy), uses an LLM to judge which are genuinely interesting to the community, and serves them to a Slack slash command.

How selection works

1. 1.Domain filter (domain_filter.py) — Hub API search= queries per domain keyword, then a verification pass against tags/description so a fuzzy search match doesn't slip through as a false positive.
2. 2.Recency window (config.LOOKBACK_DAYS, default 30) — domain-science content on HF is a long tail with sparse engagement, so a short window (e.g. 24h) often returns nothing. 30 days is a reasonable POC default.
3. 3.Heuristic pre-filter (scoring.py) — combines HF's own trendingScore, likes, and log-scaled downloads into one number, used ONLY to cap the candidate pool (config.MAX_CANDIDATES_FOR_LLM, default 12 per type) before the more expensive LLM step. This is not the final ranking.
4. 4.LLM interestingness judging (interest_judge.py) — each pre-filtered candidate is sent to an LLM via Hugging Face's Inference Providers router, which scores it 1-10 on genuine relevance/novelty to the science/ML community, independent of its popularity metrics. Items below config.LLM_INTEREST_THRESHOLD (default 6) are dropped; the rest are ranked by LLM score.
5. 5.Format & deliver (formatter.py, app.py) — final picks become a Slack Block Kit message.

Setup

pip install -r requirements.txt
export HF_TOKEN=hf_...   # https://huggingface.co/settings/tokens

Test the pipeline without Slack

python3 digest.py

Prints the digest as plain text plus each item's LLM-assigned score and one-line reason — the fastest way to sanity-check the selection quality and tune config.py (lookback window, thresholds, model choice) before wiring up Slack.

Run the backend

uvicorn app:app --reload --port 8000

POST /health → {"status": "ok"} to confirm it's up.

Wire up the Slack slash command

1. 1.Expose the local server publicly for testing (e.g. ngrok http 8000).
2. 2.In your Slack app config → Slash Commands → create /hf-digest (or any name) with Request URL: https://<your-domain>/slack/digest.
3. 3.Slack will POST form-encoded data including response_url on each invocation; the endpoint acks immediately and posts the real digest to that URL once ready (needed because building the digest — Hub calls + LLM judging — usually takes longer than Slack's 3-second ack window).

Before exposing this beyond your own testing: add Slack request signature verification using your app's signing secret (https://api.slack.com/authentication/verifying-requests-from-slack) — not yet implemented in this POC.

Known POC limitations / next steps

• —No caching/dedupe wired into the on-demand path — seen_store.py exists (flat-file, tracks which artifact IDs have already been shown) but isn't called yet. Needed once you add a scheduled digest, so it doesn't repost the same item every run.
• —Sequential LLM calls — judge_batch calls the LLM one item at a time. Fine at this volume (≤12 candidates/type); parallelize if the candidate pool grows.
• —No persistence/history — nothing is stored between runs beyond the optional seen-store.
• —Blog RSS feed has no engagement metric — blogs are pre-filtered by recency only before the LLM step, unlike models/datasets which also get the heuristic score.
• —Domain keyword dictionary is a starting point — domain_filter.py's verify_terms lists are intentionally short; expand as you see false negatives (real science content that isn't matching) in practice.
• —Scheduled mode — the pipeline (digest.py) has no Slack/FastAPI dependency, so a cron job calling run_digest() and posting to an Incoming Webhook is a small addition, not a rewrite.