bloomee-app/bloomee-sft-nasasmd-grounded-5m

<div align="center"> <img src="assets/logo.png" alt="Bloomee Logo"/>

🌸 bloomee-sft-nasasmd-grounded-5m

Supervised fine-tuning corpus of grounded, tool-calling conversations about flowering phenology

Every answer traced back to the chunks it was drawn from, and scored against them

   

🏆 Part of the Bloomee platform — NASA Space Apps Challenge 2025

Built by Team Ganespace for Bandung, Indonesia

</div>

🌍 About

1,899 conversations that teach a small model to behave like the Bloomee agent: call the right NDVI tool with the right arguments, ground answers in retrieved phenology literature, refuse places outside the catalogue, ask before guessing, and stay useful when the upstream API is down.

It was produced by distilling ibm-granite/granite-4.1-8b through the production agent graph — the same retrieval node, the same four tools, the same prompt — against a recorded fixture grid, then discarding every sample that failed validation. Running the real graph is the point: the corpus reflects how the application actually behaves rather than a reimplementation that could drift from it.

Of 5,652 generated samples, 2,226 passed validation (39.4%), and 1,899 survived best-of-n selection, diversity demotion and intent balancing. The high rejection rate is by design — the rules discard behaviours that would actively harm a student model.

This corpus trained `bloomee-v1-clm-nasasmd-granite4.1-3b-adapter`.

🚀 Usage

python load_dataset.py

from load_dataset import load, split_by_seed

rows = load()
train, evaluation = split_by_seed(rows)   # 1,796 / 103 rows over 754 / 39 seeds

⚠️ Split by seed_id, never by row

One seed produces up to eight samples of the same user turn, and a multi-turn conversation contributes one row per turn. A row-level split puts near-identical siblings on both sides, so the eval loss measures memorisation rather than generalisation — it reads better than the truth, which is the worst way for a metric to be wrong.

split_by_seed() reproduces the published split exactly (seed=20260731, eval_fraction=0.05). The fine-tune additionally dropped 11 rows exceeding a 4,096-token cap, training on 1,787 / 101; that filter is tokenizer-dependent and is not applied here.

Rows are in OpenAI chat format, so they feed a TRL/PEFT pipeline directly. Two things to carry over from how they were built:

📐 Schema

One JSON object per line: {"messages": [...], "meta": {...}}.

messages follows OpenAI chat format — system, user, assistant (optionally with tool_calls), tool (with tool_call_id). Role counts across the corpus: 1,899 system · 2,212 user · 5,079 assistant · 2,954 tool.

NASA-QA rows carry three extra keys: dataset, gold_span, rewritten.

📊 Composition

Eight regions are covered: japan_cherry, usa_cherry_dc, netherlands_tulips, france_lavender, uk_bluebells, california_poppies, texas_bluebonnets, bandung_floriculture.

🛰️ Provenance

The nasasmd-grounded in the name refers to NASA's Science Mission Directorate sentence transformer, which scored how well each candidate answer was supported by the chunks actually retrieved for it. It is deliberately not the retrieval encoder: one model both selecting the chunks and judging groundedness would measure self-agreement rather than grounding — it would reward answers that echo its own notion of similarity, which is the exact failure the check exists to catch. The two tied on every retrieval metric, so the tie was broken on that principle.

Grounding was a ranking term, never a gate. It re-ordered survivors so best-of-n picked better samples; it never rejected one. 536 of the 1,899 rows carry a score.

83 rows come from `nasa-impact/nasa-smd-qa-benchmark` (train split only, Sahara-colliding pairs removed), supplying Earth-science register rather than domain knowledge — only 3 of the 83 touch vegetation.

NDVI values come from a recorded fixture grid (559 responses + 162 genuine upstream failures) replayed against a clock frozen to 2026-07-30, so every number is reproducible and no traffic reached the live service.

📚 Sources & attribution

Retrieval ran over six phenology and remote-sensing documents, and every one of the 1,899 rows embeds extracts from them in its system message — 7.0 MB in total, about 40% of the file, averaging 3,703 characters per row.

Rights in those extracts remain with their original authors and publishers. They are included as reference context for the conversations, not as a redistribution of the underlying works, and the terms of each source govern any reuse of that portion. If you intend to redistribute this corpus, check each source's licence first — they are not uniform.

⚠️ Limitations

• —`science_keywords` is empty on every row. The topic classifier (nasa-impact/science-keyword-classification) failed a domain-fitness probe — it returns OCEANS > SALINITY for "the normalized difference vegetation index measures canopy greenness" — and was gated off rather than allowed to write confidently wrong metadata.
• —Three intents sit under 3%: injection (52), catalogue (42), invalid_region (41). These are the corrective behaviours, and they are the scarcest because the teacher produced the correct shape for them only 9–18% of the time even sampled 6–8 deep.
• —`score` is a within-seed ranking, not a quality measure. Validation already removed everything unacceptable; a low score means "a sibling was better", not "this row is bad".
• —Values are replayed fixtures, not live API responses, under a clock frozen to 2026-07-30.
• —English-dominant — 72 of 1,899 rows are non-English (Indonesian, French, Dutch, German, Spanish, romanised Japanese), all hand-written rather than machine-translated.
• —The adapter trained on this corpus regressed on its acceptance metric. Golden conversations fell 13/15 → 11/15 while eval loss dropped 66% and first-tool accuracy rose 63% → 98%. The regression is redundant tool calls, not wrong answers. See the adapter card before assuming this corpus is ready to train on unmodified.

📄 License

Mixed. The generated conversations — prompts, tool calls, assistant answers and all metadata — are released under Apache-2.0. The quoted extracts embedded in each system message (~40% of the file) are not ours to license; see Sources & attribution.

🌟 Acknowledgments

• —NASA Earth Science Division for the Earth observation data behind Bloomee
• —NASA Space Apps Challenge for the challenge this was built for
• —NASA IMPACT for nasa-smd-ibm-st-v2 and the SMD QA benchmark
• —IBM Granite for the teacher and embedding models
• —The authors of the six source documents listed above

<div align="center">

🌸 Bloomee — Unveiling Earth's Flowering Rhythms Through Space and Time 🌸

Built with ❤️ by Team Ganespace for NASA Space Apps Challenge 2025

</div>