CoolFace
Datasetpublic

NuTonic/brief-composer-sft-v1

BriefComposer SFT Multi-image analytical brief rows composed from completed FireWatch, OceanScout, LandShift, and FloodPulse dataset folders (metadata/ + images/). Each sample stitches 1–4 images and metadata-derived headlines into one executive-style assistant reply. Record counts (this build) Split JSONL lines train 6307 validation 851 test 842 total 8000 Inputs Source roots: one or more --source-root directories (each must… See the full description on the dataset page: https://huggingface.co/datasets/NuTonic/brief-composer-sft-v1.

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face
0likes817downloads
Dataset Card

BriefComposer SFT

Multi-image analytical brief rows composed from completed FireWatch, OceanScout, LandShift, and FloodPulse dataset folders (metadata/ + images/). Each sample stitches 1–4 images and metadata-derived headlines into one executive-style assistant reply.

Record counts (this build)

SplitJSONL lines
train6307
validation851
test842
total8000

Inputs

  • —Source roots: one or more --source-root directories (each must contain images/ and metadata/ from the profile builders).
  • —CLI: build_lfm_vl_brief_sft.py --samples N controls JSONL line count. Run after the four temporal profile datasets are built so metadata and PNGs exist.

Dataset layout

  • —data/train.jsonl, data/validation.jsonl, data/test.jsonl — VLM SFT samples (messages list with system/user/assistant content; images referenced by relative paths under this folder).
  • —images/ — PNG chips (typically mixed profile images per row) consumed by the JSONL.
  • —metadata/ — one JSON sidecar per tile/sample (scene ids, bbox, optional regions, profile tag).

Splits

Train / validation / test use a stable hash of the synthetic sample id (brief_######) assigned at compose time.

Regenerating locally

Run from the nutonic repository root (paths relative to that root):

bash
python data/scripts/build_lfm_vl_brief_sft.py <see --help for required args>

Upload to the Hub (requires a token with write access to the target dataset repo):

bash
export HF_TOKEN="hf_..."   # or HUGGING_FACE_HUB_TOKEN
python data/scripts/build_lfm_vl_brief_sft.py ... --upload-repo NuTonic/brief-composer-sft-v1 --hf-token "$HF_TOKEN"

Skip upload with --no-upload. Resume partial STAC downloads with --skip-existing.

Orchestrator (all PRO profiles)

bash
python data/scripts/run_pro_sft_orchestrator.py \
  --events-per-profile 0 \
  --hf-org YOUR_ORG \
  --upload-repo-firewatch YOUR_ORG/firewatch-sft-v1 \
  --hf-token "$HF_TOKEN"

Use --upload-repo-<profile> per profile, or --hf-org to upload to YOUR_ORG/<default-repo-suffix> for each builder. See run_pro_sft_orchestrator.py --help.

Environment

VariablePurpose
HF_TOKEN or HUGGING_FACE_HUB_TOKENHub upload if --hf-token is not passed.
HTTP_PROXY / HTTPS_PROXYOptional; if your network requires a proxy to reach STAC and Hugging Face.

STAC: scenes are resolved from Element84 Earth Search (sentinel-2-l2a by default). No Earth Engine or Mapbox keys are required for this pipeline.

Hub target for this artifact

Dataset repo id: NuTonic/brief-composer-sft-v1

Card URL: https://huggingface.co/datasets/NuTonic/brief-composer-sft-v1

Limitations

  • —Optical-only Sentinel-2; cloud cover and revisit gaps can drop events or leave empty change masks.
  • —Detections and captions are heuristic (indices / thresholds), not operational truth labels.
  • —Assistant text follows conservative wording (candidates, confidence qualifiers).

Hub layout (sharded)

This snapshot was processed with python data/scripts/shard_lfm_vl_dataset_for_hub.py so that `images/, mapbox_stills/, overlays/, and metadata/ use at most **8000** files per leaf directory (Hub git limit: 10k files per directory). JSONL paths may include sNNNNN/` shard segments where needed.

NuTonic/brief-composer-sft-v1 · CoolFace