harborframework/terminal-bench-importer

Replace snapshot_download with level-by-level listing for agent files.

Add reward hacking validation check.

Fix missing await on get_public_url causing all trial inserts to fail

Update the importer to be asyncio compatible.

Log git stderr on clone/fetch failures for easier debugging

Update DATASET_REPO to harborframework org

Return webhook response immediately, process in background thread

Add concurrent uploads and also validation.

Upload trial archives and trajectories to Supabase storage on import

Fix the dockerfile.

Update the importer

Include accuracy in PR validation comments

Only validate submissions that changed in PR

Fix trial job_id foreign key constraint and add validation

Use raw dict for trial results to support custom environment types

Use raw dict for job config to support custom environment types

Add comprehensive logging throughout import process

Use insert with ignore_duplicates for models

Fix config.json parsing and add model token cost lookup

Detect PR from updatedRefs when discussion payload is missing

Remove comment deduplication code

Add comment deduplication to prevent duplicate validation comments

Improve trial validation and remove redundant agent_name from metadata

Validate trial result.json has required fields instead of strict schema validation

Fix validation: use lenient JSON check for trials, limit error count in comments

Use lenient validation to allow custom environment types

Use git clone instead of snapshot_download for much faster downloads

Fix PR detection: check isPullRequest field and add logging

Fix PR download: use refs/pr/N instead of head.sha

Fix repo name comparison - name already includes namespace

Fix webhook: use WebhooksServer with custom UI instead of conflicting decorator

Fix Docker deployment: bind to 0.0.0.0:7860

Fix Pydantic deprecation warning: use ConfigDict instead of class Config

Switch to Docker SDK to fix multipart package conflict

Add python-multipart to fix gradio import error

Specify Python 3.12 for harbor package compatibility

Use job result.json for started_at, ended_at, stats; add package_version

Use harbor package for JobConfig and TrialResult models

Initial importer Space with webhook handler

initial commit