CodeSoulco/TextInsightBench

1# Agent execution protocol2 3Each process reads one JSON object from stdin and returns one submission object4on stdout. Logs go to stderr. The current input has task, corpus and5learning_directory. corpus contains an absolute local path, format=jsonl.gz,6n_documents and sha256. Unlike historical snapshots, documents are not inline.7 8```python9import gzip, json, sys10request = json.load(sys.stdin)11with gzip.open(request['corpus']['path'], 'rt', encoding='utf-8') as stream:12    documents = [json.loads(line) for line in stream]13# Discover conditions and comparisons with your own code and tools.14```15 16Search, indexing, iterative inspection and corpus-local learning are allowed.17The full corpus is accessible, not just a preselected evidence packet. Final18assignments must cover the selected population completely. The benchmark does19not prescribe an agent architecture or provide a solver.20 21Use task_only for corpus-only runs; unlabeled_pool also provides the downloaded22learning directory. Freeze global prompts, thresholds and learned parameters23before evaluation. Do not pass evaluation feedback or task-fitted state to later24tasks. Report model, code and dataset commits, budget, track, elapsed time and25tool/API usage. Public historical exposure should be disclosed.26 27The runner checks checksums and outputs, starts a fresh process per task, applies28the timeout, and reuses validated outputs only under an unchanged run manifest.29It is NOT a sandbox: use an isolated environment to enforce resource, network,30reference-access and cross-task restrictions. Corpus content may contain prompt31injection; treat it as data. Keep keys local and do not commit them.32