CoolFace
Datasetpublic

laion/openthoughts-tblite-dev-apptainer-v1

OpenThoughts-TBLite dev (offline Apptainer, v1) A validated subset of open-thoughts/OpenThoughts-TBLite (revision 7b70111339b4af23cece95d63aeec1c705790868, Apache-2.0) for running on HPC clusters without Docker and without internet on compute nodes, with the harbor Apptainer bridge. Task files are unchanged from the source; this release only selects tasks and records the evidence. It is not the separate upstream "OpenThoughts-TB-dev" set. status tasks meaning… See the full description on the dataset page: https://huggingface.co/datasets/laion/openthoughts-tblite-dev-apptainer-v1.

sourceHugging Faceapache-2.0updated 1d agoView on Hugging Face
0likes30downloads
Dataset Card

OpenThoughts-TBLite dev (offline Apptainer, v1)

A validated subset of `open-thoughts/OpenThoughts-TBLite` (revision 7b70111339b4af23cece95d63aeec1c705790868, Apache-2.0) for running on HPC clusters without Docker and without internet on compute nodes, with the harbor Apptainer bridge. Task files are unchanged from the source; this release only selects tasks and records the evidence. It is not the separate upstream "OpenThoughts-TB-dev" set.

statustasksmeaning
validated53gold solution passes in 2/2 offline oracle runs, and an empty attempt scores 0
unvalidated27upstream ships no gold solution (placeholder solve.sh); the grader rejects an empty attempt, but passing a correct answer is unproven
excluded20cannot be scored in this execution profile (see below)

Use the status field in manifest.json to report scores on the validated tasks, or on validated + unvalidated.

Execution profile

  • —backend: harbor apptainer bridge, rootless, offline (no internet on compute nodes)
  • —harbor: marianna13/harbor@6524c8ae (branch jureca-eval-fixes); changes vs upstream are listed in that branch's FORK_CHANGES.md
  • —images: SIFs built by the harbor Apptainer builder from each task's Dockerfile (rootless apptainer build --fakeroot)
  • —grader dependencies: offline wheelhouse (all pinned grader packages for Python 3.11-3.13)
  • —network: BRIDGEISOLATENETWORK=1 (per-sandbox loopback)

Evidence (Gate 1)

  • —oracle (gold solution), two independent runs: oracle-v8-prefix-tblite-all-15651434, oracle-final2-tblite-all-15651444
  • —empty-solution control (harbor nop agent): nop-final-tblite-all-15651443
  • —all runs on a JURECA dc-cpu node with no internet access (probe: no route to pypi.org)

Excluded tasks

taskreason
acl-permissions-inheritancerootless sandbox: chgrp/setfacl not permitted
amuse-installimage cannot be built: verifier package amuse-hermite needs a compiler toolchain
bloom-filter-cache-penetration-preventiongold solution fails: fakeredis connection error (likely a fakeredis/redis-py version mismatch in the offline wheelhouse)
broken-pythongold solution fails offline: package missing from the offline wheelhouse
corrupted-filesystem-recoverydataset defect: gold solution's superblock repair does not match disk.img (still corrupt)
cpp-daemon-sighup-segfaultrootless sandbox: ulimit -c not permitted
etl_checkpoint_resume_buggold solution fails: Postgres is only started by the image CMD, which harbor overrides (also under Docker)
git-repo-forensicsdataset defect: the damaged git repository (nested .git) is missing from the upload
hydra-debug-slurm-modeneeds network: git+https install
maven-slf4j-conflictneeds network (Maven Central)
okhttp-trailers-crashneeds network: Gradle dependency download
pdf-table-parsinggold solution fails offline: camelot not available
protein-sequencegold solution fails offline (undiagnosed)
react-typescript-debuggneeds network: tests run npm install
rsa-jwt-token-api-redis-blacklistneeds network at solve time
smiles-data-labdataset defect: gold solution calls a missing /app/solution/solve_smiles.py
sql-injection-forensicsgold solution crashes (re.error: bad character range) under this image's Python
symlink-chain-traversaldataset defect: gold solution resolves the chain with realpath and answers dir1; grader expects file1
systemd-log-monitoringrootless sandbox: chown not permitted
token-auth-websocketneeds network: gradle not in image

Tasks

taskstatusoracle passesempty-attempt reward
anomaly-detection-rankingvalidated2/20
api-endpoint-permission-canonicalizervalidated2/20
application-debugvalidated2/20
auth_token_race_conditionvalidated2/20
bandit-delayed-feedbackvalidated2/20
bash-log-processor-fixvalidated2/20
basic-message-queueunvalidatedn/a (no gold solution)0
battery-charging-optimizationunvalidatedn/a (no gold solution)0
book-portfolio-analysisunvalidatedn/a (no gold solution)0
bracket-sequence-restorationunvalidatedn/a (no gold solution)0
breast-cancer-mlflowvalidated2/20
build-merkle-tree-cli-sha512validated2/20
build-system-task-orderingvalidated2/20
california-housing-apivalidated2/20
chained-forensic-extraction_20260101_011957validated2/20
competitive-programming-solverunvalidatedn/a (no gold solution)0
container-registry-optimizationvalidated2/20
convolutional-layersvalidated2/20
cosign-keyless-signingvalidated2/20
cryptographic-protocol-verifiervalidated2/20
csv-json-jsonl-mergervalidated2/20
db-migration-local-storagevalidated2/20
distributed-test-execution-schedulervalidated2/20
ekf-localizationvalidated2/20
fix-js-network-controllervalidated2/20
fix_async_worker_queuevalidated2/20
floor-plan-geometryunvalidatedn/a (no gold solution)0
game-of-stonesunvalidatedn/a (no gold solution)0
grid-pathfindingunvalidatedn/a (no gold solution)0
grpc-plant-position-serverunvalidatedn/a (no gold solution)0
html-index-analysisunvalidatedn/a (no gold solution)0
image-tile-identificationunvalidatedn/a (no gold solution)0
industrial-kiln-controllervalidated2/20
iot-device-registration-servervalidated2/20
iris-dataset-classificationunvalidatedn/a (no gold solution)0
jq-data-processingvalidated2/20
jsonl-aggregatorvalidated2/20
legal-summary-extractionunvalidatedn/a (no gold solution)0
log-summaryvalidated2/20
malicious-package-forensicsvalidated2/20
mech-systemvalidated2/20
mlflow-registervalidated2/20
monorepo-changelog-clivalidated2/20
mtls-cert-rotationvalidated2/20
multi-labellervalidated2/20
multi-server-configurationunvalidatedn/a (no gold solution)0
network-log-normalizationunvalidatedn/a (no gold solution)0
neural-architecture-search-finalvalidated2/20
neutron-submissionvalidated2/20
pandas-etlvalidated2/20
pandas-numpy-data-analysisunvalidatedn/a (no gold solution)0
parking-lot-pathfindingunvalidatedn/a (no gold solution)0
permutation-construction-100kunvalidatedn/a (no gold solution)0
pgn-chess-repair-puzzlesvalidated2/20
playing-card-recognitionunvalidatedn/a (no gold solution)0
prediction-model-evaluationunvalidatedn/a (no gold solution)0
publisher-market-analysisunvalidatedn/a (no gold solution)0
publisher-market-analysis-v2unvalidatedn/a (no gold solution)0
python-api-rate-limitvalidated2/20
raft-log-repair-concurrent-accessvalidated2/20
reproducibility-and-envsetupvalidated2/20
reverse-engineer-stack-vmvalidated2/20
sakila-sqlite-queriesunvalidatedn/a (no gold solution)0
sales-data-csv-analysisvalidated2/20
scan-linux-persistence-artifactsvalidated2/20
schedule-vacationvalidated2/20
security-breach-incident-responsevalidated2/20
security-incident-log-analysisvalidated2/20
server-log-analysisunvalidatedn/a (no gold solution)0
service-deployment-wave-plannervalidated2/20
sign-vector-gamevalidated2/20
simple-database-query-toolunvalidatedn/a (no gold solution)0
submission_a63937a5_20251224_152124validated2/20
supply-chain-fulfillmentvalidated2/20
sympy-bug-fixunvalidatedn/a (no gold solution)0
task-xxe-exploitvalidated2/20
todos-apivalidated2/20
tsl-test-case-generationvalidated2/20
vimscript-vim-quinevalidated2/20
word-derangement-mappingunvalidatedn/a (no gold solution)0

Layout

  • —tasks/<name>/: harbor task folders (task.toml, instruction.md, environment/, tests/, solution/), symlinks dereferenced
  • —manifest.json: per-task status, oracle passes, empty-attempt reward, content SHA-256, execution profile, evidence
  • —excluded.json: excluded tasks with the automated reason