hf-gcp-tpu-internal/torch-tpu-model-support

torch-tpu-model-support

Support sweep results for HuggingFace models on Google Cloud TPU with torch_tpu.

Each row is one model tested in one sweep. The sweep loads the model and generates 8 tokens with the fixed prompt "The future of AI is" in bf16.

Result values

• —PASS — model loaded and generated successfully
• —FAIL(load) — exception during model load
• —FAIL(gen) — load succeeded, generation raised
• —FAIL(output) — generation returned but the text was judged not valid English
• —FAIL(timeout) — killed after exceeding the sweep wall-clock budget
• —FAIL(other) — unclassified failure (render error, OOM, etc.)

Output judge

• —english — verdict of a small CPU instruct model asked whether the generated text is sensible English continuing the prompt: Y, N, or null for runs that predate the judge (or where it could not run).

Token counts

• —tokens_in — number of tokens submitted as input (after any chat templating for instruction-tuned models)
• —tokens_out — number of tokens generated beyond the input
• —Both columns are null for runs from sweeps that predate token counting.

Links

• —Source repo: https://github.com/huggingface/tpu-test
• —Raw logs bucket: hf://buckets/{user}/torch-tpu-transformers/