datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
2l-bilinear-attn
pile
Quadratic/bilinear attention causal language model trained with the tensor-mars research stack. This repository packages the final checkpoint, configuration, and reference model code.
## Training configuration
```yaml
batch_size: 384
max_steps: 33333
warmup_steps: 200
lr: 0.0003
optimizer: Muon + AdamW
dtype: bfloat16
grad_clip: 1.0
```
## Data + tokenizer
- Context length: 512 | Vocab size: 4096
## Metrics
- **train_loss**: 3.9820… See the full description on the dataset page: https://huggingface.co/datasets/melephant/2l-bilinear-attn.Custom-OpenThinker-32B_1754027458_eval_e0d7_gpqadiamond_prn_attn_0
chengfu0118/Custom-OpenThinker-32B_1754027458_eval_e0d7_gpqadiamond_prn_attn_0
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 59.09% ± 2.51%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
58.59%
116
198
2
54.04%
107
198
3
64.65%
128
198
gromov-max
Gromov-Max
Mathematical dataset with reasoning traces, generated synthetically.Dataset contains ~87k * 2gen synthetical examples out of DeepSeek-R1-Qwen3 distill (0528).Dataset contains difficulties from 1 to 10:
Difficulty distribution
1: 2036 examples
2: 16006 examples
3: 19579 examples
4: 7234 examples
5: 10437 examples
6: 10702 examples
7: 14312 examples
8: 6067 examples
9: 246 examples
10: 7 examples
Mean difficulty -… See the full description on the dataset page: https://huggingface.co/datasets/attn-signs/gromov-max.Unroll-Qwen2.5-7B-Instruct_1754599205_eval_3444_gpqadiamond_skip_attn_idx_2
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599205_eval_3444_gpqadiamond_skip_attn_idx_2
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 38.38% ± 1.09%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
36.36%
72
198
2
37.88%
75
198
3
40.91%
81
198
Custom-OpenThinker-32B_1754027748_eval_e0d7_gpqadiamond_prn_attn_1
chengfu0118/Custom-OpenThinker-32B_1754027748_eval_e0d7_gpqadiamond_prn_attn_1
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 60.94% ± 0.69%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
60.10%
119
198
2
60.10%
119
198
3
62.63%
124
198
Custom-OpenThinker-32B_1754027544_eval_e0d7_gpqadiamond_prn_attn_2
chengfu0118/Custom-OpenThinker-32B_1754027544_eval_e0d7_gpqadiamond_prn_attn_2
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 56.06% ± 0.41%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
55.05%
109
198
2
56.57%
112
198
3
56.57%
112
198
Unroll-Qwen2.5-7B-Instruct_1754599336_eval_3444_gpqadiamond_skip_attn_idx_5
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599336_eval_3444_gpqadiamond_skip_attn_idx_5
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 27.27% ± 1.72%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
23.23%
46
198
2
28.28%
56
198
3
30.30%
60
198
Unroll-Qwen2.5-7B-Instruct_1754599421_eval_3444_gpqadiamond_skip_attn_idx_7
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599421_eval_3444_gpqadiamond_skip_attn_idx_7
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 33.84% ± 1.95%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
30.30%
60
198
2
32.83%
65
198
3
38.38%
76
198
hunyuan-image3-dit-attn-capture
HunyuanImage-3.0 DiT attention capture — handoff
Real attention inputs (q/k/v + mask) captured from the HunyuanImage-3.0 DiT, for
kernel work: choosing/implementing an attention backend that can serve this
model. The mask here is structural, not a padding mask, so it is the part
that constrains what a kernel can accept.
What's in this bundle
File
Size
Contents
out/call0_rank0.npz
16.5 MiB
prefill — q/k/v + mask
out/call32_rank0.npz
14.0 MiB
denoise —… See the full description on the dataset page: https://huggingface.co/datasets/Yi30/hunyuan-image3-dit-attn-capture.Unroll-Qwen2.5-7B-Instruct_1754599159_eval_3444_gpqadiamond_skip_attn_idx_1
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599159_eval_3444_gpqadiamond_skip_attn_idx_1
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 16.16% ± 0.41%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
15.66%
31
198
2
15.66%
31
198
3
17.17%
34
198
Unroll-Qwen2.5-7B-Instruct_1754577788_eval_3444_gpqadiamond_prn_attn_idx_0Unroll-Qwen2.5-7B-Instruct_1754599248_eval_3444_gpqadiamond_skip_attn_idx_3
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599248_eval_3444_gpqadiamond_skip_attn_idx_3
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 22.56% ± 0.73%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
21.21%
42
198
2
22.22%
44
198
3
24.24%
48
198
Unroll-Qwen2.5-7B-Instruct_1754599292_eval_3444_gpqadiamond_skip_attn_idx_4
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599292_eval_3444_gpqadiamond_skip_attn_idx_4
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 27.61% ± 1.35%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
26.77%
53
198
2
25.25%
50
198
3
30.81%
61
198
Unroll-Qwen2.5-7B-Instruct_1754599506_eval_3444_gpqadiamond_skip_attn_idx_9
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599506_eval_3444_gpqadiamond_skip_attn_idx_9
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 23.91% ± 0.77%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
22.73%
45
198
2
23.23%
46
198
3
25.76%
51
198
Unroll-Qwen2.5-7B-Instruct_1754599547_eval_3444_gpqadiamond_skip_attn_idx_10
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599547_eval_3444_gpqadiamond_skip_attn_idx_10
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 27.95% ± 2.15%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
23.23%
46
198
2
28.28%
56
198
3
32.32%
64
198
Unroll-Qwen2.5-7B-Instruct_1754599117_eval_3444_gpqadiamond_skip_attn_idx_0
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599117_eval_3444_gpqadiamond_skip_attn_idx_0
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 6.23% ± 0.36%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
7.07%
14
198
2
5.56%
11
198
3
6.06%
12
198
Unroll-Qwen2.5-7B-Instruct_1754592824_eval_759d_aime24_skip_attn_idx_10
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754592824_eval_759d_aime24_skip_attn_idx_10
Precomputed model outputs for evaluation.
Evaluation Results
AIME24
Average Accuracy: 10.00% ± 1.15%
Number of Runs: 10
Run
Accuracy
Questions Solved
Total Questions
1
6.67%
2
30
2
10.00%
3
30
3
16.67%
5
30
4
6.67%
2
30
5
6.67%
2
30
6
6.67%
2
30
7
10.00%
3
30
8
10.00%
3
30
9
16.67%
5
30
10
10.00%
3
30
Unroll-Qwen2.5-7B-Instruct_1754599679_eval_3444_gpqadiamond_skip_attn_idx_11
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599679_eval_3444_gpqadiamond_skip_attn_idx_11
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 29.46% ± 2.98%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
23.23%
46
198
2
29.29%
58
198
3
35.86%
71
198
Unroll-Qwen2.5-7B-Instruct_1754599633_eval_3444_gpqadiamond_skip_attn_idx_12
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599633_eval_3444_gpqadiamond_skip_attn_idx_12
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 37.04% ± 0.36%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
37.88%
75
198
2
36.36%
72
198
3
36.87%
73
198
Custom-Bespoke-Stratos-7B_1753961306_eval_466d_math500_skip_attn_1
chengfu0118/Custom-Bespoke-Stratos-7B_1753961306_eval_466d_math500_skip_attn_1
Precomputed model outputs for evaluation.
Evaluation Results
MATH500
Accuracy: 1.80%
Accuracy
Questions Solved
Total Questions
1.80%
9
500
Unroll-Qwen2.5-7B-Instruct_1754599715_eval_3444_gpqadiamond_skip_attn_idx_14
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599715_eval_3444_gpqadiamond_skip_attn_idx_14
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 31.65% ± 1.10%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
30.30%
60
198
2
30.30%
60
198
3
34.34%
68
198
Unroll-Qwen2.5-7B-Instruct_1754599800_eval_3444_gpqadiamond_skip_attn_idx_16
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599800_eval_3444_gpqadiamond_skip_attn_idx_16
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 31.48% ± 1.45%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
28.79%
57
198
2
30.81%
61
198
3
34.85%
69
198
Unroll-Qwen2.5-7B-Instruct_1754599380_eval_3444_gpqadiamond_skip_attn_idx_6
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599380_eval_3444_gpqadiamond_skip_attn_idx_6
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 40.40% ± 0.48%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
40.40%
80
198
2
39.39%
78
198
3
41.41%
82
198
Unroll-Qwen2.5-7B-Instruct_1754599884_eval_3444_gpqadiamond_skip_attn_idx_18
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599884_eval_3444_gpqadiamond_skip_attn_idx_18
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 31.65% ± 2.84%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
26.77%
53
198
2
29.80%
59
198
3
38.38%
76
198
Custom-OpenThinker-32B_1754022354_eval_104b_aime24_prn_attn_1
chengfu0118/Custom-OpenThinker-32B_1754022354_eval_104b_aime24_prn_attn_1
Precomputed model outputs for evaluation.
Evaluation Results
AIME24
Average Accuracy: 59.00% ± 0.95%
Number of Runs: 10
Run
Accuracy
Questions Solved
Total Questions
1
56.67%
17
30
2
56.67%
17
30
3
60.00%
18
30
4
63.33%
19
30
5
53.33%
16
30
6
63.33%
19
30
7
56.67%
17
30
8
60.00%
18
30
9
60.00%
18
30
10
60.00%
18
30
Custom-OpenThinker-32B_1754028628_eval_07bc_livecodebench_prn_attn_1
chengfu0118/Custom-OpenThinker-32B_1754028628_eval_07bc_livecodebench_prn_attn_1
Precomputed model outputs for evaluation.
Evaluation Results
LiveCodeBench
Average Accuracy: 69.11% ± 0.45%
Number of Runs: 6
Run
Accuracy
Questions Solved
Total Questions
1
70.84%
362
511
2
69.08%
353
511
3
69.28%
354
511
4
67.51%
345
511
5
68.49%
350
511
6
69.47%
355
511
Unroll-Qwen2.5-7B-Instruct_1754599675_eval_3444_gpqadiamond_skip_attn_idx_13
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599675_eval_3444_gpqadiamond_skip_attn_idx_13
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 33.33% ± 1.26%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
30.30%
60
198
2
35.35%
70
198
3
34.34%
68
198
Unroll-Qwen2.5-7B-Instruct_1754599757_eval_3444_gpqadiamond_skip_attn_idx_15
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754599757_eval_3444_gpqadiamond_skip_attn_idx_15
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 37.37% ± 0.63%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
36.36%
72
198
2
36.87%
73
198
3
38.89%
77
198
Unroll-Qwen2.5-7B-Instruct_1754691065_eval_6a28_math500_top-5-voting_num_prune_attn_6
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754691065_eval_6a28_math500_top-5-voting_num_prune_attn_6
Precomputed model outputs for evaluation.
Evaluation Results
MATH500
Accuracy: 54.80%
Accuracy
Questions Solved
Total Questions
54.80%
274
500
Unroll-Qwen2.5-7B-Instruct_1754600010_eval_3444_gpqadiamond_skip_attn_idx_21
chengfu0118/Unroll-Qwen2.5-7B-Instruct_1754600010_eval_3444_gpqadiamond_skip_attn_idx_21
Precomputed model outputs for evaluation.
Evaluation Results
GPQADiamond
Average Accuracy: 34.34% ± 2.18%
Number of Runs: 3
Run
Accuracy
Questions Solved
Total Questions
1
30.30%
60
198
2
33.33%
66
198
3
39.39%
78
198
