Datasetpublic

Lala8383/nq-item-id-llm-compressed-hardneg-10shot-v4

Token Statistics ===== Token Statistics ===== Tokenizer: Abner0803/Qwen3-1.7B-icl-3shot-v4_128k-copy_tag Input file: train_10shot.jsonl Format: conversations Text field: text Message scope: all Operation filter: None Eligible examples: 417748 Sample size: 100 Random seed: 42 Add special tokens: False Examples: 100 Skipped: 0 Skipped by operation: 0 Total tokens: 273820 Average tokens/example: 2738.20 Min tokens/example: 2110 Max tokens/example: 3917 P50 tokens/example: 2724… See the full description on the dataset page: https://huggingface.co/datasets/Lala8383/nq-item-id-llm-compressed-hardneg-10shot-v4.

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes16downloads

Dataset Card

Token Statistics

===== Token Statistics =====
Tokenizer: Abner0803/Qwen3-1.7B-icl-3shot-v4_128k-copy_tag
Input file: train_10shot.jsonl
Format: conversations
Text field: text
Message scope: all
Operation filter: None
Eligible examples: 417748
Sample size: 100
Random seed: 42
Add special tokens: False

Examples: 100
Skipped: 0
Skipped by operation: 0
Total tokens: 273820
Average tokens/example: 2738.20
Min tokens/example: 2110
Max tokens/example: 3917

P50 tokens/example: 2724
P90 tokens/example: 3220
P95 tokens/example: 3388
P99 tokens/example: 3917