Datasetpublic

Lala8383/nq-item-id-llm-compressed-hardneg-20shot-v4

Token Statistics ===== Token Statistics ===== Tokenizer: Abner0803/Qwen3-1.7B-icl-3shot-v4_128k-copy_tag Input file: train_20shot.jsonl Format: conversations Text field: text Message scope: all Operation filter: None Eligible examples: 417748 Sample size: 100 Random seed: 42 Add special tokens: False Examples: 100 Skipped: 0 Skipped by operation: 0 Total tokens: 537835 Average tokens/example: 5378.35 Min tokens/example: 4165 Max tokens/example: 6844 P50 tokens/example: 5401… See the full description on the dataset page: https://huggingface.co/datasets/Lala8383/nq-item-id-llm-compressed-hardneg-20shot-v4.

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes8downloads

Dataset Card

Token Statistics

===== Token Statistics =====
Tokenizer: Abner0803/Qwen3-1.7B-icl-3shot-v4_128k-copy_tag
Input file: train_20shot.jsonl
Format: conversations
Text field: text
Message scope: all
Operation filter: None
Eligible examples: 417748
Sample size: 100
Random seed: 42
Add special tokens: False

Examples: 100
Skipped: 0
Skipped by operation: 0
Total tokens: 537835
Average tokens/example: 5378.35
Min tokens/example: 4165
Max tokens/example: 6844

P50 tokens/example: 5401
P90 tokens/example: 6040
P95 tokens/example: 6322
P99 tokens/example: 6844