Datasetpublic

ermiaazarkhalili/orca-mini-v1-high-prob-qwen-0.5b-10k

High-Probability Sentence Predictions Dataset Dataset Description This dataset contains sentences from pankajmathur/orca_mini_v1_dataset where the model Qwen/Qwen2.5-0.5B predicts the token before the final period with ≥90% probability. Source Dataset Attribution This dataset is derived from pankajmathur/orca_mini_v1_dataset and inherits its license terms (apache-2.0). Please cite the original dataset when using this data. Extraction… See the full description on the dataset page: https://huggingface.co/datasets/ermiaazarkhalili/orca-mini-v1-high-prob-qwen-0.5b-10k.

sourceHugging Faceapache-2.0updated 10mo agoView on Hugging Face

0likes11downloads

Dataset Card

High-Probability Sentence Predictions Dataset

Dataset Description

This dataset contains sentences from pankajmathur/orca_mini_v1_dataset where the model Qwen/Qwen2.5-0.5B predicts the token before the final period with ≥90% probability.

Source Dataset Attribution

This dataset is derived from [pankajmathur/orca_mini_v1_dataset](https://huggingface.co/datasets/pankajmathur/orca_mini_v1_dataset) and inherits its license terms (apache-2.0). Please cite the original dataset when using this data.

Extraction Parameters

Parameter	Value
Source Dataset	pankajmathur/orca_mini_v1_dataset
Model	Qwen/Qwen2.5-0.5B
Probability Threshold	0.9
Seed	42
Source Columns	output
Extraction Date	2025-12-15
Total Samples	10,000

Schema

Field	Type	Description
`model_id`	string	Model used for prediction
`dataset_id`	string	Source dataset identifier
`columns`	list[string]	Source columns extracted from
`seed`	int64	Random seed used for reproducibility
`sample_idx`	int64	Index in source dataset
`sentence_prefix`	string	Text before predicted token
`predicted_token`	string	Model's top prediction
`actual_token`	string	Ground truth token
`probability`	float64	Prediction confidence (0-1)
`num_tokens`	int32	Token count in sentence

Usage

python

from datasets import load_dataset

dataset = load_dataset("ermiaazarkhalili/orca-mini-v1-high-prob-qwen-0.5b-10k")
print(dataset["train"][0])

Citation

bibtex

@dataset{high_prob_sentences_2025,
    title = {High-Probability Sentence Predictions from pankajmathur/orca_mini_v1_dataset},
    year = {2025},
    publisher = {Hugging Face},
    howpublished = {\url{https://huggingface.co/datasets/ermiaazarkhalili/orca-mini-v1-high-prob-qwen-0.5b-10k}},
    note = {Derived from pankajmathur/orca_mini_v1_dataset, model: Qwen/Qwen2.5-0.5B}
}

License

This dataset inherits the license from the source dataset: apache-2.0

See pankajmathur/orca_mini_v1_dataset for full license terms.

Reproducibility

To reproduce this dataset extraction:

bash

python scripts/extract_high_prob_sentences.py \
    --dataset "pankajmathur/orca_mini_v1_dataset" \
    --model "Qwen/Qwen2.5-0.5B" \
    --threshold 0.9 \
    --seed 42 \
    --columns output \
    --output data/output.parquet