datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ovos-wake-word-bench-synthetic-wakewords-hey_neon
OVOS wake_word bench — synthetic-wakewords-hey_neon
Per-clip detection decisions predictions of the registered
OVOS Plugin Arena
wake_word fighters over
OpenVoiceOS/synthetic-wakewords.
One dedicated repo per modality; one dataset split per language; one JSONL
file per fighter under predictions/<lang>/<competitor_id>.jsonl. Rows follow
the arena §3.2 contract (pinned dataset_revision, plugin_version,
latency_ms). Produced by the reproducible benchmark script in the arena repo;… See the full description on the dataset page: https://huggingface.co/datasets/OpenVoiceOS/ovos-wake-word-bench-synthetic-wakewords-hey_neon.neonredact-tr
NeonRedact-TR: Turkish PII Detection Dataset
A synthetic, span-annotated dataset for detecting personally identifiable
information (PII) in Turkish text.
Open multilingual PII models now list Turkish among many languages, but they are not built for it. This dataset trains models for the parts of Turkish they miss: suffixes, Turkey-specific identifiers, and people named by role.
Models trained on it are evaluated on a separate, independently written test set: NeonRedact-TR Bench.… See the full description on the dataset page: https://huggingface.co/datasets/neondijital/neonredact-tr.qald9base64-decode-v1
Dataset: Base64 decode version1
This dataset is for improving base64 decoding capabilities.
The number of bytes that are in the base64 encoded data spans between 0..127 bytes.
GPT 4o is great at base64 decoding.
However llama3 is terrible at base64 decoding.
Short examples of what data.jsonl looks like:
{"instruction": "Transform base64 to HEX", "input": "464pNBlIObA=", "output": "e3ae2934194839b0"}
{"instruction": "Decode Base64 to json", "input": "NQ==", "output": "[53]"}… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/base64-decode-v1.simon-arc-solve-symmetry-v8
Version 1
ARC-AGI Tasks where the job is to transform symmetric images.
example count: 2-4.
test count: 1-2.
image size: 2-3.
symmetry types: hstack2, hstack3, vstack2, vstack3, grid2x2.
Version 2
image size: 2-4.
Version 3
Added HSTACK4, VSTACK4.
Version 4
Added HSTACK5, VSTACK5.
Version 5
Added ImageSymmetrySquare, so images can be rotated by 90 degrees, and flipped over the diagonals.
Version 6
Only exercising… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-solve-symmetry-v8.structured-stern-neon-articles
Structured Stern NEON Community Articles
This repository contains approximately 20k user written texts,
articles, and poetry pulled from archives of the Stern NEON website.
Stern NEON was a community platform where users could write and publish their own articles.
Many of the articles are personal stories, poems, or opinion pieces.
The articles are structured in a way that they can be used for further analysis.
Dataset Details
Uses
This dataset can be used for… See the full description on the dataset page: https://huggingface.co/datasets/dotwee/structured-stern-neon-articles.neonredact-tr-bench
NeonRedact-TR Bench v1
An independent evaluation set for personally identifiable information (PII) detection in Turkish text.
400 fictional Turkish documents, 1,617 gold entities, 24 labels. Court records, bank receipts, insurance policies, e-Devlet printouts, hospital reports, payslips, WhatsApp chats, server logs.
Results for NeonRedact, OpenAI Privacy Filter and OpenMed v2 are in RESULTS.md.
Split
Documents
Entities
Use
dev
100
411
Tuning and error analysis
test… See the full description on the dataset page: https://huggingface.co/datasets/neondijital/neonredact-tr-bench.simon-arc-combine-v113
Version 1
A combination of multiple datasets.
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 2
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 3
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 4
Added a shared dataset name for all these datasets: SIMON-SOLVE-V1. There may be higher… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-combine-v113.gameoflife-v1
Dataset with Conways Game of Life
Wikipedia - Game of life
This dataset contains 42300 items in total. There are 9 curriculums each containing 4700 items.
The images are between 3x3 and 14x14.
Each item in this dataset is a markdown file.
The markdown file has these sections: Input, Output without wrap, Output with wrap, Status.
Each of the Input images are unique.
The Perform N steps has these variants:
When it's Perform 1 step then one iteration is performed, easy.
When it's… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/gameoflife-v1.base64-decode-v2
Dataset: Base64 decode version2
This dataset is for improving base64 decoding capabilities.
This improves on the neoneye/base64-decode-v1 dataset.
Here number of bytes that are in the base64 encoded data spans between 0..255 bytes. Where version 1 spans between 0..127.
Here 3 different random functions are used. Where version 1 uses 1 random function.
GPT 4o is great at base64 decoding.
However llama3 is terrible at base64 decoding.
Short examples of what data.jsonl looks like:… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/base64-decode-v2.arc-bad-prediction
ARC Bad Prediction
Visualization of bad predictions: ARC-AGI training, ARC-AGI evaluation.
My goal during the ARC-AGI contests has been to make a stepwise refinement algorithm, that can improve on earlier predictions.
This repo is intended for stepwise refinement algorithms. This dataset contains incorrect predictions that are somewhat close to the target.
I have manually inspected these predictions and removed the worst predictions. However there may still be more bad predictions.… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/arc-bad-prediction.simon-arc-combine-v205
Version 1
A combination of multiple datasets.
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 2
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 3
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 4
Added a shared dataset name for all these datasets: SIMON-SOLVE-V1. There may be higher… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-combine-v205.simon-arc-combine-v4
Version 1
A combination of multiple datasets.
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 2
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 3
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 4
Added a shared dataset name for all these datasets: SIMON-SOLVE-V1. There may be higher… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-combine-v4.simon-arc-solve-gravity-v11
Version 1
ARC-AGI Tasks where the job is to apply gravity in the directions up/down/left/right.
example count: 2-4.
test count: 1-2.
image size: 3-10.
number of pixels to apply gravity to: 2-5.
Exercises image_gravity_move().
Version 2
Exercises image_gravity_draw().
Version 3
Exercises image_gravity_move() and image_gravity_draw().
Increased max_number_of_positions from 5 to 8.
Version 4
image size: 3-30.
max number of positions: 5.
Version… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-solve-gravity-v11.simon-arc-combine-v208
Version 1
A combination of multiple datasets.
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 2
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 3
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 4
Added a shared dataset name for all these datasets: SIMON-SOLVE-V1. There may be higher… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-combine-v208.histogram-comparisons-small-v1This is a small subset of the huge histogram-comparisons-v1 dataset with 3M rows.
This dataset contains 150000 items in total. There are 3 curriculums each containing 50000 items.
Each item is a markdown document.
Each item contains between 2 and 6 image comparisons, with a Summary at the bottom.
The images are between 3x3 and 14x14.
The markdown document contains a ## Response, that separates the prompt from the answer.
The structure of the markdown document with 3 comparisons: A, B, C.
#… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/histogram-comparisons-small-v1.simon-arc-combine-v194
Version 1
A combination of multiple datasets.
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 2
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 3
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 4
Added a shared dataset name for all these datasets: SIMON-SOLVE-V1. There may be higher… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-combine-v194.base64-encode-v1
Dataset: Base64 encode version1
This dataset is for improving base64 encoding capabilities.
GPT 4o is great at base64 encoding.
user:
convert this hex data to base64:
880567a1
assistant:
The base64 encoding of the hex data `880567a1` is `iAVnoQ==`.
user:
convert this json data representing a byte sequence to base64:
[30,41,183]
assistant:
The base64 encoding of the JSON data `[30,41,183]` is `Him3`.
However llama3 is terrible at base64 encoding.
Short examples of what… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/base64-encode-v1.simon-arc-combine-v206
Version 1
A combination of multiple datasets.
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 2
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 3
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 4
Added a shared dataset name for all these datasets: SIMON-SOLVE-V1. There may be higher… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-combine-v206.histogram-comparisons-v1If you want a small subset of this dataset, there is histogram-comparisons-small-v1 with 150k rows.
This dataset contains 3000000 items in total. There are 3 curriculums each containing 1000000 items.
Each item is a markdown document.
Each item contains between 2 and 6 image comparisons, with a Summary at the bottom.
The images are between 3x3 and 14x14.
The markdown document contains a ## Response, that separates the prompt from the answer.
The structure of the markdown document with 3… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/histogram-comparisons-v1.simon-arc-shape-v1
Version 1
Detect shape2x2 and shape3x3.
The image sizes are between 1 and 30 pixels.
simon-arc-image-v52
Version 1
Have dataset items that are somewhat evenly of each type. The LLM learned some of the types fine. However rotated images are causing problems.
The image sizes are between 1 and 10 pixels.
Version 2
Here the majority of dataset items are rotated images. Since this is what my LLM is struggling with.
Smaller images. Here the image sizes are between 1 and 5 pixels.
This helped a lot on the validation loss.
Version 3
Main focus is now on… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-image-v52.simon-arc-symmetry-v1
Version 1
From an input image, create a symmetric output image. image size 1-10.
hstack(a b)
hstack(a b c)
vstack(a b)
vstack(a b c)
2x2(a b c d)
The abcd can be: orig, flipx, flipy, 180.
simon-arc-combine-v92
Version 1
A combination of multiple datasets.
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 2
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 3
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 4
Added a shared dataset name for all these datasets: SIMON-SOLVE-V1. There may be higher… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-combine-v92.simon-arc-combine-v196
Version 1
A combination of multiple datasets.
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 2
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 3
Datasets: dataset_solve_color.jsonl, dataset_solve_rotate.jsonl, dataset_solve_translate.jsonl.
Version 4
Added a shared dataset name for all these datasets: SIMON-SOLVE-V1. There may be higher… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-combine-v196.simon-arc-mass-v17
Version 1
Measure the mass of objects for pixel connectivity 4 and pixel connectivity 8.
image size: 1-10.
max_mass: 4.
Version 2
image size: 1-20.
max_mass: 5.
This converged too slowly. I was too optimistic. I will have to proceed slower.
Version 3
image size: 1-12.
max_mass: 5.
Too big spikes in the training loss. I will have to lower the max_mass, and gradually increase it.
Version 4
image size: 1-15.
max_mass: 2.
The validation loss for this is… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-mass-v17.simon-arc-image-v32
Version 1
Have dataset items that are somewhat evenly of each type. The LLM learned some of the types fine. However rotated images are causing problems.
The image sizes are between 1 and 10 pixels.
Version 2
Here the majority of dataset items are rotated images. Since this is what my LLM is struggling with.
Smaller images. Here the image sizes are between 1 and 5 pixels.
This helped a lot on the validation loss.
Version 3
Main focus is now on… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-image-v32.simon-arc-image-v46
Version 1
Have dataset items that are somewhat evenly of each type. The LLM learned some of the types fine. However rotated images are causing problems.
The image sizes are between 1 and 10 pixels.
Version 2
Here the majority of dataset items are rotated images. Since this is what my LLM is struggling with.
Smaller images. Here the image sizes are between 1 and 5 pixels.
This helped a lot on the validation loss.
Version 3
Main focus is now on… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-image-v46.simon-arc-mass-v8
Version 1
Measure the mass of objects for pixel connectivity 4 and pixel connectivity 8.
image size: 1-10.
max_mass: 4.
Version 2
image size: 1-20.
max_mass: 5.
This converged too slowly. I was too optimistic. I will have to proceed slower.
Version 3
image size: 1-12.
max_mass: 5.
Too big spikes in the training loss. I will have to lower the max_mass, and gradually increase it.
Version 4
image size: 1-15.
max_mass: 2.
The validation loss for this is… See the full description on the dataset page: https://huggingface.co/datasets/neoneye/simon-arc-mass-v8.simon-arc-solve-fractal-v1
Version 1
ARC-AGI Tasks where the job is to transform fractal input/output images.
example count: 2-4.
test count: 1-2.
image size: 2-3.