occ
Datasets
All datasets matching “occ”tokenizer-wiki-bench
Multilingual Tokenizer Benchmark
This dataset includes pre-processed wikipedia data for tokenizer evaluation in 45 languages. We provide more information on the evaluation task in general this blogpost.
Usage
The dataset allows us to easily calculate tokenizer fertility and the proportion of continued words on any of the supported languages. In the example below we take the Mistral tokenizer and evaluate its performance on Slovak.
from transformers import AutoTokenizer… See the full description on the dataset page: https://huggingface.co/datasets/occiglot/tokenizer-wiki-bench.OccTrack360occiglot-fineweb-v1.0
Occiglot Fineweb v1.0
We present a more mature version of the multilingual Occiglot Fineweb corpus. In this early form, the dataset contains roughly 430M heavily cleaned documents from 10 languages.
Occiglot Fineweb builds on our existing collection of curated datasets and pre-filtered web data.
Subsequently, all documents were filtered with language-specific derivatives of the fine-web processing pipeline and different levels of depuplicated.
We provide the data at 3 levels of… See the full description on the dataset page: https://huggingface.co/datasets/occiglot/occiglot-fineweb-v1.0.Nuplan-OccupancyOccuFly
OccuFly Dataset
Following its acceptance as a CVPR 2026 Oral, we release OccuFly: the first real-world, large-scale camera-based benchmark for Semantic Scene Completion and Metric Monocular Depth Estimation from the aerial perspective.
📚 Full Documentation on GitHub: github.com/markus-42/occufly🌐 Project Page: markus-42.github.io/publications/2026/occufly/🤗 Aerial DepthAnything2: huggingface.co/markus-42/OccuFly-DepthAnythingV2📜 Paper: arXiv:2512.20770… See the full description on the dataset page: https://huggingface.co/datasets/markus-42/OccuFly.pick_and_place_occlusion
pick_and_place
QuestArmTeleop LeRobot v3 dataset. RGB is encoded as LeRobot video and lossless aligned depth is retained under rgbd/.
Point cloud
Point clouds are stored directly in Parquet as observation.pointcloud.xyz (float32 metres, shape [1024, 3], base_link), observation.pointcloud.rgb (uint8 RGB), and observation.pointcloud.valid_mask. They share the same row, frame index and timestamp as state/action.
See conversion_info.json for coordinate frames, units… See the full description on the dataset page: https://huggingface.co/datasets/Keith-Luo/pick_and_place_occlusion.