Datasetpublic

finetrainers/OpenVid-1k-split

Combination of part_id's from bigdata-pw/OpenVid-1M and video data from nkp37/OpenVid-1M. This is a 1k video split of the original dataset for faster iteration during testing. The split was obtained by filtering on aesthetic and motion scores by iteratively increasing their values until there were at most 1000 videos. Only videos containing between 80 and 240 frames were considered. Loading the data: from datasets import load_dataset, disable_caching, DownloadMode from… See the full description on the dataset page: https://huggingface.co/datasets/finetrainers/OpenVid-1k-split.

sourceHugging Facecc-by-4.0updated 1y agoView on Hugging Face

3likes87downloads

Dataset Card

Combination of part_id's from bigdata-pw/OpenVid-1M and video data from nkp37/OpenVid-1M.

This is a 1k video split of the original dataset for faster iteration during testing. The split was obtained by filtering on aesthetic and motion scores by iteratively increasing their values until there were at most 1000 videos. Only videos containing between 80 and 240 frames were considered.

Loading the data:

python

from datasets import load_dataset, disable_caching, DownloadMode
from torchcodec.decoders import VideoDecoder

# disable_caching()

def decode_float(sample):
    return float(sample.decode("utf-8"))

def decode_int(sample):
    return int(sample.decode("utf-8"))

def decode_str(sample):
    return sample.decode("utf-8")

def decode_video(sample):
    decoder = VideoDecoder(sample)
    return decoder[:1024]

def decode_batch(batch):
    decoded_sample = {
        "__key__": batch["__key__"],
        "__url__": batch["__url__"],
        "video": list(map(decode_video, batch["video"])),
        "caption": list(map(decode_str, batch["caption"])),
        "aesthetic_score": list(map(decode_float, batch["aesthetic_score"])),
        "motion_score": list(map(decode_float, batch["motion_score"])),
        "temporal_consistency_score": list(map(decode_float, batch["temporal_consistency_score"])),
        "camera_motion": list(map(decode_str, batch["camera_motion"])),
        "frame": list(map(decode_int, batch["frame"])),
        "fps": list(map(decode_float, batch["fps"])),
        "seconds": list(map(decode_float, batch["seconds"])),
        "part_id": list(map(decode_int, batch["part_id"])),
    }
    return decoded_sample

ds = load_dataset("finetrainers/OpenVid-1k-split", split="train", download_mode=DownloadMode.REUSE_DATASET_IF_EXISTS)
ds.set_transform(decode_batch)
iterator = iter(ds)

for i in range(10):
    data = next(iterator)
    breakpoint()

Environment tested:

- huggingface_hub version: 0.25.2
- Platform: macOS-15.3.1-arm64-arm-64bit
- Python version: 3.11.10
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Running in Google Colab Enterprise ?: No
- Token path ?: /Users/aryanvs/Desktop/huggingface/token
- Has saved token ?: True
- Who am I ?: a-r-r-o-w
- Configured git credential helpers: osxkeychain
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.6.0
- Jinja2: 3.1.4
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 10.4.0
- hf_transfer: 0.1.8
- gradio: 5.6.0
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: 2.10.1
- aiohttp: 3.10.10
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /Users/aryanvs/Desktop/huggingface/hub
- HF_ASSETS_CACHE: /Users/aryanvs/Desktop/huggingface/assets
- HF_TOKEN_PATH: /Users/aryanvs/Desktop/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: True
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10