Datasetpublic

gavinlaw/chinese-lips-longform-debug

Chinese-LiPS Long-Form (zh long streaming speech) Reconstructed continuous long-speech streams from BAAI/Chinese-LiPS, for slide-aware / streaming speech-translation development and evaluation. Each source video (one speaker, one scripted lecture with slides) was released as pre-segmented clips; here they are re-joined into the full talk. Two variants of the same 3 talks (~97 min speech total): config how segments are placed use orig_timeline at their original session… See the full description on the dataset page: https://huggingface.co/datasets/gavinlaw/chinese-lips-longform-debug.

sourceHugging Facecc-by-nc-sa-4.0updated 2mo agoView on Hugging Face

0likes66downloads

Dataset Card

Chinese-LiPS Long-Form (zh long streaming speech)

Reconstructed continuous long-speech streams from BAAI/Chinese-LiPS, for slide-aware / streaming speech-translation development and evaluation. Each source video (one speaker, one scripted lecture with slides) was released as pre-segmented clips; here they are re-joined into the full talk.

Two variants of the same 3 talks (~97 min speech total):

config	how segments are placed	use
`orig_timeline`	at their original session timestamps, real inter-segment silence restored (from the raw release's per-segment `startTime`/`endTime`)	realistic streaming: pauses, READ/WRITE timing, latency
`silence_removed`	back-to-back, no gaps	compact debugging, dense transcript coverage

video_id	topic	speaker	segments	orig span	speech
13042M_TY	体育 sports	42 M	265	43.0 min	37.6 min
10224M_KJ	科技 technology	24 M	206	32.4 min	29.7 min
04128F_RWLS	人文历史 humanities	28 F	222	31.4 min	29.4 min

—<video_id>.longform.wav — 16 kHz mono 16-bit
—<video_id>.longform.jsonl — one line per clip: {video_id, clip_id, start, end, zh_transcript, ocr_text, vl2_text, ppt_frame[, orig_start, orig_end]}; in orig_timeline, start/end reproduce the original talk timeline (max drift 0.001 s vs source timestamps).

About the slide modality (why this is a strong slide-aware source)

Chinese-LiPS ships, per segment, a 1080p slide-region video (PPT/*.mp4) and a face/lip crop (FACE/*.mp4) alongside the audio. The slides are clean, dense, high-resolution Chinese text/graphics (e.g. a titled map slide "地理位置 / 广西南部") — substantially higher visual quality than typical web-talk frames. The per-clip ocr_text / vl2_text fields carry slide OCR and visual labels from the source. Fetch the PPT/FACE media from the upstream Chinese-LiPS release and align by clip_id.

Toward a zh->En benchmark

The source provides Chinese transcripts but no English translation reference. To use this as a zh->En slide-aware ST benchmark, English references must be added (and, being bilingual-verifiable, checked by a zh/en reader). Until then this is a development/diagnostic resource for the translation direction, and a ready-made long-form zh ASR + slide benchmark as-is.

Reproduce

bash

python repo/scripts/build_chinese_lips_longform.py \
  --meta-json meta_test.json --processed-dir processed_test \
  --video-ids 130_42_M_TY 102_24_M_KJ 041_28_F_RWLS \
  --timeline-dir raw_json --out-dir orig_timeline      # add --timeline-dir for original timeline

License

Derived from Chinese-LiPS (BAAI), CC BY-NC-SA 4.0; shared under the same license for non-commercial research.