puttatidam/software-documentation-tha-bitextmining

1---2tags:3- sea-bed4- deduplicated5dataset_info:6  features:7  - name: source8    dtype: large_string9  - name: target10    dtype: large_string11  - name: source_row_index12    dtype: int6413  - name: flags_fired14    dtype: large_string15  splits:16  - name: train17    num_bytes: 58039518    num_examples: 191719  download_size: 20692620  dataset_size: 58039521configs:22- config_name: default23  data_files:24  - split: train25    path: data/train-*26---27 28# software-documentation-tha-bitextmining29 30Deduplicated copy of [`kornwtp/software-documentation-tha-bitextmining`](https://huggingface.co/datasets/kornwtp/software-documentation-tha-bitextmining),31part of the SEA-BED data-quality work.32 33- **Source dataset:** `kornwtp/software-documentation-tha-bitextmining`34- **Deduplicated on:** 2026-09-0435- **Task type:** bitext_mining36- **Splits:** train37 38## What changed39 40Duplicate (source, target) pairs collapsed to one row, and per-side duplicate sources/targets collapsed so the split has a retrieval ceiling of 1.0 on both sides. Every removed row was an exact duplicate after normalization; no row was removed for being low quality.41 42Row counts and the exact policy used are recorded in `bitext_mining_dedup_summary.csv` in the43analysis repo. This dataset is *not* identical to the published SEA-BED data --44scores computed on it are not directly comparable to published SEA-BED numbers.45