depinwang/jinyang-gse138866-rseqc-metrics-v1
jinyang-gse138866-rseqc-metrics-v1 FINAL (130/130) per-sample QC table for GSE138866 FFPE omental metastatic HGSOC bulk RNA-seq, complete production run. Pipeline: STAR 2-pass alignment (split pass1/pass2 sbatch steps, GRCh38 Ensembl-113) -> samtools markdup -> RustQC rna (all QC modules in one BAM pass) -> per-sample JSON -> this aggregate table. Both Stage A (STAR alignment, job 75309971) and Stage B (markdup+RustQC, job 75317062) reached 130/130 real completions with ZERO… See the full description on the dataset page: https://huggingface.co/datasets/depinwang/jinyang-gse138866-rseqc-metrics-v1.
jinyang-gse138866-rseqc-metrics-v1
FINAL (130/130) per-sample QC table for GSE138866 FFPE omental metastatic HGSOC bulk RNA-seq, complete production run. Pipeline: STAR 2-pass alignment (split pass1/pass2 sbatch steps, GRCh38 Ensembl-113) -> samtools markdup -> RustQC rna (all QC modules in one BAM pass) -> per-sample JSON -> this aggregate table. Both Stage A (STAR alignment, job 75309971) and Stage B (markdup+RustQC, job 75317062) reached 130/130 real completions with ZERO failures/OOMs/timeouts across the entire pipeline. Full redteambrief.md validation at n=130: exactly 130 unique rows (no silent array-job skips); every sample has all six RustQC tool output dirs + CITATIONS.md (spot-checked on the last-finished samples); inferexperiment fractions sum to 1.000 +/- 0.0001 on all 130 rows; exonic+intronic+otherpct sums to 100.00 +/- 0.01 on all 130 rows (same for Qualimap's independent exonic/intronic/intergenic split); TIN in [0,100] on all rows with 100 gene-body bins; samtoolsduplicates > 0 on all 130 rows (confirms markdup ran everywhere). TIN median ranges 16.2-58.x (mean ~41), confirming genuine FFPE RNA degradation across the full cohort (data-validator-confirmed CLEAN on the 45-row partial, pattern holds at full scale): 130/130 samples flag/severe against the a-priori TIN>=60 pass threshold (121 flag, 9 severe) -- expected under redteambrief.md risk #3, not a bug. TIN median 16.2-59.6 (mean 45.6); TIN mean-column 19.6-52.7 (mean 42.5). Strand-call check (redteambrief item 8, only fully testable at n=130): 96/130 samples call reverse (RF/dUTP) and 34/130 call ambiguous under the a-priori 0.8 antisensefraction cutoff -- this is NOT silently averaged away, every ambiguous sample surfaces 'strand-ambiguous' in its verdict. Independently confirmed by a final data-validator pass (2026-09-08): 30/34 ambiguous samples cluster tightly just under the cutoff (antisensefraction 0.771-0.7995, no TIN/dupfraction/exonicpct group difference vs the reverse group, consistent with one uniform reverse-stranded protocol at the noise boundary of a hard threshold) -- but 4/34 (GSM4120718, GSM4120715, GSM4120658, GSM4120659) are genuine outliers at 0.63-0.77, a real weaker strand-specificity signal for those specific specimens, not noise; does not overturn the one-protocol conclusion for the cohort. Two other n=130-scale patterns surfaced by the final data-validator (not pipeline bugs, but worth knowing before downstream use): (1) totalrecords/original-FASTQ-pair ratio spans 0.98x-4.62x (mean 2.7x) and uniquemapped/FASTQ-pairs spans 6.2%-77.1% (mean 30%), correlated with TIN (r=0.68) -- some severely-degraded samples have as few as ~6% uniquely-mapped reads, which the `verdict` column does not itself surface (verdict only checks TIN/exonic/dup/strand/junction-sat, not mapping rate; see uniquemapped/totaltags columns directly for this). (2) raw duplication rate (samtoolsduplicates/totalrecords) is sharply bimodal: GSM4120712-GSM4120720 (9 sequential samples) plus GSM4120689 sit at 9-13%, then a clean gap, then the remaining 120 samples span 20.5-58.5% continuously; the low-duplication subset independently also shows higher unique-mapping rate and higher TIN across all three metrics -- an internally consistent 'less-degraded FFPE block' signature for those 10 samples, not a processing artifact (same sequencing platform/submission/date in SRA metadata for all 130). Two known metric-reliability caveats apply throughout (see EXPERIMENTREADME.md): insertsizemean/stdev are unreliable for this spliced RNA-seq data (use innerdistancemedian instead), and genebody3p5pratio vs Qualimap's own bias5p3p can disagree on 5'-3' bias direction for the same sample. Full 130-row qc_summary.csv and the 4 aggregate QC plots (TIN histogram, gene-body coverage, duplication histogram, read-distribution) are attached in this repo alongside the parquet table.
Dataset Info
- Rows: 130
- Columns: 38
Columns
Generation Parameters
{
"script_name": "run_rustqc.py + aggregate_qc.py",
"tool": "RustQC v0.2.1 (seqeralabs/RustQC) -- single-pass RSeQC+Qualimap+dupRadar+featureCounts+preseq+samtools reimplementation, glibc-2.28-compatible build",
"model": "n/a (QC pipeline, no model inference)",
"description": "FINAL (130/130) per-sample QC table for GSE138866 FFPE omental metastatic HGSOC bulk RNA-seq, complete production run. Pipeline: STAR 2-pass alignment (split pass1/pass2 sbatch steps, GRCh38 Ensembl-113) -> samtools markdup -> RustQC rna (all QC modules in one BAM pass) -> per-sample JSON -> this aggregate table. Both Stage A (STAR alignment, job 75309971) and Stage B (markdup+RustQC, job 75317062) reached 130/130 real completions with ZERO failures/OOMs/timeouts across the entire pipeline. Full red_team_brief.md validation at n=130: exactly 130 unique rows (no silent array-job skips); every sample has all six RustQC tool output dirs + CITATIONS.md (spot-checked on the last-finished samples); infer_experiment fractions sum to 1.000 +/- 0.0001 on all 130 rows; exonic+intronic+other_pct sums to 100.00 +/- 0.01 on all 130 rows (same for Qualimap's independent exonic/intronic/intergenic split); TIN in [0,100] on all rows with 100 gene-body bins; samtools_duplicates > 0 on all 130 rows (confirms markdup ran everywhere). TIN median ranges 16.2-58.x (mean ~41), confirming genuine FFPE RNA degradation across the full cohort (data-validator-confirmed CLEAN on the 45-row partial, pattern holds at full scale): 130/130 samples flag/severe against the a-priori TIN>=60 pass threshold (121 flag, 9 severe) -- expected under red_team_brief.md risk #3, not a bug. TIN median 16.2-59.6 (mean 45.6); TIN mean-column 19.6-52.7 (mean 42.5). Strand-call check (red_team_brief item 8, only fully testable at n=130): 96/130 samples call reverse (RF/dUTP) and 34/130 call ambiguous under the a-priori 0.8 antisense_fraction cutoff -- this is NOT silently averaged away, every ambiguous sample surfaces 'strand-ambiguous' in its verdict. Independently confirmed by a final data-validator pass (2026-09-08): 30/34 ambiguous samples cluster tightly just under the cutoff (antisense_fraction 0.771-0.7995, no TIN/dup_fraction/exonic_pct group difference vs the reverse group, consistent with one uniform reverse-stranded protocol at the noise boundary of a hard threshold) -- but 4/34 (GSM4120718, GSM4120715, GSM4120658, GSM4120659) are genuine outliers at 0.63-0.77, a real weaker strand-specificity signal for those specific specimens, not noise; does not overturn the one-protocol conclusion for the cohort. Two other n=130-scale patterns surfaced by the final data-validator (not pipeline bugs, but worth knowing before downstream use): (1) total_records/original-FASTQ-pair ratio spans 0.98x-4.62x (mean 2.7x) and unique_mapped/FASTQ-pairs spans 6.2%-77.1% (mean 30%), correlated with TIN (r=0.68) -- some severely-degraded samples have as few as ~6% uniquely-mapped reads, which the `verdict` column does not itself surface (verdict only checks TIN/exonic/dup/strand/junction-sat, not mapping rate; see unique_mapped/total_tags columns directly for this). (2) raw duplication rate (samtools_duplicates/total_records) is sharply bimodal: GSM4120712-GSM4120720 (9 sequential samples) plus GSM4120689 sit at 9-13%, then a clean gap, then the remaining 120 samples span 20.5-58.5% continuously; the low-duplication subset independently also shows higher unique-mapping rate and higher TIN across all three metrics -- an internally consistent 'less-degraded FFPE block' signature for those 10 samples, not a processing artifact (same sequencing platform/submission/date in SRA metadata for all 130). Two known metric-reliability caveats apply throughout (see EXPERIMENT_README.md): insert_size_mean/stdev are unreliable for this spliced RNA-seq data (use inner_distance_median instead), and genebody_3p_5p_ratio vs Qualimap's own bias_5p3p can disagree on 5'-3' bias direction for the same sample. Full 130-row `qc_summary.csv` and the 4 aggregate QC plots (TIN histogram, gene-body coverage, duplication histogram, read-distribution) are attached in this repo alongside the parquet table.",
"experiment_name": "jinyang-gse138866-rseqc",
"job_id": "turso:75317062(Stage-B,complete) turso:75309971(Stage-A-pass2,complete)",
"cluster": "turso",
"artifact_status": "final",
"canary": false,
"hyperparameters": {
"genome": "GRCh38 Ensembl-113 (project-standard index, same as prior rMATS/Shiba/MAJIQ BAMs)",
"star_mode": "2-pass split into sbatch_star_pass1.sh (mapping only, SJ.out.tab) + sbatch_star_pass2.sh (--sjdbFileChrStartEnd on-the-fly junction insertion, no --twopassMode)",
"markdup_pipeline": "samtools sort -n | fixmate -m | sort | markdup | index",
"rustqc_invocation": "rustqc rna <bam> --gtf <ensembl113.gtf> -p -t 8 -s unstranded",
"stage_a_concurrency": "--array=1-130%50 (bumped from %20 mid-run per user request to make fuller use of idle cluster capacity)",
"stage_b_concurrency": "--array=1-130%30 (bumped from unused default %20 based on a fresh cluster-headroom check at Stage A completion; memory-bound at 96G/task, kept below theoretical max to respect other users' pending jobs)",
"stage_b_mem": "96G (bumped from 40G after both canary samples OOM'd at ~27.9G MaxRSS)",
"platform": "turso (Slurm, short partition)"
},
"input_datasets": []
}Usage
from datasets import load_dataset
dataset = load_dataset("depinwang/jinyang-gse138866-rseqc-metrics-v1", split="train")
print(f"Loaded {len(dataset)} rows")