gabrielaltay/tcga-kirc-tabular-open

TCGA-KIRC — Tabular (Open Access)

Open-access [TCGA-KIRC][gdc-project] data from the NCI Genomic Data Commons, reshaped into one table per GDC data_type. Clinical, biospecimen and every open molecular modality for this cohort, in one place, queryable without downloading a single .tar or parsing a single TSV.

• —GDC data release: Data Release 46.0 - August 10, 2026
• —Built: 2026-09-12 03:58:45 UTC
• —Scope: one TCGA project — see [the family][repo] for the others

from datasets import load_dataset

REPO_ID = "gabrielaltay/tcga-kirc-tabular-open"
cases = load_dataset(REPO_ID, "cases", split="train")
expr = load_dataset(REPO_ID, "gene_expression_quantification", split="train")

Each table is its own config, so you can load one without pulling the rest — useful when a single project's expression table is larger than everything else combined. Nothing here requires joining against another dataset.

[gdc-project]: https://portal.gdc.cancer.gov/projects/TCGA-KIRC

Tables

Every table is a HuggingFace config. Row counts are for TCGA-KIRC.

How the tables join

cases is the hub. Every molecular table repeats the case, sample and aliquot foreign keys it needs, so the common queries are joins on an id rather than a walk down the nested tree.

Two exceptions to know before writing a query:

• —RPPA attaches to a `portion`, so protein_expression_quantification carries portion_id where its siblings carry aliquot_id.
• —`masked_somatic_mutation` carries `tumor_sample_id` / `matched_normal_sample_id` — a variant call is about a pair of samples.

The full biospecimen hierarchy (sample -> portion -> analyte -> aliquot, with slides, centres and annotations at each level) is nested inside cases.samples.

The gene_model join

Every GDC per-gene file repeats the same GENCODE v36 model, which cost 51% of the expression table's bytes. It lives once in gene_model, and the two per-gene tables carry only gene_id. The source file is exactly reconstructible by joining — verified value-for-value including row order.

SELECT e.*, g.gene_name, g.gene_type
FROM gene_expression_quantification e
JOIN gene_model g USING (gene_id)

gene_model is assembled from the two GDC sources that each hold half of it, so nothing is imported from outside the GDC. The 37 chrM genes carry null coordinates because the copy number callers exclude the mitochondrial genome.

Coverage

One table per GDC data_type; a data_type's workflows are separated by a workflow_type column rather than split across tables.

files has a row for every open-access GDC file for TCGA-KIRC, carried here or not, so the dataset describes its own scope. in_dataset says whether the content is in a table, dataset_table says which, and gdc_download_url is on every row either way.

SELECT in_dataset, count(*) AS files, sum(file_size)/1e9 AS gb
FROM files GROUP BY in_dataset;

Indexing is nearly free where carrying is not: the table is under a megabyte and describes far more data than this dataset stores.

Not carried, all raw or redundant rather than analysis results:

• —Slide Image — whole-slide .svs, an order of magnitude larger than everything else here combined, and not tabular.
• —Masked Intensities — the raw .idat behind the betas; methylation_beta_value is the analysis-ready form.
• —The per-case BCR XML supplements. Each supplement data_type ships as both a project-level bcr biotab TSV and per-case XML; the tables here are parsed from the biotabs, and the XML is the same data under different element names. Measured, not assumed: 918 of 918 mapped values agree between bcr ssf xml and ssf_tumor_samples, and 99.3% between bcr xml and clinical_patient.

Controlled-access files are not listed — a URL nobody reading an open dataset can use is noise, and cases.summary.data_categories already reports that controlled data exists for a case.

Reading the molecular tables

Copy number — four tables, not interchangeable

Filter on `workflow_type`. Several callers ship for the same aliquot and genuinely disagree — each fits purity and ploidy independently, so one aliquot can be modal CN 2 under ASCAT2 and 4 under ASCAT3. Not filtering pools different answers to the same question.

• —Allele-specific is absolute integer CN with purity and ploidy corrected; the masked and unmasked tables are ratios against a diploid reference. In a hyperdiploid tumour, CN 3 is copy-neutral against its own baseline but still reads near log2 0.
• —num_probes is array probes for DNAcopy, sequencing bins for GATK4 — comparable only within a workflow.
• —chromosome is written as each source writes it: bare (1) in the DNAcopy tables, chr-prefixed elsewhere.
• —ABSOLUTE LiftOver appears only at gene level — it ships no segment file anywhere in the GDC.
• —A small tail of masked-segment files is over-fragmented (noisy arrays); num_probes is the filter.

Methylation

SeSAMe level-3 beta, the methylated fraction in [0, 1].

• —`platform` matters. TCGA spans three Illumina generations with different probe sets; betas compare only within a platform.
• —Nulls are real — ~15% of probes in a 450k file. SeSAMe masks probes it cannot trust, so null means "masked", not "unmethylated".

Expression, miRNA and isoforms

gene_expression_quantification is STAR counts with the four N_* alignment-summary rows dropped; join gene_model for annotation. mirna_expression_quantification gives one value per mature miRNA; isoform_expression_quantification splits the same reads across the pileups collapsed into it (~4,500 isoforms vs ~1,881 mature miRNAs, same aliquots and run). In both, cross_mapped = "Y" marks reads that also aligned elsewhere, so the count is not uniquely attributable.

Protein expression (RPPA)

The narrowest coverage here: RPPA ran on a subset of cases and the antibody panel grew over time (set_id distinguishes versions), so a missing target usually means "not on that panel", not "zero". Missing values are the source's literal string `NA`, not empty cells — testing for empty strings finds nothing and looks like a bug.

Pathology reports

pdf_bytes holds the scanned PDF verbatim. These are page images, mostly with no text layer, so no text extraction is shipped rather than one that silently returns empty strings.

Clinical and biospecimen data

Two complementary views, not duplicates.

`cases` is the GDC's harmonized view: one row per patient with the /cases entity tree nested as structs and lists. Fetched with every expandable group the API offers except files.*, so it carries demographic, diagnoses (with treatments, pathology details, annotations), follow-ups (with molecular tests and other clinical attributes), exposures, family histories, the biospecimen hierarchy, curator annotations, tissue source site, program, and GDC's per-case file tallies.

*`clinical_supplement_ / biospecimensupplement`* are the original BCR biotab forms, one table per form. They carry what the harmonized API drops or under-populates — notably treatment_outcome_first_course, the disease-free signal behind DFI — plus the specimen chain: per-slide percent_tumor_nuclei and percent_necrosis, analyte a260_a280_ratio, plate and shipment provenance for batch-effect work, and site-specific factors the pan-cancer schema has no column for.

These are flex-schema: the column set differs by project and submitting centre, so each form gets its own inferred schema. Union across projects with NULL padding, as the GDC and cBioPortal do for their own exports.

Survival endpoints (survival_derived)

We have provided a supplement to the GDC source data: re-derived survival endpoints — Overall Survival (OS), Disease-Specific Survival (DSS), Progression-Free Interval (PFI), Disease-Free Interval (DFI) — following the algorithm published by Liu et al. 2018 (DOI 10.1016/j.cell.2018.02.052).

Surfaced as a standalone `survival_derived` table (one row per patient, joined to cases on case_submitter_id) with eight columns: os_event / os_time, dss_event / dss_time, pfi_event / pfi_time, dfi_event / dfi_time. *_event is 0/1 (event observed vs censored); *_time is days from index_date (TCGA: diagnosis date). DFI is null for SKCM / THYM / UVM / LAML — Liu specifies no DFI for those tumor types.

We've reimplemented Liu's method against the current TCGA data and find broad agreement with the original curated CDR. Differences exist and are expected: this is a newer release of the underlying GDC data, so re-curated clinical values, post-2018 patient additions, and schema migrations all contribute to the gap. This work is evolving; see the [repository][repo] for the full reproduction report and per-endpoint methodology.

Why we don't ship Liu's curated 2018 values directly: the CDR is a frozen 2018 snapshot derived from a since-modified GDC release. Including those values would lock in irreproducible source-data drift. We re-derive on every build, so the values reflect the current GDC and are reproducible from this dataset's other tables alone.

Pathway activity (ssGSEA)

Single-sample gene set enrichment for every RNA-Seq aliquot: one ssgsea_scores_<collection> table per MSigDB collection, each row a (aliquot, gene set) score with a pathway_url to the set's definition.

Barbie et al. (2009) ssGSEA as implemented by Bioconductor GSVA, reimplemented in Python and validated against GSVA 2.6.6 to floating-point noise. alpha=0.25, scored on tpm_unstranded over protein-coding genes plus functional Ig/TCR segments, gene sets filtered to >=10 genes after mapping. MSigDB is pinned to a single release and verified by md5, since set membership changes between releases and feeds straight into the scores.

Scores are raw and composition-dependent. ssGSEA ranks each sample against the gene universe, so a score's meaning depends on which samples were scored together — raw values are not comparable across studies. The matching ssgsea_stats_<collection> table carries the reference distribution needed to normalize them; divide by the range or z-score against it rather than comparing raw scores to another cohort's.

Because ssGSEA weights ranks, any strictly monotonic transform of the input leaves scores unchanged — there is no reason to log-transform first.

What is GDC's, and what is ours

Every measured value in every table is GDC's, copied as written — column names are lowercased and a few illegal characters replaced (cross-mapped -> cross_mapped), but no number is recomputed or re-normalized.

Four things are added, all clearly separated:

Nothing derived is mixed into a source table, so a table you did not ask for cannot quietly change a measurement you did.

Provenance

The GDC API only ever serves the current data release, so when a file was fetched says nothing about whether its bytes changed. files therefore pins each file individually: gdc_version is the file's own version, gdc_first_release the release it first appeared in, and gdc_superseded flags a file the GDC has since replaced under a different id. With md5sum and gdc_download_url, that is enough to re-verify any row against the GDC directly.

GDC references

• —[Data dictionary][gdc-dict] (every entity + field definition)
• —Biospecimen Encyclopedia
• —MAF format spec
• —[mRNA analysis pipeline][gdc-mrna] (STAR counts, TPM / FPKM / FPKM-UQ, GENCODE v36)
• —Sample Type codes
• —TCGA Barcode reference

License & redistribution

Per the NCI GDC Data Analysis Policy:

The GDC itself places no restrictions (other than attempts at reidentification) on analysis or publication of open access data provided through the GDC Data Portal.

Per the NCI TCGA citation page:

Moratoria on all cancer types are now lifted and all TCGA data are available without restrictions on their use in publications or presentations.

Per the GDC Data Access Processes and Tools page:

Open access data generally includes high level genomic data that is not individually identifiable, as well as most clinical and all biospecimen data elements.

Restrictions on use

Users of any data provided by GDC, whether open or controlled access, agree not to attempt to reidentify any individual participant in any study represented by GDC data, for any purpose whatever. (source)

Required acknowledgement

If you publish or present results derived from this dataset, include the NCI-required TCGA acknowledgement:

The results <published or shown> here are in whole or part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga.

Suggested citations:

• —Grossman, R. L., et al. (2016). Toward a Shared Vision for Cancer Genomic Data. NEJM, 375(12), 1109-1112.
• —The Cancer Genome Atlas Research Network. https://www.cancer.gov/tcga
• —NCI Genomic Data Commons. https://gdc.cancer.gov

Policy references: GDC Policies, [GDC Encyclopedia — Controlled Access][controlled] (defines what is not in this dataset), NIH Genomic Data Sharing Policy.

[controlled]: https://docs.gdc.cancer.gov/Encyclopedia/pages/Controlled_Access/

Disclaimer

This project is not affiliated with the NCI, GDC, or the TCGA Research Network. It is an experimental open-source pipeline that may change significantly between versions. Pipeline source: [galtay/tcga2hf][repo].

[gdc-dict]: https://docs.gdc.cancer.gov/DataDictionary/ [gdc-mrna]: https://docs.gdc.cancer.gov/Data/BioinformaticsPipelines/ExpressionmRNAPipeline/ [gdc-sample-types]: https://gdc.cancer.gov/resources-tcga-users/tcga-code-tables/sample-type-codes [repo]: https://github.com/galtay/tcga2hf [patients]: https://huggingface.co/datasets/gabrielaltay/tcga-patients-open [tabular]: https://huggingface.co/datasets/gabrielaltay/tcga-tabular-open