data-lake
science-datalake
Science Data Lake
A unified, portable science data lake integrating 7 scholarly datasets (~525 GB Parquet) with cross-dataset DOI normalization, 13 scientific ontologies (1.3M terms), and a reproducible ETL pipeline.
Note: One additional source (Semantic Scholar S2AG) is supported by the pipeline but is not redistributed here due to its API terms of service. See Not Included in This Upload below.
What's Unique
This dataset enables queries… See the full description on the dataset page: https://huggingface.co/datasets/J0nasW/science-datalake.my-cloud-data-lake
🌊 Zero-Cost Cloud Data Lake on Hugging Face
A complete serverless data pipeline that converts 12GB+ of CSV data to optimized Parquet format and serves it via a FastAPI on Hugging Face Spaces.
🏗️ Architecture Overview
📁 Local CSV Data (12GB+)
↓
🔄 Conversion Script (DuckDB + SNAPPY)
↓
📦 Parquet Files (4.4GB - 67% compression)
↓
☁️ Hugging Face Dataset
↓
🚀 Serverless FastAPI
↓
🌐 Public API Endpoint
📊 Dataset Information
Source: 793… See the full description on the dataset page: https://huggingface.co/datasets/OMCHOKSI108/my-cloud-data-lake.carbon-emission-datalake
Carbon Emission Data Lake
Dataset ini berisi data historis & prediksi emisi karbon serta suhu untuk wilayah
Texas dan Jakarta, dihasilkan secara otomatis oleh pipeline Prefect + dbt.
Kolom: generated_date, generated_at, region, live_temperature,
base_emission_mt, carbon_emission_forecast_mt, prophet_upper, prophet_lower,
is_anomaly (0=Normal, 1=Anomaly, NULL=model belum cukup data), recommended_carbon_cap_mt.
Modul ML: Prophet (forecast 30 hari), IsolationForest (deteksi… See the full description on the dataset page: https://huggingface.co/datasets/sigit48/carbon-emission-datalake.biomni-data-lakeQuant_DataLake
Quant_DataLake 数据湖存储层详解
文档版本: v3.0 (Doc-Code 对齐: Content-Addressable 因子注册表 + 12 因子 + _factor_states 路径修正 + 数据完备性标注)生成日期: 2026-07-24定位: 仅描述数据湖仓物理存储层 (Parquet+JSON 文件布局、Schema 契约、分区规范)。ETL 管道代码设计见 data_pipeline/README.md。语言: 中文 (Technical Chinese)
一、 数据湖层级解剖 (DataLake Anatomy)
1.1 三层架构总览 (Bronze → Silver → Gold)
数据湖采用 Medallion Architecture (奖章架构) 三层递进设计，所有数据以 Hive 风格 year=YYYY/ 分区 存储于 Parquet 文件中。
┌──────────────────────────────────────┐… See the full description on the dataset page: https://huggingface.co/datasets/zzlmgg/Quant_DataLake.science-datalake
Science Data Lake
A unified, portable science data lake integrating 7 scholarly datasets (~525 GB Parquet) with cross-dataset DOI normalization, 13 scientific ontologies (1.3M terms), and a reproducible ETL pipeline.
Note: One additional source (Semantic Scholar S2AG) is supported by the pipeline but is not redistributed here due to its API terms of service. See Not Included in This Upload below.
What's Unique
This dataset enables queries… See the full description on the dataset page: https://huggingface.co/datasets/GodotCN/science-datalake.