hdfs
Datasets
All datasets matching “hdfs”HDFS_v1
Dataset Card for logfit-project/HDFS_v1
Dataset Summary
The HDFS v1 log dataset captures Hadoop Distributed File System (HDFS) console logs that were collected
from a private cloud deployment while benchmark workloads were executed. Each log line can be associated
with one or more block identifiers; block-level anomaly labels were generated by manually crafted rules.
This script preserves the raw line structure while attaching a binary anomaly flag for downstream anomaly… See the full description on the dataset page: https://huggingface.co/datasets/logfit-project/HDFS_v1.hdfs-logs-encoded-blocks
HDFS Logs Train/Val/Test Splits
This dataset contains preprocessed HDFS log sequences split into train, validation, and test sets for anomaly detection tasks.
Dataset Description
The dataset is derived from the HDFS log dataset, which contains system logs from a Hadoop Distributed File System (HDFS).
Each sequence represents a block of log messages, labeled as either normal or anomalous. The dataset has been preprocessed
using the Drain algorithm to extract structured… See the full description on the dataset page: https://huggingface.co/datasets/honicky/hdfs-logs-encoded-blocks.HDFSHDFS_DATASET_WINDOW_SIZE_20hdfs-logsHDFS_v1_blocks
HDFS v1 Block-Level Dataset
Resources:
Video Explainer
GitHub Repo
Dataset
Dataset Summary
This dataset is a block-level transformation of the HDFS_v1 log dataset. While the original dataset contains individual log lines (~11M rows), this version aggregates all log entries belonging to the same block into a single text sequence, making it suitable for LLM-based anomaly classification.
Each row represents a unique HDFS block with its complete log history concatenated in… See the full description on the dataset page: https://huggingface.co/datasets/shawhin/HDFS_v1_blocks.