datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
phishing_urlsats-career-page-urls
ATS Career Page URLs
69,638 canonical career page URLs for public job boards hosted on 40 applicant tracking system (ATS) platforms, including Greenhouse, Lever, Workable, Ashby, Workday, and BambooHR.
Each row is the canonical entry point to a public job board. The dataset is deduplicated, URL-normalized, and intended as a starting point for job-market research, labor-market analytics, ATS ecosystem analysis, and job aggregation pipelines.
Released as part of Latmay, a semantic… See the full description on the dataset page: https://huggingface.co/datasets/latmay/ats-career-page-urls.phishing_url_classification
Phishing URL Classification Dataset
This dataset contains URLs labeled as 'Safe' (0) or 'Not Safe' (1) for phishing detection tasks.
Dataset Summary
This dataset contains URLs labeled for phishing detection tasks. It's designed to help train and evaluate models that can identify potentially malicious URLs.
Dataset Creation
The dataset was synthetically generated using a custom script that creates both legitimate and potentially phishing URLs. This approach… See the full description on the dataset page: https://huggingface.co/datasets/imanoop7/phishing_url_classification.ARCHIVE-TEXT-URLS
Internet Archive English Text URLs Dataset
Dataset Description
This dataset contains 11,151,637 direct download URLs to OCR-processed text files from the Internet Archive's digital library. All entries are English-language texts spanning books, documents, historical records, and various other written materials.
Dataset Summary
Total Rows: 11,151,637
Language: English
Source: Internet Archive
Format: CSV with metadata and direct text file URLs
Text… See the full description on the dataset page: https://huggingface.co/datasets/Navanjana/ARCHIVE-TEXT-URLS.JP-TH_Literary_Translation_URL_Alignment_Index
JP–TH Literary Translation URL Alignment Index
This release provides a copyright-conscious metadata index and reproducibility package for a Japanese–Thai literary translation dataset associated with the study Context-Aware Prompting for Japanese–Thai Literary Translation in a Low-Resource Setting.
Overview
The release is designed to support reproducible academic research on Japanese–Thai literary machine translation, context-aware prompting, prompt engineering… See the full description on the dataset page: https://huggingface.co/datasets/Gsk068/JP-TH_Literary_Translation_URL_Alignment_Index.hf-arxiv-url-bench
arXiv URL Extraction Benchmark & Longitudinal Corpus
Dataset Description
This repository hosts datasets designed to assess format-specific coverage gaps in URL extraction across the arXiv corpus. The collection facilitates large-scale reproducibility studies and evaluates how different document formats (LaTeX, HTML, XML, Markdown, TXT, PNG) impact automated extraction pipelines.
The repository is divided into two primary corpora:
The 200-Paper Benchmark: A… See the full description on the dataset page: https://huggingface.co/datasets/rochanaro/hf-arxiv-url-bench.urlsurl-classifications
Model Card: URL Classifications Dataset
Dataset Summary
The URL Classifications Dataset is a collection of URL classifications for PDF documents, primarily derived from the SafeDocs corpus. It contains multiple CSV files with different subsets of classifications, including both raw and processed data.
Supported Tasks
This dataset supports the following tasks:
Text Classification
URL-based Document Classification
PDF Content Inference
Languages
The… See the full description on the dataset page: https://huggingface.co/datasets/snats/url-classifications.YFCC15M_page_and_download_urls
YFCC15M subset used for VLMs
This dataset contains the ~15M subset of YFCC100M used for training the models in the paper Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP. The metadata provided in this repo contains both the page-urls and image-download-urls for downloading the dataset.
This dataset can be easily downloaded with img2dataset:
img2dataset --url_list yfcc15m_final_split_pageandimageurls.csv --input_format "csv" --output_format… See the full description on the dataset page: https://huggingface.co/datasets/vishaal27/YFCC15M_page_and_download_urls.ats-career-page-urls
ATS Career Page URLs
69,638 canonical career page URLs for public job boards hosted on 40 applicant tracking system (ATS) platforms, including Greenhouse, Lever, Workable, Ashby, Workday, and BambooHR.
Each row is the canonical entry point to a public job board. The dataset is deduplicated, URL-normalized, and intended as a starting point for job-market research, labor-market analytics, ATS ecosystem analysis, and job aggregation pipelines.
Released as part of Latmay, a semantic… See the full description on the dataset page: https://huggingface.co/datasets/Vera-001/ats-career-page-urls.reddit-syac-urlslottie-urls
Dataset Card for [Dataset Name]
Dataset Summary
List of lottiefiles uri for research purposes
Supported Tasks and Leaderboards
[More Information Needed]
Languages
[More Information Needed]
Dataset Structure
Data Instances
[More Information Needed]
Data Fields
[More Information Needed]
Data Splits
[More Information Needed]
Dataset Creation
Curation Rationale
[More Information Needed]… See the full description on the dataset page: https://huggingface.co/datasets/AmirulOm/lottie-urls.unlabeled-urlsPhishing_Urlssql-urlshape-quote-0910
Controlled SQL URL shape probe
phishing_url_classification
Dataset for Detecting Phishing URLs
This dataset contains URLs labeled as 'Safe' (0) or 'Not Safe' (1) for phishing detection tasks.
Dataset Summary
This dataset contains URLs labeled for phishing detection tasks. It's designed to help train and evaluate models that can identify potentially malicious URLs.
Dataset Creation
The dataset was synthetically generated using a custom script that creates both legitimate and potentially phishing URLs. This approach… See the full description on the dataset page: https://huggingface.co/datasets/darshan8950/phishing_url_classification.transformers_dump_urlPhishing_URLURL_Key_Integrator_Classification_Set
URL_Key_Integrator_Classification_Set
tags: Classification, URL Key Integration, Data Classification
Note: This is an AI-generated dataset so its content may be inaccurate or false
Dataset Description:
The 'URL_Key_Integrator_Classification_Set' dataset comprises URLs with unique key integrators and their corresponding transformation funktion labels. This dataset is tailored for machine learning practitioners focusing on URL classification and key integrator analysis. Each entry… See the full description on the dataset page: https://huggingface.co/datasets/infinite-dataset-hub/URL_Key_Integrator_Classification_Set.news_urlsA collection of about 21k urls of news articles taken from RAI news sites (national and regionals). The file ("urls_train_set.csv") contains around 20k of them reffering to articles published in
the period 01/01/2022 – 09/03/2023 while the file ("urls_test_set.csv") contains urls referring to articles published in the period 10/03/2023 - 04/05/2023.
URLLinearTransforms
URLLinearTransforms
tags: url-processing, linear-model, transform-dataset
Note: This is an AI-generated dataset so its content may be inaccurate or false
Dataset Description:
The 'URLLinearTransforms' dataset is a curated collection of URL transformations performed by a URL Processing Integrator Key Funktion. Each entry represents a pre-processing step that has been applied to a specific URL to facilitate data mining and content analysis. The dataset is designed for researchers and… See the full description on the dataset page: https://huggingface.co/datasets/infinite-dataset-hub/URLLinearTransforms.URLs_FunctionalIntegration_Regression
URLs_FunctionalIntegration_Regression
tags: Machine Learning, Functional Integration, Regression
Note: This is an AI-generated dataset so its content may be inaccurate or false
Dataset Description:
The 'URLs_FunctionalIntegration_Regression' dataset comprises a collection of unique URLs from various music industry-related platforms, event pages, and tools that integrate functionalities of AI and ML for analyzing trends, predicting event attendance, and managing artist contracts. The… See the full description on the dataset page: https://huggingface.co/datasets/infinite-dataset-hub/URLs_FunctionalIntegration_Regression.URLs_Function_Integration_AI_Classification
URLs_Function_Integration_AI_Classification
tags: Web_Scraping, Machine_Learning, URL_Analytics
Note: This is an AI-generated dataset so its content may be inaccurate or false
Dataset Description:
The 'URLs_Function_Integration_AI_Classification' dataset comprises URLs from various online platforms, specifically focusing on events, music artist contracts, and AI-related content. The dataset is constructed to facilitate the training of machine learning models to classify URLs based… See the full description on the dataset page: https://huggingface.co/datasets/infinite-dataset-hub/URLs_Function_Integration_AI_Classification.URLs_VIPTop_Recommendation
URLs_VIPTop_Recommendation
tags: URLs, TOP VIP, Recommendation
Note: This is an AI-generated dataset so its content may be inaccurate or false
Dataset Description:
The 'URLs_VIPTop_Recommendation' dataset comprises URLs and corresponding metadata for VIP top-tier events, music artist contracts, and management services that are predicted to be of interest to AI-based recommendation tools. Each entry is labeled based on its relevance to the topics of VIP events, artist management, and… See the full description on the dataset page: https://huggingface.co/datasets/infinite-dataset-hub/URLs_VIPTop_Recommendation.czech_traffic_urlsKey_Transform_Funktion_URL_Dataset
Key_Transform_Funktion_URL_Dataset
tags: Machine Learning, Keyword Transformation, URL Funktion Study
Note: This is an AI-generated dataset so its content may be inaccurate or false
Dataset Description:
The 'Key_Transform_Funktion_URL_Dataset' contains a collection of URLs with metadata for a machine learning study on the transformation and integration of keyword funktionen (functionalities) in web addresses. Each entry includes the original URL, a transformed version of the URL… See the full description on the dataset page: https://huggingface.co/datasets/infinite-dataset-hub/Key_Transform_Funktion_URL_Dataset.cdcgov-cleanslate-and-relaunch-url-mappings
CDC.gov CleanSlate and Relaunch URL Mappings
Description
CDC is planning a major relaunch of CDC.gov on May 15th, followed by subsequent monthly batches of sites restructuring their CDC.gov content into the fall of 2024.
Because this restructuring includes merged or changed content, CDC will be updating all URLs to a new standard naming structure. About 70% of these updates will occur during the May 15th launch, but another 30% of our content and URLs will be restructured… See the full description on the dataset page: https://huggingface.co/datasets/HHS-Official/cdcgov-cleanslate-and-relaunch-url-mappings.url_defanged
🛡️ Phishing vs Legitimate URLs Dataset
This dataset contains phishing-like and legitimate URLs designed for training and fine-tuning machine learning models (e.g., BERT) to detect phishing attempts.All phishing samples are defanged (e.g., hxxps://example[.]com) to ensure safety when browsing the dataset.
📂 Dataset Structure
The dataset includes the following columns:
url_defanged (string) → The URL in a safe format (hxxps://, [.] instead of .)
label (int) →
1… See the full description on the dataset page: https://huggingface.co/datasets/Gamortsey/url_defanged.URLsFunctionIntegration
URLsFunctionIntegration
tags: prediction, web, API
Note: This is an AI-generated dataset so its content may be inaccurate or false
Dataset Description:
The 'URLsFunctionIntegration' dataset is designed to assist Machine Learning (ML) practitioners in creating predictive models that can analyze web functionalities and their integrations with various platforms. The dataset focuses on capturing the interactions between URLs and web-based functions, particularly those that integrate… See the full description on the dataset page: https://huggingface.co/datasets/infinite-dataset-hub/URLsFunctionIntegration.ScienceQA_Image_url