Datasetpublic

yashassnadig/wikimovies

Wikipedia Movies Dataset Dataset Description This dataset contains 58.1k movie information scraped from Wikipedia's "List of films" pages. The data includes basic movie metadata, infobox information, and introductory text from individual movie Wikipedia pages. NOTE: This is an uncleaned dataset containing raw scraped data. The content is sourced from Wikipedia and is not owned by the dataset creator. All content remains under Wikipedia's licensing terms.… See the full description on the dataset page: https://huggingface.co/datasets/yashassnadig/wikimovies.

sourceHugging Facemitupdated 1y agoView on Hugging Face

1likes181downloads

Dataset Card

Wikipedia Movies Dataset

Dataset Description

This dataset contains 58.1k movie information scraped from Wikipedia's "List of films" pages. The data includes basic movie metadata, infobox information, and introductory text from individual movie Wikipedia pages.

NOTE: This is an uncleaned dataset containing raw scraped data. The content is sourced from Wikipedia and is not owned by the dataset creator. All content remains under Wikipedia's licensing terms.

Dataset Summary

—Source: Wikipedia (en.wikipedia.org)
—Pages Scraped: Wikipedia's "List of films" pages (A-Z, numbers)
—Languages: English

Data Collection Process

The dataset was collected using a Scrapy spider that:

1.Crawled Wikipedia's "List of films" pages organized alphabetically
2.Extracted movie links from these list pages
3.Visited individual movie Wikipedia pages
4.Scraped structured data from movie infoboxes
5.Extracted introductory paragraphs and metadata

Dataset Structure

Data Fields/Columns

Column	Type	Description
`title`	String	Movie title as extracted from Wikipedia
`movie_url`	String	Full URL to the movie's Wikipedia page
`year`	String/Integer	Release year (when available)
`source_url`	String	URL of the Wikipedia list page where movie was found
`table_data`	JSON	Structured data from movie's infobox table (uncleaned)
`info`	Array	Movie Info and Plot summary
`imdb_id`	String	IMDB ID (when available in Wikipedia page)
`created_at`	Timestamp	When the record was scraped

Column Details

`table_data`

Contains structured information from the movie's Wikipedia infobox, which may include:

—Director(s)
—Producer(s)
—Writer(s)
—Starring cast
—Release date
—Runtime
—Budget
—Box office
—Genre
—Country of origin
—Language
—And other metadata fields

`info`

Array of introductory paragraphs from the movie's Wikipedia page, typically containing:

—Movie info
—Plot summary

Data Quality Notes

—Uncleaned Data: Contains raw scraped content with potential inconsistencies.
—Missing Values: Fields may be empty or null.
—Duplicate Detection: No deduplication has been performed.

Known Issues

1.Not all movies have complete information.
2.Year information may be missing or inaccurate.
3.Some entries may contain unescaped HTML or special characters
4.Data reflects Wikipedia content at time of scraping

Disclaimer

—All scraped content belongs to Wikipedia and its contributors. This dataset simply provides a structured export of publicly available information for research purposes.