Datasetpublic

kalkehcoisa/The-Pushshift-Reddit-Dataset

The Pushshift Reddit Dataset This dataset is a copy from the dataset available here. The work belongs to Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020, janeiro 14). The Pushshift Reddit Dataset. https://doi.org/10.5281/zenodo.3608135 I'm simply making it available on more sources. Now, about the dataset. It provides a small sample of the Pushshift Reddit dataset. The sample consists of two files: RS_2019-04.zst: All Reddit submissions that were posted during… See the full description on the dataset page: https://huggingface.co/datasets/kalkehcoisa/The-Pushshift-Reddit-Dataset.

sourceHugging Facecc-by-4.0updated 7mo agoView on Hugging Face

0likes12downloads

Dataset Card

The Pushshift Reddit Dataset

This dataset is a copy from the dataset available here. The work belongs to Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020, janeiro 14). The Pushshift Reddit Dataset. https://doi.org/10.5281/zenodo.3608135 I'm simply making it available on more sources.

Now, about the dataset. It provides a small sample of the Pushshift Reddit dataset. The sample consists of two files: RS2019-04.zst: All Reddit submissions that were posted during April 2019. RC2019-04.zst: All Reddit comments that were posted during April 2019.

Both files are "small" compared to the full dataset. They are considerable in size, specially if you are going to process them locally. Here are their details: | File | Type | Total records | Size | |---|---|---|---| | RS_2019-04.zst | Submissions | 18,310,157 | 5.6 GB | | RC_2019-04.zst | Comments | 138,473,643 | 15.5 GB |

RC_2019-04.zst — Comments

Field	Type	Description
`id`	String	Unique comment identifier
`author`	String	Username of the comment's author
`author_fullname`	String	Reddit's internal full user ID, e.g. `"t2_2tbsul75"`
`author_created_utc`	Integer	UNIX timestamp of when the author's account was created
`body`	String	Full text content of the comment
`subreddit`	String	Name of the subreddit where the comment was posted
`subreddit_id`	String	Reddit's internal subreddit ID, e.g. `"t5_2qi58"`
`subreddit_type`	String	Subreddit visibility, e.g. `"public"`
`parent_id`	String	ID of the parent — `t1_` prefix for a comment, `t3_` prefix for a submission
`link_id`	String	ID of the submission this comment belongs to
`permalink`	String	Relative URL path to the comment on Reddit
`created_utc`	Integer	UNIX timestamp of when the comment was created
`retrieved_on`	Integer	UNIX timestamp of when Pushshift crawled the comment
`score`	Integer	Net upvotes (upvotes minus downvotes)
`gilded`	Integer	Number of times the comment received Reddit Gold
`gildings`	Object	Breakdown of awards received
`all_awardings`	Array	List of awards received
`total_awards_received`	Integer	Total number of awards received
`controversiality`	Integer	Whether the comment is controversial (`0` or `1`)
`is_submitter`	Boolean	Whether the author is also the submission's author
`stickied`	Boolean	Whether the comment is stickied in the thread
`locked`	Boolean	Whether the comment is locked
`collapsed`	Boolean	Whether the comment is collapsed by default
`collapsed_reason`	String	Reason for collapse, if any
`distinguished`	String	Moderator or admin distinction, if any
`edited`	Boolean	Whether the comment has been edited
`no_follow`	Boolean	Whether links in the comment have `nofollow`
`send_replies`	Boolean	Whether the author receives reply notifications
`can_gild`	Boolean	Whether the comment can receive awards
`can_mod_post`	Boolean	Whether the author can moderate the post
`quarantined`	Boolean	Whether the subreddit is quarantined
`removal_reason`	String	Reason for removal, if applicable
`author_flair_text`	String	Text of the author's flair in the subreddit
`author_flair_css_class`	String	CSS class of the author's flair
`author_flair_type`	String	Flair type, e.g. `"text"`
`author_flair_background_color`	String	Background color of the author's flair
`author_flair_text_color`	String	Text color of the author's flair
`author_flair_template_id`	String	Template ID of the author's flair
`author_flair_richtext`	Array	Rich text representation of the author's flair
`author_patreon_flair`	Boolean	Whether the author has a Patreon flair

RS_2019-04.zst — Submissions

Field	Type	Description
`id`	String	Unique submission identifier
`author`	String	Username of the submission's author
`author_fullname`	String	Reddit's internal full user ID
`author_created_utc`	Integer	UNIX timestamp of when the author's account was created
`title`	String	Title of the submission
`selftext`	String	Body text of the submission (empty for link posts)
`subreddit`	String	Name of the subreddit where the submission was posted
`subreddit_id`	String	Reddit's internal subreddit ID
`subreddit_type`	String	Subreddit visibility, e.g. `"public"`
`subreddit_subscribers`	Integer	Number of subscribers in the subreddit at the time
`subreddit_name_prefixed`	String	Subreddit name with prefix, e.g. `"r/AskReddit"`
`url`	String	URL of the submission or linked content
`permalink`	String	Relative URL path to the submission on Reddit
`domain`	String	Domain of the linked URL, e.g. `"self.AskReddit"`
`thumbnail`	String	Thumbnail type or URL
`thumbnail_height`	Integer	Height of the thumbnail in pixels
`thumbnail_width`	Integer	Width of the thumbnail in pixels
`created_utc`	Integer	UNIX timestamp of when the submission was created
`retrieved_on`	Integer	UNIX timestamp of when Pushshift crawled the submission
`score`	Integer	Net upvotes (upvotes minus downvotes)
`num_comments`	Integer	Total number of comments on the submission
`num_crossposts`	Integer	Number of times the submission was crossposted
`gilded`	Integer	Number of times the submission received Reddit Gold
`gildings`	Object	Breakdown of awards received
`all_awardings`	Array	List of awards received
`total_awards_received`	Integer	Total number of awards received
`upvote_ratio`	Float	Ratio of upvotes to total votes (0.0–1.0)
`is_self`	Boolean	Whether it is a text post (`true`) or a link post (`false`)
`is_video`	Boolean	Whether the submission links to a video
`is_original_content`	Boolean	Whether marked as original content
`is_meta`	Boolean	Whether the post is meta (about the subreddit itself)
`is_crosspostable`	Boolean	Whether the submission can be crossposted
`is_reddit_media_domain`	Boolean	Whether the URL is hosted on Reddit's media domain
`is_robot_indexable`	Boolean	Whether the submission is indexable by search engines
`over_18`	Boolean	Whether the submission is marked as NSFW
`spoiler`	Boolean	Whether the submission is marked as a spoiler
`stickied`	Boolean	Whether the submission is stickied in the subreddit
`locked`	Boolean	Whether the submission is locked for new comments
`pinned`	Boolean	Whether the submission is pinned
`hidden`	Boolean	Whether the submission is hidden
`contest_mode`	Boolean	Whether contest mode is enabled
`media_only`	Boolean	Whether the post contains media only
`quarantine`	Boolean	Whether the subreddit is quarantined
`no_follow`	Boolean	Whether links have `nofollow`
`send_replies`	Boolean	Whether the author receives reply notifications
`distinguished`	String	Moderator or admin distinction, if any
`edited`	Boolean	Whether the submission has been edited
`removal_reason`	String	Reason for removal, if applicable
`suggested_sort`	String	Suggested comment sort order, if set
`category`	String	Category of the submission, if any
`content_categories`	Array	Content categories assigned by Reddit
`media`	Object	Media metadata, if applicable
`media_embed`	Object	Embed metadata for media content
`secure_media`	Object	Secure media metadata, if applicable
`secure_media_embed`	Object	Secure embed metadata for media content
`pwls`	Integer	Partial whitelist status code
`wls`	Integer	Whitelist status code
`parent_whitelist_status`	String	Ad whitelist status, e.g. `"all_ads"`
`whitelist_status`	String	Ad whitelist status of the submission
`link_flair_text`	String	Flair text applied to the submission
`link_flair_css_class`	String	CSS class of the submission's flair
`link_flair_type`	String	Flair type, e.g. `"text"`
`link_flair_background_color`	String	Background color of the submission's flair
`link_flair_text_color`	String	Text color of the submission's flair
`link_flair_richtext`	Array	Rich text representation of the submission's flair
`author_flair_text`	String	Text of the author's flair in the subreddit
`author_flair_css_class`	String	CSS class of the author's flair
`author_flair_type`	String	Flair type, e.g. `"text"`
`author_flair_background_color`	String	Background color of the author's flair
`author_flair_text_color`	String	Text color of the author's flair
`author_flair_template_id`	String	Template ID of the author's flair
`author_flair_richtext`	Array	Rich text representation of the author's flair