x65617379/lesswrong_260509

<div style=" display: inline-block; width: fit-content; max-width: 100%; background: #0b0f14; border: 1px solid #223044; border-left: 5px solid #20c997; border-radius: 8px; padding: 14px 18px; margin-bottom: 20px; color: #e6edf3; font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, 'Liberation Mono', monospace; box-shadow: 0 8px 24px rgba(15, 23, 42, 0.18); "> <p style=" margin: 0 0 6px 0; color: #8aa2bd; font-size: 12px; line-height: 1.4; text-transform: uppercase; ">codex dataset card</p> <p style=" margin: 0; color: #e6edf3; font-size: 15px; font-weight: 650; line-height: 1.45; "><span style="color:#20c997;">&gt;</span> generated by <span style="color:#ffffff;">Codex</span><span style="color:#20c997;">_</span></p> </div>

Dataset Card for lesswrong_260509

Dataset Summary

A structured crawl of public longform posts and comments from LessWrong, a ForumMagnum-powered discussion site focused on rationality, cognitive science, AI alignment, forecasting, philosophy, and related topics. The dataset contains 45,933 posts and 906,338 comments, preserving one flat comment list per post with parent IDs, nesting depth, orphan markers, timestamps, scores, vote counts, author metadata, and post-level flags.

The crawl targets public, approved, non-draft, non-future, listed, non-shortform posts visible to an anonymous LessWrong GraphQL requester.

Dataset Details

Dataset Description

• —Language(s): English
• —Crawl date: 2026-05-06 to 2026-05-09

Dataset Sources

• —Source Website: https://www.lesswrong.com
• —Source API: https://www.lesswrong.com/graphql

Uses

Direct Use

Suitable for:

• —Academic research into forum dynamics and longform online discussion
• —Computational social science research on rationalist and AI safety communities
• —Linguistic analysis of argumentative, technical, and speculative web discourse
• —Studying thread/reply structures, voting patterns, and moderation markers
• —Information retrieval and semantic search over LessWrong posts and discussions

Dataset Structure

Schema

Source Struct

Post Struct

Comment Struct

Statistics

The most prolific post authors include Eliezer Yudkowsky (1,048 posts), Zvi (989), jefftk (921), Stuart_Armstrong (811), and KatjaGrace (632). The most prolific comment authors include ChristianKl (13,987 comments), wedrifid (13,141), Lumifer (11,480), gwern (10,952), and Vladimir_Nesov (8,612).

Dataset Creation

Curation Rationale

LessWrong is a long-running forum for rationality, AI alignment, decision theory, philosophy, cognitive science, and adjacent technical discussion. A structured archive makes this corpus available for research into longform online discourse, intellectual community dynamics, and the development of AI safety and rationalist discussion over time.

Source Data

Data Collection and Processing

• —Tool: Custom Python script using the public LessWrong GraphQL API
• —Scope: Public, approved, non-draft, non-future, listed, non-shortform posts visible to anonymous GraphQL requests
• —No filtering was applied to fetched content; all crawled posts and comments are included
• —Comments are stored as a single flat list in the comments column
• —Parent-child relationships are preserved using parentCommentId, topLevelCommentId, position, and depth
• —Comments that could not be attached to the main comment tree are retained in comments with orphan = true
• —Score and vote counts are snapshot values from crawl time

Who are the source data producers?

LessWrong users posting under real names or pseudonymous usernames.

Bias, Risks, and Limitations

Content Risks

• —Content reflects the interests, norms, and epistemic commitments of the LessWrong community
• —Posts may contain speculative, controversial, or technical claims presented with varying levels of evidence
• —Discussions may include sensitive topics such as AI existential risk, mental health, bioethics, politics, religion, self-experimentation, and personal life decisions
• —Historical posts may contain outdated claims, broken links, or references to community controversies

Structural Limitations

• —The crawl was conducted from 2026-05-06 to 2026-05-09 and includes no content discovered beyond that snapshot
• —The latest post in the shards is dated 2026-05-06, while the latest comment is dated 2026-05-09
• —Vote counts, scores, curation status, frontpage status, deletion markers, and retraction markers are snapshot values and may have changed since crawl time
• —The dataset is limited to content discoverable through the public GraphQL API without privileged access
• —Some older imported LessWrong content has missing author metadata or legacy formatting artifacts in htmlBody
• —A small number of comments are marked with orphan = true because they could not be attached to the main comment tree

Personal and Sensitive Information

• —No systematic PII detection or redaction was performed
• —Many users post under stable pseudonyms, and some post under real names
• —Posts and comments may contain references to real individuals, organizations, events, or personal experiences
• —The dataset should be treated as containing potentially sensitive information

Licence and Redistribution

No licence was identified for the source forum content. This dataset is therefore marked as license: unknown.

The dataset consists of scraped public forum posts authored by third parties. The dataset publisher does not claim ownership of the underlying post text and does not grant additional rights beyond any rights users may already have under applicable law.

Users are responsible for determining whether their intended use is lawful, including with respect to copyright, privacy, data protection, and platform terms.