CompassioninMachineLearning/lmsys-chat-1m-animal-welfare-coarse-filter

LMSYS Chat 1M Animal Welfare Coarse Filter

Public working dataset for Compassion in Machine Learning.

Source dataset: lmsys/lmsys-chat-1m

Filter package: flpc

Filter used: original coarse animal-welfare keyword list provided by the project team.

Counts:

• —total conversations scanned: 1,000,000
• —matched conversations: 16,527
• —match rate: 1.6527%

Files:

• —matches.parquet: one row per matched conversation, preserving all original source fields/columns plus _matched_terms, _text_preview, and _source_dataset.
• —samples.jsonl: reservoir sample for sanity checking, using the same all-columns-preserving structure as matches.parquet rows.
• —summary.txt: run summary and top matched terms.

Schema notes:

• —ShareLM keeps all original fields, including timestamp, source, user_id, conversation_metadata, and user_metadata when present.
• —LMSYS keeps all original fields: conversation_id, model, conversation, turn, language, openai_moderation, redacted.
• —ShareChat is conversation-grouped in matches.parquet with a messages list containing every original message row/CSV column; matched_messages.parquet is also included for flat message-level analysis.
• —Added filter fields are prefixed with _.
• —This is a coarse, high-recall keyword filter and intentionally tolerates false positives.