mmarkusmalone/journal-entries-emotion-detection-vad

Reddit Diary of a Redditor VAD Dataset Dataset Creation Process

• —Scraping Reddit Posts
• —Posts were scraped from the r/diaryofaredditor subreddit using the Reddit API.
• —The script used for scraping is shown below:

    import requests
    import csv
    import time

    access_token = ""
    headers = {
        "Authorization": f"bearer {access_token}",
        "User-Agent": "ChangeMeClient/0.1"
    }

    url = "https://oauth.reddit.com/r/diaryofaredditor/new"
    params = {"limit": 100}
    after = None

    csv_path = "/Users/mayamarkus-malone/Documents/VADMAP/data/reddit_posts.csv"

    # Open CSV in append mode (do not write header)
    with open(csv_path, "a", encoding="utf-8", newline='') as csvfile:
        csv_writer = csv.writer(csvfile)
        fetched = 0
        for _ in range(10):  # 10 requests × 100 posts = 1000 posts
            if after:
                params["after"] = after
            response = requests.get(url, headers=headers, params=params)
            data = response.json()
            posts = data.get("data", {}).get("children", [])
            if not posts:
                print("No more posts found.")
                break
            for post in posts:
                selftext = post["data"].get("selftext", "")
                content = (selftext).strip()
                csv_writer.writerow([content, ""])
                fetched += 1
            after = data.get("data", {}).get("after", None)
            if not after:
                print("Reached end of listing.")
                break
            time.sleep(2)  # Be nice to Reddit's API

    print(f"Fetched and appended {fetched} posts.")

• —Chunking
• —Each post was split into smaller chunks by newlines to ensure manageable input sizes for downstream processing.

• —Automatic VAD Labeling
• —Each chunk was automatically labeled for Valence, Arousal, and Dominance (VAD) using a custom script (data_labeling_auto.py).
• —This script uses a transformer-based model (SamLowe/roberta-base-go_emotions-onnx) and a mapping from emotion labels to VAD values.
• —The model output was mapped to VAD scores using a weighted average based on the predicted emotion probabilities.

• —Manual Labeling with Custom Interface
• —A custom web interface was developed to review and refine the auto-generated VAD labels.
• —The interface pre-populated the sliders with the auto labels, allowing a human annotator to accept or adjust the VAD values for each chunk.
• —Final labels were saved to vad_data.csv.
• —

Dataset Statistics

• —Average Valence (V): -0.0281
• —Average Arousal (A): 0.1176
• —Average Dominance (D): 0.0016

Files

• —reddit_posts_clean.csv: Raw, cleaned Reddit posts.
• —reddit_posts_vad.csv: Posts with auto-generated VAD labels.
• —vad_data.csv: Final, manually-labeled VAD values.

Credits

• —Auto-labeling pipeline: Adapted from SamLowe/roberta-base-go_emotions-onnx and custom mapping logic in data_labeling_auto.py.
• —Manual labeling interface: Custom-built for this project.

Credits For questions or collaboration, please open an issue or contact the dataset creator.