matcav/datacookers-processed

1---2license: cc-by-4.03language:4- en5size_categories:6- 100M<n<1B7configs:8- config_name: channels_clean9  data_files: "channels_clean.parquet"10- config_name: grammy_raw11  data_files: "grammy_raw.parquet"12- config_name: grammy_metadata13  data_files: "grammy_metadata.parquet"14- config_name: grammy_channels15  data_files: "grammy_channels.parquet"16- config_name: grammy_metadata_extended17  data_files: "grammy_metadata_extended.parquet"18- config_name: grammy_timeseries19  data_files: "grammy_timeseries.parquet"20- config_name: music_metadata21  data_files: "music_metadata.parquet"22- config_name: music_video_ids23  data_files: "music_video_ids.parquet"24- config_name: music_comments25  data_files: "music_comments.parquet"26- config_name: cf_item_factors27  data_files: "CF_item_factors.parquet"28- config_name: cf_user_factors29  data_files: "CF_user_factors.parquet"30---31 32# Preprocessed Data for the ADA Project 202533 34## By DataCookers35 36### General Datasets37 38* **channels_clean.parquet**: A cleaned version of the `df_channels_en` split from the Youniverse Dataset. This dataset corrects parsing errors where missing values in the **Category** column caused subsequent columns to shift one position to the left.39* **grammy_raw.parquet**: The foundational dataset containing Grammy winners and nominees (1965–2024), sourced from [Kaggle](https://www.kaggle.com/datasets/johnpendenque/grammy-winners-and-nominees-from-1965-to-2024).40* **grammy_metadata.parquet**: A merged dataset combining the *grammy_raw* data with the `yt_metadata_en` split from Youniverse. It is further enriched with web-scraped song lyrics.41* **grammy_channels.parquet**: A filtered subset of `df_channels_en` containing only those YouTube channels that have published a Grammy-nominated or winning song.42* **grammy_metadata_extended.parquet**: An expansion of the metadata containing all videos belonging to the channels identified in *grammy_channels* (not just the Grammy-winning videos).43* **grammy_timeseries.parquet**: Temporal data (time series) specifically associated with the channels found in the *grammy_channels* dataset.44* **music_metadata.parquet**: A subset of the original `yt_metadata_en` Youniverse split, filtered to include only entries classified under the **Music** Category.45* **music_video_ids.parquet**: A lightweight dataset containing the specific video IDs filtered from the `yt_metadata_en` split, strictly for videos listed in the **Music** category.46* **music_comments.parquet**: A collection of user comments obtained by filtering `youtube_comments.tsv.gz`. It retains only comments posted on videos present in the *music_video_ids* dataset.47 48### Collaborative Filtering Data49 50The following files represent the output of Matrix Factorization performed on the `merged_comments.tsv.gz` split. 51 52**Methodology:** 1.  We constructed a sparse interaction matrix of shape `(users, items)` with binary entries (**0** or **1**), where a **1** indicates the user commented on a specific video.532.  We performed Matrix Factorization using the Alternating Least Squares (ALS) algorithm via the `implicit` library.54 55**Resulting Files:**56 57* **CF_item_factors.parquet**: The latent space vectors representing every `video_id` (Item Factors).58* **CF_user_factors.parquet**: The latent space vectors representing every `author` (User Factors).59* **CF_als_model.pkl**: The serialized weights of the trained ALS model.60* **CF_user_id_map.pkl**: A dictionary mapping the **original** `author` (string) to the integer ID used during ALS model training.61* **CF_item_id_map.pkl**: A dictionary mapping the **original** `video_id` (string) to the integer ID used during ALS model training.