Datasetpublic

dataproc5/intermediate-danbooru2025-metadata-prioritized

dataproc5/intermediate-danbooru2025-metadata-prioritized (Auto-generated summary) Basic Info: Shape: 9113291 rows × 61 columns Total Memory Usage: 35.96 GB Duplicates: 7 (0.00%) Column Stats: Error generating stats table. Column Summaries: → file_url (object) Unique values: 8836519 → approver_id (float64) Min: 1.000, Max: 1215532.000, Mean: 318955.604, Std: 309967.830 → bit_flags (int64) Min: 0.000, Max: 3.000, Mean: 0.401… See the full description on the dataset page: https://huggingface.co/datasets/dataproc5/intermediate-danbooru2025-metadata-prioritized.

sourceHugging Faceupdated 1y agoView on Hugging Face

1likes395downloads

Dataset Card

dataproc5/intermediate-danbooru2025-metadata-prioritized

(Auto-generated summary)

Basic Info:

—Shape: 9113291 rows × 61 columns
—Total Memory Usage: 35.96 GB
—Duplicates: 7 (0.00%)

Column Stats:

Error generating stats table.

Column Summaries:

→ file_url (object)

—Unique values: 8836519

→ approver_id (float64)

—Min: 1.000, Max: 1215532.000, Mean: 318955.604, Std: 309967.830

→ bit_flags (int64)

—Min: 0.000, Max: 3.000, Mean: 0.401, Std: 0.801

→ created_at (object)

—Unique values: 9112565

→ down_score (int64)

—Min: -179.000, Max: 8.000, Mean: -0.238, Std: 0.792

→ fav_count (int64)

—Min: 0.000, Max: 4458.000, Mean: 32.490, Std: 54.924

→ file_ext (object)

—Unique values: 9
—'jpg': 6680308 (73.30%)
—'png': 2313492 (25.39%)
—'mp4': 47075 (0.52%)
—'gif': 36300 (0.40%)
—'zip': 18212 (0.20%)
—'webp': 13889 (0.15%)
—'webm': 2687 (0.03%)
—'swf': 1202 (0.01%)
—'avif': 126 (0.00%)

→ file_size (int64)

—Min: 49.000, Max: 106133556.000, Mean: 1516887.645, Std: 2966698.233

→ hasactivechildren (bool)

—True: 912245 (10.01%), False: 8201046 (89.99%)

→ has_children (bool)

—True: 975691 (10.71%), False: 8137600 (89.29%)

→ has_large (bool)

—True: 6433666 (70.60%), False: 2679625 (29.40%)

→ hasvisiblechildren (bool)

—True: 938904 (10.30%), False: 8174387 (89.70%)

→ image_height (int64)

—Min: 1.000, Max: 54250.000, Mean: 1760.387, Std: 1184.680

→ image_width (int64)

—Min: 1.000, Max: 35102.000, Mean: 1471.753, Std: 963.848

→ is_banned (bool)

—True: 103425 (1.13%), False: 9009866 (98.87%)

→ is_deleted (bool)

—True: 487053 (5.34%), False: 8626238 (94.66%)

→ is_flagged (bool)

—True: 55 (0.00%), False: 9113236 (100.00%)

→ is_pending (bool)

—True: 1030 (0.01%), False: 9112261 (99.99%)

→ largefileurl (object)

—Unique values: 8836519

→ lastcommentbumped_at (object)

—Unique values: 753609

→ lastcommentedat (object)

—Unique values: 862772

→ lastnotedat (object)

—Unique values: 666345

→ md5 (object)

—Unique values: 8836519

→ mediaassetcreated_at (object)

—Unique values: 9099187

→ mediaassetduration (float64)

—Min: 0.000, Max: 405.610, Mean: 13.035, Std: 21.803

→ mediaassetfile_ext (object)

—Unique values: 9
—'jpg': 6680308 (73.30%)
—'png': 2313492 (25.39%)
—'mp4': 47075 (0.52%)
—'gif': 36300 (0.40%)
—'zip': 18212 (0.20%)
—'webp': 13889 (0.15%)
—'webm': 2687 (0.03%)
—'swf': 1202 (0.01%)
—'avif': 126 (0.00%)

→ mediaassetfile_key (object)

—Unique values: 8836519

→ mediaassetfile_size (int64)

—Min: 49.000, Max: 106133556.000, Mean: 1516887.645, Std: 2966698.233

→ mediaassetid (int64)

—Min: 1.000, Max: 25544968.000, Mean: 8588859.003, Std: 8300218.532

→ mediaassetimage_height (int64)

—Min: 1.000, Max: 54250.000, Mean: 1760.387, Std: 1184.680

→ mediaassetimage_width (int64)

—Min: 1.000, Max: 35102.000, Mean: 1471.753, Std: 963.848

→ mediaassetis_public (bool)

—True: 9113291 (100.00%), False: 0 (0.00%)

→ mediaassetmd5 (object)

—Unique values: 8836519

→ mediaassetpixel_hash (object)

—Unique values: 9087959

→ mediaassetstatus (object)

—Unique values: 1
—'active': 9113291 (100.00%)

→ mediaassetupdated_at (object)

—Unique values: 9036440

→ mediaassetvariants (object)

—Unique values: 8836519

→ parent_id (float64)

—Min: 27.000, Max: 9159019.000, Mean: 4888907.932, Std: 2428185.567

→ pixiv_id (float64)

—Min: 20.000, Max: 130571504.000, Mean: 70977427.831, Std: 35090233.521

→ previewfileurl (object)

—Unique values: 8835372

→ rating (object)

—Unique values: 4
—'s': 4514108 (49.53%)
—'g': 2682333 (29.43%)
—'q': 1024235 (11.24%)
—'e': 892615 (9.79%)

→ score (int64)

—Min: -167.000, Max: 2693.000, Mean: 26.151, Std: 47.964

→ source (object)

—Unique values: 8400965

→ tag_count (int64)

—Min: 1.000, Max: 1250.000, Mean: 36.316, Std: 15.692

→ tagcountartist (int64)

—Min: 0.000, Max: 272.000, Mean: 0.987, Std: 0.245

→ tagcountcharacter (int64)

—Min: 0.000, Max: 1203.000, Mean: 1.620, Std: 2.633

→ tagcountcopyright (int64)

—Min: 0.000, Max: 582.000, Mean: 1.389, Std: 0.929

→ tagcountgeneral (int64)

—Min: 0.000, Max: 654.000, Mean: 30.021, Std: 14.631

→ tagcountmeta (int64)

—Min: 0.000, Max: 17.000, Mean: 2.300, Std: 1.342

→ tag_string (object)

—Unique values: 9010785

→ tagstringartist (object)

—Unique values: 467603

→ tagstringcharacter (object)

—Unique values: 941333

→ tagstringcopyright (object)

—Unique values: 210056

→ tagstringgeneral (object)

—Unique values: 8797626

→ tagstringmeta (object)

—Unique values: 99785

→ up_score (int64)

—Min: 0.000, Max: 2700.000, Mean: 25.871, Std: 48.155

→ updated_at (object)

—Unique values: 9111678

→ uploader_id (int64)

—Min: 1.000, Max: 1340604.000, Mean: 450762.181, Std: 325888.208

→ id (int64)

—Min: 1.000, Max: 9158800.000, Mean: 4590531.375, Std: 2635819.924

→ balancingtags (object)

—Contains unhashable sequence: ndarray
—Typical length: mean=31.64, min=0, max=1212

→ _priority (float64)

—Min: -149.400, Max: 3071.160, Mean: 30.321, Std: 54.949

Sample Rows (first 3):

                                          file_url  approver_id  bit_flags                    created_at  down_score  fav_count file_ext  file_size  has_active_children  has_children  has_large  has_visible_children  image_height  image_width  is_banned  is_deleted  is_flagged  is_pending                                     large_file_url        last_comment_bumped_at             last_commented_at last_noted_at                              md5        media_asset_created_at  media_asset_duration media_asset_file_ext media_asset_file_key  media_asset_file_size  media_asset_id  media_asset_image_height  media_asset_image_width  media_asset_is_public                  media_asset_md5           media_asset_pixel_hash media_asset_status        media_asset_updated_at                               media_asset_variants  parent_id  pixiv_id                                   preview_file_url rating  score                                             source  tag_count  tag_count_artist  tag_count_character  tag_count_copyright  tag_count_general  tag_count_meta                                         tag_string                                  tag_string_artist                               tag_string_character           tag_string_copyright                                 tag_string_general                                    tag_string_meta  up_score                    updated_at  uploader_id      id                                    _balancing_tags  _priority
https://cdn.donmai.us/original/b9/19/b919d2ed5e...          NaN          0 2022-01-25T13:09:21.305-05:00          -7       2635      mp4   51123503                 True          True      False                  True           720         1280      False       False       False       False https://cdn.donmai.us/original/b9/19/b919d2ed5e... 2022-01-25T13:12:33.563-05:00 2022-02-11T11:25:51.543-05:00          None b919d2ed5e8ba0b5f0834bf10f61898e 2022-01-25T13:07:53.997-05:00            119.066667                  mp4            AxqeAPbj8               51123503         5173798                       720                     1280                   True b919d2ed5e8ba0b5f0834bf10f61898e b919d2ed5e8ba0b5f0834bf10f61898e             active 2023-03-26T18:30:25.941-04:00 {'type': '180x180', 'url': 'https://cdn.donmai....        NaN       NaN https://cdn.donmai.us/180x180/b9/19/b919d2ed5e8...      e   2693 https://twitter.com/nagoonimation/status/148599...         68                 4                    1                    2                 52               9 1boy 1girl 3d all_fours animated anus artist_co... icedev mustard_sfm nagoonimation pleasedbyviole...                                         marie_rose  dead_or_alive dead_or_alive_5 1boy 1girl 3d all_fours anus artist_name ass ba... animated artist_collaboration audible_music aud...      2700 2024-11-10T20:22:40.421-05:00       110107 5083150 ['black_ribbon' 'marie_rose' 'cum_overflow' 'lo...    3071.16
https://cdn.donmai.us/original/d0/8f/d08fc57713...     499293.0          0 2023-11-22T19:54:58.864-05:00          -4       2456      mp4    1343460                False         False      False                 False          1056          720      False       False       False       False https://cdn.donmai.us/original/d0/8f/d08fc57713... 2024-07-13T22:54:19.677-04:00 2024-07-13T22:54:19.677-04:00          None d08fc57713a577967532cb0c9c01ffb4 2023-11-22T19:38:26.768-05:00             21.866667                  mp4            6DOh1SPqp                1343460        16670487                      1056                      720                   True d08fc57713a577967532cb0c9c01ffb4 d08fc57713a577967532cb0c9c01ffb4             active 2023-11-22T19:38:30.447-05:00 {'type': '180x180', 'url': 'https://cdn.donmai....  6906725.0       NaN https://cdn.donmai.us/180x180/d0/8f/d08fc57713a...      e   2521 https://twitter.com/librarylewds/status/1727477...         62                 2                    1                    1                 52               6 1boy 1girl after_fellatio animated bare_shoulde...                                lehornysfx the_atko                                    saki_(the_atko)                       original 1boy 1girl after_fellatio bare_shoulders blue_e... animated commentary english_commentary second-p...      2525 2024-10-13T01:05:23.354-04:00       616194 6907632 ['blush' 'looking_at_viewer' 'breasts' 'hair_be...    2925.52
https://cdn.donmai.us/original/18/57/1857365cff...          NaN          2 2019-02-08T07:07:50.692-05:00         -19       4458      gif    1817114                False         False      False                 False           720          960      False       False       False       False https://cdn.donmai.us/original/18/57/1857365cff... 2024-12-09T00:50:35.846-05:00 2024-12-09T00:50:35.846-05:00          None 1857365cffdeaabaef57258d4ac94430 2019-02-08T07:07:50.692-05:00              0.580000                  gif            GfdXaftu8                1817114         3376579                       720                      960                   True 1857365cffdeaabaef57258d4ac94430 1857365cffdeaabaef57258d4ac94430             active 2023-02-27T09:47:07.201-05:00 {'type': '180x180', 'url': 'https://cdn.donmai....        NaN       NaN https://cdn.donmai.us/180x180/18/57/1857365cffd...      e   2651              https://nijie.info/view.php?id=300349         83                 1                    4                    1                 73               4 4girls animated animated_gif arms_behind_back b...                              mushiro_(nijie728995) himesaka_noa hoshino_hinata hoshino_miyako_(wat... watashi_ni_tenshi_ga_maiorita! 4girls arms_behind_back backpack bag belly blac... animated animated_gif commentary looping_animation      2670 2024-10-27T00:28:47.936-04:00        49984 3410029 ['blush' 'orange_hair' 'shirosaki_hana' 'solo_f...    2864.16

Usage Example:

python

import unibox as ub
df = ub.loads("hf://dataproc5/intermediate-danbooru2025-metadata-prioritized").to_pandas()

Saving to dataset:

python

import unibox as ub
ub.saves(df, "hf://dataproc5/intermediate-danbooru2025-metadata-prioritized")

(last updated: 2025-04-19 02:57:09.408793)