ZeroAurora/Arknights-6th-ThankYou-Celebration-PV-Comments
《明日方舟》2025「感谢庆典」活动宣传PV评论区数据 - 自述卡 任务: 内容爬取 匿名化处理 爬虫工具 blblcd: https://github.com/WShihan/blblcd ClickHouse SQL create table comments (rpid UInt64, parent UInt64, ctime UInt64, mid UInt64, level UInt8, sex String, like UInt64, content String) engine = MergeTree primary key rpid insert into comments from infile 'test.csv' format CSVWithNames alter table comments add column hashed_mid UInt64 after ctime alter table comments update hashed_mid =… See the full description on the dataset page: https://huggingface.co/datasets/ZeroAurora/Arknights-6th-ThankYou-Celebration-PV-Comments.
09
《明日方舟》2025「感谢庆典」活动宣传PV评论区数据 - 自述卡
任务:
- [x] 内容爬取
- [x] 匿名化处理
爬虫工具
blblcd: https://github.com/WShihan/blblcd
ClickHouse SQL
create table comments (rpid UInt64, parent UInt64, ctime UInt64, mid UInt64, level UInt8, sex String, like UInt64, content String) engine = MergeTree primary key rpid
insert into comments from infile 'test.csv' format CSVWithNames
alter table comments add column hashed_mid UInt64 after ctime
alter table comments update hashed_mid = xxh3(concat([toString(mid), '[SALT_REDACTED]'])) where true
alter table comments drop column mid
select * from comments into outfile 'comments-maxctime-minctime.parquet' format Parquet待补充
