HuggingFaceM4/IDEFICS_Data_Measurement_Tool

12023-04-11 03:13:38,542:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:462Tokenized the dataset.32023-08-22 02:28:44,417:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:464Tokenized the dataset.52023-08-22 02:31:19,198:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:466Tokenized the dataset.72023-08-22 03:23:39,876:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:468Tokenized the dataset.92023-08-22 03:25:10,857:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:4610Tokenized the dataset.112023-08-22 03:26:59,957:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:4612Tokenized the dataset.132023-08-22 03:27:39,910:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:4614Tokenized the dataset.152023-08-22 03:46:36,797:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:4616Tokenized the dataset.172023-08-22 03:50:40,779:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:4618Tokenized the dataset.192023-08-22 05:10:28,430:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:4620Tokenized the dataset.212023-08-22 05:12:07,202:/Users/ezi/Desktop/HF/DMT/data-measurements-tool/DMT2023/data-measurements-tool/data_measurements/tokenize.py,  tokenize:4622Tokenized the dataset.23