Greatriver/SimBarca

1INFO: Pandarallel will run on 8 workers.2INFO: Pandarallel will use Memory file system to transfer data between the main process and workers.3 4 Selecting 5.0% vehicles from 160946 vehicles5Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/00_traj_1271.5.json.gz6Loading a file takes 43.02s7Creating data frames takes 16.71s8Filling data types and values takes 5.96s9 10 Processing a file takes 17.95s11Concatenating time series takes 0.40s12Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/01_traj_1895.0.json.gz13Loading a file takes 57.68s14Creating data frames takes 21.27s15Filling data types and values takes 5.75s16 17 Processing a file takes 17.25s18Concatenating time series takes 0.28s19Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/02_traj_2453.0.json.gz20Loading a file takes 57.83s21Creating data frames takes 20.35s22Filling data types and values takes 5.69s23 24 Processing a file takes 17.06s25Concatenating time series takes 0.44s26Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/03_traj_2984.5.json.gz27Loading a file takes 54.71s28Creating data frames takes 23.32s29Filling data types and values takes 5.87s30 31 Processing a file takes 17.24s32Concatenating time series takes 0.50s33Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/04_traj_3478.0.json.gz34Loading a file takes 58.59s35Creating data frames takes 20.60s36Filling data types and values takes 5.90s37 38 Processing a file takes 17.23s39Concatenating time series takes 0.48s40Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/05_traj_3956.5.json.gz41Loading a file takes 55.84s42Creating data frames takes 22.26s43Filling data types and values takes 5.92s44 45 Processing a file takes 17.99s46Concatenating time series takes 0.41s47Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/06_traj_4399.0.json.gz48Loading a file takes 59.45s49Creating data frames takes 20.68s50Filling data types and values takes 5.95s51 52 Processing a file takes 17.36s53Concatenating time series takes 0.42s54Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/07_traj_4817.0.json.gz55Loading a file takes 59.45s56Creating data frames takes 21.50s57Filling data types and values takes 5.86s58 59 Processing a file takes 17.62s60Concatenating time series takes 0.37s61Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/08_traj_5225.0.json.gz62Loading a file takes 56.20s63Creating data frames takes 23.42s64Filling data types and values takes 5.96s65 66 Processing a file takes 17.44s67Concatenating time series takes 0.37s68Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/09_traj_5614.5.json.gz69Loading a file takes 59.92s70Creating data frames takes 22.13s71Filling data types and values takes 6.14s72 73 Processing a file takes 18.34s74Concatenating time series takes 0.43s75Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/10_traj_5973.5.json.gz76Loading a file takes 59.81s77Creating data frames takes 25.44s78Filling data types and values takes 6.10s79 80 Processing a file takes 18.05s81Concatenating time series takes 0.58s82Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/11_traj_6309.5.json.gz83Loading a file takes 62.16s84Creating data frames takes 22.77s85Filling data types and values takes 5.88s86 87 Processing a file takes 17.83s88Concatenating time series takes 0.37s89Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/12_traj_6629.5.json.gz90Loading a file takes 54.52s91Creating data frames takes 21.42s92Filling data types and values takes 5.90s93 94 Processing a file takes 17.72s95Concatenating time series takes 0.38s96Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/13_traj_6929.0.json.gz97Loading a file takes 58.15s98Creating data frames takes 19.73s99Filling data types and values takes 5.77s100 101 Processing a file takes 16.45s102Concatenating time series takes 0.46s103Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/14_traj_7205.0.json.gz104Loading a file takes 50.84s105Creating data frames takes 20.49s106Filling data types and values takes 5.79s107 108 Processing a file takes 23.30s109Concatenating time series takes 0.30s110Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/15_traj_7501.0.json.gz111Loading a file takes 55.73s112Creating data frames takes 21.59s113Filling data types and values takes 5.79s114 115 Processing a file takes 35.11s116Concatenating time series takes 0.34s117Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/16_traj_7867.0.json.gz118Loading a file takes 56.65s119Creating data frames takes 21.11s120Filling data types and values takes 5.46s121 122 Processing a file takes 16.65s123Concatenating time series takes 0.33s124Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/17_traj_8321.0.json.gz125Loading a file takes 56.34s126Creating data frames takes 21.00s127Filling data types and values takes 5.97s128 129 Processing a file takes 16.29s130Concatenating time series takes 0.40s131Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/18_traj_8888.0.json.gz132Loading a file takes 55.54s133Creating data frames takes 21.13s134Filling data types and values takes 6.10s135 136 Processing a file takes 16.41s137Concatenating time series takes 0.40s138Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/19_traj_9539.0.json.gz139Loading a file takes 60.37s140Creating data frames takes 21.44s141Filling data types and values takes 6.22s142 143 Processing a file takes 15.18s144Concatenating time series takes 0.35s145Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/20_traj_10244.0.json.gz146Loading a file takes 60.77s147Creating data frames takes 20.39s148Filling data types and values takes 5.95s149 150 Processing a file takes 14.75s151Concatenating time series takes 0.32s152Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/21_traj_11032.5.json.gz153Loading a file takes 57.00s154Creating data frames takes 21.42s155Filling data types and values takes 6.01s156 157 Processing a file takes 14.98s158Concatenating time series takes 0.30s159Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/22_traj_11893.5.json.gz160Loading a file takes 56.38s161Creating data frames takes 22.06s162Filling data types and values takes 6.12s163 164 Processing a file takes 13.77s165Concatenating time series takes 0.34s166Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/23_traj_12806.5.json.gz167Loading a file takes 61.17s168Creating data frames takes 22.92s169Filling data types and values takes 6.03s170 171 Processing a file takes 13.72s172Concatenating time series takes 0.28s173Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/24_traj_13761.5.json.gz174Loading a file takes 60.80s175Creating data frames takes 23.09s176Filling data types and values takes 5.99s177 178 Processing a file takes 14.26s179Concatenating time series takes 0.26s180Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/25_traj_14996.0.json.gz181Loading a file takes 60.99s182Creating data frames takes 22.31s183Filling data types and values takes 6.21s184 185 Processing a file takes 13.79s186Concatenating time series takes 0.39s187Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/26_traj_16809.5.json.gz188Loading a file takes 56.25s189Creating data frames takes 22.75s190Filling data types and values takes 5.94s191 192 Processing a file takes 13.84s193Concatenating time series takes 0.21s194Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/27_traj_18720.0.json.gz195Loading a file takes 58.94s196Creating data frames takes 23.00s197Filling data types and values takes 5.83s198 199 Processing a file takes 12.67s200Concatenating time series takes 0.15s201Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/28_traj_20712.0.json.gz202Loading a file takes 59.22s203Creating data frames takes 20.95s204Filling data types and values takes 5.79s205 206 Processing a file takes 10.47s207Concatenating time series takes 0.18s208Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/29_traj_22793.0.json.gz209Loading a file takes 54.26s210Creating data frames takes 22.34s211Filling data types and values takes 5.58s212 213 Processing a file takes 12.39s214Concatenating time series takes 0.14s215Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/trajectory/30_traj_end.json.gz216Loading a file takes 19.90s217Creating data frames takes 5.65s218Filling data types and values takes 1.20s219 220 Processing a file takes 5.42s221Concatenating time series takes 0.12s222Saving all the section time series takes 124.49s223Saving all the junction time series takes 18.21s224Saving network entrance and exit takes 0.10s225Aggregating the raw statistics to different intervals...226
  0%|          | 0/1525 [00:00<?, ?it/s]
  0%|          | 1/1525 [00:00<20:27,  1.24it/s]
  1%|          | 18/1525 [00:00<01:02, 24.03it/s]
  3%|▎         | 44/1525 [00:01<00:23, 61.88it/s]
  4%|▍         | 59/1525 [00:01<00:19, 76.12it/s]
  5%|▌         | 82/1525 [00:01<00:14, 101.03it/s]
  7%|▋         | 112/1525 [00:01<00:10, 136.91it/s]
  9%|▉         | 134/1525 [00:01<00:09, 140.69it/s]
 10%|█         | 156/1525 [00:01<00:08, 155.30it/s]
 11%|█▏        | 175/1525 [00:01<00:08, 163.23it/s]
 13%|█▎        | 194/1525 [00:01<00:07, 167.23it/s]
 14%|█▍        | 213/1525 [00:02<00:07, 172.12it/s]
 15%|█▌        | 233/1525 [00:02<00:07, 173.39it/s]
 17%|█▋        | 252/1525 [00:02<00:07, 173.40it/s]
 18%|█▊        | 270/1525 [00:02<00:08, 149.47it/s]
 19%|█▉        | 286/1525 [00:02<00:08, 147.45it/s]
 20%|██        | 309/1525 [00:02<00:07, 162.72it/s]
 21%|██▏       | 326/1525 [00:04<00:37, 31.79it/s] 
 23%|██▎       | 347/1525 [00:04<00:26, 43.70it/s]
 25%|██▍       | 379/1525 [00:04<00:16, 67.94it/s]
 26%|██▌       | 399/1525 [00:04<00:13, 81.84it/s]
 27%|██▋       | 419/1525 [00:04<00:11, 92.27it/s]
 29%|██▊       | 437/1525 [00:04<00:10, 103.47it/s]
 30%|██▉       | 455/1525 [00:05<00:09, 115.05it/s]
 31%|███       | 475/1525 [00:05<00:07, 131.49it/s]
 32%|███▏      | 493/1525 [00:05<00:08, 124.76it/s]
 33%|███▎      | 509/1525 [00:05<00:08, 125.58it/s]
 34%|███▍      | 524/1525 [00:05<00:07, 128.88it/s]
 35%|███▌      | 539/1525 [00:05<00:07, 123.84it/s]
 36%|███▋      | 553/1525 [00:05<00:11, 88.31it/s] 
 37%|███▋      | 564/1525 [00:06<00:13, 72.82it/s]
 38%|███▊      | 581/1525 [00:06<00:10, 88.61it/s]
 39%|███▉      | 596/1525 [00:06<00:09, 100.31it/s]
 40%|███▉      | 609/1525 [00:06<00:09, 100.22it/s]
 41%|████      | 622/1525 [00:06<00:08, 106.02it/s]
 42%|████▏     | 639/1525 [00:06<00:07, 115.29it/s]
 43%|████▎     | 653/1525 [00:06<00:07, 115.39it/s]
 44%|████▍     | 672/1525 [00:06<00:06, 124.81it/s]
 46%|████▌     | 694/1525 [00:07<00:05, 146.97it/s]
 47%|████▋     | 710/1525 [00:07<00:05, 141.99it/s]
 48%|████▊     | 729/1525 [00:07<00:05, 140.60it/s]
 49%|████▉     | 748/1525 [00:07<00:05, 149.55it/s]
 50%|█████     | 764/1525 [00:07<00:05, 152.17it/s]
 51%|█████     | 780/1525 [00:07<00:05, 148.69it/s]
 52%|█████▏    | 796/1525 [00:07<00:04, 150.45it/s]
 53%|█████▎    | 812/1525 [00:07<00:04, 146.54it/s]
 54%|█████▍    | 827/1525 [00:08<00:05, 128.84it/s]
 55%|█████▌    | 841/1525 [00:08<00:05, 123.99it/s]
 56%|█████▌    | 856/1525 [00:08<00:05, 121.28it/s]
 57%|█████▋    | 871/1525 [00:08<00:05, 127.95it/s]
 58%|█████▊    | 885/1525 [00:08<00:05, 115.41it/s]
 59%|█████▉    | 902/1525 [00:08<00:04, 127.02it/s]
 60%|██████    | 916/1525 [00:08<00:04, 127.66it/s]
 61%|██████    | 930/1525 [00:08<00:04, 120.46it/s]
 62%|██████▏   | 945/1525 [00:09<00:04, 125.74it/s]
 63%|██████▎   | 965/1525 [00:09<00:04, 137.40it/s]
 65%|██████▍   | 986/1525 [00:09<00:03, 156.47it/s]
 66%|██████▌   | 1002/1525 [00:09<00:03, 136.57it/s]
 67%|██████▋   | 1017/1525 [00:09<00:03, 129.96it/s]
 68%|██████▊   | 1031/1525 [00:09<00:04, 107.34it/s]
 68%|██████▊   | 1043/1525 [00:09<00:04, 104.78it/s]
 70%|██████▉   | 1061/1525 [00:09<00:03, 116.11it/s]
 70%|███████   | 1075/1525 [00:10<00:03, 118.48it/s]
 71%|███████▏  | 1088/1525 [00:10<00:04, 106.21it/s]
 73%|███████▎  | 1106/1525 [00:10<00:03, 119.13it/s]
 73%|███████▎  | 1119/1525 [00:10<00:03, 115.62it/s]
 74%|███████▍  | 1131/1525 [00:10<00:03, 103.72it/s]
 75%|███████▍  | 1142/1525 [00:10<00:03, 104.29it/s]
 76%|███████▌  | 1158/1525 [00:10<00:03, 114.52it/s]
 77%|███████▋  | 1170/1525 [00:10<00:03, 113.13it/s]
 78%|███████▊  | 1182/1525 [00:11<00:03, 107.28it/s]
 78%|███████▊  | 1195/1525 [00:11<00:02, 110.02it/s]
 79%|███████▉  | 1208/1525 [00:11<00:02, 106.31it/s]
 80%|████████  | 1224/1525 [00:11<00:02, 113.87it/s]
 82%|████████▏ | 1247/1525 [00:11<00:02, 128.74it/s]
 83%|████████▎ | 1262/1525 [00:11<00:01, 133.05it/s]
 84%|████████▍ | 1284/1525 [00:11<00:01, 155.29it/s]
 86%|████████▌ | 1309/1525 [00:11<00:01, 176.77it/s]
 87%|████████▋ | 1327/1525 [00:12<00:01, 165.01it/s]
 88%|████████▊ | 1344/1525 [00:12<00:01, 145.95it/s]
 89%|████████▉ | 1361/1525 [00:14<00:06, 26.85it/s] 
 90%|█████████ | 1374/1525 [00:14<00:04, 32.91it/s]
 91%|█████████ | 1389/1525 [00:14<00:03, 41.41it/s]
 92%|█████████▏| 1406/1525 [00:14<00:02, 53.97it/s]
 93%|█████████▎| 1420/1525 [00:14<00:01, 64.13it/s]
 94%|█████████▍| 1440/1525 [00:14<00:01, 84.15it/s]
 95%|█████████▌| 1456/1525 [00:14<00:00, 81.37it/s]
 97%|█████████▋| 1475/1525 [00:14<00:00, 97.04it/s]
 98%|█████████▊| 1492/1525 [00:15<00:00, 103.89it/s]
100%|██████████| 1525/1525 [00:15<00:00, 100.31it/s]227Constructing time series for each modality ...228Saving aggregated time series to /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_094/timeseries/agg_timeseries.pkl229Extracting samples from time series ...230number of samples: 20231Packing the samples into numpy arrays ...232