Greatriver/SimBarca

1INFO: Pandarallel will run on 8 workers.2INFO: Pandarallel will use Memory file system to transfer data between the main process and workers.3 4 Selecting 5.0% vehicles from 164470 vehicles5Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/00_traj_1265.5.json.gz6Loading a file takes 52.96s7Creating data frames takes 23.83s8Filling data types and values takes 7.74s9 10 Processing a file takes 19.23s11Concatenating time series takes 0.40s12Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/01_traj_1882.0.json.gz13Loading a file takes 60.05s14Creating data frames takes 28.81s15Filling data types and values takes 7.32s16 17 Processing a file takes 17.13s18Concatenating time series takes 0.23s19Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/02_traj_2443.0.json.gz20Loading a file takes 60.98s21Creating data frames takes 28.55s22Filling data types and values takes 7.19s23 24 Processing a file takes 17.71s25Concatenating time series takes 0.23s26Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/03_traj_2964.5.json.gz27Loading a file takes 59.73s28Creating data frames takes 42.89s29Filling data types and values takes 8.08s30 31 Processing a file takes 17.04s32Concatenating time series takes 0.39s33Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/04_traj_3442.0.json.gz34Loading a file takes 52.73s35Creating data frames takes 24.56s36Filling data types and values takes 5.69s37 38 Processing a file takes 17.28s39Concatenating time series takes 0.30s40Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/05_traj_3880.0.json.gz41Loading a file takes 55.15s42Creating data frames takes 22.25s43Filling data types and values takes 5.71s44 45 Processing a file takes 16.82s46Concatenating time series takes 0.61s47Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/06_traj_4296.5.json.gz48Loading a file takes 65.29s49Creating data frames takes 94.83s50Filling data types and values takes 5.88s51 52 Processing a file takes 18.19s53Concatenating time series takes 0.59s54Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/07_traj_4687.5.json.gz55Loading a file takes 63.57s56Creating data frames takes 22.31s57Filling data types and values takes 5.85s58 59 Processing a file takes 17.69s60Concatenating time series takes 0.37s61Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/08_traj_5053.5.json.gz62Loading a file takes 55.30s63Creating data frames takes 23.64s64Filling data types and values takes 5.92s65 66 Processing a file takes 17.82s67Concatenating time series takes 0.43s68Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/09_traj_5396.5.json.gz69Loading a file takes 58.83s70Creating data frames takes 22.67s71Filling data types and values takes 5.94s72 73 Processing a file takes 17.01s74Concatenating time series takes 0.48s75Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/10_traj_5718.5.json.gz76Loading a file takes 57.75s77Creating data frames takes 22.78s78Filling data types and values takes 5.98s79 80 Processing a file takes 17.69s81Concatenating time series takes 0.79s82Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/11_traj_6027.5.json.gz83Loading a file takes 59.41s84Creating data frames takes 20.31s85Filling data types and values takes 6.11s86 87 Processing a file takes 17.49s88Concatenating time series takes 0.37s89Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/12_traj_6323.5.json.gz90Loading a file takes 57.74s91Creating data frames takes 22.15s92Filling data types and values takes 6.00s93 94 Processing a file takes 18.29s95Concatenating time series takes 0.42s96Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/13_traj_6607.0.json.gz97Loading a file takes 57.81s98Creating data frames takes 22.36s99Filling data types and values takes 5.88s100 101 Processing a file takes 17.19s102Concatenating time series takes 0.40s103Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/14_traj_6876.5.json.gz104Loading a file takes 57.95s105Creating data frames takes 22.52s106Filling data types and values takes 6.00s107 108 Processing a file takes 16.81s109Concatenating time series takes 0.35s110Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/15_traj_7137.0.json.gz111Loading a file takes 57.83s112Creating data frames takes 26.14s113Filling data types and values takes 6.13s114 115 Processing a file takes 17.54s116Concatenating time series takes 0.47s117Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/16_traj_7411.0.json.gz118Loading a file takes 54.85s119Creating data frames takes 28.15s120Filling data types and values takes 5.95s121 122 Processing a file takes 21.00s123Concatenating time series takes 0.32s124Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/17_traj_7740.0.json.gz125Loading a file takes 54.84s126Creating data frames takes 26.44s127Filling data types and values takes 5.84s128 129 Processing a file takes 25.75s130Concatenating time series takes 0.45s131Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/18_traj_8113.0.json.gz132Loading a file takes 54.74s133Creating data frames takes 26.55s134Filling data types and values takes 5.41s135 136 Processing a file takes 25.26s137Concatenating time series takes 0.33s138Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/19_traj_8511.0.json.gz139Loading a file takes 49.40s140Creating data frames takes 24.69s141Filling data types and values takes 5.82s142 143 Processing a file takes 18.68s144Concatenating time series takes 0.22s145Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/20_traj_8928.5.json.gz146Loading a file takes 57.00s147Creating data frames takes 24.52s148Filling data types and values takes 5.77s149 150 Processing a file takes 14.23s151Concatenating time series takes 0.34s152Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/21_traj_9363.0.json.gz153Loading a file takes 54.55s154Creating data frames takes 22.12s155Filling data types and values takes 5.78s156 157 Processing a file takes 13.80s158Concatenating time series takes 0.34s159Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/22_traj_9832.0.json.gz160Loading a file takes 52.70s161Creating data frames takes 23.03s162Filling data types and values takes 6.13s163 164 Processing a file takes 15.03s165Concatenating time series takes 0.39s166Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/23_traj_10358.5.json.gz167Loading a file takes 60.06s168Creating data frames takes 23.57s169Filling data types and values takes 6.01s170 171 Processing a file takes 15.13s172Concatenating time series takes 0.31s173Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/24_traj_10917.0.json.gz174Loading a file takes 59.81s175Creating data frames takes 25.30s176Filling data types and values takes 6.02s177 178 Processing a file takes 14.24s179Concatenating time series takes 0.33s180Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/25_traj_11521.0.json.gz181Loading a file takes 55.39s182Creating data frames takes 23.32s183Filling data types and values takes 6.09s184 185 Processing a file takes 14.33s186Concatenating time series takes 0.31s187Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/26_traj_12191.5.json.gz188Loading a file takes 60.13s189Creating data frames takes 26.74s190Filling data types and values takes 6.11s191 192 Processing a file takes 14.36s193Concatenating time series takes 0.33s194Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/27_traj_12971.5.json.gz195Loading a file takes 55.88s196Creating data frames takes 27.40s197Filling data types and values takes 6.04s198 199 Processing a file takes 15.39s200Concatenating time series takes 0.65s201Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/28_traj_13918.0.json.gz202Loading a file takes 60.44s203Creating data frames takes 24.75s204Filling data types and values takes 6.03s205 206 Processing a file takes 14.65s207Concatenating time series takes 0.26s208Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/29_traj_15084.0.json.gz209Loading a file takes 59.83s210Creating data frames takes 25.01s211Filling data types and values takes 5.96s212 213 Processing a file takes 14.18s214Concatenating time series takes 0.27s215Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/30_traj_16527.5.json.gz216Loading a file takes 55.09s217Creating data frames takes 21.96s218Filling data types and values takes 5.90s219 220 Processing a file takes 12.66s221Concatenating time series takes 0.18s222Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/31_traj_18340.0.json.gz223Loading a file takes 58.00s224Creating data frames takes 32.35s225Filling data types and values takes 5.84s226 227 Processing a file takes 10.75s228Concatenating time series takes 0.18s229Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/32_traj_20207.0.json.gz230Loading a file takes 55.66s231Creating data frames takes 21.79s232Filling data types and values takes 5.28s233 234 Processing a file takes 9.55s235Concatenating time series takes 0.10s236Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/33_traj_22091.0.json.gz237Loading a file takes 56.34s238Creating data frames takes 24.39s239Filling data types and values takes 5.42s240 241 Processing a file takes 9.20s242Concatenating time series takes 0.12s243Working on file /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/trajectory/34_traj_end.json.gz244Loading a file takes 34.84s245Creating data frames takes 14.02s246Filling data types and values takes 3.48s247 248 Processing a file takes 7.21s249Concatenating time series takes 0.12s250Saving all the section time series takes 131.00s251Saving all the junction time series takes 21.00s252Saving network entrance and exit takes 0.11s253Aggregating the raw statistics to different intervals...254
  0%|          | 0/1526 [00:00<?, ?it/s]
  0%|          | 1/1526 [00:00<19:31,  1.30it/s]
  0%|          | 2/1526 [00:00<11:17,  2.25it/s]
  0%|          | 4/1526 [00:01<05:07,  4.94it/s]
  2%|▏         | 30/1526 [00:01<00:30, 49.06it/s]
  3%|▎         | 48/1526 [00:01<00:21, 68.33it/s]
  6%|▌         | 94/1526 [00:01<00:09, 145.24it/s]
  8%|▊         | 115/1526 [00:01<00:11, 125.40it/s]
  9%|▉         | 141/1526 [00:01<00:09, 152.17it/s]
 11%|█         | 161/1526 [00:01<00:09, 151.10it/s]
 12%|█▏        | 181/1526 [00:02<00:08, 155.06it/s]
 13%|█▎        | 199/1526 [00:02<00:08, 150.11it/s]
 14%|█▍        | 220/1526 [00:02<00:08, 159.28it/s]
 16%|█▌        | 238/1526 [00:02<00:08, 156.00it/s]
 17%|█▋        | 255/1526 [00:02<00:08, 155.02it/s]
 18%|█▊        | 272/1526 [00:02<00:08, 143.39it/s]
 19%|█▉        | 287/1526 [00:02<00:09, 125.68it/s]
 20%|██        | 312/1526 [00:04<00:32, 37.74it/s] 
 22%|██▏       | 333/1526 [00:04<00:23, 50.24it/s]
 24%|██▍       | 365/1526 [00:04<00:15, 74.75it/s]
 26%|██▋       | 402/1526 [00:04<00:10, 109.25it/s]
 28%|██▊       | 425/1526 [00:04<00:08, 122.89it/s]
 29%|██▉       | 447/1526 [00:04<00:08, 131.97it/s]
 31%|███       | 467/1526 [00:05<00:08, 127.44it/s]
 32%|███▏      | 485/1526 [00:05<00:07, 137.15it/s]
 33%|███▎      | 503/1526 [00:05<00:08, 122.91it/s]
 34%|███▍      | 519/1526 [00:05<00:07, 125.98it/s]
 35%|███▍      | 534/1526 [00:05<00:07, 124.93it/s]
 36%|███▌      | 548/1526 [00:05<00:08, 119.18it/s]
 37%|███▋      | 561/1526 [00:05<00:08, 112.89it/s]
 38%|███▊      | 578/1526 [00:05<00:07, 121.47it/s]
 39%|███▉      | 595/1526 [00:06<00:06, 133.30it/s]
 40%|███▉      | 609/1526 [00:06<00:07, 130.77it/s]
 41%|████      | 626/1526 [00:06<00:06, 140.64it/s]
 42%|████▏     | 644/1526 [00:06<00:06, 135.91it/s]
 44%|████▍     | 675/1526 [00:06<00:04, 180.22it/s]
 45%|████▌     | 694/1526 [00:06<00:04, 182.76it/s]
 47%|████▋     | 713/1526 [00:06<00:04, 183.73it/s]
 48%|████▊     | 732/1526 [00:06<00:04, 185.08it/s]
 49%|████▉     | 751/1526 [00:06<00:04, 181.94it/s]
 50%|█████     | 770/1526 [00:07<00:04, 175.42it/s]
 52%|█████▏    | 788/1526 [00:07<00:04, 170.77it/s]
 53%|█████▎    | 806/1526 [00:07<00:04, 170.51it/s]
 54%|█████▍    | 824/1526 [00:07<00:04, 157.12it/s]
 55%|█████▌    | 840/1526 [00:07<00:04, 151.60it/s]
 56%|█████▌    | 857/1526 [00:07<00:04, 154.88it/s]
 57%|█████▋    | 873/1526 [00:07<00:04, 146.30it/s]
 58%|█████▊    | 888/1526 [00:07<00:04, 130.48it/s]
 59%|█████▉    | 902/1526 [00:08<00:04, 126.10it/s]
 60%|██████    | 917/1526 [00:08<00:04, 128.94it/s]
 61%|██████    | 933/1526 [00:08<00:04, 132.93it/s]
 62%|██████▏   | 947/1526 [00:08<00:04, 123.19it/s]
 64%|██████▍   | 976/1526 [00:08<00:03, 161.80it/s]
 65%|██████▌   | 993/1526 [00:08<00:03, 159.95it/s]
 66%|██████▋   | 1012/1526 [00:08<00:03, 166.84it/s]
 67%|██████▋   | 1029/1526 [00:08<00:03, 135.90it/s]
 68%|██████▊   | 1044/1526 [00:09<00:03, 126.95it/s]
 69%|██████▉   | 1058/1526 [00:09<00:03, 128.33it/s]
 70%|███████   | 1072/1526 [00:09<00:04, 104.05it/s]
 71%|███████   | 1087/1526 [00:09<00:04, 104.35it/s]
 72%|███████▏  | 1104/1526 [00:09<00:03, 112.95it/s]
 73%|███████▎  | 1117/1526 [00:09<00:03, 112.48it/s]
 74%|███████▍  | 1129/1526 [00:09<00:03, 111.93it/s]
 75%|███████▌  | 1150/1526 [00:09<00:02, 135.81it/s]
 77%|███████▋  | 1168/1526 [00:10<00:02, 143.59it/s]
 78%|███████▊  | 1183/1526 [00:10<00:02, 134.90it/s]
 78%|███████▊  | 1197/1526 [00:10<00:02, 117.94it/s]
 80%|███████▉  | 1219/1526 [00:10<00:02, 142.37it/s]
 81%|████████▏ | 1241/1526 [00:10<00:01, 160.01it/s]
 83%|████████▎ | 1261/1526 [00:10<00:01, 160.75it/s]
 84%|████████▍ | 1288/1526 [00:10<00:01, 187.26it/s]
 86%|████████▌ | 1308/1526 [00:10<00:01, 163.90it/s]
 87%|████████▋ | 1333/1526 [00:11<00:01, 184.35it/s]
 89%|████████▊ | 1354/1526 [00:11<00:00, 190.61it/s]
 90%|█████████ | 1374/1526 [00:13<00:04, 32.13it/s] 
 91%|█████████ | 1389/1526 [00:13<00:03, 39.14it/s]
 92%|█████████▏| 1409/1526 [00:13<00:02, 51.85it/s]
 94%|█████████▎| 1429/1526 [00:13<00:01, 62.00it/s]
 95%|█████████▍| 1448/1526 [00:13<00:01, 76.55it/s]
 96%|█████████▌| 1464/1526 [00:13<00:00, 86.27it/s]
 97%|█████████▋| 1481/1526 [00:13<00:00, 99.74it/s]
 99%|█████████▊| 1504/1526 [00:13<00:00, 123.71it/s]
100%|██████████| 1526/1526 [00:13<00:00, 109.69it/s]255Constructing time series for each modality ...256Saving aggregated time series to /home/weijiang/Projects/Netsanut/datasets/simbarca/simulation_sessions/session_062/timeseries/agg_timeseries.pkl257Extracting samples from time series ...258number of samples: 20259Packing the samples into numpy arrays ...260