eljavatar/pretrain_utg4java_codet5p

1SUMMARY MODEL:2 3Model Params Size: 2228820484Model Params Size Formatted: 222.88 M5Model Disk Size: 8916482556Model Disk Size Formatted: 891.65 MB7 8 9TRAINING AND VALIDATION RESULTS:10 11Training batch size: 6412Validation batch size: 12813Total expected epochs: 4014Total expected trainig steps: 9404015Total expected trainig steps 2: 9404016Total trained epochs: 40.017Total trained steps: 9404018Elapsed time: 54841.370337963104 seconds19Elapsed time (formatted): 15:14:0120Total flos: 3.6650496018087936e+1821Total flos (formatted): 3.665050e+1822Best epoch val_loss: 0.2090902775526046823Best model checkpoint: /root/pretrain_utg4java_02/checkpoint-9168924 25 26 27SUMMARY DATASETS:28 29Loaded Dataset:30DatasetDict({31    train: Dataset({32        features: ['text'],33        num_rows: 15052334    })35    valid: Dataset({36        features: ['text'],37        num_rows: 1881638    })39    test: Dataset({40        features: ['text'],41        num_rows: 1881542    })43})44 45Tokenized Dataset:46DatasetDict({47    train: Dataset({48        features: ['input_ids'],49        num_rows: 15052350    })51    valid: Dataset({52        features: ['input_ids'],53        num_rows: 1881654    })55    test: Dataset({56        features: ['input_ids'],57        num_rows: 1881558    })59})60 61