fals3/methods2test

1---2language:3- en4license: mit5task_categories:6- text-generation7configs:8- config_name: fm9  data_files:10  - split: train11    path: data/fm/train-*12  - split: test13    path: data/fm/test-*14  - split: validation15    path: data/fm/validation-*16- config_name: fm_indented17  data_files:18  - split: train19    path: data/fm_indented/train-*20  - split: test21    path: data/fm_indented/test-*22  - split: validation23    path: data/fm_indented/validation-*24- config_name: fm+t25  data_files:26  - split: train27    path: data/fm+t/train-*28  - split: test29    path: data/fm+t/test-*30  - split: validation31    path: data/fm+t/validation-*32- config_name: fm+fc33  data_files:34  - split: train35    path: data/fm+fc/train-*36  - split: test37    path: data/fm+fc/test-*38  - split: validation39    path: data/fm+fc/validation-*40- config_name: fm+fc+t+tc41  data_files:42  - split: train43    path: data/fm+fc+t+tc/train-*44  - split: test45    path: data/fm+fc+t+tc/test-*46  - split: validation47    path: data/fm+fc+t+tc/validation-*48- config_name: fm+fc+c49  data_files:50  - split: train51    path: data/fm+fc+c/train-*52  - split: test53    path: data/fm+fc+c/test-*54  - split: validation55    path: data/fm+fc+c/validation-*56- config_name: fm+fc+c+t+tc57  data_files:58  - split: train59    path: data/fm+fc+c+t+tc/train-*60  - split: test61    path: data/fm+fc+c+t+tc/test-*62  - split: validation63    path: data/fm+fc+c+t+tc/validation-*64- config_name: fm+fc+c+m65  data_files:66  - split: train67    path: data/fm+fc+c+m/train-*68  - split: test69    path: data/fm+fc+c+m/test-*70  - split: validation71    path: data/fm+fc+c+m/validation-*72- config_name: fm+fc+c+m+t+tc73  data_files:74  - split: train75    path: data/fm+fc+c+m+t+tc/train-*76  - split: test77    path: data/fm+fc+c+m+t+tc/test-*78  - split: validation79    path: data/fm+fc+c+m+t+tc/validation-*80- config_name: fm+fc+c+m+f81  data_files:82  - split: train83    path: data/fm+fc+c+m+f/train-*84  - split: test85    path: data/fm+fc+c+m+f/test-*86  - split: validation87    path: data/fm+fc+c+m+f/validation-*88- config_name: fm+fc+c+m+f+t+tc89  data_files:90  - split: train91    path: data/fm+fc+c+m+f+t+tc/train-*92  - split: test93    path: data/fm+fc+c+m+f+t+tc/test-*94  - split: validation95    path: data/fm+fc+c+m+f+t+tc/validation-*96- config_name: t97  data_files:98  - split: train99    path: data/t/train-*100  - split: test101    path: data/t/test-*102  - split: validation103    path: data/t/validation-*104- config_name: t_indented105  data_files:106  - split: train107    path: data/t_indented/train-*108  - split: test109    path: data/t_indented/test-*110  - split: validation111    path: data/t_indented/validation-*112- config_name: t+tc113  data_files:114  - split: train115    path: data/t+tc/train-*116  - split: test117    path: data/t+tc/test-*118  - split: validation119    path: data/t+tc/validation-*120dataset_info:121- config_name: fm122  features:123  - name: id124    dtype: string125  - name: text126    dtype: string127  splits:128  - name: train129    num_bytes: 440444124130    num_examples: 624022131  - name: test132    num_bytes: 59407291133    num_examples: 78388134  - name: validation135    num_bytes: 57170315136    num_examples: 78534137  download_size: 99172217138  dataset_size: 557021730139- config_name: fm+fc140  features:141  - name: id142    dtype: string143  - name: text144    dtype: string145  splits:146  - name: train147    num_bytes: 506130678148    num_examples: 624022149  - name: test150    num_bytes: 68407490151    num_examples: 78388152  - name: validation153    num_bytes: 65318956154    num_examples: 78534155  download_size: 109141139156  dataset_size: 639857124157- config_name: fm+fc+c158  features:159  - name: id160    dtype: string161  - name: text162    dtype: string163  splits:164  - name: train165    num_bytes: 569209100166    num_examples: 624022167  - name: test168    num_bytes: 75552573169    num_examples: 78388170  - name: validation171    num_bytes: 73101169172    num_examples: 78534173  download_size: 117996353174  dataset_size: 717862842175- config_name: fm+fc+c+m176  features:177  - name: id178    dtype: string179  - name: text180    dtype: string181  splits:182  - name: train183    num_bytes: 1354004338184    num_examples: 624022185  - name: test186    num_bytes: 187724929187    num_examples: 78388188  - name: validation189    num_bytes: 184349299190    num_examples: 78534191  download_size: 222922572192  dataset_size: 1726078566193- config_name: fm+fc+c+m+f194  features:195  - name: id196    dtype: string197  - name: text198    dtype: string199  splits:200  - name: train201    num_bytes: 1476073209202    num_examples: 624022203  - name: test204    num_bytes: 201686811205    num_examples: 78388206  - name: validation207    num_bytes: 201259950208    num_examples: 78534209  download_size: 240405885210  dataset_size: 1879019970211- config_name: fm+fc+c+m+f+t+tc212  features:213  - name: id214    dtype: string215  - name: source216    dtype: string217  - name: target218    dtype: string219  splits:220  - name: train221    num_bytes: 2019918359222    num_examples: 624022223  - name: test224    num_bytes: 269021331225    num_examples: 78388226  - name: validation227    num_bytes: 272958781228    num_examples: 78534229  download_size: 371500476230  dataset_size: 2561898471231- config_name: fm+fc+c+m+t+tc232  features:233  - name: id234    dtype: string235  - name: source236    dtype: string237  - name: target238    dtype: string239  splits:240  - name: train241    num_bytes: 1897682665242    num_examples: 624022243  - name: test244    num_bytes: 255053799245    num_examples: 78388246  - name: validation247    num_bytes: 256030595248    num_examples: 78534249  download_size: 360175965250  dataset_size: 2408767059251- config_name: fm+fc+c+t+tc252  features:253  - name: id254    dtype: string255  - name: source256    dtype: string257  - name: target258    dtype: string259  splits:260  - name: train261    num_bytes: 1109827485262    num_examples: 624022263  - name: test264    num_bytes: 142558255265    num_examples: 78388266  - name: validation267    num_bytes: 144523616268    num_examples: 78534269  download_size: 251861137270  dataset_size: 1396909356271- config_name: fm+fc+t+tc272  features:273  - name: id274    dtype: string275  - name: source276    dtype: string277  - name: target278    dtype: string279  splits:280  - name: train281    num_bytes: 1046592848282    num_examples: 624022283  - name: test284    num_bytes: 135403379285    num_examples: 78388286  - name: validation287    num_bytes: 136729952288    num_examples: 78534289  download_size: 243052074290  dataset_size: 1318726179291- config_name: fm+t292  features:293  - name: id294    dtype: string295  - name: source296    dtype: string297  - name: target298    dtype: string299  splits:300  - name: train301    num_bytes: 868034154302    num_examples: 624022303  - name: test304    num_bytes: 114371187305    num_examples: 78388306  - name: validation307    num_bytes: 112688219308    num_examples: 78534309  download_size: 217267853310  dataset_size: 1095093560311- config_name: fm_indented312  features:313  - name: id314    dtype: string315  - name: text316    dtype: string317  splits:318  - name: train319    num_bytes: 473170158320    num_examples: 624022321  - name: test322    num_bytes: 64280367323    num_examples: 78388324  - name: validation325    num_bytes: 61093848326    num_examples: 78534327  download_size: 103174190328  dataset_size: 598544373329- config_name: t330  features:331  - name: id332    dtype: string333  - name: source334    dtype: string335  - name: target336    dtype: string337  splits:338  - name: train339    num_bytes: 380207303340    num_examples: 624022341  - name: test342    num_bytes: 47993188343    num_examples: 78388344  - name: validation345    num_bytes: 49808813346    num_examples: 78534347  download_size: 113820250348  dataset_size: 478009304349- config_name: t+tc350  features:351  - name: id352    dtype: string353  - name: source354    dtype: string355  - name: target356    dtype: string357  splits:358  - name: train359    num_bytes: 550955294360    num_examples: 624022361  - name: test362    num_bytes: 68323462363    num_examples: 78388364  - name: validation365    num_bytes: 72740770366    num_examples: 78534367  download_size: 136767271368  dataset_size: 692019526369- config_name: t_indented370  features:371  - name: id372    dtype: string373  - name: source374    dtype: string375  - name: target376    dtype: string377  splits:378  - name: train379    num_bytes: 405853738380    num_examples: 624022381  - name: test382    num_bytes: 51457514383    num_examples: 78388384  - name: validation385    num_bytes: 52970428386    num_examples: 78534387  download_size: 117732776388  dataset_size: 510281680389tags:390- unit test391- java392- code393---394 395## Dataset Description396Microsoft created the methods2test dataset, consisting of Java Junit test cases with its corresponding focal methods. 397It contains 780k pairs of JUnit test cases and focal methods which were extracted from a total of 91K398Java open source project hosted on GitHub.399 400This is an assembled version of the methods2test dataset. It provides convenient access to the different context levels based on the raw source code (e.g. newlines are preserved). The test cases and associated classes are also made available.401 402The mapping between test case and focal methods are based heuristics rules and Java developer's best practice.403 404More information could be found here:405- [methods2test Github repo](https://github.com/microsoft/methods2test)406- [Methods2Test: A dataset of focal methods mapped to test cases](https://arxiv.org/pdf/2203.12776.pdf)407 408## Dataset Schema409```410t: <TEST_CASE>411t_tc: <TEST_CASE> <TEST_CLASS_NAME>412fm: <FOCAL_METHOD>413fm_fc: <FOCAL_CLASS_NAME> <FOCAL_METHOD>414fm_fc_c: <FOCAL_CLASS_NAME> <FOCAL_METHOD> <CONTRSUCTORS>415fm_fc_c_m: <FOCAL_CLASS_NAME> <FOCAL_METHOD> <CONTRSUCTORS> <METHOD_SIGNATURES>416fm_fc_c_m_f: <FOCAL_CLASS_NAME> <FOCAL_METHOD> <CONTRSUCTORS> <METHOD_SIGNATURES> <FIELDS>417```418 419## Focal Context420- fm: this representation incorporates exclusively the source421code of the focal method. Intuitively, this contains the most422important information for generating accurate test cases for423the given method.424- fm+fc: this representations adds the focal class name, which425can provide meaningful semantic information to the model.426- fm+fc+c: this representation adds the signatures of the constructor methods of the focal class. The idea behind this427augmentation is that the test case may require instantiating428an object of the focal class in order to properly test the focal429method.430- fm+fc+c+m: this representation adds the signatures of the431other public methods in the focal class. The rationale which432motivated this inclusion is that the test case may need to433invoke other auxiliary methods within the class (e.g., getters,434setters) to set up or tear down the testing environment.435- fm+fc+c+m+f : this representation adds the public fields of436the focal class. The motivation is that test cases may need to437inspect the status of the public fields to properly test a focal438method.439 440![image/png](https://huggingface.co/datasets/fals3/methods2test/resolve/main/figure-1-focal-context.png)441 442 The different levels of focal contexts are the following:443```444T: test case445T_TC: test case + test class name446FM: focal method447FM_FC: focal method + focal class name448FM_FC_C: focal method + focal class name + constructor signatures449FM_FC_C_M: focal method + focal class name + constructor signatures + public method signatures450FM_FC_C_M_F: focal method + focal class name + constructor signatures + public method signatures + public fields451```452 453## Limitations454The original authors validate the heuristics by inspecting a455statistically significant sample (confidence level of 95% within 10%456margin of error) of 97 samples from the training set. Two authors457independently evaluated the sample, then met to discuss the disagreements. We found that 90.72% of the samples have a correct458link between the test case and the corresponding focal method459 460## Contribution461All thanks to the original authors.462