patched-codes/generate-readme-eval

Generate README Eval The generate-readme-eval is a dataset (train split) and benchmark (test split) to evaluate the effectiveness of LLMs when summarizing entire GitHub repos in form of a README.md file. The datset is curated from top 400 real Python repositories from GitHub with at least 1000 stars and 100 forks. The script used to generate the dataset can be found here. For the dataset we restrict ourselves to GH repositories that are less than 100k tokens in size to allow us… See the full description on the dataset page: https://huggingface.co/datasets/patched-codes/generate-readme-eval.