cabbage972/GitChameleon-2.0

GitChameleon 2.0 GitChameleon 2.0 is an AI coding benchmark comprising 328 Python-based problems conditioned on specific versions of popular libraries for scientific computing and web development. It evaluates whether AI code generation models can correctly use library APIs as they existed at a particular version — a challenging test of version-specific knowledge. Note: This is GitChameleon 2.0, a distinct and newer work from the original GitChameleon benchmark. Please do not… See the full description on the dataset page: https://huggingface.co/datasets/cabbage972/GitChameleon-2.0.