CoolFace
Datasetpublic

dhruveshpatel/classeval-structured-v1

ClassEval Structured v1 Derived from FudanSELab/ClassEval (Du et al. 2023, arXiv:2308.01861). License: CC BY-NC 4.0 (upstream data license). Non-commercial use only. One row per (task_id, variant) with variant in {1,2,3} (100 tasks × 3 = 300 rows; Hub split test). solution_code, test, and methods_info_json come from upstream. We add rendered prompts/targets and stratification fields. Missing bodies use .... Variants: Signatures and docstrings kept; every method body is ....… See the full description on the dataset page: https://huggingface.co/datasets/dhruveshpatel/classeval-structured-v1.

sourceHugging Facecc-by-nc-4.0updated 2mo agoView on Hugging Face
0likes25downloads
Dataset Card

ClassEval Structured v1

Derived from FudanSELab/ClassEval (Du et al. 2023, arXiv:2308.01861). License: CC BY-NC 4.0 (upstream data license). Non-commercial use only.

One row per (task_id, variant) with variant in {1,2,3} (100 tasks × 3 = 300 rows; Hub split test). solution_code, test, and methods_info_json come from upstream. We add rendered prompts/targets and stratification fields. Missing bodies use ....

Variants:

  1. 1.Signatures and docstrings kept; every method body is ....
  2. 2.Same anchors; some methods keep a gold body prefix, rest is ....
  3. 3.Required methods listed in the module docstring; prompt ends at class ClassName: (no pass, no def). target_text is the indented class body.

Fields

FieldMeaning
task_idClassEval task id (e.g. ClassEval_0).
variant1, 2, or 3 (above).
instructionExternal instruction for this variant.
prompt_textPrompt file. Variants 1–2: full file with ... holes. Variant 3: docstring + imports + class ClassName:.
target_textEmpty for Variants 1–2. Variant 3: indented class body; prompt_text + target_text is a full file.
solution_codeGold full-class implementation.
testClass-level unittest source (Pass@1).
methods_info_jsonJSON list of per-method objects (see below).
class_nameTarget class name.
num_functionsMethod count.
num_cross_depsMethod/field dependency edge count.
difficulty_splitparallel, mixed, or sequential.
nondeterminism_flagsSoft flags (e.g. datetime.now); tasks are kept.
prompt_parse_okprompt_text parsed during build.
prompt_parse_errorError string if not prompt_parse_ok; else empty.
sourceAlways FudanSELab/ClassEval.
source_idUpstream id (same as task_id).

methods_info_json elements:

SubfieldMeaning
method_nameShort name (e.g. filter).
method_descriptiondef name(...): plus docstring.
test_classPer-method unittest class name.
test_codePer-method unittest source.
solution_codeGold body for this method.
dependenciesStandalone, lib_dependencies, field_dependencies, method_dependencies.