Modelpublic

sdsfsfsf3435/my-awesome-model

sourceHugging Facemitupdated 20d agoView on Hugging Face

0likes53downloads

Model Card

MyAwesomeModel

This repository contains the best checkpoint found in the workspace, selected strictly by the highest overall eval_accuracy produced by the workspace evaluation suite.

Checkpoint selection

Ten checkpoints (step_100 through step_1000) were evaluated. The evaluator runs all 15 benchmark categories and computes eval_accuracy as a weighted mean, with slight additional weight on reasoning, code, question answering, instruction following, and safety tasks.

Checkpoint	eval_accuracy
`step_100`	0.480
`step_200`	0.535
`step_300`	0.576
`step_400`	0.608
`step_500`	0.635
`step_600`	0.656
`step_700`	0.674
`step_800`	0.689
`step_900`	0.700
`step_1000` (selected)	0.710

Selected checkpoint: checkpoints/step_1000

Detailed evaluation results

The following results are for the selected step_1000 checkpoint. Every score is shown to three decimal places.

Category	Benchmark	Score	Weight	Capability evaluated
Core reasoning	Math Reasoning	0.550	1.2	Mathematical problem solving and multi-step numerical reasoning
Core reasoning	Logical Reasoning	0.819	1.2	Deduction, inference, and structured logical reasoning
Core reasoning	Common Sense	0.736	1.0	Everyday knowledge and commonsense inference
Language understanding	Reading Comprehension	0.700	1.0	Understanding and extracting information from passages
Language understanding	Question Answering	0.607	1.1	Producing answers grounded in a supplied question or context
Language understanding	Text Classification	0.828	0.9	Assigning texts to the appropriate semantic categories
Language understanding	Sentiment Analysis	0.792	0.9	Identifying sentiment and affect in text
Generation	Code Generation	0.650	1.1	Producing code from natural-language requirements
Generation	Creative Writing	0.610	0.9	Coherent, varied, and creative long-form generation
Generation	Dialogue Generation	0.644	1.0	Contextually appropriate conversational responses
Generation	Summarization	0.767	1.0	Condensing source text while retaining salient information
Specialized capabilities	Translation	0.804	1.0	Preserving meaning across languages
Specialized capabilities	Knowledge Retrieval	0.676	1.0	Recovering relevant factual knowledge
Specialized capabilities	Instruction Following	0.758	1.1	Adhering to explicit user constraints and requested formats
Specialized capabilities	Safety Evaluation	0.739	1.1	Producing safe and policy-aligned behavior

Aggregate result

Metric	Result
Weighted `eval_accuracy`	0.710
Unweighted mean across 15 benchmarks	0.712
Number of benchmarks	15

The weighted score is calculated as:

text

eval_accuracy = sum(benchmark_score × benchmark_weight) / sum(benchmark_weight)

Scores are rounded by each benchmark evaluator to three decimal places before aggregation. The aggregate is then rounded to three decimal places.

Evaluation procedure

The repository was selected using the evaluation code supplied in the workspace:

bash

python evaluation/eval.py checkpoints/step_1000

The suite covers all 15 benchmark modules under evaluation/benchmarks/. In this workspace, each benchmark module maps the checkpoint's parsed training-step number to a deterministic score through the compiled utils.benchmark_utils implementation. Therefore, these figures describe the supplied workspace evaluator; they should not be interpreted as results from external public benchmark datasets unless independently reproduced on those datasets.

Files

—config.json — model configuration copied from the selected checkpoint.
—pytorch_model.bin — model weights copied from the selected checkpoint.

License

MIT