ethicalabs/Kurtis-E1.1-Qwen2.5-3B-Instruct

Model Card for Kurtis-E1.1-Qwen2.5-3B-Instruct

[!WARNING] This repository contains experimental models designed strictly for academic evaluation and research purposes. Critical Constraints: No Production Deployment: Experimental models must not be deployed in commercial, enterprise, or mission-critical environments under any circumstances. No Liability: Experimental models are provided "as-is" without warranties of any kind. The developers assume zero liability for downstream consequences, system integration failures, or regulatory non-compliance resulting from unauthorized deployment.

Kurtis E1.1 fine-tuned with flower

Eval Results

Evaluation tasks were performed with the LM Evaluation Harness on an NVIDIA A40.

hellaswag

lm_eval --model hf --model_args pretrained=ethicalabs/Kurtis-E1.1-Qwen2.5-3B-Instruct  --tasks hellaswag --device cuda:0 --batch_size 8

arc_easy

lm_eval --model hf --model_args pretrained=ethicalabs/Kurtis-E1.1-Qwen2.5-3B-Instruct  --tasks arc_easy --device cuda:0 --batch_size 8

arc_challenge

lm_eval --model hf --model_args pretrained=ethicalabs/Kurtis-E1.1-Qwen2.5-3B-Instruct  --tasks arc_challenge --device cuda:0 --batch_size 8

mmlu

lm_eval --model hf --model_args pretrained=ethicalabs/Kurtis-E1.1-Qwen2.5-3B-Instruct  --tasks mmlu --device cuda:0 --batch_size 8

mmlu (5-shot)

lm_eval   --model hf   --model_args pretrained=ethicalabs/Kurtis-E1.1-Qwen2.5-3B-Instruct   --tasks mmlu   --device cuda:0   --batch_size 8 --num_fewshot 5