JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-Agentic-Tessa-8K-2048-Experimental-LoRA

Qwen3 4B Thinking 2507 Heretic CodeFeedback — Agentic Tessa 8K 2048 Experimental LoRA

This repository contains an experimental LoRA adapter trained on top of:

`JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback`

This adapter is a larger continuation experiment using:

`smirki/Agentic-Coding-Tessa`

This is not recommended as the main version.

Status

This is an experimental adapter kept for transparency, comparison, and future analysis.

In a small local Python coding benchmark, this adapter regressed compared with the previous CodeFeedback checkpoint.

Delta:

Observed behavior

The adapter did not fail completely, but it became worse at strict executable-code output.

Observed regressions:

• —flatten failed with a type error.
• —valid_parentheses failed to output executable code.
• —lru_cache remained incomplete.
• —The model showed more explanatory / agentic behavior instead of always returning compact executable code.

This suggests that the larger Agentic Tessa continuation pushed the model toward a more verbose agentic-assistant style, which may be useful for some workflows but is worse for strict code-output benchmarks.

Training configuration

Training result

Benchmark files

Benchmark artifacts are included under:

~~~text benchmark/ ~~~

Files:

~~~text benchmark/beforesummary.md benchmark/aftersummary.md benchmark/COMPARISON.md benchmark/comparison.json benchmark/beforeresults.jsonl benchmark/afterresults.jsonl ~~~

Intended use

This adapter is intended for:

• —comparison against the previous CodeFeedback checkpoint
• —studying regression from larger agentic fine-tuning
• —analyzing output-style drift
• —future experiments with smaller learning rates or filtered datasets

It is not recommended for strict agentic coding workflows that require compact executable code output.

For the stronger current baseline, prefer:

`JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback`

Loading example

~~~python from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel import torch

base_model = "JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback" adapter = "JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-Agentic-Tessa-8K-2048-Experimental-LoRA"

tokenizer = AutoTokenizer.frompretrained(basemodel, trustremotecode=True)

bnbconfig = BitsAndBytesConfig( loadin4bit=True, bnb4bitquanttype="nf4", bnb4bitcomputedtype=torch.float16, bnb4bitusedouble_quant=True, )

model = AutoModelForCausalLM.frompretrained( basemodel, quantizationconfig=bnbconfig, devicemap="auto", trustremote_code=True, )

model = PeftModel.from_pretrained(model, adapter) model.eval() ~~~

Important notes

This is an experimental LoRA adapter.

It should not be treated as a universal improvement over the previous CodeFeedback model.

The benchmark used here is small and should not be treated as a formal coding leaderboard. It is mainly useful for local before/after regression testing.