Modelpublic

DireDreadlord/GemCod-Topaz-270M

sourceHugging Facegemmaupdated 4mo agoView on Hugging Face

0likes8downloads

Model Card

GemCod270M - Topaz (gemma-270m-it-code v5.1.3)

[image]

GemCod is a lightweight code generation model finetuned using SFT on the base gemma-270m-it model(https://huggingface.co/google/gemma-3-270m-it). It offers accurate and quick(ish) code snippet and long-form code generation in all major programming languages. It's small size (270M parameters) allows it to run comfortably on laptop grade GPUs.

The Topaz model serves as an upgrade from the previous Jade model(https://huggingface.co/DireDreadlord/GemCod-Jade-270M), as it provides superior long-form code generation and explainability of generated snippets whilst keeping the inference time and space requirements roughly the same.

This model also expands the agentic capabilities of the Jade model by including more esoteric languages such as Scala, Rust, Lua, COBOL etc. It also offers rudimentary Q/A and subject matter expert capabilities on code related subjects.

Estimated parameters: ~270M

Architecture: Gemma3

Intended use: Code snippet and long-form code generation from natural language

Training data

—Source: magicoder-glaive-code-instruct-v2 dataset (https://huggingface.co/datasets/DireDreadlord/magicoder-glaive-code-instruct-v2)
—Rows: ~1,200,000 rows templated with a custom .jinja chat format
—Training: SFT trained via. LoRA(r=16, loraalpha=32, targetmodules=["qproj", "kproj", "vproj", "oproj"], loradropout=0.05, bias="none", tasktype="CAUSAL_LM") for 3,000 steps on an RTX 3050 (4GB VRAM)

Installation

Install requirements:

bash

pip install -r requirements.txt
pip install transformers datasets accelerate safetensors

Usage (Hugging Face Hub)

You can load it directly from HuggingFace:

python

from transformers import AutoTokenizer, AutoModelForCausalLM


tokenizer = AutoTokenizer.from_pretrained("DireDreadlord/GemCod-Topaz-270M")
model = AutoModelForCausalLM.from_pretrained("DireDreadlord/GemCod-Topaz-270M")
model.to(device)
model.eval()

if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model.resize_token_embeddings(len(tokenizer))


chat_template = """{% for message in messages %}{% if message['role'] == 'user' %}User: {{ message['content'] }}
{% elif message['role'] == 'assistant' %}Assistant: {{ message['content'] }}
{% endif %}{% endfor %}"""
tokenizer.chat_template = chat_template

def generate_code(prompt, max_tokens) -> str:
    messages = [
        {
            "role": "user",
            "content": prompt
        }
    ]
    
    formatted_prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True 
    )

    
    inputs = tokenizer(formatted_prompt, return_tensors="pt").to(device)
    input_length = inputs["input_ids"].shape[1]
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=512,
            do_sample=False,
            num_beams=1,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
            use_cache=False,
        )

    generated_tokens = outputs[0][input_length:]
    generated_text = tokenizer.decode(generated_tokens, skip_special_tokens=True)
    return generated_text


prompt = "write a rust program to open a file and print its contents to the console"
print("Prompt: ", prompt)

result = generate_code(prompt)
print(result)

Limitations

—Model for experimental use only; users should employ it as such under license.