AMFORGE/cofos

Cofos Code Model (v1-logic)

Cofos is a logic- and coding-oriented decoder-only language model built on AMEFORGE's SparseMind architecture. It is specialized for code generation, docstring construction, and code explanation in Python, JavaScript, and Rust.

Developed by AMEFORGE.

Architecture & Parameters

• —Architecture: SparseMind v15 (dynamic neural type management, sparse attention, and sparse FFN)
• —Parameters: 36,824,070
• —Hidden dimension: 384
• —Layers: 8
• —Context length: 512 tokens
• —Tokenizer: Custom BPE Tokenizer (vocab size 16384) with atomic indents/operators/keywords

Training Details

• —Trained on 30,000,000+ procedurally generated programming templates on-the-fly.
• —Masked loss to learn code completion and documentation formatting.
• —Best Python syntax-validity on held-out validation: 8.8%

How to use

import torch
import sentencepiece as spm

# Load checkpoint
ckpt = torch.load("cofos_best.pt", map_location="cpu")
cfg_dict = ckpt["config"]

# Instantiate model architecture
# model = SparseMind(Config(**cfg_dict))
# model.load_state_dict(ckpt["model"])
# model.eval()