FWKTechnologies/CodeMind-Code-Intelligence

Model Card for CodeMind Model 1

Release scope: This model card documents CodeMind Model 1, the first public architecture of the CodeMind project. Details of later architectures are intentionally excluded from this document. Evidence policy: This document distinguishes implemented behavior, design/configuration values, and measured results. Where the available source material does not provide a verified measurement, the field is explicitly marked as not available, not yet measured, or not sufficiently tested rather than being estimated as a benchmark result.

Model Details

Model Description

CodeMind Model 1 is a custom code-intelligence system built primarily in Python and PyTorch around a Heterogeneous Graph Transformer (HGT) operating on a Code Property Graph (CPG). The CPG represents multiple program relations rather than treating source code as a plain text sequence. The implementation combines structural program analysis, graph representation learning, dual-brain processing, custom routing, graph-aware generation, validation, and a policy-optimization component into one training/inference pipeline.

The central design principle is that source code is represented as a structured graph containing multiple kinds of program relationships. In the Model 1 implementation, the graph is derived from structures including AST, CFG, DFG, PDG, Call Graph, and SDG. The graph can also be merged across multiple source files and languages, with explicit cross-language relations for supported project structures.

The model is not described here as a conventional text-only language model, code completion model, or simple ensemble. Its primary representation is graph-based, and its generation path uses a graph-action representation rather than relying solely on plain text token generation.

The architecture contains two primary graph-processing brains:

• —Brain A — understanding: deeper structural representation learning intended to model the code/program state.
• —Brain B — editing: a second graph-processing stage used for editing/generation-oriented transformation while preserving information from Brain A.

The two brains are connected through an AntiForgetBridge, which combines their representations and includes a preservation signal intended to reduce destructive drift from the knowledge carried by Brain A.

A FlyPromptRouter provides task-conditioned sparse routing across expert modules. The router uses top-k expert selection, routing noise during training, temporal expert behavior, and auxiliary balancing terms. The implementation should therefore be understood as a custom routing system, not assumed to be identical to a textbook MoE implementation.

A custom Curiosity PPO / PPO-style policy optimization subsystem supplies additional learning signals based on correctness, curiosity, calibration, honesty/abstention behavior, integrity, safety, logic evidence, and bridge retention. The project code uses the PPO name, but the surrounding reward construction and integration are specific to CodeMind and should not be interpreted as an unmodified reference implementation of PPO.

Model Summary

Architecture Overview

The following diagrams are intentionally included as actual rendered graphs. GitHub can render Mermaid diagrams directly in Markdown.

High-level model graph

flowchart LR
    I[Source Code / Project Files]
    P[Multi-language Parser]
    G[Unified Code Property Graph]

    A1[AST]
    A2[CFG]
    A3[DFG]
    A4[PDG]
    A5[Call Graph]
    A6[SDG]

    T[Graph Tensorizer]
    BA[Brain A\nUnderstand\n8 HGT layers]
    R[FlyPromptRouter\n6 experts / top-k 2]
    BB[Brain B\nEdit\n7 HGT layers]
    BR[AntiForgetBridge\nFusion + retention state]
    S[Semantic Representation]
    D[GraphAction Decoder\n9 layers / 16 heads]
    V[Validator / Static Analysis]
    PPO[Custom Curiosity PPO\nPolicy + Value + Reward]
    O[Generated / Edited Code]

    I --> P --> G
    G --> A1
    G --> A2
    G --> A3
    G --> A4
    G --> A5
    G --> A6
    G --> T --> BA
    BA --> R --> BB
    BA --> BR
    BB --> BR
    BR --> S --> D --> O
    O --> V
    V --> PPO
    BR --> PPO
    PPO --> R

Program graph structure

graph TD
    F[Function]
    V1[Variable]
    C[Call]
    E[Expression]
    B[Basic Block]
    S[Statement]
    M[Module / File]

    F -->|AST| S
    S -->|AST| E
    E -->|DFG| V1
    F -->|CFG| B
    B -->|CFG| B
    C -->|Call Graph| F
    M -->|Contains| F
    S -->|PDG| S
    F -->|SDG| F

Dual-brain coupling graph

graph LR
    A[Brain A\nUnderstanding State]
    B[Brain B\nEditing State]
    R[FlyPrompt Router]
    X[AntiForgetBridge]
    E[EWC-lite Anchor / Fisher State]
    Q[Bridge Retention Reward]
    P[PPO Policy Optimization]

    A --> R
    R --> B
    A --> X
    B --> X
    E --> X
    X --> Q
    Q --> P
    P --> R

This last graph is important for interpreting the architecture correctly: Model 1 is not simply “Model A + Model B = averaged output.” The components interact through representations, routing, preservation state, validation results, and reward signals.

Model Sources

• —Repository: ./ (this repository)
• —Primary implementation: CodeMind.py
• —Dual-brain implementation: codemind_brain_dual.py
• —Paper: Not published at the time of this model card
• —Demo: Not provided in the available release materials
• —License: LICENSE — CodeMind Code Intelligence License, CM-CIL-1.0

Uses

Direct Use

Model 1 is intended for code-intelligence workflows where source code structure matters in addition to surface text. Supported intended uses include:

• —source-code understanding and structural analysis;
• —project/file graph construction and graph-aware inspection;
• —static validation and issue analysis;
• —code generation and code editing through the graph-action decoder;
• —code-quality analysis and validation-assisted generation;
• —multi-language project analysis where the parser and graph linker support the language involved;
• —research into graph-native or tightly coupled code-intelligence architectures.

Downstream Use

Model 1 can be integrated into larger systems such as:

• —IDE and developer-tooling assistants;
• —repository analysis and code-review systems;
• —automated refactoring or code-editing pipelines;
• —internal software-engineering agents;
• —research systems that use graph representations as an intermediate program state.

Any downstream system should run its own validation, sandboxing, and security checks rather than treating model output as executable truth.

Out-of-Scope Use

Model 1 is not presented as a general-purpose safety-certified autonomous coding system. In particular, the available materials do not provide sufficient validation for:

• —unattended deployment of generated code to production;
• —high-assurance safety-critical software;
• —security-critical systems without independent review and testing;
• —domains where an incorrect program can directly cause physical, financial, or safety-critical harm;
• —claims of general conversational intelligence comparable to a general-purpose frontier language model.

Bias, Risks, and Limitations

Technical Limitations

1. Evaluation is incomplete.

The provided source material contains implementation and validation machinery but does not include a complete, independently reproduced benchmark suite with published accuracy numbers. Therefore this model card does not claim state-of-the-art performance, benchmark superiority, or a validated percentage of correct generations.

2. Parameter count is currently documented as a design estimate.

The Model 1 configuration comments estimate approximately 3.29 billion parameters. This is not an arbitrary round number: the source itself contains real runtime parameter-counting logic (sum(p.numel() for p in ...parameters()), aggregated across the decoder, encoder, and both brains), and the release-packaging step (_release_readme / _release_model_card) writes an authoritative, measured parameter count into the release metadata from that logic — by design, with no hardcoded figures. What the source comments themselves note, in the exact environment the code was last edited in, is that no GPU/PyTorch runtime was available to actually execute that count at the time of writing — so ~3.29B is the design-time estimate carried in the comments, not a number pulled from a completed run. The exact release parameter count should be copied from the release metadata generated by an actual release run before publishing a final immutable model artifact.

3. Graph size is bounded.

Model 1 uses a graph capacity of 4,096 nodes. Larger graphs may need collapsing or trimming, which is an explicit trade-off between memory/compute and retained structural detail.

4. Graph parsing and linking are imperfect.

The implementation supports multi-language graph construction and cross-language linking, but the code explicitly notes that route matching and related linking logic are not full semantic/type-aware program reasoning. A graph edge should therefore not automatically be interpreted as proof of semantic equivalence.

5. Decoder context is not equivalent to guaranteed quality at every position.

Model 1 exposes a 32,768-position decoder capacity while the main training length is configured at 20,480. The implementation includes long-context sampling and position-offset augmentation, but the source comments explicitly note that this does not constitute an architectural guarantee of equal-quality performance at every position up to the maximum context.

6. Custom PPO does not imply standard PPO behavior.

The system uses a PPO-based policy-optimization component with custom reward decomposition and bridge-retention signals. It should be treated as a CodeMind-specific implementation rather than as a claim that every behavior matches a canonical PPO reference implementation.

7. Custom routing does not imply canonical MoE behavior.

The FlyPromptRouter uses sparse top-k expert routing, but its routing, temporal expert behavior, task conditioning, and auxiliary balancing logic are implementation-specific.

8. Validation is not a universal proof of correctness.

The validator can inspect syntax, static issues, integrity/purity indicators, and—when reference data and execution are available—behavioral or graph-logic evidence. None of these checks alone proves correctness for arbitrary software.

9. The contrastive loss term required a real, documented fix.

The unified loss (see Training Procedure, below) trains on one (source, target) graph pair per step rather than a batch. An earlier version of the contrastive term ran InfoNCE cross-entropy on a 1×1 similarity matrix — one query, one positive, zero negatives — which makes the softmax (and its gradient) identically 1.0 for every input, every step. Since w_contrastive = 0.30 is the single largest term in the unified loss, this meant close to a third of the intended training signal was structurally inert regardless of data or hyperparameters. The fix, present in the current source, keeps a small FIFO bank of recent target embeddings (detached, CPU-resident) as in-batch negatives so the similarity row has real alternatives to discriminate against. This is disclosed here because it directly affects how any checkpoint trained before this fix should be interpreted — the contrastive objective could not have been contributing a real gradient before it.

Security and Misuse Risks

Generated or edited code can contain defects, unsafe operations, incorrect assumptions, or dependency-related problems. Outputs should be treated as untrusted until reviewed, tested, and executed in an appropriate sandbox.

The project also includes protective mechanisms around validation and code integrity, but the available source material does not establish a security guarantee against all adversarial prompts, hostile repositories, poisoned inputs, or novel attack techniques.

Recommendations

Users should:

• —validate generated code before use;
• —run generated code in a sandbox during development and evaluation;
• —use unit tests, integration tests, static analysis, and human review for important code;
• —record the model version and checkpoint hash when comparing results;
• —avoid interpreting internal confidence or reward signals as proof of correctness;
• —report failures using reproducible source/target examples whenever possible.

How to Get Started with the Model

The implementation is designed to be used directly from Python.

from CodeMind import CodeMind

brain = CodeMind()
brain.connect_data()

# Train
report = brain.train(epochs=2)

# Understand code
result = brain.understand(source_code)

# Evaluate code quality / validation through the model's built-in tooling
score = brain.quality_score(source_code)

print(report)
print(result)
print(score)

For generation/editing workflows, use the release API and generation functions provided by the exact repository version. The public interface can change between releases, so users should pin the repository commit or release tag.

Training Details

Training Data

Model 1 is trained for code-intelligence tasks over structured source-code examples and associated graph/validation signals. The provided configuration contains a planning estimate of roughly 90 GB of code text for the large training setup. The repository material available for this model card does not include a final, independently audited dataset manifest with an exact measured byte count, exact token count, complete provenance list, or complete license-by-license accounting.

Therefore:

• —Exact final dataset size: not available in the supplied release materials.
• —Exact token count: not available in the supplied release materials.
• —Complete source-by-source provenance: not available in the supplied release materials.
• —Complete deduplication report: not available in the supplied release materials.
• —Final train/validation/test corpus manifest: not available in the supplied release materials.

A dataset card should be published alongside the model once those records are finalized.

Training Procedure

Model 1 training combines graph representation learning, semantic alignment, validation-related objectives, graph regularization, and a PPO-based policy term in a unified training pipeline.

The source implementation defines a unified loss with these Model 1 weights, confirmed directly against CodeMindLoss.__init__:

The implementation also contains a correction for the degenerate one-positive InfoNCE case by maintaining a FIFO negative bank so the contrastive term has actual alternatives rather than a mathematically trivial 1×1 comparison (see Technical Limitations, item 9, for why this fix mattered).

Preprocessing

The preprocessing pipeline converts source files into graph structures and associated validation information. Relevant implementation stages include:

1. 1.language detection and parsing;
2. 2.construction of program-structure nodes and edges;
3. 3.optional project-level graph merging;
4. 4.graph-to-tensor conversion for HGT processing;
5. 5.target graph preparation and validation;
6. 6.caching/prefetching to reduce repeated CPU work during training.

The project includes multiprocessing-based parsing/validation paths because parsing and validation are CPU-heavy Python operations and can otherwise leave the accelerator idle.

Training Hyperparameters

• —Training regime: BF16 autocast on supported CUDA hardware, with FP32 fallback where BF16/CUDA is unavailable.
• —Optimizer: AdamW.
• —Base learning rate: 2e-4 for the main training configuration.
• —Weight decay: 0.01 (with parameter grouping in the current implementation).
• —Batch size: 128 configured batch size.
• —Gradient accumulation: 8.
• —Nominal effective batch: 1024 before OOM/backoff adjustments.
• —Validation split: 10% configured validation ratio.
• —Quality target: 85.0 in the training configuration.
• —Gradient checkpointing: enabled in the training-memory configuration.
• —TF32: enabled on supported NVIDIA CUDA paths.

These values are configuration values, not empirical claims about the optimal training regime.

Custom PPO Configuration

The Model 1 PPO component is configured with the following defaults in the supplied implementation, confirmed directly against PPOAgent.__init__:

The optimizer backing the PPO policy is AdamW with betas=(0.9, 0.95) (rather than the library-default 0.999) and a cosine-annealing-with-warm-restarts LR schedule keyed to the warmup/floor values above — a detail chosen for RL convergence stability, not carried over unmodified from a default AdamW config.

The reward is decomposed rather than being a single scalar generated from one check. The Model 1 reward implementation includes terms for curiosity, correctness, calibration, honesty/abstention, integrity, and safety, with additional logic-sensitive and bridge-retention behavior in the broader pipeline.

Speeds, Sizes, Times

The source configuration targets an NVIDIA H100 SXM 80 GB system with approximately 20 vCPU, 125 GB RAM, and 50 GB SSD cache.

The supplied source material does not contain a reproducible end-to-end training-time measurement for a final public release. Therefore:

• —Training hours: not measured in the supplied materials.
• —Examples/second: not measured in the supplied materials.
• —Tokens/second: not measured in the supplied materials.
• —Final checkpoint size: not measured from a final immutable release artifact in the supplied materials. The source's own checkpoint-size estimator uses ~12 bytes per parameter (FP32 weights plus FP32 Adam first/second moments), which — applied to the ~3.29B design estimate above, and only as an order-of-magnitude planning figure rather than a measurement — works out to roughly 39 GB for a full training checkpoint. This is the training-checkpoint estimate, not the packaged release size, which strips optimizer state and can down-cast weights (see the repository's release/packaging documentation for that distinct number).
• —Inference latency: not measured in a standardized benchmark.

These should be added after a fixed hardware/software benchmark run.

Evaluation

Testing Data, Factors & Metrics

Testing Data

A complete independent test-set manifest is not included in the available release materials. The training code supports validation splitting and reference-based code validation, but this model card does not claim a finalized public benchmark set.

Status: Not sufficiently documented for a reproducible public benchmark.

Factors

The following evaluation factors are directly relevant to the architecture and should be reported in the final benchmark report:

• —programming language;
• —code length;
• —graph node count;
• —project vs. single-file input;
• —presence of cross-file calls;
• —presence of cross-language links;
• —code generation vs. code understanding;
• —syntax validity vs. behavioral correctness;
• —graph-structural similarity;
• —validation/purity/integrity outcomes.

The source material contains machinery for several of these dimensions, but does not provide a completed statistical breakdown.

Metrics

Recommended metrics supported by the implementation or directly aligned with its outputs include:

Results

No verified benchmark results are claimed in this model card.

This is intentional. The supplied source code demonstrates that evaluation and validation mechanisms exist, but it does not provide enough reproducible test output to justify publishing numerical performance claims.

Summary

At the time of publication of this document, the correct status is:

Architecture documented; implementation substantially specified; independent benchmark results not yet sufficiently available for publication.

Model Examination

Structural Examination

The architecture has been reviewed at the source-code level to identify major subsystems and their explicit interfaces. The main components are:

flowchart TB
    P[Parser / Graph Builder]
    C[CPG]
    HGA[HGT Brain A]
    HGB[HGT Brain B]
    R[FlyPromptRouter]
    B[AntiForgetBridge]
    F[Semantic Fusion]
    G[GraphActionCodec]
    D[Decoder]
    V[Validator]
    L[Unified Loss]
    PPO[Custom PPO]

    P --> C --> HGA
    HGA --> R --> HGB
    HGA --> B
    HGB --> B
    B --> F --> D
    G --> D
    D --> V
    V --> L
    L --> PPO
    PPO --> R
    PPO --> L

What Has Been Examined

The source material supports the following concrete observations:

• —the primary graph representation is a heterogeneous CPG;
• —two HGT-based brains are explicitly constructed in the dual-brain path;
• —Brain A and Brain B have different depths in Model 1;
• —the bridge contains learned projections, gating, normalization, retention state, and an EWC-lite penalty mechanism;
• —the router uses sparse top-k expert routing and auxiliary balancing logic;
• —the decoder operates over graph-action labels rather than a plain token-only output space;
• —validation can optionally compare generated code against reference code and/or a reference graph;
• —training contains a unified multi-objective loss and a PPO-based policy term;
• —the unified loss's contrastive term required a real fix (a FIFO negative bank) to produce a non-zero gradient at all, given the one-pair-per-step training setup.

What Has Not Been Established

The source material does not provide enough evidence to claim:

• —mechanistic interpretability of individual attention heads;
• —causal attribution of a particular model output to a particular expert;
• —proof that the bridge prevents catastrophic forgetting in all settings;
• —proof that the graph representation always improves over a text-only baseline;
• —proof that the custom PPO subsystem improves generalization over ablations;
• —proof that routing diversity improves downstream quality;
• —proof of superior performance relative to existing code models.

Those claims require controlled experiments and ablation studies.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator introduced by Lacoste et al. (2019).

• —Hardware Type: NVIDIA H100 SXM 80 GB (training target)
• —Hours used: Not available
• —Cloud Provider: RunPod is the configured target environment in the source documentation
• —Compute Region: Not available
• —Carbon Emitted: Not available

No carbon figure is reported here because the available materials do not contain the actual training duration, measured accelerator power draw, region, or energy-mix information required for a defensible estimate.

Technical Specifications

Model Architecture and Objective

Model 1 is built from the following main layers of computation:

1. 1.Parsing: source code is parsed into language-aware graph structures.
2. 2.CPG construction: multiple program relations are represented as heterogeneous graph edges.
3. 3.Graph encoding: graph nodes and relations are converted into HGT inputs.
4. 4.Brain A: an 8-layer HGT processes the program graph for understanding.
5. 5.Routing: FlyPromptRouter selects a sparse subset of 6 experts, using top-k=2 routing in Model 1.
6. 6.Brain B: a 7-layer HGT receives routed information for editing/transformation-oriented processing.
7. 7.Bridge: AntiForgetBridge fuses A/B representations and tracks a preservation-oriented state.
8. 8.Generation: a 9-layer decoder produces graph-action sequences from the semantic representation.
9. 9.Validation: generated candidates can be statically analyzed and, when a reference is available, behaviorally and/or structurally compared.
10. 10.Policy optimization: a custom PPO subsystem uses reward signals from model quality and related system-level criteria.

Conceptually:

L_total =
    0.30 * L_contrastive
  + 0.25 * L_semantic
  + 0.20 * L_validate
  + 0.10 * L_graph
  + 0.15 * L_ppo

The expression above describes the Model 1 configured unified loss. It is a training objective, not a claim that each term contributes equally in practice or that the chosen weights are optimal.

Compute Infrastructure

The Model 1 configuration was sized for:

• —GPU: NVIDIA H100 SXM, 80 GB HBM
• —CPU: 20 vCPU
• —RAM: 125 GB
• —SSD cache: 50 GB
• —Training precision: BF16 autocast on supported CUDA devices
• —Optimizer: AdamW, with a fused CUDA path when available
• —Gradient checkpointing: enabled
• —CUDA acceleration: TF32 enabled where supported

Hardware

The repository also retains CPU-only and reduced-hardware fallback paths. However, the Model 1 architecture and its long-context configuration were primarily sized for accelerator-backed training rather than for consumer-grade hardware.

Software

• —Python
• —PyTorch
• —PyTorch CUDA acceleration when available
• —Project-specific graph, parsing, validation, routing, bridge, and training modules

The architecture is implemented primarily in project-owned Python/PyTorch code. External libraries are treated as implementation dependencies or extensions; they do not define the full CodeMind architecture.

Reproducibility and Release Notes

For reproducible evaluation, users should record:

• —repository commit or release tag;
• —model checkpoint hash;
• —Python version;
• —PyTorch version;
• —CUDA version and GPU model;
• —dataset version/hash;
• —configuration file;
• —random seeds;
• —exact evaluation scripts;
• —whether reference-code execution was enabled.

The release should not be compared directly across commits without checking configuration compatibility.

Citation

BibTeX

No paper has been published for CodeMind Model 1 at the time of this model card.

@software{codemind_model_1,
  title        = {CodeMind Model 1},
  author       = {CodeMind Project},
  year         = {2026},
  version      = {1},
  note         = {Graph-based code intelligence system built with Python and PyTorch}
}

APA

CodeMind Project. (2026). CodeMind Model 1 (Version 1) [Computer software].

Glossary

More Information

The most useful additional documentation to publish alongside this model card is:

1. 1.Dataset Card — provenance, licenses, language/domain distribution, deduplication, and exact token counts.
2. 2.Benchmark Report — fixed test split, metrics, baselines, ablations, confidence intervals, and hardware details.
3. 3.Architecture Notes — deeper explanation of the graph pipeline, dual-brain interaction, routing behavior, and decoder action representation.
4. 4.Release Manifest — exact runtime parameter count, checkpoint hashes, software versions, and configuration.

Public Disclosure Boundary

This document intentionally focuses on Model 1. Details of later CodeMind architectures are outside the scope of this release and are not used as evidence for Model 1's capabilities or results.

Model Card Authors

CodeMind Project

This model card was prepared from the Model 1 implementation and configuration available in the repository. Unverified performance claims have been intentionally omitted.

Model Card Contact

Use the repository's issue tracker or project contact channel for questions, bug reports, benchmark contributions, and reproducibility reports.