Shuu12121/NightJar-CodeSearch-Embedding

NightJar-CodeSearch-Embedding

NightJar-CodeSearch-Embedding is a 768-dimensional dense embedding model for natural-language-to-code retrieval and code-edit retrieval. It is based on Shuu12121/NightJar and fine-tuned with hard negatives and large in-batch negatives.

The model embeds a text query and a code document into the same vector space. Higher cosine similarity indicates a stronger match. It does not require query or document prefixes.

Model details

Usage

Sentence Transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Shuu12121/NightJar-CodeSearch-Embedding")

query = "parse a JSON string and return an error when it is invalid"
code = [
    "def parse_json(text):\n    return json.loads(text)",
    "def read_lines(path):\n    return Path(path).read_text().splitlines()",
]

query_embedding = model.encode(query, normalize_embeddings=True)
code_embeddings = model.encode(code, normalize_embeddings=True)
scores = code_embeddings @ query_embedding

for score, snippet in sorted(zip(scores, code), reverse=True):
    print(float(score), snippet)

No query:, passage:, or task-specific prefix is needed. Use the same model and encoding settings for queries and code. Normalized embeddings make the dot product equivalent to cosine similarity.

Transformers

Loading through SentenceTransformer is recommended because the repository's pooling configuration is part of the model. If the backbone is loaded directly with Transformers, use the final hidden state of the CLS token and L2-normalize the resulting vector.

Intended uses

• —Semantic code search from natural-language queries
• —Retrieval of code relevant to a requested edit
• —Candidate generation for reranking or retrieval-augmented generation
• —Code clustering, deduplication, and nearest-neighbor exploration

The model is a retriever, not a code generator or correctness verifier. A high similarity score does not guarantee that code is safe, correct, or appropriate for execution.

Supported programming languages

The code-search training mixture includes:

Bash, C, C++, C#, Dart, Go, Java, JavaScript, Kotlin, Lua, PHP, Python, Ruby, Rust, Scala, Swift, and TypeScript.

Performance may transfer to related languages, but languages not represented in training have not been systematically validated.

Training

The model was fine-tuned on 4,073,472 examples using knowledge-distilled hard negative datasets for code search and code-edit retrieval. Each example contained one positive and up to 15 explicit hard negatives; other positives in the 1,024-example logical batch also served as in-batch negatives.

The three training groups were sampled at a 1:8:8 batch ratio:

Each 17-batch weighted cycle was shuffled deterministically, rather than always presenting the groups in a fixed order. Languages within each group were balanced, and rows within each batch were shuffled.

Training data and decontamination

The training mixture uses the same decontaminated retrieval datasets as NightOwl-CodeEmbedding: code-search and code-comment pairs, together with commitpackft-derived code-edit pairs. All examples were constructed with one positive and 15 hard negatives mined by Qwen/Qwen3-Embedding-0.6B.

Before training, overlaps were removed between:

• —The code-search/code-comment data and the CodeSearchNet test splits
• —The commitpackft-derived code-edit data and the CodeEditSearchRetrieval benchmark evaluation examples

Although MTEB names the CodeEditSearchRetrieval evaluation split train, those evaluated examples were not included in this model's fine-tuning data.

Main hyperparameters

Evaluation

MTEB CodeSearchNetRetrieval

The model was evaluated with MTEB 2.5.1 on the CodeSearchNetRetrieval test sets. The benchmark's main_score is nDCG@10. The reported average is an unweighted macro average over all six languages.

MTEB CodeEditSearchRetrieval

Code-edit retrieval was evaluated with MTEB 2.5.1. The task's main_score is nDCG@10. The macro average across the 13 language subsets is 0.7382.

CodeEditSearchRetrieval does not provide a standard test split in MTEB, so its official train split is used for evaluation. The evaluated examples were removed from the fine-tuning data and were not used to train this model. The score therefore measures in-domain retrieval on held-out benchmark examples; it is not training-set performance or a strictly zero-shot result.

In-training validation

The following results use cosine retrieval on 1,000 validation examples per CodeSearchNet language. Each query has one relevant document, so Accuracy@k and Recall@k are equivalent in this setup. The macro average is an unweighted mean over the six evaluated languages.

These numbers come from an in-training validation setup and should not be treated as directly comparable to results produced with a different corpus, candidate pool, preprocessing pipeline, or benchmark implementation.

License

The model weights and original code in this repository are released under the Apache License 2.0.

The author's code-search training dataset was constructed from repositories licensed under MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, Unlicense, CC0-1.0, and ISC.

The Apache-2.0 license for this model does not supersede any third-party rights or license obligations that may apply to source materials used during training. Users are responsible for complying with applicable licenses when redistributing or reusing original source code obtained independently of the model.

Limitations

• —Evaluation above covers six CodeSearchNet languages, not every training language.
• —Inputs longer than 1,024 tokens are truncated.
• —The training data may contain public-code biases, duplicated patterns, or insecure implementations inherited from its sources.