Shuu12121/NightOwl-CodeEmbedding
1410
1---2tags:3- sentence-transformers4- feature-extraction5- code-search6- code-embedding7- retrieval8- modernbert9- dense10base_model: Shuu12121/NightOwl11pipeline_tag: feature-extraction12library_name: sentence-transformers13license: apache-2.014datasets:15- Shuu12121/coir_hard_negative_datasets_v3_kd16- Shuu12121/owl_code_search_hard_negative_datasets_V2_kd17- Shuu12121/codeedit_hard_negative_datasets_kd18---19 20# NightOwl-CodeEmbedding 🦉21 22`NightOwl-CodeEmbedding` is a compact, 768-dimensional dense embedding model built for23code retrieval, code-edit retrieval, and technical question answering.24 25It is fine-tuned from [`Shuu12121/NightOwl`](https://huggingface.co/Shuu12121/NightOwl),26a ModernBERT-based code encoder. The model uses CLS pooling with cosine similarity and27does **not** require `query:` / `passage:` style prefixes.28 29## Highlights30 31* Compact (150.8M parameters) yet competitive on CoIR-style code retrieval benchmarks32* Ranks **18th out of 241 models overall** on the33 [MTEB(Code, v1)](https://huggingface.co/spaces/mteb/leaderboard) leaderboard, and is34 the **top-scoring dense single-vector model under 300M parameters** among scored35 entries — ahead of many models an order of magnitude larger36 (see [Leaderboard Standing](#leaderboard-standing))37* Standard single-vector retrieval: one 768-d vector per document and plain38 dot-product / cosine search, so it drops straight into any existing vector-database39 stack40* Covers **eight programming languages**: the six CodeSearchNet languages plus Rust41 and TypeScript42* Handles a broad range of retrieval scenarios: NL-to-code search, code-to-code43 retrieval, **code-edit retrieval**, and technical QA44* Trained with hard negatives mined by `Qwen/Qwen3-Embedding-0.6B`45 (15 hard negatives per anchor)46* Decontaminated against the CodeSearchNet test splits and the47 CodeEditSearchRetrieval benchmark (see [Data Decontamination](#data-decontamination))48* Drop-in compatible with `sentence-transformers`; Apache-2.0 licensed49 50## Supported Languages51 52* Go, Java, JavaScript, PHP, Python, Ruby (CodeSearchNet languages)53* **Rust, TypeScript** (additional)54 55Performance on languages outside this set is untested and may vary.56 57## Usage58 59```python60from sentence_transformers import SentenceTransformer61 62model = SentenceTransformer("Shuu12121/NightOwl-CodeEmbedding")63 64queries = ["Python function that sorts a list in descending order"]65documents = [66 "def sort_desc(values): return sorted(values, reverse=True)",67 "def average(values): return sum(values) / len(values)",68]69 70query_embeddings = model.encode(queries)71document_embeddings = model.encode(documents)72 73# Cosine similarity (embeddings are normalized internally by similarity())74scores = model.similarity(query_embeddings, document_embeddings)75print(scores)76```77 78## Model Details79 80| Property | Value |81| ----------------------- | -------------------- |82| Base model | `Shuu12121/NightOwl` |83| Architecture | ModernBERT |84| Parameters | 150,779,136 |85| Embedding dimension | 768 |86| Pooling | CLS pooling |87| Maximum sequence length | 1,024 tokens |88| Similarity | Cosine similarity |89| Query/document prefixes | Not required |90| Weight dtype | FP32 |91| Weight memory | 575 MiB |92| License | Apache-2.0 |93 94## MTEB Results95 96The model was evaluated with MTEB on code-related retrieval and technical QA tasks.97 98Evaluation setup:99 100* Model revision: `c7c8a57b9539297e192d5cf39b9aecf1fb376edd`101* MTEB version: `2.15.1`102* Metric: `NDCG@10`103* Hardware: NVIDIA GeForce RTX 5090104* Batch size: 64105 106Multi-subset task scores are reported as macro averages.107 108| Task | Split | NDCG@10 |109| -------------------------------- | ------: | ----------: |110| AppsRetrieval | test | 0.39177 |111| COIRCodeSearchNetRetrieval | test | 0.84264 |112| CodeEditSearchRetrieval | train¹ | 0.74808 |113| CodeFeedbackMT | test | 0.76690 |114| CodeFeedbackST | test | 0.85207 |115| CodeSearchNetCCRetrieval | test | 0.91805 |116| CodeSearchNetRetrieval | test | 0.89239 |117| CodeTransOceanContest | test | 0.75953 |118| CodeTransOceanDL | test | 0.36057 |119| CosQA | test | 0.42810 |120| StackOverflowQA | test | 0.86608 |121| SyntheticText2SQL | test | 0.68266 |122| **Macro average, all 12 tasks** | | **0.70907** |123| **CoIR macro average, 10 tasks** | | **0.68684** |124 125¹ `CodeEditSearchRetrieval` does not provide a standard `test` split in MTEB, so the126official `train` split is used for evaluation. These examples were **not** used for127fine-tuning. See [Data Decontamination](#data-decontamination).128 129### Leaderboard Standing130 131On the public [MTEB(Code, v1)](https://huggingface.co/spaces/mteb/leaderboard)132leaderboard, the model's task-mean score (70.91, i.e. the 12-task macro average above133×100) places it as follows:134 135* **#18 of 241 models overall**, ahead of many models an order of magnitude larger136* **#6 of 155 among sub-1B-parameter dense single-vector models** — and the137 **smallest model in that top six**. The five models ranked above it138 (`F2LLM-v2-0.6B/330M`, `pplx-embed-v1-0.6b`, `C2LLM-0.5B`, `Qwen3-Embedding-0.6B`)139 are all roughly 0.33–0.6B parameters, i.e. 2–4× larger.140* **#1 among ranked dense single-vector models under 300M parameters**141 (the leaderboard's small-model view)142 143> **Reading the numbers fairly.** MTEB(Code, v1) reports a *zero-shot %* for each144> model — the fraction of leaderboard tasks the model was *not* trained on.145> `NightOwl-CodeEmbedding` is **8%** zero-shot: it was trained on most of these task146> families, so its score reflects strong **in-domain** retrieval rather than zero-shot147> transfer. Models marked **100%** (e.g. `embeddinggemma-300m`, the148> `granite-embedding` r2 family, `Qwen3-Embedding`) are evaluated fully out-of-domain,149> so raw score comparisons across rows with different zero-shot % are not150> apples-to-apples. The fairest direct comparisons are to other code-specialized151> models at similar zero-shot levels (e.g. the `F2LLM` / `C2LLM` families at 8–58%).152 153### Comparison with similar-sized models154 155The table below compares `NightOwl-CodeEmbedding` with other compact **dense156single-vector (bi-encoder)** models on MTEB(Code, v1), with a size ladder of larger157single-vector models for reference. Score is the leaderboard task mean (higher is158better); the *Zero-shot* column is the share of tasks the model did not train on.159 160| Model | Params | Emb. dim | Max tokens | Zero-shot | MTEB(Code, v1) ↑ |161| ---------------------------------------------------- | ------: | -------- | ---------: | --------: | ---------------: |162| **`NightOwl-CodeEmbedding`** (this model) | 150.8M | 768 | 1,024 | 8% | **70.91** |163| `codefuse-ai/F2LLM-v2-160M` | 159M | 640 | 40,960 | 58% | 70.38 |164| `google/embeddinggemma-300m` | 308M | 768 | 2,048 | 100% | 68.76 |165| `codefuse-ai/F2LLM-v2-80M` | 80M | 320 | 40,960 | 58% | 67.97 |166| `ibm-granite/granite-embedding-311m-multilingual-r2` | 312M | 768 | 8,192 | 100% | 63.84 |167| _Larger single-vector reference (size ladder)_ | | | | | |168| `codefuse-ai/F2LLM-v2-0.6B` (#1 sub-1B) | 596M | 1,024 | 40,960 | 58% | 77.41 |169| `Qwen/Qwen3-Embedding-0.6B` | 596M | 1,024 | 32,768 | 100% | 75.42 |170| `codefuse-ai/F2LLM-v2-14B` (#1 overall) | 13.99B | 5,120 | 40,960 | 58% | 80.75 |171 172Takeaways:173 174* Among compact single-vector dense models, `NightOwl-CodeEmbedding` is the strongest175 entry in the leaderboard's small-model view while also being one of the smallest,176 edging out `F2LLM-v2-160M` and clearly ahead of `embeddinggemma-300m`.177* The sub-1B leaders (`F2LLM-v2-0.6B`, `Qwen3-Embedding-0.6B`) score ~4–6.5 points178 higher but are ~4× the parameter count with larger embedding dimensions, which179 directly increases index size and inference cost.180* The 14B model at the top of the overall board is ~10 points higher but ~93× larger —181 a different deployment cost regime entirely.182 183## Base Model: the NightOwl Backbone184 185`NightOwl-CodeEmbedding` is fine-tuned from186[`Shuu12121/NightOwl`](https://huggingface.co/Shuu12121/NightOwl), a ModernBERT-style187code encoder that was **pre-trained from scratch** — tokenizer included — rather than188adapted from a general-purpose checkpoint. The whole stack, from tokenization to the189pre-training objective, is designed around code.190 191**Code-aware tokenizer.** NightOwl uses a custom 50,368-token BPE tokenizer in which192whitespace is tokenized **independently** of adjacent words, so indentation gets its193own tokens instead of being merged into "leading-whitespace + word" pieces. In code,194the same identifier recurs at many indentation depths; folding whitespace into those195pieces would waste large parts of the vocabulary on near-duplicate "indent + token"196variants. Keeping whitespace separate lets the fixed vocabulary budget cover more197genuinely distinct subwords while still representing indentation faithfully — which198matters for whitespace-significant languages such as Python.199 200**Two-phase pre-training with line-level masking.** NightOwl is trained with201masked-language modeling (`mlm_probability = 0.3`) in two phases:202 203* *Phase 1 — mixed pre-training:* standard random-token MLM over code, natural204 language, and technical documentation (producing `NightOwl-Pre`).205* *Phase 2 — code-only continuation:* **line-level MLM**, where entire source-code206 lines are masked instead of random tokens. This aligns the pre-training objective207 with code search and retrieval, where the unit of meaning is closer to a line or208 statement than an isolated token. The recommended `NightOwl` checkpoint is this209 Phase-2 result.210 211Backbone architecture (base):212 213| Property | Value |214| ------------------------------ | ----------------------------------------------------- |215| Architecture | ModernBERT (alternating local/global attention, RoPE) |216| Parameters | ≈150M |217| `hidden_size` / layers / heads | 768 / 19 / 12 |218| Vocabulary | 50,368 (custom code BPE) |219| Max sequence length | 1,024 (Phase 1) → 2,048 (Phase 2) |220 221Pre-training data mixes `bigcode/starcoder2data-extras` (Kaggle notebooks,222StackOverflow threads, GitHub issues, technical documentation, …) with whole-file223source from `Shuu12121/github-file-programs-dataset` across the eight supported224languages. Long examples are split into chunks so all tokens are used rather than225truncated.226 227As a raw backbone — before any embedding fine-tuning — NightOwl reaches **0.8436228average MRR** on MTEB `CodeSearchNetRetrieval` under a fixed SentenceTransformer229fine-tuning protocol, ahead of CodeBERT-base (0.7944), GraphCodeBERT-base (0.8325),230UniXCoder-base (0.8003), and ModernBERT-base (0.8182) evaluated the same way.231`NightOwl-CodeEmbedding` builds the retrieval model described in this card on top of232that backbone.233 234## Training235 236The model was trained with `CachedMultipleNegativesRankingLoss` using bidirectional237query-to-document and document-to-query objectives.238 239| Property | Value |240| -------------------------- | ------------------------------------ |241| Training samples | 2,534,400 |242| Positives per anchor | 1 |243| Negatives per anchor | 15 |244| Loss | `CachedMultipleNegativesRankingLoss` |245| Objective | Bidirectional retrieval training |246| Hard-negative mining model | `Qwen/Qwen3-Embedding-0.6B` |247| Epochs | 1 |248| Learning rate | 6e-5 |249| Batch size | 1024 |250 251### Training Data252 253The training data is a mixture of:254 2551. **Public code-retrieval datasets** covering the following CoIR task families:256 AppsRetrieval, COIRCodeSearchNetRetrieval, CodeFeedbackMT, CodeFeedbackST,257 CodeSearchNetCCRetrieval, CodeSearchNetRetrieval, CodeTransOceanContest,258 CodeTransOceanDL, CosQA, StackOverflowQA, and SyntheticText2SQL.2592. **Custom code–comment pair data**: code snippets paired with natural-language260 description comments across the eight supported languages.2613. **Code-edit data** derived from `commitpackft`, pairing edit intents with code262 changes.263 264All datasets were constructed as hard-negative retrieval datasets: for each anchor,265one positive and fifteen hard negatives. Hard negatives were mined with266[`Qwen/Qwen3-Embedding-0.6B`](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B), which267retrieves semantically similar but non-matching candidates — considerably harder than268random negatives. The mining model is used only during dataset construction and is not269required at inference time.270 271This setup is intended to sharpen discrimination between code snippets, programming272questions, edit examples, and technically similar retrieval candidates.273 274### Data Decontamination275 276To reduce benchmark contamination, the following overlaps were removed from the277training data **before** training:278 279* Overlaps between the custom code–comment pair data and the **CodeSearchNet test280 split**281* Overlaps between the `commitpackft`-derived code-edit data and the282 **CodeEditSearchRetrieval** benchmark evaluation data283 284For `CodeEditSearchRetrieval`, note that MTEB labels the evaluation split `train`.285This is just the official split name for the task; the evaluated examples were not286included in this model's fine-tuning data. The reported score should therefore be read287as **in-domain generalization on held-out benchmark examples** — not training-set288performance, though, given the in-domain training distribution, not strictly zero-shot289performance either.290 291## Intended Use292 293* Natural language to code search294* Code-to-code retrieval and similar-function search295* Code-edit retrieval (matching edit intents to code changes)296* Retrieval over programming Q&A and technical questions297* Local semantic code search systems298* RAG systems over codebases and developer documentation299 300Example use cases include indexing functions, snippets, programming solutions,301StackOverflow-style answers, code-review examples, and edit-related code examples.302 303## Limitations304 305* Specialized for code-related retrieval; may underperform general-purpose text306 embedding models on unrelated natural-language tasks.307* Inputs longer than 1,024 tokens are truncated — a shorter context window than some308 competitors (e.g. the 8K+ token `F2LLM` and `granite` models), so very long files309 must be chunked.310* MTEB(Code, v1) is largely in-domain for this model (8% zero-shot). On code domains,311 query styles, or languages far from the training distribution, expect lower312 performance than the leaderboard numbers suggest.313* Performance may vary by programming language, query style, and the granularity of314 indexed code chunks; languages outside the eight supported ones are untested.315* The model produces dense single-vector embeddings. For applications that need very316 fine-grained token-level matching, other retrieval approaches — such as317 late-interaction (multi-vector) models or cross-encoder rerankers — may be worth318 considering, with different trade-offs in index size and retrieval infrastructure.319 320## Recommended Indexing Settings321 322Encode both queries and documents with normalized embeddings:323 324```python325embeddings = model.encode(texts, normalize_embeddings=True)326```327 328With normalized embeddings, dot product is equivalent to cosine similarity.329 330For codebase search, indexing function-level or class-level chunks usually works best.331Very long files may exceed the 1,024-token context limit and should be split into332smaller semantic chunks.333 334## Citation335 336If you use this model, please cite it together with the base model and Sentence337Transformers.338 339```bibtex340@misc{nightowl_codeembedding,341 title = {NightOwl-CodeEmbedding},342 author = {Shuu12121},343 year = {2026},344 publisher = {Hugging Face},345 url = {https://huggingface.co/Shuu12121/NightOwl-CodeEmbedding}346}347```