Modelpublic

AXERA-TECH/jina-embeddings-v5-omni-small

sourceHugging Facecc-by-nc-4.0updated 2mo agoView on Hugging Face

1likes30downloads

Model Card

jina-embeddings-v5-omni-small on AXERA NPU

Ready-to-run deployment package for `jinaai/jina-embeddings-v5-omni-small` on AX650 / NPU3.

—Runtime: bundled axllm service with an OpenAI-compatible /v1/embeddings API
—Inputs: text and images through the packaged API and demo; fixed-profile image and audio encoder models are included
—Output: one L2-normalized 1024-dimensional embedding per request
—Tasks: retrieval, clustering, classification, and text matching, selected at runtime without loading four complete text backbones
—Included assets: compiled .axmodel files, tokenizer data, packed task adapters, sample images, validation results, and a browser demo

Supported Platform

—[x] AX650 / NPU3
—[x] Single NPU

Performance

All measurements below were taken on one AX650 NPU with the files in this package. Standalone .axmodel latency uses 3 warm-up runs followed by 20 measured runs. API latency includes tokenization, adapter selection, model execution, and local HTTP overhead.

End-to-End Text Embedding and Task Switching

Scenario	Average time
Same-task short-text embedding request	256.933 ms
Alternating `retrieval` / `clustering` requests	378.273 ms
Internal switch between loaded adapters	117.000 ms

The model encodes each input in one forward pass. It has a prefill stage but no autoregressive decode stage.

Standalone Module Latency

Module	Fixed input profile	Average time
Text backbone, sum of 28 layer models	256 tokens, shape group 1	189.957 ms
Embedding post-processing	`[1, 1024]` output	8.192 ms
Vision encoder	256×256 image	50.675 ms
Vision merger, retrieval	64 visual tokens	1.144 ms
Vision merger, clustering	64 visual tokens	1.140 ms
Vision merger, classification	64 visual tokens	1.142 ms
Vision merger, text matching	64 visual tokens	1.142 ms
Audio encoder	800 mel frames	209.989 ms
Audio projector, retrieval	task-specific	0.290 ms
Audio projector, clustering	task-specific	0.290 ms
Audio projector, classification	task-specific	0.289 ms
Audio projector, text matching	task-specific	0.298 ms

Standalone results were measured with ax_run_model --warmup=3 --repeat=20; the text layer models additionally use --group=1. The recorded averages are available in `validation/ax_run_model_average.json`. Task-switch samples are available in `validation/task_switch_client_latency.json` and `validation/task_switch_internal_latency.json`.

Runtime Memory

Measured runtime state	CMM used	OS memory used
`axllm` initialized and retrieval/clustering smoke test completed	1438.258 MiB	89.048 MiB PSS

CMM used is the increase in allocated CMM between the stopped-service baseline and the measured running state on the same board. PSS is the proportional set size reported for the axllm process after the smoke test. These are consumed amounts and do not depend on the board's installed CMM capacity.

The audio encoder is initialized lazily and was not loaded in this text smoke-test state. The measurement record is available in `validation/runtime_memory.json`.

Package Size

Packaged component	Size
Shared text backbone, embedding table, and post model	1007.6 MiB
Four packed LoRA task adapters	154.0 MiB total
Vision models	402.1 MiB
Audio models	675.1 MiB
Complete runtime package	2.21 GiB

The four tasks share the same text backbone. Package sizes describe files on storage and are independent of the runtime CMM and OS-memory measurements above.

Functional Validation

Check	Result
Retrieval relevant cosine	0.738921
Retrieval irrelevant cosine	0.034047
Clustering within-topic cosine	0.758002
Clustering cross-topic cosine	0.033445
Retrieval round-trip max absolute difference	0.0
Complete web-demo checks	5 / 5 PASS

Machine-readable results are stored under `validation/`.

Package Layout

text

.
├── README.md
├── config.json
├── start_axllm.sh
├── bin/axllm
├── jina_v5_omni_tokenizer.txt
├── tokenizer.json
├── tokenizer_config.json
├── model.embed_tokens.weight.bfloat16.bin
├── jina_embeddings_v5_omni_p256_l0_together.axmodel
├── ...
├── jina_embeddings_v5_omni_p256_l27_together.axmodel
├── jina_embeddings_v5_omni_post.axmodel
├── jina_v5_omni_small_vision_tower_256x256.axmodel
├── jina_v5_omni_small_vision_merger_<task>_256x256.axmodel
├── jina_v5_omni_small_audio_tower_800frames.axmodel
├── jina_v5_omni_small_audio_projector_<task>_800frames.axmodel
├── lora/{retrieval,clustering,classification,text-matching}/
├── scripts/test_retrieval_clustering.py
├── demo/
├── assets/
├── validation/
└── SHA256SUMS

The root directory is the axllm model directory. Keep the relative paths unchanged because config.json resolves the text, media, and adapter files from this layout.

Download the Package

bash

mkdir -p AXERA-TECH/jina-embeddings-v5-omni-small
cd AXERA-TECH/jina-embeddings-v5-omni-small
hf download AXERA-TECH/jina-embeddings-v5-omni-small --local-dir .

Copy the downloaded directory to the AX650 board before continuing.

Run on the Board

From the package root, start the bundled service on port 18201:

bash

bash ./start_axllm.sh 18201

In another terminal, verify the service and model id:

bash

curl http://127.0.0.1:18201/health
curl http://127.0.0.1:18201/v1/models

A healthy service returns:

json

{
  "concurrency": 0,
  "max_concurrency": 1,
  "status": "healthy"
}

The model list contains AXERA-TECH/jina-embeddings-v5-omni-small.

Select a Task Adapter

Set task_id in each embedding request:

`task_id`	Intended use	Input role
`retrieval`	Cross-modal search, semantic retrieval, and RAG	`query` for queries; `document` for indexed items
`clustering`	Topic discovery, grouping, and deduplication	`document`
`classification`	Features for zero-shot, few-shot, or downstream classifiers	`document`
`text-matching`	Semantic similarity, paraphrase, and near-duplicate matching	`document`

Adapter files are validated during model initialization. A task switch then loads the selected matrices without repeating the full file validation.

OpenAI-Compatible Text Request

bash

curl http://127.0.0.1:18201/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/jina-embeddings-v5-omni-small",
    "task_id": "retrieval",
    "input_type": "query",
    "input": "a photo of a cat"
  }'

The embedding is returned in data[0].embedding. It contains 1024 floating-point values with an L2 norm of approximately 1.0. For retrieval, use input_type: query for the search query and input_type: document for indexed text, images, or audio.

The packaged smoke test verifies retrieval ranking, clustering separation, and a bit-exact retrieval -> clustering -> retrieval round trip:

bash

python3 scripts/test_retrieval_clustering.py \
  --url http://127.0.0.1:18201/v1/embeddings

Browser Demo

Keep axllm running and start the packaged web application in another terminal:

bash

python3 demo/openai_task_switch_web.py \
  --axllm-url http://127.0.0.1:18201/v1/embeddings \
  --host 0.0.0.0 \
  --port 8080

The web process checks the embedding service before it starts. Open http://<board-ip>:8080 to search the 12 packaged animal images with natural language. The first search creates an in-memory image index; later searches reuse the cached image embeddings and compute only the new text embedding.

Open http://<board-ip>:8080/index.html for the complete capability validation. It exercises all four tasks and then checks a bit-exact switch back to retrieval.

For the query 找出所有猫的图像, all three cat images are ranked first:

<table> <tr> <td align="center"><img src="assets/cat2.jpg" alt="cat2.jpg" width="220"> 1. cat2.jpg cosine 0.343559</td> <td align="center"><img src="assets/cat1.jpeg" alt="cat1.jpeg" width="220"> 2. cat1.jpeg cosine 0.336765</td> <td align="center"><img src="assets/cat0.jpeg" alt="cat0.jpeg" width="220"> 3. cat_0.jpeg cosine 0.296639</td> </tr> </table>

The complete log below was captured from the packaged demo on AX650.

<details> <summary>Complete AX650 web-demo log</summary>

text

Jina Embeddings v5 Omni - OpenAI API 多任务切换 Demo
Model: AXERA-TECH/jina-embeddings-v5-omni-small
Assets: cat=3, dog=3, fox=3, rabbit=3

--- retrieval：文本查询召回动物图片 ---

=== task switch: <startup> -> retrieval ===
[request] task=retrieval     input=text:a photo of a  latency=  256.29 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:cat_0.jpeg   latency=  526.62 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:cat_1.jpeg   latency=  694.87 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:cat_2.jpg    latency=  491.13 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:dog_0.jpeg   latency=  382.88 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:dog_1.jpg    latency=  406.06 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:dog_2.jpeg   latency=  823.02 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:fox_0.jpeg   latency=  400.50 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:fox_1.jpeg   latency=  358.35 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:fox_2.jpeg   latency=  368.29 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:rabbit_0.jpeg latency=  361.31 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:rabbit_1.jpeg latency=  346.02 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:rabbit_2.jpeg latency=  345.37 ms dim=1024 norm=1.000002
[result] retrieval ranking:
   1. cat_1.jpeg        cosine=0.376784
   2. cat_2.jpg         cosine=0.374494
   3. cat_0.jpeg        cosine=0.333584
   4. fox_0.jpeg        cosine=0.255053
   5. rabbit_1.jpeg     cosine=0.250108
   6. fox_2.jpeg        cosine=0.245141
   7. fox_1.jpeg        cosine=0.240511
   8. rabbit_2.jpeg     cosine=0.228791
   9. rabbit_0.jpeg     cosine=0.220517
  10. dog_1.jpg         cosine=0.210978
  11. dog_0.jpeg        cosine=0.185020
  12. dog_2.jpeg        cosine=0.156387
[check] cat top-1: PASS

--- clustering：同类与跨类图片聚类 ---

=== task switch: retrieval -> clustering ===
[request] task=clustering    input=image:cat_0.jpeg   latency=  678.72 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:cat_1.jpeg   latency=  703.22 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:cat_2.jpg    latency=  492.12 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:dog_0.jpeg   latency=  386.99 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:dog_1.jpg    latency=  406.84 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:dog_2.jpeg   latency=  828.96 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:fox_0.jpeg   latency=  398.71 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:fox_1.jpeg   latency=  358.83 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:fox_2.jpeg   latency=  368.50 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:rabbit_0.jpeg latency=  363.37 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:rabbit_1.jpeg latency=  345.02 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:rabbit_2.jpeg latency=  345.02 ms dim=1024 norm=1.000001
[result] within-class average cosine=0.722070
[result] cross-class  average cosine=0.504200
[check] clustering separation: PASS

--- classification：动物文本分类特征分离 ---

=== task switch: clustering -> classification ===
[request] task=classification input=text:A cat sleeps  latency=  543.08 ms dim=1024 norm=1.000001
[request] task=classification input=text:A kitten res  latency=  255.83 ms dim=1024 norm=1.000002
[request] task=classification input=text:A dog runs i  latency=  256.54 ms dim=1024 norm=1.000001
[request] task=classification input=text:A puppy play  latency=  256.36 ms dim=1024 norm=1.000001
[request] task=classification input=text:A red fox wa  latency=  257.11 ms dim=1024 norm=1.000001
[request] task=classification input=text:A wild fox h  latency=  257.35 ms dim=1024 norm=1.000001
[request] task=classification input=text:A rabbit eat  latency=  257.31 ms dim=1024 norm=1.000001
[request] task=classification input=text:A bunny sits  latency=  256.40 ms dim=1024 norm=1.000001
[result] cat    same-class cosine=0.892006
[result] dog    same-class cosine=0.883812
[result] fox    same-class cosine=0.890544
[result] rabbit same-class cosine=0.859263
[result] same-class average cosine=0.881406
[result] cross-class average cosine=0.835429
[check] classification separation: PASS

--- text-matching：动物文本匹配 ---

=== task switch: classification -> text-matching ===
[request] task=text-matching input=text:A red fox st  latency=  424.52 ms dim=1024 norm=1.000001
[request] task=text-matching input=text:A domestic c  latency=  255.87 ms dim=1024 norm=1.000001
[request] task=text-matching input=text:A pet dog ru  latency=  257.68 ms dim=1024 norm=1.000001
[request] task=text-matching input=text:A wild red f  latency=  255.85 ms dim=1024 norm=1.000001
[request] task=text-matching input=text:A small rabb  latency=  256.55 ms dim=1024 norm=1.000001
[result] text-matching ranking:
  1. fox    cosine=0.927695
  2. rabbit cosine=0.539868
  3. dog    cosine=0.536886
  4. cat    cosine=0.363692
[check] fox top-1: PASS

--- round-trip：切回 retrieval ---

=== task switch: text-matching -> retrieval ===
[request] task=retrieval     input=text:a photo of a  latency=  378.39 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:cat_0.jpeg   latency=  555.39 ms dim=1024 norm=1.000001
[result] retrieval query max_abs_diff=0.0000000000
[result] retrieval image max_abs_diff=0.0000000000
[check] retrieval round-trip: PASS

=== summary ===
retrieval       PASS
clustering      PASS
classification  PASS
text-matching   PASS
round-trip      PASS

</details>

Fixed Input Profiles

The final embedding output is always [1, 1024] and L2 normalized.

Input	Compiled profile
Text	256-token processing chunks; 2047-token runtime limit
Image	256×256
Audio	800 mel frames

The vision and audio task-specific mapper files are selected together with the active adapter. Image resolutions or audio lengths outside these profiles require rebuilding the corresponding media modules.

Adapter selection is mutable state within one service instance. Serialize requests that change task_id, or run separate service instances when requests must be isolated by task. The first switch may include filesystem cache overhead; use the warm measurements above for steady-state planning.

Conversion References

If you need the original model files or want to rebuild the deployment artifacts, start with:

—Original Hugging Face model: jinaai/jina-embeddings-v5-omni-small
—AXERA conversion and deployment workflow: AXERA-TECH/jina_embeddings_v5_omni.axera

The public conversion workflow requires Pulsar2 7.0 or later. Pulsar2 is needed only to rebuild the deployment artifacts, not to run this downloaded package.

Discussion

—GitHub Issues
—QQ group: 139953715