groxaxo/Nemotron-3-Embed-1B-oQ4-MLX

Nemotron-3-Embed-1B-oQ4-MLX

<!-- polished-overview:start -->

Overview

Nemotron-3-Embed-1B-oQ4-MLX is an MLX-formatted checkpoint optimized for Apple silicon, published by `groxaxo`. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

At a glance

What is included

• —*.safetensors (1 file)
• —config.json
• —tokenizer.json
• —tokenizer_config.json
• —Additional configuration, tokenizer, processor, or shard files (14 visible artifacts total)

Quick start

MLX

Use an up-to-date MLX-compatible runtime on Apple silicon and point it at this repository:

mlx_lm.generate --model groxaxo/Nemotron-3-Embed-1B-oQ4-MLX --prompt "Write a concise technical summary."

Embedding and audio repositories may require the task-specific MLX package documented by the upstream project.

Compatibility and responsible use

• —Use a runtime that explicitly supports this format, architecture, and modality.
• —Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
• —Review the source model card and license before redistribution or deployment.
• —Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
• —Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws. <!-- polished-overview:end -->

An Apple-Silicon-native OMLX oQ4 conversion of `nvidia/Nemotron-3-Embed-1B-BF16`.

This derivative was produced locally on Apple Silicon with the same streaming, sensitivity-driven OMLX pipeline used for the oQ5 release. It uses a base 4-bit affine plan, group size 64, and bfloat16 working dtype. The calibration ran on the checkpoint's bidirectional encoder architecture rather than through a causal-LM wrapper. No weights were retrained.

The quantized safetensors payload is 670,214,436 bytes. OMLX promoted 26 sensitivity-selected projections to 5, 6-bit precision.

Embedding contract

• —bidirectional attention (is_causal: false)
• —average/mean pooling
• —L2-normalized 2,048-dimensional output
• —query: prefix for queries
• —passage: prefix for documents

BF16 fidelity smoke test

The source and this oQ4 checkpoint were evaluated using a query, a matching passage, and an unrelated passage. All outputs were 2,048-dimensional and L2-normalized.

Quick start on Apple Silicon

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade mlx mlx-lm transformers huggingface_hub

hf download groxaxo/Nemotron-3-Embed-1B-oQ4-MLX --local-dir ./Nemotron-3-Embed-1B-oQ4-MLX
python ./Nemotron-3-Embed-1B-oQ4-MLX/embed_mlx.py \
  --model ./Nemotron-3-Embed-1B-oQ4-MLX \
  --query "Which city is known as the City of Sails?" \
  --passage "Auckland is widely known as the City of Sails."

The included script deliberately loads the bare Ministral3 backbone and applies non-causal attention. The regular MLX-LM generation path is causal and is not the correct embedding interface for this checkpoint.

Quantization details

See QUANTIZATION_REPORT.md for artifact SHA-256 values and the full local validation result.

License

This derivative inherits the `OpenMDW-1.1` license from the source model. LICENSE, NOTICE, and THIRD_PARTY_NOTICES.md are included unchanged from NVIDIA's source repository.