TiGa-RCE/needle-mlx

1---2license: mit3library_name: mlx4tags:5  - mlx6  - function-calling7  - tool-use8  - encoder-decoder9  - apple-silicon10---11 12# Needle MLX13 14An MLX safetensors format conversion of15[Cactus-Compute/needle](https://huggingface.co/Cactus-Compute/needle), a1626M-parameter encoder-decoder function-calling model.17 18This is not a fine-tune or a quantization. The published checkpoint is loaded19using Needle's declared `bfloat16` inference dtype, then stored as MLX20safetensors so it preserves the upstream runtime's executable values.21 22## What is included23 24- `model.safetensors`: 31 converted tensors, 26,315,421 parameters.25- `needle_mlx.py`: a small custom MLX inference implementation for Needle's26  architecture.27- `tokenizer/`: the upstream SentencePiece tokenizer files.28- `config.json` and `manifest.json`: architecture and conversion provenance.29 30## Use31 32```python33from needle_mlx import NeedleModel34 35model = NeedleModel.from_pretrained(".")36logits = model.forward([[1, 2, 3]], [[1, 4]])37```38 39This is an MLX-native package, not an MLX-LM or oMLX model yet. Needle uses a40custom JAX/Flax architecture, so load it with the included runtime or add a41dedicated adapter to the serving runtime.42 43## Conversion and verification44 45The published `needle.pkl` checkpoint was read with a restricted NumPy-only46pickle loader, converted to MLX `bfloat16` safetensors, and checked47tensor-for-tensor after the same dtype cast used by Needle's upstream runtime.48 49- Source SHA-256:50  `40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40`51- Converted `model.safetensors` SHA-256:52  `7b9d5f0d6ddeb7fbb20f4e45f3f616919357e5d08b5778859fdb762a33d60dae`53- Verification: 31/31 tensors equal the source after Needle's `bfloat16`54  runtime cast; an MLX encoder-decoder smoke pass produced logits with shape55  `(1, 2, 8192)` and generated the upstream weather tool-call example.56 57The converter source is available at58[seeker-cyber-maker/needle-mlx-depicklinator](https://github.com/seeker-cyber-maker/needle-mlx-depicklinator).59 60## Credits and license61 62Needle was created by Cactus Compute. See the63[upstream model card](https://huggingface.co/Cactus-Compute/needle) and64[source repository](https://github.com/cactus-compute/needle) for the model,65training details, and citation. The upstream model card publishes Needle under66the MIT license; this converted package retains that license.67