mlx-community/Arch-Router-1.5B-mlx-8Bit

1---2base_model: katanemo/Arch-Router-1.5B3language:4- en5library_name: transformers6license: other7license_name: katanemo-research8license_link: https://huggingface.co/katanemo/Arch-Router-1.5B/blob/main/LICENSE9pipeline_tag: text-generation10tags:11- routing12- preference13- arxiv:2506.1665514- llm15- mlx16- mlx-my-repo17paper: https://arxiv.org/abs/2506.1665518---19 20# mlx-community/Arch-Router-1.5B-mlx-8Bit21 22The Model [mlx-community/Arch-Router-1.5B-mlx-8Bit](https://huggingface.co/mlx-community/Arch-Router-1.5B-mlx-8Bit) was converted to MLX format from [katanemo/Arch-Router-1.5B](https://huggingface.co/katanemo/Arch-Router-1.5B) using mlx-lm version **0.26.4**.23 24## Use with mlx25 26```bash27pip install mlx-lm28```29 30```python31from mlx_lm import load, generate32 33model, tokenizer = load("mlx-community/Arch-Router-1.5B-mlx-8Bit")34 35prompt="hello"36 37if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:38    messages = [{"role": "user", "content": prompt}]39    prompt = tokenizer.apply_chat_template(40        messages, tokenize=False, add_generation_prompt=True41    )42 43response = generate(model, tokenizer, prompt=prompt, verbose=True)44```45