mlx-community/IQuest-Coder-V1-40B-Loop-Instruct-6bit

1---2license: other3license_name: iquestcoder4license_link: https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Instruct/blob/main/LICENSE5language:6- en7library_name: mlx8pipeline_tag: text-generation9tags:10- mlx11base_model: IQuestLab/IQuest-Coder-V1-40B-Loop-Instruct12---13 14# mlx-community/IQuest-Coder-V1-40B-Loop-Instruct-6bit15 16This model [mlx-community/IQuest-Coder-V1-40B-Loop-Instruct-6bit](https://huggingface.co/mlx-community/IQuest-Coder-V1-40B-Loop-Instruct-6bit) was17converted to MLX format from [IQuestLab/IQuest-Coder-V1-40B-Loop-Instruct](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Loop-Instruct)18using mlx-lm version **0.30.0**.19 20## Use with mlx21 22```bash23pip install mlx-lm24```25 26```python27from mlx_lm import load, generate28 29model, tokenizer = load("mlx-community/IQuest-Coder-V1-40B-Loop-Instruct-6bit")30 31prompt = "hello"32 33if tokenizer.chat_template is not None:34    messages = [{"role": "user", "content": prompt}]35    prompt = tokenizer.apply_chat_template(36        messages, add_generation_prompt=True, return_dict=False,37    )38 39response = generate(model, tokenizer, prompt=prompt, verbose=True)40```41