jedisct1/NextCoder-32B-q8-mlx

1---2license: mit3language:4- en5base_model: microsoft/NextCoder-32B6pipeline_tag: text-generation7library_name: mlx8tags:9- code10- chat11- microsoft12- nextcoder13- selekt14- mlx15datasets:16- microsoft/NextCoderDataset17- microsoft/NextCoderDataset-Conversational18- bigcode/commitpackft19- bigcode/starcoderdata20---21 22# jedisct1/NextCoder-32B-q8-mlx23 24This model [jedisct1/NextCoder-32B-q8-mlx](https://huggingface.co/jedisct1/NextCoder-32B-q8-mlx) was25converted to MLX format from [microsoft/NextCoder-32B](https://huggingface.co/microsoft/NextCoder-32B)26using mlx-lm version **0.25.3**.27 28## Use with mlx29 30```bash31pip install mlx-lm32```33 34```python35from mlx_lm import load, generate36 37model, tokenizer = load("jedisct1/NextCoder-32B-q8-mlx")38 39prompt = "hello"40 41if tokenizer.chat_template is not None:42    messages = [{"role": "user", "content": prompt}]43    prompt = tokenizer.apply_chat_template(44        messages, add_generation_prompt=True45    )46 47response = generate(model, tokenizer, prompt=prompt, verbose=True)48```49