funnygeeker/Ornith-1.0-35B-uncensored-heretic-MLX-4bit

English | 中文

English

This model was converted to MLX format from `llmfan46/Ornith-1.0-35B-uncensored-heretic`.

Subjective Evaluation

• —This model has some playability and is worth trying.

Things To Note

• —I ran some tests, HumanEval's score may be a little low (More obvious), This model may see more retries in some agent tasks that involve writing short scripts.
• —I've noticed that some of my cron jobs are taking longer to run than they used to.（Hermes-Agent）
• —Ornith-1.0-35B-uncensored-heretic-MLX-4bit: ~66.7% (this model)
• —Ornith-1.0-35B-4bit: ~86.7%
• —Software: oMLX-v0.4.4
• —Sample Size: 30

Recommended Inference Parameters

01

02

• —When the value of repeat_penalty is set to 1.0, the model is more prone to thinking loops.
• —When the value of repeat_penalty is set to 1.08 or greater, in certain tasks, it is easier to change the English period of some file suffix names into Chinese periods, thereby affecting tool invocation.
• —Other parameters have not been fully verified and are currently for reference only.

Code To Quantify This Model

from mlx_vlm import convert # mlx_vlm >= 0.6.4

convert(
    hf_path="/Volumes/original_model_path/Ornith-1.0-35B-uncensored-heretic",
    mlx_path="/Volumes/quantized_model_path/Ornith-1.0-35B-uncensored-heretic-MLX-4bit",
    quantize=True,
    q_bits=4,
    q_group_size=32
)

中文（Chinese）

该模型从 `llmfan46/Ornith-1.0-35B-uncensored-heretic` 转换为 MLX 格式。

主观评价

• —这个模型有一定的可玩性，值得尝试。

注意事项

• —我进行了一些测试，HumanEval 的分数可能偏低（比较明显）。在一些涉及编写短脚本的 Agent 任务中，该模型可能需要更多重试。
• —我注意到我的一些 cron 作业运行时间比过去更长了（Hermes-Agent）。
• —Ornith-1.0-35B-uncensored-heretic-MLX-4bit：~66.7%（此模型）
• —Ornith-1.0-35B-4bit：~86.7%
• —软件：oMLX-v0.4.4
• —样本量：30

推荐的参数

01

02

• —当 repeat_penalty 的值设置为 1.0 时，模型更容易出现思维循环。
• —当 repeat_penalty 的值设置为 1.08 或更大时，在某些任务中，更容易将某些文件后缀名的英文句点更改为中文句点，从而影响工具调用。
• —其他参数尚未完全验证，目前仅供参考。

其他规格

6bit：经测试，平均性能提升 ~3%（由于我的流量不多了，所以暂时还没上传）

量化此模型的代码

from mlx_vlm import convert # mlx_vlm >= 0.6.4

convert(
    hf_path="/Volumes/original_model_path/Ornith-1.0-35B-uncensored-heretic",
    mlx_path="/Volumes/quantized_model_path/Ornith-1.0-35B-uncensored-heretic-MLX-4bit",
    quantize=True,
    q_bits=4,
    q_group_size=32
)