CoolFace
Modelpublic

yugeshkarunamurthy/Agents-A1-4B-oQe6

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
2likes95downloads
Model Card

Agents-A1-4B-oQe6

Apple Silicon Optimized oQe6 MLX Quantized Release

This repository contains an oQe6 mixed-precision MLX quantized version of Agents-A1-4B, optimized for efficient local inference on Apple Silicon.

The original Agents-A1-4B model was developed by InternScience. This repository provides an optimized MLX conversion using oQe6 sensitivity-aware mixed-precision quantization and does not include any additional training or fine-tuning.

About Agents-A1

Agents-A1-4B is a compact agentic reasoning model designed for long-horizon task execution and tool-augmented reasoning.

Despite having only 4B parameters, it demonstrates impressive performance across:

  • —Agentic reasoning
  • —Long-horizon search
  • —Software engineering
  • —Scientific research
  • —Instruction following
  • —Tool calling
  • —Function calling
  • —Professional knowledge
  • —Research assistance

The model is trained using a multi-stage agent training pipeline that combines:

  • —Large-scale supervised fine tuning
  • —Domain-specialized teachers
  • —Multi-domain knowledge distillation
  • —Long-horizon trajectory optimization

The result is a highly capable local agent model with strong planning and reasoning abilities. :contentReference[oaicite:1]{index=1}

Quantization

This release uses oQe6 mixed-precision quantization.

Specifications

  • —Format: MLX
  • —Quantization: oQe6
  • —Method: Sensitivity-Aware Mixed Precision
  • —Target Platform: Apple Silicon
  • —Inference Engine: MLX / oMLX

Unlike conventional fixed-bit quantization, oQe6 dynamically assigns precision according to layer sensitivity, preserving quality while significantly reducing memory usage.

Benefits include:

  • —Better reasoning retention
  • —Higher tool-use accuracy
  • —Lower memory usage
  • —Faster inference
  • —Optimized Apple Silicon performance

Recommended Settings

For best performance:

yaml
temp: 0.85
top_p: 0.95
top_k: 20
min_p: 0
rep_penalty: 1.0
presence_penalty: 1.1
enable_thinking: true

These settings closely follow the recommendations from the original Agents-A1 team and provide stable multi-turn agentic reasoning, planning, and tool usage. :contentReference[oaicite:2]{index=2}

Example Usage

python
from mlx_lm import load, generate

model, tokenizer = load("yugeshkarunamurthy/Agents-A1-4B-oQe6")

messages = [
    {
        "role": "user",
        "content": "Research the advantages of speculative decoding and summarize them."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    temp=0.85,
    top_p=0.95,
    top_k=20,
    max_tokens=16384,
)

print(response)

Optimized For

This release is optimized for:

  • —Apple M1
  • —Apple M2
  • —Apple M3
  • —Apple M4

Compatible with:

  • —MLX
  • —oMLX
  • —Open WebUI
  • —LM Studio (MLX)
  • —MLX-LM
  • —Local AI Agent Frameworks

Model Highlights

  • —Long-Horizon Agentic Reasoning
  • —Native Function Calling
  • —Tool Use
  • —Scientific Reasoning
  • —Engineering Tasks
  • —Software Development
  • —Research Workflows
  • —Multi-step Planning
  • —Instruction Following
  • —Long Context Support

Intended Use

Agents-A1-4B-oQe6 is ideal for:

  • —AI Agents
  • —Coding Assistants
  • —Research Automation
  • —Tool Calling
  • —Scientific Analysis
  • —Software Engineering
  • —Deep Research
  • —Multi-step Planning
  • —Local AI Applications

Hardware Recommendations

Recommended systems:

  • —Apple M1 Pro / Max / Ultra
  • —Apple M2 Pro / Max / Ultra
  • —Apple M3 Series
  • —Apple M4 Series

Higher-memory Apple Silicon systems are recommended for large context windows and complex multi-step agent workflows.

About oQe6 Quantization

oQe6 is a sensitivity-aware mixed-precision quantization technique designed to preserve model quality while substantially reducing memory requirements.

Instead of quantizing every layer equally, oQe6 identifies the most important weights and allocates precision accordingly, allowing efficient execution with minimal impact on reasoning performance.

Benefits include:

  • —Improved reasoning retention
  • —Better planning quality
  • —Lower memory consumption
  • —Faster inference
  • —Apple Silicon optimization

Credits

Original Model

All credit for the original model, datasets, training methodology, evaluation, benchmarks, and research belongs entirely to:

InternScience

Original Repository:

https://huggingface.co/InternScience/Agents-A1-4B

Technical Report:

https://arxiv.org/abs/2606.30616

oQe6 MLX Quantized Release

This repository provides an Apple Silicon optimized oQe6 MLX quantized version of the original model.

No additional fine-tuning has been performed.

Acknowledgements

  • —InternScience
  • —Shanghai Artificial Intelligence Laboratory
  • —Qwen Team
  • —Apple MLX
  • —Hugging Face
  • —Transformers
  • —SGLang
  • —vLLM
  • —oMLX
  • —OptiQ Quantization

Citation

If you use this model in research, please cite the original Agents-A1 paper:

bibtex
@misc{bai2026scalinghorizonparametersreaching,
  title={Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent},
  author={Lei Bai and others},
  year={2026},
  eprint={2606.30616},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

License

This release inherits the Apache-2.0 license from the original model.

Please refer to the original repository for complete licensing information.

Disclaimer

This repository contains an optimized oQe6 MLX quantized conversion intended for efficient local inference on Apple Silicon devices.

All original model architecture, datasets, training methodology, benchmarks, evaluations, and research remain entirely the work of the original authors.