hongpingzhang/ecocompute-energy-efficiency

---
license: cc-by-4.0
task_categories:
  - text-generation
tags:
  - energy-efficiency
  - quantization
  - benchmark
  - gpu
  - green-ai
size_categories:
  - n<1K
---

# EcoCompute: Energy Efficiency Benchmark for Quantized Language Models

Systematic energy efficiency measurements for quantized language models across 0.5B-14B parameters on NVIDIA RTX 5090 (Blackwell), RTX 4090D (Ada Lovelace), and A800 80GB (Ampere).

**113+ configurations** covering five precision methods: FP16, NF4, INT8 (default), INT8 (pure bnb), and FP8.

## Key Findings

- **Small-Model Quantization Paradox**: +25-56% energy for models <3B parameters
- **Break-even threshold**: 4.2B (Ada) / 5.2B (Blackwell)
- **INT8 default**: 4.6x less efficient than NF4 for small models
- **FP8 Paradox**: up to +701% energy overhead on RTX 5090 (software immaturity)

## Try It Interactively

Query these benchmarks conversationally with the **EcoLobster** AI advisor on ClawHub:

👉 **[EcoCompute on ClawHub](https://clawhub.ai/hongping-zh/ecocompute)** — Ask questions like "Which quantization saves the most energy for Qwen2-7B on RTX 5090?"

No setup required. Works directly in the OpenClaw client.

## Dataset Details

- **Hardware**: RTX 5090, RTX 4090D, A800 80GB
- **Models**: Qwen2 (0.5B/1.5B/7B/14B), TinyLlama-1.1B, Mistral-7B, Yi-1.5 (6B/9B)
- **Precisions**: FP16, NF4, INT8 (default), INT8 (pure bnb), FP8
- **Quality**: CV < 2%, n=2 repeated trials per configuration

## Links & References

| Resource | URL |
|----------|-----|
| ClawHub Skill (interactive) | https://clawhub.ai/hongping-zh/ecocompute |
| Zenodo Archive (DOI) | https://doi.org/10.5281/zenodo.18900289 |
| Interactive Dashboard | https://hongping-zh.github.io/ecocompute-dynamic-eval/ |
| GitHub Repository | https://github.com/hongping-zh/ecocompute-ai |
| HF Optimum Docs (cites this data) | https://huggingface.co/docs/optimum/concept_guides/quantization |
| HF Optimum PR #2410 | https://github.com/huggingface/optimum/pull/2410 |
| torchao FP8 Issue #4094 | https://github.com/pytorch/ao/issues/4094 |

## Community Adoption

- Referenced in **HuggingFace Optimum official documentation** ([PR #2410](https://github.com/huggingface/optimum/pull/2410), merged Mar 2026)
- FP8 energy anomaly confirmed by **torchao maintainers** ([Issue #4094](https://github.com/pytorch/ao/issues/4094))
- Available as interactive AI skill on **[ClawHub](https://clawhub.ai/hongping-zh/ecocompute)**
- Related contributions: [bitsandbytes PR #1882](https://github.com/bitsandbytes-foundation/bitsandbytes/pull/1882), [Transformers PR #44407](https://github.com/huggingface/transformers/pull/44407)

## Citation

## License

CC BY 4.0