lil-lab/CoLMLM-360M-FW

CoLMLM-360M-FW

The 360M-parameter, SmolLM2-based Co-LMLM retriever from the paper **Co-LMLM: Continuous-Query Limited Memory Language Models**, trained on FineWeb-Edu.

Co-LMLM is a retrieval-aware language model: at each <FACT> position it emits a continuous query from its hidden state that retrieves the fact's content from an external index at inference time, instead of storing it in its weights.

Usage

The model is used together with a released retrieval index. See the code repository for setup, index downloads, and a copy-pasteable quick start:

👉 [github.com/lil-lab/Co-LMLM](https://github.com/lil-lab/Co-LMLM)

This model is part of the **Co-LMLM** collection.

Citation

@misc{feldman2026colmlmcontinuousquerylimitedmemory,
      title={Co-LMLM: Continuous-Query Limited Memory Language Models},
      author={Yair Feldman and Linxi Zhao and Nathan Godey and Dongyoung Go and Yilun Hua and Kilian Q. Weinberger and Jennifer J. Sun and Yoav Artzi},
      year={2026},
      eprint={2607.07707},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.07707},
}