jniimi/boltzmann-gpt-smartphone

boltzmann-gpt-smartphone

The Smartphone checkpoint of Energy-Based Attribute Models for Controllable Review Generation with Frozen LLMs (Junichiro Niimi, TMLR 2026). A Deep Boltzmann Machine over binary review attributes, plus the MLP adapter that turns its mean-field beliefs into soft prompts for a frozen Qwen/Qwen2.5-0.5B-Instruct. Load it with the `boltzmann-gpt` package.

• —Paper: OpenReview
• —Code: https://github.com/jniimi/boltzmann-gpt

Everything this model generates is synthetic review text. It is not a real customer's opinion and it describes no real purchase. Do not post generations as genuine reviews; if you publish or redistribute them, state clearly that they are model output.

Usage

uv add git+https://github.com/jniimi/boltzmann-gpt

from boltzmann_gpt import AttributeModel

model = AttributeModel.from_pretrained("jniimi/boltzmann-gpt-smartphone")
model.attributes()                         # group -> allowed values

v = model.encode(brand="samsung", rating="5", price="Premium")
print(model.energy(v))                     # mean-field energy score, lower = more coherent
print(model.energy(model.clamp(v, price="Entry")))
print(model.generate(v, seed=0))           # renders via the frozen generator
print(model.generate(model.clamp(v, rating="1"), seed=0))

generate() downloads Qwen/Qwen2.5-0.5B-Instruct from the Hub on first use. Without a prompt= argument it uses the paper's one-shot prompt layout (paper, Appendix B): an instruction naming the domain ("smartphone"), one example review, and the task fields. The example review is synthetic and author-written, and the product is generic ("Unlocked Android Smartphone", $299.99); both live in config.json. Override them with generate(v, product_name=..., price=..., example={"product_name": ..., "price": ..., "review": ...}), where a partial example dict is merged over the default. The average rating is 3.0 for both example and product, because it was constant at 3.0 in every training prompt. Pass prompt= to supply the full text prompt yourself; model.default_prompt(...) returns the prompt generate() would use.

What is in this repository

The adapter reads the concatenation of all converged hidden-layer means (3584 dimensions) and emits 30 soft-prompt embeddings, which are prepended to the text prompt. The generator's weights are never modified. This is the seed-0 run reported in the paper; the other seeds are not released.

Attribute schema

Groups left out of encode() take their modal training value. The modal defaults are marginal modes taken group by group, so the default configuration as a whole is not a typical review (the topic and TF-IDF groups are empty, for instance) and its energy is higher than that of a typical training vector. For meaningful comparisons, start from a fully specified configuration.

Training data

Built from Amazon Reviews 2023, the Cell Phones and Accessories category (smartphones only; accessories excluded): verified purchases, English only, one review per user; 52,952 / 1,024 / 1,024 train / validation / test reviews. Attribute construction is described in the paper's preprocessing appendix. No review text or user data is included here.

Limitations

The DBM models how attributes co-occur in this one training domain. It is not a causal model: clamping an attribute fixes visible units in the learnt distribution and re-runs mean-field inference, so the resulting changes reflect model-internal consistency, not real-world effects. Generations inherit the biases of the review corpus and of the generator, and a small generator often produces repetitive or ungrammatical text.

License

The DBM and adapter weights in this repository are released under the MIT License. The frozen generator, Qwen/Qwen2.5-0.5B-Instruct, is not included and is distributed by its authors under the Apache License 2.0.

Citation

@article{niimi2026energybased,
    title = {Energy-Based Attribute Models for Controllable Review Generation with Frozen {LLM}s},
    author = {Junichiro Niimi},
    journal = {Transactions on Machine Learning Research},
    issn = {2835-8856},
    year = {2026},
    url = {https://openreview.net/forum?id=pOIFHY4dOJ}
}