minastik-ai/llm-movielens
LLM-MovieLens A portable pipeline that turns a catalogue's structured metadata into LLM-synthesized item features, with the evaluation harness needed to find out whether they help. Instantiated on two catalogues that share no metadata source: 10,381 films in MovieLens 20M, and 9,289 books. Accompanies the ECIR 2027 Resource-track submission by Tan Nghia Duong and Manh Hoang Tran (School of Electrical and Electronic Engineering, Hanoi University of Science and Technology).… See the full description on the dataset page: https://huggingface.co/datasets/minastik-ai/llm-movielens.
LLM-MovieLens
A portable pipeline that turns a catalogue's structured metadata into LLM-synthesized item features, with the evaluation harness needed to find out whether they help. Instantiated on two catalogues that share no metadata source: 10,381 films in MovieLens 20M, and 9,289 books.
Accompanies the ECIR 2027 Resource-track submission by Tan Nghia Duong and Manh Hoang Tran (School of Electrical and Electronic Engineering, Hanoi University of Science and Technology). [PAPER_URL]
Where the code and the record live
- Code, harness and verifiers: <https://github.com/minastik-ai/llm-movielens>
- Archived record (DOI): 10.5281/zenodo.22766653 — the concept DOI, which resolves to the newest version
Everything on this page is data. Every script named below lives in the code repository above.
What is here
The identity ablation, and what it bounds
A fair question about any LLM-generated item feature is whether the text describes the metadata it was given or the model's prior knowledge of the item. The release answers it with a paired ablation you can inspect directly.
Each catalogue was generated twice: once normally, and once with the item's structured identity withheld — title, year, director and cast on MovieLens; title, developer and publisher on Steam — while every descriptive field was kept. Encoding both arms with bge-large-en-v1.5, we ask whether a profile written without the identity can still retrieve its own unmasked version.
MovieLens profiles are reproducible from the metadata this release ships. Steam profiles are not. That is the boundary condition of the approach: below roughly a hundred tokens of item metadata, a request for 80–120 words asks the model to supply most of the content itself.
Why 998 and not 1,000. Two games produced no masked profile at all. With identity withheld, the entire remaining input for Hot Dish is genres: none, tags: [Casual], specs: [Single-player], and for Mightier it is genres: [Indie], tags: [Indie], specs: [Single-player, Game demo]. The model declined to write, replying that the input was insufficient. Both generated normally with the title present. We report them as refusals rather than pressing the model past them — they are the boundary condition at its limit, and coercing a round number out of them would have hidden the clearest evidence in the experiment.
Read it with one caveat, which is in the paper too: this withholds structured identity. MovieLens free text still leaks identity — the plot names characters, and user tags carry actor names — so 94.1% means the title is redundant when the description already determines the profile, not that the model draws on no prior knowledge.
What is deliberately not here
No source material is redistributed. That means no MovieLens interaction data, no tag genome or any transformation of it, no genre strings, and no cached third-party item metadata. Two consequences worth knowing before you start:
- Benchmark splits are rebuilt, not downloaded. Get MovieLens 20M from GroupLens, then run
tools/rebuild_splits.pyfrom the companion code repository (linked above; it is not part of this dataset repository). It reconstructs the splits deterministically and verifies them against the SHA-256 of the files the paper's results were computed on — so a byte-exact match is provable rather than assumed. - Baseline features are regenerated too. The genome-PCA and title-embedding baselines are transformations of source data, so the harness rebuilds them.
Licences
Split, because the parts differ in what they derive from. human_eval/, metadata/ and all code are CC BY 4.0. profiles/ and embeddings/ are CC BY-NC 4.0: they are our output, but generated from sources whose own terms are non-commercial, and we cannot grant rights we do not hold. Research use is unrestricted under both. See LICENSE.
Maintenance and versioning
What is versioned. Every release is archived under its own DOI, and the concept DOI above resolves to whichever is newest. A result cited against a version DOI stays checkable against the exact bytes it was computed on — v1.0 is 10.5281/zenodo.22766654 — while a citation of the concept DOI follows the resource as it moves on.
What is maintained is the harness, not the file. A new sentence encoder, generating model or prompt is a path override rather than a fork -- a new model architecture is a class plus a branch in build_model, not a path. The pipeline reads whatever structured description a catalogue has, and the evaluation harness runs any feature through the same temporal split, five seeds and full-ranking protocol. That is the design decision that keeps this usable as models change, and the cross-provider regeneration reported in the paper is the evidence that it works rather than an intention.
How you can check it still holds. The one-command verifier recomputes every headline number in the paper from the released result files in about a minute without a GPU. That is deliberate: a resource whose claims can only be re-checked on a cluster stops being re-checked. SHA256SUMS covers every released file, so shasum -c SHA256SUMS tells you immediately whether what you have is what we published.
What we commit to. Corrections to the released files are published as a new version with the change recorded, never as a silent overwrite of an existing one. Issues and questions go to the contact below.
Citation
Cite the paper, not this repository — one canonical target keeps citations from fragmenting across the preprint, the archival DOI and the proceedings entry.
[CITATION_BIBTEX]
Contact
Questions, defect reports and errata: this repository's issue tracker. It is public, searchable and archived, and does not depend on any individual mailbox remaining active. Correspondence: the corresponding author at the institutional address on the paper.
Funding and interests
Funded in full by Minastik Technology JSC, which also hosts this artifact. T.N.D. is Chief AI Officer at, and holds an equity interest in, Minastik Technology JSC. The funder placed no restriction on publication or on the release of this benchmark. See the paper's Declarations section.
This product uses the TMDB API but is not endorsed, certified, or otherwise approved by TMDB.