KSU-HW-SEC/food-science-llm-protocol

Food Science LLM Text-Mining Protocol

Pipeline and derived data accompanying:

Guo X, Fu W. Data Mining and Text Mining Using Large Language Models. In: Li Y, Zhang D, Guo Z (eds), AI in Food Science: Methods and Protocols. Methods and Protocols in Food Science. Springer.

The chapter prints one protocol as 26 numbered steps with abbreviated code listings. This repository is the executable form of that protocol. Every step has a corresponding function here, and every number in the case study is produced by running it.

No result in this repository is typed by hand. The figures are drawn from pipeline output, and case_study.tex regenerates the LaTeX macros that carry the counts, so prose and data cannot drift apart.

What the pipeline does

Given a PubMed query, it retrieves abstracts, splits them into sentences, extracts food-science entities and the relations between them with an LLM, builds a knowledge graph, clusters food sources by bioactive profile, and evaluates a retrieval-augmented question-answering system against a no-retrieval control.

Installation

Python 3.12 is required. spaCy has no prebuilt wheels for 3.13 at the time of writing and building it from source fails there.

python3.12 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
pip install https://s3-us-west-2.amazonaws.com/ai2-s2-scispacy/releases/v0.5.4/en_core_sci_lg-0.5.4.tar.gz
cp .env.example .env      # then fill in any keys you need

LLM endpoint

Any OpenAI-compatible endpoint works. The choice lives under llm: in the run configuration. The default is a local Ollama server, which makes the protocol runnable at no API cost:

llm:
  model: qwen2.5:72b-instruct
  base_url: http://localhost:11434/v1
  api_key: ollama
  workers: 8          # concurrent requests

To use the OpenAI API, set model: gpt-4o-mini, drop base_url, and put OPENAI_API_KEY in .env. Nothing else changes. This is the substitution described in Note 3 of the chapter.

Running it

python -m foodlm.cli --config config/bioactive_peptides.yaml retrieve
python -m foodlm.cli --config config/bioactive_peptides.yaml preprocess
python -m foodlm.cli --config config/bioactive_peptides.yaml ner
python -m foodlm.cli --config config/bioactive_peptides.yaml normalize
python -m foodlm.cli --config config/bioactive_peptides.yaml relations
python -m foodlm.cli --config config/bioactive_peptides.yaml index
python -m foodlm.cli --config config/bioactive_peptides.yaml rag-eval --qa eval/qa_pairs.jsonl
python -m foodlm.cli --config config/bioactive_peptides.yaml mining
python -m foodlm.cli --config config/bioactive_peptides.yaml stats
python -m foodlm.cli --config config/bioactive_peptides.yaml figures

or ... all to run every stage in order. config/smoke.yaml is the same pipeline over 40 abstracts, for checking the environment before a full run.

Every LLM stage is resumable and cached. Responses are cached on disk by (model, prompt), and the extraction stages skip work already recorded in their output files, so an interrupted run restarts for free and re-running a completed stage costs nothing.

To measure what the Step 6 keyword pre-filter discards, as Note 5 advises:

python -m foodlm.cli --config config/bioactive_peptides.yaml filter-recall --sample 100

Where the outputs go

data/<run-name>/
  corpus.csv                 retrieved abstracts
  corpus_meta.json           query, retrieval date, hit counts   (Note 4)
  sentences.jsonl            segmented sentences + filter decision
  entities.jsonl             grounded entity mentions
  entities_normalized.jsonl  mentions with a canonical label attached
  canonical_map.json         synonym to canonical mapping; hand edits win
  relations.jsonl            every classified pair, kept or not, with its status
  cooccurrence.csv           food source x biological activity matrix
  mining.json                graph, clustering and classification results
  rag_eval.json              per-question RAG vs direct-query outcomes
  case_study_stats.json      every number quoted in the case study
  case_study.tex             the same numbers as LaTeX macros

Reference run

RESULTS.md reports a complete run, regenerated by python scripts/render_results.py.

Treat it as one iteration, not a fixed benchmark. The output depends on the model, its version, the query, and the state of PubMed on the day it ran. Model behaviour moves quickly, and a run six months from now will not reproduce these counts. The pipeline therefore records the model, endpoint, query and retrieval date alongside every result. What carries across runs is the shape of the findings, not the digits.

Three findings are worth reading before you trust your own run.

• —Self-reported confidence carries almost no signal. 96% of relation predictions came back labelled high, and the confidence filter the protocol suggests removed nothing. The filter that worked was checking the model's own supporting quotation against the sentence. About a fifth of high-confidence predictions cited evidence that was not there.
• —Normalisation is load-bearing, not housekeeping. Raw extraction produced about four thousand distinct BIOLOGICAL_ACTIVITY strings for under ten thousand mentions. That fragmentation makes clustering and classification meaningless, and it inflates apparent model disagreement. Measured on raw surface forms, two models appeared to agree on 0.27 of mentions; measured on canonical forms, 0.64. Most of the gap was spelling.
• —Two strong models agree on about two thirds of what they extract. gpt-4o-mini and llama-3.3-70b reached a span-and-type F1 of 0.64 on the same sentences. Agreement is consistency, not correctness, but it bounds how far any single run should be trusted, and it puts a number on the chapter's cross-model validation advice.

Where this departs from the printed listings

The chapter's listings are written for readability in print. The differences below exist because the printed version does not survive contact with a real corpus.

1. 1.The PubMed query is tuned, and its hit count is recorded. Phrase queries are brittle. Requiring exact [tiab] phrases from two separate lists in the same abstract can shrink a result set by an order of magnitude, and PubMed does not stem inside quotation marks, so singular and plural forms retrieve different records. The query in config/bioactive_peptides.yaml keeps the peptide requirement strict but accepts either a food-source phrase or a food-related MeSH heading as the food anchor. Check your own hit count before building on a query. This run's query, date and count are in corpus_meta.json.

1. 1.Entity offsets are recomputed, not trusted. Step 7 asks the model for character offsets. Models get offsets wrong far more often than they get spans wrong, so ner.ground() locates each span itself and discards any entity whose text does not occur literally in the sentence. This is the cheapest hallucination filter in the pipeline.

1. 1.Relation extraction is restricted and batched. Only entity-type combinations that some relation in Table 1 can connect are sent to the LLM, and pairs sharing a type combination are classified several per call. Food sources are still linked to activities through the mediating compound, which is what Table 1 licenses. A predicted triple is dropped if it violates the type schema or if its evidence quote does not occur in the sentence.

1. 1.Umbrella compounds are excluded from the food/activity join. Terms such as bioactive peptide and bioactivity attach to hundreds of food sources and to every activity. Joining through them yields the product of the two sets rather than any association: in this corpus the ten broadest such terms generated 98% of all pairs, giving a matrix with near-identical rows. A compound above a configurable breadth is treated as a category label.

1. 1.The number of clusters is chosen, not fixed. Step 23 sets k = 6. Here k is selected by silhouette score over a range, so the grouping is a property of the data rather than of the protocol.

1. 1.Step 10 is a stage, not a line. The protocol describes normalisation in one clause. foodlm/normalize.py implements it as two passes, deterministic rules followed by LLM synonym grouping, writing an inspectable canonical_map.json that hand edits override.

1. 1.Inline markup is stripped at ingestion. 31% of the abstracts in this corpus contain <i>, <sub> or <sup> tags. Left in place they end up inside entity spans (<i>lactis</i> bd17) and fragment the graph.

1. 1.Louvain comes from networkx. networkx 3.0 and later ship the Louvain method, so the separate python-louvain package is no longer needed.

Data and licensing

MIT licensed (see LICENSE).

These labels are model output, not ground truth. They were produced by an LLM and filtered by the checks described above, not verified by a domain expert. Use them as a starting point, a weak-supervision source, or a reproduction target, not as an evaluation benchmark.

Abstract text is not redistributed. PubMed abstracts carry publisher copyright that varies by journal, so corpus.csv and sentences.jsonl are not published. What is here is the derived output: PMIDs, extracted entities and relations, the co-occurrence matrix and the aggregate results. Running the retrieve stage with the recorded query rebuilds the source text. PubMed is continuously indexed, so a later retrieval returns a superset of these records.

Republishing

huggingface-cli login
python scripts/publish_hf.py --repo <user>/<name> --dry-run   # review first
python scripts/publish_hf.py --repo <user>/<name>

The uploader excludes abstract text, the vector store, the response cache and the print-resolution TIFFs. Everything else is published together: pipeline source, configurations, extractions and results, so that reading a number and reading the code that produced it are the same visit.

Tests

python tests/test_units.py

These cover the parts that need neither a network nor a GPU: JSON recovery from malformed model output, entity grounding, normalisation rules, the keyword filter, record cleaning and the relation type schema.

Citation

@incollection{guo2026datamining,
  author    = {Guo, Xiaolong and Fu, Weimin},
  title     = {Data Mining and Text Mining Using Large Language Models},
  booktitle = {AI in Food Science: Methods and Protocols},
  series    = {Methods and Protocols in Food Science},
  publisher = {Springer},
  year      = {2026}
}