CoolFace
Apppublic

DylYol/ai6122-yelp-ir

sourceHugging Facemitupdated 11mo agoView on Hugging Face
1likes
App README

AI6122 – Yelp IR Assignment

This starter repo should speed you up on the setup, let's focus more on enhancement and results.

  • —Dataset analysis (sampling, stemming comparison, style analysis)
  • —Search engine with Whoosh (keyword by business name) and similar-reviews (TF‑IDF + cosine)
  • —Trend analysis (sentiment over time)
  • —Application example (comparison sentence detector)
  • —One CLI entrypoint using Typer
Place the Yelp dataset files yelpacademicdatasetbusiness.json and yelpacademicdatasetreview.json in data/

How to Setup

bash
conda create -n ai6122 python=3.10 -y
conda activate ai6122
pip install -r requirements.txt

Decide your metro area

bash
python -m src.cli list-areas
# Example: "Saint Louis, MO"

Sample the area (creates sampled/*.jsonl)

this will generate sample files under sampled folder

bash
python -m src.cli sample --area "Saint Louis, MO"

Stemming analysis

this will generate different stemming figure under reports/figs

TODO: Silvia: kindly explore other stemming techniques for comparison

bash
python -m src.cli stemming

Style analysis

TODO: Silvia: kindly refer to note below

Aim for at least ~5–10 sentences per article so the stats are meaningful.

If ST is paywalled, you can still use two short ST pieces you can access; worst case, use two mainstream SG news articles with similar tone.

bash
python -m src.cli style

Build the index & search

TODO: Kein Yau: refer to notes below

1. Files are newline-delimited JSON (NDJSON) – streamed line-by-line for memory safety.

2. Memory: avoid storing huge text fields unless needed for snippets; if memory tight, store only review_text and IDs you display.

3. Latency: cache the parser; reuse the Searcher for multiple queries inside the same process.

bash
python -m src.cli build-index
python -m src.cli search-name --q "pizza"

Document granularity: one business = one document. All reviews for that business are aggregated into review_text at indexing time.

Index schema (Whoosh):

  • —name, categories, city – free-text fields
  • —review_text – aggregated reviews
  • —stars – numeric
bash
python -m src.cli search --q "ramen AND rich broth"

Find similar reviews search

TODO: Yee Kiat: kindly change stemming/stopwords or add Sentence-BERT for better similarity.

bash
python -m src.cli similar --b1 <BUSINESS_ID> --topn 10

Trend analysis

this will generate the trend figure under reports/figs

TODO: Yee Kiat: Can consider following recommendation and enhance trend/compare algorithm, whichever you think suit our project.

1. Stars vs sentiment: track both. People often ask if text sentiment correlates with star ratings.

2. Change-point detection: automatically flag structural shifts in the time series.

3. Aspect (topic) trends: split sentiment into aspects like staff, price, wait time.

bash
python -m src.cli trend --bid <BUSINESS_ID>
python -m src.cli compare --bid <BUSINESS_ID>

TODO: Kirubha and Dylan: based on current project structure and method in place, you should be able to start with the documentation. Once the enhancement is ready, you can update the documentation.

TODO: Kirubha and Dylan: with the working cli command, you should be able to build the simple GUI application.

Notes:

Please refer to the history file to check out the steps and output of each cli.

Run Yelp IR application locally

bash
streamlit run app.py