aiacademy-kg/house_kg_full_dataset_frames

house.kg — Kyrgyzstan Real Estate, over time

Sale and rental listings scraped from house.kg, the largest real-estate board in Kyrgyzstan, re-measured on a schedule. Field names are English; values are kept in the original language (Russian), exactly as the site renders them.

Coverage: 2026-09-08. This is the baseline snapshot; later runs append new partitions.

Subsets

How the time dimension is organised

Three tables, three different questions:

listing_observations is a panel: every live listing is measured on every run. changes is the derived event log — cheaper to scan when you only care about price cuts, bumps or delistings.

from datasets import load_dataset

panel   = load_dataset("<repo>", "listing_observations", split="train")
changes = load_dataset("<repo>", "changes",             split="train")
ads     = load_dataset("<repo>", "listings",            split="train")

# every price cut, most recent first
cuts = changes.filter(lambda r: r["field"] == "price_usd"
                      and float(r["new_value"]) < float(r["old_value"]))

Each subset is one config holding many Parquet files, so load_dataset returns the whole history concatenated — filter on snapshot_id to slice it.

Relations

listings.house_kg_id            <- listing_observations.house_kg_id   (panel)
listings.house_kg_id            <- changes.entity_key   (where entity_type='listing')
listings.author_user_id         -> users.user_id        (private sellers only)
listings.company_slug           -> companies.slug
listings.complex_slug           -> complexes.slug
reviews.subject_slug            -> companies.slug | complexes.slug
photo_index.listing_id          -> listings.id      (metadata only, ~10 MB)
photos.listing_id               -> listings.id      (with embedded images)
listing_observations.snapshot_id-> snapshots.snapshot_id

listings.id is a uuid5 of `house_kg_id`, and review_id is a content hash — both are reproducible, so ids are stable across snapshots and joins survive a re-crawl.

Read before you analyse

• —Check `snapshots.complete` first. A run that was interrupted covers only part of the board. Delistings are not recorded for such a run (see refresh.min_completeness in the scraper), but its observations are still partial — treat an incomplete snapshot as a gap, not as a market movement.
• —A missing row is not a delisting. A listing is delisted when it has an observation with is_active = false; after that it simply stops appearing.
• —`views` and `favourites` only ever grow, so they are recorded in the panel but deliberately not tracked in changes — otherwise every listing would be "changed" on every run. favourites is null when the site rendered no counter.
• —Prices are not comparable across deals. A sale price is a total; a rent price is a rate. Always filter on price_period (total / month / day).
• —*A bump is a drop in age, not a change in `upped_date`.* The site renders relative dates ("2 месяца назад") which are resolved against the moment of reading, so an untouched listing's upped_date slides forward with the clock. Only a real bump makes an advertisement younger, so that is what changes reports — but the raw upped_date in the panel still carries the drift, so compare ages there too rather than differencing the timestamps yourself.
• —`offer_type` vs `seller_type`. The first is what the seller claims; the second is what their account is. Disagreements are flagged by seller_mismatch.
• —Photos are fetched once per listing and carry the snapshot_id that fetched them — they are not re-downloaded when a listing changes.
• —The board is Bishkek-centric: ~92% of listings are in Chui/Bishkek.
• —Reviews are capped at 20 per entity by the site itself — compare reviews_count with reviews_scraped and the reviews_truncated flag.

The full field-by-field guide is in DATASET_GUIDE.md.