sdhwroot/rental-scout

Rental Scout

A research agent that finds rentals in a target area, enriches each listing with neighbourhood intelligence — nearest school & shops, every creche within 5 km (with ratings, reviews and peak-hour drive times), and SAPS crime stats for the suburb — then renders a flip-through HTML report. A modern web form (mint/teal/magenta pastel theme) passes your search brief to the agent.

Neighbourhood intelligence

Each property card shows:

• —Nearest school & shops — from OpenStreetMap (free, no key).
• —Creches within 5 km — all of them, nearest first, each with walk distance and, when a Google key is set, a star rating, review count, a top review, and the peak-hour driving time from the property.
• —Area safety — real SAPS crime stats for the suburb's police precinct (murder, assault, burglary, vehicle crime, etc.) with a per-100k rate.

Two optional one-time setups light these up:

# 1) Crime stats — build the compact precinct dataset (real SAPS data, ~once)
python scripts/build_crime_data.py        # writes data/crime_by_precinct.json (~0.6 MB)

# 2) Creche reviews + peak-hour traffic — set a Google Maps Platform key
export GOOGLE_MAPS_API_KEY=...            # enable Places API + Distance Matrix API

Both degrade gracefully: no crime file → the safety panel is omitted; no Google key → creches still list (distance/walk) without ratings or drive times. Crime is matched by suburb name (no coordinates needed); see crime.py for the suburb→precinct aliases.

┌──────────────┐   POST /search   ┌──────────────────────┐   render   ┌──────────────┐
│  form.html   │ ───────────────► │  rental_agent.run()  │ ─────────► │  report.py   │
│ (search form)│                  │  scrape → geocode →  │            │ (flipbook)   │
│              │ ◄─────────────── │  amenities (OSM)     │            │              │
└──────────────┘   { url }        └──────────────────────┘            └──────────────┘
       ▲                                   app.py (FastAPI)                    │
       └──────────────────────  GET /report/{id}  ◄─────────────────────────────┘

Quick start

./run.sh              # sets up .venv, installs deps, serves http://127.0.0.1:8000
./run.sh 8123         # custom port

Then open the URL, fill in the brief, and hit Run search →. Start with the Sample data source — it uses built-in Cape Town listings and needs no network.

Manual setup, if you prefer:

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn app:app --reload      # http://127.0.0.1:8000

Note: this project targets Python 3.11+. run.sh auto-detects a healthy interpreter; to force a specific one, set PYTHON=/path/to/python3 ./run.sh.

Configuration (API keys)

Copy .env.example to .env and fill in your keys — the app auto-loads .env on startup (config.py). Real environment variables (e.g. set in Render) always override the file, and .env is gitignored so keys never get committed.

cp .env.example .env   # then edit

No fake data in production. APP_ENV=production removes Sample from the form and rejects source=sample server-side. Nothing else falls back to fabricated listings — a live source that fails returns an error, never sample data.

Files

Data sources

Selected via the Data source control on the form:

• —Sample — built-in Cape Town listings. No network, great for a quick look.
• —All sites — aggregates every live portal (Property24 + Private Property), merges the results, and de-duplicates the overlap (the same home is often posted on both). A source that fails is skipped, not fatal — you still get the others. Each listing in the report is labelled with the portal it came from.
• —Property24 — live scrape of property24.com.
• —Private Property — live scrape of privateproperty.co.za.
• —JSON file — load listings you already collected (Apify / ScrapFly / an export). Provide a path to a JSON array of listing objects (see JSONScraper in rental_agent.py for the accepted fields).

Each live portal works the same way: type any SA area (e.g. Burgundy Estate, Sea Point, Blouberg) and the scraper

1. 1.resolves it to that portal's numeric location ID via the site's own autocomplete API;
2. 2.requests the canonical /to-rent/.../<id> page (both portals redirect an id-only URL to the right slug, so nothing is guessed);
3. 3.parses the listing tiles (price, beds/baths, size, photos, link), keeping residential rentals only.

If a plain HTTP request is blocked, the scraper transparently retries through a headless browser when Playwright is installed (optional):

pip install playwright && playwright install chromium

Adding another portal

Subclass BaseScraper, implement fetch(params) -> list[Property] (set p.source), and drop it into the AggregateScraper(...) list in app.py. The Property24Scraper / PrivatePropertyScraper pattern — resolve area → id, fetch id-redirect page, parse tiles — is the template.

Geocoding uses OpenStreetMap Nominatim and amenities use the Overpass API — both free, no API key. Please respect their usage policies (the agent rate-limits itself with a polite delay).

Command line

The agent also runs standalone, without the web form:

python rental_agent.py "Sea Point, Cape Town" --source json --json listings.json \
  --max-price 25000 --min-beds 2 -o report.html