luc-prr904/ipid-corpus

A navigation bar on every page, above the fold

Home page carries the findings; a method page carries the method

Grow the thin markets: 405 -> 450 documents

A ceiling on the thing insured is not a cover limit

An excess that is stated but not quantified now reads "stated"

Probe templates for Allianz Partners and SquareTrade: 402 asked, 1 found

Cover limits, two amount-reader bugs, and 376 -> 405 documents

Second turn of the loop: notices for the brands that had none — 371 -> 376

Promote eight discovered intermediaries into the registry

ipid intermediaries: find an intermediary by the number it gives for itself

ipid links: the documents say where their own conditions are — 357 -> 371

Spain's DGSFP register: agency contracts, and no personal data on disk

Probe templates for Domestic & General and Chubb, and the utility channel in six markets

Following the register's leads: 340 -> 347 IPIDs, and what the register cannot say

FCA API: 24 firms, 226 relations, EE now under Chubb

FCA API client, with the hand-read pages as fallback

FCA register (UK): 48 appointed-representative relations, 12 confirmations

Registers (ORIAS, IVASS) and predictable-address probes: 309 -> 340 IPIDs

Brokers: SPB entities grouped, broker-first search, brokers table

Roles of intermediaries, read from IPIDs and the notices behind them

France depth pass: 275 -> 295 IPIDs (France 78 -> 98 documents)

Extractor 3.2.2: cover features, with an audited precision

Extractor 2.9.0: sections read reliably (what-is-insured 183 -> 266 of 275)

New markets: Austria and Portugal (252 -> 275 IPIDs)

Corpus growth, fourth pass (platform-first): 247 -> 252 IPIDs

Corpus growth, third pass: 234 -> 247 IPIDs (depth in IT, ES, BE)

Corpus growth, second pass: 222 -> 234 IPIDs

Corpus growth pass: 186 -> 222 IPIDs across all eight markets

Explorer: draw each document as brand -> platform -> insurer

Add the explorer page: network, coverage and insurer matrices, evidence table

Phase 2: extract parties and sections with evidence, score them, export a dataset

Deepen the thinnest markets; read encrypted PDFs; Spanish heading variant

Add Poland as a market and hand-search it

Deepen BE, NL, UK, IT, ES and DE with the platform-research method

Formalise the platform-research method: skill, registries, ipid parties

Add hand-searched candidates from Assurant, bolttech, Allianz Partners, Chubb and Neat

Add hand-searched candidates from SPB's white-label estate

Read robots.txt as RFC 9309 says, not as urllib.robotparser does

Add hand-searched French candidates for the lines the campaigns left thin

Stop reading a 403 on robots.txt as a refusal to crawl

Read six section headers as an IPID even with no keyword in the text

Stop Stage A discarding the documents it finds

Add unattended discovery campaigns, expand France, and fix four fetch and search bugs

Make discovery search the open web through DuckDuckGo, and refuse wrong-market documents

Use real contact details on the crawler page

Add IPID acquisition layer, project page and crawler information

initial commit