open-index/umi-golden

umi golden corpus Ten thousand real web pages, kept so that HTML extraction can be proved to produce byte identical output on every machine that runs it. This is not a training corpus. It is a test fixture. The crawler it belongs to is umi, which lets people outside the project run fetchers, and the only thing that makes a fetcher somebody else runs trustworthy is that its extraction of a page can be compared against everyone else's byte for byte. A one in ten thousand… See the full description on the dataset page: https://huggingface.co/datasets/open-index/umi-golden.