CoolFace
Datasetpublic

V4ldeLund/faroese-torshavn-fo

Tórshavnar kommuna (torshavn_fo) Status: Standalone website collection. It is not part of Faroese Dynaword. About this data Council records, budgets, annual accounts, municipal rules, plans, guidance and municipal web pages. Source: Tórshavnar kommuna Size Measure Value Documents 3,230 Characters 60,332,912 Tokens 26,695,185 Possible official documents 270 Tokens in possible official documents 1,099,125 File type… See the full description on the dataset page: https://huggingface.co/datasets/V4ldeLund/faroese-torshavn-fo.

sourceHugging Faceupdated 1mo agoView on Hugging Face
0likes31downloads
Dataset Card

Tórshavnar kommuna (torshavn_fo)

Status: Standalone website collection. It is not part of Faroese Dynaword.

About this data

Council records, budgets, annual accounts, municipal rules, plans, guidance and municipal web pages.

Source: Tórshavnar kommuna

Size

MeasureValue
Documents3,230
Characters60,332,912
Tokens26,695,185
Possible official documents270
Tokens in possible official documents1,099,125
File typeDocuments
HTML1,841
PDF1,389
DOCX0
CSV0
plain text0

The token count uses the AI-Sweden-Models/Llama-3-8B-instruct tokenizer. It counts all collected text, including other languages and repeated website text.

Files

  • —rows.parquet contains the collected documents.
  • —parser/ contains the crawler, site settings and Python requirements.

Crawl details

  • —Crawl started: 2026-08-15
  • —Crawl completed: 2026-08-15

The crawler used public sitemaps and links from the website. It stayed within the websites and file hosts listed in parser/sites.csv and followed robots.txt.

Text was taken from web pages, text-based PDFs, Word files and CSV files. Scanned PDFs and images were not read with OCR. Very short pages and exact duplicate texts were left out.

Columns

ColumnMeaning
source_site, source_homepageWebsite name and homepage
source_url, requested_urlFinal URL and the URL first requested
title, text, content_typeDocument title, extracted text and file type
retrieved_atWhen the document was downloaded
text_sha256, text_charactersText hash and character count
rights_status, rights_basisAutomatic rights label and why it was assigned
extraction_source_url, source_page_urlExtra source links when available

Rights

Some laws, decisions, notices and similar public documents may be outside copyright under section 9 of the Faroese Copyright Act.

likely_official_document only means that the title or URL matched a list of words. It is not a legal classification.

Note for this site

Council records, municipal rules, approved budgets and final accounts are the clearest candidates. News, reports, public submissions and outside attachments may still be copyrighted.

Known limits

  • —The crawl may not cover every page or old document.
  • —The text has not been checked by hand or filtered by language.
  • —Menus, headers and other repeated website text may remain.
  • —Text from PDFs may appear in the wrong order.
  • —Scanned PDFs and text inside images are missing.
  • —Some pages may contain personal information.

Parser

The crawler used for this website is included in the parser folder. The copy of sites.csv contains only this website.

bash
cd parser
python -m pip install -r requirements.txt
python scrape.py --site torshavn_fo