CoolFace
Datasetpublic

vc940/business-entity-resolution-normalized

Business Entity Resolution: normalised records Normalised copies of the six source files of the ML Challenge 2026 Business Entity Resolution task (business records from three sources, US / India in train, plus France in test). The goal of the task is to find, for every Source 1 record, the Source 2 / Source 3 records that describe the same business. File Rows train_s1.parquet 2,206,821 train_s2.parquet 5,034,616 train_s3.parquet 5,285,603 test_s1.parquet 1,732… See the full description on the dataset page: https://huggingface.co/datasets/vc940/business-entity-resolution-normalized.

sourceHugging Faceupdated 8h agoView on Hugging Face
0likes
Dataset Card

Business Entity Resolution: normalised records

Normalised copies of the six source files of the ML Challenge 2026 Business Entity Resolution task (business records from three sources, US / India in train, plus France in test). The goal of the task is to find, for every Source 1 record, the Source 2 / Source 3 records that describe the same business.

FileRows
train_s1.parquet2,206,821
train_s2.parquet5,034,616
train_s3.parquet5,285,603
test_s1.parquet1,732,544
test_s2.parquet4,887,273
test_s3.parquet5,082,316

No ground-truth labels are included.

How the records were processed

Each record goes through four rule-based steps (no translation or language models, no external lookups):

  1. 1.Languages: Indian-script words (Devanagari, Tamil, Telugu, Kannada, Gujarati, Bengali, Malayalam, Oriya, Gurmukhi) are turned into English with a word dictionary learned from training pairs. Words the dictionary does not know go through a rule-based transliteration script.
  2. 2.Transformations: a learned address word map (st -> street, hwy -> highway, ...), Indian city renames (bombay -> mumbai), state / region forms (codes, native script, French departments -> regions), French street abbreviations (R -> rue, AV -> avenue).
  3. 3.Normalisation: names (alias split on DBA / formerly / aka, domains and hashtags, glued-word splitting, initials collapse A.B.C. -> abc, digit-for-letter fixes, legal forms split off) and addresses (tokens, state, house number, placeholders removed).
  4. 4.Typo fixing: Source 2 / 3 words not in the Source 1 vocabulary of the same country are replaced by the most frequent Source 1 word within 1-2 edits (_fix columns).

Version note: this upload was built with the first version of the rules, which used hand-written lists for US state names, French street words / regions and Indian city renames.

Columns

GroupColumns
Keys and raw textentity_id, source, country, name_raw, addr_raw
Name, language stepname_latin, n_indic_words, n_indic_fallback
Name, normalisedname_core, name_core_fix, name_alts, name_alts_fix, name_legal
Name flagsflag_alias, flag_domain, flag_id_tag, flag_junk_prefix, flag_initials, flag_leet
Addressaddr_norm, addr_norm_fix, state, house_number, addr_numbers, addr_blank, addr_placeholder, addr_indic
  • —name_core: the main name without legal forms or honorifics, space-separated tokens.
  • —name_alts: all alternative names separated by | (only set when the name contains DBA / formerly / aka ...).
  • —name_legal: canonical legal forms (inc, llc, ltd, pvt, sarl, ...).
  • —n_indic_fallback: number of Indian-script words the learned dictionary did not know.
  • —house_number: first number of the raw address, leading zeros removed.
python
import pandas as pd
s1 = pd.read_parquet("hf://datasets/vc940/business-entity-resolution-normalized/test_s1.parquet")