FrenchCastle/isora-tax-administration

ISORA — International Survey on Revenue Administration, FY2014–FY2024

Every published answer of every ISORA survey round, in one clean long-format panel, with the metadata you need to use it responsibly: what each question means in each questionnaire generation, which questions changed wording (or meaning) between rounds, which jurisdictions answered which question in which year, and how published values were revised between releases.

ISORA is the joint survey of national tax administrations run by the Asian Development Bank (ADB), the Inter-American Center of Tax Administrations (CIAT), the International Monetary Fund (IMF), the Intra-European Organisation of Tax Administrations (IOTA) and the OECD. It covers revenue collections, budgets and staffing, registration, filing and payment, arrears, audit and compliance risk management, dispute resolution, taxpayer services, digitalisation, governance and institutional arrangements. The IMF publishes the data through the ISORA Data Portal (isoradata.org), and this dataset is built from the IMF SDMX API that sits behind that portal.

Unofficial redistribution. This dataset is not produced or endorsed by the IMF, ADB, CIAT, IOTA, the OECD or any tax administration. The data remain subject to the ISORA Data Portal Terms and Conditions and the IMF Copyright and Usage policy — read the `LICENSE` file. You may publish ISORA data provided the source is acknowledged; see Citation.

Quick start

Easiest: the consolidated panel. One row per jurisdiction and fiscal year, 76 headline indicators with short names, plus region and World Bank income group. Values are only taken from questionnaire generations where the question is the same (see `panel_dictionary`).

import pandas as pd

panel = pd.read_parquet("hf://datasets/FrenchCastle/isora-tax-administration/data/panel.parquet")
panel[panel.fiscal_year == 2023].groupby("income_group_wb")["cost_of_collection_pct"].median()

Loader script. `isora.py` is a single file (pandas + huggingface_hub) with helpers for the common tasks; download it or copy it into your project:

import isora                                     # python isora.py --help for the CLI

panel = isora.panel()                            # consolidated panel
obs = isora.load("observations")                 # every published answer, long format
isora.search("arrears")                          # find codes by keyword
isora.describe("337_092")                        # definition + history of one code
wide = isora.wide(["337_001", "337_012"])        # jurisdiction x year table of chosen codes
fra = isora.series("80040_3", ["FRA"])           # tidy time series, money in base LCU

`datasets` library.

from datasets import load_dataset

panel = load_dataset("FrenchCastle/isora-tax-administration", "panel", split="train").to_pandas()
obs = load_dataset("FrenchCastle/isora-tax-administration", "observations", split="train").to_pandas()
ind = load_dataset("FrenchCastle/isora-tax-administration", "indicators", split="train").to_pandas()
hist = load_dataset("FrenchCastle/isora-tax-administration", "indicator_history", split="train").to_pandas()

# 1. Pick a question and look at its history across questionnaire generations first.
hist.set_index("indicator_code").loc["80040_3", ["label_2016", "label_2018", "label_2020plus", "comparability_flag"]]

# 2. Build a country × year table of one indicator (money already in base local-currency units).
net_revenue = (
    obs[obs.indicator_code == "80040_3"]
    .pivot(index="jurisdiction_code", columns="fiscal_year", values="value_local_currency_units")
)

# 3. Categorical answers: the closed list of options is in indicators.answer_categories.
ind.loc[ind.indicator_code == "80700", ["questionnaire_generation", "label", "answer_categories"]]

DuckDB works directly on the Parquet files:

SELECT jurisdiction_name, fiscal_year, value_numeric
FROM 'data/observations/*.parquet'
WHERE indicator_code = '337_001'      -- net revenue collected as % of GDP (derived by ISORA)
ORDER BY 1, 2;

About ISORA

ISORA collects tax administration data from national or federal tax administrations through an online platform administered by the IMF, using common questions and definitions agreed by the five partner organisations. Participation is voluntary; after collection the partners review the data for accuracy, completeness and consistency, then publish the finalised round. Participants in ISORA 2020 and later rounds agree in advance that all data they provide can be placed in the public domain; every row in this dataset carries the source flag PUBLIC_DATA = true.

Until 2021 the survey ran every two years and collected two fiscal years at a time. After ISORA 2018 the questionnaire was redesigned: a smaller annual core is asked every year and a larger periodic module (governance, human resources, compliance risk management, taxpayer services, tax operations) is asked every four years — it was included in ISORA 2023 (FY2022), which is why that year has roughly twice as many indicators as its neighbours.

The IMF exposes the published data as three SDMX dataflows, one per questionnaire generation. This dataset keeps that distinction because the question codes and wording differ between them:

Coverage by fiscal year:

The tables

panel and panel_dictionary

panel is the consolidated, analysis-ready view: one row per jurisdiction × fiscal year (1,730 rows) with 76 headline indicators as columns. Each column is fed by one source code per questionnaire generation, chosen so that the question is the same in every generation it draws from; where a code changed meaning between generations (for instance 337_015, a staffing share in 2018 and a capital-expenditure ratio in 2020+) only the matching generation is used. Money columns end in _lcu and are in base units of local currency (harmonised across rounds). Attribute columns: jurisdiction_code, jurisdiction_name, fiscal_year, survey_round, questionnaire_generation, imf_region, world_bank_region, income_group_wb (World Bank income group in force for that fiscal year, see Enrichment), income_group_wb_classification_fy, income_group_wb_current, member_oecd, member_eu, member_iota, member_ciat, member_adb.

panel_dictionary documents every column: unit, description, and for each generation the source indicator_code and its original label. Keep it next to any analysis — the questionnaire_generation column in panel tells you where a series crosses a questionnaire redesign, and level shifts at those points (for example France's revenue_to_gdp_pct moving from 17.5 in FY2017 to 25.7 in FY2018 as the definition of revenue reported changed) are real features of the source, not of this dataset.

observations

One row per jurisdiction × indicator × fiscal year. Keys are unique within each questionnaire generation and, because fiscal years do not overlap between generations, unique overall.

indicators

One row per questionnaire generation × indicator code (3,569 rows), i.e. the three source codelists flattened with every annotation the IMF attaches to a code, plus statistics computed from the observations. Key columns: label, display_label, description (rarely filled at source), form_code / form_name (the survey form, e.g. Form F. Operational metrics), question_ref (e.g. Form D - Q2, ISORA 2020+ only), section, topic_group / topic_subgroup (the IMF Indicators by Topic hierarchy, ISORA 2020+ only), report_table_index / report_table_title (where the code appears in the IMF review tables), indicator_type (the source's declared type: binary, count, currency, percent, nominal, ordinal, text, date, unspecified — unreliable, see caveats), observed_value_kind (inferred from data), answer_categories (the closed list of answers actually observed, most frequent first), is_derived / formula / numerator / denominator / legend (ISORA-computed ratios such as 337_001 net revenue as % of GDP), is_monetary, is_local_currency, label_mentions_thousands, is_periodic (periodic-module question), is_review_indicator, n_observations, n_observations_with_value, n_jurisdictions, fiscal_years_with_data, has_observations (codelists contain codes that were never published with data).

indicator_history

One row per indicator code (2,551 codes) describing how the code appears across the three questionnaire generations: in_2016 / in_2018 / in_2020plus, the label in each generation, label_changed_2016_to_2018, label_changed_2018_to_2020plus, similarity scores of the normalised labels (0–1), n_generations, fiscal_years_with_data, observation counts and a comparability_flag:

226 codes exist in all three generations, 566 in two, 1,759 in one. 183 codes changed wording between ISORA 2016 and ISORA 2018, 215 between ISORA 2018 and ISORA 2020+.

jurisdictions

One row per jurisdiction with data (182 rows): jurisdiction_code, jurisdiction_name, imf_numeric_code, IMF region / sub-region / regional technical-assistance centre, World Bank region and FY2015 income group as carried in the IMF codelist, the current World Bank region, income group and lending category (world_bank_region_current, world_bank_income_group_current, world_bank_lending_category_current, world_bank_code), WEO group, fragile / small-developing-state flags, membership flags (ADB, CIAT, IOTA, OECD, OECD Forum on Tax Administration, EU, G20, G7, WCO, WAEMU) as recorded in the IMF codelist, and participation computed from the data (fiscal_years_with_data, survey_rounds_with_data, in_isora_2016 / in_isora_2018 / in_isora_2020plus, n_observations).

coverage

One row per generation × indicator × fiscal year (6,752 rows) counting how many jurisdictions were published for that question in that year, split by value_status (n_value, n_not_available, n_not_applicable, n_empty, n_unrecognized_code, n_jurisdictions_reporting). This is the questionnaire matrix: it tells you which questions were asked (or at least published) in which year, and how well they were answered.

revisions

The IMF API still serves earlier published versions of the consolidated FY2018+ dataflow. Each version is the dataset as released after a survey round, so differences between versions are revisions of previously published answers. Three vintages were compared on every jurisdiction × indicator × fiscal year key they share (359,777 keys):

The table lists, in long format (one row per key × vintage), only the keys where something meaningful changed (42,552 keys), with a change_type:

Differences that are only formatting (thousands separators, float precision, rounding to the coarser published precision, encoding glitches, letter case) are not listed. Example of a real revision: Australia's 337_084 (on-time filing rate, CIT) for FY2022 was published as 72.88 in the 2023 and 2024 releases and as 68.68 in the 2025 release — the latter equals the FY2021 value of the earlier releases. Users who need "the value as first published" can rebuild it from this table; users who need "the latest view" should simply use observations.

Working with questions that changed over time

ISORA question codes are not stable identifiers of meaning across the three questionnaire generations. Three patterns occur:

1. 1.Same code, same question, new wording. 80250_3 is Non-tax revenue - Net in 2016 and 2018 and Net revenue collected by the tax administration (in thousands in local currency)-Non-tax revenue in 2020+. Comparable.
2. 2.Same code, narrower or broader question. 88360 is Administration pre-fills returns or assessments (2016, 2018) but Administration pre-fills PIT returns or assessments (2020+). 85710_268 is Other verification interventions - Total additional assessments… (2016), Automated audits - Total additional assessments… (2018) and Value of additional assessments raised from audits and verification actions… - Electronic compliance checks (2020+). Comparability is a judgement call.
3. 3.Same code, unrelated question. 92670 is Categories of third party information used to pre-fill returns - Other income - description (2018) and Description of tax deductible expenses that are prefilled in PIT tax returns and assessments (2020+). Not comparable.

Recommended workflow:

• —Start from indicator_history; filter comparability_flag == "label_stable" for series that can be stitched with little risk, and read both labels for label_changed codes.
• —Join observations to indicators on (questionnaire_generation, indicator_code) so each value carries the definition that applied when it was collected. Never join on the code alone.
• —Use coverage to see in which years a question was actually asked; the periodic module (indicators.is_periodic) only has data for FY2022 within the consolidated generation.
• —Within ISORA 2020+ the questionnaire is stable across FY2018–FY2024 (the same codelist version is published for all seven years); the revisions table shows which earlier answers were revised in later rounds.
• —The ISORA 2016 → ISORA 2018 transition is smoother (605 shared codes, mostly same questions) than ISORA 2018 → ISORA 2020+ (a redesigned, much shorter questionnaire).

Units and currency

• —Money is in the jurisdiction's own currency and is not converted. indicators.is_local_currency marks national-currency questions. Derived ratios (337_*, 398_*, 111_*) are unit-free.
• —The three generations publish money differently. ISORA 2016 and 2018 published amounts in thousands (as asked on the form) with SCALE = 0. The consolidated ISORA 2020+ dataflow publishes the same questions already multiplied out to base currency units and marks them with SCALE = 3 (verified against GDP: France's 398_001 for FY2022 is published as 2 638 008 000 000 with SCALE = 3, i.e. EUR 2.64 trillion; the ISORA 2023 release had published 2 638 008 000, in thousands). Do not multiply ISORA 2020+ values by 1 000.
• —value_local_currency_units removes the ambiguity: it is always base units (45,359 rows converted from thousands, 39,451 rows taken as published). value_numeric stays exactly as published for traceability.
• —Counts (staff, taxpayers, returns), percentages and ratios are published as-is.

What was changed relative to the source (transformation notice)

Values were not altered. The following was done, and is reversible through value_raw:

1. 1.Three SDMX dataflows were stacked into one long table with a common schema; the dataset-level and series-level attribute rows of the SDMX-CSV were dropped.
2. 2.Numeric IMF jurisdiction codes (ISORA 2016/2018) were mapped to the alpha-3 codes used by the consolidated dataflow, via the ISO annotation of the IMF codelist (Kosovo: 967 → KOS, the IMF's current code; the 2018 codelist annotated it UVK).
3. 3.The mixed-type OBSERVATION string was split into value_numeric / value_text / value_status. D → not_available; Not Applicable / N/A → not_applicable; P → unrecognized_code; digit strings with space grouping (163 310 020) → number.
4. 4.Text answers and footnotes: HTML fragments such as <br/> and entities removed, whitespace collapsed, and 64 values plus 141 footnotes with double-encoded UTF-8 (‘ → ‘, Türkiye → Türkiye) repaired.
5. 5.Monetary harmonisation (monetary_unit, value_local_currency_units) as described above.
6. 6.Indicator metadata flattened from SDMX annotations; declared types normalised (Counting → count, trailing spaces removed); observed value kinds, answer categories, coverage, cross-generation history and inter-release revisions computed.
7. 7.Jurisdiction attributes taken from the IMF CL_ISORA_ISO_COUNTRY codelist; Yes/No flags converted to booleans.
8. 8.World Bank income classifications joined (see Enrichment); the consolidated panel built from the curated crosswalk in panel_dictionary. The four derived expenditure aggregates 337_176–337_179, published in thousands with SCALE = 0, are flagged as such and harmonised like the other money questions.

Nothing was imputed, interpolated, deduplicated or filtered out.

Caveats and known issues in the source

• —Self-reported, voluntary. Answers are provided by the administrations and reviewed by the partners, but definitions are applied locally; read the ISORA guide before comparing countries.
• —`D` and `P`. 113,785 cells are D — the ISORA convention for no data available on a numeric question, distinct from a question that was skipped. 1,526 ISORA 2016 cells contain P, a code that does not appear in the surviving documentation; it occurs only on numeric questions and is treated as missing (unrecognized_code).
• —Zeros that mean "missing". The ISORA 2016 derived ratios (codes 10010–10790) publish an exact 0 when one of their inputs was not reported (107 of the 262 FY2014–15 revenue-to-GDP values are 0 although revenue was reported and GDP was not). In observations these zeros are kept as published; in panel they are set to null only for the four columns where zero is impossible (zeros_treated_as_missing in panel_dictionary). Treat other exact zeros in FY2014–FY2015 ratio columns with suspicion.
• —Declared types are unreliable. In the 2020+ codelist 817 of 1 098 codes have no declared type and several count questions (Total number of returns received - CIT) are typed currency. Use indicator_value_kind / observed_value_kind, which are inferred from data.
• —Scale attribute inconsistency between generations (see Units). The label_mentions_thousands flag exists because in ISORA 2016/2018 the unit is only stated in some labels.
• —Categorical answers are not fully harmonised at source: InPlace and In Place, Implmenting and Implementing, option a) with a stray <br/>, leading spaces in ISORA 2016 answers. Cleaning removed markup and whitespace but did not merge spellings.
• —Codelists include codes without data (283 in 2016, 116 in 2018, 397 in 2020+): questions suppressed from publication or never asked. indicators.has_observations flags them.
• —Fiscal years are the administrations' own fiscal years and do not align across countries.
• —Combined tax-and-customs administrations sometimes report total staff or expenditure for both functions (the IMF notes this on the staff tables).
• —World Bank income groups in jurisdictions are the FY2015 classification stored in the IMF codelist. Join current classifications yourself if you need them.
• —Revisions: the observations table is the latest published view (ISORA 2025 release). If you compare with figures quoted in older IMF/OECD publications, consult revisions.
• —Territorial names follow IMF practice (e.g. China, P.R.: Hong Kong, Taiwan, Kosovo, Republic of, Türkiye, Rep of) and are without prejudice to the status of any territory.

Enrichment: World Bank income groups

Two columns were added from the World Bank's Country and Lending Groups classification (datahelpdesk.worldbank.org/knowledgebase/articles/906519):

• —jurisdictions.world_bank_income_group_current (and region, lending category): the current classification file (CLASS.xlsx, FY27 edition, based on 2025 GNI per capita).
• —panel.income_group_wb: the historical classification (OGHIST.xlsx) aligned so that fiscal year Y gets the World Bank group computed from year-Y GNI per capita, i.e. World Bank fiscal year FY(Y+2), shown in income_group_wb_classification_fy. 1,692 of 1,730 jurisdiction-years are matched.

177 of the ISORA jurisdictions have a World Bank classification; AIA, COK, MSR, NIU, SRP are territories the World Bank does not classify. Code differences are mapped (KOS → XKX, UAE → ARE). The World Bank publishes these files under its open data terms (Creative Commons Attribution 4.0, worldbank.org/data-terms); the FY2015 income group carried in the IMF codelist is kept for reference.

Provenance and reproducibility

Everything comes from the public IMF SDMX API (https://api.imf.org/external/sdmx/3.0, agency ISORA), retrieved on 2026-09-20T04:42:35Z:

Structures used: DSD_ISORA_PUBLISHED 1.0.0 / 2.0.0 / 6.0.0 with their codelists (CL_INDICATOR 1.0.2, CL_ISORA_TAX 1.0.3 and 6.0.6, CL_COUNTRY, CL_JURISDICTION 4.8.4, CL_ISORA_ISO_COUNTRY 2.0.1), the hierarchies H_CL_INDICATORS_BY_TOPIC 2.2.0, H_CL_PERIODIC_INDICATORS 2.0.0, H_CL_DERIVED_INDICATORS 2.0.0, H_CL_REVIEW_INDICATORS 2.1.0 and the label codelist CL_RAFIT_LABELS. Dataset-level attributes (license URL, citations, publication dates) were read from the SDMX 2.1 CSV endpoint and are stored in metadata/build_summary.json.

The full pipeline (download, transformation rules, unit tests, this card's template) is in `pipeline/` and is MIT-licensed; python -m isora_hf.sdmx_client && python -m isora_hf.build && python -m isora_hf.card rebuilds the dataset from scratch. Re-running it after the next ISORA release (expected mid-2027 for FY2025) is how this dataset will be updated.

Official documentation — questionnaires, completion guides, review and derived tables per round — is in the ISORA Documents Catalog]) (the files are served through the portal's download buttons and are not mirrored here). The IMF also publishes analytical reports on each round (ISORA 2016: Understanding Revenue Administration, 2019; ISORA 2018: Understanding Revenue Administration, 2021; ISORA 2023: Tax Administration: Performance and Practices, 2026), and the OECD's annual Tax Administration series is built on the same data for 58 jurisdictions.

Citation and acknowledgement

Any publication using these data must acknowledge the source. The citation requested by the publisher (from the source metadata) is:

The International Survey on Revenue Administration (ISORA). http://isoradata.org. Accessed on [date].

Full source citation:

The Asian Development Bank (ADB), the Inter-American Center of Tax Administrations (CIAT); the International Monetary Fund (IMF); the Intra-European Organisation of Tax Administrations (IOTA); and the Organisation for Economic Co-operation and Development (OECD), International Survey on Revenue Administration: https://ISORADATA.ORG

If you also want to credit this cleaned redistribution:

@misc{isora_hf_2026,
  title        = {ISORA -- International Survey on Revenue Administration, FY2014--FY2024 (cleaned redistribution)},
  howpublished = {Hugging Face dataset \url{https://huggingface.co/datasets/FrenchCastle/isora-tax-administration}},
  year         = {2026},
  note         = {Unofficial restructuring of data published by the IMF on behalf of ADB, CIAT, IMF, IOTA and OECD through the ISORA Data Portal (https://isoradata.org). Data subject to the ISORA Data Portal Terms and Conditions.}
}

Licence

license: other — ISORA Data Portal Terms and Conditions of Data Access and Use plus the IMF Copyright and Usage policy, reproduced in `LICENSE`. In short: you may use and publish the data with appropriate acknowledgement of the source; the data are provided as is, without warranty; you indemnify the partner organisations against third-party claims arising from your use; the partner organisations' immunities are preserved; contact copyright@imf.org for commercial reuse questions. The pipeline code is MIT-licensed.

Dataset version

• —1.1.0 (2026-09-20): added the consolidated panel and panel_dictionary tables, the isora.py loader, current and per-year World Bank income groups, and the thousands flag on the derived expenditure aggregates.
• —1.0.0 (2026-09-20): first release, built from ISORA_2016_DATA_PUB 2.0.0, ISORA_2018_DATA_PUB 2.0.0 and ISORA_LATEST_DATA_PUB 5.0.0 (ISORA 2025 release, FY2024 data published June 2026).