Datasetpublic

overthelex/crossjur-citation-coupling

Cross-Jurisdictional Legal Citation Coupling — Ukrainian data & tools Reproducibility artifacts for the paper "Bibliographic Coupling as a Cross-Jurisdictional Evaluation Framework for Legal Case Retrieval: Civil-Law and Common-Law Citation Graphs Compared." The paper compares a civil-law citation graph (Ukrainian, from the EDRSR state court-decision registry) with a common-law one (Canadian, from the COLIEE 2026 Task 1 corpus) under a single coupling / co-citation / temporal… See the full description on the dataset page: https://huggingface.co/datasets/overthelex/crossjur-citation-coupling.

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

0likes58downloads

Dataset Card

Cross-Jurisdictional Legal Citation Coupling — Ukrainian data & tools

Reproducibility artifacts for the paper "Bibliographic Coupling as a Cross-Jurisdictional Evaluation Framework for Legal Case Retrieval: Civil-Law and Common-Law Citation Graphs Compared."

The paper compares a civil-law citation graph (Ukrainian, from the EDRSR state court-decision registry) with a common-law one (Canadian, from the COLIEE 2026 Task 1 corpus) under a single coupling / co-citation / temporal methodology. This repository holds the tools and the Ukrainian-side derived data so the Ukrainian results can be reproduced and the numbers cited in the paper can be traced to their source.

The Canadian (COLIEE Task 1) texts are not redistributed here — they are governed by the COLIEE data memorandum. The same tools reproduce the Canadian figures from the COLIEE corpus once its licence terms are met.

Update, 25 July 2026

Two changes, both prompted by a re-check of the coupling metrics after the ICTIR '26 framework paper appeared (doi:10.1145/3805713.3820412).

Correction. Earlier versions of the paper stated that Cite(d) was available over the whole Ukrainian pool while the COLIEE side had it only for the labelled query cases, and that the Canadian figures were therefore the only lower bound. That is true of the extraction but not of the outcome: of the 2,217 pool documents with a non-empty citation list, 2,000 are the query cases, and only 3 of the 3,650 pure distractors carry one. Both coupling layers rest on query-case citations, so both sets of figures are lower bounds. Section 3.4, the RQ3 discussion and the Limitations are corrected accordingly. The paper also now reports a corpus property it had omitted: Ukrainian queries carry a mean of 1.31 gold cases (84.5% have exactly one) against 4.12 for the Canadian queries.

New tools.

Tool	Purpose
`bc_diagnostics.py`	Popularity floor, Coverage decomposed by how often each gold case is cited, and growth of the coupling layer with citation knowledge
`reachability_gain.py`	Reachability Gain, a nugget/diversity formulation whose corner cases reproduce Extended Precision and Coverage exactly (`--verify` asserts it)
`dump_rankings.py`	Dump dense-encoder top-k lists, so rank-aware measures can be scored beyond the BM25 baseline

Outputs are in data/bc_diagnostics_{ua,ca}.json and data/rg_{ua,ca}.json.

`tools/`

Standard-library / sentence-transformers Python (and R for figures); no third-party services beyond a PostgreSQL source and, for retrieval, a GPU.

Tool	Purpose
`build_canada_citation_graph.py`	Build the case→case graph from a COLIEE-format zip; coupling, co-citation, citation-age + Mann-Kendall
`build_ua_coupling_sample.py`	UA article-level coupling / co-citation on a fixed-seed decision sample (runs on the EDRSR Postgres)
`build_ua_temporal.py`	Full UA case→case citation-age trend + Mann-Kendall over all resolved precedent edges
`build_ua_case_retrieval_package.py`	Assemble a COLIEE-format UA case-retrieval task (gold = resolved precedent edges)
`probe_ua_case_retrieval.py`	Read-only scoping probe used to size the UA task
`retrieval_experiment.py`	Dense first-stage retrieval (E5, BGE-M3) + exponential recency-weighting sweep; `--years-json` for non-English dates
`figures/`	`prepare_fig_data.py` (JSON→CSV), R + tikzDevice figure scripts, `build_claims_source.py` (flatten stats for fact-checking)

`data/` (Ukrainian, derived from EDRSR)

File	What
`ua_graph_stats.json`	UA article-level coupling / co-citation sample statistics
`ua_temporal_stats.json`	UA full case→case citation-age-by-year series + Mann-Kendall (6.59M precedent edges)
`ua_retrieval_results.json`	UA case-retrieval macro P/R/F1 @k, baseline vs recency-weighting sweep (E5, BGE-M3)
`ua_case_retrieval.zip`	The UA case-retrieval benchmark in COLIEE Task 1 format: `ua_cases/cases/<doc_id>.txt` (2,000 queries + ~7,700-decision pool, texts truncated) + `clean_ua_case_labels.json` (query → gold precedent)
`ua_years.json`	`{<doc_id>.txt: decision_year}` for recency weighting

Reproduce the Ukrainian retrieval result

bash

python tools/retrieval_experiment.py \
  --zip data/ua_case_retrieval.zip \
  --years-json data/ua_years.json \
  --models e5 bge-m3 --max-chars 4000 --num-gpus 1 \
  --out ua_retrieval_results.json

Headline: exponential recency weighting degrades case retrieval in both Ukraine (−37% to −41% at λ=0.2) and Canada (−64% to −67%), while it improves Ukrainian statute retrieval — so the effect tracks the target type (superseded statute vs. cumulative precedent), not the legal tradition. Coupling density, by contrast, is tradition-dependent (Ukrainian decisions couple ~18× more densely than Canadian cases).

Source & ethics

Ukrainian data derives from the public EDRSR (Єдиний державний реєстр судових рішень). Decision texts are truncated and included only as a research benchmark. See also overthelex/ukrainian-court-decisions and overthelex/ua-court-citation-graph.

Citation

Bibliographic Coupling as a Cross-Jurisdictional Evaluation Framework for Legal Case Retrieval. Working draft, 2026 -- UAlberta / COLIEE collaboration; authorship to be finalized.