gkour/israeli_law

Open Israeli Law (Hebrew Wikisource)

Israel's entire body of law — every statute, regulation, and order that Hebrew Wikisource volunteers have transcribed for the "Open Book of Laws" project (ספר החוקים הפתוח) — in one file you can actually load and query.

Nearly 6,000 pages. Almost 100 million characters. One jsonl.

This isn't a scrape of a summary or a curated subset — it's the raw material: full MediaWiki wikitext, straight from the source, with the metadata you need to trace every page back to its exact revision. Bring your own parser, chunker, or fine-tuning pipeline.

Good for:

• —RAG — semantic search and retrieval over Israeli legal text
• —Fine-tuning — instruction tuning, after you clean and normalize
• —Legal NLP research — segmentation, citation linking, summarization, QA

By the numbers

Where it comes from

Every page was pulled from Hebrew Wikisource (he.wikisource.org), radiating out from the project's home page:

• —"ויקיטקסט:ספר החוקים הפתוח" (Open Book of Laws)

Each record carries its pageid, revid, and timestamp, so you can always trace a snippet back to the exact Wikisource revision it came from — no guessing which version of a law you're looking at.

License & legal notes (read this before you redistribute)

License: CC BY-SA 4.0 — the safest default for anything built on Wikisource.

Here's the nuance that makes this interesting:

• —The laws themselves (statutes, regulations, court decisions) aren't copyrighted under Israeli law — official publications are free for anyone to use.
• —But the Wikisource edits on top of them — formatting, transcription choices, structure, templates — are contributor work, and contributors released it under CC BY-SA 4.0.

So the underlying law is free, but the version you're holding has a license attached because of how it got here.

If you redistribute (including cleaned/derived versions), you need to:

1. 1.Attribute Hebrew Wikisource — link back to the project if you can.
2. 2.Keep it ShareAlike — CC BY-SA 4.0 or a compatible license, no exceptions.
3. 3.Not imply Wikisource endorses your work.

Want to go license-free? If you rebuild the dataset straight from an official government source (Nevo, Reshumot, the Knesset site) and strip out anything Wikisource contributors added, the resulting text could plausibly be public domain / CC0. This dataset deliberately isn't that — it's the Wikisource version, so treat it as CC BY-SA.

Data schema

Each line in israeli_laws.jsonl is one JSON object:

Example record

{"title":"חוק איגודי ערים","pageid":382257,"revid":1459999,"timestamp":"2022-12-14T10:21:11Z","wikitext":"..."}

Understanding the wikitext format

The wikitext field is MediaWiki markup — the same markup language Wikipedia is written in, not plain text or HTML. The basics:

On top of that, this project uses its own template dialect prefixed ח: (חוק, "law") that Hebrew Wikisource volunteers built to structure statutes consistently. A real excerpt from חוק הרשויות המקומיות (אגרת העברת מקרקעין):

{{ח:התחלה}}                            law starts here
{{ח:כותרת|חוק ... תשי״ט–1959}}         the law's title
{{ח:סעיף|1|הגדרות}}                    section 1, titled "Definitions"
{{ח:ת}} בחוק זה –                      a paragraph of legal text
{{ח:ת|סוג=הגדרה}} ”רשות מקומית“ – ...  a definition-type paragraph
{{ח:תתת|(1)}} ...                       a nested sub-clause
{{ח:הערה|...}}                          an editorial/explanatory note
{{ח:חיצוני|target#anchor|display text}} a link to another law/section
{{ח:סוגר}}                              closes a block
{{ח:מפריד}}                             a visual divider

So a typical law's body is a sequence of {{ח:סעיף|number|title}} section markers, each followed by {{ח:ת}} calls wrapping the actual paragraphs, with {{ח:תת}} / {{ח:תתת}} for nested sub-clauses.

Not every page follows this pattern — portal/index pages (like the project's own home page, "ספר החוקים הפתוח") have no legal content at all, just large nested [[wikilink]] lists pointing to the real law pages. Filter these out by title if you're building a corpus of actual laws.

Getting from wikitext to something usable

The wikitext field is exactly what a Wikisource editor typed — templates, tables, formatting markup and all. That's a feature (nothing was thrown away) but it means you'll want a pass before feeding it to a model:

1. 1.Strip the markup — render or regex out templates, tables, and category tags. Keep section headers (== כותרת ==) as structural signal; don't just delete them.
2. 2.Normalize — punctuation, whitespace, Hebrew quote marks.
3. 3.Chunk with legal structure in mind — split by siman/se'if (סעיפים), not by arbitrary character count. 300–1,000 tokens per chunk with overlap works well for RAG.
4. 4.Enrich (optional) — attach canonical law IDs, dates, and cross-references to Nevo/Reshumot/Knesset sources; link amendments to the laws they amend.

One catch: "complete dump" means index and portal pages came along for the ride, not just individual laws. If you're building RAG, filter out anything that looks like a project/category page (titles like "ספר החוקים הפתוח" itself) before you embed everything.

What people build with this

• —Hebrew-language RAG for legal Q&A
• —Statute-level summarization
• —Section-level QA datasets, once segmented
• —Citation graphs mapping how laws reference each other

Know before you rely on it

• —This is not an official source of law. Wikisource is community-maintained — expect typos, occasional missing amendments, and uneven coverage on obscure regulations.
• —Templates can hide content. A naive strip-all-braces parser will occasionally eat something you wanted to keep. Test on a sample first.
• —Every page is frozen at one revision. The law may have changed since that timestamp — always check it against the current consolidated text if precision matters.

Citing this dataset

If it ends up in a paper or product, a line like this covers you:

"Text sourced from Hebrew Wikisource (he.wikisource.org), 'ספר החוקים הפתוח' project, under CC BY-SA."

How it was built

Pulled straight from the MediaWiki API — page content plus revision metadata, one page per API call, flattened into one JSON object per line.