opnsrcntrbtrian/sebi-circulars

SEBI Circulars Dataset

A comprehensive, structured dataset of Indian Securities and Exchange Board (SEBI) regulatory circulars, public-domain government works compiled and annotated for AI/ML research.

Date: 2026-08-14 Snapshot Version: v2026.08 Corpus: 728 circulars (2010–2026)

Dataset Configurations

Schema Details

corpus

Columns: circular_number, issue_date, effective_date, subject, issuing_department, supersession_status, version_lineage, source_url, text, excerpt, extraction_date, circular_type, validity_status, superseded_by_id, supersession_edges

• —circular_number (str): Unique identifier (e.g., SEBI/HO/CFD/P/CIR/2023/123).
• —issue_date (date): Publication date.
• —effective_date (date, nullable): When the circular takes effect.
• —subject (str): Circular title/summary.
• —issuing_department (str): Issuing SEBI department (e.g., CFD, MRD). Known limitation: 176/728 records have issuing_department=UNKNOWN due to pre-existing parsing artifacts.
• —supersession_status (str): in_force, superseded, or amended.
• —version_lineage (list[str]): Prior circular numbers this updates/references.
• —source_url (str): Original SEBI publication page.
• —text (str): Full circular text.
• —excerpt (bool): Whether the text is a partial excerpt.
• —extraction_date (date): When this record was extracted from source.

Known data-quality caveat: Some master-circular subject fields capture body text (~2900 chars) due to a pre-existing PDF parsing artifact in src/sebi_rag/ingest_pdf.py. This is not a regression from this work; document it in your analysis.

chunks

Columns: chunk_id, doc_id, section, context_header, text, circular_number, issue_date, effective_date, subject, issuing_department, supersession_status, version_lineage, circular_type, validity_status, superseded_by_id

78,585 section-aware retrieval chunks derived from corpus text, one row per chunk.

• —chunk_id (str): Unique chunk identifier (e.g., SEBI/HO/CFD/P/CIR/2023/123#preamble#0).
• —doc_id (str): Parent circular number.
• —section (str): Section path (e.g., SEBI/HO/CFD/.../preamble/p0).
• —context_header (str): Repeated contextual header extracted from chunk text (circular_number | subject | section).
• —text (str): Chunk text body (header removed for clarity).
• —Flattened metadata: circular_number, issue_date, effective_date, subject, issuing_department, supersession_status, version_lineage.

lineage

Columns: source_circular, relation, target_circular, source_issue_date, target_in_corpus

4,577 regulatory supersession/amendment edges (forward-direction only).

• —source_circular (str): Circular that supersedes/amends another.
• —relation (str): supersedes or amends.
• —target_circular (str): Older circular being superseded/amended.
• —source_issue_date (date): Publication date of source (for temporal reasoning).
• —target_in_corpus (bool): Whether the target circular is in this corpus (allows filtering for pair-classification tasks).

Note: Inverse relationships (superseded_by, amended_by) are omitted to avoid duplication; regenerate them at query time.

citation-normalization

Columns:

8,903 in-text reference citations mined and normalized.

• —raw_reference (str): Raw citation text as it appears in the circular (e.g., CIR/CFD/CMD/4/2015).
• —normalized_circular_number (str): Canonical form (lowercase, standardized).
• —context_window (str): Surrounding text (~60 characters on each side, whitespace collapsed).
• —source_doc_id (str): Circular containing the reference.
• —format_family (str): Reference format category:
• —new-standard: SEBI/HO/DEPT/P/CIR/YYYY/NNN (post-2015 format).
• —old-standard: CIR/DEPT/YYYY/NNN (legacy format).
• —dept-order-2026: HO/(NN)YYYY-DEPT (departmental order format, 2026).

Task: Seq2seq/NER: predict normalized circular number from raw reference; or use as training set for reference extraction/normalization models.

supersession-pairs

Columns:

2,769 labeled circular pairs: positives from lineage, negatives sampled same-department (2:1 ratio).

• —circular_a_number (str): First circular.
• —circular_a_subject (str): Subject of circular A.
• —circular_b_number (str): Second circular.
• —circular_b_subject (str): Subject of circular B.
• —label (str): supersedes, amends, or unrelated.

Task: Pair classification: does circular A supersede/amend circular B? Positives from lineage edges (both endpoints in corpus); negatives sampled deterministically (seed=42) from same-department non-linked pairs.

Licensing & Compliance

Underlying Regulatory Text: SEBI circulars are Indian government works. Per India's Copyright Act 1957 §52(1)(q), government orders/notifications may be freely reproduced. We attribute SEBI and provide source_url per record for verification.

Compilation & Annotations: The metadata extraction, chunking, lineage graph, normalized citations, and pair labels are original annotations licensed under CC-BY-4.0.

Disclaimers

1. 1.Not legal advice. These circulars are informational only. Verify against sebi.gov.in before regulatory reliance.
2. 2.Not SEBI-endorsed. This dataset is independent; not affiliated with or endorsed by the Securities and Exchange Board of India.
3. 3.Coverage: Corpus spans 2010–2026 and is not exhaustive of all SEBI circulars.
4. 4.Data quality: issuing_department is UNKNOWN for 176 records (parsing artifact). Some master-circular subject fields may be oversized (~2900 chars, also a parsing artifact).

Citation

Please cite this dataset if you use it:

@dataset{sebi_circulars_2026,
  title={SEBI Circulars: Indian Regulatory Texts, 2010–2026},
  author={OpenSourceContributor},
  year={2026},
  url={https://huggingface.co/datasets/...},
  license={CC-BY-4.0}
}

Repository

Full dataset extraction pipeline and reproducibility information:

• —GitHub: https://github.com/opnsrcntrbtr/sebi-circular-rag
• —Extraction date: 2026-08-14 Snapshot: v2026.08 (max issue_date across corpus)

Suggested Use Cases

• —Retrieval & RAG: chunks config for dense/hybrid retrieval pipelines.
• —Citation Mining: citation-normalization for training sequence-to-sequence or NER models.
• —Regulatory Reasoning: lineage for link prediction, temporal reasoning, and regulatory change tracking.
• —Pair Classification: supersession-pairs for supervised learning on relationship prediction.
• —Benchmark: eval config (56 curated queries) for domain-specific retrieval evaluation.

Contact

For questions or issues: [https://github.com/opnsrcntrbtr/sebi-circular-rag/issues]