kapilrao/SEC_filings_1994_2024

Dataset Card for SEC EDGAR Filings Master Index

Dataset Details

Dataset Description

This dataset contains metadata for all submissions to the Securities and Exchange Commission (SEC) through their EDGAR system from 1994 until December 14, 2024. The data is extracted from quarterly master files and includes key information about company filings such as CIK numbers, company names, form types, and filing dates.

• —Curated by: Arthur (arthur@cicero.chat)
• —Language(s): English
• —License: Likely public domain as US government data

Dataset Sources

• —Repository: Based on SEC EDGAR database
• —Primary Source: SEC EDGAR system (https://www.sec.gov/edgar.shtml)

Uses

Direct Use

• —Financial research and analysis
• —Corporate compliance monitoring
• —Historical filing pattern analysis
• —Company information retrieval
• —Regulatory filing tracking
• —Market research and analytics

Out-of-Scope Use

• —Real-time filing monitoring (dataset is historical)
• —Analysis of filing content (contains only metadata)
• —Legal advice or compliance determinations
• —Personal identification or contact information gathering

Dataset Structure

The dataset contains the following fields:

1. 1._id: Unique identifier (36 characters)
2. 2.timestamp: Timestamp of record creation
3. 3.cik: Company Identifier (1-7 digits)
4. 4.company_name: Name of filing company (0-150 characters)
5. 5.form_type: Type of SEC form filed (734 distinct values)
6. 6.date_filed: Date of filing submission
7. 7.filename: Path to filing in EDGAR system (37-43 characters)
8. 8.url: Direct link to filing (66-72 characters)
9. 9.quarter: Filing quarter (1-4)
10. 10.year: Filing year (1994-2024)
11. 11.master_metadata: Dictionary containing:
12. 12.Description
13. 13.Last Data Received
14. 14.Comments
15. 15.Anonymous FTP
16. 16.Cloud HTTP
17. 17.idx_type: Filing index type

Dataset Creation

Curation Rationale

The dataset was created to provide a comprehensive index of all SEC EDGAR filings, enabling research, analysis, and monitoring of corporate financial disclosures and regulatory compliance.

Source Data

Data Collection and Processing

• —Data is collected from SEC EDGAR quarterly master files
• —Processing includes structuring the data into a consistent format
• —URLs are standardized to SEC.gov domain
• —Timestamps are standardized to a consistent format
• —Unique identifiers are assigned to each record

Who are the source data producers?

• —Primary data producers are publicly traded companies and other entities required to file with the SEC
• —Data is collected and distributed by the U.S. Securities and Exchange Commission

Personal and Sensitive Information

The dataset contains only publicly available information from SEC filings. While it includes company names and identifiers, it does not contain personal information beyond what is already public in SEC filings.

Bias, Risks, and Limitations

• —Historical data may be incomplete for earlier years
• —Company names may change over time
• —CIK numbers may be reused in rare cases
• —Form types may be deprecated or changed over time
• —URLs may become invalid if SEC changes its archive structure
• —Coverage limited to SEC-regulated entities

Recommendations

• —Verify current company information through SEC's current EDGAR system
• —Cross-reference CIK numbers when tracking companies over time
• —Consider form type changes when analyzing historical patterns
• —Implement appropriate error handling for URL access
• —Use appropriate date ranges for analysis based on regulation changes
• —Consider regulatory context when analyzing filing patterns

Dataset Card Contact

Arthur (arthur@cicero.chat)

Code

https://github.com/arthrod/sec-edgar-bulker