aekankpatel/finrag

1---2title: FinRAG3emoji: 📈4colorFrom: green5colorTo: blue6sdk: streamlit7sdk_version: "1.41.1"8app_file: app.py9pinned: false10---11 12# FinRAG13 14FinRAG is a Retrieval-Augmented Generation (RAG) system built for querying real financial documents. You type a natural language question — "What did NVIDIA say about AI demand?" or "What are Tesla's key risk factors?" — and the system finds the most relevant passages from a corpus of SEC filings and earnings call transcripts, then generates a grounded answer using an LLM. Every answer cites the exact source passages it was derived from, so you can verify the claim directly.15 16Live demo: [huggingface.co/spaces/aekankpatel/finrag](https://huggingface.co/spaces/aekankpatel/finrag)17 18---19 20## What problem it solves21 22Financial documents are long and dense. A 10-K filing can run 150+ pages. An earnings call transcript is 30–50 pages of executive commentary. Reading across multiple companies to find a specific piece of information — say, how three different companies characterize AI investment risk — takes hours.23 24FinRAG compresses that into a single search. It lets you ask questions in plain language and surfaces the relevant passages across all 24 documents in the corpus instantly, with an LLM synthesizing a clear answer from them.25 26---27 28## How it works29 30### Step 1 — Document ingestion (offline, done once)31 32Each financial PDF is parsed to plain text using `pdfplumber` (`ingest_pdfs.py`). The text is then chunked into overlapping token windows using LlamaIndex's `TokenTextSplitter`:33 34- **Chunk size:** 128 tokens per passage35- **Chunk overlap:** 20 tokens (so context is not cut off at boundaries)36- **Metadata attached to each chunk:** the source filename, so answers can be traced back to a specific document37 38Each chunk is embedded into a 384-dimensional vector using `BAAI/bge-small-en-v1.5`, a lightweight but accurate sentence embedding model from HuggingFace. The resulting vectors — along with the original text and metadata — are saved to a LlamaIndex `SimpleVectorStore` and persisted to disk as JSON files.39 40This entire process is run once via `build_index.py` and the output is stored in the [finrag-index](https://github.com/aekankpatel/finrag-index) repository.41 42### Step 2 — Index loading (on app startup)43 44When the HuggingFace Space starts, `app.py` checks if the index files exist locally. If not, it downloads them from GitHub:45 46- `docstore.json` (~26 MB) — all text chunks with metadata, downloaded via `raw.githubusercontent.com`47- `index_store.json` (~1 MB) — index structure and node ID mappings48- `default__vector_store.json` (~124 MB) — the embedding vectors, stored in Git LFS and downloaded via `media.githubusercontent.com`49- `graph_store.json` — empty placeholder, not used50 51LlamaIndex reconstructs the full in-memory index from these files. The embedding model (`bge-small-en-v1.5`) is also loaded at this point so that query embeddings use the same vector space as the stored document embeddings.52 53### Step 3 — Query and retrieval54 55When you submit a question:56 571. The question is embedded using the same `bge-small-en-v1.5` model582. The query vector is compared against all stored chunk vectors using cosine similarity593. The top-k most similar chunks are retrieved (default: 8, configurable via slider)604. If a document filter is active (auto-detected from keywords or manually selected), only chunks from that document are considered61 62The **auto-detect** feature works by matching keywords in your question against a company/topic map. If you mention "Tesla" or "tsla", it restricts retrieval to the Tesla 10-K. If no keyword matches, retrieval runs across the full corpus.63 64### Step 4 — Answer generation65 66The top 5 retrieved chunks (up to 500 tokens each) are assembled into a context block and sent to the Groq API along with the question. The model used is `llama-3.1-8b-instant`, which is fast and runs on Groq's inference hardware.67 68The system prompt instructs the model to answer strictly from the provided context and not to introduce outside knowledge. This keeps answers grounded and prevents hallucination on financial specifics like revenue figures or risk disclosures.69 70### Step 5 — Display71 72The app shows:73- The document(s) searched74- A retrieval confidence bar (based on the top cosine similarity score)75- The generated answer76- Each source passage with its document name and similarity score77- A download button to export the full answer + sources as a `.txt` file78 79```80User types question81        |82        v83bge-small-en-v1.5 embeds the question into a 384-dim vector84        |85        v86Cosine similarity search across all stored chunk vectors87        |88        v89Top-k chunks retrieved (optionally filtered to one document)90        |91        v92Top 5 chunks sent as context to llama-3.1-8b-instant via Groq93        |94        v95LLM generates answer grounded in context96        |97        v98Answer + source passages + confidence score displayed99```100 101---102 103## Documents covered104 105| Company / Topic | Documents |106|---|---|107| Apple | 10-K 2025, 10-Q Q1 2025, 10-Q Q4 2025 |108| Amazon | 10-K 2025, 10-Q Q3 2025, Q4 2025 earnings call |109| NVIDIA | 10-Q Q3 2025, Q4 2025 earnings call |110| Meta | 10-K 2025 |111| Microsoft | 10-Q Q3 2025, Q2 2025 earnings call |112| Tesla | 10-K 2025, 10-Q Q3 2025 |113| Goldman Sachs | BDC 10-Q Q2 2025, 2026 M&A outlook |114| Bank of America | 2024 Annual Report, Q4 2025 earnings call |115| JPMorgan | Q4 2025 earnings call |116| Walmart | Q4 2026 earnings call |117| Global macro | World Bank Global Economic Prospects Jan 2026 |118| Banking sector | EY Global Banking Outlook 2025 |119| Capital markets | Capital Markets Forecast 2026 |120 121---122 123## Features124 125- **Auto-detect** — Keywords in your question (company names, tickers) automatically narrow the search to the most relevant document126- **Manual filter** — Override auto-detect and pin the query to any specific document127- **Compare mode** — Run the same question against two documents side by side to directly compare how companies describe the same topic128- **Confidence score** — The top cosine similarity score is shown as a percentage bar, giving a rough signal of how well the corpus covers your question129- **Chat history** — Previous questions and answers are shown in the session so you can scroll back through your research130- **Export** — Every answer can be downloaded as a `.txt` file with the full source passages included131 132---133 134## Tech stack135 136| Layer | Technology |137|---|---|138| Frontend | Streamlit |139| Embeddings | `BAAI/bge-small-en-v1.5` (HuggingFace) |140| Vector index | LlamaIndex `SimpleVectorStore` |141| LLM | `llama-3.1-8b-instant` via Groq API |142| Index storage | GitHub + Git LFS ([aekankpatel/finrag-index](https://github.com/aekankpatel/finrag-index)) |143| Hosting | HuggingFace Spaces |144 145---146 147## Project structure148 149```150finrag/151├── app.py               # Streamlit app — handles index loading, retrieval, and UI152├── requirements.txt     # Python dependencies153├── build_index.py       # Offline script — chunks documents and builds the vector index154├── ingest_pdfs.py       # Parses PDFs to plain text files155├── evaluate.py          # Evaluation script for retrieval quality156├── query.py             # Standalone query script (no UI)157├── data/158│   ├── raw/             # Source PDF files159│   └── processed/       # Extracted plain text files (one per document)160└── finrag/161    └── index/           # Vector index files (downloaded at runtime from finrag-index repo)162```163 164### Key files explained165 166**`ingest_pdfs.py`** — Reads each PDF from `data/raw/` using `pdfplumber`, extracts the text page by page, and writes it to `data/processed/` as a `.txt` file. Handles encoding issues and strips junk characters.167 168**`build_index.py`** — Reads the processed `.txt` files, attaches the filename as `source` metadata to each document, splits everything into 128-token chunks with 20-token overlap using `TokenTextSplitter`, embeds each chunk with `bge-small-en-v1.5`, and persists the resulting index to `finrag/index/`.169 170**`app.py`** — On startup: downloads the pre-built index from GitHub if not cached, loads it into memory, and initializes the embedding model. On each query: embeds the question, runs similarity search, sends context to the Groq API, and renders the answer with sources.171 172---173 174## Running locally175 176```bash177git clone https://github.com/aekankpatel/finrag.git178cd finrag179pip install -r requirements.txt180```181 182Create `.streamlit/secrets.toml`:183 184```toml185GROQ_API_KEY = "your_groq_api_key"186```187 188Run the app:189 190```bash191streamlit run app.py192```193 194On first run the vector index (~150 MB total) is downloaded from GitHub automatically into `finrag/index/`. Subsequent runs load from the local cache.195 196To rebuild the index from scratch (e.g. after adding new documents):197 198```bash199python ingest_pdfs.py     # parse PDFs to text200python build_index.py     # embed and index201```202 203---204 205## Index repository206 207The pre-built vector index is stored separately at [github.com/aekankpatel/finrag-index](https://github.com/aekankpatel/finrag-index). It is kept in its own repo to avoid bloating the main repo with large binary files. The `default__vector_store.json` file (~124 MB) is stored using Git LFS because it exceeds GitHub's 100 MB file size limit.208