CoolFace
Apppublic

arunSeervi/offline-ai-docs

sourceHugging Facegpl-3.0updated 3mo agoView on Hugging Face
0likes
App README

CPU-First Offline AI Document Intelligence Platform

An offline-first document processing pipeline that transforms unstructured documents (PDFs, DOCXs, and images) into structured, validated relational database profiles. All operations run entirely on local CPU resources with zero network dependencies.

[!IMPORTANT] Offline Compliance: This application does not connect to external APIs (no OpenAI, no Anthropic, no Gemini). All OCR, embeddings, vector indexing, and LLM inference run locally on the host's CPU.

🚀 Key Features

  • —100% Offline execution: Zero HTTP calls to third-party services.
  • —CPU-Optimized Inference: Utilizes quantized GGUF models via llama.cpp and onnxruntime CPU threads.
  • —Decoupled AI Pipeline: Swap components (OCR, embedder, vector store, LLM) without altering business logic.
  • —Strict JSON Outputs: Validated against configurable Pydantic schemas with a single-retry recovery mechanism.
  • —Relational Storage: SQLite database for documents, chunk relationships, extracted entities, and metadata.
  • —Interactive Side-by-Side Canvas: React/Vite dashboard illustrating text block maps alongside extracted database fields.

🛠️ Tech Stack & Constraints

LayerTechnologyVersion / Configuration
Backend FrameworkFastAPI + UvicornPython 3.13.5
Dependency ManageruvExclusive package manager (pyproject.toml / uv.lock)
Local LLM Runtimellama.cpp (llama-cpp-python)Qwen2.5-1.5B-Instruct GGUF Q4KM
Local Embeddingonnxruntime (CPU)all-MiniLM-L6-v2 ONNX Model
Vector SearchFAISS (CPU)Local index storage
OCR FallbackTesseractUsed when native digital text extraction fails
DatabaseSQLite + SQLAlchemyStrict foreign keys and cascaded deletions
FrontendReact + Vite + TypeScriptTailwind CSS styling with dark glassmorphism

📂 Project Architecture

The repository enforces a clean, modular structure:

text
offline_first/
├── .specify/                   # GitHub Spec Kit templates and constitution
│   └── memory/
│       ├── constitution.md     # Engineering principles and coding standards
│       ├── specification.md    # Product features specification
│       └── plan.md             # Technical implementation blueprints
├── backend/                    # FastAPI python app
│   ├── api/
│   │   └── v1/
│   │       └── endpoints/      # REST API route handlers
│   ├── core/                   # Single-responsibility AI and DB modules
│   │   ├── config/             # System settings and environment variables
│   │   ├── ocr/                # Native layout parsing and Tesseract engine
│   │   ├── chunking/           # Recursive text splitting logic
│   │   ├── embedding/          # ONNX model loader and vector computation
│   │   ├── rag/                # FAISS vector database and retriever
│   │   ├── llm/                # llama.cpp client and prompt grammar
│   │   ├── schemas/            # Pydantic structures for validation
│   │   ├── storage/            # Local file persistence
│   │   ├── services/           # Decoupled processing pipelines
│   │   └── database/           # Relational models and session managers
│   ├── tests/                  # Pytest unit suites
│   └── main.py                 # Backend Entrypoint
├── frontend/                   # React + Vite client app
│   ├── src/
│   │   ├── components/         # Reusable canvas widgets (DocumentViewer)
│   │   ├── pages/              # Views (Upload, Status, Search)
│   │   ├── App.tsx             # Root page containing layout rules
│   │   └── index.css           # Styling entrypoint with Tailwind directives
│   ├── postcss.config.js       # PostCSS config with @tailwindcss/postcss
│   └── tailwind.config.js      # Tailwind scan paths configuration
├── docs/                       # Architectural specs and API documentation
├── scripts/                    # Command utilities
│   ├── setup.sh                # Install and synchronize packages
│   └── run.sh                  # Run dev servers
└── models/                     # Weight store for GGUF and ONNX models

🔧 Prerequisites & Setup

1. System Requirements

  • —Python: 3.13.*
  • —Node.js: >= 20.0.0
  • —uv: Python packaging CLI
  • —Tesseract: Local OCR binaries (required only for image files/scanned PDFs)
  • —Debian/Ubuntu: sudo apt-get install tesseract-ocr
  • —macOS: brew install tesseract

2. Initialization

Run the project bootstrap script:

bash
./scripts/setup.sh

This script will build the Python virtual environment using uv sync and install frontend npm packages.

3. Placing Model Weights

Download and copy required local model weights to the models/ directory before starting the application:

  1. 1.Qwen2.5-1.5B-Instruct-GGUF (Q4KM): Download the weights from HuggingFace and place them as models/qwen2.5-1.5b-instruct-q4_k_m.gguf.
  2. 2.all-MiniLM-L6-v2 ONNX: Execute the download script to automatically fetch and configure ONNX weights:
bash
   uv run scripts/download_models.py

🏃 Running the Application

Option A: Running Both Servers Simultaneously (Recommended)

Start both frontend and backend development environments together:

bash
./scripts/run.sh

Option B: Running the Servers Separately

If you prefer to run or debug the components individually, open two terminal sessions:

Terminal 1: Backend Server

Start the FastAPI application on port 8000:

bash
uv run uvicorn backend.main:app --host 127.0.0.1 --port 8000 --reload
Terminal 2: Frontend Server

Start the Vite dev server on port 5173:

bash
cd frontend
npm run dev

🧪 Static Quality Gates & Testing

We enforce strict validation checks. Run these before pushing changes:

bash
# Apply code styling
uv run black backend
uv run isort backend

# Run linters and checks
uv run ruff check backend
uv run mypy backend

# Run security checks
uv run bandit -r backend --exclude backend/tests

# Run unit test suites
uv run pytest

# Execute all pre-commit hooks manually
uv run pre-commit run --all-files

📝 License

This project is licensed under the GNU Affero General Public License v3.0 - see the LICENSE file for details.