itachi023/Markdown-Layout-Extractor

1---2title: Markdown Layout Extractor3emoji: 📄4colorFrom: red5colorTo: yellow6sdk: docker7app_port: 78608pinned: false9---10 11<p>12  <div align="center">13  <h1>14    PDF to Markdown MCP15    <br /> <br />16    <a href="">17      <img18        src="https://img.shields.io/badge/python%20%7C%203.12-blue"19        alt="Python 3.12"20      />21    </a>22    <a href="https://github.com/astral-sh/uv">23      <img24        src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json"25        alt="uv"26      />27    </a>28    <a href="https://modelcontextprotocol.io/">29      <img30        src="https://img.shields.io/badge/MCP-FastMCP-6C47FF"31        alt="FastMCP"32      />33    </a>34    <a href="https://mistral.ai/">35      <img36        src="https://img.shields.io/badge/Mistral%20AI-FF7000?logoColor=white"37        alt="Mistral AI"38      />39    </a>40    <a href="https://www.starlette.io/">41      <img42        src="https://img.shields.io/badge/Starlette-ASGI-009688"43        alt="Starlette"44      />45    </a>46    <a href="https://www.uvicorn.org/">47      <img48        src="https://img.shields.io/badge/Uvicorn-server-4051B5"49        alt="Uvicorn"50      />51    </a>52    <a href="https://loguru.readthedocs.io/">53      <img54        src="https://img.shields.io/badge/Loguru-logging-FF6B6B"55        alt="Loguru"56      />57    </a>58  </h1>59  </div>60</p>61 62An MCP (Model Context Protocol) server that converts PDFs and documents into Markdown using **Mistral OCR**.63 64## Features65 66- **`pdf_to_markdown`** — Convert any publicly accessible PDF/document URL to merged Markdown.67- **`pdf_to_structured_markdown`** — Convert and get per-page structured output (page index, individual markdown, merged result).68- CORS-enabled SSE transport — connect from any MCP client or inspector.69- `/health` endpoint for liveness probing.70- Structured, colorized logging via Loguru.71 72## Project Structure73 74```75pdf_to_md_mcp/76├── main.py                       # Entry point — uvicorn runner77├── pyproject.toml78├── sample.env                    # Secrets reference (copy to .env)79├── development.yml               # Non-secret config (server, CORS, OCR model)80└── app/81    ├── server.py                 # ASGI app factory (MCP + CORS + health)82    ├── core/83    │   ├── config.py             # Pydantic settings (loads .env + development.yml)84    │   ├── logger.py             # Loguru logger85    │   ├── lifespan.py           # AppContext + Mistral client lifecycle86    │   └── exceptions.py         # Domain exceptions87    ├── services/88    │   └── ocr_service.py        # Mistral OCR business logic89    ├── tools/90    │   └── markdown_tools.py     # @mcp.tool() definitions91    └── utils/92        ├── response.py           # create_response() helper93        └── validators.py         # URL validation94```95 96## Setup97 98```bash99# Install uv if not already installed100curl -LsSf https://astral.sh/uv/install.sh | sh101 102# Install dependencies103uv sync104 105# Configure secrets106cp sample.env .env107# Edit .env — set MISTRAL_API_KEY108# Non-secret config (server, CORS, OCR model) lives in development.yml109```110 111## Run112 113```bash114uv run main.py115```116 117Server starts at `http://127.0.0.1:8000` by default.118 119| Endpoint | Description |120| --- | --- |121| `GET /health` | Liveness probe |122| `GET /sse` | MCP SSE transport |123| `POST /messages/` | MCP message handler |124 125## MCP Tools126 127### `pdf_to_markdown`128 129Convert a document URL to merged Markdown (all pages concatenated).130 131**Input**132 133| Parameter | Type | Description |134| --- | --- | --- |135| `document_url` | `string` | Publicly accessible URL of a PDF or image document |136 137**Returns** — `string`138 139```140# Introduction141 142This paper presents...143 144## Section 2145 146...147```148 149---150 151### `pdf_to_structured_markdown`152 153Convert a document URL and get per-page structured output alongside the merged result.154 155**Input**156 157| Parameter | Type | Description |158| --- | --- | --- |159| `document_url` | `string` | Publicly accessible URL of a PDF or image document |160 161**Returns** — `object`162 163```json164{165  "page_count": 3,166  "pages": [167    { "index": 0, "markdown": "# Page 1\n..." },168    { "index": 1, "markdown": "## Page 2\n..." },169    { "index": 2, "markdown": "### Page 3\n..." }170  ],171  "markdown": "# Page 1\n...\n\n## Page 2\n...\n\n### Page 3\n..."172}173```174 175## Debugging with MCP Inspector176 177```bash178npx -y @modelcontextprotocol/inspector179```180 181Connect to `http://127.0.0.1:8000/sse` locally or your Railway URL in production.182 183## Deploy to Railway184 185### 1. Push to GitHub186 187```bash188git init189git add .190git commit -m "initial commit"191gh repo create pdf-to-md-mcp --public --source=. --push192```193 194### 2. Create a Railway project195 196Go to [railway.app](https://railway.app) → **New Project** → **Deploy from GitHub repo** → select your repo.197 198Railway detects the `railway.json` and uses `uv run main.py` as the start command automatically.199 200### 3. Set environment variables201 202In Railway → your service → **Variables**, add:203 204| Variable | Value |205|---|---|206| `MISTRAL_API_KEY` | your Mistral API key |207| `HOST` | `0.0.0.0` |208 209> `PORT` is injected automatically by Railway — do **not** set it manually.  210> All other config (`MISTRAL_OCR_MODEL`, `LOG_LEVEL`, etc.) is read from `development.yml`.211 212### 4. Deploy213 214Railway triggers a deploy on every push to your default branch. Once live, your public SSE URL will be:215 216```217https://<your-service>.up.railway.app/sse218```219 220Use that URL in any MCP client or pass it to the inspector:221 222```bash223npx -y @modelcontextprotocol/inspector224# connect to: https://<your-service>.up.railway.app/sse225```226 227### Why it works228 229- Railway injects `PORT` as an env var — pydantic-settings reads env vars before `development.yml`, so it's picked up automatically.230- `HOST=0.0.0.0` (set via Railway Variables) overrides the local `127.0.0.1` default so the container is reachable.231- `proxy_headers=True` in `main.py` makes uvicorn trust Railway's `X-Forwarded-*` headers.232- `/health` is set as Railway's healthcheck path in `railway.json`.233 234 235 236## Configuration237 238Configuration is split across two files to separate secrets from non-sensitive settings.239 240### `.env` — Secrets only241 242```dotenv243MISTRAL_API_KEY=your_mistral_api_key_here244```245 246### `development.yml` — Non-secret config247 248```yaml249# Mistral250MISTRAL_OCR_MODEL: mistral-ocr-latest251MISTRAL_TABLE_FORMAT: markdown252 253# Server254APP_NAME: "Markdown & Layout Extractor"255HOST: "127.0.0.1"256PORT: 8000257LOG_LEVEL: INFO258 259# CORS260CORS_ALLOW_ORIGINS:261  - "*"262CORS_ALLOW_METHODS:263  - "*"264CORS_ALLOW_HEADERS:265  - "*"266```267 268**Priority (highest → lowest):** environment variables → `.env` → `development.yml`269 270### All settings271 272| Variable | File | Default | Description |273| --- | --- | --- | --- |274| `MISTRAL_API_KEY` | `.env` | **required** | Mistral AI API key |275| `MISTRAL_OCR_MODEL` | `development.yml` | `mistral-ocr-latest` | OCR model identifier |276| `MISTRAL_TABLE_FORMAT` | `development.yml` | `markdown` | Table output format |277| `APP_NAME` | `development.yml` | `Markdown & Layout Extractor` | MCP server name |278| `HOST` | `development.yml` | `127.0.0.1` | Bind address |279| `PORT` | `development.yml` | `8000` | Bind port |280| `LOG_LEVEL` | `development.yml` | `INFO` | Log level (`DEBUG`, `INFO`, `WARNING`, `ERROR`) |281| `CORS_ALLOW_ORIGINS` | `development.yml` | `["*"]` | Allowed CORS origins |282| `CORS_ALLOW_METHODS` | `development.yml` | `["*"]` | Allowed HTTP methods |283| `CORS_ALLOW_HEADERS` | `development.yml` | `["*"]` | Allowed HTTP headers |284 285## Design Notes286 287- **Single Starlette app** — `sse_app()` is the sole ASGI application; the health route and CORS middleware are injected directly onto it to prevent double-middleware stacking (which causes the `http.response.start` crash).288- **Separation of concerns** — Tools are thin wrappers around `OCRService`; business logic is independently testable.289- **Lifespan-managed client** — The Mistral client is initialized once at startup and shared across all tool calls.290- **Loguru logging** — Structured, colorized logs across all layers via Loguru.291- **Pydantic Settings** — Type-safe, `.env`-driven configuration with an LRU-cached singleton.292