internationalscholarsprogram/handbook-engine

1---2title: ISP Handbook Engine3emoji: 📘4colorFrom: blue5colorTo: indigo6sdk: docker7pinned: false8---9 10# ISP Handbook Service — Python Migration11 12A Python/FastAPI service that generates the ISP (International Scholars Program) Handbook as PDF or HTML. This is a drop-in replacement for the PHP handbook generation pipeline, designed to be called over HTTP from the existing PHP application.13 14## Architecture15 16```17python_service/18├── app/19│   ├── main.py              # FastAPI entry point20│   ├── api/21│   │   └── routes.py        # REST endpoints22│   ├── core/23│   │   ├── config.py        # Environment-based settings24│   │   ├── database.py      # SQLAlchemy engine (MySQL)25│   │   ├── fonts.py         # Century Gothic font management26│   │   └── logging.py       # Logging setup27│   ├── models/              # SQLAlchemy models (if needed)28│   ├── repositories/29│   │   └── handbook_repo.py # Direct DB access (fallback)30│   ├── schemas/31│   │   └── handbook.py      # Pydantic request/response models32│   └── services/33│       ├── data_fetcher.py   # Fetch data from external JSON APIs34│       ├── html_builder.py   # Build full handbook HTML35│       ├── pdf_service.py    # HTML -> PDF via WeasyPrint36│       ├── renderers.py      # TOC, sections, university renderers37│       └── utils.py          # Shared helpers (h, money format, etc.)38├── tests/39│   ├── test_api.py40│   └── test_renderers.py41├── fonts/                    # Century Gothic TTF files42├── images/                   # Handbook images (cover, header, etc.)43├── css/                      # Base stylesheet44├── Dockerfile45├── requirements.txt46├── .env.example47└── README.md48```49 50## API Endpoints51 52| Method | Path | Description |53|--------|------|-------------|54| `GET` | `/health` | Health check |55| `GET` | `/diagnostics/fonts` | Font file diagnostics |56| `GET` | `/api/v1/sections/global?catalog_id=0` | Fetch normalised global sections |57| `GET` | `/api/v1/sections/universities` | Fetch normalised university sections |58| `GET` | `/api/v1/handbook/pdf?catalog_id=0` | Generate PDF (download) |59| `POST` | `/api/v1/handbook/pdf` | Generate PDF with JSON body |60| `GET` | `/api/v1/handbook/html?catalog_id=0` | Generate HTML preview |61| `POST` | `/api/v1/handbook/render` | Generate PDF or HTML based on `output_format` |62| `GET` | `/docs` | Swagger UI |63| `GET` | `/redoc` | ReDoc UI |64 65## Local Development66 67### Prerequisites68 69- Python 3.11+70- MySQL database (existing schema — unchanged)71- Century Gothic font files in `fonts/` directory72 73### Setup74 75```bash76cd python_service77 78# Create virtualenv79python -m venv .venv80.venv\Scripts\activate    # Windows81# source .venv/bin/activate  # Linux/Mac82 83# Install dependencies84pip install -r requirements.txt85 86# Copy and configure environment87copy .env.example .env88# Edit .env with your database credentials and API URLs89```90 91### Run92 93```bash94uvicorn app.main:app --reload --host 0.0.0.0 --port 786095```96 97Visit http://localhost:7860/docs for the interactive API documentation.98 99### Run Tests100 101```bash102pytest tests/ -v103```104 105## Docker106 107### Build108 109```bash110docker build -t isp-handbook-service .111```112 113### Run114 115```bash116docker run -d \117  --name handbook-service \118  -p 7860:7860 \119  -e DB_HOST=host.docker.internal \120  -e DB_USER=root \121  -e DB_PASSWORD=secret \122  -e DB_NAME=handbook \123  -e API_BASE_URL=https://finsapdev.qhtestingserver.com \124  isp-handbook-service125```126 127Or with an env file:128 129```bash130docker run -d --name handbook-service -p 7860:7860 --env-file .env isp-handbook-service131```132 133## Hugging Face Spaces Deployment134 1351. Create a new Space on Hugging Face with **Docker** SDK1362. Upload/push the `python_service/` directory as the Space root1373. Ensure `fonts/`, `images/`, and `css/` directories are included1384. Set environment variables (Secrets) in Space settings:139   - `DB_HOST`, `DB_USER`, `DB_PASSWORD`, `DB_NAME`140   - `API_BASE_URL`141   - `PORT=7860` (default for HF Spaces)1425. The `Dockerfile` is already configured for HF Spaces (port 7860, `0.0.0.0`)143 144**Important**: Hugging Face Spaces may not allow outbound MySQL connections. If direct DB access is needed, use the external API endpoint approach (the service fetches data from the PHP JSON APIs over HTTP, not from the database directly).145 146## PHP Integration Example147 148The PHP application can call this Python service over HTTP using cURL:149 150```php151<?php152/**153 * PHP client for the ISP Handbook Python Service.154 * Replace HANDBOOK_SERVICE_URL with your actual deployment URL.155 */156 157define('HANDBOOK_SERVICE_URL', 'http://localhost:7860');158 159/**160 * Check service health.161 */162function handbook_health(): array {163    $url = HANDBOOK_SERVICE_URL . '/health';164    $ch = curl_init($url);165    curl_setopt_array($ch, [166        CURLOPT_RETURNTRANSFER => true,167        CURLOPT_TIMEOUT        => 5,168    ]);169    $body = curl_exec($ch);170    $code = (int)curl_getinfo($ch, CURLINFO_HTTP_CODE);171    curl_close($ch);172 173    if ($code !== 200) {174        return ['ok' => false, 'error' => 'Service unreachable', 'http_code' => $code];175    }176    return json_decode($body, true) ?? ['ok' => false, 'error' => 'Invalid response'];177}178 179/**180 * Generate and download the handbook PDF.181 */182function handbook_download_pdf(int $catalogId = 0, bool $debug = false): void {183    $params = http_build_query([184        'catalog_id' => $catalogId,185        'debug'      => $debug ? 'true' : 'false',186    ]);187    $url = HANDBOOK_SERVICE_URL . '/api/v1/handbook/pdf?' . $params;188 189    $ch = curl_init($url);190    curl_setopt_array($ch, [191        CURLOPT_RETURNTRANSFER => true,192        CURLOPT_TIMEOUT        => 120,193        CURLOPT_FOLLOWLOCATION => true,194    ]);195    $body = curl_exec($ch);196    $code = (int)curl_getinfo($ch, CURLINFO_HTTP_CODE);197    $contentType = curl_getinfo($ch, CURLINFO_CONTENT_TYPE);198    curl_close($ch);199 200    if ($code !== 200 || strpos($contentType, 'application/pdf') === false) {201        http_response_code(502);202        header('Content-Type: text/plain');203        echo "PDF generation failed (HTTP $code)";204        return;205    }206 207    header('Content-Type: application/pdf');208    header('Content-Disposition: attachment; filename="ISP_Handbook.pdf"');209    header('Content-Length: ' . strlen($body));210    echo $body;211}212 213/**214 * Fetch global sections via the Python service.215 */216function handbook_get_sections(int $catalogId = 0): array {217    $url = HANDBOOK_SERVICE_URL . '/api/v1/sections/global?catalog_id=' . $catalogId;218    $ch = curl_init($url);219    curl_setopt_array($ch, [220        CURLOPT_RETURNTRANSFER => true,221        CURLOPT_TIMEOUT        => 25,222    ]);223    $body = curl_exec($ch);224    curl_close($ch);225    return json_decode($body, true) ?? [];226}227 228/**229 * Generate handbook via POST with custom options.230 */231function handbook_generate(array $options = []): string {232    $url = HANDBOOK_SERVICE_URL . '/api/v1/handbook/render';233    $payload = json_encode(array_merge([234        'catalog_id'                => 0,235        'include_inactive_programs' => false,236        'debug'                     => false,237        'output_format'             => 'pdf',238    ], $options));239 240    $ch = curl_init($url);241    curl_setopt_array($ch, [242        CURLOPT_RETURNTRANSFER => true,243        CURLOPT_POST           => true,244        CURLOPT_POSTFIELDS     => $payload,245        CURLOPT_HTTPHEADER     => ['Content-Type: application/json'],246        CURLOPT_TIMEOUT        => 120,247    ]);248    $body = curl_exec($ch);249    curl_close($ch);250    return $body;251}252```253 254### Usage in PHP255 256```php257// Health check258$status = handbook_health();259if ($status['status'] === 'ok') {260    echo "Service is running\n";261}262 263// Stream PDF to browser264handbook_download_pdf(catalogId: 1);265 266// Get sections data267$sections = handbook_get_sections(catalogId: 1);268print_r($sections);269```270 271## Migration Notes & Assumptions272 273### What was migrated274 275| PHP Component | Python Equivalent | Notes |276|---|---|---|277| `common.php` (URL builder, HTTP client) | `data_fetcher.py` | Uses `httpx` instead of cURL |278| `cors.php` | FastAPI CORS middleware | Same origins preserved |279| `helpers.php` (`h()`, `respondJson()`) | Built into FastAPI + `utils.py` | |280| `fetchers.php` (global/uni data fetch) | `data_fetcher.py` | Identical normalisation logic |281| `renderers.php` (TOC, blocks, university) | `renderers.py` | All block types preserved |282| `html_builder.php` (`buildHandbookHtml`) | `html_builder.py` | Same HTML structure |283| `pdf.php` (Dompdf render) | `pdf_service.py` | **WeasyPrint** replaces Dompdf |284| `images.php` (image config) | `pdf_service.py` `_get_images_config()` | |285| `font_diagnostics.php` | `GET /diagnostics/fonts` | |286| `db.php` (mysqli) | `database.py` (SQLAlchemy) | Available but not primary path |287 288### Key differences289 2901. **PDF engine**: WeasyPrint replaces Dompdf. Layout may differ slightly in edge cases (table widths, page breaks). Both support `@font-face` with base64 TTF and `@page` rules.291 2922. **TOC page numbers**: The PHP code uses a 2-pass Dompdf render to inject exact TOC page numbers via named destinations. WeasyPrint doesn't expose named destinations the same way. TOC pages are assigned sequentially in the initial migration. Exact page numbers can be added via a post-processing PDF pass if needed.293 2943. **No auth**: The PHP code has no authentication. The Python service also has none. Add API key middleware if this service is exposed publicly.295 2964. **Data source**: The service fetches data from the same two PHP JSON APIs over HTTP (not directly from the database). The `repositories/handbook_repo.py` provides a DB fallback if you want to bypass the PHP APIs entirely.297 2985. **SSL verification**: Disabled for internal API calls (`verify=False` in httpx), matching the PHP behavior (`CURLOPT_SSL_VERIFYPEER => false`).299 300### Risks301 302- **Font rendering**: Century Gothic rendering may differ slightly between Dompdf (PHP) and WeasyPrint (Python). Test with actual fonts.303- **Page break behavior**: Dompdf and WeasyPrint handle CSS `page-break-*` properties slightly differently.304- **Image embedding**: Remote campus images are fetched at generation time. Network issues will result in placeholder cells (same as PHP behavior).305- **Memory**: Large handbooks with many university images may require significant memory. The Dockerfile doesn't set memory limits — Hugging Face Spaces has its own limits.306