sscorp/ecosystem-intel

1---2title: Ecosystem Intelligence3emoji: 🧠4colorFrom: yellow5colorTo: gray6sdk: docker7app_port: 78608---9 10# Should I Build This? — Ecosystem Intelligence for Open Source11 12> Before you start an open-source project, this tells you **what's alive, what's a13> graveyard (and exactly why each project died), and where the genuinely unfilled14> gap is** — questions a search box cannot answer.15 16Built on self-hosted **[Cognee](https://www.cognee.ai)** for the WeMakeDevs × Cognee17**"Where's My Context?"** hackathon — **Track 1: Best Use of Cognee Open Source**.18 19**Live demo:** https://sscorp-ecosystem-intel.hf.space20 21![Landing page — Build only what earns the right to exist](assets/landing.png)22 23![Analysis of "an open-source LLM eval framework": active projects, failed or abandoned projects with cited source links, and the open opportunity](assets/analysis.png)24 25*One analysis, three answers: what's **alive**, what's **dead and why** (every26graveyard carries the maintainer's own deprecation notice as a clickable27receipt), and the **gap** — demand no maintained project serves.*28 29[![CI](https://github.com/SamChawla/ecosystem-intel/actions/workflows/ci.yml/badge.svg)](https://github.com/SamChawla/ecosystem-intel/actions/workflows/ci.yml)30![Tests](https://img.shields.io/badge/tests-86%20passing-brightgreen)31![Coverage](https://img.shields.io/badge/coverage-72%25-green)32![Python](https://img.shields.io/badge/python-3.11-blue)33[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)34 35---36 37## The theme, taken literally38 39The hackathon asks ***"Where's my context?"*** — an AI that woke up with no memory40of last night. **Open source wakes up like that every morning.** In any niche,41dozens of projects have already lived and died, and the context — what was tried,42why it failed, what users still ask for — is scattered across closed issues,43archived READMEs, and deprecation notices nobody reads. The ecosystem has amnesia.44 45This project is the morning-after reconstruction, built on Cognee's memory46lifecycle: it **remembers** the ecosystem (add + cognify), **recalls** why each47dead project died with the source as the receipt, **improves** what it knows when48reality changes (memify), and **forgets** projects that no longer belong. Ask it49*"should I build this?"* and it answers with the context everyone else lost.50 51---52 53## Table of contents54 551. [The theme](#the-theme-taken-literally)561. [The problem](#the-problem)572. [Why a knowledge graph](#why-a-knowledge-graph)583. [How it uses Cognee](#how-it-uses-cognees-memory-lifecycle)594. [Architecture](#architecture)605. [Getting started](#getting-started)616. [Configuration reference](#configuration-reference)627. [API reference](#api-reference)638. [Project structure](#project-structure)649. [Development](#development)6510. [Deployment](#deployment)6611. [Status](#status)67 68---69 70## The problem71 72Asking "does something like this already exist?" is a similarity search — solved,73commodity, and useless over a sea of abandoned repos. The questions that actually74decide whether you should build are **relational** and live in scattered prose, not75in repo descriptions:76 77- Which similar projects were **abandoned, and why?** (the reason is buried in a78  closing issue or a README deprecation notice)79- Where is the **unmet gap** — a capability people keep requesting that no80  maintained project supplies?81 82Vector search cannot answer either. Both require a graph.83 84## Why a knowledge graph85 86| Question | Why it needs the graph |87|---|---|88| Why did project X die? | Multi-hop: `Project → ABANDONED_BECAUSE → FailureReason`, with the source issue as provenance |89| Where's the gap? | Bridge entity: a `Capability` with `REQUESTS` edges (demand) but no active `Project` with a `PROVIDES` edge (supply hole) |90| What's the living landscape? | Entity resolution + status across heterogeneous sources (repos, issues) |91 92## How it uses Cognee's memory lifecycle93 94| Lifecycle stage | Where | Status |95|---|---|---|96| **add** (remember) | `ingest_documents()` — structured documents per project/failure/demand signal | Working, smoke-verified |97| **cognify** | Graph build per dataset (one dataset per project, so forget stays surgical) | Working, smoke-verified |98| **search** (recall) | `GRAPH_COMPLETION` queries for the three hero answers | Working, smoke-verified |99| **forget** | Native `cognee.forget(dataset=...)` — removes a project from the active picture | Working — `POST /forget`, smoke-verified end-to-end |100| **memify** | Feedback re-ingestion (add + cognify into a feedback dataset) — the graph learns from outcomes | Working — `POST /memify`, smoke-verified before/after |101 102## Architecture103 104```105GitHub API ──▶ documents ──▶ Cognee (cognify) ──▶ graph + vector store (Kuzu + LanceDB)106                                                          │107              Web UI ◀── FastAPI /analyze ◀── recall (multi-hop) ◀┘108```109 110| Layer | Choice |111|---|---|112| Knowledge engine | Cognee **1.2.2** (self-hosted, pinned) |113| LLM | Euri / euron.one (OpenAI-compatible chat) |114| Embeddings | local fastembed `all-MiniLM-L6-v2` (384d, free, offline) |115| Graph store | Kuzu (embedded) |116| Vector store | LanceDB (embedded) |117| Backend | FastAPI |118| Frontend | static HTML |119 120Ingestion runs **offline** to pre-build the store; the deployed service only reads121it. This keeps deployment light and free.122 123## Getting started124 125### Mock mode (2 minutes, no keys, no Cognee)126 127```bash128cd backend129python -m venv .venv && source .venv/bin/activate130pip install fastapi "uvicorn[standard]" python-dotenv requests pydantic131cp .env.example .env                      # then set USE_MOCK=1132uvicorn app.main:app --reload133# open http://127.0.0.1:8000134```135 136### Real mode137 138> **Windows users:** the real stack (cognee + kuzu + fastembed) runs under **WSL139> Ubuntu**. Create the venv inside WSL and run all commands below from there.140 141```bash142# 1) Environment (uv recommended; plain pip works too)143uv venv ~/eco-venv --python 3.11144uv pip install --python ~/eco-venv/bin/python -r backend/requirements.txt145 146# 2) Configure147cp backend/.env.example backend/.env      # fill LLM_* and GITHUB_TOKEN, set USE_MOCK=0148 149# 3) GATE: prove LLM + embeddings + graph round-trip before anything else150#    WARNING: the gate starts with a store reset — always BEFORE step 4, never after.151cd backend && ~/eco-venv/bin/python smoke_test.py152# PASS = the printed answer names RepoB as abandoned (burnout / superseded)153 154# 4) Pre-ingest demo domains (offline)155~/eco-venv/bin/python -m app.ingest "llm eval framework" --max 15156 157# 5) Serve158~/eco-venv/bin/uvicorn app.main:app --reload159```160 161## Configuration reference162 163All configuration is via `backend/.env` (see `backend/.env.example`). **Never commit164`.env`** — it is gitignored; every secret lives there and nowhere else.165 166| Variable | Purpose | Notes |167|---|---|---|168| `USE_MOCK` | `1` = canned answers, no deps; `0` = real Cognee | Mock is the demo safety net |169| `LLM_PROVIDER` | `openai` | Euri is OpenAI-compatible |170| `LLM_ENDPOINT` | Euri base URL | `https://api.euron.one/api/v1/euri` |171| `LLM_API_KEY` | Euri API key | secret |172| `LLM_MODEL` | `openai/<model-id>` | the `openai/` prefix is litellm routing syntax |173| `EMBEDDING_PROVIDER` | `fastembed` | local, no API calls |174| `EMBEDDING_MODEL` | `sentence-transformers/all-MiniLM-L6-v2` | 384 dimensions |175| `EMBEDDING_DIMENSIONS` | `384` | must match the model |176| `GRAPH_DATABASE_PROVIDER` | `kuzu` | embedded |177| `VECTOR_DB_PROVIDER` | `lancedb` | embedded |178| `REQUIRE_AUTHENTICATION` | `false` | cognee 1.2 defaults to multi-user auth; this is a single-user store |179| `ENABLE_BACKEND_ACCESS_CONTROL` | `false` | same reason |180| `CACHING` | `false` | disable cognee session cache |181| `GITHUB_TOKEN` | PAT with `public_repo` scope | ingestion only; lifts rate limit to 5000/hr |182 183Do **not** set `DATA_ROOT_DIRECTORY` in `.env`: cognee validates it at import time184and rejects relative paths. The store location (`backend/data/`) is set in code by185`cognee_engine.configure()`.186 187## API reference188 189| Method | Path | Body | Returns |190|---|---|---|---|191| `GET` | `/health` | — | `{ok, mode, cognee_installed}` |192| `POST` | `/analyze` | `{"idea": "..."}` | `{idea, living, dead, gap, mode}` — the three buckets |193| `POST` | `/ingest` | `{"query": "...", "max_repos": 12}` | live single-domain ingestion |194| `POST` | `/forget` | `{"project_name": "..."}` | removes the project from the active landscape |195| `POST` | `/memify` | `{"feedback": "..."}` | teaches the graph from an outcome |196| `GET` | `/` | — | static frontend (`index.html` + `css/` + `js/`) |197| `GET` | `/docs` | — | interactive Swagger UI |198 199Fail-soft contract: if the real pipeline errors, every endpoint returns the200error in the body (e.g. `"mode": "mock-fallback"`, `"forgotten": false`) —201nothing 500s during a demo.202 203The API layer is class-based: controllers in `app/api/routes.py` own their204routers, and every router carries the `get_current_user` dependency from205`app/api/deps.py` — the single seam where authentication can be added later206without touching any route.207 208## Project structure209 210```211backend/212  app/213    api/214      routes.py        # class-based controllers (Health, Analysis, Lifecycle)215      schemas.py       # Pydantic request models216      deps.py          # shared dependencies — the auth seam217    cognee_engine.py   # THE ONLY file with version-sensitive Cognee calls (pinned 1.2.2)218    github_client.py   # GitHub REST wrapper: repo search, tiered provenance, demand signals219    provenance.py      # cognee-free ledger: project -> exact source URL220    schema.py          # graph ontology models + document builders221    ingest.py          # offline ingestion, one dataset per project222    queries.py         # product logic: mock / cognee / fail-soft, ADR-0010 JSON contract223    main.py            # app factory: wires controllers + mounts the frontend224  tests/               # pytest suite (contracts, provenance tiers, fail-soft)225  smoke_test.py        # six-leg gate: remember, cognify, recall, forget, memify226  Dockerfile           # HF Spaces deployment (port 7860)227  requirements.txt228frontend/229  index.html           # markup only230  css/app.css          # styles231  js/app.js            # API calls + rendering232pyproject.toml         # ruff + pylint + pytest config233```234 235**The one rule:** every version-sensitive Cognee call lives in236`backend/app/cognee_engine.py`, annotated with the verified 1.2.2 signature. If a237Cognee upgrade breaks something, the fix is in that one file.238 239## Development240 241### Run locally for feedback (mock mode — instant, no keys)242 243```bash244# from backend/ (in WSL if on Windows):245USE_MOCK=1 uvicorn app.main:app --host 0.0.0.0 --port 8000246# then open http://localhost:8000  (UI)  and  http://localhost:8000/docs  (Swagger)247```248 249`USE_MOCK=1` on the command line overrides `.env`, so the full UI + API run250with canned data and zero dependencies on Cognee/Euri/GitHub. Drop the251override to serve real graph answers from the pre-ingested store.252 253### Quality gates254 255```bash256# Lint + format (both must be clean; pylint is held at 10.00/10)257ruff format backend && ruff check backend258cd backend && pylint app smoke_test.py tests259 260# Unit/contract tests (fast, no network)261cd backend && pytest262 263# The real-Cognee integration gate (needs .env keys; ~4 min)264cd backend && python smoke_test.py265```266 267- The smoke test is the gate: no feature work lands while `smoke_test.py` fails.268- Multi-file changes are test-first (see `backend/tests/`).269 270### Test suite & coverage271 27286 tests, all green, in ~10s with **no network and no Cognee installed** — the273suite proves the API contracts, the fail-soft behavior, the provenance tiers,274the status heuristic, and the LLM failover chain entirely in mock mode.275Coverage of `backend/app` is **72%**, and the gap is deliberate:276 277| Module | Coverage | Why |278|---|---|---|279| `queries.py`, `manifest.py`, `deps.py`, `schemas.py` | 100% | product logic and contracts are fully unit-tested |280| `routes.py`, `provenance.py`, `schema.py`, `main.py` | 83–92% | contract-tested through the HTTP layer (httpx) |281| `cognee_engine.py` | 48% | the real-Cognee paths are exercised by `smoke_test.py` against live Cognee — mocking `cognee.*` would verify nothing (its API drifts between versions) |282| `ingest.py` | 18% | offline batch script; proven by the real ingestion runs that build `backend/data/` |283 284```bash285pytest --cov=backend/app --cov-report=term-missing   # reproduce the numbers286```287 288### Continuous integration289 290Every push runs [`.github/workflows/ci.yml`](.github/workflows/ci.yml):291`ruff format --check` + `ruff check`, `pylint --fail-under=10`, and the full292pytest suite with coverage — on a machine with **cognee deliberately not293installed**, which continuously proves the mock/fail-soft boot path (ADR-0008).294 295The real-Cognee integration is **not** CI-able (it needs live LLM keys and296resets the store), so it is gated locally instead:297 298### The six-leg real-Cognee gate299 300`backend/smoke_test.py` runs against real Cognee 1.2.2 + Euri + local301fastembed and must print six `PASS` lines:302 303| Leg | Proves |304|---|---|305| 1. `configure()` | LLM + embedding + Kuzu + LanceDB config accepted |306| 2. `reset()` | clean slate (⚠️ this **wipes the store** — run before ingest, never after) |307| 3. ingest + cognify | remember: per-project datasets build a real graph |308| 4. recall | "why did RepoB die?" returns *burnout / superseded* with the source URL |309| 5. forget | `POST /forget` path removes RepoB from the landscape |310| 6. memify | feedback re-ingestion changes what the graph answers |311 312Run order for a deployable store: **smoke → bulk ingest → deploy**.313 314## Deployment315 316- **Backend** → Hugging Face Spaces (Docker). Commit `backend/data/` (the prebuilt317  store) at ingest time, set `USE_MOCK=0` + the LLM vars as Space Variables.318  Serves on port 7860.319- **Frontend** → Vercel (static). Point `API_BASE` in `frontend/index.html` at the320  Space URL.321 322## Status323 324- [x] Environment validated: cognee 1.2.2 + Euri LLM + local fastembed325- [x] Six-leg smoke gate passing (remember, cognify, recall, forget, memify)326- [x] pytest suite green (API contracts, fail-soft, provenance tiers, status heuristic)327- [x] Provenance-tiered GitHub ingestion (README notices → maintainer issues → archived default → none)328- [x] `/forget` + `/memify` HTTP endpoints and UI controls329- [x] Class-based API layer with an auth seam (`app/api/`)330- [x] Lint clean: ruff all-pass, pylint 10.00/10331- [x] CI on every push: ruff + pylint + pytest with coverage (mock-mode, cognee-free)332- [x] MIT licensed333- [x] Pre-ingested demo domains (96 datasets, 1157 nodes / 1608 edges) + deployed334      live on HF Spaces with an hourly keep-alive335