soumya-ai/Knowledge-Graph

1---2title: Knowledge Graph3emoji: 🕸️4colorFrom: indigo5colorTo: blue6sdk: gradio7sdk_version: 5.23.18app_file: app.py9pinned: false10license: mit11python_version: 3.1112---13 14# GraphRAG — Neo4j Aura + OpenAI + LangChain15 16Query a Neo4j Aura knowledge graph with OpenAI + LangChain GraphRAG (vector + multi-hop retrieval).17 18**Query Space:** [Knowledge-Graph](https://huggingface.co/spaces/soumya-ai/Knowledge-Graph) · **Ingest Space:** [Knowledge-Graph-Ingest](https://huggingface.co/spaces/soumya-ai/Knowledge-Graph-Ingest)19 20A GraphRAG implementation using:21- **Neo4j Aura** — knowledge graph + vector indexes (cloud)22- **OpenAI** — entity/relation extraction, embeddings, and answers23- **LangChain LCEL** — ingestion and query pipelines24 25---26 27## Hugging Face Spaces28 29| Space | App file | Role |30|-------|----------|------|31| [Knowledge-Graph](https://huggingface.co/spaces/soumya-ai/Knowledge-Graph) | `app.py` | Query |32| [Knowledge-Graph-Ingest](https://huggingface.co/spaces/soumya-ai/Knowledge-Graph-Ingest) | `ingest_app.py` | Ingest (use `README.ingest.md` as README on that Space) |33 34Add secrets on **both** Spaces: `NEO4J_URI`, `NEO4J_USERNAME`, `NEO4J_PASSWORD`, `NEO4J_DATABASE`, `OPENAI_API_KEY` — then **Restart**.35 36---37 38## Architecture39 40```41Documents42   │43   ▼  [INDEXING — runs once]44Text Chunking (LangChain splitter)45   │46   ▼47OpenAI GPT — entity & relation extraction (JSON)48   │49   ▼50OpenAI text-embedding-3-small — node & chunk embeddings51   │52   ▼53Neo4j Aura — nodes, edges, vector indexes54 55 56User Query57   │58   ▼  [QUERYING — runs per question]59OpenAI embeddings — embed query60   │61   ▼62Neo4j vector search — find similar chunks & entities63   │64   ▼65Neo4j Cypher — multi-hop graph traversal (1-2 hops)66   │67   ▼68Context assembly (entities + paths + chunks)69   │70   ▼71OpenAI GPT-4o-mini — generate grounded answer72```73 74---75 76## Local development77 78### Prerequisites79 80| Tool | Purpose | Install |81|------|---------|---------|82| Neo4j Aura | Graph + vector store | [console.neo4j.io](https://console.neo4j.io) |83| OpenAI API key | Extract, embed, answer | [platform.openai.com](https://platform.openai.com) |84| Python 3.11+ | Runtime | [python.org](https://python.org) |85| Docker (optional) | Local Neo4j only | [docs.docker.com](https://docs.docker.com) |86 87---88 89### Setup90 91#### 1. Neo4j database92 93**Option A — Neo4j Aura (recommended for cloud)**94 951. Create an instance at [console.neo4j.io](https://console.neo4j.io).962. Wait ~60 seconds until the instance is **Running**.973. Copy connection details into `.env` (see `.env.example`):98 99```bash100NEO4J_URI=neo4j+s://YOUR_INSTANCE.databases.neo4j.io101NEO4J_USERNAME=neo4j102NEO4J_PASSWORD=<from Aura console>103NEO4J_DATABASE=neo4j104```105 106Use the `neo4j+s://` URI from Aura (TLS).107 108### 2. Configure environment109 110```bash111cp .env.example .env112```113 114Set **Neo4j Aura** credentials and **OPENAI_API_KEY** in `.env`:115 116```bash117OPENAI_API_KEY=sk-...118OPENAI_MODEL=gpt-4o-mini119OPENAI_EXTRACT_MODEL=gpt-4o-mini120OPENAI_EMBED_MODEL=text-embedding-3-small121OPENAI_EMBED_DIMENSIONS=1536122```123 124### 3. Install Python dependencies125 126```bash127python -m venv venv128source venv/bin/activate   # Windows: venv\Scripts\activate129pip install -r requirements.txt130```131 132---133 134## Usage135 1361. **Ingest** — open [Knowledge-Graph-Ingest](https://huggingface.co/spaces/soumya-ai/Knowledge-Graph-Ingest) → Connect → **Ingest sample corpus**1372. **Query** — open [Knowledge-Graph](https://huggingface.co/spaces/soumya-ai/Knowledge-Graph) → ask a question138 139Local Gradio: `python ingest_app.py` or `python app.py`140 141---142 143## Project structure144 145```146├── README.md               # Query Space (app.py)147├── README.ingest.md        # Ingest Space (ingest_app.py) — copy as README there148├── requirements.txt149├── app.py                  # Query Gradio app150├── ingest_app.py           # Ingest Gradio app151└── src/152    ├── config.py153    ├── neo4j_client.py154    ├── embeddings.py155    ├── graph_builder.py156    ├── ingest_service.py157    ├── sample_corpus.py158    ├── retriever.py159    └── rag_chain.py160```161 162---163 164## How the LCEL chain works165 166```python167# rag_chain.py — simplified view168 169chain = (170    RunnableLambda(retrieve)       # OpenAI embed + Neo4j vector + Cypher171    | RunnableLambda(format_context)  # Build prompt dict172    | RAG_PROMPT                   # System + user template173    | ChatOpenAI(model="gpt-4o-mini")  # Generate answer174    | StrOutputParser()            # Return plain string175)176 177answer = chain.invoke({"question": "...", "top_k": 5})178```179 180---181 182## Tuning tips183 184| Parameter | Location | Effect |185|-----------|----------|--------|186| `chunk_size` | `graph_builder.py` | Larger = more context per extraction, slower |187| `top_k` | `retriever.py` / `app.py` | More chunks retrieved per query |188| `min_score` | `retriever.py` | Minimum cosine similarity threshold |189| `*1..2` in Cypher | `retriever.py` | Hop depth — increase for deeper traversal |190| `OPENAI_EXTRACT_MODEL` | `.env` | Model for entity/relation JSON extraction |191| `OPENAI_EMBED_MODEL` | `.env` | Embedding model (must match `OPENAI_EMBED_DIMENSIONS`) |192| `OPENAI_MODEL` | `.env` | Swap to `gpt-4o` for best answer quality |193 194---195 196## Neo4j graph schema197 198```199(:Chunk)          -[:MENTIONS]->  (:Entity)200(:Entity {type})  -[:REL_TYPE]->  (:Entity)201 202Node properties:203  Entity: id, name, type, description, embedding (1536-dim default)204  Chunk:  id, text, embedding (1536-dim default)205 206Vector indexes:207  entity_embedding — cosine, dims = OPENAI_EMBED_DIMENSIONS208  chunk_embedding  — cosine, dims = OPENAI_EMBED_DIMENSIONS209```210 211---212 213## Troubleshooting214 215**Neo4j not accepting connections** — check Aura instance is Running at [console.neo4j.io](https://console.neo4j.io)216 217**OpenAI API errors**218- Confirm `OPENAI_API_KEY` is set in `.env`219- Check billing and rate limits at [platform.openai.com](https://platform.openai.com)220 221**Wrong embedding dimensions**222If you changed `OPENAI_EMBED_DIMENSIONS` after ingesting, drop old vector indexes in Aura and re-ingest.223 224**Empty retrieval results**225- Check Neo4j browser: run `MATCH (n) RETURN count(n)` — should be > 0 after ingestion226- Lower `min_score` thresholds in `retriever.py`227- Confirm embeddings exist: `size(e.embedding)` should equal `OPENAI_EMBED_DIMENSIONS`228 229**Check that indexes exist and are ONLINE**230SHOW VECTOR INDEXES;231 232-- Peek at a few entity embeddings (first 5 dimensions shown)233```234MATCH (e:Entity) WHERE e.embedding IS NOT NULL235RETURN e.name, e.type, size(e.embedding) AS dims,236       e.embedding[0..5] AS first5237LIMIT 5;238```239 240-- Same for chunks241```242MATCH (c:Chunk) WHERE c.embedding IS NOT NULL243RETURN c.id, size(c.embedding) AS dims,244       c.embedding[0..5] AS first5245LIMIT 5;246```247 248-- Count nodes missing embeddings (a sign of silent embed failures)249```250MATCH (e:Entity) WHERE e.embedding IS NULL OR size(e.embedding) = 0251RETURN count(e) AS entities_without_embedding;252```253