jycln/the-body-extension-archive

Last update (YYYY-MM-DD): 2026-01-11 16:42

RAG Body-Extension Archive

This is an experimental relational system.

Responses do not aim to explain, conclude, or resolve. They emerge through proximity, fragments, and guarded synthesis.

The system does not accumulate memory. Each query is encountered as a situated presence.

Runtime notes

• —CPU-based inference
• —Latency may vary
• —Silence, refusal, or neutrality are part of the system’s behavior

Built as a field, not an interface.

CPU-based RAG system with persistent llama-cpp GGUF backend for fast query runtime on CPU.

Models

• —Embedding Model: sentence-transformers/all-MiniLM-L6-v2 (384 dimensions)
• —Vector Database: FAISS (IndexFlatL2)
• —LLM Model: TinyLlama-1.1B-Chat-v1.0-GGUF (Q4KM quantization, ~170MB)

Features

• —Persistent model loading - Load once, query many times (no reload per query)
• —llama-cpp GGUF backend - 3-5x faster than transformers CPU
• —TinyLlama 1.1B Q4_K_M - Quantized GGUF for speed and low memory (~170MB)
• —Dual query mode - Collage (extractive) + Story (generative) outputs from single query
• —Fast CPU runtime - Story mode completes in ~20 seconds on CPU
• —Pool-based generation - Story mode uses candidate pool selection for quality
• —Gradio interface - Web UI for interactive queries
• —HF Spaces ready - Deployed on Hugging Face Spaces

Quick Start

1. Install Dependencies

pip install llama-cpp-python==0.2.90
pip install sentence-transformers faiss-cpu transformers gradio huggingface_hub

2. Model Download

The GGUF model is automatically downloaded from QuantFactory/TinyLlama-1.1B-Chat-v1.0-GGUF on first run. Or manually:

mkdir -p models
# Model will be downloaded automatically via huggingface_hub

3. Initialize and Query

from scripts.main_pipeline import initialize_rag

# Initialize once - model loads and stays in memory
rag = initialize_rag()

# Query with dual mode (Collage + Story)
result = rag.query_dual(
    "How do sensing technologies reshape the boundary of the body?",
    k=5,
    temperature=0.6
)

# Access results
print(result["collage"]["answer"])  # Extractive collage (2 paragraphs max)
print(result["story"]["answer"])    # Generated story (2-3 sentences)

Performance

• —First initialization: 30-60 seconds (model loading)
• —Subsequent queries: ~20 seconds (Story mode), <5 seconds (Collage mode)
• —Memory usage: ~500MB with Q4KM quantization
• —Model size: 170MB GGUF file

Architecture

scripts/
├── config.py              # System configuration (v71: HF Spaces paths)
├── main_pipeline.py       # Main orchestration - RAG orchestration with persistent loading
├── data_loader.py         # Document loader - With sentence fragment generation
├── retrieval_pipeline.py  # FAISS retrieval - FAISS-based chunk-first
├── synthesizer.py         # Answer generation - Collage & Story (v78 pool-based) synthesis
├── llm_client.py          # LLM interface - Unified LLM client (supports both backends)
├── llm_llamacpp.py        # Persistent llama-cpp client
├── llama_worker.py        # Worker process isolation - llama-cpp backend (v70)
├── gguf_utils.py          # Model utilities - GGUF model bootstrap and validation
├── sentence_utils.py      # Text processing - Sentence splitting and fragment filtering
├── tone_tagger.py         # Evidence annotation
├── guards.py              # Content guards and validation
├── v156_metrics.py        # Keyword metrics and similarity scoring
└── query_logger.py        # Query logging to JSONL

Modes

Collage Mode (Extractive)

• —No LLM calls - Strictly extractive from retrieved evidence
• —Output: 2 paragraphs max, 2 sentences per paragraph
• —Speed: <5 seconds (no generation overhead)

Story Mode (Generative)

• —Pool-based generation (v78) - Generates 8-10 candidates, selects best 2-3
• —Output: 2-3 new relational sentences
• —Quality filters: Non-English, meta/validator text, verbatim evidence rejection
• —Speed: ~20 seconds on CPU
• —Constraints: No verbatim evidence quotes, encounter voice (not explainer)

Configuration

Edit scripts/config.py to customize:

class Config:
    # Backend selection
    LLM_BACKEND = "llama_cpp"  # or "transformers" for fallback
    
    # llama-cpp GGUF model (auto-downloaded)
    LLAMA_CPP_REPO_ID = "QuantFactory/TinyLlama-1.1B-Chat-v1.0-GGUF"
    LLAMA_CPP_FILENAME = "TinyLlama-1.1B-Chat-v1.0.Q4_K_M.gguf"
    LLAMA_CPP_CTX = 512        # Context window
    LLAMA_CPP_THREADS = 2      # CPU threads
    
    # Story generation
    STORY_MAX_NEW_TOKENS = 64  # Token budget for ~20s runtime
    
    # Collage constraints
    COLLAGE_MAX_PARAS = 2
    COLLAGE_MAX_SENTENCES_PER_PARA = 2

Troubleshooting

"llama-cpp-python not installed"

pip install llama-cpp-python==0.2.90

"Model file not found"

• —Model auto-downloads on first run via gguf_utils.py
• —Check config.py for correct repo ID and filename

"Falling back to transformers"

• —llama-cpp installation failed or model download failed
• —System will use slower transformers backend (~3 minutes per query)

Usage Notes

• —Initialize once per session - Reuse the same rag instance for multiple queries
• —Query logging - Enabled by default in data/05_logs/*.jsonl
• —Evidence display - Left column shows selected 5 chunks
• —Collage mode display - Middle column shows retrieved fragments with source attribution
• —Story mode display - Right column chatbox
• —Retrieval mode - Not using now. Uses chunk-first retrieval (configurable in config.py)

Export Logs

Export to Dataset Repo

Type ::EXPORT_LOGS:: in the chat interface to commit the current session log file to a Hugging Face dataset repo.

Requirements:

• —HF_TOKEN environment variable must be set (with write permission)
• —Dataset repo will be created automatically if it doesn't exist

What happens:

• —The current session .jsonl file from data/05_logs/ is committed to the dataset repo
• —Logs are stored at the repo root (no subdirectory)
• —Commit hash is returned in the chat response

Download logs locally:

1. 1.Clone the dataset repo into your local data/05_logs folder:

   git clone https://huggingface.co/datasets/jycln/TheBody-ExtensionArchive_logs data/05_logs

1. 1.To update with new logs:

   cd data/05_logs
   git pull

Note: The dataset repo is separate from the Space repo, so exports won't trigger Space rebuilds.