Back to Projects
📄 Case Study · Solo Project

CiteGrain — Long-Document Traceable QA System

Academic-PDF question answering, cited back to page and section, inside a hard 10,000-token context
Solo project · Built 2026 · Runs entirely local except one text-only model call
GitHub Repository
Problem System Flow Tech Stack Chunking & Tables Trade-offs My Role Features Challenges Results Demo

Problem Background

System Flow

CiteGrain system architecture diagram
Five layers: Client (document panel, chat, sources panel) / Proxy (nginx: static files, API forwarding, health check) / Services (Documents, Q&A, Metrics, MCP) / Core (Parsing, Chunking, Routing, Retrieval, Tables, Summary — the actual assignment) / Data (local files + local ONNX embedding + gpt-4o-mini as the one external call)
Upload path · once · ~11s
Read PDF→ Find sections→ Rebuild tables→ Chunk→ Index
Query path · every question · first token ~1s
Route→ Retrieve→ Pack evidence→ Generate→ Resolve citations

Offline indexing is tested explicitly: upload and indexing still succeed when the external model is unavailable — only answer generation needs the API.

Tech Stack

Language Model & Retrieval

gpt-4o-mini (text-only) multilingual-MiniLM-L12-v2 (ONNX, local) BM25 + jieba segmentation RRF rank fusion (k=60)

Backend

FastAPI + uvicorn (async) PyMuPDF Server-Sent Events

Frontend & Interfaces

React · Vite · Tailwind MCP server (4 tools) Swagger / OpenAPI docs

Deployment & Observability

Docker Compose nginx reverse proxy Prometheus + Grafana

The embedding model was picked by benchmark, not by default: four candidates were timed on the same four queries, and multilingual-MiniLM-L12-v2 ranked first on all of them (11.7s vs. 26.4s for bge-zh and 60.2s for e5-small) — bge-zh and e5-small placed 8th and 4th specifically on the Chinese-language queries.

Structure-Aware Chunking & Table Verification

Engineering Trade-offs

Cost
    Speed
      Accuracy
        Reliability

          My Role

          Solo Full-Stack & AI Engineer

          Features

          Hybrid Retrieval

            Verified Table Reconstruction

              Query Routing

                Hierarchical Summarization

                  Traceable Citations

                    Challenges & Solutions

                    1 The PDF library's built-in table detection failed both ways

                    Problem

                    Solution

                    2 A "sound" fix to filter footnote superscripts made tables worse

                    Problem

                    Solution

                    3 An assumption about throttling was wrong by 17×

                    Problem

                    Solution

                    4 "Summarize this document" can't be answered by ordinary retrieval

                    Problem

                    Solution

                    5 An early summarizer quietly dropped the numbers that mattered most

                    Problem

                    Solution

                    6 A fluent, fully-cited answer was still wrong

                    Problem

                    Solution

                    Evaluation Results

                    Retrieval accuracy: 0% → 100%

                    On 12 test questions (half Chinese, half English), the fixed-length baseline found the correct section in its top 8 results 0% of the time. Adding section-aware chunking alone brought that to 83%; adding hybrid retrieval and table handling on top reached 100% (12/12).

                    Table fidelity: 13/13, zero wrong cells indexed

                    17 tables were detected across four test papers; all 13 numeric tables passed both conservation checks and were indexed, the 2 that failed validation fell back to raw text, and zero incorrect cell values ever entered the index.

                    100% citation rate on real answers

                    Every one of 233 answered queries carried a citation — zero answers with no traceable source. Separately, 47 queries were correctly refused when the uploaded document didn't cover the question, rather than answered anyway.

                    Retrieval itself is fast; the model is the bottleneck

                    Finding the right chunks takes 19ms at the median. End-to-end latency (~3.5s at 1 concurrent user) is dominated by waiting on the language model to generate its answer (~2.2s), not by the retrieval this project actually built.

                    Generalizes past the assigned paper

                    Beyond the assigned LightRAG paper, the same pipeline was verified on Attention Is All You Need, BERT, and RAG — spanning single- and double-column layouts, and papers with and without a built-in PDF outline.

                    US$0.000847 per question, 0 external calls while indexing

                    Median context usage is 2,736 of the available 6,000 evidence tokens. Parsing, chunking, and embedding a document never touch an external API — only the final answer-generation step does.

                    Demo

                    A real query against the assigned LightRAG paper, answered entirely from a table the system reconstructed and verified.

                    CiteGrain answering a question about LightRAG's ablation study, with citations to the source table