Intelligent Knowledge Assistant for Technical Books

We chose BAAI/bge-small-en-v1.5 to satisfy our 2–5s SLA on commodity hardware without requiring expensive NVIDIA CUDA GPUs or burning paid API tokens per search query:

Vector Storage Economics: 384 dimensions require 4x less memory in Qdrant HNSW graphs than 1536-dimensional OpenAI embeddings, allowing the entire 2,810-chunk index to execute inside sub-millisecond RAM caches.

Chunking technical books requires solving the Context Truncation Dilemma:

• If chunks are too small (200–300 tokens): Multi-line SQL CTE queries, recursive Python functions, and mathematical proofs are cut in half, destroying syntactic integrity.
• If chunks are too large (1,500+ tokens): Embeddings suffer from semantic dilution, making specific concepts harder to isolate in ANN vector space.
• The 700-token sweet spot: Perfectly captures 2 to 3 cohesive textbook paragraphs or a complete code block alongside its explanatory commentary.

15% Overlap (105 tokens): Implemented via token-aware sentence boundary snapping. If an essential algorithmic definition lands across a chunk break, it is guaranteed to appear in full in both adjacent chunks.

Dense embeddings produce cosine similarity scores bounded in $[0, 1]$, whereas BM25 Okapi produces unbounded frequency scores in $[0, \infty)$. Linear weighted sum ($\alpha \cdot \text{Dense} + (1-\alpha) \cdot \text{BM25}$) fails because score scales shift wildly between short and long queries.

RRF eliminates score normalization vulnerabilities: It operates strictly over ordinal positions: $\text{RRF}(d) = \sum \frac{1}{60 + \text{Rank}_m(d)}$. As demonstrated below in our interactive sandbox, this allows BM25 to rescue exact terminology matches (like Chunk #4, promoted +16 ranks into the context window) while preventing single-keyword spam from overwhelming semantic relevance.