← back
Bypassing the Multimodal Tax: Hybrid RAG, SQL RRF & UI Telemetry - Abed Matini, Ogilvy
Takeaway
A transparent local ingestion pipeline and hybrid retrieval can simplify document assistants and reduce repeated document-processing costs.
Summary
- The workshop builds an HR FAQ assistant using FastAPI, React, PostgreSQL, Docker, local Ollama models, and Langfuse observability.
- Documents are converted locally to Markdown with Docling before chunking and embedding, exposing document structure and avoiding repeated multimodal processing.
- The planned retrieval pipeline combines keyword and semantic search with reciprocal-rank fusion and passes relevant results to the language model.
- An admin interface supports inspecting ingestion and chunking choices, while tracing is intended to expose latency, conversation behavior, and safety checks.
hybrid-ragdoclingpostgresql
Original description
Building a prototype chatbot interface takes an afternoon, but scaling it into a secure, predictable enterprise app requires solving production infrastructure challenges: minimizing unpredictable cloud API token bills, handling broken layout extractions, mapping session telemetry, and executing deterministic search. In this practical, demo-driven session, we will bypass high-level wrapper libraries to dissect the complete technical implementation of a local-first document ingestion application. While our demo runs on local developer environments, the underlying design patterns mirror exactly how you build high-performance, cost-effective architectures using enterprise infrastructure like Azure Database for PostgreSQL with pgvector. Key Architectural Blueprints We Will Explore: Optimizing the Data Ingest: How parsing documents to clean structural Markdown locally eliminates cloud vision token overhead. We will look at implementing dynamic sidebar toggles to switch into heavy-duty local OCR pipelines when corrupted document font layers are encountered. Database-First RAG without Framework Bloat: Why heading-based semantic chunking outperforms arbitrary sliding token windows. We will walk through the raw SQL schemas and indexes (HNSW tracking) required to combine dense embedding vectors and sparse keyword text indices natively inside a single database query, using Reciprocal Rank Fusion (RRF) for top-tier retrieval performance. UX Telemetry and Application Guardrails: A deep dive into injecting live client-side floating widgets to display session telemetry, tracking token limits and cumulative ROI. Finally, we will touch on building a zero-dependency, background-threaded heuristic validation layer (_check_injection()) to scan incoming Markdown text for prompt-injection markers before they hit your persistence layer. Speakers: Abed Matini (Ogilvy): Abed Matini is a software developer and AI engineer focused on practical RAG systems, with a strong emphasis on local and edge LLMs and lightweight production architectures. X/Twitter: https://x.com/abedmatini LinkedIn: / matini GitHub: https://github.com/abedmatini