← back
When All Context Matters: Extended Cache Augmented Generation - Luis Romero-Sevilla, Orbis
Takeaway
Parallel cached contexts can support collection-wide reasoning over rapidly changing data when retrieval and graph rebuilding are poor fits.
Summary
- The target workload requires reasoning across an entire document collection that is frequently replaced, limiting both selective vector retrieval and expensive graph reconstruction.
- Cache-augmented generation loads documents into model context and retains KV caches, but a single context window limits scale and can degrade answer quality.
- Extended CAG distributes documents across parallel cached context buckets that a stronger supervisor queries and revisits before synthesizing an answer.
- Documents are not divided into named domains because that encouraged the supervisor to overlook relevant buckets; cache lifetimes provide a cost-control lever.
context-cachingmulti-document-reasoningcag
Original description
This session addresses a critical challenge in knowledge representation: extracting accurate answers from a rapidly changing dataset where every document is highly interconnected and relevant. Explore the limitations of standard retrieval methods for dynamic, high-context scenarios—including the constraints of Simple RAG and the computational bottlenecks of constantly recomputing a GraphRAG. To overcome these hurdles, this talk introduces a novel solution: Extended Cache Augmented Generation (ECAG). Speakers: Luis Romero-Sevilla (Orbis Operations): Luis Romero-Sevilla is an AI strategist and full-stack software engineer with over 13 years of experience driving mission-critical technological innovation across defense, healthcare, and the public sector, currently serving as the Vice President of AI at Orbis Operations. X/Twitter: https://x.com/lurose15 LinkedIn: / luis-romero-sevilla GitHub: https://github.com/lurose5