← back
On AI and Knowledge — Pablo Castro, Distinguished Engineer & CVP for AI Knowledge, Microsoft
Takeaway
Enterprise agents need coordinated access to organizational knowledge and retrieval methods beyond the model's training data.
Summary
- Knowledge is framed as intrinsic model knowledge, external organizational knowledge, and learned knowledge.
- Microsoft IQ connects agents to workplace documents, analytics assets, custom data, and web information through several specialized offerings.
- Retrieval evaluations favor combining lexical and vector methods with ranking over relying on cosine similarity alone.
- Foundry IQ layers managed ingestion and agentic retrieval over lower-level controls for indexing and relevance.
retrievalcontext-engineeringmicrosoft
Original description
Pablo Castro explores AI and knowledge systems for building better applications and agents. Speaker: Pablo Castro —Distinguished Engineer and CVP, Microsoft, leads the AI Knowledge team in Microsoft's CoreAI division, where he focuses on state-of-the-art information understanding and retrieval systems for AI applications and agents, including Foundry IQ, Azure AI Search, and Azure Content Understanding. LinkedIn: / pabloc Timestamps: 0:00 Introduction and speaker background 1:14 Defining the nature of knowledge: Intrinsic, Extrinsic, and Learned 1:27 Intrinsic knowledge and the history of AI coding tools 4:38 Extrinsic knowledge and corporate data grounding 7:06 Evolution of retrieval systems and Foundry IQ 9:56 Foundry IQ demo: Building a knowledge base 13:08 Learned knowledge: The agent learning loop 14:25 Foundry agent optimization demo 16:49 Closing remarks and resources Key quotes Intrinsic Knowledge Perspective: This knowledge represents the foundational parametric memory of models. "Intrinsic knowledge is just the knowledge that comes with the models... it's what started many of the scenarios that then grew on all the things we're doing with agents today." (1:27 - 1:48) "I would argue that GitHub Copilot and ChatGPT, those sort of experiences, were heavily grounded on this intrinsic memory—what the models already knew." (2:59 - 3:04) Extrinsic Knowledge Perspective: To be truly useful in an organization, agents must access private, ambient data through sophisticated retrieval. "Intrinsic model got us here, but it only gets you so far if you're building a system that or an agent that needs to participate in what's happening in an organization." (4:41 - 4:49) "The trick is how do you build a platform that allows you to combine all these building blocks without putting the complexity right in front of you." (8:02 - 8:10) "For more sophisticated cases you do want a system that can reflect on what's in the data set and decide whether or not we've satisfied the information need." (9:09 - 9:18) Learned Knowledge & Future Predictions Perspective: Knowledge is compounded by observing processes and enabling agents to self-optimize. "The idea that we can actually observe the processes and get better at them by reflecting and improving every step of it is something that is really changed now." (13:20 - 13:29) "Satya wrote about this recently and reflected on the fact that people and agents can really compound in how they do the work and how they can create this learning loop." (13:35 - 13:43) "This is a real learning loop materialized in practice... we can enable this learning loops that will capture this differentiated capability that lives in each one of the companies and organizations we work on." (16:40 - 17:03)