Consultants searched years of engagement documents by memory and by asking around. We built an agent on Databricks that answers with the sources: documents ingested as they are saved, chunked and embedded into Vector Search, retrieved with the reader’s document permissions checked before passages reach the model, and served behind the AI Gateway with every request logged and every release evaluated.
Proposals, deliverables and lessons learned sat in document libraries by client and year. Finding the last comparable engagement meant asking a partner. Nothing could be opened to a model without opening everything to everyone.
Documents indexed in Vector Search inside Unity Catalog, with a permission-aware retriever controlling which passages reach the model; answers from a foundation model through Model Serving with the passages cited; every request through the AI Gateway with logging, rate limits and guardrails; every release gated by evaluation.
Document indexing and user requests meet at an explicit permission-aware retrieval boundary before passages reach the answer.
The retrieval boundary enforces which engagement content the reader may access. Table row filters are not assumed to propagate into the shared search index.
A shared Vector Search index does not automatically inherit Unity Catalog table row filters. Retrieval must enforce the reader’s authorization explicitly.
Auto Loader lands new documents as they are saved. A Lakeflow pipeline parses, chunks and embeds them into Delta tables, and a Vector Search index with Delta Sync follows those tables, so the index is never rebuilt by hand.
The retriever checks the reader’s document permissions before passages reach the model. The shared index requires explicit permission filtering; row filters on the underlying chunk tables alone do not secure vector query results.
Built with the Mosaic AI Agent Framework: retrieval, a model from the Foundation Model APIs, and tools that read engagement metadata. Deployed to Model Serving with MLflow tracing on every request, so a bad answer can be read back step by step.
MLflow evaluation runs a curated question set with LLM judges and expert-reviewed answers before each release; a version that scores below the current one does not ship. Production traces feed the next question set.
The AI Gateway fronts the endpoint with rate limits, payload logging and guardrails. Consultants use the agent through a Databricks App that shows the answer and the cited passages side by side.
Permission-aware retrieval limits the passages available to the agent to documents the consultant is allowed to read.
Passages are cited and one click away; an answer without a source is not shown.
A version that scores lower than the current one on the question set does not ship.
Gateway logs and traces show what was asked, what was retrieved and what it cost, per team.
The index follows the Delta tables; nothing is re-indexed by hand.