Two ways to give an agent your company's context

An agent that can write well and call tools competently will still do a mediocre job for you if it doesn’t know that Acme churned last month, that Priya owns renewals now, or that your standard payment terms changed in March. The model isn’t the constraint. What it knows about your company is.

There are two ways to solve that, and they lead to different products. From the outside both get described the same way: the agent knows your business.

Fetch it when the question arrives

Jean-Denis Greze, the CEO of Town, described his architecture to Casey Newton in August in more detail than founders usually give. Town samples your email and calendar to build a small wiki, five to 20 pages, deliberately abstracted so it holds little private information. Then, in his words:

“You can think of the wiki as a map: when you ask the system to do something, it looks in the wiki, which refers to emails or calendar events, and pulls those into the session. And in addition to that, it does federated search. When you ask a question, we use the search engines of Gmail, Calendar, Notion, all the other services, just for that question, to pull in the right data live. We don’t reindex all the other data.”

That last sentence reveals the design decision. The map is small and cheap to keep. The heavy lifting happens per question, against search engines someone else already built and already keeps current.

It’s a good answer. You inherit Gmail’s index instead of paying to rebuild it. Nothing goes stale, because you’re reading the source at the moment you need it. You store much less, which shrinks what there is to protect. And you’re never wrong about what’s in an email, because you just read the email.

What live retrieval reaches

Retrieval only finds what a query can name.

Three problems come up constantly:

Things written in words you’d never search for. Someone ends a long thread about refunds with “let’s go back to the old way”. That message changed your return policy, and it never contains the phrase “return policy”. A search for the policy skips it and hands you the older message that does use those words, which is now wrong. Reading the thread as it arrives catches the change, because the conversation around it says what the old way was.

Things that only exist across messages. A customer’s address changed in a signature in June, and the contract you’re about to renew still has the old one. You only catch that if the knowledge layer kept both.

Things you don’t know to ask for. Live retrieval answers the question you thought to ask. Say a customer runs 480 stores and mentioned once, in passing, that they’re closing 100 of them. You open the account to prep a renewal, see 480 against a plan that tops out at 500, and pitch the bigger tier to a company that’s about to shrink. Nobody types “are they closing stores” into a search box, because nobody knew to wonder.

You can widen the map to cover more of this. But the map grows, and the moment it holds real content rather than pointers, you’re maintaining a knowledge layer and the privacy and freshness advantages start converting into upkeep.

Maintain the layer

We embraced building the knowledge layer to solve these problems. I led a product team building knowledge management products at Facebook, for internal use and for external customers via Workplace, so I’ve watched this approach fail before.

Our approach processes messages as they arrive. Commitments, decisions, contact changes and open questions get written into a layer that persists, and automations update it as things change. You’re not asking a question at read time. You’re keeping a model of the business that a question can be answered from.

This addresses the three problems above. The knowledge layer records that refund thread as a change to the return policy, so a later question finds it without having to guess which words the thread used. Contradictions surface because it holds the most recent data and the history. And the store closure lands on the account before anyone thinks to ask, so nothing depends on the right query occurring to you.

The costs are large:

Upkeep never stops. The knowledge bases I’ve worked on before died on upkeep rather than on the idea being wrong. Somebody has to maintain it, and since it’s not typically the maintainer’s primary job, it rots. The bet is that extraction and updating happen automatically, or the layer is worthless.

Building the initial layer is critical. You can import what’s already structured, a CRM most obviously, and you can process the back catalog of messages. Until that finishes the layer is empty and worth nothing, which leaves a bad first impression. Live retrieval has no cold start at all, because it reads systems that were already full.

The question I don’t think either of us has settled

Cold start is where I’d push back on my own side hardest, and it’s the part I’d most like to be argued with about.

The obvious answer is to build the initial layer out of the message history that’s already sitting there, since a year of email contains most of what a company knows about its customers. We’ve been testing whether that works. Early results on narrow slices are encouraging.

Both are reasonable, and which one works better is still an open question. Right now a buyer can’t even tell them apart, because both get sold as “the agent knows your business”.