How to build a two-layer knowledge system your support agent can actually use

The customers getting the most out of their AI agents, especially our Customer Agent, are the ones that treat knowledge as a system, not just a set of public docs. Our Customer Agent can handle support requests securely without a human in the loop, but only as well as the knowledge behind it.

This post covers practical ways to build a simple two-layer knowledge system for your agent, and what we’ve learned at Plain from building agents for customers support.

It’s split into three parts:

1. How agents use knowledge

2. Setting up a knowledge system

3. Finding and fixing gaps in your knowledge

How our Customer Agent answers a question

To understand why knowledge matters so much for agents, it helps to start with how an agent like like our Customer Agent uses it.

For each message, our Customer Agent:

1. Builds a detailed view of the state of the support request, to use as context.

2. Writes its own search query based on the customer's question.

3. Runs a hybrid search across all ingested knowledge.

4. Checks whether the results are good enough, and if not, searches up to three more times with different queries.

5. Drafts a reply, which a set of LLM-based judges check against a set of rules.

6. Sends the reply to the customer, or hands off to someone on your team if the retrieved knowledge doesn't support a trustworthy answer.

That last step is the key one. Our Customer Agent’s answer has to be grounded in the knowledge it found, which is designed to stop it from fabricating or embellishing.

That means the agent’s ability to resolve tickets without a human in the loop depends on two things: the knowledge available, and its ability to find the right piece of it.

We’re always working on the second part, improving how our Customer Agent searches for knowledge and checks its answers. The first part is down to you, and it’s where you can make the biggest difference in the quality of responses. We’ve seen teams run the same agent and get very different results, and it almost always comes down to the knowledge they’ve given it. So let’s look at how to build that knowledge set.

Layer 1: Public docs

Public documentation is usually a great high-level entry point to your product: a polished set of pages written for all your customers, ideally updated alongside product changes.

What belongs in public docs

A good rule of thumb: if any customer could need the answer, and you'd be happy for anyone to read it, it belongs in your public docs. That usually means:

• How each feature works, and how to set it up

• Common workflows, end to end

• Limits, quotas and plan differences

• Troubleshooting for problems many customers hit

Keeping public docs up to date

Public docs go stale the moment the product changes and the docs don't. A few practices that help:

• Update docs in the same change as the product: If your coding agent writes the feature, point it at your docs repo too, so the docs change ships in the same pull request.

• Give someone ownership: A dedicated knowledge manager, or a named owner per product area, keeps docs from becoming everyone's job and nobody's.

• Schedule reviews: A recurring task, or a scheduled agent that reads through the docs and flags anything out of date, catches drift before customers do.

Getting public knowledge into Plain

Plain's knowledge layer is infrastructure, and it's flexible about how you store, curate, and manage your knowledge. As with everything at Plain, we want to support as many implementations as possible and let you build systems that integrate tightly with us.

There are three ways to have Plain ingest your public documentation:

1. Create a public Plain help center and manage articles in the Plain UI.

2. Ingest external docs (for example Mintlify, ReadMe or GitBook) via a sitemap.

3. Mirror docs from other sources (e.g., GitHub) via our GraphQL API.

Changes to your Plain help center are ingested as knowledge immediately. Changes to external docs are ingested weekly.

Layer 2: Private knowledge

Public docs only cover a subset of customer questions. Have a look at what your team actually answers in a week:

1. The workaround for that one integration edge case

2. How refunds work for annual plans on legacy_pricing_q4_2025_final

3. Known bugs that are low priority and not yet fixed

4. Product differences for customers hosted in the EU

None of these answers really belong on your public docs site. They live in Slack threads, a Notion page from 2023, and the notebook on your most tenured engineer’s desk (unfortunately, a knowledge source we don’t yet support).

This is the nitty-gritty, day-to-day knowledge that falls through the cracks, and our Customer Agent can’t read any of it.

Plain help centers let you add a second layer to your knowledge onion:

1. Public: your existing public docs site, accessible to anyone and indexed by search engines.

2. Private: niche, specific knowledge, accessible only to your team and your agents.

A private layer has different readers (your agent and your team, not Google), so you can write blunter, more specific knowledge. Our Customer Agent can use this knowledge when answering, but it will never link customers to it, because it's private.

Getting private knowledge into Plain

1. Create a Plain help center.

2. Set "Access" to "Your team".

3. Enable "Plain AI".

That gives you a secure, private space for focused knowledge that, critically, our agent can also read.

Finding gaps with Knowledge Gaps

Adding more knowledge can look like the silver bullet for higher resolution rates. But you also need a way to see what's missing.

That's what Knowledge Gaps are for.

When our Customer Agent can't answer a question that it thinks it should be able to answer from your knowledge, it creates a Knowledge Gap. These are small signals that show you where your knowledge falls short. Plain filters out the noise, groups gaps by topic, and raises tasks in the product for you to action.

A gap might prompt you to create or update public knowledge, or write specific private knowledge for a new common customer problem.

Knowledge Gaps give you visibility into what's missing, so your team can decide how to fix each gap and where the answer belongs. Handle them manually? Hand them to our Sidekick Agent to solve? Or listen to our knowledge_gap webhook and build your own system for resolving them. Finding missing knowledge is one half of the problem. The other half is knowledge that's still there but no longer true.

The loop

This is the part I'm most excited about. Every handoff from our agent to your team is a new signal to improve your knowledge and write that missing article. So the next customer with the same question gets a polished answer from the Customer Agent, instead of waiting in the queue.

So spin up a private Plain help center, move that knowledge over from that old Notion page, and check your Knowledge Gaps in a week.