AI agents for SMBs: what actually works.
Three patterns we saw work (short RAG, human handoff, weekly metrics) and two that don't.
We’ve been implementing AI agents for regional SMBs for over a year. The marketing promise is enormous; the reality is more nuanced. This post is about what we actually saw work.
What works
Short, well-curated RAG: Best results with small, carefully curated knowledge bases — 20–50 well-maintained documents, not 500 raw PDFs. An agent that knows 20 things with certainty is more useful than one that knows 500 with noise.
Explicit human handoff: The agent needs to know when not to know. We implement a confidence threshold: if the query doesn’t match the knowledge base well, the agent routes to a human with a context summary. This prevents hallucinations and builds trust.
Real weekly metrics: A simple dashboard: how many conversations, how many escalations, how many resolved without a human. Without metrics, the agent becomes an ignored chatbot.
What doesn’t work
Trying to automate 100%, and uploading unstructured data. Design the system assuming there will be exceptions — it’s more robust than trying to eliminate them.
AI agents work when the scope is small, the knowledge base is honest, and there’s a clear human backup. They work as intelligent first filters — not complete replacements.