Make Your AI Tools Say What They Don't Know

Ask an AI assistant today’s naira exchange rate, last month’s inflation figure, or who represents a particular Nigerian constituency, and it will usually answer confidently and wrongly. That data changes faster than models are trained, and much of it is thin online.

So I built an MCP server for Plus234Feed. It gives assistants like Claude 13 tools over the same data the site runs on: a news archive of 170,000+ articles, exchange rates, stock market data, official statistics and the National Assembly roster.

Getting the data to the model was the easy part. The harder part was making sure the model could not misread what came back. A tool’s output is all the model knows, and it will believe whatever the output implies.

An empty result is an answer

The legislator tool looks up senators and representatives. Its source data is incomplete: it returns fewer members than either chamber has seats.

That sounds like a data problem. It is actually a correctness problem. Ask “who is my senator?” for a state that is missing, get back an empty list, and the assistant will tell the user “there isn’t one”. That is a false statement about the world, not a gap in a dataset.

So the tool reports its own coverage: “X of Y seats on file”. Both numbers are counted from the database at query time, never hardcoded. As the missing records are filled in, the warning shrinks and disappears by itself, with no code change.

If your tool can return nothing for two different reasons, “this does not exist” and “I don’t have it”, the model needs to know which one it got.

Say when the data is from

Nigeria’s official statistics bureau publishes months in arrears. The latest inflation figure might describe a month that ended a quarter ago.

Every statistics response leads with the period it is quoting, so whatever the model repeats carries its date with it. This is the same idea as a timestamp on a dashboard, applied to text a model will pass on to someone.

Return both numbers when both matter

Nigeria has two exchange rates that matter: the official central bank rate and the parallel market rate. They often disagree, and the gap between them is usually the story.

The exchange rate tool always returns both, with the spread. Returning either one alone would be technically correct and practically misleading. When a fact is contested or has more than one valid answer, a tool that picks one for the model is making an editorial decision the user never sees.

Two search tools, because they fail in opposite ways

The archive has two search tools, on purpose.

Semantic search finds articles by meaning. “Why is electricity so expensive?” finds coverage of tariffs and the regulator even when none of those words appear. But it can drift to the wrong subject: the same question can pull in articles about the price of mobile data.

Keyword search is exact. It is the right tool for names, companies and stock tickers, and it finds nothing when the user’s words differ from the publisher’s.

Merging them into one tool would hide the trade-off. Keeping them separate, with descriptions that say when to use each, lets the model choose the one that fits the question.

Fail loudly, and point somewhere useful

Some failures are silent unless you design against them.

Semantic search only works if the question is embedded with the same model the archive was built with. Use a different one and every similarity score becomes meaningless, while nothing appears to fail. So the server pins query embeddings to the model the archive was built with, text-embedding-3-small, and the documentation warns against changing it.

When semantic search cannot run, for example because no API key is configured, it returns an error telling the caller to use keyword search instead. When keyword search times out, it redirects to semantic search. An error that names the alternative gets the model unstuck. A bare failure invites it to guess.

Read-only by design

Every tool is read-only. There is no tool that writes, deletes or publishes, so an assistant using the server cannot change the site. That is what makes it safe to attach to a general-purpose assistant.

The common thread

I believe AI should be the interface to authoritative data, not the source of truth. I make that argument at length in Nigeria’s AI Ambition Needs a Data Foundation. Building this server made it concrete.

The model is good at turning a question into the right tool call and explaining the result. It cannot tell whether an empty list means “none” or “unknown”, whether a figure is current, or whether a single number hides a second one. The tool has to say. Most of the work in a good AI tool is not the AI. It is making sure the data tells the truth about itself.