When I built the conversational agent for Ubunifu Madness, my NCAA prediction platform, I had a specific problem: the model knew enough about basketball to fabricate plausible-sounding facts. It would confidently report scores that never happened. It would invent player statistics. None of this was malicious. It was the model doing what models do, which is generate plausible continuations of text.
The solution I landed on was structural: a rule at the top of the prompt, and seven tools underneath it that could actually answer the question.
The rule, and the seven tools behind it
The agent had access to seven custom tools, each backed by a deterministic database query:
lookup_team: a team's Elo, record, conference, seed, Four Factors, and advanced statsget_matchup_prediction: the ensemble's win probability for a matchup, with a confidence flag and a plain-language explanationget_conference_info: conference strength metrics and the teams in a conferenceget_top_teams: the top teams by Elo, optionally filtered to one conferenceget_todays_scores: live scores and results for a given dateget_upset_candidates: games where the lower-seeded team has a real chance of winningbuild_bracket: all sixty-three tournament games under a chalk, balanced, or chaos strategy
The system prompt's first rule was blunt: only state facts that come directly from your tool results. If a tool did not return a specific piece of data, do not guess or infer it.
That's the grounding rule. In practice it governs every answer the agent gives about teams, players, games, and predictions.
What it changed
The agent became more cautious, more explicit about where its numbers came from, and considerably less impressive in the surface-level demo. It also became checkable, because every number it stated came back from a tool rather than from the model.
A user could ask "who would win between Houston and Duke?" and the agent would call get_matchup_prediction, then state the model's probability along with the factors behind it, since both come back from that one call. Ask it for the day's upset picks and it calls get_upset_candidates; ask how strong a conference is and it calls get_conference_info. Within that set of questions, the facts came back from a tool result.
When a user asked something the tools couldn't answer ("what's Coach Calipari's career record against Kentucky?"), the agent would say so, instead of confabulating a number.
Where this approach breaks down
The grounding pattern worked here because most of the useful questions reduced to a small set of database operations. Basketball stats fit that shape. Conversations about strategy, narrative, or context ("tell me about the rivalry between these two programs") fall outside the tool boundary, and the agent has to either decline or rely on its own knowledge with an explicit caveat.
I haven't fully solved that boundary. For now, anything that isn't backed by a tool gets a hedge. It makes the agent less smooth, and it makes the unverified parts easy to spot.
What the 2026 tournament showed
The 2026 NCAA tournament was the agent's first live test. The goal, for that season and for the long arc of the project, was the same: gather the performances, look at where the agent and model did well or didn't, and feed that back into next year's features and tools.
| Correct | Accuracy | |
|---|---|---|
| Men (agent) | 46/63 | 73.0% |
| Men (ensemble, since deleted) | 43/63 | 68.3% |
| Women (agent) | 49/63 | 77.8% |
| Women (ensemble, since deleted) | 48/63 | 76.2% |
Headline accuracy clusters between about 68 and 78 percent, about what you'd expect from single-elimination games stacked with close matchups. I originally read that as a little below the model's roughly 80 percent on its 2023-to-2026 holdout. That comparison was invalid, which is what the note at the top retracts: the 80 percent was leaking, and the ensemble rows above are the model I later deleted. The tournament is more useful to me as a postmortem: which seeds did the ensemble overrate? Which features under-weighted late-season form? Where did the grounding rule force the agent to hedge in ways that turned out to be correct? Those are the questions that drive what gets re-engineered before next March.
A word on why the agent and the model disagree. The model produces one input: a win probability for a matchup. The agent surfaces that probability faithfully, because the grounding rule forbids it from inventing or distorting the number. But the agent's final pick isn't a passthrough of the model. It reasons over the model's probability alongside the other grounded tool results, recent form, rankings, the strength-of-schedule context, and lands its own call. That extra reasoning is where the agent's picks can diverge from the raw model's, three games on the men's side and one on the women's. At sixty-three games each, a gap that small is well inside the noise; I read it as the agent doing no worse while weighing more of the evidence, not as proof it forecasts better.
So the grounding rule didn't make the agent a better forecaster, and I didn't expect it to. On accuracy it came out even with the ensemble, inside the margin you'd expect at this sample size. What it changed was where the numbers came from: each one traceable to a tool call rather than to the model's own recall. I wrote at the time that this bought trust. It bought traceability, which is a smaller and more useful thing.
There is more to say here: how the seven tools were scoped, where the grounding rule needs nuance for queries that synthesize across several tools, and how I'd extend the pattern to domains where the answer depends on context the database doesn't hold. The postmortem linked at the top answers part of it, from an angle I did not expect when I wrote this.