How to stop an AI from inventing apartment numbers in your CRM
A dev.to engineering guide shows how to make call-summary AI admit what it doesn't know, so guesses never become entries in a customer record.
The short version
- Missing and null mean different things: a missing key breaks the format, while null honestly says 'not supplied'.
- Actions should come from a closed list; widening the list until every model phrase passes weakens the safeguard.
- Invalid output gets one bounded repair attempt using the original transcript plus the validation error.
- A well-formed result can still contradict the call, so a source-fidelity review is needed before any CRM write.
- Compare AI providers on the same transcripts, rules and retry budget, and measure time to a usable action.
Why it matters
Anyone using AI to fill business records faces the same risk: a fluent summary can hide a made-up detail or an overeager action. The post argues the safeguard belongs at the point where data is written, not in the model's prompt alone. The idea also works without code: give the AI an allowed list of answers, and let it say 'nothing here'.
The failure that looks fine
As dev.to reported, the post starts from a hypothetical sales-call scenario for a property-management company. A caller says to send the floor plan, says they haven't picked a unit, and says "I'm not booking a tour yet." The AI's output is a readable summary with the unit field left out entirely and the action labelled "interested."
That label sounds sensible, but the CRM (the customer-records system) has no operation called that. A tidy paragraph and a green 'request succeeded' light on a dashboard don't show the software received a valid instruction. The author's worry is the opposite failure too: an unknown apartment number turning into an invented one, or tentative interest turning into a scheduled tour.
Missing is not the same as null
The core advice is to separate two ideas. A missing key means the output broke the expected shape. A key that is present with the value null (programmer shorthand for 'no value') can truthfully say the caller never gave that information.
The post's example contract requires three keys: summary, unit and action. Unit and action are allowed to be null. If a field is required but 'nothing' isn't an allowed answer, the model feels pressure to fill the slot with something, which is the wrong incentive when the destination is a customer record.
One technical catch: when a field is limited to a fixed list of values (an enum), null has to be added to that list explicitly. Allowing the null type alone doesn't lift the restriction. Placeholder strings such as unknown or N/A should be rejected rather than stored as if they were real identifiers, though the author says each business should check that this rule doesn't block a legitimate value.
Keep the action list short and closed
The example allows only two actions: send_details and schedule_tour. The post treats that list as a deliberate decision. Each entry is a real CRM operation, so a new label shouldn't be accepted without a change to the application.
When the model produces a phrase like "interested," the author's advice is to keep it in the free-text summary, where nuance belongs, and map it later if the business wants to. Stretching the list until every phrase passes makes the check meaningless.
There is also a warning about overcorrecting. Converting every invalid value to null makes errors disappear but throws away information. If the caller asked for the floor plan, the source supports a details action, and a good repair recovers that meaning instead of blanking it.
One retry, then a human
The recommended flow is simple. Check the AI's output against the contract. If it fails, give the model one repair attempt, showing it the original transcript and the specific error. If that also fails, hold the item for review. Nothing touches the CRM until the check passes.
The post includes a complete example program in Go that exercises this loop without writing to any CRM. Its single repair and 30-second overall time limit are demonstration settings, not speed measurements. Even when the output passes, the program prints a reminder that a source-fidelity review is still needed.
That reminder matters because a valid format isn't proof of truth. A perfectly formed schedule_tour label can still contradict what the caller said. Rejected extractions and actions that actually reached the CRM should also be counted separately, so one success percentage doesn't hide where the boundary failed.
Comparing providers fairly
The post also discusses choosing the AI behind the pipeline. A fair comparison of OpenAI, Anthropic Claude and Google Gemini needs the same transcripts, nullable fields, action list, rules and retry budget. Otherwise one vendor can look better simply because it was asked for less.
The author says each direct integration means maintaining a separate adapter, while a compatible intermediary can keep an existing client's interface steady when the provider changes. The post names Infrai as one such service, describing it as one key, one bill and one REST API covering 295 routes across 20 modules. That is the post's own description of an integration convenience, and the author is explicit that it is not evidence of better summaries.
On speed, the guidance is to measure time to a validated, usable action, including repairs and review, rather than the first response. No latency winner is claimed.
What to watch
- Whether source-fidelity checks, confirming the output matches the transcript, become a standard step alongside format checks.
- How different AI providers handle required-but-nullable fields and closed action lists when tested on identical transcripts.
Try it
Take five real call notes or emails and ask your AI tool for a summary plus one action from a short list you write, with 'none' allowed. Check whether it ever invents a detail or forces an action.
Sources and links
- Reportingdev.to: the article we sourced
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.
Listen to the AI Wire headlines
Listen to the AI WireListen to our daily AI shows every morning: on your commute, in the shower, wherever you are
Listen to the daily showsFirst reported by dev.to · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
