An AI demo is built to show what can happen. A production AI system has to manage what will happen, what might happen, and what must never happen.
I have built custom AI models and supporting platforms for fraud prevention, customer service, voice generation, communications, and back-office operations. Across those use cases, the difficult work has been remarkably consistent. The model has to operate inside a product with incomplete information, impatient users, real costs, adversarial behavior, privacy constraints, and consequences when it is wrong.
That changes the question. Instead of asking, “How intelligent is the model?” I ask, “What decision are we improving, how will we know, and what does the system do when confidence is not enough?”
Begin with the decision, not the technology
“We should add AI” is not a product requirement. It is an expression of anxiety, enthusiasm, or both. A useful implementation begins with a specific job: identify likely abuse sooner, assemble the context a support agent needs, generate a voice response with the right pronunciation, or route a communication without making the customer repeat the problem.
The narrower definition creates better design choices. It tells us what inputs matter, what latency is acceptable, what a false positive costs, whether a person should approve the result, and which existing rule-based behavior is already good enough.
AI is not automatically better than deterministic software. If a rule can solve the problem reliably and cheaply, use the rule. Models earn their complexity when the input is variable, the pattern is difficult to express directly, or the output benefits from learned context.
The product includes the context
A model without the right context is being asked to improvise. In customer service, that context may include account state, previous conversations, product configuration, known incidents, and policy. In fraud prevention, it may include behavior over time, relationships among events, and operational history. In voice generation, it includes pronunciation, tone, language, timing, and the purpose of the message.
More context is not always better. It increases cost, latency, attack surface, and the chance that irrelevant information steers the result. The product work is deciding what the model is allowed to know, when it is allowed to know it, and how that context is assembled and verified.
This is why I prefer AI inside the operating platform rather than beside it. A separate chatbot often asks the user to reconstruct context the company already has. A well-integrated system can act on the same permissions, data, and workflow as the rest of the product.
Evaluations are product specifications
Traditional software gives us a comforting relationship between input and output. Models give us distributions. The same system can be excellent across most cases and dangerously wrong in a narrow category. A handful of successful prompts tells us almost nothing about that shape.
I treat an evaluation set as a living product specification. It should include ordinary cases, high-value cases, ambiguous cases, adversarial cases, and the failures the team has already observed. The scoring should reflect the business consequence, not only linguistic similarity or a generic quality metric.
For fraud prevention, a missed abuse case and a blocked legitimate customer are different failures with different costs. For customer service, a fluent but invented policy answer may be worse than no answer. For voice, intelligibility and correct names may matter more than an impressive imitation of human texture.
Quality also includes latency, cost, consistency, and the amount of human review required. A system that produces a beautiful result too slowly for the workflow has failed the product requirement.
Uncertainty needs an interface
People often describe “human in the loop” as if adding an approval button solves the problem. It does not. The reviewer needs to understand what the system did, what evidence it used, and what kind of uncertainty remains. Otherwise the human becomes a rubber stamp for a machine that appears more confident than it is.
A trustworthy AI product needs explicit behaviors for uncertainty: ask for missing information, present alternatives, abstain, fall back to a deterministic path, or escalate to someone with the right authority. The correct behavior depends on the consequence of the decision.
The user experience should also make reversibility clear. Can the action be undone? Can the original input be recovered? Is there a record of who or what made the decision? Confidence grows when people know the system will not quietly trap them in an irreversible mistake.
Security changes when language becomes an input
AI expands the input surface. Instructions can arrive inside documents, customer messages, retrieved content, or tool output. A system that can read sensitive context and take actions has to treat every piece of external language as potentially untrusted.
The basic disciplines still matter: least privilege, data minimization, isolation, validation, rate limits, audit trails, and careful handling of secrets. The AI-specific design should sit on top of those controls, not replace them with a promise that the model has been told to behave.
Fraud systems make this especially visible because users actively adapt to the controls. But the same principle applies to support and operations. If the system changes behavior based on uncontrolled input, somebody will eventually discover a way to steer it.
Production is a feedback system
Models change. Providers update them, data drifts, user behavior evolves, and the product itself changes the environment that generated the original examples. A passing launch evaluation is therefore the beginning of measurement, not the end.
I want to see which cases are being escalated, which outputs are edited, which customers repeat the same request, where latency spikes, and which categories create disagreement among reviewers. Those signals turn production behavior into the next evaluation set and the next product decision.
This feedback loop is also where teams should resist the urge to automate everything. If human review keeps catching a category the system does not understand, the answer may be better context, a narrower scope, or a deterministic rule. Expansion is only progress when the measured system becomes more useful.
The real advantage is operational
The organizations that benefit most from AI will not necessarily be the ones with the flashiest demo. They will be the ones that redesign work around the capability: deciding what should disappear, what should accelerate, where human judgment becomes more valuable, and how learning returns to the product.
That is the work I find interesting. Not adding AI as decoration, but building a system that makes a difficult product more responsive, a risky decision safer, or a complicated operation feel simpler—and then proving that it keeps doing so in the real world.
Further listening
I discuss related ideas in XTraw AI, Data Unchained, and Avant-Garde Entrepreneur.