When a server fails, you know. Alerts fire, pages load slowly or not at all, someone gets paged. The system tells you it’s in trouble.
Machine learning models don’t do that. A model is a snapshot of the world at the moment it was trained - the shapes of the data, the meanings of the fields, the relationships between them. It doesn’t crash when those change. It keeps answering. The answers just quietly stop being true.
The ways this happens are mundane. A source system renames a field, or starts sending nulls where numbers used to be. A rounding change shifts values just enough to cross the model’s internal thresholds. Customer behaviour drifts season by season until the patterns the model learned no longer describe the people it’s scoring. None of these will ever page anyone - unless you’ve engineered for them.
This is why we treat AI as a run problem, not just a build problem. The disciplines are unglamorous and decisive: validate data before the model sees it, and quarantine what doesn’t conform. Version every model, keep its training data reproducible, keep its outputs traceable. Pin the runtime so the model you tested is exactly the model in production. And monitor for drift the way you monitor uptime - because a confidently wrong model is worse than a down one. A down model gets fixed; a wrong one gets believed.
We’ve practised these disciplines on models compiled into wearable hardware in an R&D fleet, where you can’t ssh in and check - and they are the difference between AI as an asset and AI as a slow-burning incident.
If you’ve shipped a model and nobody’s watching it, that’s worth a conversation. Launch is the halfway point - for AI, more than anything.