AI in production
What “production” means for an agent
Eleven questions that separate a system from a demo. None of them are about model quality, and most projects that stall have not answered more than three.
Syntrigen
LLM workflows
Prompts are code, and they need the same discipline
A prompt change alters system behaviour in production. If it can reach users without review, a test, or a way to roll back, then the most volatile part of your system is also the least governed.
LLM workflows
Structured output is the difference between a demo and a feature
A model that returns prose is a conversation. A model that returns a validated object is a component you can build on. Getting from one to the other is mostly unglamorous contract work.
Data and integration
The integration work nobody scopes
The model is rarely the hard part. Getting clean, current, permissioned data out of the systems of record is the hard part, and it is where most projects quietly stop.
RAG
RAG is a retrieval problem wearing a language model costume
When a retrieval-augmented system gives a bad answer, the model is almost never the reason. It answered correctly from the wrong documents, because retrieval handed it the wrong documents.
Agent architecture
Multi-agent systems are usually one agent and a queue
Most problems described as needing a team of collaborating agents need one competent agent, a work queue, and a way to hand off to a person. The orchestration is the easy part to add and the hard part to debug.
Agent architecture
Tool permissions are an authorisation problem, not a prompt problem
"You must never issue a refund over £500" in a system prompt is not a control. It is a request. The boundary belongs in the tool layer, where it can be enforced.
LLM workflows
Where an LLM belongs in a workflow, and where it does not
The useful question is not whether a language model can do a step. It is whether the step tolerates being wrong occasionally, and what happens when it is.
Evaluation
Why your evaluation suite passes and production still breaks
An eval set is a sample from a distribution you chose. Production is a sample from one you did not. Almost every surprising failure lives in that gap.