Observability
Observability for LLM systems: what to log when the output is text
Conventional monitoring tells you the request succeeded. For a model-backed system that is nearly useless — the interesting failure returns 200 and says something wrong.