Whitepaper
Evaluating agents in production
A working method for deciding whether a change made an agent better or worse — how to build the set, what to measure, how to gate a release, and how to keep it honest as production moves.
Whitepaper
A working method for deciding whether a change made an agent better or worse — how to build the set, what to measure, how to gate a release, and how to keep it honest as production moves.
Evaluation
An eval set is a sample from a distribution you chose. Production is a sample from one you did not. Almost every surprising failure lives in that gap.