Tag
Everything tagged "Evaluation": 3 pieces across articles and the journal.
Running a coding agent unattended is tempting and mostly a trap. The thing that makes it safe isn't a better prompt. It's an external, deterministic gate the model cannot talk its way past. Here is the principle, a working pipeline that embodies it, and the failure modes that matter.
Fetching the right documents is necessary but not sufficient. Grounding (answers that are actually entailed by the retrieved evidence) is a separate property you have to design for and measure. Here is how I think about the gap, and the evaluation that closes it.
Rule I keep relearning: build the evaluation before the feature, not after. It feels slower. You sit down to add a capability and instead spend the first hour assembling twenty lab…