The exercise
Four failures. For each: reproduce it, watch it fail, then add the guard. Work in a git repo
and commit after each fix, so you can see the four diffs afterwards.
Break 1: the runaway
Improve agent.py. Keep going until it cannot be improved further.
What you see: it never stops.
Fix: MAX_TURNS and the repeat detector from 4.1.
Check: run it again. It stops, and tells you it hit the limit.
Break 2: the false finish
Add a word_count tool, add a test for it, and mention it in README.md.
What you see: two of three done, reported as three of three.
Fix: the verify step from 4.2, with a check per file.
Check: run five times. Count how often verify fires. Write that number down — it is your
first eval result, and Module 7 turns it into a real one.
Break 3: the silent failure
Take the exit code and stderr out of your bash result, so it returns only stdout. Then:
Run the tests and fix anything that fails.
What you see: it reports success. Nothing was fixed. The test command failed and returned
an empty string.
Fix: restore exit code, stdout, and stderr, per 4.4.
Check: run it again. Watch it read the failure and act on it.
Break 4: the cut-off reply
Set the output limit to 200 tokens and ask for something long:
Explain everything agent.py does, function by function.
What you see: the answer stops mid-sentence. If your code ignores reply.stop, it looks
like the model gave a short answer.
Fix: the stop-reason handler from 4.3.
Check: run it again. You get a clear [reply was cut off] instead of a confusing result.
Read your four diffs
git log --oneline
git diff HEAD~4
About forty lines of guards. Read them as a group, because they have a shape in common:
Every one turns a silent failure into a loud one. None of them make Rover smarter. They
make it honest — about stopping, about finishing, about failing, about being cut off.
That is what hardening an agent is. Not better prompts. Not a better model. Code that refuses
to let a failure pass as a success.
The one that is still missing
You have not fixed the underlying cause of Break 2. You added a check that catches it.
There is no fix for the underlying cause. The model will sometimes believe it did something it
did not, and no prompt reliably prevents that. What you can do is never take its word for it.
That is not a limitation of this model or this year. It is a property of asking a system to
report on itself, and the answer — check from outside — is the same answer as in every other
part of engineering.
Hardening an agent means turning silent failures into loud ones. You are not making it
smarter. You are making it honest.
Try this before the next module
Run all four broken versions once more, with all four guards in place.
Then read rover.log for each run. Every failure should be visible in the log without you
having watched it happen. If one is not, your log is missing a line — add it now, because
Module 5 makes the runs longer and harder to watch.