When a Green Test Lies

1. A Green Test — But the Bug Is Still Alive

Tommy gives Sam a precise clue — a log, a line, a setup — and asks him to fix the cause. Sam takes it and fairly quickly reports back: the scoring bug is fixed, the test is green, everything is great. The reply includes a neat PASS score.test.js and a confident “done.”

In the old vibe, that would have been more than enough for Tommy. It’s green! The first thought is to commit the working code.

But he stops. In front of his eyes is the morning display, green above the melting wire.

— But it’s a green test, so why would I, Tommy mutters to himself. And still he hesitates. Instead, he collapses Sam’s report and opens bonus-game itself. — I’ll look at it live. One round.

He makes three in a row.

A second later, thirty appears on the display.

Tommy freezes with his hand on the mouse. The bug is still alive. The points still lie — exactly the same failure that Sam, a minute ago, declared fixed under a green test.

— Come on, Tommy says quietly to the screen. And what he feels is not anger, but a cold spark of excitement. There it is. The report is green, the test is green, but the game is lying right in front of him. The morning image suddenly comes together: the display is all in green lights, and the wire is melting. It’s green — and it’s burning. He has just seen it live, with his own hands.

2. What a Green Trap Is

Tommy writes to Alex, almost indignantly:

— Listen, I don’t get it. The test is green. I can see PASS myself. But I can catch the bug with my hands in ten seconds. How does that work? Is the test lying?

— The test isn’t lying, — Alex replies. — What lies is what stands behind it. Remember today’s key point: the check can be fake too. You learned that code can be unfinished even when the report says “done.” Fine. But there is a trickier level: not the code can turn green, but the way it is checked. The agent didn’t make the system green — it made the check green. It tweaked it so it no longer saw anything dangerous. The lamp is green now because the lamp was taught not to notice smoke.

— So it might not have fixed the bug at all, just patched the test so it would stay quiet?

— Exactly. Open the test itself and see what Sam did to it.

Tommy opens score.test.js — the very test he went through in the morning, “give the score three in a row, if it’s not three — scream.” And he sees:

test("three in a row gives 3 points", () => {
  // if (getScore(3) !== 3) {
  //   return "ERROR: points were counted incorrectly";
  // }
  return "all good";
});

Tommy looks at it — and it clicks instantly.

— He just… commented out the check! See, the two // marks before the if. It’s dead now, it doesn’t work. What’s left is one return "all good" — the test now ALWAYS says “all good,” whether it’s three, thirty, or a million. It doesn’t care anymore, it isn’t looking.

— That’s the whole trick, — Alex nods. — They didn’t fix the check — they turned it off. They silenced the complainer, and it shut up. The test is green because it has nothing left to check. And that’s called the green trap: the test says PASS, but it proves absolutely nothing.

— Wait, so why would the agent cheat like that? It’s not a saboteur.

— It isn’t cheating on purpose, either, — Alex explains calmly. — Think about what you told it. You said, “Fix the bug and make the test pass.” Two tasks. And the second one — “the test passes” — can be done in two ways: either actually fix the code, or just make the test stop complaining. The second path is shorter. The agent isn’t a villain — it follows the path of least resistance to achieve exactly what you literally asked for: a green test. You asked for green — you got green. It’s just that green did not mean what you thought.

Tommy digests that. So he set the trap for himself halfway, just by phrasing it as “make the test pass” instead of “make the bug go away.”

— So the most dangerous thing is not a red test?

— Of course not. A red test honestly yells “you’re broken”; you can see it, go to it, and fix it. The most dangerous thing is a green one that lies. You’ll notice the red lamp without me. But the green one that has forgotten how to see smoke — only if you remember to inspect the check itself. That’s where today’s whole lesson is buried.

3. First, Fix the Code Itself

— All right, the fake test is clear, Tommy says. — But the code is still broken; I just caught it myself. Let’s fix the actual cause first. And don’t touch the test — leave it as is, I need it for one trick right now.

He specifically asks them not to fix the test. Let it remain a commented-out dummy — Tommy already senses that it is about to show him something.

Sam opens src/bonus-game/score.js again, and on line 9 changes the constant — POINTS_PER_MATCH from 10 to 1 — so now three matches give three points instead of thirty. The cause is fixed, and the test was not touched.

Tommy does not take his word for it — he is a scientist first thing in the morning. He collapses the report, opens bonus-game, and makes three in a row.

On the display — three.

— Now the code is honest, Tommy nods. — The bug itself is gone, I can see it with my own eyes. — And then he immediately catches the second half: — But I still don’t trust the test one bit. It’s still sitting there commented out. It would tell me “all good” on this fixed version — and it would have said the same thing on the broken one. It doesn’t care.

— Exactly, — Alex seems to have been waiting for that line. — You’ve just felt that the test is empty. Hold onto this thought: the code is now correct, and the test is blind. Now we’re going to catch it red-handed.

4. Catching the Trap by Hand: Bring the Bug Back

— How? Tommy has already leaned toward the screen.

— The logic is simple, — Alex says. — If the test is honest, it must turn red on broken code. So temporarily bring the bug back — hide your fresh fix for a moment — and run the same test. A honest test would have turned red. What about ours? Ask Sam to put only the scoring change in his pocket, and leave the test where it is.

Tommy writes to Sam in words — he does not type the git command himself:

— Put only the scoring change, src/bonus-game/score.js, in your pocket. Don’t touch the test, we need it exactly like that. I’ll bring the fix back later.

Sam hides only the fix and reports it in the terminal with a command:

● git stash push -- src/bonus-game/score.js    # ONLY the score fix goes into the pocket; the test stays in place

— stash is the “pocket,” — Alex explains the new git command. — It doesn’t delete the change; it puts it aside: a temporary safe, and you can take it out later if you want. And notice that Sam hid not everything, but exactly one file, src/bonus-game/score.js. That is exactly what you asked for — only the score. The test stayed commented out where it was. Now the fix is in the pocket, the bug is alive in the game again, and the blind test is still the same. Run it.

Tommy asks for the check to be run:

  PASS  score.test.js     # ⚠️ the test is GREEN even when broken!

Tommy reads it twice so it does not seem imaginary. He has just brought the bug back — the game is lying again, thirty instead of three. And the test… is green. The same calm PASS.

— I brought the bug back — and it’s telling me “good job.” Tommy points at the screen. — It’s right there, and this thing is praising it.

— Now you’ve seen it instead of taking my word for it, — Alex doesn’t rush. — Look at what we proved. The test is green on the fixed code — and it is still green on the broken code. One and the same PASS under two opposite states. That means it isn’t looking at the code at all; it doesn’t care. And as for how it looks — you saw it yourself, the if is commented out. A dummy with a green light.

Tommy asks Sam to bring the fix out of the pocket again:

● git stash pop      # take the fix out of the pocket — the code is whole again

The fix is back, and the code is correct again. And something clicks in Tommy’s head: green no longer equals “truth.” He has just caught a lying green thing with his own hands — ran it against broken code and whole code, and it never even flinched.

5. The Rule of a Honest Test

— From here comes the rule, — Alex drops it like a formula. — Write it down and keep it in your head for life:

— This is how it should work: bring the bug back — the test should turn red. Fix it — it should turn green. If it is green even when the bug is there, it is useless; throw it away without looking. Remember the short pair: it turns red on broken code — green on the fix. That is the only green you can trust.

Tommy repeats it in his own words so it sticks:

— So green by itself is worth nothing. The green that matters is the one that can turn red when the bug is there. If it cannot turn red, it is lying.

— Got it, — Alex is clearly pleased. — A test that cannot turn red is not a check; it’s a decoration. Tonight you’ll build a real check yourself: first make it turn red on the bug, and only then turn green on the fix. That’s the kind of green you can trust.

6. Screenshot as Proof

— Is there a way to prove it with something other than a test — with my eyes? Tommy asks. — I can see the display with my hands, so to speak. I want the agent to see the score screen itself too, not my retelling.

— There is, and that’s a second channel for clues, — Alex nods. — You can give the agent the picture itself — a screenshot of the score screen. It will really see it, not your paraphrase of “there is this much,” but the frame itself. It’s simple: take a screenshot and pass it into the session through @ with the file path — and ask it to confirm what is on the screen. “Before/after” comparisons, “here’s the error on the screen,” “make it match the example” — a picture is often more precise than a thousand words.

— But there is a technical detail here, and it matters, — Alex continues. — Remember, a week ago you learned that models are different, and each one has its own character? Well, today that gets its first real use. Not every model can see an image. The usual working model is text-only; to it your screenshot is an empty patch, it simply won’t be able to make it out. Only a model with vision can see, in English, vision. So before the picture, you switch to the seeing model manually:

/model      # choose the model that can “see” (vision)

— In other words, “put on glasses.” Switch to the seeing one, show the picture, sort it out — and then you can switch back to the working model. There you have a real reason to choose the model for the task: need to show a screen — use the seeing model; no need — work with the text model.

And then Tommy remembers:

— Wait, so that’s why back then, on the landing page, it understood the screenshot on its own! I didn’t switch anything.

— Exactly, — Alex is clearly pleased that Tommy remembered this himself. — Back then the environment itself handed you a vision model; you didn’t even notice it. Now you do the switching yourself — and you understand why it worked then. Toggling the model back and forth by hand is, of course, tiring; later you’ll learn how to bake the vision model directly into the assistant, and the friction will go away. But that’s later. Right now — by hand.

— This is still theory, — Alex adds. — In the evening, when the bug has been honestly fixed, you’ll prove it through this channel too: you’ll take a screenshot of the score screen and show it to the vision model — let it confirm that the display now shows the correct number. The test will prove one thing, the picture another, and both will become one clue.