Skip to content

Defensive-choice-with-justifying-comment

Defensive or uncertain choices in AI-generated code are often paired with a comment that justifies the choice in prose. The comment narrates an intent or constraint that does not survive verification: either the constraint does not exist, or it is no longer current, or the justification is a hope rather than a fact.

Where the observation appears

Nine-plus entries demonstrate this:

Entry The defensive choice The justifying comment
unreachable-defensive-guard if x is None: return None for inputs the caller cannot produce "Belt and suspenders" (aabtzu's framing)
swallowed-exceptions except Exception: pass on memory-system init # Memory is optional -- don't break agent init
brittle-error-detection String-match on error message instead of typed exception (often paired with a comment explaining what the substring means)
wrong-tool-for-job TAG instead of TEXT for Valkey memory field # Using TAG instead of TEXT for Valkey compatibility (Valkey supports TEXT)
sleep-based-synchronization await asyncio.sleep(0.5) before driving test step # hope discovery completes
narrating-comments Decorative Args: section restating type info (the docstring IS the narration; the choice and the comment are the same act)
print-instead-of-logging print() in production code (often paired with # Status update for debugging or # Show progress to user)
hardcoded-config-values client.get_or_create_collection(...) omits embedding_function (the omission is the choice; no comment needed — but downstream the EmbedderIdentityMismatchError enforces a constraint that the choice doesn't honor)
resource-leak-no-context-manager Streaming file handles opened outside with (ProjectScylla) # noqa: SIM115 (justification: "streaming writes require persistent open handles") — a legitimate variant where the constraint DOES survive verification, but the suppression becomes a cleanup-discipline risk

The ProjectScylla case is the legitimate variant of the pattern: the comment names a real constraint, the suppression is principled, and the choice is defensible. But the principled suppression is itself a discipline-risk surface — if the class's cleanup raises, the streaming handles leak anyway. The legitimate variant teaches that even correct justifying comments can mask risks if the alternative defense (cleanup discipline) is not separately enforced.

Mechanism

A language model generates code in steps. When the model produces a defensive or uncertain choice, the local generation context often also produces a comment in the same step or the adjacent step. The comment is not an independent verification of the choice — it is another product of the same generation context, drawing on the same priors.

The justifying comment is doing three jobs poorly:

  1. Performative reassurance. The comment looks like the developer thought about the choice. Readers see the comment, see the justification, and treat the choice as informed. The comment narrates intent without verifying the constraint.
  2. Substitute for verification. The model has produced a defensive choice instead of looking up whether the defense is needed (does the cycle exist? does Valkey require TAG? is memory really optional?). The comment is the model's stand-in for the verification step.
  3. Future-fragile narration. Even when the comment was correct at generation time, the constraint may be stale. Valkey adds TEXT support; the comment "for Valkey compatibility" now misleads. The memory system becomes mandatory in a future release; the comment "memory is optional" now lies. The comment does not age with the codebase.

The most diagnostic instances make the unreliability explicit:

  • # hope discovery completes (aios) — the author wrote hope in the comment. The wager is acknowledged in prose; the code itself is the unsafe primitive.
  • # Using TAG instead of TEXT for Valkey compatibility (mem0) — the comment narrates a constraint that is verifiable and turns out to be false.
  • # Memory is optional (hermes-agent) — the comment narrates a design intent; the code's except Exception: pass does not distinguish "memory was optional and not configured" from "memory was meant to work and broke."

In all three, the comment performs a function the code should have performed: distinguish the intentional case from the broken case, verify the constraint, or use a primitive that does not need a hopeful explanation.

How readers can use this observation

The comment-as-justification shape is itself a calibration cue. When auditing AI-generated code:

  • A defensive choice with no comment is sometimes correct, sometimes reflexive — read the code, decide.
  • A defensive choice with a comment that justifies the choice in prose deserves a second look. The comment is not evidence the constraint exists; it is evidence the model produced both the choice and the explanation.
  • Verify the comment against the codebase or upstream documentation:
  • "Valkey compatibility" — check whether Valkey has the alternative.
  • "Memory is optional" — check whether the call path treats memory as optional or as required.
  • "Belt and suspenders" — check whether either belt or suspenders is actually doing work.
  • A comment that says "hope" or "should" or "to be safe" is acknowledging uncertainty in prose. The right cure is usually to remove the uncertainty in code, not to expand the explanation.

Why this is a note, not an entry

This observation was considered for elevation to its own entry and is documented as a note instead because:

  • The defect in each instance is the underlying defensive choice (swallowed exception, hardcoded value, wrong tool), not the comment. The comment is a diagnostic signal, not a defect class.
  • The mechanism is adjacent-generation-product, which is also visible in narrating-comments and in the structural mechanism of how attention-based generation works. Promoting it to an entry would either duplicate narrating-comments or create a "comment-class" entry that doesn't fit the taxonomy's defect-focused framing.
  • The most useful framing of this observation is as a reading-AI-generated-code skill: when you see a comment justifying a defensive choice, treat the comment as a hypothesis to verify, not as evidence.

Implications

For readers of AI-generated code:

  • Justifying comments are not evidence. Verify the constraint they describe.
  • Hopeful comments (# hope X completes, # should be enough) are explicit acknowledgments of uncertainty — read them as the author saying "I am not sure"
  • The pattern provides a quick first-pass audit: find defensive code, read its comment, check whether the comment's reason is real

For projects using AI-assisted development:

  • Code reviews can flag comments that justify rather than explain. "WHY-comments" should describe non-obvious constraints (correct); "JUSTIFY-comments" describe defensive intents (suspect).
  • Test cases that exercise the constraint named in the comment can catch the false-constraint case at CI rather than at audit.