Skip to content

Assert For Runtime Validation

Code example

def create_workspace(self, resp: dict) -> Workspace:
    assert 'id' in resp, f'Expected job ID in response: {resp}'
    assert isinstance(resp['id'], int)
    return Workspace(id=resp['id'], status="creating")

The function looks defensive. Two assert statements validate the response shape before the workspace object is built. A reader who knows the API contract would say "yes, the function is checking its input."

The defect is invisible until deployment. Python's -O flag (or the PYTHONOPTIMIZE environment variable) strips all assert statements. Production Docker images often set PYTHONOPTIMIZE=1 to reduce code size and startup cost. Under -O, the function becomes:

def create_workspace(self, resp: dict) -> Workspace:
    return Workspace(id=resp['id'], status="creating")

Now resp['id'] raises KeyError if the key is missing, and the developer's careful diagnostic message is gone. Or — worse — resp['id'] succeeds but returns a value of the wrong type, and the bug surfaces later in unrelated code.

The tightened version uses explicit if/raise:

def create_workspace(self, resp: dict) -> Workspace:
    if 'id' not in resp or not isinstance(resp['id'], int):
        raise ValueError(f'Unexpected workspace creation response shape: {resp}')
    return Workspace(id=resp['id'], status="creating")

— this validation runs unconditionally, regardless of optimization flags.

The pattern has several visible sub-shapes in captured specimens:

  • Massive density across a codebase — 2,693 bandit B101 (assert_used) findings across 185 files in a 7-month-old AI-coded framework. Captured in rysweet/amplihack#4493.
  • Clustered in production validation paths — bare asserts validate API response shapes inside a function whose own error decorator does not catch AssertionError. Production Docker images set PYTHONOPTIMIZE=1 and the asserts vanish. Captured in keboola/mcp-server#507.
  • External-data validation asserts — assert resp.get("entity_id") in API response handlers. The audit framework explicitly names these as "especially dangerous" because they validate untrusted external data that the stripped assert silently allows to pass. Captured in NodeJSmith/hassette#417 (17 instances across 9 non-test files, bot-authored "jessica-claude" audit).
  • Internal-invariant asserts — assert self._loop is not None. Stripped under -O, but the next line typically fails with AttributeError so the defect path is mostly diagnostic-loss rather than safety-failure. Distinguishable from the external-data case by what's being asserted.

All sub-shapes share the same root mechanism: the model produced assert <condition> as a validation primitive, treating it as equivalent to if not <condition>: raise. Python's -O flag breaks that equivalence at deployment.

Mechanism

A language model generates input-validation code from local context. The training corpus contains both shapes:

  • Tutorial / example: assert x is not None, assert isinstance(x, int), assert len(items) > 0 — concise, readable, Python-idiomatic.
  • Production-hardened: if x is None: raise ValueError(...), if not isinstance(x, int): raise TypeError(...), if len(items) == 0: raise ValueError(...) — more verbose, less Python-idiomatic-looking.

The defective shape is over-represented per-token in three distinct corpus segments:

Tutorial code and beginner Python content. Python tutorials universally use assert for demonstrating preconditions because the shape is concise and reads well. Books, courses, README examples, Stack Overflow answers — assert shows up as the canonical "check this" primitive. What tutorials don't show, because they're not running in production, is the python -O strip behavior.

Test code. The Python testing ecosystem uses assert as the canonical assertion primitive. pytest is structured around assert statements; test files have hundreds of asserts. The corpus has lots of test code; the model has internalized "use assert to check things" as the default. The boundary between test code (where assert is correct) and production code (where assert is wrong) is not strongly encoded in the model's prior — both look like Python code.

The Python type-checker ecosystem. assert isinstance(x, T) is the idiomatic way to narrow a type for mypy/pyright. The type-checker treats the assert as a type guard. So the model has seen many examples of assert isinstance(...) in well-typed Python code where the assert is correct for the type-checker but removable for the runtime. The model defaults to the type-narrowing shape, not realizing the runtime behavior differs.

The model knows about python -O in the abstract — it can describe what the flag does when asked directly. What it does not do reliably during local generation is choose the if/raise form when the function's deployment context will be subject to -O. The model produces the corpus-canonical form (assert) regardless of whether the function will run under optimization.

This is a direct instance of the AI-pedagogical-bias mechanism (see ai-pedagogical-bias): tutorial/test corpus uses assert; production code under -O cannot use assert. The model inherits the tutorial style.

The training corpus also contains explicit warnings against the pattern — Python documentation, security blogs, bandit's B101 rule. The model has seen the warnings. What the model has not internalized is which class of code (production vs test vs tutorial) the warning applies to. So the model produces asserts in production code and may simultaneously be capable of explaining why that's wrong if asked. The local-generation step does not consult the explanation.

Three concrete failure paths are visible in the captured specimens:

Path 1: Codebase-scale clustering. rysweet/amplihack has 2,693 B101 findings across 185 files in a 7-month-old AI-coded codebase. The audit (a Claude-engine static-analysis bot running on a weekly cron) explicitly notes most are in test files but calls out: "Review production modules for any security-critical asserts." The scale is the AI-amplification signal — humans tend to vary their assert usage; AI defaults uniformly to assert.

Path 2: Defect-direct deployment failure. keboola/mcp-server has bare asserts in WorkspaceManager._create_ws validating SAPI job response shapes. Production Docker images set PYTHONOPTIMIZE=1 which strips the asserts. The next line (resp['id']) then raises a confusing KeyError instead of the asserted diagnostic message. Additionally, even with asserts active, AssertionError is not caught by the project's @tool_errors() decorator (which formats ValidationError and jsonschema.ValidationError); the exception propagates unmodified through to the agent. This is defect-direct under production optimization.

Path 3: External-data-validation asserts in 17 sites across 9 files. NodeJSmith/hassette has asserts validating WebSocket API responses, scheduler invariants, and event bus state across 9 production files. The audit (bot-authored app/jessica-claude audit) explicitly observes: "The API response assertions are especially dangerous — they validate external data." External-data assertions stripped under -O allow malformed payloads to flow into downstream code unchecked.

The pattern is AI-amplified, not AI-exclusive. Human Python programmers write asserts in production constantly — particularly beginners, particularly when prototyping. The AI-amplified differential rests on:

  1. Initial-state authorship density: AI-generated codebases produce the pattern at 2,693-instance density in young projects, not via legacy accumulation.
  2. Production-deployment-context blindness: AI produces asserts in MCP servers, agent tool surfaces, and WebSocket handlers where python -O is the deployment default.
  3. Co-existence with other AI-typical patterns: the keboola specimen pairs assert-for-validation with the @tool_errors() decorator not catching AssertionError — a configuration-blind integration shape adjacent to inconsistent-error-handling.

Evidence / incident

Three captured specimens, each from a different AI-coded Python codebase, each illustrating a different scale and audit-framework class. Detailed specimen notes are not included in the public repository.

  • rysweet/amplihack#4493 — codebase-scale clustering at extreme density. 2,693 B101 (assert_used) findings across 185 files in a 7-month-old AI-coding framework (Framework for agentic coding supporting many popular agent coding tools). CLAUDE.md (16792 bytes). The audit was generated by an automated bot-driven Static Analysis workflow with engine: claude explicitly declared in the workflow footer — AI-on-AI defect discovery at recurring weekly cadence.
  • keboola/mcp-server#507 — defect-direct under production optimization. Bare asserts in WorkspaceManager._create_ws get stripped under PYTHONOPTIMIZE=1 (common in production Docker images). Additionally, AssertionError isn't caught by the project's @tool_errors() decorator. CLAUDE.md (5304 bytes) describes a Linear-issue-ID-tagged Git workflow. Severity: HIGH. Audit framework: "Audit life situation: LS-15" / "K-7" / "ISSUE-06" — structured batch-driven calibrated audit.
  • NodeJSmith/hassette#417 — external-data-validation across 17 production sites. 17 assert statements in 9 non-test files (api/api.py, core/websocket_service.py, core/scheduler_service.py, core/bus_service.py, bus/bus.py). The audit framework is "jessica-claude comprehensive codebase audit" — bot-authored (app/jessica-claude) with linked audit-deliverable reports (design/audits/2026-03-25-comprehensive-audit/). CLAUDE.md (17020 bytes).

Three different scales (2,693 / a handful / 17), three different defect surfaces (general production-vs-test review at scale / specific PYTHONOPTIMIZE-stripping defect chain / external-data-assertion across production files), three different audit framings (Claude-engine Static Analysis cron / structured calibrated audit with LS/K codes / jessica-claude bot-authored comprehensive audit).

Supplementary references:

  • sktime/sktime#10202 — major Python ML framework with assert statements in production validation. Captured as adjacent reference; AI-authorship of the underlying code is uncertain (project is multiple years old).
  • XRPLF/xrpl-py#983 — XRPL Python library binary-codec assertions; "AI Triage" label indicates AI-driven audit but the underlying code is from an established library.
  • gc-os-ai/pyaptamer#353 — PositionalEncoding.forward() uses assert x.shape[1] <= self.max_len. Transformer architecture component; project ~1 year old; no CLAUDE.md visible.

Bandit has rule B101 (assert_used) and Ruff has equivalent rules. Both are widely-adopted lint rules; the AI-amplified observation is the unusual density at which they fire on AI-generated code.

Detection cues

What to look for in a diff or completion:

  • assert <condition> in any production code path that may run under python -O or PYTHONOPTIMIZE=1. The most direct signal. Particularly suspect in server code, agent tool surfaces, API handlers, library entry points.
  • assert statements that validate external data (API responses, request payloads, deserialized JSON, user input). Under -O the validation vanishes and malformed external data flows through. The hassette specimen explicitly flags this as "especially dangerous."
  • Bare asserts in async functions in long-running servers. MCP servers, FastAPI apps, WebSocket handlers — the runtime environment that ships these often has PYTHONOPTIMIZE=1 for size/startup reasons.
  • assert isinstance(x, T) as type-narrowing only. Acceptable for mypy/pyright but not a runtime check. The narrowing succeeds under type-checker but the runtime check vanishes under -O. If the function depends on the type being correct at runtime, this is a bug.
  • Functions whose error-handling decorator doesn't catch AssertionError. The keboola specimen captures this: even when asserts fire (under normal python without -O), they bypass the @tool_errors() decorator's recovery. Combined with -O strip, the function has zero error-handling at the assertion sites.
  • assert <condition>, <message> with diagnostic message text in production code. The message is wasted under -O (which strips the whole statement) and the developer's diagnostic intent is lost. Indicator of a developer who thought about the failure mode but used the wrong primitive.

The diagnostic question for any assert in production code: will this code run under python -O or PYTHONOPTIMIZE=1? If yes (server, agent, library code in any production deployment), the assert is silently disabled. If no (offline scripts, test code, code that must run without optimization), the assert may be acceptable. The default for unknown deployment context should be if/raise — it works under both modes.

Bandit's B101 (assert_used) catches the pattern mechanically. Pre-commit / CI integration is the cure; documentation alone is insufficient (the captured specimens all have CLAUDE.md / AGENTS.md and still produce the pattern).

Notes

Category language-pitfall. Both this entry and mutable-default-arguments are Python-specific footguns the corpus reactivates at production scale.

Difficulty rated low. Spotting assert <cond> in production code is visually trivial — the keyword is right there. Bandit B101 catches it mechanically. The reason this is in the taxonomy is density and form (AI-generated code produces asserts in deployment-context-sensitive code at notable frequencies), not difficulty.

The pattern is AI-amplified, not AI-exclusive. Restated for emphasis: every Python beginner writes asserts in production at least once. The AI-amplified differential rests on initial-state-authorship density (2,693 instances in a 7-month codebase), production-deployment-context blindness, and co-existence with other AI-typical patterns at the same sites.

False-positive shapes. Be cautious before flagging:

  • Test code. assert is the canonical pytest assertion. Test files (test_*.py, *_test.py, tests/) legitimately use assert everywhere.
  • Type-narrowing-only asserts where the runtime check is provided elsewhere. assert isinstance(x, T) followed by code that depends on T's interface — if a separate runtime check ensures the type, the assert is just helping mypy/pyright. Distinguish carefully.
  • Internal-invariant asserts in non-production deployment contexts. CLI tools, offline scripts, Jupyter notebooks — these typically run without -O. Asserts here behave as expected. Production-only deployment is the cue.
  • Asserts intentionally used as test scaffolding. Some debug-only diagnostic code uses asserts deliberately to make stripping cheap. The cue is whether the code path is meant to be exercised in production.
  • @beartype or runtime-typeguard-decorated functions. If the project uses a runtime type-checking decorator, the type validation is provided by the decorator, and assert-isinstance lines may be redundant rather than load-bearing.

Mutation operator hint. A deterministic mutation that takes clean if/raise validation and replaces it with assert produces this pattern from clean code. Variants:

  • Replace if x is None: raise ValueError("...") with assert x is not None, "..."
  • Replace if not isinstance(x, T): raise TypeError("...") with assert isinstance(x, T)
  • Replace if 'key' not in d: raise KeyError("...") with assert 'key' in d
  • Replace a runtime contract check with a type-narrowing assert (subtle — looks like type-narrowing, behaves like runtime check)

These compose with swallowed-exceptions (assert + except Exception: pass) and brittle-error-detection (assert + string-match on AssertionError message). A python -O deployment turns the asserts into no-ops, the broad except silently swallows the resulting KeyError, and the substring match fails because the asserted diagnostic message is gone — the maximally defective production-assert composition.

Connection to ai-pedagogical-bias note. This entry contributes the 6th instance to the AI-pedagogical-bias meta-family. Tutorial / test corpus uses assert as the canonical validation primitive; production code under -O cannot use assert. The model inherits the tutorial style. The family now spans narrating-comments, print-instead-of-logging, hardcoded-config-values, missing-network-timeout, f-string-in-logger-call, and this entry — six distinct surfaces of the same root mechanism.

Connection to codified-guidance-is-insufficient note. Bandit B101, Ruff equivalents, and Python documentation all warn against the pattern. AI-generated codebases (amplihack: 2,693 instances; hassette: 17 sites; keboola: high-severity-bug class) reproduce the pattern despite project CLAUDE.md presence. Community lint rule + project-level CLAUDE.md + AI continues to produce the pattern = canonical codified-guidance-is-insufficient instance.

Connection to deployment-context awareness. This entry is part of a small emerging cluster of patterns where the defect surfaces only in production deployment contexts: missing-network-timeout hangs only under realistic upstream latency, print-instead-of-logging corrupts MCP stdout only when MCP transport is bound to stdout, f-string-in-logger-call breaks structured-log aggregation only with log aggregators, and this entry vanishes only under -O. The model lacks deployment-context-sensitivity in its corpus-default behavior; the cure across the cluster is to encode deployment-context awareness in CI/lint rather than in documentation.