Files
Alexandre 9443356b36 docs(skill): ask "is this the simplest way possible" before implementing (#3062)
* docs(skill): ask "is this the simplest way possible" before implementing

Every question the loop asks — of me and of Codex — is about what could go
wrong, and each answer argues for more code. Nothing pushed the other way
until the maintainer asked, and by then the code existed.

On PR #3061 that cost a 309-line Python registry client to read one OCI
label, in a script whose neighbouring branch does the same kind of job in
inline curl and jq. The plan review and the code review both passed it.
Asked outright, Codex said to delete all 309 lines; the inline version came
to 21 and returned byte-identical output on every image tested. It is the
second time the question has had to come from the maintainer.

Step 3 now asks it before any code exists, of myself in writing and of Codex
as a question of its own with the smaller alternative sketched out to argue
for. Step 5 gains an Idiom check, on whether a hunk looks like the code it
sits beside. Adding a new file or a new mechanism now means the full loop
however small the diff looks, and codex-review.md records that Codex will
never raise proportionality unprompted but answers it bluntly when asked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(skill): give the proportionality self-question a home on the light path

Three independent reviewers on #3062 (CodeRabbit, Codex, Copilot) all caught
the same gap: the self-question and its written answer lived only inside step
3, marked full-loop-only, while the light path went straight from scope to
implement. A light task following the documented sequence never reached the
instruction telling it to ask "is this the simplest way?" at all.

Moved the self-question (and the written answer it requires) into the shared
triage section, before the light/full-loop split, so both paths ask it before
implementing. Step 3 now only adds Codex to that question, for full loop.

Also: clarified that a version bump plus its CHANGELOG entry is release
bookkeeping, not a "new file", so it does not trigger the light-to-full-loop
escalation on its own (Codex's second finding on #3062, which read the
one-file threshold as counting every file the PR touches rather than whether
a new mechanism was added). Fixed step 5's stale "six questions" to seven,
after Idiom was added in the prior commit. Fixed simplify.md's case study to
say the 309-line client was "initially proposed", not shipped, matching the
prior PR's own body.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(skill): split "test the smaller candidate" by whether the diff exists yet

CodeRabbit on #3062 caught that step 3's "build the smaller one and run it"
never says to compare it against the larger candidate, only to run it. As
written it applied at plan time, before either candidate has code — where the
larger candidate is a plan, not a diff, so there is nothing to compare against
without building both in full, which defeats the point of picking the cheaper
one first. Testing the smaller sketch against the actual requirement is the
correct and complete check there.

The comparison CodeRabbit is describing is real, but it happens at step 6, not
step 3: by the time code review runs, the larger candidate already exists as
the diff, so sketching the smaller one and diffing its output against the
shipped code costs nothing extra and is exactly what settled #3061. Moved the
build-and-compare instruction there instead of applying it, unchanged, to a
step where nothing exists to compare against yet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 17:04:54 +02:00

4.6 KiB
Raw Permalink Blame History

Simplify — case studies

Evidence for why the mechanism ladder in SKILL.md step 3 exists, and why levels 4-6 need a reason that survives being said out loud. In each case the simpler option existed and was skipped. Being able to build the complicated thing is not a reason to.

  • A rejected PR spent a 388-line TCP proxy plus a 142-line monkeypatch of a private upstream method to reclaim 159 MB — placing custom transport code in the path of every API request. Both independent reviewers said close it rather than iterate on it.

  • A ~180-line ctypes probe was written to decide whether to enable GPU flags. It worked perfectly, proved the driver was fine, and the change still did nothing, because the question it answered was not the question that mattered.

  • A resolution cap shipped as a new init script writing an s6 envdir — the wrong mechanism entirely (ladder level 4). Renaming the option to the env var the service already reads (level 1) would have worked, and the new script did not.

  • A calibre-web trusted-ips fix injected the add-on's per-boot IP, which forced a rewrite-every-boot design that erased user entries; preserving them then needed merge logic plus a state file recording what was injected (+34 lines, PR #3009 — closed unmerged). Asking "is there a constant that makes the rewrite unnecessary?" gave the shipped fix: trust the static supervisor range 172.30.32.0/23, one idempotent statement, net −6 lines (PR #3010). Complexity spent working around a changing value is a sign to hunt for the constant instead.

  • A .templates/ha_entrypoint.sh fix went to review at 25 lines of code and merged at 10. Two sources of the excess, and neither was caught by the loop: a pure-bash fallback written at implement time for images shipping with-contenv but not s6-dumpenv — reasoned from the two binaries living in different s6 packages, never demonstrated on any real image, and the case it defended would have degraded to the pre-fix behaviour anyway — and a helper function plus a second reset that existed only to serve that fallback. Deleting the fallback deleted all of it. The rest of the review's objections were correct and cost two tokens on an existing line. It took the maintainer asking "is this the simplest way possible?" to run the pass that step 6 now requires.

  • The immich updater fix (issue #3060, PR #3061) initially proposed a 309-line Python registry client to read one OCI label, into a script whose neighbouring dockerhub branch does the same kind of job in a handful of inline curl | jq lines. Both the plan review and the code review passed it, because both were asked what could break rather than what it should cost. Asked "is this the simplest way possible?", Codex answered "delete all 309 lines"; the inline version came to 21 and was measured to return byte-identical output on all five images. About a third of what went was defending cases never observed in this repo — an architecture mismatch, a hostile label, an unsupported registry — each of which already failed closed on its own. Second time the maintainer has had to ask the question, which is why step 3 now asks it before any code exists and step 5 asks whether the hunk looks like its neighbours.

Where SKILL.md step 5's questions get hard

Deletion is not automatically the safe direction. The /dev/shm case in evidence.md is a removal that reintroduced a crash loop on hosts unlike this one. A removal that depends on a host default needs the same verification as an addition.

Naming the case is the bar for a defensive branch, not reproducing it. Docker's 64 MB /dev/shm default is documented behaviour that Home Assistant does not override, so that guard stays even though this host measured 7.7 GB. Nobody could name a single image shipping with-contenv without s6-dumpenv, so that fallback went. If you cannot name the case, delete the branch: the situation then fails the way it already fails today, visibly, instead of through a second path that is never exercised and silently rots as the base images move. Weigh the cost both ways — a one-flag guard against a crash you cannot rule out is cheap; a second code path that degrades to the pre-fix behaviour anyway is not. Write in the PR body what you cut and why, so the next person does not re-add it from the same reasoning.

"Bigger than the thing it fixes" is a smell, not a rule — but it usually means the problem was framed one level too deep.

Three-year failure favours code that reads a documented knob. Code that reaches into private internals does not survive the base image moving.