From a68ef0dde2e2115f4f36ebab5b0af1523c4da384 Mon Sep 17 00:00:00 2001 From: Alexandre <44178713+alexbelgium@users.noreply.github.com> Date: Tue, 25 Aug 2026 11:01:24 +0200 Subject: [PATCH] fix(skill): stop the workflow scripts reporting verdicts they have not established (#3018) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(skill): stop the workflow scripts reporting verdicts they have not established Every defect here is the same species: a check printed as passed that never ran, or never proved what it claims. All six were reproduced before and after. validate.sh - `run`/`finish` were selected by filename and fed to bash -n and shellcheck. 25 of the 97 such files here are `#!/usr/bin/execlineb`, so 21 add-ons -- calibre_web and seerr among them -- reported `local validation FAILED` and a wall of parse errors no matter what the diff contained. One list, filtered by shebang, now feeds both checks; seerr went from 15 shellcheck findings and two bash -n failures to the single finding the test diff actually introduced. - `grep -q "$ADDON/CHANGELOG.md"` was unanchored with `.` as a wildcard, so a diff bumping zzz_archived_overseerr/CHANGELOG.md reported seerr's as updated -- a false green on the one hard CI gate. Five such name collisions exist in this repo (also birdnet-pi/battybirdnet-pi, mealie/social_to_mealie, plex/spotify_to_plex). `grep -Fxq`. - The --vs-master section skipped any file absent from origin/master, so a newly added script's findings -- all of which are by definition added by the diff -- were never reported, under a line reading "your diff introduced no new lint findings". An added file now compares against an empty base. A deleted one is skipped: it was being linted at a path that no longer exists, which turned every deletion into a fabricated `openBinaryFile: does not exist` finding. - That loop ran as the right-hand side of a pipe, so it could not have reached `fail` even had it tried. It now runs in this shell and new findings fail the script; the all-clear line is printed only when nothing was listed. Findings that merely moved lines still cancel -- the comparison strips file:line:col before comm. - `bash -n ok` stood for an add-on with no shell files at all, and hadolint could print `clean` directly after printing findings (`A && {...} || C` with pipefail). Counted and branched properly. preflight.sh - `MATCH -- this checkout corresponds to the running image` was concluded from config.yaml's version equalling $BUILD_VERSION. Version is bumped once per PR, so any later commit or a dirty tree matches while differing from what runs -- the one conclusion the script exists to establish was the one it overstated. pr_review.sh - `watch` exhausting its minutes with checks still pending fell out of the loop and exited 0, reporting success for checks that never settled. Unsettled is now exit 2. Checks reported as `skipping` still count as passing, which is correct -- for this PR itself, three jobs skipped because no */config.* changed, and that is the right outcome, not a failure. But a skipped job tested nothing, so `watch` now says so. Reviewed by Codex (gpt-5.6-sol), which corrected two claims in the audit behind this: the .templates CHANGELOG assertion (CI skips the gate entirely for a template-only PR, so that fix is not in this diff) and a tradeoff that did not exist. The deleted-file and watch-timeout defects are its finds. Co-Authored-By: Claude Opus 5 * fix(skill): address review — cwd independence, no pass verdict for an empty check --------- Co-authored-by: Claude Opus 5 --- .../scripts/pr_review.sh | 8 ++- .../scripts/preflight.sh | 3 +- .../hassio-addon-workflow/scripts/validate.sh | 62 +++++++++++++------ 3 files changed, 51 insertions(+), 22 deletions(-) diff --git a/.claude/skills/hassio-addon-workflow/scripts/pr_review.sh b/.claude/skills/hassio-addon-workflow/scripts/pr_review.sh index 24eca0dbc8..56fa01e982 100755 --- a/.claude/skills/hassio-addon-workflow/scripts/pr_review.sh +++ b/.claude/skills/hassio-addon-workflow/scripts/pr_review.sh @@ -90,6 +90,7 @@ resolve) ;; watch) MINS="${3:-180}" # the addon build alone has taken ~3h; 20 was far too short + wfail=2 # not 0: running out of minutes with checks still pending is not a pass for i in $(seq 1 "$MINS"); do c=$(gh pr checks "$PR" 2> /dev/null | awk '{print $1"="$2}' | tr '\n' ' ') if [ -z "$c" ]; then @@ -102,11 +103,14 @@ watch) case "$c" in *pending*) sleep 60 ;; *fail* | *error* | *cancel*) echo "settled — with FAILURES (see above)"; wfail=1; break ;; - *) echo "settled — all passing"; break ;; + *) echo "settled — all passing"; wfail=0; break ;; esac done + [ "$wfail" -eq 2 ] && echo "gave up after ${MINS}m, checks still unsettled — NOT a pass" + # A PR touching no */config.* skips the CHANGELOG, linter and build jobs outright (#3018). + case "${c:-}" in *skipping*) echo " ...of which some were SKIPPED — a skipped job tested nothing" ;; esac echo "note: long queues here are usually account runner contention, not your diff." - exit "${wfail:-0}" + exit "$wfail" ;; *) echo "unknown: $CMD"; exit 1 ;; esac diff --git a/.claude/skills/hassio-addon-workflow/scripts/preflight.sh b/.claude/skills/hassio-addon-workflow/scripts/preflight.sh index 52dd3233c6..a61495b00d 100755 --- a/.claude/skills/hassio-addon-workflow/scripts/preflight.sh +++ b/.claude/skills/hassio-addon-workflow/scripts/preflight.sh @@ -64,7 +64,8 @@ if [ -n "${BUILD_VERSION:-}" ]; then echo " $SLUG/config.yaml version = $here" echo " running image BUILD_VERSION = $BUILD_VERSION" if [ "$here" = "$BUILD_VERSION" ]; then - echo " MATCH — this checkout corresponds to the running image." + # version is bumped once per PR, so a later commit or a dirty tree matches here. + echo " VERSION MATCH — source revision itself is not verified." else echo " MISMATCH — this branch is NOT what is running." git fetch origin master --quiet 2> /dev/null diff --git a/.claude/skills/hassio-addon-workflow/scripts/validate.sh b/.claude/skills/hassio-addon-workflow/scripts/validate.sh index 0ecea9e2d2..442f5361ae 100755 --- a/.claude/skills/hassio-addon-workflow/scripts/validate.sh +++ b/.claude/skills/hassio-addon-workflow/scripts/validate.sh @@ -4,10 +4,13 @@ # rather than implying the build was checked. # # --vs-master re-lints each changed file at origin/master and prints only findings your diff -# ADDED. Without it you will chase warnings that were already in the file. +# ADDED, and fails if there are any. Without it you will chase warnings that were already there. # # Usage: validate.sh [addon-dir] [--vs-master] set -uo pipefail +# git diff prints repo-root-relative paths and $ADDON is a top-level directory name: neither +# resolves from a subdirectory, where the -f guard below would skip every file and report clean. +if root=$(git rev-parse --show-toplevel 2> /dev/null); then cd "$root" || exit 1; fi ADDON="${1:-}" [ "${ADDON:-}" = "--vs-master" ] && { ADDON=""; set -- --vs-master; } @@ -31,17 +34,21 @@ echo "== validating $ADDON ==" fail=0 note() { printf ' %-13s %s\n' "$1" "$2"; } +# execline `run`/`finish` files are not shell (25 of them here, across 21 add-ons). Neither +# linter below can read one, so anything either says about it is noise. +is_execline() { local l; IFS= read -r l < "$1" 2> /dev/null; [[ $l == '#!'*execlineb* ]]; } -# Shell: bash -n then shellcheck -x (follows sourced files, as CI does). -while IFS= read -r f; do - [ -f "$f" ] || continue +# Shell: bash -n then shellcheck -x (follows sourced files, as CI does). One list for both. +files=() +while IFS= read -r f; do is_execline "$f" || files+=("$f"); done \ + < <(find "$ADDON" -type f \( -name '*.sh' -o -name 'run' -o -name 'finish' -o -name 'autostart' \) 2> /dev/null) +for f in "${files[@]}"; do if ! out=$(bash -n "$f" 2>&1); then note "bash -n" "FAIL $f"; echo "$out" | sed 's/^/ /'; fail=1; fi -done < <(find "$ADDON" -type f \( -name '*.sh' -o -name 'run' -o -name 'finish' -o -name 'autostart' \) 2> /dev/null) -[ "$fail" -eq 0 ] && note "bash -n" "ok" +done +[ "$fail" -eq 0 ] && note "bash -n" "${#files[@]} file(s) checked" -if command -v shellcheck > /dev/null 2>&1; then - sc=$(find "$ADDON" -type f \( -name '*.sh' -o -name 'autostart' -o -name 'run' -o -name 'finish' \) -print0 2> /dev/null | - xargs -0 -r shellcheck -x -f gcc 2>&1) +if [ "${#files[@]}" -gt 0 ] && command -v shellcheck > /dev/null 2>&1; then + sc=$(shellcheck -x -f gcc "${files[@]}" 2>&1) if [ -n "$sc" ]; then note "shellcheck" "$(printf '%s\n' "$sc" | grep -c .) finding(s)" printf '%s\n' "$sc" | sed 's/^/ /' | head -20 @@ -50,7 +57,10 @@ fi command -v hadolint > /dev/null 2>&1 && [ -f "$ADDON/Dockerfile" ] && { hl=$(hadolint "$ADDON/Dockerfile" 2>&1) - [ -n "$hl" ] && { note "hadolint" "$(printf '%s\n' "$hl" | grep -c .) finding(s)"; printf '%s\n' "$hl" | sed 's/^/ /' | head -10; } || note "hadolint" "clean" + if [ -n "$hl" ]; then + note "hadolint" "$(printf '%s\n' "$hl" | grep -c .) finding(s)" + printf '%s\n' "$hl" | sed 's/^/ /' | head -10 + else note "hadolint" "clean"; fi } if [ -f "$ADDON/config.yaml" ]; then @@ -79,11 +89,14 @@ $VS_MASTER && command -v npx > /dev/null 2>&1 && [ -f "$ADDON/CHANGELOG.md" ] && echo echo "== CI requirements ==" -if git diff --name-only origin/master...HEAD 2> /dev/null | grep -q "$ADDON/CHANGELOG.md"; then +# -Fxq, not -q: unanchored, seerr's is matched by zzz_archived_overseerr's, and . is a wildcard. +# Stricter than the gate itself, whose quoted =~ accepts that same collision +# (onpr_check-pr.yaml:75), so this can fail where CI passes — the wrong add-on's is still wrong. +if git diff --name-only origin/master...HEAD 2> /dev/null | grep -Fxq "$ADDON/CHANGELOG.md"; then note "CHANGELOG" "updated" else # This one IS gated: onpr_check-pr.yaml exits 1 without it. - note "CHANGELOG" "NOT UPDATED — this is the one CI hard-gate"; fail=1 + note "CHANGELOG" "NOT UPDATED for $ADDON — CI hard-gates this"; fail=1 fi if git diff origin/master...HEAD -- "$ADDON/config.yaml" 2> /dev/null | grep -q '^+version:'; then note "version" "bumped" @@ -98,8 +111,15 @@ if $VS_MASTER; then echo echo "== findings ADDED by this diff (pre-existing ones filtered out) ==" tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT - git diff --name-only origin/master...HEAD -- "$ADDON" 2> /dev/null | while IFS= read -r f; do - git show "origin/master:$f" > "$tmp/base" 2> /dev/null || continue + added=0 + # Fed by process substitution, not a pipe: a pipeline runs this in a subshell, where the + # findings below could never reach $fail and the verdict would contradict the list. + while IFS= read -r f; do + # Deleted: linting the path that is gone invents a finding. Added: no base, and an + # empty one says the right thing — every finding in it is one this diff added. + [ -f "$f" ] || continue + is_execline "$f" && continue + git show "origin/master:$f" > "$tmp/base" 2> /dev/null || : > "$tmp/base" # A missing linter must be a visible skip, not a silent "no new findings": # its "command not found" error is identical for base and head, so comm would # cancel it out and report a false clean. @@ -115,12 +135,16 @@ if $VS_MASTER; then cmd() { hadolint "$1" 2>&1 | sed 's/^[^:]*//; s/^:[0-9]*//'; } ;; *) continue ;; esac - cp "$tmp/base" "$tmp/base_f"; b=$(cmd "$tmp/base_f" | sort) + b=$(cmd "$tmp/base" | sort) a=$(cmd "$f" | sort) - new=$(comm -13 <(printf '%s\n' "$b") <(printf '%s\n' "$a") | grep -c .) - [ "$new" -gt 0 ] && { echo " $f: $new NEW finding(s)"; comm -13 <(printf '%s\n' "$b") <(printf '%s\n' "$a") | sed 's/^/ /' | head -5; } - done - echo " (nothing listed above = your diff introduced no new lint findings)" + new=$(comm -13 <(printf '%s\n' "$b") <(printf '%s\n' "$a")) + [ -n "$new" ] && { + echo " $f: $(printf '%s\n' "$new" | grep -c .) NEW finding(s)" + printf '%s\n' "$new" | sed 's/^/ /' | head -5 + added=1; fail=1 + } + done < <(git diff --name-only origin/master...HEAD -- "$ADDON" 2> /dev/null) + [ "$added" -eq 0 ] && echo " (none — this diff introduced no new lint findings)" fi echo