fix(ci): retry a failed add-on build once before auto-reverting the push (#2971)

A single failing matrix leg in the builder reverts the entire push, so any
transient error inside a build silently undoes a good version bump.

Observed on zoneminder 1.38.4 (run 31678876409, attempt 1): the aarch64 leg
failed after 44 s inside the ha_autoapps.sh layer with

  curl: (92) HTTP/2 stream 1 was not closed cleanly: REFUSED_STREAM (err 7)
  gzip: stdin: unexpected end of file
  tar: Error is not recoverable: exiting now

while the amd64 leg built and pushed 1.38.4 to GHCR. revert-on-failure then
pushed 0ee26fc72 reverting the bump; a manual re-run of the same source went
fully green. Because that re-run flips the run conclusion to success, these
incidents do not even show up in run-conclusion statistics.

The build-image step is now run tolerantly (continue-on-error) and repeated
once when the first attempt fails. Only a second failure reaches
revert-on-failure, so genuinely broken add-ons are still reverted, one build
later than before.

The retry is unconditional rather than gated on the log text looking
transient: BuildKit reformats error strings and registry/runner failures
spell themselves many different ways, so a text classifier would eventually
stop reverting real breakage. It is also cheap - the add-ons that fail
deterministically on every push (ente, comixed, binance-trading-bot) each
fail in 16-34 s.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Alexandre
2026-08-13 11:55:05 +02:00
committed by GitHub
parent 2ea7fb0d90
commit d70a76298a

View File

@@ -285,8 +285,52 @@ jobs:
echo "${{ matrix.arch }} is not a valid architecture for ${{ matrix.addon }}, skipping build."
fi
# A single failing leg here auto-reverts the whole push (revert-on-failure
# below), so a transient error inside the build - a truncated download in
# a RUN layer, a registry hiccup - silently undoes a perfectly good
# version bump. Seen on zoneminder 1.38.4 (run 31678876409): the aarch64
# leg died on "curl: (92) HTTP/2 stream 1 was not closed cleanly:
# REFUSED_STREAM", the commit was reverted, and a manual re-run of the
# identical source then went green. (A lost runner or a cancelled job is
# not covered by this - no later step gets to run at all.)
#
# So: build once tolerantly, and only let a second failure reach the
# revert. A genuinely broken add-on fails twice and is still reverted,
# one build later. Retrying is unconditional rather than gated on the
# log text looking "transient" - BuildKit reformats error strings and
# registry/runner failures spell themselves a dozen ways, so classifying
# by log text would silently stop reverting real breakage. It is also
# cheap: the add-ons that fail deterministically on every push each fail
# in 16-34 s.
- name: Build ${{ matrix.addon }} add-on
id: build
if: steps.info.outputs.build_arch == 'true' && steps.info.outputs.has_dockerfile == 'true'
continue-on-error: true
uses: home-assistant/builder/actions/build-image@2026.06.0
with:
arch: ${{ matrix.arch }}
cache-gha: "false"
cache-gha-scope: ${{ matrix.addon }}-${{ matrix.arch }}
context: ./${{ matrix.addon }}
file: ${{ steps.info.outputs.dockerfile }}
image: ${{ steps.info.outputs.image }}
image-tags: |
${{ steps.info.outputs.version }}
latest
version: ${{ steps.info.outputs.version }}
push: "true"
cosign: "false"
container-registry-password: ${{ secrets.GITHUB_TOKEN }}
labels: ${{ steps.info.outputs.labels }}
build-args: ${{ steps.info.outputs.build_args }}
# Keep these inputs identical to the first attempt above - this step is
# that attempt, run a second time, and nothing else. There is deliberately
# no pause in between: re-running the earlier layers already spaces the
# two network windows apart, and a sleep step would only add somewhere
# else for the retry to be skipped from.
- name: Build ${{ matrix.addon }} add-on (retry after failed attempt)
if: steps.build.outcome == 'failure'
uses: home-assistant/builder/actions/build-image@2026.06.0
with:
arch: ${{ matrix.arch }}