From 2087da7f18b421a77868ea08523ed272589f5e7f Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 28 May 2026 17:01:42 +0000 Subject: [PATCH 1/3] birdnet-go: harden config init, auto-wire MQTT/MariaDB, trim ingress proxy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bugs fixed in 01-structure.sh: - Database backup created during BIRDSONGS_FOLDER migration was written to the script's CWD instead of /config, and the restore path was recomputed with a fresh timestamp — so any second-boundary crossing between backup and restore left the user unable to recover. Backup path is now absolute and reused for restore. - Path inputs are validated against [A-Za-z0-9._/-]+ before being interpolated into the SQL UPDATE statement. - Default-config download tolerates network failure instead of leaving an empty config.yaml behind. - output.sqlite.path and logging.file_output.* are now seeded with the "set-if-missing" idiom (`//=`) so user edits to config.yaml survive restarts. (Breaking: addon options for log rotation now only seed defaults on first run.) - Path normalization centralized; trailing-slash juggling removed. UX upgrades: - 33-mqtt.sh now auto-configures realtime.mqtt.* in config.yaml from the HA Mosquitto addon (with new mqtt_disable opt-out option). - 33-mariadb.sh now auto-switches output.mysql.* to the HA MariaDB addon and disables SQLite (with mariadb_disable opt-out option). Cleanup: - Dockerfile: upstream entrypoint sed-patch now warns (not silently succeeds) when the target pattern is missing in a new nightly. - Removed dead nginx upstream.conf pointing at unused port 8096. - Trimmed redundant nginx HTML-attribute sub_filters; upstream birdnet-go handles those itself via X-Ingress-Path. JS string rewrites kept since the upstream HTML rewriter does not touch JS. (Breaking UI-side if upstream regresses — see CHANGELOG.) --- birdnet-go/CHANGELOG.md | 11 +++ birdnet-go/Dockerfile | 8 +- birdnet-go/README.md | 10 ++ birdnet-go/config.yaml | 6 +- .../rootfs/etc/cont-init.d/01-structure.sh | 97 ++++++++++++------- .../rootfs/etc/cont-init.d/33-mariadb.sh | 61 ++++++++++-- birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh | 53 ++++++++-- .../rootfs/etc/nginx/includes/upstream.conf | 3 - birdnet-go/rootfs/etc/nginx/nginx.conf | 1 - .../rootfs/etc/nginx/servers/ingress.conf | 20 ++-- 10 files changed, 200 insertions(+), 70 deletions(-) delete mode 100644 birdnet-go/rootfs/etc/nginx/includes/upstream.conf diff --git a/birdnet-go/CHANGELOG.md b/birdnet-go/CHANGELOG.md index d7a86bc0bf..dbfbea413a 100644 --- a/birdnet-go/CHANGELOG.md +++ b/birdnet-go/CHANGELOG.md @@ -1,3 +1,14 @@ +## nightly-20260525-3 (28-05-2026) +- Auto-configure the Home Assistant MQTT addon: when Mosquitto is active, `realtime.mqtt.{enabled,broker,username,password}` are written directly to `config.yaml`. Set the new `mqtt_disable: true` addon option to opt out. +- Auto-configure the Home Assistant MariaDB addon: when active, `output.mysql.*` is filled in and `output.sqlite.enabled` is set to `false`. Set the new `mariadb_disable: true` addon option to opt out. +- **Breaking**: `output.sqlite.path` and `logging.file_output.*` are now seeded only when missing from `config.yaml` (previously overwritten every restart). Values changed through the BirdNET-Go UI or by hand-editing `config.yaml` now survive container restarts. If you relied on `LOG_MAX_SIZE_MB` / `LOG_MAX_AGE_DAYS` addon options to override an existing setting in `config.yaml`, remove the existing key from `config.yaml` or edit it directly — the option will only be applied on first run. +- **Breaking (UI only)**: The nginx ingress reverse-proxy no longer rewrites HTML `href`/`src`/`action` attributes; upstream BirdNET-Go handles those itself via `X-Ingress-Path`. JavaScript string-literal rewrites are unchanged. Please file an issue if you see broken images, links, or forms in the ingress UI after upgrade. +- Fix database-migration restore: the timestamped backup created during a `BIRDSONGS_FOLDER` change was being written to the script's working directory and looked up under a fresh timestamp on restore, so a SQL failure left the user unable to recover. Backup path is now absolute and reused for restore. +- Harden the `BIRDSONGS_FOLDER` SQL/YAML path substitution: paths containing characters outside `[A-Za-z0-9._/-]` are now rejected up front instead of being interpolated raw into the SQL UPDATE statement. +- Tolerate a missing internet connection on first boot: if the default `config.yaml` cannot be downloaded from GitHub, BirdNET-Go now falls back to its embedded defaults instead of starting with an empty config. +- Warn (without failing the build) if the upstream `entrypoint.sh` patch target drifts in a new nightly. +- Remove a dead nginx upstream definition that pointed at an unused port. + ## nightly-20260525-2 (26-05-2026) - Suppress noisy startup logs: silence `chmod /dev/snd` errors on the read-only HA mount, and hide unavailable ALSA plugins (JACK, OSS, dsnoop) from device enumeration so libjack and pcm_oss/dsnoop probes no longer print at launch. ALSA overrides are written to `/root/.asoundrc` (since `/etc/asound.conf` is read-only in this environment). - Allow advanced users to override the ALSA config by dropping a custom `asound.conf` into the addon config folder. diff --git a/birdnet-go/Dockerfile b/birdnet-go/Dockerfile index c5664ed3cf..cd22b1912e 100644 --- a/birdnet-go/Dockerfile +++ b/birdnet-go/Dockerfile @@ -39,8 +39,14 @@ RUN find . -type f \( -name "*.sh" -o -name "run" -o -name "finish" \) -print -e # Silence "chmod: Read-only file system" noise from upstream entrypoint: # Home Assistant mounts /dev/snd read-only, so the chmod is a no-op and the # stderr output is just noise. The "|| true" already absorbs the exit code. +# Warn (without failing the build) if the upstream pattern drifts so we notice +# in CI logs and can update the patch. RUN if [ -f /usr/bin/entrypoint.sh ]; then \ - sed -i 's#chmod -R a+rw /dev/snd || true#chmod -R a+rw /dev/snd 2>/dev/null || true#' /usr/bin/entrypoint.sh; \ + if grep -q 'chmod -R a+rw /dev/snd || true' /usr/bin/entrypoint.sh; then \ + sed -i 's#chmod -R a+rw /dev/snd || true#chmod -R a+rw /dev/snd 2>/dev/null || true#' /usr/bin/entrypoint.sh; \ + else \ + echo "WARN: upstream entrypoint.sh /dev/snd chmod pattern not found; nightly drift?" >&2; \ + fi; \ fi # Uses /bin for compatibility purposes diff --git a/birdnet-go/README.md b/birdnet-go/README.md index 9def9953db..419416b833 100644 --- a/birdnet-go/README.md +++ b/birdnet-go/README.md @@ -49,6 +49,8 @@ Options can be configured through three ways : ALSA_CARD : number of the card (0 or 1 usually), see https://github.com/tphakala/birdnet-go/blob/main/doc/installation.md#deciding-alsa_card-value TZ: Etc/UTC specify a timezone to use, see https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List COMMAND : realtime --rtsp url # allows to provide arguments to birdnet-go +mqtt_disable: false # set true to skip auto-wiring of the Home Assistant MQTT addon +mariadb_disable: false # set true to skip auto-wiring of the Home Assistant MariaDB addon ``` - Config.yaml @@ -57,6 +59,14 @@ Additional variables can be configured using the config.yaml file found in /conf - Config_env.yaml Additional environment variables can be configured there +### MQTT and MariaDB auto-configuration + +If the Home Assistant **MQTT** addon is installed and running, BirdNET-Go is now wired to it automatically on startup: `realtime.mqtt.enabled`, `broker`, `username`, and `password` in `config.yaml` are populated from the HA Mosquitto credentials, and the topic defaults to `birdnet`. Set `mqtt_disable: true` in the addon options to keep manual control. + +If the Home Assistant **MariaDB** addon is installed and running, BirdNET-Go is switched to it automatically: `output.mysql` is enabled with the HA credentials (database `birdnet`, created on first connect) and `output.sqlite.enabled` is set to `false`. Set `mariadb_disable: true` to keep using SQLite or to point at a different MySQL server manually. + +The addon also seeds `output.sqlite.path` and `logging.file_output.*` defaults only when those keys are missing from `config.yaml`, so values you change through the BirdNET-Go UI now survive container restarts. + ### Mounting Drives This addon supports mounting both local drives and remote SMB shares: diff --git a/birdnet-go/config.yaml b/birdnet-go/config.yaml index 5c5d5ad94a..db7a9be833 100644 --- a/birdnet-go/config.yaml +++ b/birdnet-go/config.yaml @@ -89,6 +89,8 @@ options: LOG_MAX_SIZE_MB: 50 LOG_MAX_AGE_DAYS: 7 homeassistant_microphone: false + mqtt_disable: false + mariadb_disable: false panel_admin: false panel_icon: mdi:bird ports: @@ -114,6 +116,8 @@ schema: cifsusername: str? homeassistant_microphone: bool? localdisks: str? + mariadb_disable: bool? + mqtt_disable: bool? networkdisks: str? services: - mysql:want @@ -122,4 +126,4 @@ slug: birdnet-go udev: true url: https://github.com/alexbelgium/hassio-addons/tree/master/birdnet-go usb: true -version: "nightly-20260525-2" +version: "nightly-20260525-3" diff --git a/birdnet-go/rootfs/etc/cont-init.d/01-structure.sh b/birdnet-go/rootfs/etc/cont-init.d/01-structure.sh index dcf44ecd81..616a2e29f3 100755 --- a/birdnet-go/rootfs/etc/cont-init.d/01-structure.sh +++ b/birdnet-go/rootfs/etc/cont-init.d/01-structure.sh @@ -3,12 +3,38 @@ set -e # Default Variables -DEFAULT_BIRDSONGS_FOLDER="/data/clips/" +DEFAULT_BIRDSONGS_FOLDER="/data/clips" CONFIG_LOCATION="/config/config.yaml" +# Strip trailing slashes; canonical internal form has none. +normalize_path() { + local p="$1" + while [[ "$p" == */ && "$p" != "/" ]]; do + p="${p%/}" + done + printf '%s' "$p" +} + +# Reject paths containing characters that would break the SQL/YAML literals +# we substitute into below. We deliberately allow only "safe" filename chars. +validate_safe_path() { + local p="$1" + if [[ ! "$p" =~ ^[A-Za-z0-9._/-]+$ ]]; then + bashio::log.fatal "Refusing unsafe path: '$p' (only [A-Za-z0-9._/-] allowed)" + exit 1 + fi +} + if [ ! -f "$CONFIG_LOCATION" ]; then bashio::log.warning "There is no config.yaml yet in the config folder, downloading a default one. Please customize" - curl -L -s -S https://raw.githubusercontent.com/tphakala/birdnet-go/refs/heads/main/internal/conf/config.yaml -o "$CONFIG_LOCATION" + # Network may be unreachable on first boot; tolerate failure and let + # birdnet-go fall back to its embedded default on first run. + if ! curl -fL -s -S \ + https://raw.githubusercontent.com/tphakala/birdnet-go/refs/heads/main/internal/conf/config.yaml \ + -o "$CONFIG_LOCATION"; then + bashio::log.warning "Could not download default config.yaml; birdnet-go will create one from its embedded defaults" + rm -f "$CONFIG_LOCATION" + fi fi ################# @@ -22,20 +48,22 @@ fi ###################### # Birdsongs Location ###################### -CURRENT_BIRDSONGS_FOLDER="clips/" -# Read the current folder from config files +# Read the current folder from config.yaml; fall back to the legacy default. CURRENT_BIRDSONGS_FOLDER="$(yq '.realtime.audio.export.path' "$CONFIG_LOCATION" | tr -d '\"')" CURRENT_BIRDSONGS_FOLDER="${CURRENT_BIRDSONGS_FOLDER:-$DEFAULT_BIRDSONGS_FOLDER}" - -# Adjust default path if it matches the default string -if [[ "$CURRENT_BIRDSONGS_FOLDER" == "clips/" ]]; then +# Treat the upstream-shipped relative "clips/" as the legacy default. +if [[ "$CURRENT_BIRDSONGS_FOLDER" == "clips" || "$CURRENT_BIRDSONGS_FOLDER" == "clips/" ]]; then CURRENT_BIRDSONGS_FOLDER="$DEFAULT_BIRDSONGS_FOLDER" fi +CURRENT_BIRDSONGS_FOLDER="$(normalize_path "$CURRENT_BIRDSONGS_FOLDER")" -# Set the new birdsongs folder +# Set the new birdsongs folder from addon options (default: relative "clips"). BIRDSONGS_FOLDER="$(bashio::config "BIRDSONGS_FOLDER")" -BIRDSONGS_FOLDER="${BIRDSONGS_FOLDER:-clips/}" -BIRDSONGS_FOLDER="${BIRDSONGS_FOLDER%/}" # Remove trailing slash if present +BIRDSONGS_FOLDER="$(normalize_path "${BIRDSONGS_FOLDER:-clips}")" + +validate_safe_path "$BIRDSONGS_FOLDER" +validate_safe_path "$CURRENT_BIRDSONGS_FOLDER" + if [[ ! "$BIRDSONGS_FOLDER" == /* ]]; then if [ ! -d "/config/$BIRDSONGS_FOLDER" ]; then mkdir -p "/config/$BIRDSONGS_FOLDER" @@ -52,42 +80,43 @@ fi bashio::log.info "... audio clips saved to $BIRDSONGS_FOLDER according to addon options" # Migrate data if the folder has changed -if [[ "${CURRENT_BIRDSONGS_FOLDER%/}" != "${BIRDSONGS_FOLDER%/}" ]]; then +if [[ "$CURRENT_BIRDSONGS_FOLDER" != "$BIRDSONGS_FOLDER" ]]; then bashio::log.warning "Birdsongs folder changed from $CURRENT_BIRDSONGS_FOLDER to $BIRDSONGS_FOLDER" - # Update config files with the new birdsongs folder path - yq -i -y ".realtime.audio.export.path = \"$BIRDSONGS_FOLDER/\"" "$CONFIG_LOCATION" + # Update config.yaml with the new birdsongs folder path (trailing slash + # restored only at the boundary, since birdnet-go expects it). + yq -i -y ".realtime.audio.export.path = \"${BIRDSONGS_FOLDER}/\"" "$CONFIG_LOCATION" # Move files only if sqlite paths changed if [[ -d "$CURRENT_BIRDSONGS_FOLDER" && "$(ls -A "$CURRENT_BIRDSONGS_FOLDER")" ]]; then bashio::log.warning "Migrating files from $CURRENT_BIRDSONGS_FOLDER to $BIRDSONGS_FOLDER" cp -rnf "$CURRENT_BIRDSONGS_FOLDER"/* "$BIRDSONGS_FOLDER"/ - mv "${CURRENT_BIRDSONGS_FOLDER%/}" "${CURRENT_BIRDSONGS_FOLDER%/}_migrated" + mv "$CURRENT_BIRDSONGS_FOLDER" "${CURRENT_BIRDSONGS_FOLDER}_migrated" fi # Adapt the database if [ -f /config/birdnet.db ]; then - # Prepare backup="$(date +%Y%m%d_%H%M%S)" - bashio::log.warning "Modifying database paths from $CURRENT_BIRDSONGS_FOLDER to $BIRDSONGS_FOLDER. A backup named birdnet.db_$backup will be created before" + BACKUP_FILE="/config/birdnet.db_${backup}" + bashio::log.warning "Modifying database paths from $CURRENT_BIRDSONGS_FOLDER to $BIRDSONGS_FOLDER. A backup will be created at ${BACKUP_FILE}" - # Create backup - if ! cp /config/birdnet.db "birdnet.db_$backup"; then + # Create backup at the absolute path we'll restore from on failure. + if ! cp /config/birdnet.db "$BACKUP_FILE"; then bashio::log.error "Failed to create a backup of the database. Aborting path modification." exit 1 fi - # Execute the query using sqlite3 - SQL_QUERY="UPDATE notes SET clip_name = '${BIRDSONGS_FOLDER%/}/' || substr(clip_name, length('${CURRENT_BIRDSONGS_FOLDER%/}/') + 1) WHERE clip_name LIKE '${CURRENT_BIRDSONGS_FOLDER%/}/%';" + # Paths were validated above against [A-Za-z0-9._/-]+ so quote + # escaping in the SQL literal is not a concern. + SQL_QUERY="UPDATE notes SET clip_name = '${BIRDSONGS_FOLDER}/' || substr(clip_name, length('${CURRENT_BIRDSONGS_FOLDER}/') + 1) WHERE clip_name LIKE '${CURRENT_BIRDSONGS_FOLDER}/%';" if ! sqlite3 /config/birdnet.db "$SQL_QUERY"; then bashio::log.warning "An error occurred while updating the paths. The database backup will be restored." - BACKUP_FILE="/config/birdnet.db_$(date +%Y%m%d_%H%M%S)" # Make sure this matches the earlier backup filename if [ -f "$BACKUP_FILE" ]; then mv "$BACKUP_FILE" /config/birdnet.db bashio::log.info "The database backup has been restored." else - bashio::log.error "Backup file not found! Manual intervention required." + bashio::log.error "Backup file $BACKUP_FILE not found! Manual intervention required." fi else - echo "Paths have been successfully updated." + bashio::log.info "Paths have been successfully updated." fi fi fi @@ -95,11 +124,13 @@ fi #################### # Correct Defaults #################### -bashio::log.info "Correcting configuration for defaults" +# Seed addon-specific defaults only if the user has not set them in +# config.yaml. The "//=" form leaves any user-edited value alone, so +# changes made via the BirdNET-Go UI or by hand-editing /config/config.yaml +# survive container restarts. +bashio::log.info "Seeding default configuration values (only if missing)" -# Update database location in config files -yq -i -y ".output.sqlite.path = \"/config/birdnet.db\"" "$CONFIG_LOCATION" -bashio::log.info "... database is located in /config/birdnet.db" +yq -i -y '.output.sqlite.path //= "/config/birdnet.db"' "$CONFIG_LOCATION" #################### # Log Management @@ -109,13 +140,13 @@ LOG_MAX_SIZE_MB="${LOG_MAX_SIZE_MB:-50}" LOG_MAX_AGE_DAYS="$(bashio::config "LOG_MAX_AGE_DAYS")" LOG_MAX_AGE_DAYS="${LOG_MAX_AGE_DAYS:-7}" -bashio::log.info "Configuring log rotation: max ${LOG_MAX_SIZE_MB}MB per file, max ${LOG_MAX_AGE_DAYS} days retention" +bashio::log.info "Seeding default log rotation: max ${LOG_MAX_SIZE_MB}MB per file, max ${LOG_MAX_AGE_DAYS} days retention (only applied if not already set)" -# Configure log rotation in birdnet-go config -yq -i -y ".logging.file_output.max_size = ${LOG_MAX_SIZE_MB}" "$CONFIG_LOCATION" -yq -i -y ".logging.file_output.max_age = ${LOG_MAX_AGE_DAYS}" "$CONFIG_LOCATION" -yq -i -y ".logging.file_output.max_rotated_files = 3" "$CONFIG_LOCATION" -yq -i -y ".logging.file_output.compress = true" "$CONFIG_LOCATION" +# Seed log-rotation defaults; do not clobber user-edited values. +yq -i -y ".logging.file_output.max_size //= ${LOG_MAX_SIZE_MB}" "$CONFIG_LOCATION" +yq -i -y ".logging.file_output.max_age //= ${LOG_MAX_AGE_DAYS}" "$CONFIG_LOCATION" +yq -i -y '.logging.file_output.max_rotated_files //= 3' "$CONFIG_LOCATION" +yq -i -y '.logging.file_output.compress //= true' "$CONFIG_LOCATION" # Trim existing log files that exceed the configured max age LOG_DIR="/config/logs" diff --git a/birdnet-go/rootfs/etc/cont-init.d/33-mariadb.sh b/birdnet-go/rootfs/etc/cont-init.d/33-mariadb.sh index dfd581e9aa..29f2476b0d 100755 --- a/birdnet-go/rootfs/etc/cont-init.d/33-mariadb.sh +++ b/birdnet-go/rootfs/etc/cont-init.d/33-mariadb.sh @@ -2,15 +2,56 @@ # shellcheck shell=bash set -e -# Gives mariadb information +# If the Home Assistant MariaDB addon is active, wire its credentials directly +# into BirdNET-Go's config.yaml. Upstream reads MySQL settings only from YAML +# (no env-var overrides exist). Users who prefer SQLite or a different MySQL +# server can set mariadb_disable: true in the addon options. -if bashio::services.available 'mysql'; then - bashio::log.green "---" - bashio::log.yellow "MariaDB addon is active on your system! If you want to use it instead of sqlite, here are the informations to encode :" - bashio::log.blue "Database user : $(bashio::services "mysql" "username")" - bashio::log.blue "Database password : $(bashio::services "mysql" "password")" - bashio::log.blue "Database name : birdnet" - bashio::log.blue "Host-name : $(bashio::services "mysql" "host")" - bashio::log.blue "Port : $(bashio::services "mysql" "port")" - bashio::log.green "---" +CONFIG_LOCATION="/config/config.yaml" +MYSQL_DATABASE="birdnet" + +if ! bashio::services.available 'mysql'; then + exit 0 fi + +if bashio::config.true 'mariadb_disable'; then + bashio::log.info "MariaDB auto-configuration disabled by 'mariadb_disable' addon option; skipping." + exit 0 +fi + +if [ ! -f "$CONFIG_LOCATION" ]; then + bashio::log.warning "Skipping MariaDB auto-configuration: $CONFIG_LOCATION not found" + exit 0 +fi + +MYSQL_HOST="$(bashio::services 'mysql' 'host')" +MYSQL_PORT="$(bashio::services 'mysql' 'port')" +MYSQL_USER="$(bashio::services 'mysql' 'username')" +MYSQL_PASS="$(bashio::services 'mysql' 'password')" + +bashio::log.green "---" +bashio::log.blue "Home Assistant MariaDB addon detected; auto-configuring BirdNET-Go" +bashio::log.blue "Host: ${MYSQL_HOST}:${MYSQL_PORT}" +bashio::log.blue "User: ${MYSQL_USER}" +bashio::log.blue "Database: ${MYSQL_DATABASE} (will be created by BirdNET-Go on first connect)" +bashio::log.blue "(Set 'mariadb_disable: true' in addon options to opt out)" +bashio::log.green "---" + +# Upstream config.go stores port as a string; pass it as such to match. +# $host / $port / etc. are jq/yq variables, not shell expansions — the +# single quotes around the filter are intentional. +# shellcheck disable=SC2016 +yq -i -y \ + --arg host "$MYSQL_HOST" \ + --arg port "$MYSQL_PORT" \ + --arg user "$MYSQL_USER" \ + --arg pass "$MYSQL_PASS" \ + --arg db "$MYSQL_DATABASE" \ + '.output.mysql.enabled = true + | .output.mysql.host = $host + | .output.mysql.port = $port + | .output.mysql.username = $user + | .output.mysql.password = $pass + | .output.mysql.database = $db + | .output.sqlite.enabled = false' \ + "$CONFIG_LOCATION" diff --git a/birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh b/birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh index f968102b3a..7096fc0163 100755 --- a/birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh +++ b/birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh @@ -2,13 +2,50 @@ # shellcheck shell=bash set -e -# Gives mqtt information +# If the Home Assistant MQTT addon is active, wire its credentials directly +# into BirdNET-Go's config.yaml (upstream reads MQTT settings only from YAML; +# no env-var overrides exist). Users who prefer to manage MQTT manually can +# set mqtt_disable: true in the addon options. -if bashio::services.available 'mqtt'; then - bashio::log.green "---" - bashio::log.yellow "MQTT addon is active on your system! Add the MQTT details below to the Birdnet-go config.yaml :" - bashio::log.blue "MQTT user : $(bashio::services "mqtt" "username")" - bashio::log.blue "MQTT password : $(bashio::services "mqtt" "password")" - bashio::log.blue "MQTT broker : tcp://$(bashio::services "mqtt" "host"):$(bashio::services "mqtt" "port")" - bashio::log.green "---" +CONFIG_LOCATION="/config/config.yaml" + +if ! bashio::services.available 'mqtt'; then + exit 0 fi + +if bashio::config.true 'mqtt_disable'; then + bashio::log.info "MQTT auto-configuration disabled by 'mqtt_disable' addon option; skipping." + exit 0 +fi + +if [ ! -f "$CONFIG_LOCATION" ]; then + bashio::log.warning "Skipping MQTT auto-configuration: $CONFIG_LOCATION not found" + exit 0 +fi + +MQTT_HOST="$(bashio::services 'mqtt' 'host')" +MQTT_PORT="$(bashio::services 'mqtt' 'port')" +MQTT_USER="$(bashio::services 'mqtt' 'username')" +MQTT_PASS="$(bashio::services 'mqtt' 'password')" +MQTT_BROKER="tcp://${MQTT_HOST}:${MQTT_PORT}" + +bashio::log.green "---" +bashio::log.blue "Home Assistant MQTT addon detected; auto-configuring BirdNET-Go" +bashio::log.blue "Broker: ${MQTT_BROKER}" +bashio::log.blue "User: ${MQTT_USER}" +bashio::log.blue "(Set 'mqtt_disable: true' in addon options to opt out)" +bashio::log.green "---" + +# $broker / $user / $pass / "birdnet" are jq/yq variables and literals, +# not shell expansions, so the single quotes are intentional. +# shellcheck disable=SC2016 +yq -i -y \ + --arg broker "$MQTT_BROKER" \ + --arg user "$MQTT_USER" \ + --arg pass "$MQTT_PASS" \ + '.realtime.mqtt.enabled = true + | .realtime.mqtt.broker = $broker + | .realtime.mqtt.username = $user + | .realtime.mqtt.password = $pass + | .realtime.mqtt.topic //= "birdnet"' \ + "$CONFIG_LOCATION" diff --git a/birdnet-go/rootfs/etc/nginx/includes/upstream.conf b/birdnet-go/rootfs/etc/nginx/includes/upstream.conf deleted file mode 100644 index a640ef18c0..0000000000 --- a/birdnet-go/rootfs/etc/nginx/includes/upstream.conf +++ /dev/null @@ -1,3 +0,0 @@ -upstream backend { - server 127.0.0.1:8096; -} diff --git a/birdnet-go/rootfs/etc/nginx/nginx.conf b/birdnet-go/rootfs/etc/nginx/nginx.conf index 37c27d6386..9d371deb37 100644 --- a/birdnet-go/rootfs/etc/nginx/nginx.conf +++ b/birdnet-go/rootfs/etc/nginx/nginx.conf @@ -76,7 +76,6 @@ http { } include /etc/nginx/includes/resolver.conf; - include /etc/nginx/includes/upstream.conf; include /etc/nginx/servers/*.conf; } diff --git a/birdnet-go/rootfs/etc/nginx/servers/ingress.conf b/birdnet-go/rootfs/etc/nginx/servers/ingress.conf index 7a8e36cfe9..21fc2c9319 100644 --- a/birdnet-go/rootfs/etc/nginx/servers/ingress.conf +++ b/birdnet-go/rootfs/etc/nginx/servers/ingress.conf @@ -21,29 +21,25 @@ server { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; - # Tell BirdNET-Go its proxy prefix (NEW — required) + # Tell BirdNET-Go its proxy prefix. Upstream honours X-Ingress-Path + # in internal/api/server.go and rewrites HTML href/src/action + # attributes itself, so we no longer duplicate that work here. proxy_set_header X-Ingress-Path %%ingress_entry%%; # Prevent timeouts proxy_read_timeout 86400; proxy_send_timeout 86400; - # sub_filter setup + # sub_filter setup: keep gzip off and the type filter wide so JS gets + # touched too (the upstream HTML rewriter does not look inside JS). proxy_set_header Accept-Encoding ""; sub_filter_once off; sub_filter_types *; - # HTML attribute rewrites - sub_filter href=\"/ href=\"%%ingress_entry%%/; - sub_filter src=\"/ src=\"%%ingress_entry%%/; - sub_filter src=\"'/ src=\"'%%ingress_entry%%/; - sub_filter action=\"/ action=\"%%ingress_entry%%/; - - # JavaScript string rewrites (needed for Vite dynamic imports) + # JS string-literal rewrites — paths embedded in JS code are not + # touched by upstream's HTML rewriter, so we patch them here. sub_filter EventSource('/ EventSource('%%ingress_entry%%/; sub_filter fetch('/ fetch('%%ingress_entry%%/; - - # Backtick template literal rewrites sub_filter `/api/v `%%ingress_entry%%/api/v; sub_filter "'/api/v" "'%%ingress_entry%%/api/v"; sub_filter \"/api/v \"%%ingress_entry%%/api/v; @@ -53,8 +49,6 @@ server { sub_filter `/asset `%%ingress_entry%%/asset; sub_filter "'/asset" "'%%ingress_entry%%/asset"; sub_filter \"/asset \"%%ingress_entry%%/asset; - - # Streaming/EventSource fix sub_filter window.location.origin} window.location.origin}%%ingress_entry%%; } } From 854f90cbe84d219e0d20ecb4dd65d3b45b1d0e40 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 28 May 2026 17:21:12 +0000 Subject: [PATCH 2/3] birdnet-go: invert MQTT/MariaDB auto-config to opt-in MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace mqtt_disable / mariadb_disable (opt-out, default-on) with mqtt_auto_config / mariadb_auto_config (opt-in, default-off). When the HA addon is detected but the option is off, still log the broker / database credentials and a hint pointing the user at the option — so discoverability stays the same without surprise config rewrites. --- birdnet-go/CHANGELOG.md | 4 +- birdnet-go/README.md | 10 ++--- birdnet-go/config.yaml | 8 ++-- .../rootfs/etc/cont-init.d/33-mariadb.sh | 34 ++++++++++------- birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh | 38 +++++++++++-------- 5 files changed, 54 insertions(+), 40 deletions(-) diff --git a/birdnet-go/CHANGELOG.md b/birdnet-go/CHANGELOG.md index dbfbea413a..51e60fbff4 100644 --- a/birdnet-go/CHANGELOG.md +++ b/birdnet-go/CHANGELOG.md @@ -1,6 +1,6 @@ ## nightly-20260525-3 (28-05-2026) -- Auto-configure the Home Assistant MQTT addon: when Mosquitto is active, `realtime.mqtt.{enabled,broker,username,password}` are written directly to `config.yaml`. Set the new `mqtt_disable: true` addon option to opt out. -- Auto-configure the Home Assistant MariaDB addon: when active, `output.mysql.*` is filled in and `output.sqlite.enabled` is set to `false`. Set the new `mariadb_disable: true` addon option to opt out. +- New `mqtt_auto_config` addon option (default `false`). When `true` and the Home Assistant MQTT addon is active, `realtime.mqtt.{enabled,broker,username,password}` are written directly to `config.yaml` on every restart. When `false` but Mosquitto is detected, the addon still logs the broker details and reminds you about the option — nothing is written. +- New `mariadb_auto_config` addon option (default `false`). When `true` and the Home Assistant MariaDB addon is active, `output.mysql.*` is filled in and `output.sqlite.enabled` is set to `false`. When `false` but MariaDB is detected, the addon logs the credentials and reminds you about the option. - **Breaking**: `output.sqlite.path` and `logging.file_output.*` are now seeded only when missing from `config.yaml` (previously overwritten every restart). Values changed through the BirdNET-Go UI or by hand-editing `config.yaml` now survive container restarts. If you relied on `LOG_MAX_SIZE_MB` / `LOG_MAX_AGE_DAYS` addon options to override an existing setting in `config.yaml`, remove the existing key from `config.yaml` or edit it directly — the option will only be applied on first run. - **Breaking (UI only)**: The nginx ingress reverse-proxy no longer rewrites HTML `href`/`src`/`action` attributes; upstream BirdNET-Go handles those itself via `X-Ingress-Path`. JavaScript string-literal rewrites are unchanged. Please file an issue if you see broken images, links, or forms in the ingress UI after upgrade. - Fix database-migration restore: the timestamped backup created during a `BIRDSONGS_FOLDER` change was being written to the script's working directory and looked up under a fresh timestamp on restore, so a SQL failure left the user unable to recover. Backup path is now absolute and reused for restore. diff --git a/birdnet-go/README.md b/birdnet-go/README.md index 419416b833..2c26ae7ec6 100644 --- a/birdnet-go/README.md +++ b/birdnet-go/README.md @@ -49,8 +49,8 @@ Options can be configured through three ways : ALSA_CARD : number of the card (0 or 1 usually), see https://github.com/tphakala/birdnet-go/blob/main/doc/installation.md#deciding-alsa_card-value TZ: Etc/UTC specify a timezone to use, see https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List COMMAND : realtime --rtsp url # allows to provide arguments to birdnet-go -mqtt_disable: false # set true to skip auto-wiring of the Home Assistant MQTT addon -mariadb_disable: false # set true to skip auto-wiring of the Home Assistant MariaDB addon +mqtt_auto_config: false # set true to auto-wire the Home Assistant MQTT addon into config.yaml +mariadb_auto_config: false # set true to auto-wire the Home Assistant MariaDB addon into config.yaml (also disables SQLite) ``` - Config.yaml @@ -59,11 +59,11 @@ Additional variables can be configured using the config.yaml file found in /conf - Config_env.yaml Additional environment variables can be configured there -### MQTT and MariaDB auto-configuration +### MQTT and MariaDB auto-configuration (opt-in) -If the Home Assistant **MQTT** addon is installed and running, BirdNET-Go is now wired to it automatically on startup: `realtime.mqtt.enabled`, `broker`, `username`, and `password` in `config.yaml` are populated from the HA Mosquitto credentials, and the topic defaults to `birdnet`. Set `mqtt_disable: true` in the addon options to keep manual control. +If the Home Assistant **MQTT** addon is installed and running and you set `mqtt_auto_config: true` in the addon options, the addon writes the HA Mosquitto credentials directly into BirdNET-Go's `config.yaml` on every startup: `realtime.mqtt.enabled`, `broker`, `username`, and `password` are populated, and the topic defaults to `birdnet`. When the option is `false` (the default), the addon still logs the broker details and reminds you about the option whenever Mosquitto is detected — nothing is written. -If the Home Assistant **MariaDB** addon is installed and running, BirdNET-Go is switched to it automatically: `output.mysql` is enabled with the HA credentials (database `birdnet`, created on first connect) and `output.sqlite.enabled` is set to `false`. Set `mariadb_disable: true` to keep using SQLite or to point at a different MySQL server manually. +If the Home Assistant **MariaDB** addon is installed and running and you set `mariadb_auto_config: true`, the addon writes the HA credentials into `output.mysql.*` and sets `output.sqlite.enabled` to `false` (database name `birdnet`, created on first connect). When the option is `false` (the default), the addon only logs the credentials so you can configure them manually. The addon also seeds `output.sqlite.path` and `logging.file_output.*` defaults only when those keys are missing from `config.yaml`, so values you change through the BirdNET-Go UI now survive container restarts. diff --git a/birdnet-go/config.yaml b/birdnet-go/config.yaml index db7a9be833..ee98904ad6 100644 --- a/birdnet-go/config.yaml +++ b/birdnet-go/config.yaml @@ -89,8 +89,8 @@ options: LOG_MAX_SIZE_MB: 50 LOG_MAX_AGE_DAYS: 7 homeassistant_microphone: false - mqtt_disable: false - mariadb_disable: false + mqtt_auto_config: false + mariadb_auto_config: false panel_admin: false panel_icon: mdi:bird ports: @@ -116,8 +116,8 @@ schema: cifsusername: str? homeassistant_microphone: bool? localdisks: str? - mariadb_disable: bool? - mqtt_disable: bool? + mariadb_auto_config: bool? + mqtt_auto_config: bool? networkdisks: str? services: - mysql:want diff --git a/birdnet-go/rootfs/etc/cont-init.d/33-mariadb.sh b/birdnet-go/rootfs/etc/cont-init.d/33-mariadb.sh index 29f2476b0d..51a7f484ac 100755 --- a/birdnet-go/rootfs/etc/cont-init.d/33-mariadb.sh +++ b/birdnet-go/rootfs/etc/cont-init.d/33-mariadb.sh @@ -2,10 +2,12 @@ # shellcheck shell=bash set -e -# If the Home Assistant MariaDB addon is active, wire its credentials directly -# into BirdNET-Go's config.yaml. Upstream reads MySQL settings only from YAML -# (no env-var overrides exist). Users who prefer SQLite or a different MySQL -# server can set mariadb_disable: true in the addon options. +# When the Home Assistant MariaDB addon is active, optionally wire its +# credentials directly into BirdNET-Go's config.yaml. Upstream reads MySQL +# settings only from YAML (no env-var overrides exist), so this is the only +# way to auto-configure them. The behaviour is opt-in via the +# mariadb_auto_config addon option. When the option is off but MariaDB is +# detected, we log a one-shot hint pointing users at the option. CONFIG_LOCATION="/config/config.yaml" MYSQL_DATABASE="birdnet" @@ -14,8 +16,20 @@ if ! bashio::services.available 'mysql'; then exit 0 fi -if bashio::config.true 'mariadb_disable'; then - bashio::log.info "MariaDB auto-configuration disabled by 'mariadb_disable' addon option; skipping." +MYSQL_HOST="$(bashio::services 'mysql' 'host')" +MYSQL_PORT="$(bashio::services 'mysql' 'port')" +MYSQL_USER="$(bashio::services 'mysql' 'username')" +MYSQL_PASS="$(bashio::services 'mysql' 'password')" + +if ! bashio::config.true 'mariadb_auto_config'; then + bashio::log.green "---" + bashio::log.yellow "Home Assistant MariaDB addon detected. Set 'mariadb_auto_config: true' in the addon options to wire it into BirdNET-Go automatically (and disable SQLite). Connection details:" + bashio::log.blue "Database user : ${MYSQL_USER}" + bashio::log.blue "Database password: ${MYSQL_PASS}" + bashio::log.blue "Database name : ${MYSQL_DATABASE}" + bashio::log.blue "Host-name : ${MYSQL_HOST}" + bashio::log.blue "Port : ${MYSQL_PORT}" + bashio::log.green "---" exit 0 fi @@ -24,17 +38,11 @@ if [ ! -f "$CONFIG_LOCATION" ]; then exit 0 fi -MYSQL_HOST="$(bashio::services 'mysql' 'host')" -MYSQL_PORT="$(bashio::services 'mysql' 'port')" -MYSQL_USER="$(bashio::services 'mysql' 'username')" -MYSQL_PASS="$(bashio::services 'mysql' 'password')" - bashio::log.green "---" -bashio::log.blue "Home Assistant MariaDB addon detected; auto-configuring BirdNET-Go" +bashio::log.blue "mariadb_auto_config enabled; writing Home Assistant MariaDB credentials into BirdNET-Go config and disabling SQLite" bashio::log.blue "Host: ${MYSQL_HOST}:${MYSQL_PORT}" bashio::log.blue "User: ${MYSQL_USER}" bashio::log.blue "Database: ${MYSQL_DATABASE} (will be created by BirdNET-Go on first connect)" -bashio::log.blue "(Set 'mariadb_disable: true' in addon options to opt out)" bashio::log.green "---" # Upstream config.go stores port as a string; pass it as such to match. diff --git a/birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh b/birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh index 7096fc0163..e130c7fd86 100755 --- a/birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh +++ b/birdnet-go/rootfs/etc/cont-init.d/33-mqtt.sh @@ -2,10 +2,12 @@ # shellcheck shell=bash set -e -# If the Home Assistant MQTT addon is active, wire its credentials directly -# into BirdNET-Go's config.yaml (upstream reads MQTT settings only from YAML; -# no env-var overrides exist). Users who prefer to manage MQTT manually can -# set mqtt_disable: true in the addon options. +# When the Home Assistant MQTT addon is active, optionally wire its +# credentials directly into BirdNET-Go's config.yaml. Upstream reads MQTT +# settings only from YAML (no env-var overrides exist), so this is the only +# way to auto-configure them. The behaviour is opt-in via the +# mqtt_auto_config addon option. When the option is off but Mosquitto is +# detected, we log a one-shot hint pointing users at the option. CONFIG_LOCATION="/config/config.yaml" @@ -13,27 +15,31 @@ if ! bashio::services.available 'mqtt'; then exit 0 fi -if bashio::config.true 'mqtt_disable'; then - bashio::log.info "MQTT auto-configuration disabled by 'mqtt_disable' addon option; skipping." - exit 0 -fi - -if [ ! -f "$CONFIG_LOCATION" ]; then - bashio::log.warning "Skipping MQTT auto-configuration: $CONFIG_LOCATION not found" - exit 0 -fi - MQTT_HOST="$(bashio::services 'mqtt' 'host')" MQTT_PORT="$(bashio::services 'mqtt' 'port')" MQTT_USER="$(bashio::services 'mqtt' 'username')" MQTT_PASS="$(bashio::services 'mqtt' 'password')" MQTT_BROKER="tcp://${MQTT_HOST}:${MQTT_PORT}" +if ! bashio::config.true 'mqtt_auto_config'; then + bashio::log.green "---" + bashio::log.yellow "Home Assistant MQTT addon detected. Set 'mqtt_auto_config: true' in the addon options to wire it into BirdNET-Go automatically. Connection details:" + bashio::log.blue "MQTT user : ${MQTT_USER}" + bashio::log.blue "MQTT password: ${MQTT_PASS}" + bashio::log.blue "MQTT broker : ${MQTT_BROKER}" + bashio::log.green "---" + exit 0 +fi + +if [ ! -f "$CONFIG_LOCATION" ]; then + bashio::log.warning "Skipping MQTT auto-configuration: $CONFIG_LOCATION not found" + exit 0 +fi + bashio::log.green "---" -bashio::log.blue "Home Assistant MQTT addon detected; auto-configuring BirdNET-Go" +bashio::log.blue "mqtt_auto_config enabled; writing Home Assistant MQTT credentials into BirdNET-Go config" bashio::log.blue "Broker: ${MQTT_BROKER}" bashio::log.blue "User: ${MQTT_USER}" -bashio::log.blue "(Set 'mqtt_disable: true' in addon options to opt out)" bashio::log.green "---" # $broker / $user / $pass / "birdnet" are jq/yq variables and literals, From a4f6ab3b2fb59440fbf6546e471a0102f8f7c031 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 28 May 2026 17:24:11 +0000 Subject: [PATCH 3/3] birdnet-go: don't abort init when default-config download fails Removing /config/config.yaml after a failed first-boot curl left the next yq read (.realtime.audio.export.path) trying to open a missing file; under set -e that aborts the entire cont-init script, so the addon would never get to seed its defaults or start BirdNET-Go. Seed an empty YAML document ({}) instead. The existing "//=" defaults block then populates output.sqlite.path, logging.file_output.*, and the migration block writes realtime.audio.export.path. Result: an offline first boot now produces a valid minimal config.yaml and the container starts cleanly. Also harden the yq read with "// """ so a freshly seeded "{}" doc returns an empty string (caught by the existing :-DEFAULT fallback) rather than the literal string "null". --- birdnet-go/CHANGELOG.md | 2 +- .../rootfs/etc/cont-init.d/01-structure.sh | 17 +++++++++++------ 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/birdnet-go/CHANGELOG.md b/birdnet-go/CHANGELOG.md index 51e60fbff4..2e2669245b 100644 --- a/birdnet-go/CHANGELOG.md +++ b/birdnet-go/CHANGELOG.md @@ -5,7 +5,7 @@ - **Breaking (UI only)**: The nginx ingress reverse-proxy no longer rewrites HTML `href`/`src`/`action` attributes; upstream BirdNET-Go handles those itself via `X-Ingress-Path`. JavaScript string-literal rewrites are unchanged. Please file an issue if you see broken images, links, or forms in the ingress UI after upgrade. - Fix database-migration restore: the timestamped backup created during a `BIRDSONGS_FOLDER` change was being written to the script's working directory and looked up under a fresh timestamp on restore, so a SQL failure left the user unable to recover. Backup path is now absolute and reused for restore. - Harden the `BIRDSONGS_FOLDER` SQL/YAML path substitution: paths containing characters outside `[A-Za-z0-9._/-]` are now rejected up front instead of being interpolated raw into the SQL UPDATE statement. -- Tolerate a missing internet connection on first boot: if the default `config.yaml` cannot be downloaded from GitHub, BirdNET-Go now falls back to its embedded defaults instead of starting with an empty config. +- Tolerate a missing internet connection on first boot: if the default `config.yaml` cannot be downloaded from GitHub, the init script now seeds an empty YAML document so the addon-defaults block populates a usable config (rather than aborting the script on the next `yq` call under `set -e`). - Warn (without failing the build) if the upstream `entrypoint.sh` patch target drifts in a new nightly. - Remove a dead nginx upstream definition that pointed at an unused port. diff --git a/birdnet-go/rootfs/etc/cont-init.d/01-structure.sh b/birdnet-go/rootfs/etc/cont-init.d/01-structure.sh index 616a2e29f3..a8d1e13ed4 100755 --- a/birdnet-go/rootfs/etc/cont-init.d/01-structure.sh +++ b/birdnet-go/rootfs/etc/cont-init.d/01-structure.sh @@ -27,13 +27,16 @@ validate_safe_path() { if [ ! -f "$CONFIG_LOCATION" ]; then bashio::log.warning "There is no config.yaml yet in the config folder, downloading a default one. Please customize" - # Network may be unreachable on first boot; tolerate failure and let - # birdnet-go fall back to its embedded default on first run. + # Network may be unreachable on first boot. If the download fails, seed + # an empty YAML document so the yq reads/writes below succeed and the + # default-value seeding logic later in this script populates a usable + # config. (We can't remove the file and continue — subsequent yq calls + # under set -e would abort the init script.) if ! curl -fL -s -S \ https://raw.githubusercontent.com/tphakala/birdnet-go/refs/heads/main/internal/conf/config.yaml \ -o "$CONFIG_LOCATION"; then - bashio::log.warning "Could not download default config.yaml; birdnet-go will create one from its embedded defaults" - rm -f "$CONFIG_LOCATION" + bashio::log.warning "Could not download default config.yaml; seeding an empty document so addon defaults can populate it" + echo '{}' > "$CONFIG_LOCATION" fi fi @@ -48,8 +51,10 @@ fi ###################### # Birdsongs Location ###################### -# Read the current folder from config.yaml; fall back to the legacy default. -CURRENT_BIRDSONGS_FOLDER="$(yq '.realtime.audio.export.path' "$CONFIG_LOCATION" | tr -d '\"')" +# Read the current folder from config.yaml; "// """ collapses both missing +# keys and explicit nulls (e.g. in a freshly seeded "{}" doc) to an empty +# string so the ${VAR:-DEFAULT} fallback below kicks in. +CURRENT_BIRDSONGS_FOLDER="$(yq -r '.realtime.audio.export.path // ""' "$CONFIG_LOCATION")" CURRENT_BIRDSONGS_FOLDER="${CURRENT_BIRDSONGS_FOLDER:-$DEFAULT_BIRDSONGS_FOLDER}" # Treat the upstream-shipped relative "clips/" as the legacy default. if [[ "$CURRENT_BIRDSONGS_FOLDER" == "clips" || "$CURRENT_BIRDSONGS_FOLDER" == "clips/" ]]; then