Files
392d712365 wger: add optional PowerSync service so the mobile app can sync offline (#2950) (#3125)
* feat(wger): add optional PowerSync service for the mobile app (#2950)

The official wger mobile app (2.0.3+) syncs offline through a PowerSync
service it expects at <SITE_URL>/ps/. The add-on only ran the Django
server, so the app reported the sync service as unreachable and the log
showed "Not Found: /ps/probes/liveness".

PowerSync now ships in the image and can be started alongside wger with
the new POWERSYNC_ENABLED option. It is served under /ps/ by the add-on's
own nginx, which is the topology upstream's docker deployment uses, so the
app reaches it over the same host, port and remote-access path as wger
itself, with no second endpoint to publish.

With the option unset nothing changes: no new process runs, the nginx
include matches no file, and SQLite stays the default database.

PowerSync replicates from PostgreSQL only, so enabling it means pointing
wger at Postgres with PS_DATABASE_URI, which overrides the DJANGO_DB_*
defaults and keeps Django and PowerSync on one database by construction.
There is no migration path from the existing SQLite database; the README
and CHANGELOG say so explicitly.

The service tree and its own node binary are copied from a pinned
journeyapps/powersync-service release. Both architectures' node binaries
need at most GLIBC_2.28 / GLIBCXX_3.4.21 and the tree holds a single
native module needing at most GLIBC_2.17, all satisfied by the Ubuntu
24.04 base; a node --version in the build fails CI loudly if that ever
stops being true. The sync rules are vendored at a recorded upstream
commit rather than fetched at build time, so they cannot drift away from
the schema on their own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(wger): rename the PowerSync configuration heading so it is not a duplicate

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(wger): read PowerSync settings from the config.yaml file too

Review of #3125 pointed out that the CONFIG_LOCATION config.yaml file -
the mechanism this add-on documents for extra environment variables - does
not reach 89-powersync.sh. Confirmed by reading .templates/01-config_yaml.sh:
it writes every assignment to /.env and injects it into /entrypoint.sh,
/etc/services.d/*/*run* and /etc/cont-init.d/*run*, and 89-powersync.sh
matches none of those globs. 90-run.sh does, which is why Django saw those
values and this script did not.

So a user who configured PS_DATABASE_URI that way got a refusal to start
PowerSync, and file-configured JWT keys were ignored and then overridden by
a freshly generated pair, because the generated keys are appended to /.env
after the file's own lines and the last assignment wins.

The six settings are now read from /.env, which both mechanisms write.
/.env takes precedence over the environment deliberately: that is the
precedence 90-run.sh produces for Django, and this script has to validate
the values Django will actually use.

Also correct the README's PostgreSQL privileges. CREATE SCHEMA ...
AUTHORIZATION requires the creator to be able to SET ROLE to the owner, and
on PostgreSQL 16 and later a CREATEROLE user only receives ADMIN OPTION on
roles it creates because createrole_self_grant defaults to empty - so the
documented non-superuser recipe could not have worked against the Postgres
17 add-on. And warn that both URIs carry a password which the shared
startup code prints to the add-on log unmasked, since neither variable name
matches its secret-name pattern.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(wger): make a refused PowerSync configuration stop the script in both runner modes

ha_entrypoint.sh runs a cont-init script one of two ways: it execs it, or
with ha_entry_source=true it rewrites "exit N" into "return N || exit N"
and sources it. The refusal path used an exit inside refuse(), which in the
second mode returns from that function only — so the script would carry on
and route /ps/ to a PowerSync it had just refused to start.

Nothing in this repo sets ha_entry_source today, so the path was not
reachable for this add-on, but being correct by coincidence is not worth
keeping. Validation now lives in prepare(), refusals return, and the single
caller turns that into the one top-level exit the rewrite handles. Verified
by applying that exact sed to the script and sourcing it with a SQLite URI:
the refusal is logged, /ps/ stays unrouted and the caller continues.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(wger): reject a SITE_URL with a query or fragment

wger builds the app's sync endpoint as "<SITE_URL>/ps/", so a SITE_URL like
https://wger.example.com?ref=app produces https://wger.example.com?ref=app/ps/
— the /ps/ lands inside the query string and the app never reaches the route.
The glob accepted it because there is no slash after the host.

Raised by CodeRabbit on #3125, and by the Codex review before it where I
skipped it as low value; two independent reviewers and a two-line fix
changes that judgement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(wger): warn that a config.yaml password with $ or a backtick is mangled

Testing the /.env round-trip against .templates/00-global_var.sh's
dotenv_quote() showed the env_vars route preserves $, backticks, hashes,
quotes, backslashes and spaces intact, while 01-config_yaml.sh writes the
value unquoted — so a password containing $ or a backtick is expanded by
the shell and silently truncated. That happens on the way to Django as well
(90-run.sh re-exports the raw line into /data/env.sh, which the entrypoint
sources), so it is not specific to PowerSync, but PowerSync is the first
feature here to ask users for a password-bearing URI. Point them at the
env_vars option for those values.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(wger): copy the SQLite database into Postgres when PowerSync is enabled

Enabling PowerSync means moving wger to Postgres, which until now meant
starting from an empty instance. The add-on now does the transfer itself on
the first start with PowerSync enabled.

Two guarantees, both of them tested:

The SQLite database is never written to. It is copied into /data first and
every read happens against the copy, so the original keeps its contents,
mtime and mode, and no -wal or -shm file appears beside it. Removing
PS_DATABASE_URI puts wger straight back onto it.

An import only ever happens into a Postgres database holding no user
accounts. That is wger's own test for a database that was never set up
(wger/tasks.py database_initialised(), where the admin user is created as
the last step of the bootstrap), so turning PowerSync off and on again
later leaves whatever is in Postgres alone.

Mechanism: dumpdata from the copy with PS_DATABASE_URI unset so wger's
settings fall back to the DJANGO_DB_* sqlite values, then migrate and
loaddata against Postgres. Primary keys are kept, so rows land unchanged
and loaddata resets the Postgres sequences for every model it loaded.
Content types, permissions, admin log entries and sessions are excluded
because migrate rebuilds the first two and the others point at them.

It runs in cont-init, before the upstream entrypoint bootstraps, and the
wait for Postgres is what stops that bootstrap from racing it: cont-init is
synchronous. Afterwards wger's own bootstrap finds a database with users and
leaves it alone, and ensure_admin_user() does not touch an existing admin.

This commit also carries the second round of Codex review:

- Rename the script to 89-powersync-run.sh and delete the /.env loader it
  gained in the previous round. Settings reach a cont-init script by being
  injected after its shebang, and 01-config_yaml.sh injects into
  /etc/cont-init.d/*run* while 00-global_var.sh injects into *.sh — so the
  name alone makes both mechanisms reach this script the way they already
  reach 90-run.sh. That removes 35 lines, stops sourcing /.env six times as
  root before the enable check, and fixes the loader's own flaw: an
  explicitly empty value could not override the environment.
- Copy only JWT_PUBLIC_KEY and JWT_PRIVATE_KEY lines out of the persisted
  key file into /.env. jwt_file_is_complete tolerates extra lines, and the
  entrypoint evaluates every assignment it finds in /.env.
- Refuse a SITE_URL with an empty host such as http://:8080, which the
  previous patterns accepted while the comment claimed otherwise.
- Drop the wal_errors temp file and let that command's stderr reach the
  add-on log directly.
- Log the wait for wger every five minutes rather than every minute.
- Document the missing ALTER ROLE wger REPLICATION, say that REPLICATION
  belongs to the PS_DATABASE_URI role rather than powersync_storage, note
  that the GRANT needs a role allowed to hand out that membership, and stop
  overstating that nothing at all changes for users who leave the feature
  off: a stale /ps/ route is cleared and one unused nginx map is parsed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(wger): make the import directory reachable, and stop a failed import silently

Two Major findings from the CodeRabbit review of the import, both reproduced
before fixing.

The wger user could not reach the import directory, so every import would
have failed on the default path. prepare() creates /data/powersync as root
with mode 0700 for the JWT key file, and opening anything beneath a
directory needs search permission on it, so chowning the import directory
inside it changed nothing. Demonstrated with a root-owned 0700 parent: the
unprivileged user gets "Permission denied", and 0711 lets it through. The
key file's own 0600 is what protects it, so the directory is now 0755, set
explicitly on both paths that create it so a 0700 left behind by the
previous version of this code is repaired rather than inherited - /data
survives upgrades. A failing chown is now fatal with its own message
instead of a warning, since the import cannot work without it.

A failed import also let the boot continue. The upstream entrypoint would
then bootstrap the empty Postgres database into a new instance with its own
admin user, after which target_database_state reports "populated" and the
SQLite import is skipped for good - the data is still in the file, but the
add-on would never offer to move it again. On failure the script now leaves
a marker and a third sed in the Dockerfile makes the patched entrypoint stop
on it, before "wger bootstrap". The add-on fails to start with the reason in
the log, which keeps the data importable. The marker is removed at the top
of every boot, so it can never outlive the boot that wrote it: verified both
for a later successful import and for the feature being switched off.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(wger): arm the import guard up front instead of on each failure path

A refused prerequisite short-circuited the guard. With PS_DATABASE_URI set
to a Postgres URI but PS_STORAGE_PG_URI missing - exactly the mistake the
refusal messages exist to catch - prepare() returned before
import_sqlite_data() ever ran, so no marker was written, the boot continued,
and the upstream entrypoint bootstrapped the empty Postgres database into a
new instance. From the next start the target held users, the import was
skipped for good, and the SQLite rows were stranded. Reproduced before
fixing: marker absent, boot would proceed.

The marker is now armed as soon as an import is pending rather than written
on each way out, so anything that stops this script leaves it in place,
including an error nobody anticipated. Only two paths clear it: a completed
import, and a target that already holds data. Also cleared when there is no
SQLite database to move.

It is armed only when Django is actually about to use Postgres and there is
something to move, so it cannot block a boot that was never at risk.
Verified across both axes: armed for a refused prerequisite, a failed
dumpdata, a failed loaddata and an unreachable database; cleared for a
successful import, an already-populated target, the feature switched off,
PS_DATABASE_URI unset or pointing at SQLite, and no SQLite file present.

Also narrow the claim in the README and CHANGELOG. The test is
User.objects.exists() and nothing more, so a Postgres database holding rows
but no users at all still counts as never set up and would be imported into.
Saying it "never overwrites what is already there" was broader than the code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Update config.yaml

---------

Co-authored-by: BirdNET-Go Addon Builder <addon-builder@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-18 07:43:15 +03:00

150 lines
6.8 KiB
YAML

# yamllint disable rule:line-length rule:document-start rule:comments
# Vendored, unmodified below this header, from wger-project/docker at commit
# 77fad3d9fae7180ce1ac132f4d5ed4cc30a8c20e
# (2026-08-21), file services/config-powersync/sync_rules.yaml.
# These files describe the schema PowerSync replicates, so they move together
# with the wger/server image rather than being fetched independently.
# Note that changes to this file are not watched.
# The service needs to be restarted for changes to take effect.
#
# A column added server-side does not reach rows already in bucket storage:
# the queries below (SELECT * included) only run when a row is replicated, and
# an AddField emits no WAL. After such a migration either run a backfill that
# touches the rows, or bump the version line below so the changed file forces
# a reprocess
#
# Sync rules version: 1
# Every table referenced here must be part of the "powersync" publication, which
# lists its tables explicitly (see the server's core migration 0027). Reading from
# a table that was not synced before also needs
# `ALTER PUBLICATION powersync ADD TABLE <table>;`
# Warning: a user may have at most 1000 buckets, i.e. parameter-query results
# summed across all streams. This counts the *parameter* rows, not the data
# rows inside a bucket (a single bucket can hold any number of rows). For a
# stream with a `with:` CTE the count is the number of rows the CTE returns,
# so for `user_ingredients` below that is the number of distinct ingredients
# a user has ever referenced. Exceeding the limit is a hard error
# (PSYNC_S2305 "Too many parameter query results").
# See https://docs.powersync.com/sync/rules/parameter-queries
#
# Streams are split by update frequency (cold / medium / hot) so that
# bucket compaction can collapse the head of hot buckets without being
# blocked by long-lived rows from cold tables.
#
# For details, see the documentation:
# https://docs.powersync.com/sync/streams/overview
# https://docs.powersync.com/maintenance-ops/compacting-buckets
config:
edition: 3
streams:
# Global reference data, shared by all users, changes rarely enough
core:
auto_subscribe: true
queries:
- SELECT * FROM core_language
- SELECT * FROM core_license
- SELECT * FROM core_repetitionunit
- SELECT * FROM core_weightunit
- SELECT * FROM exercises_exercise
- SELECT * FROM exercises_translation
- SELECT * FROM exercises_alias
- SELECT * FROM exercises_exercisecomment
- SELECT * FROM exercises_muscle
- SELECT * FROM exercises_exercise_muscles
- SELECT * FROM exercises_exercise_muscles_secondary
- SELECT * FROM exercises_equipment
- SELECT * FROM exercises_exercise_equipment
- SELECT * FROM exercises_exercisecategory
- SELECT * FROM exercises_exerciseimage
- SELECT * FROM exercises_exercisevideo
# COLD, per-user data that almost never changes after creation.
user_profile:
auto_subscribe: true
queries:
- SELECT * FROM core_userprofile WHERE CAST(user_id AS TEXT) = auth.user_id()
- SELECT * FROM gallery_image WHERE CAST(user_id AS TEXT) = auth.user_id()
# COLD but potentially large, only the per-user *filter set* changes when the user logs new foods
user_ingredients:
auto_subscribe: true
with:
user_ingredients: |
SELECT DISTINCT nutrition_synced_ingredient.id
FROM nutrition_synced_ingredient
WHERE nutrition_synced_ingredient.id IN (
SELECT nutrition_logitem.ingredient_id FROM nutrition_logitem
JOIN nutrition_nutritionplan ON nutrition_logitem.plan_id = nutrition_nutritionplan.id
WHERE CAST(nutrition_nutritionplan.user_id AS TEXT) = auth.user_id())
OR nutrition_synced_ingredient.id IN (
SELECT nutrition_mealitem.ingredient_id FROM nutrition_mealitem
JOIN nutrition_meal ON nutrition_mealitem.meal_id = nutrition_meal.id
JOIN nutrition_nutritionplan ON nutrition_meal.plan_id = nutrition_nutritionplan.id
WHERE CAST(nutrition_nutritionplan.user_id AS TEXT) = auth.user_id())
queries:
- SELECT * FROM nutrition_synced_ingredient AS nutrition_ingredient WHERE id IN user_ingredients
- |
SELECT nutrition_image.* FROM nutrition_image
WHERE nutrition_image.ingredient_id IN user_ingredients
- |
SELECT nutrition_ingredientweightunit.* FROM nutrition_ingredientweightunit
WHERE nutrition_ingredientweightunit.ingredient_id IN user_ingredients
# MEDIUM. Edited e.g. when the user builds or edits their routine or nutrition plan,
# but not on every workout.
user_planning:
auto_subscribe: true
queries:
# Routines (templates excluded)
- SELECT * FROM manager_routine WHERE CAST(user_id AS TEXT) = auth.user_id() AND is_template = FALSE
# Measurement categories (the measurements themselves are HOT, see below)
- SELECT * FROM measurements_category WHERE CAST(user_id AS TEXT) = auth.user_id()
# Nutrition plan structure (not the log items)
- SELECT * FROM nutrition_nutritionplan WHERE CAST(user_id AS TEXT) = auth.user_id()
- |
SELECT nutrition_meal.*
FROM nutrition_meal
JOIN nutrition_nutritionplan
ON nutrition_meal.plan_id = nutrition_nutritionplan.id
WHERE CAST(nutrition_nutritionplan.user_id AS TEXT) = auth.user_id()
- |
SELECT nutrition_mealitem.*
FROM nutrition_mealitem
JOIN nutrition_meal
ON nutrition_mealitem.meal_id = nutrition_meal.id
JOIN nutrition_nutritionplan
ON nutrition_meal.plan_id = nutrition_nutritionplan.id
WHERE CAST(nutrition_nutritionplan.user_id AS TEXT) = auth.user_id()
# HOT. Generates one or more new rows per workout / meal. Compaction has the
# biggest impact here, so it must stay isolated from the other streams above
user_activity:
auto_subscribe: true
queries:
# Measurements, including body weight entries, so typically one or more new rows per day
- |
SELECT measurements_measurement.*
FROM measurements_measurement
INNER JOIN measurements_category
ON measurements_measurement.category_id = measurements_category.id
WHERE CAST(measurements_category.user_id AS TEXT) = auth.user_id()
# Workout sessions and per-set logs
- SELECT * FROM manager_workoutsession WHERE CAST(user_id AS TEXT) = auth.user_id()
- SELECT * FROM manager_workoutlog WHERE CAST(user_id AS TEXT) = auth.user_id()
# Nutrition log entries
- |
SELECT nutrition_logitem.*
FROM nutrition_logitem
JOIN nutrition_nutritionplan
ON nutrition_logitem.plan_id = nutrition_nutritionplan.id
WHERE CAST(nutrition_nutritionplan.user_id AS TEXT) = auth.user_id()