Skip to content

Update tonies.json periodically, download it only when it changed - #563

Open
zimmermq wants to merge 1 commit into
toniebox-reverse-engineering:developfrom
zimmermq:tonies-json-periodic-update
Open

zimmermq wants to merge 1 commit into
toniebox-reverse-engineering:developfrom
zimmermq:tonies-json-periodic-update

Conversation

@zimmermq

Copy link
Copy Markdown

Fixes #120

What

"Auto-Update tonies.json" only updates at startup. This adds the task for the periodic check that was missing, and makes the check cheap, which was the condition for #136: an unchanged tonies.json is not downloaded again.

  • A task started in server_init runs tonies_update every core.tonies_json_update_interval hours (new setting, default 6, 0 = only at startup as before) while core.tonies_json_auto_update is on. It looks at the clock and the settings every 5 seconds, so a changed interval or a shutdown does not wait.
  • tonies_update remembers the ETag of the tonies.json it downloaded and sends it as If-None-Match on the next update. GitHub answers an unchanged file with 304 and no body, also through the redirect from api.revvox.de (the header is part of the callback context and goes along with the redirect). On 304 it logs "tonies.json is unchanged, keeping it" and reloads nothing. The ETag is kept in memory only, so the update at startup downloads the whole file as before.
  • web_request gets one optional header, If-None-Match, from a new if_none_match field of cbr_ctx_t. Every cbr_ctx_t is zeroed (fillBaseCtx, {0}), so other callers send nothing new.
  • Only one update runs at a time: the startup update, the task and /api/toniesJsonUpdate can overlap, and two downloads into the same temp file would mix. A second one returns ERROR_ALREADY_RUNNING and logs "tonies.json is being updated already". The flag is set under MUTEX_TONIES_JSON_CACHE but the mutex is not held during the download, so the mutex manager does not warn about a long lock.
  • Shutdown waits for an update that is running (at most 30 seconds) before tonies_deinit, cache_deinit and mutex_manager_deinit; otherwise a download that ends after them would reload the catalogue with a destroyed mutex. Once internal.exit is set, tonies_update refuses to start ("Not updating tonies.json while teddyCloud shuts down"), and the task checks it after each wait, so no update starts during shutdown. This also covers /api/toniesJsonUpdate shortly before an exit, which can hit the same today.
  • The download uses its own copies of the paths: tonies_deinit frees the shared ones, which saving tonies.custom.json or /api/toniesJsonReload may do while a download runs.

Tested (Linux debug build with ASan and UBSan, in Docker; interval switched to minutes in the test build only)

Case Result
startup 200, full download
periodic check after one interval 304 through the api.revvox.de redirect, nothing downloaded or reloaded
two /api/toniesJsonUpdate at once one asks, the other logs "tonies.json is being updated already"
restart 200, full download, as before
/api/triggerExit while a full download runs shutdown waits, the download finishes and is loaded, then teddyCloud exits (without the wait: it exits in the middle of the download)
/api/toniesJsonUpdate after /api/triggerExit refused, "Not updating tonies.json while teddyCloud shuts down"

No sanitizer reports in any of them.

Not in this PR

The lookups (tonies_byAudioId and others) return pointers into the cache that callers use after the mutex is released, and a reload frees that cache. That race exists today with /api/toniesJsonUpdate; a periodic check reloads only when the catalogue changed, a few times a week. I would follow up with a separate PR that keeps the replaced cache until the next reload.

@SciLor SciLor added this to the 0.8.0 milestone Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants