feat(storage+octopus): fetch_cached helper + migrate Octopus onto the storage component#4039
Open
mgazza wants to merge 6 commits into
Open
feat(storage+octopus): fetch_cached helper + migrate Octopus onto the storage component#4039mgazza wants to merge 6 commits into
mgazza wants to merge 6 commits into
Conversation
Wrap fetch_fn in the stale and hard-miss branches so a refresh exception degrades to the cached/stale value (or None) instead of propagating, per SWR semantics. Adds tests for None-in-stale, raise-in-stale, raise-in-miss.
…poke file cache Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d code
- _download normalises falsy results to None so storage.fetch_cached does not
cache an empty {} error response (async_download_octopus_url returns {} on
HTTP/parse errors) for the fresh window
- add regression test asserting an empty error is returned falsy and not cached
- remove duplicate/late json imports, late hashlib import, dead self.url_cache,
unreachable tariffs guard; refresh module docstring
…storage error return
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
fetch_cachedstale-while-revalidate helper to the storage component and migrates the Octopus integration off its bespoke file cache ontoself.storage— bringing Octopus in line withsolcast.py,gecloud.py, andfuturerate.py, which already use the storage component.Motivation
octopus.pywas the last component still doing its ownopen()/YAML caching (URL cache + per-user cache) with a bespoke file-lock stale-while-revalidate loop. Routing it throughStorageComponentmeans there's a single, testable cache path across all components, and the cache backend can be swapped (e.g. for a shared/distributed backend) without touching component logic.What changed
storage.py— newfetch_cachedhelper (single-instance-clean):StorageBase.fetch_cached(module, filename, fetch_fn, fresh_minutes=30, stale_minutes=35, format="yaml")— fresh hit returns cache without callingfetch_fn; stale window serves cached while one caller refreshes; hard miss fetches and stores. A refresh that returnsNoneor raises degrades gracefully to the cached/Nonevalue (never propagates)._acquire_refresh_lock/_release_refresh_lockhooks default to no-ops (a single instance never contends). A subclass can override them to add real coordination; the OSS base stays single-instance with no notion of instances or distributed locks.StorageComponent.fetch_cacheddelegates to the backend.octopus.py— migrate ontoself.storage:"octopus"; account data, Kraken token, saving sessions and intelligent devices cache under module"octopus_user".fetch_url_cachednow usesstorage.fetch_cached; its_downloadwrapper normalises the{}error return ofasync_download_octopus_urltoNoneso error responses are never cached.load_url_from_cache/save_url_to_cache/clean_url_cache, the file-lock loop, the cache-directory plumbing, and a duplicate import.Behaviour
Single-instance behaviour is unchanged: same 30-minute fresh window / 30–35-minute stale window, same user-cache fields preserved across restart. Self-hosted users see no functional difference.
Tests
test_storage.py:fetch_cachedmiss, fresh hit, stale-refresh (success / returns-None / raises), hard-miss-raises.test_fetch_url_cached.py: rewritten for storage-backed behaviour — first-fetch, fresh-hit-no-redownload, independent multi-URL caching, invalid-URL handling, and an empty-{}-error-not-cached regression test.test_octopus_cache.py: user-cache save/load round-trip via storage, missing-init-empty, None normalisation, key sanitisation.test_octopus_misc.py: updated for the removedclean_url_cache.All targeted tests and
run_all --quickpass.🤖 Generated with Claude Code