Skip to content

feat(storage+octopus): fetch_cached helper + migrate Octopus onto the storage component#4039

Open
mgazza wants to merge 6 commits into
mainfrom
feat/octopus-storage-migration
Open

feat(storage+octopus): fetch_cached helper + migrate Octopus onto the storage component#4039
mgazza wants to merge 6 commits into
mainfrom
feat/octopus-storage-migration

Conversation

@mgazza

@mgazza mgazza commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a fetch_cached stale-while-revalidate helper to the storage component and migrates the Octopus integration off its bespoke file cache onto self.storage — bringing Octopus in line with solcast.py, gecloud.py, and futurerate.py, which already use the storage component.

Motivation

octopus.py was the last component still doing its own open()/YAML caching (URL cache + per-user cache) with a bespoke file-lock stale-while-revalidate loop. Routing it through StorageComponent means there's a single, testable cache path across all components, and the cache backend can be swapped (e.g. for a shared/distributed backend) without touching component logic.

What changed

storage.py — new fetch_cached helper (single-instance-clean):

  • StorageBase.fetch_cached(module, filename, fetch_fn, fresh_minutes=30, stale_minutes=35, format="yaml") — fresh hit returns cache without calling fetch_fn; stale window serves cached while one caller refreshes; hard miss fetches and stores. A refresh that returns None or raises degrades gracefully to the cached/None value (never propagates).
  • _acquire_refresh_lock / _release_refresh_lock hooks default to no-ops (a single instance never contends). A subclass can override them to add real coordination; the OSS base stays single-instance with no notion of instances or distributed locks.
  • StorageComponent.fetch_cached delegates to the backend.

octopus.py — migrate onto self.storage:

  • URL/tariff responses (public, no secrets) cache under module "octopus"; account data, Kraken token, saving sessions and intelligent devices cache under module "octopus_user".
  • fetch_url_cached now uses storage.fetch_cached; its _download wrapper normalises the {} error return of async_download_octopus_url to None so error responses are never cached.
  • Removed the bespoke load_url_from_cache / save_url_to_cache / clean_url_cache, the file-lock loop, the cache-directory plumbing, and a duplicate import.
  • Falls back to a direct download (single-instance behaviour unchanged) when no storage component is present.

Behaviour

Single-instance behaviour is unchanged: same 30-minute fresh window / 30–35-minute stale window, same user-cache fields preserved across restart. Self-hosted users see no functional difference.

Tests

  • test_storage.py: fetch_cached miss, fresh hit, stale-refresh (success / returns-None / raises), hard-miss-raises.
  • test_fetch_url_cached.py: rewritten for storage-backed behaviour — first-fetch, fresh-hit-no-redownload, independent multi-URL caching, invalid-URL handling, and an empty-{}-error-not-cached regression test.
  • test_octopus_cache.py: user-cache save/load round-trip via storage, missing-init-empty, None normalisation, key sanitisation.
  • test_octopus_misc.py: updated for the removed clean_url_cache.

All targeted tests and run_all --quick pass.

🤖 Generated with Claude Code

mgazza and others added 6 commits June 6, 2026 23:47
Wrap fetch_fn in the stale and hard-miss branches so a refresh exception
degrades to the cached/stale value (or None) instead of propagating, per
SWR semantics. Adds tests for None-in-stale, raise-in-stale, raise-in-miss.
…poke file cache

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d code

- _download normalises falsy results to None so storage.fetch_cached does not
  cache an empty {} error response (async_download_octopus_url returns {} on
  HTTP/parse errors) for the fresh window
- add regression test asserting an empty error is returned falsy and not cached
- remove duplicate/late json imports, late hashlib import, dead self.url_cache,
  unreachable tariffs guard; refresh module docstring
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant