Skip to content

fix(backfill): read prices from the item page when the data endpoint 404s - #46

Merged
icemansparks merged 1 commit into
merkerhoodfrom
fix/wowauctions-html-payload
Sep 20, 2026
Merged

icemansparks merged 1 commit into
merkerhoodfrom
fix/wowauctions-html-payload

Conversation

@icemansparks

Copy link
Copy Markdown
Collaborator

Changes Proposed:

  • wowauctions.fetch_item() falls back to the item page when the data endpoint returns 404, reading the same payload out of the page's __NEXT_DATA__ blob.
  • Once the page answers where the data endpoint did not, the run stays on the page, so an item with genuinely no data still costs one request rather than two.
  • Five tests for the fallback, the latch, and the no-data case.

Issues Addressed:

SOURCE:

As of 2026-09-20 https://www.wowauctions.net/_next/data/{buildId}/auctionHouse/chromie-craft/chromiecraft/mergedAh/x-{id}.json returns 404 for every id under a buildId resolved seconds earlier, including sentinel 4389. The sentinel check therefore re-resolves and still fails, and the tool concludes the whole catalogue is unpriced.

https://www.wowauctions.net/auctionHouse/chromie-craft/chromiecraft/mergedAh/x-{id} serves the identical object at props.pageProps.item, with the same stats.avg_price / stats.minimum_buyout / item_count / item_last_seen keys transform.build_row() already reads.

Tests Performed:

  • python3 -m unittest discover: 56 tests, all pass.
  • Live against the site: items 4389, 18335 and 8170 return current stats (scan of 2026-09-20 19:25 UTC), id 999999 returns None without latching.

How to Test the Changes:

  1. cd tools/price-backfill && python3 -m unittest discover -v
  2. python3 -c "import wowauctions; b=wowauctions.resolve_build_id(); print(wowauctions.fetch_item(b, 4389))" prints the item's stats.

…404s

Every id 404s under a freshly resolved buildId, the sentinel included, so
the tool reads the whole catalogue as having no ChromieCraft data. The item
page still carries the same payload in its __NEXT_DATA__ blob.

The page is only consulted after a 404, and once it answers where the data
endpoint did not, the run stays on the page so an unpriced item does not
cost two requests.
@icemansparks
icemansparks merged commit 5e6d40d into merkerhood Sep 20, 2026
1 check passed
@icemansparks
icemansparks deleted the fix/wowauctions-html-payload branch September 20, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant