Skip to content

API: read, code and check invoices, post to MYOB with safe retries - #1

Merged
TonyTech404 merged 7 commits into
mainfrom
feat/api-pipeline
Nov 19, 2026
Merged

TonyTech404 merged 7 commits into
mainfrom
feat/api-pipeline

Conversation

@TonyTech404

Copy link
Copy Markdown
Owner

What this adds

The whole back end, from an uploaded PDF to a bill in MYOB.

  • Reading: Azure AI Document Intelligence (prebuilt-invoice) over REST. Fields keep their confidence and bounding polygon, so the UI can show where each value came from.
  • Coding: Azure OpenAI picks an account and GST code per line using the chart of accounts, bookkeeping rules and the supplier's past codings. Strict JSON schema; account codes are an enum of active accounts.
  • Checks: sums, GST against tax codes, ABN checksum, dates, duplicates and closed accounts, in plain code.
  • Posting: an outbox (sync_jobs) with backoff. A timeout is treated as "maybe it worked": the next attempt looks the bill up by supplier invoice number before posting again, because MYOB has no idempotency keys.
  • Reconciliation: posted invoices vs the MYOB company file.
  • Simulated MYOB: AccountRight request and response shapes, with flaky and dropped-response modes.

Decisions

  • MYOB rate limits come back as 403 Developer Over Qps, so the client classifies 403s by body rather than treating them as auth failures.
  • Seed data is real model output (api/scripts/build-seed.ts), not invented numbers.
  • Extraction results are cached by file hash to stay inside the Document Intelligence free tier.

Testing

  • 30 unit tests: ABN, checks, Document Intelligence mapping (real responses as fixtures), MYOB client error classification, backoff, reconciliation.
  • Integration test against Postgres (CI uses a Neon branch): lost response with the guard on gives one bill; with it off, two bills, and reconciliation flags the duplicate; a closed account fails without retrying.

Fictional Australian suppliers a coffee roaster deals with: green
coffee (GST-free beans, taxable freight), dairy, packaging, freight
with a mistyped ABN, an espresso machine repair with a capital item,
and a crooked phone photo of a receipt. Rendered from HTML with Chrome
so they can be regenerated. Phone numbers use the 02 5550 range ACMA
reserves for fiction.
…enAI

- docintel: prebuilt-invoice over REST, polling, and mapping to the
  app's shape, keeping each field's confidence and bounding polygon.
  Cleans vendor names that come back with logo text attached.
- coding: chart of accounts, bookkeeping rules and the supplier's past
  codings in the prompt; strict JSON schema with account codes as an
  enum of active accounts.
- checks: sums, GST against tax codes, ABN checksum, dates, duplicates,
  closed accounts and low-confidence reads, in plain tested code.
- Tests use real Document Intelligence responses as fixtures.
MYOB has no idempotency keys, so a timed-out POST may have created the
bill. The worker marks such attempts uncertain and searches MYOB by
supplier invoice number before posting again. Rate limits (403
Developer Over Qps) and 503s back off with jitter; 400s stop and go
back to a person.

The simulated AccountRight API follows MYOB's shapes (Location
headers, OData filters, error bodies) and can be flaky or drop
responses after saving. The client takes a transport, so the real
API is a configuration change.

Reconciliation compares posted invoices with the company file and
reports duplicates, edited amounts, deleted bills and manual entries.
- Hono app: upload, extract, code, review, approve, post, sync log,
  reconcile, settings for the failure modes, and a cron endpoint.
- Each visitor gets a workspace seeded with a month of history. The
  seed comes from running the real pipeline over past invoices once
  (scripts/build-seed.ts), not hand-written numbers.
- Free-tier guards: per-visitor and daily rate limits, and extraction
  results cached by file hash.
- esbuild bundles the API into one file for Azure Functions.
- Integration test: lost responses with and without the duplicate
  guard, and MYOB rejecting a closed account, against Postgres.
Integration tests run against a dedicated Neon branch.
… the mock

On CI the mock's own database round trips could exceed the 2.5 s
client timeout, so an unguarded retry posted a third bill. CI now
waits 8 s, and the mock loads accounts and tax codes once.
@TonyTech404
TonyTech404 merged commit c131425 into main Nov 19, 2026
1 check passed
@TonyTech404
TonyTech404 deleted the feat/api-pipeline branch November 19, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant