Skip to content

CDST rework#25

Merged
HughRunyan merged 3 commits into
mainfrom
cdst-red
Jun 25, 2026
Merged

CDST rework#25
HughRunyan merged 3 commits into
mainfrom
cdst-red

Conversation

@HughRunyan

@HughRunyan HughRunyan commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Jira: WP-344  ·  Type: Bug fix + refactor  ·  Labels: bug, refactor, test  ·  Pairs with WasteMAP PR (WP-344)

What

The model-side half of WP-344 — the city DST chart's red "Something went wrong…" error. Replaces the hand-rolled diversion allocator in City.mass_checker_math with an exact min-cost max-flow solver (SWEET_python/dst_allocation.py). The frontend half (showing the reason) is the companion WasteMAP PR.

Root cause

Splitting the compost / anaerobic / recycling sliders across waste types is a transportation problem — each treatment accepts only some types, and no type can be diverted more than it exists. The old loop solved it with local moves: it redistributed an over-allocated type's excess proportional to each sibling's current share, ignoring remaining capacity, overshot small pools, then permanently banned the overshot type — painting itself into a corner and raising CustomError on inputs that are actually feasible. (It could also silently divert 0 tons, or raise a bare AssertionError → HTTP 500.)

The fix

  • Allocate the three contended treatments (the only ones competing over wood + paper_cardboard) with min-cost max-flow — correct over the entire feasible region. Combustion keeps its existing uniform-fraction-of-leftover step, now with a divide-by-zero guard.
  • Genuine infeasibility raises an actionable CustomError naming the binding slider and its true max — e.g. "Compost can be at most 48.0% of this city's waste … but 95.0% was requested" — which the frontend surfaces (WP-344 companion).
  • No-conflict happy path untouched. city_params.py net −252 lines.

Validation

58 unit tests (solver vs an independent exact-rational oracle) + 18 integration tests (real mass_checker_math, incl. a 0–100% recycling sweep) + a ~400k-case fuzz harness (dst_allocation_prototype.py) → 0 mismatches, 0 invalid allocations, 0 crashes. Verified live on Korçë: the previously-erroring combo now renders; infeasible inputs return clean 400s.

Follow-up fix — food-waste-prevention renormalization

A separate, pre-existing bug surfaced on dev (Algiers, composting + recycling + food prevention): the food-waste-prevention step rescaled non-food fractions by old_nonfood/new_nonfood, over-inflating metal/glass/other/textiles so a non-food type went negative → spurious "Negative mass for <type>". Now rescaled by the true total-reduction factor (1 - food_waste_prevention × food_fraction), preserving waste_fractions[w] × waste_mass == waste_masses[w]. Adds tests/test_food_waste_prevention.py (DB-free, fails pre-fix / passes post-fix).

Acceptance criteria (WP-344)

  • Root cause identified — the local-redistribution heuristic above.
  • Error eliminated where mathematically possible — every physically feasible diversion mix now computes.
  • Where unavoidable, the user is told what to reduce and why — backend names the binding slider + its max (surfaced by the companion frontend PR).

Definition of Done

  • Acceptance criteria met (model side)
  • Tests pass — 86 in tests/
  • Reviewed & merged (alongside the WasteMAP WP-344 PR)

Food prevention removes food mass and shrinks the total, but the non-food
waste fractions were rescaled by old_nonfood_total / new_nonfood_total instead
of by the actual total-reduction factor. That over-inflated every non-food
share, so the diversion allocator believed more metal/glass/other/textiles
existed than the unchanged masses hold, and the downstream mass check raised a
spurious "Negative mass for <type>" -- erroring the city DST chart (e.g. Algiers
with composting + recycling + food prevention).

Rescale every fraction by (1 - food_waste_prevention * food_fraction), the same
factor waste_mass is reduced by, so the invariant
    waste_fractions[w] * waste_mass == waste_masses[w]
holds for all waste types.

Adds tests/test_food_waste_prevention.py: a DB-free regression suite built from
country defaults (dst_baseline_blank) that exercises the real
implement_dst_changes_simple_v1_5 path. The tests fail on the pre-fix code and
pass on the fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
f"{t.capitalize()} can be at most {cond * 100:.1f}% given "
f"the other diversion selections for this city, but "
f"{three_targets[t] * 100:.1f}% was requested. Reduce "
f"{t} or the other diversion sliders.",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! They'll know exactly which slider.

@@ -0,0 +1,582 @@
"""

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels weird. If this is meant to replace something in the future, should it be in a future ticket and not here?

@andre-scheinwald andre-scheinwald left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot going on here. I do want to hear your thoughts on my comment in the prototype.py file. Even if it's "no".

@andre-scheinwald andre-scheinwald left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Questions addressed!

@HughRunyan HughRunyan merged commit 3c7eb7f into main Jun 25, 2026
@HughRunyan HughRunyan deleted the cdst-red branch June 25, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants