Model curtailment on a negative export price (#3986)

[tieskuh]

What this does

Implements the request in #3986: lets Predbat model the curtailment of export that many of us apply when the export price goes negative (e.g. an automation that sets the inverter''s export limit to zero while export_price < 0). Predbat does not drive the curtailment — it only models it, so the feature is inverter-agnostic and adds no new control failure modes.

Why

With a static export_limit, Predbat models export as proceeding normally during negative-price slots, causing two problems:

1. Inaccurate projection — it books a phantom export cost/credit and models PV as exported for slots that are actually curtailed.
2. PV calibration pollution — when the inverter throttles generation during curtailment, measured generation drops and pv_calibration reads it as panel underperformance, dragging the forecast down (clamped as low as 20%). This hits Forecast.Solar users hardest, as they have no auto-dampening.

What''s included

• New control select.predbat_curtail_on_negative_export_price (expert mode, default off):
  • off — no curtailment modelled (existing behaviour, unchanged).
  • curtail_excess — grid export modelled as blocked whenever the export rate is negative, while PV still charges the battery and supplies the house.
  • solar_production_off — PV modelled as fully off during negative-price slots, for inverters that can only disable generation rather than limit export.
• Prediction (prediction.py) — per slot, when enabled and export_rate < 0, the effective export limit is set to zero (and PV to zero in solar_production_off).
• PV calibration fix (solcast.py) — negative-price slots are excluded from the calibration input (both the actual and forecast aggregates), using the recorded history of the predbat.rates_export entity, so calibration can stay enabled without curtailment being mistaken for underperformance.
• Plan view (output.py) — curtailed PV is shown in a distinct colour; no new columns/fields, so the Predbat table card is unaffected.
• Docs — customisation.md, energy-rates.md, apps-yaml.md.
• Tests — unit tests for the prediction modes and the calibration exclusion.

Design note

The original issue proposed a configurable price threshold. In testing, the only sensible threshold turned out to be 0 (curtail when the price is negative); a configurable value mainly added confusion, and export fees can be folded into the rate sensor instead. So the trigger is fixed at < 0, and the control is a single select that also captures the two real curtailment behaviours (export-only vs full PV-off).

Off by default

With the default off, nothing changes for existing users. The feature is modelling-only; the actual curtailment stays with the user''s inverter or automation.

Testing

• Unit tests added (model scenarios for both modes + a positive-price guard; calibration-exclusion test including a guard that the correct rate entity is queried).
• Validated on a live SolarEdge (dual-inverter) + Forecast.Solar system: the plan correctly curtails (colour + zero export) during negative-price slots and leaves positive-price export untouched. An A/B toggle confirmed the calibration exclusion fires only on negative-price days and leaves other days byte-identical.

Closes #3986.

[springfall2008]

Looks good, but the use of 'off' is incorrect it should be False/True for apps.yaml settings. Let me know if you are okay to update?

[tieskuh]

Happy to update if you'd like. I chose 'off ' as option because it is used in the existing convention for selects: manual_charge, manual_export, manual_demand, the manual_*_rates, manual_soc, manual_api, the freeze selects, etc. all use off as their default/option, so keeping it would be consistent with those (and the YAML-boolean caveat would apply to them equally). It's also a 3-way select (curtail_excess / solar_production_off / off), so it can't become a plain True/False.

I'm happy either way: keep off for consistency, or rename the disabled option to e.g. disabled. What would you prefer?

Below a screenshot of the plan tomorrow with the negative prices and curtailment:

I ran several A/B tests for a week to see if calibration works correctly.

[tieskuh]

Cross-referencing #4036 (Native Solar Clipping Buffer) for coordination — the two are complementary, but they touch overlapping files, so whichever merges second will need a rebase.

Both deal with PV that can''t be exported, but via a different trigger and mechanism:

	This PR (#4023)	#4036
Trigger	export price is negative (rate-conditional)	PV exceeds a power limit (inverter AC / DNO export)
Goal	avoid a phantom export cost + keep PV calibration clean	avoid solar lost to clipping
Approach	modelling-only (models the curtailment)	active control (reserves battery SoC headroom)
Related issue	#3986	#1206 (+ the static-limit family #3442 / #3481 / #3828)

The original request in #3986 noted that the static export/DNO-limit cases would need a deeper change that actively lowers SoC ahead of high-generation days — which is exactly what #4036 tackles. So they complement rather than duplicate each other.

Overlap heads-up: both touch prediction.py (the export/clip block), solcast.py, output.py, config.py, fetch.py and several test files, so a rebase + conflict resolution will be needed for whichever lands second. Happy to rebase this one on top of #4036 if that goes first.