Repository navigation

[SPARK-60325][SQL] Re-enable Arrow columnar input to Python UDFs, holding only vectors that are not reused - #59582

Draft

viirya wants to merge 3 commits into

apache:masterfrom

viirya:SPARK-60325

Draft

[SPARK-60325][SQL] Re-enable Arrow columnar input to Python UDFs, holding only vectors that are not reused#59582
viirya wants to merge 3 commits into
apache:masterfrom
viirya:SPARK-60325

Commits on Oct 18, 2026
- Add a marker for column vectors that their producer may release or overwrite
  
  viirya
  and
  Isaac
  committed
  bc2449a
- [SPARK-60306] Release the input batches of Python UDFs and non-codegen ColumnarToRow
  
  viirya
  and
  Isaac
  committed
  dcdb133
- [SPARK-60325] Hold only unmarked vectors in the Arrow columnar path of Python UDFs
  
  viirya
  and
  Isaac
  committed
  897f39f