Skip to content

Fix NeoBinary serialization of DataSets and DataTables larger than 4 KB - #251

Merged
theRainbird merged 1 commit into
theRainbird:masterfrom
danio3xl3:fix/NeoBinaryLargeDataSetSerialization
Nov 17, 2026
Merged

theRainbird merged 1 commit into
theRainbird:masterfrom
danio3xl3:fix/NeoBinaryLargeDataSetSerialization

Conversation

@danio3xl3

Copy link
Copy Markdown

Problem

NeoBinarySerializer.SerializeDataSet and SerializeDataTable (XML mode, the default) write the XML schema and the DiffGram into a MemoryStream that wraps a fixed 4096 byte buffer rented from the ArrayPool. Such a stream cannot grow. Every DataSet or DataTable whose XML is larger than 4 KB (roughly 10 to 30 rows) therefore fails:

InvalidOperationException: Failed to serialize object of type 'System.Data.DataSet'
 ---> NotSupportedException: Memory stream is not expandable.
    at System.IO.MemoryStream.set_Capacity(Int32 value)
    ...
    at System.Data.DataSet.WriteXml(XmlWriter writer, XmlWriteMode mode)
    at CoreRemoting.Serialization.NeoBinary.NeoBinarySerializer.SerializeDataSet(...)

Fix

The schema and the DiffGram are written into StringWriter instances instead. They grow as needed, match the StringReader that the deserialization side already uses, and the XML no longer takes a detour through UTF-8 bytes. The serialization is moved into one helper, WriteXmlSchemaAndDiffGram, that is shared by the DataSet and the DataTable path. The deserialization code is not changed. The data is still written as two length-prefixed strings.

Tests

Two regression tests with 2000 rows (DataSet with a relation, a primary key, a DBNull default value, a modified row and multi-byte characters; DataTable with a modified row). Both fail on master with the exception above and pass with this change. The serialization, DataSet and NeoBinary tests pass on net8.0.

SerializeDataSet and SerializeDataTable wrote the XML schema and the DiffGram
into a MemoryStream that wrapped a fixed 4096 byte buffer rented from the
ArrayPool. Such a stream cannot grow, so any DataSet or DataTable whose XML
exceeded 4 KB failed with "NotSupportedException: Memory stream is not
expandable".

Write the XML into StringWriter instances instead. They grow as needed, match
the StringReader used on the deserialization side and avoid the UTF-8 round
trip. The wire format stays a length-prefixed string.

Add regression tests with 2000 rows (relation, DBNull, modified row, multi-byte
characters) for DataSet and DataTable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@theRainbird
theRainbird self-requested a review November 17, 2026 21:50
@theRainbird
theRainbird merged commit bc7dc28 into theRainbird:master Nov 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants