[mindlace]

Bash treats \xHH and \NNN in $'...' as raw bytes, so multi-byte UTF-8 written as escapes decodes to a single character. decodeAnsiCQuoted mapped each byte to its own code point instead:

Source	Bash output bytes	Word.value before	after
$'\xc3\xa9'	c3 a9	"é"	"é"
$'\303\251'	c3 a9	"é"	"é"
$'\xe2\x82\xac'	e2 82 ac	"â\x82¬"	"€"
$'\xf0\x9f\x98\x80'	f0 9f 98 80	"ð\x9f\x98\x80"	"😀"

Change

Runs of \x and octal escapes are collected as bytes and decoded with TextDecoder when the run ends, so hex and octal can mix ($'\xc3\251').

Behavior change for malformed bytes: a byte outside a well-formed UTF-8 sequence now becomes U+FFFD, following the WHATWG and browser rule (one replacement per maximal subpart). Before, it became the code point with the same number, which made it look like a real character: $'\xc3' and $'\xc3\x83' both decoded to "Ã", though only the second is Ã in any Unicode encoding. A JS string can't hold the raw byte, so value can't be lossless here, but U+FFFD at least can't be mistaken for real text.

Source	Bash output bytes	before	after
$'\xc3'	c3	"Ã"	"�"
$'\777'	ff	"ÿ"	"�"
$'\xe2\x82'	e2 82	"â\x82"	"�"

The existing $'\777' expectation in test/word-parts.test.ts is updated to match.

TextDecoder is global since Node 11, and engines already requires Node ≥14.

Testing

• Added tests in test/word-parts.test.ts for valid sequences (expected values from bash 5.3) and for malformed ones (expected values from TextDecoder).
• Ran 3000 random byte-escape strings through parse() and through bash. The output matched on every case whose bytes are valid UTF-8.
• pnpm test:types and pnpm lint are clean. pnpm test passes except benchmark configuration is edited in the script instead of parsed from CLI flags, which also fails on main without this change.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/unbash@16

commit: 7bb72fe

[webpro]

Thanks for the fix, Ethan! Decoding byte runs as UTF-8 with U+FFFD for malformed bytes looks right, and heredoc delimiters such as $'\xc3\xa9' now close on their é line as well.

I pushed one commit on top with two changes:

• TextDecoder strips a leading byte order mark by default, and each run is decoded on its own, so $'\xef\xbb\xbf' became an empty string and sed $'1s/^\xef\xbb\xbf//' lost its BOM. The decoder now passes ignoreBOM: true.
• The lexer decodes ANSI-C quotes while scanning, so a decoder call per byte run roughly doubled parse time for scripts full of escapes like \033 or \x1b. ASCII bytes now skip the decoder.

It also adds tests for the byte order mark and the heredoc delimiter.

[webpro]

🚀 This pull request is included in v6.0.1. See Release 6.0.1 for release notes.