dslx: parse semantic sum declarations and constructors

[dank-openai]

Summary

• Add DSLX frontend support for semantic-sum enum declarations with unit, tuple, and struct payload variants.
• Add semantic-sum constructor expressions and match payload patterns so the AST preserves the source payload shape.
• Teach formatting, cloning, LSP symbols, type-info plumbing, conversion scaffolding, and diagnostics about the new AST nodes while leaving full typechecking and lowering semantics for follow-up work.
• Preserve existing numeric-enum parsing behavior, including explicitly typed empty enums.

Example

enum Option<T: type> {
  None,
  Some(T),
}

This PR makes declarations and constructors like Option::Some(x) parse and format as semantic-sum syntax. Later changes can build typechecking and conversion semantics on top of these AST nodes.

Validation

Validation ran on Ubuntu 24.04 at commit 1f8ad4299fa61d5d35d2bbd4fe4780f79314f2d8:

• bazel test //xls/dslx/fmt:ast_fmt_test //xls/dslx/frontend:parser_test //xls/dslx/frontend:ast_cloner_test //xls/dslx/frontend:scanner_test //xls/dslx:dslx_fmt_test //xls/dslx:constexpr_evaluator_test //xls/dslx/bytecode:bytecode_emitter_test //xls/dslx/exhaustiveness:exhaustiveness_match_test //xls/dslx/lsp:document_symbols_test //xls/dslx/translators:dslx_to_verilog_test //xls/fuzzer:value_generator_test //xls/dslx/ir_convert:ir_converter_test //xls/dslx/ir_convert:ir_converter_legacy_test //xls/dslx/tests/errors:error_modules_test //xls/dslx/run_routines:run_routines_test //xls/dslx/type_system:type_info_to_proto_test //xls/dslx/type_system_v2:typecheck_module_v2_enum_test --test_output=errors --keep_going --test_timeout=900 --cache_test_results=no

Misc.

This is PR 1/7 for phase 1 implementation of sum types.

[richmckeever]

The usual convention in here is to use std::optional<T*> to indicate a nullable T* and force users of it to check if it has a value.

[richmckeever]

Can we make the condition for this more generic or defer the check to sometime later than parsing? Resolving references generally is not something the parser is well equipped for, and the direction is to be less equipped for it.

[richmckeever]

It seems like it would be less surprising to have no space between the empty braces on both sides here (and in the declaration of the enum).

[richmckeever]

A few observations about this:

• This class itself is an ad hoc sum type in C++(one for tuple + one for struct). It's an element in what is already a more standard sum type, NameDefTree::Leaf.
• It would be nice to foresee how we'd use struct patterns with regular structs. Having a struct pattern thing that is not married to sum types would solve that.
• This is actually a subtree, and breaks the concept of what is a "leaf" in NameDefTree. I've always thought NameDefTree was unintuitive anyway.

Maybe we should do away with the NameDefTree class and just define PatternTree = std::variant<TuplePattern*, StructPattern*, NameDef*, NameRef*, etc. from what is now Leaf>. We replace NameDefTree uses with PatternTree. We split your proposed SumVariantPayloadPattern into TuplePattern and StructPattern. TuplePattern's constructor ref is nullable. StructPattern's can later be a ColonRef | TRTA, and we can use it for regular structs then.