fix: resolve LLVM toolchain install failure on Linux#61
Open
Sunrisepeak wants to merge 12 commits into
Open
Conversation
Three fixes for `mcpp toolchain install llvm` failing with "xpkg payload missing": 1. install_with_progress(): use direct `xlings install -y` command on ALL platforms (not just Windows). The direct command avoids stdin closure (</dev/null) that breaks xlings subprocess coordination for large packages like LLVM (~800MB). Falls back to NDJSON interface path if direct install fails. 2. package_fetcher.cppm: extend the global xlings directory fallback from Windows-only to all platforms. If xlings installs a package to ~/.xlings/ instead of the mcpp sandbox, detect and copy it. 3. ci.yml: add "Toolchain install smoke test" step that exercises `mcpp toolchain install llvm` + build with it. This core user flow was previously untested in CI.
Also replace raw WIFEXITED/WEXITSTATUS with platform::process::extract_exit_code() in xlings.cppm.
The added import changed the module dependency graph fingerprint, invalidating BMI cache on macOS CI and exposing a pre-existing Xcode 16.4 SDK incompatibility during std module precompilation. Use #if defined(_WIN32) instead of if constexpr for the USERPROFILE check to avoid changing the module import set.
macOS CI was missing the MCPP_HOME job-level env var that Linux CI has. The freshly-built mcpp resolved to a fresh ~/.mcpp/ sandbox, triggering a clean std module precompile that exposed a pre-existing Xcode 16.4 + LLVM 20.1.7 sysroot incompatibility. Fix: set MCPP_HOME=/Users/runner/.mcpp (consistent with Linux CI) and export MCPP_VENDORED_XLINGS in the test step so the sandbox reuses the pre-installed xlings binary.
Root cause: probe_sysroot() falls back to xcrun --show-sdk-path on macOS when the compiler doesn't report a sysroot via -print-sysroot. For xlings-installed LLVM 20.1.7, this sets --sysroot to the Xcode SDK path. Combined with -nostdinc++ from clang++.cfg, this breaks C runtime header resolution — macOS SDK headers reference internal macros (_CTYPE_A, etc.) that are only defined via default include paths which --sysroot overrides. Fix: remove the xcrun SDK fallback in probe_sysroot(). If the compiler itself doesn't report a sysroot, none should be used. The xcrun fallback was designed for Apple's system clang, not for standalone xlings LLVM which provides its own libc++ headers. Also revert the ci-macos.yml MCPP_HOME workaround — the real bug is fixed, no workaround needed.
probe_sysroot() called -print-sysroot first, which on macOS with xlings LLVM just echoed back the --sysroot from clang++.cfg. That cfg-baked path points to CommandLineTools SDK, which may differ from the active Xcode SDK. When the two SDKs have different header versions, std module precompilation fails with undeclared _CTYPE_A and related C runtime macro errors. Fix: on macOS, probe xcrun --show-sdk-path FIRST (always returns the active SDK), then fall back to -print-sysroot for non-macOS. The xcrun sysroot passed on the command line overrides the cfg-baked one.
Root cause: on macOS, xlings LLVM's clang++.cfg already contains a --sysroot for the macOS SDK. When mcpp additionally passes --sysroot on the command line during std module precompilation, it changes the C header include ordering. The macOS SDK's ___wctype.h references _CTYPE_A (defined in _ctype.h), but the duplicate --sysroot flag prevents _ctype.h from being included transitively during module purview compilation, causing "undeclared identifier '_CTYPE_A'" errors. Fix: detect apple/darwin target triple in stdmod.cppm and skip the --sysroot flag for std module precompile. The cfg's own --sysroot handles macOS SDK discovery correctly for this compilation mode. Regular compilation (flags.cppm) and linking still use the probed sysroot as before. Also restore probe_sysroot() to its original logic (xcrun fallback) since the issue was in stdmod.cppm passing sysroot, not in probing it.
The previous fix skipped --sysroot entirely on macOS, but xlings LLVM's clang++.cfg still applies its own --sysroot pointing to CommandLineTools SDK. When the active SDK is Xcode (different path), the cfg's stale sysroot causes _CTYPE_A undeclared errors. Fix: on macOS, always probe xcrun and pass the active SDK as --sysroot to OVERRIDE the cfg-baked stale path. This ensures the std module precompile uses the correct SDK regardless of what was baked into clang++.cfg at LLVM install time.
CI step 9 proves that clang++ WITHOUT explicit --sysroot precompiles std.cppm correctly on macOS — the clang++.cfg's built-in --sysroot and -isystem flags handle SDK header resolution properly. When mcpp passes an explicit --sysroot (even the identical value), it changes Clang's internal header search order, breaking the transitive inclusion of _ctype.h before ___wctype.h. The macOS SDK's ___wctype.h references _CTYPE_A which is only defined via the default cfg-driven include chain. Fix: detect apple/darwin target and skip --sysroot for std module precompile, letting clang++.cfg handle it. Non-macOS platforms (Linux, Windows) continue to use the probed sysroot as before.
When xlings copies LLVM to mcpp's sandbox, clang++.cfg retains hardcoded absolute paths from the original install (--sysroot and -isystem pointing to ~/.xlings/... instead of ~/.mcpp/...). These stale paths cause _CTYPE_A undeclared errors during std module precompilation. Fix: on macOS Clang, pass --no-default-config to ignore the stale cfg, then explicitly provide the correct -isystem (libc++ headers from the sandbox LLVM root) and --sysroot (active SDK from xcrun). This produces the same header search behavior as a fresh clang++ invocation with the correct paths.
The stale clang++.cfg issue affects not just std module precompile but also regular compilation via ninja. flags.cppm now applies the same --no-default-config + correct -isystem + xcrun --sysroot fix for all macOS Clang compilation, bypassing the cfg's stale paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
xlings install -yon all platforms (not just Windows) for reliable large package installationmcpp toolchain install llvm+ buildProblem
mcpp toolchain install llvmfails with "xpkg payload missing" on Linux. Dependencies (libxml2, zlib) install fine but LLVM (~800MB) doesn't. Root cause:</dev/nullstdin closure and NDJSON interface mode break xlings subprocess coordination for large packages.Test plan