ci: test suite + GitHub Actions on the Nettle build#23
Open
matthew-levan wants to merge 27 commits into
Open
Conversation
Brings the test suite (tests/, make check) onto the OpenSSL->Nettle base. Conflicts resolved in favour of jb/close-ssl's Nettle + vendored aes_siv/ implementation; the test wiring in Makefile.am is rewritten to link Nettle and the vendored libaes_siv.la (dropping the OpenSSL LIBCRYPTO_* / external -laes_siv references and the /usr/local rpath). configure.ac keeps the nettle >= 4.0 / secp256k1 recovery+Schnorr checks; README dependency and install notes updated for Nettle + building secp256k1 with the required modules. Verified: all 11 suites (198 checks) pass against nettle 4.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the stale OpenSSL-era CI (libssl-dev + cloning libaes_siv) with a workflow matching the current dependencies: - Triggers on push/PR to jb/close-ssl and master. - build & test matrix on ubuntu-latest and macos-latest. - A reusable composite action (.github/actions/setup-crypto-deps) builds the two dependencies the runners' package managers can't provide and caches them: nettle 4.0 (needed for the 2-argument *_digest() interface) from ftp.gnu.org, and libsecp256k1 v0.7.1 with the recovery + Schnorr modules. - An AddressSanitizer + UBSan job, and a non-blocking cppcheck job over urcrypt/ and aes_siv/. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vendored libscrypt (b64.c, crypto_scrypt-nosse.c, sha256.c) uses K&R-style
function definitions, which C23 removes. Recent Apple clang on the macos-latest
runner defaults to C23 and rejected them ("unknown type name 'src'"), while
gcc and older clang (defaulting to gnu17) compiled fine. Pin the scrypt library
to -std=gnu17 so the build is deterministic across compilers.
The ASan+UBSan job failed on a benign UBSan finding in the vendored ed25519 reference code (ed25519/src/fe.c:721: "left shift of negative value"), which is exercised via its own test vectors and is not urcrypt's code to fix. Build the sanitizer job with clang and pass an -fsanitize-ignorelist (a clang feature gcc lacks) that excludes the vendored libraries from UBSan, mirroring the cppcheck scope. AddressSanitizer still instruments everything. Verified locally: without the ignore list the ed25519 shift error reproduces and `make check` fails; with it, all 11 suites (198 checks) pass clean.
- Nettle is now cloned from the gnutls/nettle GitHub mirror at the 4.0 release tag (nettle_4.0_release_20260205) and bootstrapped (./.bootstrap runs autoconf, since the mirror ships no generated configure) instead of pulling the ftp.gnu.org tarball. jb/close-ssl only requires nettle >= 4.0 with no pinned version, and 4.0 is the sole 4.x release, so this matches. - On Linux the dependencies (nettle, secp256k1) are built as static libraries with -fPIC, and urcrypt is configured --enable-static --disable-shared, so the test runner links them statically -- matching how urbit/vere links urcrypt statically. macOS keeps shared/dynamic linking (it can't fully static-link). - README updated: build nettle 4.0 from the mirror, point configure at it, and run the tests. Verified locally: a static build (static nettle + static secp256k1, urcrypt --enable-static --disable-shared) passes all 11 suites (198 checks), and otool -L confirms no dynamic nettle/secp linkage in the test runner.
Switch nettle back from the gnutls/nettle GitHub mirror to the canonical ftp.gnu.org release tarball, pinned and verified by sha256 (3addbc00...f5094 for nettle-4.0.tar.gz). Rationale: the GNU tarball is the official, GPG-signed release artifact and ships a generated configure (no ./.bootstrap / autoconf needed). The GitHub mirror is a third-party mirror whose auto-generated archive checksums are not guaranteed byte-stable, making a hash pin fragile. For a dependency this security-sensitive, a checksum-pinned signed tarball is the stronger supply chain. The sha256 is verified before the tarball is used; a mismatch aborts. README updated to match (build nettle 4.0 from the tarball).
Closed
joemfb
reviewed
Jun 4, 2026
Comment on lines
+5
to
+7
| branches: [jb/close-ssl, master] | ||
| pull_request: | ||
| branches: [jb/close-ssl, master] |
Collaborator
There was a problem hiding this comment.
Let's get my branch out of here.
Collaborator
Author
There was a problem hiding this comment.
Including jb/close-ssl temporarily is necessary for CI to run. Once this PR is merged to master, we can remove it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the
ml/teststest suite onto thejb/close-ssl(OpenSSL→Nettle + vendored aes-siv) base and replaces the stale CI with one that matches the current dependencies.What's here
Merge
jb/close-sslintoml/tests— conflicts resolved in favour of the Nettle / vendoredaes_siv/implementation.Makefile.am'stest_runnernow links Nettle + the vendoredlibaes_siv.la(dropping the OpenSSLLIBCRYPTO_*, external-laes_siv, and/usr/localrpath). README install notes updated.New CI (
.github/workflows/ci.yml):jb/close-sslandmaster.ubuntu-latest+macos-latest.urcrypt/+aes_siv/(non-blocking)..github/actions/setup-crypto-deps— composite action that builds & caches the two dependencies the runners' package managers can't provide:--enable-mini-gmp),--enable-module-recovery --enable-module-schnorrsig --enable-module-ecdh.Heads-up: requires nettle 4.0
jb/close-sslcalls the 2-argument*_digest()interface and relies on deprecated symbols (e.g.struct aes_ctx) being gone — both are nettle 4.0 changes (released 2026-02-05). It does not compile against nettle 3.x. Homebrew ships 3.10.2 and Ubuntu ships 3.7–3.9, so./configurefails on stock installs; CI therefore builds nettle 4.0 from source.Verification
Built nettle 4.0 from source locally and ran
make check: all 11 suites, 198/198 checks pass.Tests and test vectors
The suite (
tests/, 11 modules, 198 checks) uses a small custom harness (tests/test_common.h) run as a singletest_runnerviamake check. The vectors are, wherever a standard publishes them, the official reference/standards vectors — eachtest_*.ccites its source in the file header:test_vectors.jsontest.c)js-ethereum-cryptographyvectors (Keccak 0x01 padding, distinct from NIST SHA-3/FIPS 202)The one exception is
test_ge_additions.c(extra Ed25519 group-element operations likescalarmult): these have no single published vector set, so they are structural/round-trip and algebraic-identity checks built on RFC 8032 scalars rather than canonical vectors.CI issues found and fixed
scrypt/b64.cK&R errors. Themacos-latestrunner's clang defaults to C23, which removes K&R function definitions; vendored libscrypt still uses them. Fixed by pinning the scrypt library to-std=gnu17(Makefile.am). Reproduced and verified locally by forcing a C23 build.ed25519/src/fe.c:721left shift of negative value. A benign UBSan finding in the vendored ed25519 reference code (validated by its own test vectors), not urcrypt's code. Fixed by building the sanitizer job with clang and an-fsanitize-ignorelist(ci/sanitizer-ignorelist.txt) that excludes the vendored libraries from UBSan while keeping AddressSanitizer on everything — mirroring the cppcheck scope. Verified locally that the error reproduces without the list and disappears with it.Dependency sourcing
Nettle is built from the canonical ftp.gnu.org release tarball (
nettle-4.0.tar.gz), pinned and verified by sha256 (3addbc00...f5094) before use. secp256k1 is built from its pinned git tag (v0.7.1).We deliberately use the GNU tarball rather than the
gnutls/nettleGitHub mirror:configure(no./.bootstrap/autoconf step).For a dependency this security-sensitive, a checksum-pinned signed tarball is the stronger supply chain. The
nettle-version/nettle-sha256pair lives at the top of the workflow, so bumping the version is a one-line change plus the new hash.🤖 Generated with Claude Code