Skip to content

Modernize for Go 1.26 and fix crashes on argument-less queries - #86

Merged
anacrolix merged 18 commits into
anacrolix:masterfrom
ShivamB25:modernize-refactor
Sep 24, 2026
Merged

anacrolix merged 18 commits into
anacrolix:masterfrom
ShivamB25:modernize-refactor

Conversation

@ShivamB25

@ShivamB25 ShivamB25 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Modernizes the codebase for Go 1.26 and fixes a remotely triggerable crash, among other bugs. The commits can be reviewed separately.

Advisor residual fixes (fd77e0b)

CI run 35857753915 passed on this exact SHA: Go 1.26, stable, and lint.

  • query and ping-nodes now use the same context-aware resolver as ping. Controlled DNS regressions reproduced that both commands ignored SIGINT before the fix; both now stop within the bounded cancellation deadline.
  • NodeAddr.MarshalBinary rejects IP lengths other than 4/16 bytes, matching its 6/18-byte decoder contract. Compact IPv4 lists also return an error for IPv6/nil input rather than panicking in the generic slice marshaler.
  • Resolver compatibility tests cover IPv4, IPv6, zones, wildcard addresses, address-family mismatches, and rejection of tcp for numeric and hostname inputs.
  • Added the negative announce-port boundary while preserving implied-port override behavior.
  • Focused regressions passed five race runs; go test -race -shuffle=on -count=2 -timeout 120s ./..., native/Windows vet, and lint (0 issues) passed.
  • Torrent 76452a2, using the local DHT replacement at fd77e0b, passed GOWORK=off go build ./... and GOWORK=off go test -short -count=1 -timeout 120s . ./tracker/....

Adversarial edge-case review (265bf4a)

CI run 35856680846 completed successfully on this exact commit: Go 1.26, stable, and lint.

Six independent review tracks covered server boundaries, traversal/routing, BEP44 storage, CLI lifetimes, test quality, and wire/getput behavior. Four reviewers returned final reports; partial server/store findings were checked directly. Findings were checked sequentially against contracts and targeted reproductions rather than applied on recommendation alone.

Confirmed corrections

  • Query each endpoint once even when queued candidates supply different or initially unknown IDs. Different endpoints sharing an ID remain eligible.
  • Reject compact peer entries other than 6-byte IPv4 or 18-byte IPv6, and reject BEP44 results with omitted v; a legitimate bencoded empty string remains valid.
  • getput.Put now returns an error when no token-eligible node accepts the write, preserving remote protocol errors. A successful remote write remains success.
  • Reject missing/out-of-range announce ports, preserving implied-port precedence. Cancel pending rate-limited replies when the server closes, and stop before socket writes when a blocklist callback closes the server.
  • Validate mutable keys before seed/array conversion, handle missing return dictionaries in ping-nodes, report KRPC errors in ping output, and prevent automatic sequence wrap at MaxInt64.
  • Ping's timeout now includes DNS lookup; hostname resolution uses the caller context while numeric addresses retain the standard resolver's address-family, zone, wildcard, and port semantics.

Test-quality corrections

  • The original LoadStats concurrency test timed out after 120 seconds with one scheduler. Both workloads are now finite and cooperative, with monotonic snapshot and exact final-count assertions.
  • Callback fixture cleanup now marks shutdown before raw socket closure, avoiding an asynchronous panic on an early test exit. Callback tests also confirm that the specific logger/store/Ranger/marshaler path actually ran.
  • Removed a Go-language defer test, an unsynchronized external rate-limiter timer test, and the refreshBucket test that could not observe the claimed stats-ordering defect.
  • TestHook now uses ephemeral ports; it passed with the old hard-coded UDP port deliberately occupied. Removed the invalid nil-IP compact-address success expectation.

Evidence on this production tree

  • GOMAXPROCS=1 go test -race -shuffle=on -count=5 -timeout 120s ./... passed; the corresponding initial run reproduced the 120-second LoadStats test timeout.
  • go test -race -shuffle=on -count=2 -timeout 120s ./... passed. Later test-only cleanup was checked with focused repeated race runs.
  • Native and Windows go vet ./... passed; golangci-lint reports 0 issues; git diff --check passed.
  • Corrected KRPC fuzz targets passed 30 seconds each: general message fuzzing 571,105 executions, compact-address fuzzing 606,625 executions.
  • Actual CLI smoke checks: a valid 32-byte mutable public key produced the expected SHA-1 target; a missing mutable key returned a usage error without panic.
  • Torrent 76452a2 with the local DHT replacement passed GOWORK=off go build ./... and GOWORK=off go test -short -count=1 -timeout 120s . ./tracker/... again.

The proposed stale-Stalled race did not reproduce in repeated bounded tests; no signaling rewrite or ineffective test was kept. This is not a proof that every possible schedule is safe. The synchronous StartingNodes callback still cannot be forcibly interrupted by a context; a blocking callback can delay bootstrap cancellation. Explicit CAS-zero presence remains the separately documented public-API limitation.

Research used Codegraph and Firecrawl Developer Index, with primary contracts checked in BEP 32, BEP 44, rate.WaitN, and Go resolver/Ed25519 documentation.

Bug fixes

  • Remote crash: announce_peer, put and get queries without an a (arguments) dict caused a nil pointer dereference, so any single UDP packet could take down the server. These queries now get a KRPC protocol error, as get_peers and find_node already did.
  • find_node and get replies returned the nodes closest to the unset info_hash instead of the queried target.
  • Incoming BEP 44 immutable puts were rejected for lacking seq, which only mutable items carry. seq is now required only for mutable puts.
  • Server.Put ignored its QueryRateLimiting argument.
  • BootstrapContext leaked a running traversal goroutine when TraversalStartingNodes failed. BootstrapContext, AnnounceTraversal and exts/getput now get starting nodes before starting a traversal.
  • refreshBucket read traversal stats in its return statement, which Go evaluates before deferred functions run, so Operation.Stats was called before Stopped. The stats are now assigned after the traversal stops.
  • Announce.TraversalStats copied the counters with a plain struct read while queries update them with atomic adds. LoadStats reads both fields atomically, and the announce, bootstrap, and get/put snapshots use it.
  • getPeers waited on traversal.Stopped while that query was still outstanding, so Close with nobody reading Peers never reached Finished. The send now also selects on the query context, which Stop cancels.
  • A cancelled BootstrapContext returned while its find_node queries kept running until they timed out. Bootstrap and bucket-refresh queries now use the traversal's context, and BootstrapContext waits for the traversal to stop before returning its stats.
  • exts/getput: fixed a nil seq dereference on unsigned responses, stopped ignoring errors from TraversalStartingNodes, and now waits for the traversal to stop before reading its stats.
  • krpc.Error decoding no longer uses recover to reject malformed errors, and NodeInfo decoding checks its input length.
  • Server.Close did not cancel an in-flight query, so Ping waited out QueryResendDelay after the server closed. Query now ends when the server context is cancelled or the caller's context ends.
  • table.closestNodes now ranks eligible nodes from all routing buckets by XOR distance to the requested target, then truncates to K. Buckets are relative to the local node, not the target; sorting a bucket prefix was insufficient. Deterministic regressions cover omitted higher-index buckets, filtering, and nonpositive limits.
  • Operation.Closest returned the live set. A response could replace it while a caller ranged it. It now returns a snapshot taken under the operation lock.
  • SetIPBlockList wrote the list under s.mu while IPBlocklist and TraversalNodeFilter read it with no lock. The list is published with an atomic pointer, not s.mu, because writeToNode already holds that lock.
  • BEP 44 nonzero cas is compared with the stored sequence, before equal/stale sequence handling, so a mismatch returns 301 rather than succeeding or returning 302. Wrapper.Put serializes read/check/write; concurrent CAS tests assert one accepted update and the stored winner. Memory.Put computes immutable targets before taking its mutex, permitting a value marshaler to call Memory.Get. Explicit cas=0 presence remains an existing API limitation; see boundaries below.
  • cmd/dht ping dereferenced SenderID() when a reply had no id. dht-server returned a table-load error without Server.Close, and the deferred socket close panicked serveUntilClosed.

Regression tests cover argument-less queries, immutable puts without seq, the find_node target, the traversal goroutine leaks, and bootstrap cancellation. TestLoadStatsDuringUpdates waits until the writer has incremented, then races atomic updates against LoadStats. TestAnnounceCloseWithoutReadingPeers answers one get_peers, does not read Peers, and requires Finished after Close. With the old select it failed: Finished did not fire. TestRefreshBucketStopsBeforeStats only checks that the refresh traversal is finished when refreshBucket returns; it does not prove the stats were unread before Stopped. There are also new int160 tests.

Modernization

  • go 1.26.0, with dependencies upgraded to their latest releases.
  • Removed the direct dependencies on anacrolix/multiless, bradfitz/iter, docopt-go and golang.org/x/exp, replacing them with cmp, slices, maps, math/rand/v2, errors.AsType, sync.OnceValue, WaitGroup.Go, atomic.Uint64, binary.Append* and range over int.
  • Split handleQuery into one handler per query method.
  • int160.BitLen uses math/bits instead of big.Int, and the Castagnoli CRC table is cached for the node ID security checks.
  • Removed dead code, unused fields, commented-out code and stale comments. Test-only helpers moved into _test.go files.

CI

  • Tests run on Go 1.26 (the go.mod minimum) and the latest stable release, using current action versions, with go vet, the race detector and a one-pass benchmark run.
  • Added a golangci-lint job pinned to v2.13.2 (.golangci.yml), taking its Go version from go.mod. errcheck is not excluded for tests; the few ignored errors in dht_test.go are now checked.

Follow-through on the concurrency audit

  • Configurable OnQuery, Logger, PeerStore, Store, Ranger, response marshalers, and status writers no longer run under the server mutex on the exercised paths. Table updates remain locked, mutation/veto behavior is preserved, and handlers recheck closure after user callbacks. Query metrics snapshot values before their goroutine starts, avoiding a race with OnQuery mutations.
  • CLI ping cancels and joins workers on timeout, resolver error, and parent cancellation. Get/put commands receive the signal context. The main command closes its server before exiting; dht-server joins bootstrap before persisting the routing table.
  • Announce/getput lifecycle tests now observe actual packets/results before cancellation. The malformed response in TestBadGetPeersResponse deliberately retains the missing response type required by PR #19.
  • Removed the sleep/overlap assertion that pinned wrapper locking internals. Replaced it with observable CAS acceptance and stored-value assertions.

Advisor follow-through (651c25d)

CI run 35850583783 passed on this exact commit: Go 1.26, stable, and lint.

  • Nearest-K coverage now compares exact node IDs and addresses against a full eligible-candidate oracle using independent bytewise XOR. Membership and monotonic-distance assertions are separate, so a sorted but wrong subset fails.
  • The malformed get_peers responder has a read deadline and bounded goroutine-exit cleanup. The intentionally missing response type from PR Don't announce peer if the sender ID is nil #19 remains unchanged.
  • Existing bounded logger/Ranger/OnQuery/Memory reentry tests, bootstrap join-before-save, outer CLI server lifetime, signal propagation, and CAS error-precedence checks remain in place. No published commits were rewritten.
  • Ten focused race runs passed. go test -race -shuffle=on -count=2 -timeout 120s ./..., native and Windows vet, and golangci-lint (0 issues) passed on this test-only follow-up.

Earlier verification

Earlier tree committed as 6bccb09:

GitHub Actions run 35847431455 completed successfully on that exact SHA: test (1.26), test (stable), and lint.

  • go test -race -shuffle=on -count=2 -timeout 120s ./... passed.
  • go vet ./... and GOOS=windows go vet ./... passed.
  • golangci-lint run ./...: 0 issues.
  • Focused callback, CAS, routing, and command regressions passed under -race; bootstrap join passed five repeated runs. CLI --help and derive-put-target immutable --string hello were exercised; the latter matched SHA-1 of bencoded hello.
  • Before fixes: higher-bucket selection omitted an eligible node; reentrant callbacks/status output/store access/blocklist lookup/marshaling deadlocked; ping timeout and resolver-error return left one active transaction; bootstrap returned before its worker completed; direct Memory marshaler reentry deadlocked; stale CAS returned success or 302 instead of 301.
  • The first CI run for a28a2e2 exposed shared-rate-limiter interference in an older bootstrap test. An exhausted-limiter reproduction failed with the same nil-error result. The isolated, packet-synchronized fixture then passed ten race runs; the temporary reproduction was removed.

Downstream verification was rerun against the production changes in a28a2e2 (the final commit changes test fixtures only): torrent 76452a2, with a local module replacement, passed GOWORK=off go build ./... and GOWORK=off go test -count=1 -short -timeout 120s . ./tracker/....

The apidiff comparison remains from 1135f58 against a669bdd, not the current head. It reported removal of Ordering from cmd/dht (package main). No new API-diff claim is made. The full torrent suite was not run; the older downstream vet attempt stopped on torrent's unkeyed composite literals.

Contract boundaries

  • Public BEP44/KRPC CAS fields remain int64; the wire field uses omitempty. Explicit cas=0 cannot currently be distinguished from omission. This is a representation limitation, not a BEP 44 rule. The PR preserves existing public API types rather than claiming complete zero-CAS support.
  • Receive callbacks remain synchronous. They can call locking accessors/Close without the demonstrated mutex cycles, but should not synchronously wait for another network query whose response needs the same receive loop. Slow callbacks still delay their caller.
  • The Memory reentry regression covers Memory.Put → value marshaler → Memory.Get; it does not promise arbitrary reentry into the serializing Wrapper from its Store/marshaler callbacks.

Primary contracts checked with Codegraph, Firecrawl Developer Index, and the specifications: BEP 5, BEP 44, and Go's context.AfterFunc documentation.

Require Go 1.26 and upgrade direct dependencies to their latest releases. Drops
the direct dependencies on anacrolix/multiless, bradfitz/iter, docopt-go and
golang.org/x/exp in favour of the standard library.
Test on Go 1.26, the minimum declared in go.mod, and on the latest stable
release with current action versions. Run go vet, and add a golangci-lint job
that uses the go.mod Go version.
Bug fixes:
- Queries that require arguments (announce_peer, put, get) no longer
  dereference a nil arguments dict, which let any remote packet crash the
  server. They now get a protocol error, as get_peers and find_node already
  did.
- find_node and get replies returned the nodes closest to the (unset)
  info_hash instead of the queried target.
- Server.Put ignored its QueryRateLimiting argument.
- getput.Get and getput.Put dereferenced a nil seq in unsigned responses,
  ignored errors from TraversalStartingNodes, and didn't wait for the
  traversal to stop before reading its stats.
- krpc.Error decoding no longer relies on recover to reject malformed
  errors, and NodeInfo decoding checks its input length.

Refactors:
- Split handleQuery into a handler per query method.
- Use the standard library in place of multiless, bradfitz/iter, docopt,
  x/exp/constraints and ioutil: cmp, slices, maps, math/rand/v2,
  errors.AsType, sync.OnceValue, WaitGroup.Go, atomic.Uint64, binary.Append*,
  range over int.
- Replace the big.Int in int160.BitLen with math/bits, and cache the
  Castagnoli CRC table used for node ID security checks.
- Remove dead code, unused fields, commented-out code, and stale or
  editorializing comments. Move test-only helpers into _test.go files.
- Rename k-nearest-nodes/k-nearest-nodes.go.go.

Adds regression tests for the argument-less queries and the find_node target,
plus int160 tests.
BootstrapContext started a traversal and then returned without stopping it
when TraversalStartingNodes failed, leaving the operation's run goroutine
blocked forever. Fetch starting nodes first in Bootstrap, AnnounceTraversal
and getput so the error paths never start a traversal, and add tests that
check no traversal goroutine survives those failures.
Drop the blanket errcheck exclusion for _test.go. The only findings were
in dht_test.go; they now assert or return on error. TestBootstrapRace
unmarshalled the whole read buffer rather than the bytes read.
BootstrapContext returned as soon as its context was done, leaving
outstanding find_node queries running until they timed out. Bootstrap
and bucket refresh queries now use the traversal's context so stopping
cancels them, and BootstrapContext always waits for the traversal to
stop, returning its stats along with any context error.
Go evaluates a return statement's results before deferred functions run, so
refreshBucket called Operation.Stats before Stopped. The named result is now
assigned after the traversal stops, and a test checks the traversal goroutine
is gone when refresh returns.
Announce.TraversalStats copied the stats struct while queries update the
same fields with atomic adds, which races. LoadStats reads both counters
atomically, and the announce, bootstrap, and get/put snapshots use it.
getPeers waited on traversal.Stopped while that query was still outstanding,
so Close never reached Finished if nobody received from Peers. The send now
also selects on the query context, which Stop cancels. The stats race test
waits for the writer to increment before it loads.
TestAnnounceCloseWithoutReadingPeers only called Close after the send was
blocked, so a failure before that left the traversal running.
Close did not cancel an in-flight query, so a ping that had already been sent waited out the resend delay. Query now ends when the server context is cancelled or the caller context ends.

closestNodes kept an arbitrary prefix of a bucket. It keeps the nearest nodes by XOR distance. Operation.Closest raced query updates and now returns a snapshot taken under the operation lock. The block list was written under the server lock and read without it, so it is published atomically.

BEP 44 cas was compared with the previous write's cas instead of the stored sequence number. Wrapper.Put holds one lock across that read and the write. ping no longer dereferences a response with no id, and dht-server closes the node when loading its table fails so the deferred socket close does not panic the serve loop.
Rank eligible nodes across the entire routing table. Release the server lock around configurable callbacks, store access, marshaling and status output, preserving veto and closed-state checks.

Cancel and join CLI queries and bootstrap before returning or saving state. Check mutable CAS before sequence handling and compute Memory targets outside its mutex. Replace implementation-pinning tests with reproduced behavioral regressions and deterministic lifecycle handshakes.
Wait for an actual bootstrap packet before cancellation and assert transaction cleanup. Give callback and cancellation fixtures their own limiter so unrelated tests cannot terminate their traversal through rate limiting.
Require complete compact addresses and present BEP44 values. Report errors when no remote put succeeds, preserving remote protocol errors. Reject invalid announce ports, cancel rate-limited replies on Close, and prevent writes after a blocklist callback closes the server.
Reject malformed mutable keys and node-list responses instead of panicking. Preserve numeric-address semantics while resolving hostnames with the command context. Surface KRPC errors and prevent automatic sequence overflow.
Skip queued aliases for an already queried endpoint. Bound and yield both stats-test workloads, make callback fixture failure cleanup safe, assert callback reachability, remove vacuous external-behavior tests, and use ephemeral ports.
Use the signal context for query and ping-nodes hostname resolution. Reject invalid generic and IPv4-family compact address encodings instead of emitting bytes the decoder refuses, and cover negative announce ports.
@anacrolix
anacrolix merged commit a9c967f into anacrolix:master Sep 24, 2026
3 checks passed
@ShivamB25
ShivamB25 deleted the modernize-refactor branch September 25, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants