Test gaps in parameterization, serialization round-trips, and getPlatform

[fohara]

While mutation-testing this repo (I'm building a tool in this space), I ran the vitest suites of the 28 hermetic packages against 1,038 mutants. 866 were caught, which is strong. The 169 survivors are heavily clustered, and three files hold about a third of them:

1. packages/client-engine-runtime/src/parameterization/parameterize.ts (27 survivors): swapping &&/|| in the selection guards (e.g. :154, :224) and negating sel.selection (:158) all pass. The parameterization logic has broad line coverage but the tests mostly don't distinguish these branch outcomes.

2. packages/param-graph/src/serialization.ts (17 survivors): serialize and deserialize each have coverage, but mutants that corrupt one direction survive because no test round-trips a value through both and compares.

3. packages/get-platform/src/getPlatform.ts (15 survivors): the platform and libssl detection branches are permissive under mutation. Examples: negating platform === 'freebsd' (:76), flipping the libssl filename guards (:371, :378, :382), and inverting the linux distro check (:507) all pass. On CI runners these branches mostly collapse to one path, so the others go unobserved.

Smaller clusters: generator-helper/src/GeneratorProcess.ts (10) and adapter-mssql/src/connection-string.ts (9).

The round-trip suggestion for serialization.ts is probably the cheapest win: a single property-style test (serialize, deserialize, deep-equal) would kill most of that file's survivors at once.

Scope note: this covers only the hermetic packages. client, migrate, cli, and the engine-integration paths run under separate DB-backed suites that I did not execute, so nothing here speaks to those.

Happy to share the full survivor list or turn the round-trip test into a PR if that's welcome. No action needed otherwise, 83.7% with this operator set is comfortably above most of what I've measured!

[fohara]

Quick follow-up: I've published a broader writeup that includes Prisma's numbers alongside nine other projects. Happy to adjust how the project is characterized if anything reads wrong: https://fixture.dev/writing/we-mutation-tested-10-projects

[Sarvagya-24-chaturvedi]

Hi @fohara,

This is a fantastic analysis. An 83.7% mutation score on a large monorepo like Prisma is genuinely impressive, but your breakdown of why the remaining 169 mutants survived highlights the exact blind spots of traditional line coverage:

---

Feedback on the Specific Survivor Clusters

1. param-graph/src/serialization.ts (Round-Trip Testing):

   • This is definitely the highest-ROI fix. Unit tests that assert on individual serialized outputs often miss asymmetries.
   • A property-based test (e.g., using fast-check or Vitest with randomized graph structures) verifying deserialize(serialize(graph)) deepEquals graph would kill almost all 17 survivors in a single, clean test file.

2. get-platform/src/getPlatform.ts (CI Runner Bias):

   • The survival of the FreeBSD (:76) and Linux distro/libssl detection branches (:371-382, :507) makes complete sense given CI environments: runners execute on standard GitHub Actions Ubuntu/macOS/Windows runners, leaving esoteric OS/distro resolution branches unexercised.
   • To kill these, unit tests would need explicit mocks for os.platform(), os.release(), and /etc/os-release file reads to exercise alternate resolution trees without needing exotic runner VMs.

3. client-engine-runtime/src/parameterization/parameterize.ts (Branch Logic Guards):

   • Classic AST/visitor trap: line coverage shows the selection logic was executed, but because the test fixtures don't assert on edge-case query shapes where && vs || alters the parameter bindings, mutant branches pass silently. Adding targeted query permutations to the fixture suite will tighten these guards.

---

Opening a PR

A PR for the serialization.ts round-trip test would definitely be welcome!

• Since packages/param-graph is hermetic and runs quickly in Vitest, a PR titled test(param-graph): add round-trip serialization property tests is low-risk, self-contained, and easily reviewable by the team.
• If you have the survivor report exported (e.g., JSON / HTML), linking a gist or uploading it to a PR will give maintainers clear line numbers to target for the parameterization guards as well.

The benchmark writeup on fixture.dev was also a great read—looking forward to seeing the PR landed!