Skip to content

Avoid Metadata.key allocations on grpc#11351

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
masterfrom
andrea.marziali/grpcs
May 13, 2026
Merged

Avoid Metadata.key allocations on grpc#11351
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
masterfrom
andrea.marziali/grpcs

Conversation

@amarziali
Copy link
Copy Markdown
Contributor

@amarziali amarziali commented May 12, 2026 •
edited
Loading

What Does This Do

Avoid allocating a Metadata.Key allocation for each grpc call (it's on the hot path). The keys can be cached in a large enough DDCache

Motivation

Allocation histogram — instrumented vs baseline (JFR captures)

  byte[]                                          instr [██████████████████████]   54   base [████                  ]   10
  java.lang.Object[]                              instr [██████                ]   14   base [████                  ]   11
  java.util.HashMap$Node                          instr [███████               ]   17   base [                      ]    0
  java.lang.String                                instr [██████                ]   15   base [█                     ]    1
  java.util.HashMap$Node[]                        instr [████                  ]    9   base [█                     ]    1
  io.grpc.Metadata$AsciiKey                       instr [████                  ]    9   base [                      ]    0
  ..ChannelStreamProvider$1RetryStream            instr [██                    ]    5   base [█                     ]    2
  ..ConcurrentLinkedQueue$Node                    instr [█                     ]    3   base [█                     ]    3
  io.grpc.internal.ClientCallImpl                 instr [█                     ]    1   base [██                    ]    4
  io.grpc.internal.RetriableStream$State          instr [█                     ]    3   base [█                     ]    2

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@amarziali amarziali requested review from a team as code owners May 12, 2026 13:23
@amarziali amarziali added type: enhancement Enhancements and improvements tag: performance Performance related changes inst: grpc gRPC instrumentation labels May 12, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 12, 2026

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 13, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/grpcs
git_commit_date 1778583733 1778592116
git_commit_sha 710ca87 8e86faf
release_version 1.63.0-SNAPSHOT~710ca87583 1.63.0-SNAPSHOT~8e86faff09
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1778599921 1778599921
ci_job_id 1675876879 1675876879
ci_pipeline_id 112701082 112701082
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-e2ohwsik 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-e2ohwsik 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 11 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:iast:AgentMeter better
[-1469.419µs; -252.985µs] or [-11.709%; -2.016%]		 11.689ms 12.550ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.63.0-SNAPSHOT~8e86faff09, baseline=1.63.0-SNAPSHOT~710ca87583

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.094 s) : 0, 1093860
Total [baseline] (11.495 s) : 0, 11495194
Agent [candidate] (1.093 s) : 0, 1092869
Total [candidate] (11.458 s) : 0, 11457666
section appsec
Agent [baseline] (1.305 s) : 0, 1304679
Total [baseline] (11.219 s) : 0, 11218587
Agent [candidate] (1.303 s) : 0, 1303042
Total [candidate] (11.341 s) : 0, 11341118
section iast
Agent [baseline] (1.283 s) : 0, 1282799
Total [baseline] (11.362 s) : 0, 11361597
Agent [candidate] (1.28 s) : 0, 1279934
Total [candidate] (11.313 s) : 0, 11313338
section profiling
Agent [baseline] (1.351 s) : 0, 1351379
Total [baseline] (11.474 s) : 0, 11473679
Agent [candidate] (1.345 s) : 0, 1345472
Total [candidate] (11.445 s) : 0, 11444625
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.094 s -
Agent appsec 1.305 s 210.819 ms (19.3%)
Agent iast 1.283 s 188.94 ms (17.3%)
Agent profiling 1.351 s 257.519 ms (23.5%)
Total tracing 11.495 s -
Total appsec 11.219 s -276.607 ms (-2.4%)
Total iast 11.362 s -133.597 ms (-1.2%)
Total profiling 11.474 s -21.516 ms (-0.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.093 s -
Agent appsec 1.303 s 210.173 ms (19.2%)
Agent iast 1.28 s 187.065 ms (17.1%)
Agent profiling 1.345 s 252.603 ms (23.1%)
Total tracing 11.458 s -
Total appsec 11.341 s -116.548 ms (-1.0%)
Total iast 11.313 s -144.328 ms (-1.3%)
Total profiling 11.445 s -13.042 ms (-0.1%) 
gantt
    title petclinic - break down per module: candidate=1.63.0-SNAPSHOT~8e86faff09, baseline=1.63.0-SNAPSHOT~710ca87583

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.279 ms) : 0, 1279
crashtracking [candidate] (1.272 ms) : 0, 1272
BytebuddyAgent [baseline] (650.451 ms) : 0, 650451
BytebuddyAgent [candidate] (649.612 ms) : 0, 649612
AgentMeter [baseline] (30.546 ms) : 0, 30546
AgentMeter [candidate] (30.519 ms) : 0, 30519
GlobalTracer [baseline] (256.156 ms) : 0, 256156
GlobalTracer [candidate] (255.839 ms) : 0, 255839
AppSec [baseline] (33.717 ms) : 0, 33717
AppSec [candidate] (33.611 ms) : 0, 33611
Debugger [baseline] (64.2 ms) : 0, 64200
Debugger [candidate] (64.362 ms) : 0, 64362
Remote Config [baseline] (635.03 µs) : 0, 635
Remote Config [candidate] (641.102 µs) : 0, 641
Telemetry [baseline] (9.673 ms) : 0, 9673
Telemetry [candidate] (8.889 ms) : 0, 8889
Flare Poller [baseline] (9.809 ms) : 0, 9809
Flare Poller [candidate] (10.647 ms) : 0, 10647
section appsec
crashtracking [baseline] (1.282 ms) : 0, 1282
crashtracking [candidate] (1.269 ms) : 0, 1269
BytebuddyAgent [baseline] (692.784 ms) : 0, 692784
BytebuddyAgent [candidate] (690.781 ms) : 0, 690781
AgentMeter [baseline] (12.558 ms) : 0, 12558
AgentMeter [candidate] (12.516 ms) : 0, 12516
GlobalTracer [baseline] (258.11 ms) : 0, 258110
GlobalTracer [candidate] (258.431 ms) : 0, 258431
IAST [baseline] (25.813 ms) : 0, 25813
IAST [candidate] (25.513 ms) : 0, 25513
AppSec [baseline] (190.793 ms) : 0, 190793
AppSec [candidate] (191.209 ms) : 0, 191209
Debugger [baseline] (67.913 ms) : 0, 67913
Debugger [candidate] (68.003 ms) : 0, 68003
Remote Config [baseline] (622.564 µs) : 0, 623
Remote Config [candidate] (616.883 µs) : 0, 617
Telemetry [baseline] (7.897 ms) : 0, 7897
Telemetry [candidate] (7.937 ms) : 0, 7937
Flare Poller [baseline] (9.43 ms) : 0, 9430
Flare Poller [candidate] (8.847 ms) : 0, 8847
section iast
crashtracking [baseline] (1.282 ms) : 0, 1282
crashtracking [candidate] (1.274 ms) : 0, 1274
BytebuddyAgent [baseline] (845.878 ms) : 0, 845878
BytebuddyAgent [candidate] (843.087 ms) : 0, 843087
AgentMeter [baseline] (12.55 ms) : 0, 12550
AgentMeter [candidate] (11.689 ms) : 0, 11689
GlobalTracer [baseline] (245.15 ms) : 0, 245150
GlobalTracer [candidate] (245.714 ms) : 0, 245714
IAST [baseline] (28.075 ms) : 0, 28075
IAST [candidate] (28.365 ms) : 0, 28365
AppSec [baseline] (33.538 ms) : 0, 33538
AppSec [candidate] (33.484 ms) : 0, 33484
Debugger [baseline] (66.745 ms) : 0, 66745
Debugger [candidate] (66.794 ms) : 0, 66794
Remote Config [baseline] (559.706 µs) : 0, 560
Remote Config [candidate] (558.629 µs) : 0, 559
Telemetry [baseline] (8.194 ms) : 0, 8194
Telemetry [candidate] (8.242 ms) : 0, 8242
Flare Poller [baseline] (3.44 ms) : 0, 3440
Flare Poller [candidate] (3.448 ms) : 0, 3448
section profiling
crashtracking [baseline] (552.24 µs) : 0, 552
crashtracking [candidate] (539.204 µs) : 0, 539
BytebuddyAgent [baseline] (711.432 ms) : 0, 711432
BytebuddyAgent [candidate] (706.727 ms) : 0, 706727
AgentMeter [baseline] (10.533 ms) : 0, 10533
AgentMeter [candidate] (10.709 ms) : 0, 10709
GlobalTracer [baseline] (213.666 ms) : 0, 213666
GlobalTracer [candidate] (213.981 ms) : 0, 213981
AppSec [baseline] (33.131 ms) : 0, 33131
AppSec [candidate] (33.123 ms) : 0, 33123
Debugger [baseline] (69.621 ms) : 0, 69621
Debugger [candidate] (69.523 ms) : 0, 69523
Remote Config [baseline] (615.92 µs) : 0, 616
Remote Config [candidate] (601.229 µs) : 0, 601
Telemetry [baseline] (8.393 ms) : 0, 8393
Telemetry [candidate] (8.337 ms) : 0, 8337
Flare Poller [baseline] (3.712 ms) : 0, 3712
Flare Poller [candidate] (3.784 ms) : 0, 3784
ProfilingAgent [baseline] (93.378 ms) : 0, 93378
ProfilingAgent [candidate] (93.019 ms) : 0, 93019
Profiling [baseline] (93.974 ms) : 0, 93974
Profiling [candidate] (93.599 ms) : 0, 93599
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.63.0-SNAPSHOT~8e86faff09, baseline=1.63.0-SNAPSHOT~710ca87583

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.098 s) : 0, 1097519
Total [baseline] (9.428 s) : 0, 9428468
Agent [candidate] (1.098 s) : 0, 1097826
Total [candidate] (9.403 s) : 0, 9402637
section iast
Agent [baseline] (1.281 s) : 0, 1280667
Total [baseline] (10.135 s) : 0, 10134532
Agent [candidate] (1.276 s) : 0, 1276439
Total [candidate] (10.127 s) : 0, 10126661
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.098 s -
Agent iast 1.281 s 183.148 ms (16.7%)
Total tracing 9.428 s -
Total iast 10.135 s 706.064 ms (7.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.098 s -
Agent iast 1.276 s 178.613 ms (16.3%)
Total tracing 9.403 s -
Total iast 10.127 s 724.024 ms (7.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.63.0-SNAPSHOT~8e86faff09, baseline=1.63.0-SNAPSHOT~710ca87583

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.276 ms) : 0, 1276
crashtracking [candidate] (1.271 ms) : 0, 1271
BytebuddyAgent [baseline] (653.635 ms) : 0, 653635
BytebuddyAgent [candidate] (652.608 ms) : 0, 652608
AgentMeter [baseline] (30.435 ms) : 0, 30435
AgentMeter [candidate] (30.694 ms) : 0, 30694
GlobalTracer [baseline] (256.174 ms) : 0, 256174
GlobalTracer [candidate] (257.098 ms) : 0, 257098
AppSec [baseline] (33.726 ms) : 0, 33726
AppSec [candidate] (33.702 ms) : 0, 33702
Debugger [baseline] (62.919 ms) : 0, 62919
Debugger [candidate] (63.827 ms) : 0, 63827
Remote Config [baseline] (642.197 µs) : 0, 642
Remote Config [candidate] (646.783 µs) : 0, 647
Telemetry [baseline] (9.796 ms) : 0, 9796
Telemetry [candidate] (9.714 ms) : 0, 9714
Flare Poller [baseline] (11.505 ms) : 0, 11505
Flare Poller [candidate] (10.76 ms) : 0, 10760
section iast
crashtracking [baseline] (1.28 ms) : 0, 1280
crashtracking [candidate] (1.276 ms) : 0, 1276
BytebuddyAgent [baseline] (844.612 ms) : 0, 844612
BytebuddyAgent [candidate] (841.274 ms) : 0, 841274
AgentMeter [baseline] (11.984 ms) : 0, 11984
AgentMeter [candidate] (11.662 ms) : 0, 11662
GlobalTracer [baseline] (245.116 ms) : 0, 245116
GlobalTracer [candidate] (245.251 ms) : 0, 245251
IAST [baseline] (30.217 ms) : 0, 30217
IAST [candidate] (28.258 ms) : 0, 28258
AppSec [baseline] (31.752 ms) : 0, 31752
AppSec [candidate] (33.419 ms) : 0, 33419
Debugger [baseline] (66.165 ms) : 0, 66165
Debugger [candidate] (65.751 ms) : 0, 65751
Remote Config [baseline] (577.163 µs) : 0, 577
Remote Config [candidate] (572.055 µs) : 0, 572
Telemetry [baseline] (8.218 ms) : 0, 8218
Telemetry [candidate] (8.209 ms) : 0, 8209
Flare Poller [baseline] (3.421 ms) : 0, 3421
Flare Poller [candidate] (3.427 ms) : 0, 3427
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/grpcs
git_commit_date 1778583733 1778592116
git_commit_sha 710ca87 8e86faf
release_version 1.63.0-SNAPSHOT~710ca87583 1.63.0-SNAPSHOT~8e86faff09
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1778600389 1778600389
ci_job_id 1675876883 1675876883
ci_pipeline_id 112701082 112701082
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-6vp27d9w 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-6vp27d9w 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 1 performance regressions! Performance is the same for 17 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load better
[-412.612µs; -148.740µs] or [-7.427%; -2.677%]		 better
[-1187.451µs; -618.781µs] or [-8.908%; -4.642%]		 unstable
[-33.521op/s; +121.271op/s] or [-4.535%; +16.407%]		 5.275ms 12.427ms 783.000op/s 5.555ms 13.330ms 739.125op/s
scenario:load:petclinic:profiling:high_load worse
[+0.835ms; +1.325ms] or [+4.679%; +7.421%]		 unsure
[+0.568ms; +2.071ms] or [+1.958%; +7.133%]		 unstable
[-37.848op/s; +10.535op/s] or [-14.779%; +4.114%]		 18.933ms 30.353ms 242.438op/s 17.853ms 29.033ms 256.094op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.63.0-SNAPSHOT~8e86faff09, baseline=1.63.0-SNAPSHOT~710ca87583
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.403 ms) : 19206, 19599
.   : milestone, 19403,
appsec (19.008 ms) : 18815, 19202
.   : milestone, 19008,
code_origins (18.095 ms) : 17917, 18273
.   : milestone, 18095,
iast (17.952 ms) : 17777, 18128
.   : milestone, 17952,
profiling (18.225 ms) : 18043, 18406
.   : milestone, 18225,
tracing (18.2 ms) : 18022, 18377
.   : milestone, 18200,
section candidate
no_agent (18.369 ms) : 18183, 18554
.   : milestone, 18369,
appsec (18.666 ms) : 18477, 18854
.   : milestone, 18666,
code_origins (17.881 ms) : 17706, 18056
.   : milestone, 17881,
iast (18.134 ms) : 17954, 18314
.   : milestone, 18134,
profiling (19.251 ms) : 19056, 19445
.   : milestone, 19251,
tracing (17.699 ms) : 17523, 17874
.   : milestone, 17699,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.403 ms [19.206 ms, 19.599 ms] -
appsec 19.008 ms [18.815 ms, 19.202 ms] -394.112 µs (-2.0%)
code_origins 18.095 ms [17.917 ms, 18.273 ms] -1.308 ms (-6.7%)
iast 17.952 ms [17.777 ms, 18.128 ms] -1.45 ms (-7.5%)
profiling 18.225 ms [18.043 ms, 18.406 ms] -1.178 ms (-6.1%)
tracing 18.2 ms [18.022 ms, 18.377 ms] -1.203 ms (-6.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.369 ms [18.183 ms, 18.554 ms] -
appsec 18.666 ms [18.477 ms, 18.854 ms] 296.97 µs (1.6%)
code_origins 17.881 ms [17.706 ms, 18.056 ms] -487.569 µs (-2.7%)
iast 18.134 ms [17.954 ms, 18.314 ms] -234.39 µs (-1.3%)
profiling 19.251 ms [19.056 ms, 19.445 ms] 881.979 µs (4.8%)
tracing 17.699 ms [17.523 ms, 17.874 ms] -669.917 µs (-3.6%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.63.0-SNAPSHOT~8e86faff09, baseline=1.63.0-SNAPSHOT~710ca87583
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.255 ms) : 1243, 1267
.   : milestone, 1255,
iast (3.297 ms) : 3248, 3346
.   : milestone, 3297,
iast_FULL (6.26 ms) : 6195, 6325
.   : milestone, 6260,
iast_GLOBAL (3.677 ms) : 3619, 3735
.   : milestone, 3677,
profiling (2.233 ms) : 2212, 2254
.   : milestone, 2233,
tracing (1.97 ms) : 1952, 1987
.   : milestone, 1970,
section candidate
no_agent (1.268 ms) : 1255, 1280
.   : milestone, 1268,
iast (3.31 ms) : 3268, 3352
.   : milestone, 3310,
iast_FULL (5.905 ms) : 5846, 5965
.   : milestone, 5905,
iast_GLOBAL (3.704 ms) : 3644, 3764
.   : milestone, 3704,
profiling (2.386 ms) : 2361, 2411
.   : milestone, 2386,
tracing (1.916 ms) : 1900, 1932
.   : milestone, 1916,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.255 ms [1.243 ms, 1.267 ms] -
iast 3.297 ms [3.248 ms, 3.346 ms] 2.043 ms (162.8%)
iast_FULL 6.26 ms [6.195 ms, 6.325 ms] 5.005 ms (398.9%)
iast_GLOBAL 3.677 ms [3.619 ms, 3.735 ms] 2.422 ms (193.0%)
profiling 2.233 ms [2.212 ms, 2.254 ms] 978.02 µs (78.0%)
tracing 1.97 ms [1.952 ms, 1.987 ms] 714.873 µs (57.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.268 ms [1.255 ms, 1.28 ms] -
iast 3.31 ms [3.268 ms, 3.352 ms] 2.042 ms (161.1%)
iast_FULL 5.905 ms [5.846 ms, 5.965 ms] 4.638 ms (365.8%)
iast_GLOBAL 3.704 ms [3.644 ms, 3.764 ms] 2.436 ms (192.2%)
profiling 2.386 ms [2.361 ms, 2.411 ms] 1.118 ms (88.2%)
tracing 1.916 ms [1.9 ms, 1.932 ms] 648.594 µs (51.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/grpcs
git_commit_date 1778583733 1778592116
git_commit_sha 710ca87 8e86faf
release_version 1.63.0-SNAPSHOT~710ca87583 1.63.0-SNAPSHOT~8e86faff09
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1778600061 1778600061
ci_job_id 1675876886 1675876886
ci_pipeline_id 112701082 112701082
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-zq58syxu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-zq58syxu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.63.0-SNAPSHOT~8e86faff09, baseline=1.63.0-SNAPSHOT~710ca87583
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.93 s) : 14930000, 14930000
.   : milestone, 14930000,
appsec (14.942 s) : 14942000, 14942000
.   : milestone, 14942000,
iast (18.743 s) : 18743000, 18743000
.   : milestone, 18743000,
iast_GLOBAL (18.212 s) : 18212000, 18212000
.   : milestone, 18212000,
profiling (14.968 s) : 14968000, 14968000
.   : milestone, 14968000,
tracing (15.193 s) : 15193000, 15193000
.   : milestone, 15193000,
section candidate
no_agent (15.227 s) : 15227000, 15227000
.   : milestone, 15227000,
appsec (15.084 s) : 15084000, 15084000
.   : milestone, 15084000,
iast (19.289 s) : 19289000, 19289000
.   : milestone, 19289000,
iast_GLOBAL (18.155 s) : 18155000, 18155000
.   : milestone, 18155000,
profiling (14.877 s) : 14877000, 14877000
.   : milestone, 14877000,
tracing (15.126 s) : 15126000, 15126000
.   : milestone, 15126000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.93 s [14.93 s, 14.93 s] -
appsec 14.942 s [14.942 s, 14.942 s] 12.0 ms (0.1%)
iast 18.743 s [18.743 s, 18.743 s] 3.813 s (25.5%)
iast_GLOBAL 18.212 s [18.212 s, 18.212 s] 3.282 s (22.0%)
profiling 14.968 s [14.968 s, 14.968 s] 38.0 ms (0.3%)
tracing 15.193 s [15.193 s, 15.193 s] 263.0 ms (1.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.227 s [15.227 s, 15.227 s] -
appsec 15.084 s [15.084 s, 15.084 s] -143.0 ms (-0.9%)
iast 19.289 s [19.289 s, 19.289 s] 4.062 s (26.7%)
iast_GLOBAL 18.155 s [18.155 s, 18.155 s] 2.928 s (19.2%)
profiling 14.877 s [14.877 s, 14.877 s] -350.0 ms (-2.3%)
tracing 15.126 s [15.126 s, 15.126 s] -101.0 ms (-0.7%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.63.0-SNAPSHOT~8e86faff09, baseline=1.63.0-SNAPSHOT~710ca87583
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.502 ms) : 1490, 1514
.   : milestone, 1502,
appsec (3.891 ms) : 3660, 4121
.   : milestone, 3891,
iast (2.291 ms) : 2221, 2361
.   : milestone, 2291,
iast_GLOBAL (2.337 ms) : 2267, 2408
.   : milestone, 2337,
profiling (2.522 ms) : 2310, 2734
.   : milestone, 2522,
tracing (2.094 ms) : 2040, 2148
.   : milestone, 2094,
section candidate
no_agent (1.498 ms) : 1486, 1510
.   : milestone, 1498,
appsec (3.868 ms) : 3647, 4090
.   : milestone, 3868,
iast (2.295 ms) : 2225, 2366
.   : milestone, 2295,
iast_GLOBAL (2.332 ms) : 2262, 2401
.   : milestone, 2332,
profiling (2.516 ms) : 2360, 2672
.   : milestone, 2516,
tracing (2.103 ms) : 2049, 2157
.   : milestone, 2103,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.502 ms [1.49 ms, 1.514 ms] -
appsec 3.891 ms [3.66 ms, 4.121 ms] 2.389 ms (159.0%)
iast 2.291 ms [2.221 ms, 2.361 ms] 789.253 µs (52.5%)
iast_GLOBAL 2.337 ms [2.267 ms, 2.408 ms] 835.394 µs (55.6%)
profiling 2.522 ms [2.31 ms, 2.734 ms] 1.02 ms (67.9%)
tracing 2.094 ms [2.04 ms, 2.148 ms] 592.067 µs (39.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.498 ms [1.486 ms, 1.51 ms] -
appsec 3.868 ms [3.647 ms, 4.09 ms] 2.37 ms (158.2%)
iast 2.295 ms [2.225 ms, 2.366 ms] 797.446 µs (53.2%)
iast_GLOBAL 2.332 ms [2.262 ms, 2.401 ms] 833.727 µs (55.7%)
profiling 2.516 ms [2.36 ms, 2.672 ms] 1.018 ms (68.0%)
tracing 2.103 ms [2.049 ms, 2.157 ms] 604.832 µs (40.4%)

vandonr
vandonr approved these changes May 13, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@vandonr vandonr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, but do you have some metrics showing this improves perf ?
It looks like the code is mostly invoked with static key names, but not all the time ? I guess it's possible to have scenarios where regular swaths of variable keys come and wipe the cache... but then it'd just mostly revert to the previous behavior which was to create keys all the time. So I guess it's fine.

@amarziali
Copy link
Copy Markdown
Contributor Author

amarziali commented May 13, 2026

looks good, but do you have some metrics showing this improves perf

Yep I added the allocation histograms on the PR description. BTW the metadata key objects are immutable but we create bunch of them for each call. Even if it's a small allocation, it's worth not doing it if we can. Also we avoid calling io.grpc.Metadata$Key.validateName(String, boolean) on the hot path (that's called for each key created)

It looks like the code is mostly invoked with static key names, but not all the time ?

Normally we are largerly under 32 keys (taking into account datadog and w3c headers). We also have stuff from appsec and potentially the baggage. Having 64 element cache should be enough for nominal cases

I guess it's possible to have scenarios where regular swaths of variable keys come and wipe the cache... but then it'd just mostly revert to the previous behavior which was to create keys all the time. So I guess it's fine.

Yes in this case all the cache misses will translate in creating a new entry so it will be less effective but it won't break anything

@amarziali
Copy link
Copy Markdown
Contributor Author

amarziali commented May 13, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented May 13, 2026 •
edited
Loading

View all feedbacks in Devflow UI.

2026-05-13 09:44:58 UTC ℹ️ Start processing command /merge

2026-05-13 09:45:03 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-05-13 10:59:37 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit af7ee34 into master May 13, 2026
759 of 764 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the andrea.marziali/grpcs branch May 13, 2026 10:59
@github-actions github-actions Bot added this to the 1.63.0 milestone May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vandonr vandonr vandonr approved these changes

Assignees

No one assigned

Labels

inst: grpc gRPC instrumentation tag: performance Performance related changes type: enhancement Enhancements and improvements

Projects

None yet

Milestone

1.63.0

Development

Successfully merging this pull request may close these issues.

2 participants

@amarziali @vandonr