Run job-log regex search in a worker thread with a timeout (ReDoS hardening) #819

Context

#812 adds substring/regex search to get_pipeline_job_output (pattern, regex, case_sensitive, context_lines, max_matches). With regex: true, the caller-supplied pattern is compiled with new RegExp(pattern, flags) and tested against every line of the full job log in utils/job-log-search.ts (searchJobLog).

CodeRabbit flagged this as a major stability issue (review comment). It was left out of scope for #812 because a full fix needs a new dependency or a new execution model. This issue tracks that follow-up.

Risk

The current safeguards in #812 are:

• pattern length capped at 500 characters (JOB_LOG_MAX_PATTERN_LENGTH)
• each line cut to 4,000 characters before matching (JOB_LOG_MAX_LINE_LENGTH)
• a 1 s search budget (JOB_LOG_SEARCH_BUDGET_MS)

The budget is only checked between lines. It does not bound a single RegExp.test() call. A catastrophically backtracking pattern such as (a+)+$ on one long line can run for a very long time, and it runs on the main thread, so it blocks the Node event loop. In remote mode (SSE / Streamable HTTP), one caller can stall every session on a shared server. The tool is read-only and regex is opt-in per call (regex: true), but nothing at the server level stops an untrusted client from setting it.

Options considered

1. re2 (linear-time engine): new RE2(pattern, flags) removes catastrophic backtracking. Downsides: native dependency (prebuilt binaries / node-gyp, affects npx, Docker and Homebrew installs), and RE2 syntax differs from JS (no lookaround or backreferences), so some patterns would be rejected.
2. worker_threads with a hard timeout: run the matching in a Worker and terminate() it when the deadline passes. No new dependency and keeps full JS regex syntax. Costs: worker startup per search (or a small pool), copying the log text to the worker, and a bit more code to test.
3. Keep it as it is, opt-in, and document it: leave regex as an explicit opt-in (optionally behind a server-level switch that is off by default for remote deployments) and add a warning to the docs. Cheapest, but the event-loop risk stays for anyone who enables it.

A smaller mitigation, lowering the line length cap for regex mode only, reduces worst-case blocking time but does not remove the risk.

Proposal

Go with option 2:

• Substring search stays on the main thread (it is linear).
• When regex: true, run the line matching in a worker_threads Worker with a hard deadline (e.g. the existing 1 s budget).
• On timeout, terminate the worker and return an isError tool result such as "Job log search exceeded the time limit. Use a simpler pattern or turn regex off.", the same message the current budget check uses.
• Keep the existing pattern and line-length caps.
• Tests: a pathological pattern (e.g. (a+)+$ on a long line of as followed by !) returns isError within the deadline, and the server stays responsive to a concurrent request.

Related: #812