[williamcaban]

Fixes #1459.

run_test calls reliability_guard() without a limit although the comment above it says 4GB, so a generated solution that allocates in a loop grows until the machine runs out of memory and the operating system kills the whole evaluation.

This sets RLIMIT_AS in the forked child to the address space it already has (read from /proc/self/status) plus LIGHTEVAL_LCB_MEMORY_LIMIT_MIB (default 4096, 0 disables). A solution over the limit raises MemoryError and fails that problem; the run continues. The limit is relative to the inherited size because the child is forked from a process that already maps several GiB, so an absolute limit would break correct solutions. RLIMIT_STACK is not changed, so threads keep their default stack size. Where /proc/self/status or RLIMIT_AS are unavailable (macOS, Windows) it does nothing, as before.

Tests (tests/unit/tasks/test_lcb_solution_memory.py, Linux only) go through codegen_metrics and its process pool: an answer that allocates 512 MiB fails with a 128 MiB limit, one that allocates 32 MiB and one that uses a thread pass, and the limit can be turned off. With the limit_solution_memory() call removed, the first test fails and the others pass. On Linux (Python 3.12, 1.8 GB of RAM) they pass alone, in tests/unit/tasks order, and 6 of 6 times together with test_lighteval_task.py. The only failures in tests/unit/tasks there are the 8 test_registry.py tests, which need Pillow. ruff check and format pass.

Does not overlap with #1401 (check_correctness target) or #1257 (grade_stdio).

Limits. This bounds the address space, not resident memory. Pages the child inherits and then writes to are copied by the kernel and are not covered (a child that ran gc.collect() in a ~1.5 GiB parent added about 280 MiB), so memory in use can exceed the limit by a few hundred MiB. The docstring says so. The 4096 MiB default lets a solution add up to 4 GiB to what the process already uses; on a smaller machine, set LIGHTEVAL_LCB_MEMORY_LIMIT_MIB lower.

Not covered: a full LiveCodeBench run with this patch against a real model. A 175-problem run in a containerized environment finished without any solution reaching a limit, so it does not exercise this code path; the runaway behaviour was tested with a deliberately greedy solution.

---

Prepared with Claude Code (Anthropic's AI coding assistant), which helped write the patch, the tests and this text. I reviewed it and am responsible for it. Signed-off-by: William Caban