From 3fc47fec8d84255f2763ebf0d118e7355a9c6100 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 3 Sep 2026 13:47:39 +0000 Subject: [PATCH 1/6] Add Python Actor debugging: debugpy + a served browser DAP client Python sibling of the Node/TS actor-debugger, same architecture and the same constraints (no infrastructure changes, browser-only debugging machine): a one-line CMD change launches the Actor under debugpy, and one HTTP server on ACTOR_WEB_SERVER_PORT serves a bundled browser debugger UI and bridges a WebSocket at /dap to the adapter's DAP-over-TCP socket. - python/: pip package (only runtime dep: debugpy; server + RFC6455 WebSocket are stdlib). The bridge translates framing, injects the adapter's loopback address into attach requests (enables re-attach), and synthesizes a disconnect when the browser drops without one so page reloads don't kill the adapter. - frontend: single-file DAP client - gutter breakpoints (persisted in localStorage), stepping, call stack, variables tree, in-frame REPL, exception breakpoints; --brk maps to --wait-for-client plus a synthetic first-line breakpoint for a Node-like entry stop. - example-actor-python/: sample Actor mirroring the Apify Python template. - python/README.md documents the design and the alternatives considered (GraalPy --inspect/CDP, DAP-to-CDP shim, code-server, web-pdb, Jupyter). - publish-python.yml: PyPI release on py-v* tags, mirroring the npm flow. Verified end-to-end: full DAP session through the bridge (protocol test) and headless Chromium driving the served UI - entry pause, breakpoint hit, step, variables, evaluate, resume, re-attach after reload. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0194oFPjsTxztgsPhyB4Wq7G --- .github/workflows/publish-python.yml | 95 +++ .gitignore | 3 + README.md | 4 + example-actor-python/.actor/actor.json | 1 + example-actor-python/Dockerfile | 6 + example-actor-python/requirements.txt | 2 + example-actor-python/src/__init__.py | 0 example-actor-python/src/__main__.py | 5 + example-actor-python/src/main.py | 10 + python/README.md | 131 ++++ python/pyproject.toml | 31 + python/src/actor_debugger/__init__.py | 3 + python/src/actor_debugger/__main__.py | 3 + python/src/actor_debugger/cli.py | 145 +++++ python/src/actor_debugger/debug_server.py | 258 ++++++++ python/src/actor_debugger/frontend/index.html | 598 ++++++++++++++++++ python/src/actor_debugger/websocket.py | 124 ++++ 17 files changed, 1419 insertions(+) create mode 100644 .github/workflows/publish-python.yml create mode 100644 example-actor-python/.actor/actor.json create mode 100644 example-actor-python/Dockerfile create mode 100644 example-actor-python/requirements.txt create mode 100644 example-actor-python/src/__init__.py create mode 100644 example-actor-python/src/__main__.py create mode 100644 example-actor-python/src/main.py create mode 100644 python/README.md create mode 100644 python/pyproject.toml create mode 100644 python/src/actor_debugger/__init__.py create mode 100644 python/src/actor_debugger/__main__.py create mode 100644 python/src/actor_debugger/cli.py create mode 100644 python/src/actor_debugger/debug_server.py create mode 100644 python/src/actor_debugger/frontend/index.html create mode 100644 python/src/actor_debugger/websocket.py diff --git a/.github/workflows/publish-python.yml b/.github/workflows/publish-python.yml new file mode 100644 index 0000000..23d12e6 --- /dev/null +++ b/.github/workflows/publish-python.yml @@ -0,0 +1,95 @@ +# Publishes the Python actor-debugger package to PyPI when a py-v* tag is pushed. +# +# Release flow: +# (bump version in python/pyproject.toml and python/src/actor_debugger/__init__.py, commit) +# git tag py-vX.Y.Z && git push --tags +# +# Setup (one-time): create a PyPI API token scoped to this project (or an account token for the +# first upload) and save it as the repository secret PYPI_TOKEN. + +name: Publish to PyPI + +on: + push: + tags: + - 'py-v*' + +jobs: + check: + runs-on: ubuntu-latest + defaults: + run: + working-directory: python + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Install package + run: pip install . + + - name: Verify tag matches pyproject version + run: | + PKG_VERSION="$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")" + TAG_VERSION="${GITHUB_REF_NAME#py-v}" + if [ "$PKG_VERSION" != "$TAG_VERSION" ]; then + echo "Tag $GITHUB_REF_NAME does not match pyproject.toml version $PKG_VERSION" >&2 + exit 1 + fi + + - name: Smoke test (entrypoint runs and its output passes through) + run: | + mkdir -p /tmp/smoke && cd /tmp/smoke + echo 'print("smoke-ok")' > main.py + OUTPUT="$(python -m actor_debugger 2>&1)" + echo "$OUTPUT" + echo "$OUTPUT" | grep -q 'smoke-ok' + + - name: Smoke test (debugger UI served over the web-server port) + run: | + mkdir -p /tmp/smoke-debug && cd /tmp/smoke-debug + printf 'import time\ntime.sleep(15)\n' > main.py + ACTOR_WEB_SERVER_PORT=4321 ACTOR_WEB_SERVER_URL=http://localhost:4321 \ + python -m actor_debugger main.py & + CLI_PID=$! + for i in $(seq 1 30); do + sleep 0.5 + if curl -sf http://127.0.0.1:4321/config > /dev/null; then break; fi + done + curl -sf http://127.0.0.1:4321/config | grep -q '"cwd"' + curl -sf http://127.0.0.1:4321/ui/ | grep -q 'Actor Debugger' + kill $CLI_PID + + - name: Build distributions + run: | + pip install build + python -m build + + publish: + needs: check + runs-on: ubuntu-latest + environment: pypi + defaults: + run: + working-directory: python + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Build distributions + run: | + pip install build + python -m build + + - name: Publish + run: | + pip install twine + twine upload dist/* + env: + TWINE_USERNAME: __token__ + TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} diff --git a/.gitignore b/.gitignore index c2658d7..6371889 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,4 @@ node_modules/ +__pycache__/ +*.egg-info/ +python/dist/ diff --git a/README.md b/README.md index 0f99b2d..1093e66 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,9 @@ # actor-debugger +> **Python Actors:** the same one-link browser debugging exists for Python — debugpy plus a served +> DAP frontend instead of the Node inspector plus Chrome DevTools. See +> [`python/README.md`](python/README.md) and the sample [`example-actor-python/`](example-actor-python/). + Drop-in remote debugging for **any Apify Node/TS Actor** with a one-line Dockerfile change. It launches your Actor under the Node inspector **and serves a full Chrome DevTools UI over the run's container URL** — so you open one link in your own browser and debug. No wstunnel, no local setup, diff --git a/example-actor-python/.actor/actor.json b/example-actor-python/.actor/actor.json new file mode 100644 index 0000000..59d09bd --- /dev/null +++ b/example-actor-python/.actor/actor.json @@ -0,0 +1 @@ +{ "actorSpecification": 1, "name": "sample-py-actor", "version": "0.0", "buildTag": "latest", "dockerfile": "./Dockerfile" } diff --git a/example-actor-python/Dockerfile b/example-actor-python/Dockerfile new file mode 100644 index 0000000..152fcf1 --- /dev/null +++ b/example-actor-python/Dockerfile @@ -0,0 +1,6 @@ +FROM apify/actor-python:3.13 +COPY requirements.txt ./ +RUN pip install --no-cache-dir -r requirements.txt +COPY . ./ +# One-line change enables the debugger (revert to the normal entrypoint to disable): +CMD ["python3", "-m", "actor_debugger", "--brk"] diff --git a/example-actor-python/requirements.txt b/example-actor-python/requirements.txt new file mode 100644 index 0000000..7bed89e --- /dev/null +++ b/example-actor-python/requirements.txt @@ -0,0 +1,2 @@ +apify +actor-debugger diff --git a/example-actor-python/src/__init__.py b/example-actor-python/src/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/example-actor-python/src/__main__.py b/example-actor-python/src/__main__.py new file mode 100644 index 0000000..a66fc48 --- /dev/null +++ b/example-actor-python/src/__main__.py @@ -0,0 +1,5 @@ +import asyncio + +from .main import main + +asyncio.run(main()) diff --git a/example-actor-python/src/main.py b/example-actor-python/src/main.py new file mode 100644 index 0000000..1632d4c --- /dev/null +++ b/example-actor-python/src/main.py @@ -0,0 +1,10 @@ +import asyncio + +from apify import Actor + + +async def main() -> None: + async with Actor: + for tick in range(1, 61): + Actor.log.info(f'sample tick {tick}') + await asyncio.sleep(1) diff --git a/python/README.md b/python/README.md new file mode 100644 index 0000000..928d798 --- /dev/null +++ b/python/README.md @@ -0,0 +1,131 @@ +# actor-debugger (Python) + +Drop-in remote debugging for **any Apify Python Actor** with a one-line Dockerfile change. It +launches your Actor under **debugpy** (the standard Python debugger that powers VS Code) **and +serves a full browser debugger UI over the run's container URL** — so you open one link in your +own browser and debug. No IDE, no wstunnel, no local setup, no rebuild of your source, and **no +browser or IDE in the Actor**. + +```dockerfile +# Get the package +RUN pip install actor-debugger + +# pause on the first executable line until you attach (for short-lived Actors): +CMD ["python3", "-m", "actor_debugger", "--brk"] + +# or point at a specific module / file: +CMD ["python3", "-m", "actor_debugger", "-m", "src"] +CMD ["python3", "-m", "actor_debugger", "main.py"] +``` + +## How it works + +This is the Python sibling of the Node/TS `actor-debugger` npm package in the repository root, +built on the same architecture — swap `node --inspect` for debugpy and Chrome DevTools for a +served DAP client: + +1. It resolves your Actor's entrypoint (see detection order below) and launches it as + `python -m debugpy --listen 127.0.0.1:5678 ` — so the debugger runs on **your** code, + in its own process, exactly as it normally runs. debugpy speaks the + [Debug Adapter Protocol](https://microsoft.github.io/debug-adapter-protocol/) (DAP), the same + protocol VS Code uses to debug Python. +2. It runs one HTTP server on `ACTOR_WEB_SERVER_PORT` that: + - **serves a browser debugger UI** (static files, bundled with the package) so your *local* + browser opens it — the DAP *client* is JavaScript running in your browser; + - **bridges a WebSocket at `/dap`** to debugpy's DAP-over-TCP socket on loopback, translating + framing (one WebSocket message per DAP JSON message ↔ `Content-Length` framed TCP) and + injecting the adapter's loopback address into `attach` requests — which is also what makes + re-attach after a browser reload work; + - serves `/source?path=...` so the UI can display files that exist only on the container's + disk (a remote frontend can never fetch `file://` URLs — the same problem the Node version + solves by inlining source maps). +3. The run log prints one URL. Open it in your browser → a debugger attached to your Actor: + click line numbers to set breakpoints, step, inspect the stack and variables, evaluate + expressions in any frame, break on exceptions. + +Because the container side is a dumb byte bridge, all protocol intelligence lives in the served +frontend — the Actor image gains only `debugpy` (~3 MB) and ~30 KB of static files. There is no +IDE, browser, Xvfb, or VNC in the image, and the Apify platform is used exactly as-is: the +standard container web server port is the only channel. + +## Activation + +Running the Actor through `python3 -m actor_debugger` **is** the switch — debugging is on +whenever the `CMD` line above is in place. To turn it off, revert the `CMD` to the Actor's normal +entrypoint (e.g. `CMD ["python3", "-m", "src"]`) and rebuild. Pass `--brk` to pause on the first +executable line of the entry file until you attach — useful for Actors that would otherwise +finish before you connect. (`--brk` maps to debugpy's `--wait-for-client` plus a synthetic +first-line breakpoint that the UI sets before resuming and removes after the first stop.) + +## Connect + +Open the URL the run log prints, in your own browser: + +``` +https://.runs.apify.net/ui/ +``` + +That page is the debugger; it connects back to the same host over `wss://…/dap`. Breakpoints are +kept in `localStorage`, so they survive page reloads and re-attach automatically. Program stdout +stays in the Actor run log as usual; the UI console is a REPL into the paused frame (plus +debugger events and exception details). + +Prefer a raw channel? `wss://.runs.apify.net/dap` speaks DAP directly, one JSON message per +WebSocket text frame — any DAP client can drive it. + +## Entrypoint detection order + +1. An explicit `-m ` or `` argument, if given. +2. `src/__main__.py` → runs `-m src` (the Apify Python Actor template convention). +3. `src/main.py` → runs `-m src.main`. +4. Conventional files: `main.py`, `__main__.py`, `app.py`. + +## Security + +The debug endpoint is **unauthenticated** — anyone who reaches the container URL can execute code +in your run (and read its env, including `APIFY_TOKEN`); `/source` additionally serves any file +readable in the container. Keep the `actor_debugger` `CMD` only in builds you are actively +debugging, prefer a restricted run/token, never ship it in a published Actor, and gate the +endpoint (owner-only) before any non-prototype use. This matches the security posture of the Node +version; both need the same hardening pass. + +## Design notes: the routes considered + +The constraint was: **no platform/infrastructure changes, and nothing but a browser on the +debugging machine.** Everything must therefore run inside the Actor container and be served over +the one exposed web-server port. Options considered: + +| Approach | Verdict | +| --- | --- | +| **debugpy + served browser DAP client** (this package) | CPython unchanged, actor code and deps run exactly as in production; debugpy is the canonical Python debugger; image cost ~3 MB. The UI is ours to grow. **Chosen.** | +| **GraalPy `--inspect`** — GraalVM's Python speaks the Chrome DevTools Protocol natively, so the Node version's chii DevTools frontend + CDP proxy would work *unchanged* | Elegant symmetry (identical UI and proxy for JS and Python), but it swaps the runtime under the Actor: a different base image, slower startup/warmup, and C-extension compatibility risk for real-world scraping deps (`lxml`, `pydantic`, `cryptography`, …). Debugging a different interpreter than production undermines the point. Worth a spike for pure-Python Actors; not the default. | +| **DAP→CDP translation shim** — keep CPython + debugpy, translate DAP into the Chrome DevTools Protocol in the bridge, and serve the same chii DevTools frontend the Node version uses | Best possible UI for free, one frontend for both languages. The mapping (breakpoints, stepping, stack, scopes, evaluate) is tractable, but the DevTools frontend is picky about `Debugger`/`Runtime` domain lifecycle, script IDs, and object groups — a meaningful project on its own. The `/dap` WebSocket this package exposes is exactly where such a shim would slot in later. | +| **code-server / openvscode-server in the container** — full VS Code in a browser tab, attach to debugpy | Great UX, zero custom code, but adds ~300 MB and a Node runtime to every Python Actor image, needs auth wiring, and its multiplexed WebSockets are more ingress-sensitive. Overkill for "set a breakpoint in a run". | +| **`web-pdb` / xterm.js + pdb over WebSocket** | Tiny and dependency-free, but a terminal `pdb` UX (no gutter breakpoints, no variable tree) and pdb can't attach to a running program the way debugpy can. Good fallback, not the goal. | +| **Jupyter server in the container** (JupyterLab's visual debugger also speaks debugpy) | The kernel model doesn't fit debugging an already-running script/Actor process; heavyweight. | +| **[`dap-python`](https://pypi.org/project/dap-python/)** — a typed Python DAP *client* library | Sits on the wrong side of this design's wire: the DAP client here is the browser, and the container side is a protocol-agnostic byte bridge. It would fit a server-orchestrated variant (Python translating DAP to a custom browser protocol), at the cost of a Pydantic dependency in every Actor image and a Python ≥3.12 floor. Noted as a building block if the bridge ever needs server-side smarts. | + +## Verified + +Tested end-to-end against a generic sample Python Actor (`example-actor-python/` in the repo): + +- The one-line `CMD` detects the entrypoint (`-m src`) and runs the Actor under debugpy; explicit + module/file and auto-detect modes all pass. +- `/ui/` and `/config` are served; `/source` returns container files; the `/dap` WebSocket + round-trips the full DAP handshake (`initialize` → `attach` → `setBreakpoints` → + `configurationDone`), hits a breakpoint, walks the stack/scopes/variables, evaluates in-frame, + and resumes — through the bridge with browser framing. +- **Headless Chromium loaded the served UI and drove a real session**: attached, auto-paused on + the entry line under `--brk`, rendered sources, stepped, hit a gutter-set breakpoint, showed + stack + variables, evaluated in the paused frame, and resumed. Re-attach after page reload + works. + +One thing to confirm on real infrastructure (shared with the Node version): the `wss://` +connection through Apify's ingress when it negotiates HTTP/2. + +## Notes + +- Only runtime dependency is `debugpy`; the server and WebSocket implementation are stdlib. +- `--wait-for-client` means a crashed debug UI never blocks a *running* Actor — the process only + waits at startup with `--brk`. +- The frontend applies "just my code" on (re)connect; untick it to step into library code. diff --git a/python/pyproject.toml b/python/pyproject.toml new file mode 100644 index 0000000..39be015 --- /dev/null +++ b/python/pyproject.toml @@ -0,0 +1,31 @@ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "actor-debugger" +version = "0.1.0" +description = "Drop-in remote debugging for Apify Python Actors: debugpy plus a browser debugger UI served over the run's container URL." +readme = "README.md" +license = { text = "Apache-2.0" } +requires-python = ">=3.9" +dependencies = ["debugpy>=1.8"] +keywords = ["apify", "actor", "debugger", "debugpy", "dap", "remote-debugging"] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "Programming Language :: Python :: 3", + "Topic :: Software Development :: Debuggers", +] + +[project.urls] +Homepage = "https://github.com/Pijukatel/actor-debugger" + +[project.scripts] +actor-debugger = "actor_debugger.cli:main" + +[tool.hatch.build.targets.wheel] +packages = ["src/actor_debugger"] + +[tool.hatch.build.targets.sdist] +include = ["src/actor_debugger", "README.md"] diff --git a/python/src/actor_debugger/__init__.py b/python/src/actor_debugger/__init__.py new file mode 100644 index 0000000..ab3bee4 --- /dev/null +++ b/python/src/actor_debugger/__init__.py @@ -0,0 +1,3 @@ +"""actor-debugger: browser-based remote debugging for Apify Python Actors.""" + +__version__ = "0.1.0" diff --git a/python/src/actor_debugger/__main__.py b/python/src/actor_debugger/__main__.py new file mode 100644 index 0000000..4e28416 --- /dev/null +++ b/python/src/actor_debugger/__main__.py @@ -0,0 +1,3 @@ +from .cli import main + +main() diff --git a/python/src/actor_debugger/cli.py b/python/src/actor_debugger/cli.py new file mode 100644 index 0000000..4dd8123 --- /dev/null +++ b/python/src/actor_debugger/cli.py @@ -0,0 +1,145 @@ +"""actor-debugger - launch an Apify Python Actor under debugpy, debuggable from a plain +browser over the run's container URL, with a one-line Dockerfile change: + + CMD ["python3", "-m", "actor_debugger"] # auto-detect the Actor's entrypoint + CMD ["python3", "-m", "actor_debugger", "-m", "src"] # explicit module + CMD ["python3", "-m", "actor_debugger", "main.py"] # explicit file + CMD ["python3", "-m", "actor_debugger", "--brk"] # pause on the first line until attached + +Running through this command is what enables debugging - revert the CMD to the normal +entrypoint to turn it off. +""" + +from __future__ import annotations + +import os +import signal +import socket +import subprocess +import sys +import threading +import time + +from .debug_server import start_debug_server + +ADAPTER_PORT = 5678 +TAG = "[actor-debugger]" + + +def log(message: str) -> None: + print(f"{TAG} {message}", file=sys.stderr, flush=True) + + +def resolve_entry(args: list[str]) -> list[str] | None: + """Return the argv tail (["-m", "pkg"] or ["path.py"]) for the Actor's entrypoint.""" + if "-m" in args: + module = args[args.index("-m") + 1 :][:1] + if module: + return ["-m", module[0]] + positional = [a for a in args if not a.startswith("-")] + if positional: + return [os.path.abspath(positional[0])] + # The Apify Python templates run `python3 -m src` (src/__main__.py + src/main.py). + if os.path.isfile("src/__main__.py"): + return ["-m", "src"] + if os.path.isfile("src/main.py"): + return ["-m", "src.main"] + for candidate in ("main.py", "__main__.py", "app.py"): + if os.path.isfile(candidate): + return [os.path.abspath(candidate)] + return None + + +def entry_file_path(entry: list[str]) -> str: + """Best-effort path of the file that runs first, for the frontend's --brk entry stop.""" + if entry[0] != "-m": + return entry[0] + base = entry[1].replace(".", os.sep) + for candidate in (os.path.join(base, "__main__.py"), base + ".py"): + if os.path.isfile(candidate): + return os.path.abspath(candidate) + return "" + + +def wait_for_adapter(port: int, timeout: float = 30.0) -> bool: + """Wait until something listens on the adapter port WITHOUT connecting to it. + + debugpy treats the first accepted connection as its DAP client and shuts the session + down when it drops, so probing with a real connect would kill the adapter. A bind + attempt failing with EADDRINUSE detects the listener without ever touching it. + """ + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + try: + probe.bind(("127.0.0.1", port)) + except OSError: + return True + finally: + probe.close() + time.sleep(0.25) + return False + + +def announce(web_server_url: str) -> None: + if not wait_for_adapter(ADAPTER_PORT): + log(f"debugpy adapter did not come up on 127.0.0.1:{ADAPTER_PORT}.") + return + base = web_server_url.rstrip("/") + log("=" * 64) + log("debugging is ON - reachable over the container URL.") + log("OPEN THIS in your local browser for a full debugger UI (no local setup):") + log(f" {base}/ui/") + host = base.split("://", 1)[-1] + log(f"raw DAP-over-WebSocket channel (one JSON message per frame): wss://{host}/dap") + log("=" * 64) + + +def main() -> None: + args = sys.argv[1:] + brk = "--brk" in args + entry = resolve_entry([a for a in args if a != "--brk"]) + if not entry: + log("could not find an Actor entrypoint. Pass one explicitly:") + log(' CMD ["python3", "-m", "actor_debugger", "-m", "src"]') + sys.exit(1) + + debugpy_argv = [ + sys.executable, + "-Xfrozen_modules=off", + "-m", + "debugpy", + "--listen", + f"127.0.0.1:{ADAPTER_PORT}", + *(["--wait-for-client"] if brk else []), + *entry, + ] + env = {**os.environ, "PYDEVD_DISABLE_FILE_VALIDATION": "1"} + child = subprocess.Popen(debugpy_argv, env=env) + + server = None + port = os.environ.get("ACTOR_WEB_SERVER_PORT") + url = os.environ.get("ACTOR_WEB_SERVER_URL") + if port and url: + frontend_dir = os.path.join(os.path.dirname(__file__), "frontend") + server = start_debug_server(int(port), ADAPTER_PORT, frontend_dir, brk=brk, entry_path=entry_file_path(entry)) + threading.Thread(target=announce, args=(url,), daemon=True).start() + if brk: + log("--brk: the Actor is paused on its first line until you attach from the browser.") + else: + log(f"ACTOR_WEB_SERVER_URL/PORT not set - DAP on 127.0.0.1:{ADAPTER_PORT} only (no container-URL bridge).") + + def forward(signum, _frame): + child.send_signal(signum) + + signal.signal(signal.SIGTERM, forward) + signal.signal(signal.SIGINT, forward) + + code = child.wait() + if server: + server.shutdown() + sys.exit(code) + + +if __name__ == "__main__": + main() diff --git a/python/src/actor_debugger/debug_server.py b/python/src/actor_debugger/debug_server.py new file mode 100644 index 0000000..39bf5bc --- /dev/null +++ b/python/src/actor_debugger/debug_server.py @@ -0,0 +1,258 @@ +"""One HTTP server on the container's web-server port that: + +- serves the bundled browser debugger UI (static files) so your LOCAL browser opens it - + no IDE, no local setup; +- bridges a WebSocket at /dap to the debugpy adapter's DAP-over-TCP socket on loopback, + translating framing (one WebSocket text message per DAP JSON message <-> Content-Length + framed TCP) so the browser client stays trivial; +- rewrites `attach` requests in flight to inject the adapter's loopback connect address, + which the browser cannot know (and which is what makes debugpy accept re-attach after + a page reload); +- serves /source?path=... so the remote frontend can display files that only exist on the + container's disk (a browser can never fetch file:// URLs from the container); +- serves /config with the working directory so the frontend can group project files apart + from library code. +""" + +from __future__ import annotations + +import json +import mimetypes +import os +import socket +import threading +import time +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from urllib.parse import parse_qs, urlparse + +from .websocket import WebSocketConnection, WebSocketClosed, accept_key + +UI_PREFIX = "/ui/" + + +def _read_dap_messages(sock: socket.socket): + """Yield DAP message bodies (bytes) from a Content-Length framed TCP stream.""" + buf = b"" + while True: + while b"\r\n\r\n" not in buf: + chunk = sock.recv(65536) + if not chunk: + return + buf += chunk + header, buf = buf.split(b"\r\n\r\n", 1) + length = None + for line in header.split(b"\r\n"): + name, _, value = line.partition(b":") + if name.strip().lower() == b"content-length": + length = int(value.strip()) + if length is None: + return + while len(buf) < length: + chunk = sock.recv(65536) + if not chunk: + return + buf += chunk + body, buf = buf[:length], buf[length:] + yield body + + +class DebugRequestHandler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + + # Injected by start_debug_server(): + frontend_dir: str = "" + adapter_host: str = "127.0.0.1" + adapter_port: int = 5678 + brk: bool = False + entry_path: str = "" + + def log_message(self, format, *args): # noqa: A002 - stdlib signature + pass # keep the Actor log clean + + # -- HTTP --------------------------------------------------------------- + + def do_GET(self): # noqa: N802 - stdlib naming + parsed = urlparse(self.path) + if parsed.path == "/dap" and "websocket" in self.headers.get("Upgrade", "").lower(): + self._handle_websocket() + return + if parsed.path in ("", "/"): + self.send_response(302) + self.send_header("Location", f"{UI_PREFIX}") + self.send_header("Content-Length", "0") + self.end_headers() + return + if parsed.path == "/config": + self._send_json({"cwd": os.getcwd(), "brk": self.brk, "entryPath": self.entry_path}) + return + if parsed.path == "/source": + self._serve_source(parsed) + return + self._serve_static(parsed.path) + + def _send_json(self, payload: dict) -> None: + body = json.dumps(payload).encode() + self.send_response(200) + self.send_header("Content-Type", "application/json; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _send_plain(self, status: int, text: str) -> None: + body = text.encode() + self.send_response(status) + self.send_header("Content-Type", "text/plain; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _serve_source(self, parsed) -> None: + path = (parse_qs(parsed.query).get("path") or [""])[0] + if not path or not os.path.isabs(path) or not os.path.isfile(path): + self._send_plain(404, "not found") + return + try: + with open(path, "rb") as f: + body = f.read(4 * 1024 * 1024) + except OSError: + self._send_plain(404, "not found") + return + self.send_response(200) + self.send_header("Content-Type", "text/plain; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _serve_static(self, url_path: str) -> None: + rel = url_path[len(UI_PREFIX):] if url_path.startswith(UI_PREFIX) else url_path.lstrip("/") + rel = rel or "index.html" + file_path = os.path.normpath(os.path.join(self.frontend_dir, rel)) + if not file_path.startswith(os.path.abspath(self.frontend_dir)): + self._send_plain(403, "forbidden") + return + if not os.path.isfile(file_path): + self._send_plain(404, "not found") + return + ctype = mimetypes.guess_type(file_path)[0] or "application/octet-stream" + with open(file_path, "rb") as f: + body = f.read() + self.send_response(200) + self.send_header("Content-Type", ctype) + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + # -- WebSocket <-> DAP bridge -------------------------------------------- + + def _handle_websocket(self) -> None: + key = self.headers.get("Sec-WebSocket-Key") + if not key: + self._send_plain(400, "bad websocket handshake") + return + upstream = None + deadline = time.monotonic() + 15 + while upstream is None: + try: + upstream = socket.create_connection((self.adapter_host, self.adapter_port), timeout=5) + except OSError: + if time.monotonic() >= deadline: + self._send_plain(502, "debug adapter unavailable") + return + time.sleep(0.5) + self.connection.sendall( + b"HTTP/1.1 101 Switching Protocols\r\n" + b"Upgrade: websocket\r\n" + b"Connection: Upgrade\r\n" + b"Sec-WebSocket-Accept: " + accept_key(key).encode() + b"\r\n\r\n" + ) + self.close_connection = True + ws = WebSocketConnection(self.connection) + + def pump_adapter_to_browser(): + try: + for body in _read_dap_messages(upstream): + ws.send_text(body.decode("utf-8", errors="replace")) + except (OSError, WebSocketClosed): + pass + finally: + ws.close() + + pump = threading.Thread(target=pump_adapter_to_browser, daemon=True) + pump.start() + saw_disconnect = False + try: + while True: + text = ws.recv_text() + if '"disconnect"' in text: + try: + parsed = json.loads(text) + if parsed.get("type") == "request" and parsed.get("command") == "disconnect": + saw_disconnect = True + except ValueError: + pass + body = self._rewrite_attach(text).encode() + upstream.sendall(b"Content-Length: %d\r\n\r\n%b" % (len(body), body)) + except (OSError, WebSocketClosed): + pass + finally: + if not saw_disconnect: + # The browser vanished (tab closed, page reloaded) without a DAP disconnect. + # An abruptly dropped client makes debugpy's adapter stop listening, killing + # re-attach - so say goodbye on the browser's behalf. + goodbye = json.dumps({ + "seq": 2_000_000_000, + "type": "request", + "command": "disconnect", + "arguments": {"terminateDebuggee": False}, + }).encode() + try: + upstream.sendall(b"Content-Length: %d\r\n\r\n%b" % (len(goodbye), goodbye)) + time.sleep(0.5) + except OSError: + pass + try: + upstream.close() + except OSError: + pass + ws.close() + + def _rewrite_attach(self, text: str) -> str: + """Inject the adapter's connect address into `attach` requests. + + debugpy treats an attach with `connect` as a socket attach and then accepts later + re-attaches on the same listen port - which is what makes browser reloads work. + The browser cannot know the loopback port, so the bridge fills it in. + """ + try: + message = json.loads(text) + except ValueError: + return text + if message.get("type") == "request" and message.get("command") == "attach": + arguments = message.setdefault("arguments", {}) + arguments["connect"] = {"host": self.adapter_host, "port": self.adapter_port} + return json.dumps(message) + return text + + +def start_debug_server( + listen_port: int, + adapter_port: int, + frontend_dir: str, + brk: bool = False, + entry_path: str = "", +) -> ThreadingHTTPServer: + handler = type( + "BoundDebugRequestHandler", + (DebugRequestHandler,), + { + "frontend_dir": os.path.abspath(frontend_dir), + "adapter_port": adapter_port, + "brk": brk, + "entry_path": entry_path, + }, + ) + server = ThreadingHTTPServer(("0.0.0.0", listen_port), handler) + server.daemon_threads = True + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + return server diff --git a/python/src/actor_debugger/frontend/index.html b/python/src/actor_debugger/frontend/index.html new file mode 100644 index 0000000..942d7a6 --- /dev/null +++ b/python/src/actor_debugger/frontend/index.html @@ -0,0 +1,598 @@ + + + + + +Actor Debugger + + + + +
+
+ 🐛 Actor Debugger +
connecting…
+ + + + + + + + +
+ +

Sources

+ +
Waiting for the debug session…

+ Set breakpoints by clicking line numbers once a file is open. Files appear on the left as the + program loads them, or when it pauses.
+ +
+

Call stack

not paused
+

Variables

+

Breakpoints

+
+ + +
+
+
+ +
+
+
› +
+
+
+ + + + diff --git a/python/src/actor_debugger/websocket.py b/python/src/actor_debugger/websocket.py new file mode 100644 index 0000000..e729d4f --- /dev/null +++ b/python/src/actor_debugger/websocket.py @@ -0,0 +1,124 @@ +"""Minimal server-side RFC 6455 WebSocket implementation (stdlib only). + +Just enough for the debugger bridge: handshake, text frames, fragmentation, +ping/pong, and close. Server-to-client frames are never masked (per spec); +client-to-server frames must be masked and are rejected otherwise. +""" + +from __future__ import annotations + +import base64 +import hashlib +import socket +import struct +import threading + +_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" + +OP_CONT = 0x0 +OP_TEXT = 0x1 +OP_BINARY = 0x2 +OP_CLOSE = 0x8 +OP_PING = 0x9 +OP_PONG = 0xA + + +def accept_key(sec_websocket_key: str) -> str: + digest = hashlib.sha1((sec_websocket_key + _GUID).encode()).digest() + return base64.b64encode(digest).decode() + + +class WebSocketClosed(Exception): + pass + + +class WebSocketConnection: + """A WebSocket over an already-upgraded socket.""" + + def __init__(self, sock: socket.socket) -> None: + self._sock = sock + self._send_lock = threading.Lock() + self._closed = False + + def _read_exact(self, n: int) -> bytes: + chunks = b"" + while len(chunks) < n: + chunk = self._sock.recv(n - len(chunks)) + if not chunk: + raise WebSocketClosed() + chunks += chunk + return chunks + + def _read_frame(self) -> tuple[bool, int, bytes]: + header = self._read_exact(2) + fin = bool(header[0] & 0x80) + opcode = header[0] & 0x0F + masked = bool(header[1] & 0x80) + length = header[1] & 0x7F + if length == 126: + length = struct.unpack(">H", self._read_exact(2))[0] + elif length == 127: + length = struct.unpack(">Q", self._read_exact(8))[0] + if not masked: + raise WebSocketClosed() # client frames must be masked + mask = self._read_exact(4) + payload = bytearray(self._read_exact(length)) + for i in range(length): + payload[i] ^= mask[i % 4] + return fin, opcode, bytes(payload) + + def recv_text(self) -> str: + """Block until a full text/binary message arrives; handle control frames inline. + + Raises WebSocketClosed when the peer closes or the socket drops. + """ + message = b"" + message_opcode = None + while True: + fin, opcode, payload = self._read_frame() + if opcode == OP_PING: + self._send_frame(OP_PONG, payload) + continue + if opcode == OP_PONG: + continue + if opcode == OP_CLOSE: + self.close() + raise WebSocketClosed() + if opcode in (OP_TEXT, OP_BINARY): + message_opcode = opcode + message = payload + elif opcode == OP_CONT and message_opcode is not None: + message += payload + else: + raise WebSocketClosed() + if fin: + return message.decode("utf-8", errors="replace") + + def _send_frame(self, opcode: int, payload: bytes) -> None: + length = len(payload) + if length < 126: + header = struct.pack(">BB", 0x80 | opcode, length) + elif length < 1 << 16: + header = struct.pack(">BBH", 0x80 | opcode, 126, length) + else: + header = struct.pack(">BBQ", 0x80 | opcode, 127, length) + with self._send_lock: + self._sock.sendall(header + payload) + + def send_text(self, text: str) -> None: + if self._closed: + raise WebSocketClosed() + self._send_frame(OP_TEXT, text.encode()) + + def close(self) -> None: + if self._closed: + return + self._closed = True + try: + self._send_frame(OP_CLOSE, struct.pack(">H", 1000)) + except OSError: + pass + try: + self._sock.close() + except OSError: + pass From 3bf3b98d9ae2961a83f6528e9092f922af209771 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 8 Sep 2026 09:52:33 +0000 Subject: [PATCH 2/6] Make the Python example actor self-contained and fix idle-session drop The example previously required the unpublished actor-debugger PyPI package, so `apify push` failed at pip install. It now vendors the actor_debugger package (copied from python/src) and its requirements are only apify + debugpy - buildable and runnable on the platform today. The demo code is restructured to be worth debugging: nested calls, rich locals, a handled KeyError for the exception-breakpoint demo, slow ticks so there is always time to attach, plus an input schema and a README walkthrough (push, run, open the printed URL, what to try). Also fixes a real bug this surfaced: socket.create_connection(timeout=5) left a 5-second timeout on the established bridge-to-adapter socket, so any debug session idle for 5+ seconds was torn down by the pump thread's recv timeout. The timeout is now cleared after connect; sessions survive arbitrary idle (verified 45s+, plus the full protocol and headless- Chromium suites re-pass). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0194oFPjsTxztgsPhyB4Wq7G --- example-actor-python/.actor/actor.json | 11 +- example-actor-python/.actor/input_schema.json | 15 + example-actor-python/Dockerfile | 9 +- example-actor-python/README.md | 50 ++ .../actor_debugger/__init__.py | 3 + .../actor_debugger/__main__.py | 3 + example-actor-python/actor_debugger/cli.py | 145 +++++ .../actor_debugger/debug_server.py | 262 ++++++++ .../actor_debugger/frontend/index.html | 598 ++++++++++++++++++ .../actor_debugger/websocket.py | 124 ++++ example-actor-python/requirements.txt | 4 +- example-actor-python/src/main.py | 61 +- python/src/actor_debugger/debug_server.py | 4 + 13 files changed, 1282 insertions(+), 7 deletions(-) create mode 100644 example-actor-python/.actor/input_schema.json create mode 100644 example-actor-python/README.md create mode 100644 example-actor-python/actor_debugger/__init__.py create mode 100644 example-actor-python/actor_debugger/__main__.py create mode 100644 example-actor-python/actor_debugger/cli.py create mode 100644 example-actor-python/actor_debugger/debug_server.py create mode 100644 example-actor-python/actor_debugger/frontend/index.html create mode 100644 example-actor-python/actor_debugger/websocket.py diff --git a/example-actor-python/.actor/actor.json b/example-actor-python/.actor/actor.json index 59d09bd..74d8d8c 100644 --- a/example-actor-python/.actor/actor.json +++ b/example-actor-python/.actor/actor.json @@ -1 +1,10 @@ -{ "actorSpecification": 1, "name": "sample-py-actor", "version": "0.0", "buildTag": "latest", "dockerfile": "./Dockerfile" } +{ + "actorSpecification": 1, + "name": "sample-py-actor", + "title": "Python browser-debugger demo", + "description": "Sample Python Actor debuggable from a plain browser: the run log prints one URL, open it, set breakpoints.", + "version": "0.0", + "buildTag": "latest", + "dockerfile": "./Dockerfile", + "input": "./input_schema.json" +} diff --git a/example-actor-python/.actor/input_schema.json b/example-actor-python/.actor/input_schema.json new file mode 100644 index 0000000..7425d82 --- /dev/null +++ b/example-actor-python/.actor/input_schema.json @@ -0,0 +1,15 @@ +{ + "title": "Python browser-debugger demo", + "type": "object", + "schemaVersion": 1, + "properties": { + "ticks": { + "title": "Ticks", + "type": "integer", + "description": "How many 2-second processing ticks to run before finishing (keeps the run alive while you debug).", + "default": 120, + "minimum": 1, + "maximum": 10000 + } + } +} diff --git a/example-actor-python/Dockerfile b/example-actor-python/Dockerfile index 152fcf1..4472710 100644 --- a/example-actor-python/Dockerfile +++ b/example-actor-python/Dockerfile @@ -1,6 +1,13 @@ FROM apify/actor-python:3.13 + COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt + +# Copies the Actor source AND the vendored actor_debugger/ package (self-contained demo - +# once the package is on PyPI this becomes `pip install actor-debugger` instead). COPY . ./ -# One-line change enables the debugger (revert to the normal entrypoint to disable): + +# The one-line debug switch. `--brk` pauses on the first line until you attach from the +# browser; drop it to let the Actor run and attach mid-flight instead. +# To disable debugging, revert to: CMD ["python3", "-m", "src"] CMD ["python3", "-m", "actor_debugger", "--brk"] diff --git a/example-actor-python/README.md b/example-actor-python/README.md new file mode 100644 index 0000000..872b7aa --- /dev/null +++ b/example-actor-python/README.md @@ -0,0 +1,50 @@ +# Python browser-debugger demo (self-contained, testable today) + +A self-contained sample Actor showing the **browser-based Python debugger**: the run prints one +URL, you open it in a plain browser, and you get breakpoints, stepping, call stack, variables, +and an in-frame REPL against the live run. No IDE, no tunnel, nothing installed locally. + +This folder is fully standalone — the `actor_debugger/` package is **vendored** (copied from +[`../python/src/actor_debugger`](../python/src/actor_debugger)) so it builds without the package +being on PyPI. Once it's published, replace the vendored folder with `pip install actor-debugger`. + +## Test it in 3 steps + +```bash +cd example-actor-python +apify push # builds and deploys under your account +apify call # or press Run in the Console (default input: 120 ticks x 2 s) +``` + +Then open the run **log** and click the URL that `[actor-debugger]` prints: + +``` +[actor-debugger] OPEN THIS in your local browser for a full debugger UI (no local setup): +[actor-debugger] https://.runs.apify.net/ui/ +``` + +Because the Dockerfile uses `--brk`, the Actor waits, paused on its first line, until you open +that page — so you can't miss the start. Hit **▶ Resume** to let it run. + +## Things to try once attached + +1. Open `src/main.py` from the Sources panel and click the gutter on the `features = ...` line + inside `score_page` → next tick pauses there; inspect `page` and `features` in the Variables + panel. +2. Step **Into** `extract_features` (F11), step **Over** a few lines (F10), step **Out** + (Shift+F11). +3. In the console at the bottom, evaluate `sum(features.values())` — or mutate live state: + `page['depth'] = 99` and watch the pushed dataset item change. +4. Tick **break on raised exceptions**: every ~5th tick `flaky_lookup` raises (and handles) a + `KeyError`, and the debugger drops you on the raise site with the full stack. +5. Reload the browser tab mid-run — it re-attaches and your breakpoints survive. + +Meanwhile the run log keeps streaming `tick N: ...` lines and the dataset keeps filling — the +Actor behaves normally except when you pause it. + +## Turning the debugger off + +Edit the `Dockerfile` `CMD` back to `["python3", "-m", "src"]` and push again. **The debug +endpoint is unauthenticated** — anyone with the container URL can execute code in the run and +read its env (including `APIFY_TOKEN`) — so keep debug builds out of published Actors and prefer +runs under a restricted token. diff --git a/example-actor-python/actor_debugger/__init__.py b/example-actor-python/actor_debugger/__init__.py new file mode 100644 index 0000000..ab3bee4 --- /dev/null +++ b/example-actor-python/actor_debugger/__init__.py @@ -0,0 +1,3 @@ +"""actor-debugger: browser-based remote debugging for Apify Python Actors.""" + +__version__ = "0.1.0" diff --git a/example-actor-python/actor_debugger/__main__.py b/example-actor-python/actor_debugger/__main__.py new file mode 100644 index 0000000..4e28416 --- /dev/null +++ b/example-actor-python/actor_debugger/__main__.py @@ -0,0 +1,3 @@ +from .cli import main + +main() diff --git a/example-actor-python/actor_debugger/cli.py b/example-actor-python/actor_debugger/cli.py new file mode 100644 index 0000000..4dd8123 --- /dev/null +++ b/example-actor-python/actor_debugger/cli.py @@ -0,0 +1,145 @@ +"""actor-debugger - launch an Apify Python Actor under debugpy, debuggable from a plain +browser over the run's container URL, with a one-line Dockerfile change: + + CMD ["python3", "-m", "actor_debugger"] # auto-detect the Actor's entrypoint + CMD ["python3", "-m", "actor_debugger", "-m", "src"] # explicit module + CMD ["python3", "-m", "actor_debugger", "main.py"] # explicit file + CMD ["python3", "-m", "actor_debugger", "--brk"] # pause on the first line until attached + +Running through this command is what enables debugging - revert the CMD to the normal +entrypoint to turn it off. +""" + +from __future__ import annotations + +import os +import signal +import socket +import subprocess +import sys +import threading +import time + +from .debug_server import start_debug_server + +ADAPTER_PORT = 5678 +TAG = "[actor-debugger]" + + +def log(message: str) -> None: + print(f"{TAG} {message}", file=sys.stderr, flush=True) + + +def resolve_entry(args: list[str]) -> list[str] | None: + """Return the argv tail (["-m", "pkg"] or ["path.py"]) for the Actor's entrypoint.""" + if "-m" in args: + module = args[args.index("-m") + 1 :][:1] + if module: + return ["-m", module[0]] + positional = [a for a in args if not a.startswith("-")] + if positional: + return [os.path.abspath(positional[0])] + # The Apify Python templates run `python3 -m src` (src/__main__.py + src/main.py). + if os.path.isfile("src/__main__.py"): + return ["-m", "src"] + if os.path.isfile("src/main.py"): + return ["-m", "src.main"] + for candidate in ("main.py", "__main__.py", "app.py"): + if os.path.isfile(candidate): + return [os.path.abspath(candidate)] + return None + + +def entry_file_path(entry: list[str]) -> str: + """Best-effort path of the file that runs first, for the frontend's --brk entry stop.""" + if entry[0] != "-m": + return entry[0] + base = entry[1].replace(".", os.sep) + for candidate in (os.path.join(base, "__main__.py"), base + ".py"): + if os.path.isfile(candidate): + return os.path.abspath(candidate) + return "" + + +def wait_for_adapter(port: int, timeout: float = 30.0) -> bool: + """Wait until something listens on the adapter port WITHOUT connecting to it. + + debugpy treats the first accepted connection as its DAP client and shuts the session + down when it drops, so probing with a real connect would kill the adapter. A bind + attempt failing with EADDRINUSE detects the listener without ever touching it. + """ + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + try: + probe.bind(("127.0.0.1", port)) + except OSError: + return True + finally: + probe.close() + time.sleep(0.25) + return False + + +def announce(web_server_url: str) -> None: + if not wait_for_adapter(ADAPTER_PORT): + log(f"debugpy adapter did not come up on 127.0.0.1:{ADAPTER_PORT}.") + return + base = web_server_url.rstrip("/") + log("=" * 64) + log("debugging is ON - reachable over the container URL.") + log("OPEN THIS in your local browser for a full debugger UI (no local setup):") + log(f" {base}/ui/") + host = base.split("://", 1)[-1] + log(f"raw DAP-over-WebSocket channel (one JSON message per frame): wss://{host}/dap") + log("=" * 64) + + +def main() -> None: + args = sys.argv[1:] + brk = "--brk" in args + entry = resolve_entry([a for a in args if a != "--brk"]) + if not entry: + log("could not find an Actor entrypoint. Pass one explicitly:") + log(' CMD ["python3", "-m", "actor_debugger", "-m", "src"]') + sys.exit(1) + + debugpy_argv = [ + sys.executable, + "-Xfrozen_modules=off", + "-m", + "debugpy", + "--listen", + f"127.0.0.1:{ADAPTER_PORT}", + *(["--wait-for-client"] if brk else []), + *entry, + ] + env = {**os.environ, "PYDEVD_DISABLE_FILE_VALIDATION": "1"} + child = subprocess.Popen(debugpy_argv, env=env) + + server = None + port = os.environ.get("ACTOR_WEB_SERVER_PORT") + url = os.environ.get("ACTOR_WEB_SERVER_URL") + if port and url: + frontend_dir = os.path.join(os.path.dirname(__file__), "frontend") + server = start_debug_server(int(port), ADAPTER_PORT, frontend_dir, brk=brk, entry_path=entry_file_path(entry)) + threading.Thread(target=announce, args=(url,), daemon=True).start() + if brk: + log("--brk: the Actor is paused on its first line until you attach from the browser.") + else: + log(f"ACTOR_WEB_SERVER_URL/PORT not set - DAP on 127.0.0.1:{ADAPTER_PORT} only (no container-URL bridge).") + + def forward(signum, _frame): + child.send_signal(signum) + + signal.signal(signal.SIGTERM, forward) + signal.signal(signal.SIGINT, forward) + + code = child.wait() + if server: + server.shutdown() + sys.exit(code) + + +if __name__ == "__main__": + main() diff --git a/example-actor-python/actor_debugger/debug_server.py b/example-actor-python/actor_debugger/debug_server.py new file mode 100644 index 0000000..da7d729 --- /dev/null +++ b/example-actor-python/actor_debugger/debug_server.py @@ -0,0 +1,262 @@ +"""One HTTP server on the container's web-server port that: + +- serves the bundled browser debugger UI (static files) so your LOCAL browser opens it - + no IDE, no local setup; +- bridges a WebSocket at /dap to the debugpy adapter's DAP-over-TCP socket on loopback, + translating framing (one WebSocket text message per DAP JSON message <-> Content-Length + framed TCP) so the browser client stays trivial; +- rewrites `attach` requests in flight to inject the adapter's loopback connect address, + which the browser cannot know (and which is what makes debugpy accept re-attach after + a page reload); +- serves /source?path=... so the remote frontend can display files that only exist on the + container's disk (a browser can never fetch file:// URLs from the container); +- serves /config with the working directory so the frontend can group project files apart + from library code. +""" + +from __future__ import annotations + +import json +import mimetypes +import os +import socket +import threading +import time +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from urllib.parse import parse_qs, urlparse + +from .websocket import WebSocketConnection, WebSocketClosed, accept_key + +UI_PREFIX = "/ui/" + + +def _read_dap_messages(sock: socket.socket): + """Yield DAP message bodies (bytes) from a Content-Length framed TCP stream.""" + buf = b"" + while True: + while b"\r\n\r\n" not in buf: + chunk = sock.recv(65536) + if not chunk: + return + buf += chunk + header, buf = buf.split(b"\r\n\r\n", 1) + length = None + for line in header.split(b"\r\n"): + name, _, value = line.partition(b":") + if name.strip().lower() == b"content-length": + length = int(value.strip()) + if length is None: + return + while len(buf) < length: + chunk = sock.recv(65536) + if not chunk: + return + buf += chunk + body, buf = buf[:length], buf[length:] + yield body + + +class DebugRequestHandler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + + # Injected by start_debug_server(): + frontend_dir: str = "" + adapter_host: str = "127.0.0.1" + adapter_port: int = 5678 + brk: bool = False + entry_path: str = "" + + def log_message(self, format, *args): # noqa: A002 - stdlib signature + pass # keep the Actor log clean + + # -- HTTP --------------------------------------------------------------- + + def do_GET(self): # noqa: N802 - stdlib naming + parsed = urlparse(self.path) + if parsed.path == "/dap" and "websocket" in self.headers.get("Upgrade", "").lower(): + self._handle_websocket() + return + if parsed.path in ("", "/"): + self.send_response(302) + self.send_header("Location", f"{UI_PREFIX}") + self.send_header("Content-Length", "0") + self.end_headers() + return + if parsed.path == "/config": + self._send_json({"cwd": os.getcwd(), "brk": self.brk, "entryPath": self.entry_path}) + return + if parsed.path == "/source": + self._serve_source(parsed) + return + self._serve_static(parsed.path) + + def _send_json(self, payload: dict) -> None: + body = json.dumps(payload).encode() + self.send_response(200) + self.send_header("Content-Type", "application/json; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _send_plain(self, status: int, text: str) -> None: + body = text.encode() + self.send_response(status) + self.send_header("Content-Type", "text/plain; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _serve_source(self, parsed) -> None: + path = (parse_qs(parsed.query).get("path") or [""])[0] + if not path or not os.path.isabs(path) or not os.path.isfile(path): + self._send_plain(404, "not found") + return + try: + with open(path, "rb") as f: + body = f.read(4 * 1024 * 1024) + except OSError: + self._send_plain(404, "not found") + return + self.send_response(200) + self.send_header("Content-Type", "text/plain; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _serve_static(self, url_path: str) -> None: + rel = url_path[len(UI_PREFIX):] if url_path.startswith(UI_PREFIX) else url_path.lstrip("/") + rel = rel or "index.html" + file_path = os.path.normpath(os.path.join(self.frontend_dir, rel)) + if not file_path.startswith(os.path.abspath(self.frontend_dir)): + self._send_plain(403, "forbidden") + return + if not os.path.isfile(file_path): + self._send_plain(404, "not found") + return + ctype = mimetypes.guess_type(file_path)[0] or "application/octet-stream" + with open(file_path, "rb") as f: + body = f.read() + self.send_response(200) + self.send_header("Content-Type", ctype) + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + # -- WebSocket <-> DAP bridge -------------------------------------------- + + def _handle_websocket(self) -> None: + key = self.headers.get("Sec-WebSocket-Key") + if not key: + self._send_plain(400, "bad websocket handshake") + return + upstream = None + deadline = time.monotonic() + 15 + while upstream is None: + try: + upstream = socket.create_connection((self.adapter_host, self.adapter_port), timeout=5) + except OSError: + if time.monotonic() >= deadline: + self._send_plain(502, "debug adapter unavailable") + return + time.sleep(0.5) + # The connect timeout must not linger on the established socket: it would make the + # idle recv() in the pump thread time out and tear the session down after 5 quiet + # seconds. A debug session idles for minutes while the developer reads code. + upstream.settimeout(None) + self.connection.sendall( + b"HTTP/1.1 101 Switching Protocols\r\n" + b"Upgrade: websocket\r\n" + b"Connection: Upgrade\r\n" + b"Sec-WebSocket-Accept: " + accept_key(key).encode() + b"\r\n\r\n" + ) + self.close_connection = True + ws = WebSocketConnection(self.connection) + + def pump_adapter_to_browser(): + try: + for body in _read_dap_messages(upstream): + ws.send_text(body.decode("utf-8", errors="replace")) + except (OSError, WebSocketClosed): + pass + finally: + ws.close() + + pump = threading.Thread(target=pump_adapter_to_browser, daemon=True) + pump.start() + saw_disconnect = False + try: + while True: + text = ws.recv_text() + if '"disconnect"' in text: + try: + parsed = json.loads(text) + if parsed.get("type") == "request" and parsed.get("command") == "disconnect": + saw_disconnect = True + except ValueError: + pass + body = self._rewrite_attach(text).encode() + upstream.sendall(b"Content-Length: %d\r\n\r\n%b" % (len(body), body)) + except (OSError, WebSocketClosed): + pass + finally: + if not saw_disconnect: + # The browser vanished (tab closed, page reloaded) without a DAP disconnect. + # An abruptly dropped client makes debugpy's adapter stop listening, killing + # re-attach - so say goodbye on the browser's behalf. + goodbye = json.dumps({ + "seq": 2_000_000_000, + "type": "request", + "command": "disconnect", + "arguments": {"terminateDebuggee": False}, + }).encode() + try: + upstream.sendall(b"Content-Length: %d\r\n\r\n%b" % (len(goodbye), goodbye)) + time.sleep(0.5) + except OSError: + pass + try: + upstream.close() + except OSError: + pass + ws.close() + + def _rewrite_attach(self, text: str) -> str: + """Inject the adapter's connect address into `attach` requests. + + debugpy treats an attach with `connect` as a socket attach and then accepts later + re-attaches on the same listen port - which is what makes browser reloads work. + The browser cannot know the loopback port, so the bridge fills it in. + """ + try: + message = json.loads(text) + except ValueError: + return text + if message.get("type") == "request" and message.get("command") == "attach": + arguments = message.setdefault("arguments", {}) + arguments["connect"] = {"host": self.adapter_host, "port": self.adapter_port} + return json.dumps(message) + return text + + +def start_debug_server( + listen_port: int, + adapter_port: int, + frontend_dir: str, + brk: bool = False, + entry_path: str = "", +) -> ThreadingHTTPServer: + handler = type( + "BoundDebugRequestHandler", + (DebugRequestHandler,), + { + "frontend_dir": os.path.abspath(frontend_dir), + "adapter_port": adapter_port, + "brk": brk, + "entry_path": entry_path, + }, + ) + server = ThreadingHTTPServer(("0.0.0.0", listen_port), handler) + server.daemon_threads = True + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + return server diff --git a/example-actor-python/actor_debugger/frontend/index.html b/example-actor-python/actor_debugger/frontend/index.html new file mode 100644 index 0000000..942d7a6 --- /dev/null +++ b/example-actor-python/actor_debugger/frontend/index.html @@ -0,0 +1,598 @@ + + + + + +Actor Debugger + + + + +
+
+ 🐛 Actor Debugger +
connecting…
+ + + + + + + + +
+ +

Sources

+ +
Waiting for the debug session…

+ Set breakpoints by clicking line numbers once a file is open. Files appear on the left as the + program loads them, or when it pauses.
+ +
+

Call stack

not paused
+

Variables

+

Breakpoints

+
+ + +
+
+
+ +
+
+
› +
+
+
+ + + + diff --git a/example-actor-python/actor_debugger/websocket.py b/example-actor-python/actor_debugger/websocket.py new file mode 100644 index 0000000..e729d4f --- /dev/null +++ b/example-actor-python/actor_debugger/websocket.py @@ -0,0 +1,124 @@ +"""Minimal server-side RFC 6455 WebSocket implementation (stdlib only). + +Just enough for the debugger bridge: handshake, text frames, fragmentation, +ping/pong, and close. Server-to-client frames are never masked (per spec); +client-to-server frames must be masked and are rejected otherwise. +""" + +from __future__ import annotations + +import base64 +import hashlib +import socket +import struct +import threading + +_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" + +OP_CONT = 0x0 +OP_TEXT = 0x1 +OP_BINARY = 0x2 +OP_CLOSE = 0x8 +OP_PING = 0x9 +OP_PONG = 0xA + + +def accept_key(sec_websocket_key: str) -> str: + digest = hashlib.sha1((sec_websocket_key + _GUID).encode()).digest() + return base64.b64encode(digest).decode() + + +class WebSocketClosed(Exception): + pass + + +class WebSocketConnection: + """A WebSocket over an already-upgraded socket.""" + + def __init__(self, sock: socket.socket) -> None: + self._sock = sock + self._send_lock = threading.Lock() + self._closed = False + + def _read_exact(self, n: int) -> bytes: + chunks = b"" + while len(chunks) < n: + chunk = self._sock.recv(n - len(chunks)) + if not chunk: + raise WebSocketClosed() + chunks += chunk + return chunks + + def _read_frame(self) -> tuple[bool, int, bytes]: + header = self._read_exact(2) + fin = bool(header[0] & 0x80) + opcode = header[0] & 0x0F + masked = bool(header[1] & 0x80) + length = header[1] & 0x7F + if length == 126: + length = struct.unpack(">H", self._read_exact(2))[0] + elif length == 127: + length = struct.unpack(">Q", self._read_exact(8))[0] + if not masked: + raise WebSocketClosed() # client frames must be masked + mask = self._read_exact(4) + payload = bytearray(self._read_exact(length)) + for i in range(length): + payload[i] ^= mask[i % 4] + return fin, opcode, bytes(payload) + + def recv_text(self) -> str: + """Block until a full text/binary message arrives; handle control frames inline. + + Raises WebSocketClosed when the peer closes or the socket drops. + """ + message = b"" + message_opcode = None + while True: + fin, opcode, payload = self._read_frame() + if opcode == OP_PING: + self._send_frame(OP_PONG, payload) + continue + if opcode == OP_PONG: + continue + if opcode == OP_CLOSE: + self.close() + raise WebSocketClosed() + if opcode in (OP_TEXT, OP_BINARY): + message_opcode = opcode + message = payload + elif opcode == OP_CONT and message_opcode is not None: + message += payload + else: + raise WebSocketClosed() + if fin: + return message.decode("utf-8", errors="replace") + + def _send_frame(self, opcode: int, payload: bytes) -> None: + length = len(payload) + if length < 126: + header = struct.pack(">BB", 0x80 | opcode, length) + elif length < 1 << 16: + header = struct.pack(">BBH", 0x80 | opcode, 126, length) + else: + header = struct.pack(">BBQ", 0x80 | opcode, 127, length) + with self._send_lock: + self._sock.sendall(header + payload) + + def send_text(self, text: str) -> None: + if self._closed: + raise WebSocketClosed() + self._send_frame(OP_TEXT, text.encode()) + + def close(self) -> None: + if self._closed: + return + self._closed = True + try: + self._send_frame(OP_CLOSE, struct.pack(">H", 1000)) + except OSError: + pass + try: + self._sock.close() + except OSError: + pass diff --git a/example-actor-python/requirements.txt b/example-actor-python/requirements.txt index 7bed89e..4bfcfd5 100644 --- a/example-actor-python/requirements.txt +++ b/example-actor-python/requirements.txt @@ -1,2 +1,2 @@ -apify -actor-debugger +apify<3.0.0 +debugpy>=1.8 diff --git a/example-actor-python/src/main.py b/example-actor-python/src/main.py index 1632d4c..c849246 100644 --- a/example-actor-python/src/main.py +++ b/example-actor-python/src/main.py @@ -1,10 +1,65 @@ +"""Demo Actor for the browser debugger - deliberately structured to be fun to step through. + +Try this once the debugger UI is open: + * breakpoint inside `score_page` and watch `features` build up across ticks, + * step INTO `extract_features` / OUT again, + * evaluate `sum(features.values())` or mutate `page['depth']` in the console, + * tick "break on raised exceptions" - `flaky_lookup` raises (and handles) KeyError + every 5th tick, so the debugger will drop you on the raise site. +""" + import asyncio +import random from apify import Actor +FAKE_PAGES = [ + {'url': 'https://example.com/', 'depth': 0, 'html': '

Welcome

'}, + {'url': 'https://example.com/a', 'depth': 1, 'html': '

Products

'}, + {'url': 'https://example.com/b', 'depth': 1, 'html': '

Blog

hello world

'}, + {'url': 'https://example.com/a/1', 'depth': 2, 'html': '

Item 1

$42'}, +] + + +def extract_features(page: dict) -> dict: + html = page['html'] + return { + 'headings': sum(html.count(tag) for tag in (' str: + try: + return catalog[key] # raises KeyError for unknown keys - nice exception-breakpoint demo + except KeyError: + return 'uncategorized' + + +def score_page(page: dict, tick: int) -> dict: + features = extract_features(page) + catalog = {'https://example.com/a': 'catalog', 'https://example.com/a/1': 'product'} + category = flaky_lookup(catalog, page['url']) + score = features['headings'] * 10 + features['links'] * 5 + features['length'] // 10 + if features['has_price']: + score *= 2 + return {'url': page['url'], 'tick': tick, 'category': category, 'score': score, **features} + async def main() -> None: async with Actor: - for tick in range(1, 61): - Actor.log.info(f'sample tick {tick}') - await asyncio.sleep(1) + actor_input = await Actor.get_input() or {} + ticks = int(actor_input.get('ticks', 120)) + Actor.log.info(f'Demo Actor started - will process pages for {ticks} ticks.') + Actor.log.info('Open the debugger URL printed above by [actor-debugger].') + + for tick in range(1, ticks + 1): + page = random.choice(FAKE_PAGES) + result = score_page(page, tick) + await Actor.push_data(result) + Actor.log.info(f"tick {tick}: {result['url']} -> score {result['score']} ({result['category']})") + await asyncio.sleep(2) + + Actor.log.info('Demo Actor finished.') diff --git a/python/src/actor_debugger/debug_server.py b/python/src/actor_debugger/debug_server.py index 39bf5bc..da7d729 100644 --- a/python/src/actor_debugger/debug_server.py +++ b/python/src/actor_debugger/debug_server.py @@ -159,6 +159,10 @@ def _handle_websocket(self) -> None: self._send_plain(502, "debug adapter unavailable") return time.sleep(0.5) + # The connect timeout must not linger on the established socket: it would make the + # idle recv() in the pump thread time out and tear the session down after 5 quiet + # seconds. A debug session idles for minutes while the developer reads code. + upstream.settimeout(None) self.connection.sendall( b"HTTP/1.1 101 Switching Protocols\r\n" b"Upgrade: websocket\r\n" From 33cb36337a5af8b431bbec6b2991eeb0e561e449 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 8 Sep 2026 09:57:57 +0000 Subject: [PATCH 3/6] Pin the example actor to apify>=4,<5 The previous apify<3.0.0 pin resolved to apify 2.x/crawlee 0.x, whose pydantic models fail to import under current pydantic 2.12+ ("cannot specify both default and default_factory") - the crash seen on the platform the moment the debugged run resumed past --brk. Reproduced locally with apify<3 and verified apify 4.0.2 imports cleanly. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0194oFPjsTxztgsPhyB4Wq7G --- example-actor-python/requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/example-actor-python/requirements.txt b/example-actor-python/requirements.txt index 4bfcfd5..d8557ca 100644 --- a/example-actor-python/requirements.txt +++ b/example-actor-python/requirements.txt @@ -1,2 +1,2 @@ -apify<3.0.0 +apify>=4.0.0,<5.0.0 debugpy>=1.8 From 5337d975b2b4a6d2557099b525c5f5035e9aec33 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 8 Sep 2026 11:06:03 +0000 Subject: [PATCH 4/6] Drop the sample Python actor; ship the debugger as a standalone pip package The Python debugger is now delivered only as the pip package in python/, injected into any Python Actor the same way as the npm version: RUN pip install actor-debugger CMD ["python3", "-m", "actor_debugger", "--brk"] Docs rewritten injection-first: python/README.md leads with the two-line Dockerfile change (plus a git-install line until the PyPI release), keeps the CLI forms, architecture, security notes and design alternatives, and records that the stack is verified on the Apify platform (UI and DAP-over-WebSocket through the run ingress against a live run). Root README pointer updated; pyproject homepage moved to apify/actor-debugger. Verified: wheel builds with the frontend bundled, installs into a clean venv, and the full DAP bridge test suite passes against that install. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0194oFPjsTxztgsPhyB4Wq7G --- README.md | 7 +- example-actor-python/.actor/actor.json | 10 - example-actor-python/.actor/input_schema.json | 15 - example-actor-python/Dockerfile | 13 - example-actor-python/README.md | 50 -- .../actor_debugger/__init__.py | 3 - .../actor_debugger/__main__.py | 3 - example-actor-python/actor_debugger/cli.py | 145 ----- .../actor_debugger/debug_server.py | 262 -------- .../actor_debugger/frontend/index.html | 598 ------------------ .../actor_debugger/websocket.py | 124 ---- example-actor-python/requirements.txt | 2 - example-actor-python/src/__init__.py | 0 example-actor-python/src/__main__.py | 5 - example-actor-python/src/main.py | 65 -- python/README.md | 192 +++--- python/pyproject.toml | 2 +- 17 files changed, 106 insertions(+), 1390 deletions(-) delete mode 100644 example-actor-python/.actor/actor.json delete mode 100644 example-actor-python/.actor/input_schema.json delete mode 100644 example-actor-python/Dockerfile delete mode 100644 example-actor-python/README.md delete mode 100644 example-actor-python/actor_debugger/__init__.py delete mode 100644 example-actor-python/actor_debugger/__main__.py delete mode 100644 example-actor-python/actor_debugger/cli.py delete mode 100644 example-actor-python/actor_debugger/debug_server.py delete mode 100644 example-actor-python/actor_debugger/frontend/index.html delete mode 100644 example-actor-python/actor_debugger/websocket.py delete mode 100644 example-actor-python/requirements.txt delete mode 100644 example-actor-python/src/__init__.py delete mode 100644 example-actor-python/src/__main__.py delete mode 100644 example-actor-python/src/main.py diff --git a/README.md b/README.md index 1093e66..95b389f 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,9 @@ # actor-debugger -> **Python Actors:** the same one-link browser debugging exists for Python — debugpy plus a served -> DAP frontend instead of the Node inspector plus Chrome DevTools. See -> [`python/README.md`](python/README.md) and the sample [`example-actor-python/`](example-actor-python/). +> **Python Actors:** the same one-link browser debugging exists for Python, injected the same way — +> `RUN pip install actor-debugger` + `CMD ["python3", "-m", "actor_debugger", "--brk"]` — built on +> debugpy plus a served DAP frontend instead of the Node inspector plus Chrome DevTools. +> See [`python/README.md`](python/README.md). Drop-in remote debugging for **any Apify Node/TS Actor** with a one-line Dockerfile change. It launches your Actor under the Node inspector **and serves a full Chrome DevTools UI over the run's diff --git a/example-actor-python/.actor/actor.json b/example-actor-python/.actor/actor.json deleted file mode 100644 index 74d8d8c..0000000 --- a/example-actor-python/.actor/actor.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "actorSpecification": 1, - "name": "sample-py-actor", - "title": "Python browser-debugger demo", - "description": "Sample Python Actor debuggable from a plain browser: the run log prints one URL, open it, set breakpoints.", - "version": "0.0", - "buildTag": "latest", - "dockerfile": "./Dockerfile", - "input": "./input_schema.json" -} diff --git a/example-actor-python/.actor/input_schema.json b/example-actor-python/.actor/input_schema.json deleted file mode 100644 index 7425d82..0000000 --- a/example-actor-python/.actor/input_schema.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "title": "Python browser-debugger demo", - "type": "object", - "schemaVersion": 1, - "properties": { - "ticks": { - "title": "Ticks", - "type": "integer", - "description": "How many 2-second processing ticks to run before finishing (keeps the run alive while you debug).", - "default": 120, - "minimum": 1, - "maximum": 10000 - } - } -} diff --git a/example-actor-python/Dockerfile b/example-actor-python/Dockerfile deleted file mode 100644 index 4472710..0000000 --- a/example-actor-python/Dockerfile +++ /dev/null @@ -1,13 +0,0 @@ -FROM apify/actor-python:3.13 - -COPY requirements.txt ./ -RUN pip install --no-cache-dir -r requirements.txt - -# Copies the Actor source AND the vendored actor_debugger/ package (self-contained demo - -# once the package is on PyPI this becomes `pip install actor-debugger` instead). -COPY . ./ - -# The one-line debug switch. `--brk` pauses on the first line until you attach from the -# browser; drop it to let the Actor run and attach mid-flight instead. -# To disable debugging, revert to: CMD ["python3", "-m", "src"] -CMD ["python3", "-m", "actor_debugger", "--brk"] diff --git a/example-actor-python/README.md b/example-actor-python/README.md deleted file mode 100644 index 872b7aa..0000000 --- a/example-actor-python/README.md +++ /dev/null @@ -1,50 +0,0 @@ -# Python browser-debugger demo (self-contained, testable today) - -A self-contained sample Actor showing the **browser-based Python debugger**: the run prints one -URL, you open it in a plain browser, and you get breakpoints, stepping, call stack, variables, -and an in-frame REPL against the live run. No IDE, no tunnel, nothing installed locally. - -This folder is fully standalone — the `actor_debugger/` package is **vendored** (copied from -[`../python/src/actor_debugger`](../python/src/actor_debugger)) so it builds without the package -being on PyPI. Once it's published, replace the vendored folder with `pip install actor-debugger`. - -## Test it in 3 steps - -```bash -cd example-actor-python -apify push # builds and deploys under your account -apify call # or press Run in the Console (default input: 120 ticks x 2 s) -``` - -Then open the run **log** and click the URL that `[actor-debugger]` prints: - -``` -[actor-debugger] OPEN THIS in your local browser for a full debugger UI (no local setup): -[actor-debugger] https://.runs.apify.net/ui/ -``` - -Because the Dockerfile uses `--brk`, the Actor waits, paused on its first line, until you open -that page — so you can't miss the start. Hit **▶ Resume** to let it run. - -## Things to try once attached - -1. Open `src/main.py` from the Sources panel and click the gutter on the `features = ...` line - inside `score_page` → next tick pauses there; inspect `page` and `features` in the Variables - panel. -2. Step **Into** `extract_features` (F11), step **Over** a few lines (F10), step **Out** - (Shift+F11). -3. In the console at the bottom, evaluate `sum(features.values())` — or mutate live state: - `page['depth'] = 99` and watch the pushed dataset item change. -4. Tick **break on raised exceptions**: every ~5th tick `flaky_lookup` raises (and handles) a - `KeyError`, and the debugger drops you on the raise site with the full stack. -5. Reload the browser tab mid-run — it re-attaches and your breakpoints survive. - -Meanwhile the run log keeps streaming `tick N: ...` lines and the dataset keeps filling — the -Actor behaves normally except when you pause it. - -## Turning the debugger off - -Edit the `Dockerfile` `CMD` back to `["python3", "-m", "src"]` and push again. **The debug -endpoint is unauthenticated** — anyone with the container URL can execute code in the run and -read its env (including `APIFY_TOKEN`) — so keep debug builds out of published Actors and prefer -runs under a restricted token. diff --git a/example-actor-python/actor_debugger/__init__.py b/example-actor-python/actor_debugger/__init__.py deleted file mode 100644 index ab3bee4..0000000 --- a/example-actor-python/actor_debugger/__init__.py +++ /dev/null @@ -1,3 +0,0 @@ -"""actor-debugger: browser-based remote debugging for Apify Python Actors.""" - -__version__ = "0.1.0" diff --git a/example-actor-python/actor_debugger/__main__.py b/example-actor-python/actor_debugger/__main__.py deleted file mode 100644 index 4e28416..0000000 --- a/example-actor-python/actor_debugger/__main__.py +++ /dev/null @@ -1,3 +0,0 @@ -from .cli import main - -main() diff --git a/example-actor-python/actor_debugger/cli.py b/example-actor-python/actor_debugger/cli.py deleted file mode 100644 index 4dd8123..0000000 --- a/example-actor-python/actor_debugger/cli.py +++ /dev/null @@ -1,145 +0,0 @@ -"""actor-debugger - launch an Apify Python Actor under debugpy, debuggable from a plain -browser over the run's container URL, with a one-line Dockerfile change: - - CMD ["python3", "-m", "actor_debugger"] # auto-detect the Actor's entrypoint - CMD ["python3", "-m", "actor_debugger", "-m", "src"] # explicit module - CMD ["python3", "-m", "actor_debugger", "main.py"] # explicit file - CMD ["python3", "-m", "actor_debugger", "--brk"] # pause on the first line until attached - -Running through this command is what enables debugging - revert the CMD to the normal -entrypoint to turn it off. -""" - -from __future__ import annotations - -import os -import signal -import socket -import subprocess -import sys -import threading -import time - -from .debug_server import start_debug_server - -ADAPTER_PORT = 5678 -TAG = "[actor-debugger]" - - -def log(message: str) -> None: - print(f"{TAG} {message}", file=sys.stderr, flush=True) - - -def resolve_entry(args: list[str]) -> list[str] | None: - """Return the argv tail (["-m", "pkg"] or ["path.py"]) for the Actor's entrypoint.""" - if "-m" in args: - module = args[args.index("-m") + 1 :][:1] - if module: - return ["-m", module[0]] - positional = [a for a in args if not a.startswith("-")] - if positional: - return [os.path.abspath(positional[0])] - # The Apify Python templates run `python3 -m src` (src/__main__.py + src/main.py). - if os.path.isfile("src/__main__.py"): - return ["-m", "src"] - if os.path.isfile("src/main.py"): - return ["-m", "src.main"] - for candidate in ("main.py", "__main__.py", "app.py"): - if os.path.isfile(candidate): - return [os.path.abspath(candidate)] - return None - - -def entry_file_path(entry: list[str]) -> str: - """Best-effort path of the file that runs first, for the frontend's --brk entry stop.""" - if entry[0] != "-m": - return entry[0] - base = entry[1].replace(".", os.sep) - for candidate in (os.path.join(base, "__main__.py"), base + ".py"): - if os.path.isfile(candidate): - return os.path.abspath(candidate) - return "" - - -def wait_for_adapter(port: int, timeout: float = 30.0) -> bool: - """Wait until something listens on the adapter port WITHOUT connecting to it. - - debugpy treats the first accepted connection as its DAP client and shuts the session - down when it drops, so probing with a real connect would kill the adapter. A bind - attempt failing with EADDRINUSE detects the listener without ever touching it. - """ - deadline = time.monotonic() + timeout - while time.monotonic() < deadline: - probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - try: - probe.bind(("127.0.0.1", port)) - except OSError: - return True - finally: - probe.close() - time.sleep(0.25) - return False - - -def announce(web_server_url: str) -> None: - if not wait_for_adapter(ADAPTER_PORT): - log(f"debugpy adapter did not come up on 127.0.0.1:{ADAPTER_PORT}.") - return - base = web_server_url.rstrip("/") - log("=" * 64) - log("debugging is ON - reachable over the container URL.") - log("OPEN THIS in your local browser for a full debugger UI (no local setup):") - log(f" {base}/ui/") - host = base.split("://", 1)[-1] - log(f"raw DAP-over-WebSocket channel (one JSON message per frame): wss://{host}/dap") - log("=" * 64) - - -def main() -> None: - args = sys.argv[1:] - brk = "--brk" in args - entry = resolve_entry([a for a in args if a != "--brk"]) - if not entry: - log("could not find an Actor entrypoint. Pass one explicitly:") - log(' CMD ["python3", "-m", "actor_debugger", "-m", "src"]') - sys.exit(1) - - debugpy_argv = [ - sys.executable, - "-Xfrozen_modules=off", - "-m", - "debugpy", - "--listen", - f"127.0.0.1:{ADAPTER_PORT}", - *(["--wait-for-client"] if brk else []), - *entry, - ] - env = {**os.environ, "PYDEVD_DISABLE_FILE_VALIDATION": "1"} - child = subprocess.Popen(debugpy_argv, env=env) - - server = None - port = os.environ.get("ACTOR_WEB_SERVER_PORT") - url = os.environ.get("ACTOR_WEB_SERVER_URL") - if port and url: - frontend_dir = os.path.join(os.path.dirname(__file__), "frontend") - server = start_debug_server(int(port), ADAPTER_PORT, frontend_dir, brk=brk, entry_path=entry_file_path(entry)) - threading.Thread(target=announce, args=(url,), daemon=True).start() - if brk: - log("--brk: the Actor is paused on its first line until you attach from the browser.") - else: - log(f"ACTOR_WEB_SERVER_URL/PORT not set - DAP on 127.0.0.1:{ADAPTER_PORT} only (no container-URL bridge).") - - def forward(signum, _frame): - child.send_signal(signum) - - signal.signal(signal.SIGTERM, forward) - signal.signal(signal.SIGINT, forward) - - code = child.wait() - if server: - server.shutdown() - sys.exit(code) - - -if __name__ == "__main__": - main() diff --git a/example-actor-python/actor_debugger/debug_server.py b/example-actor-python/actor_debugger/debug_server.py deleted file mode 100644 index da7d729..0000000 --- a/example-actor-python/actor_debugger/debug_server.py +++ /dev/null @@ -1,262 +0,0 @@ -"""One HTTP server on the container's web-server port that: - -- serves the bundled browser debugger UI (static files) so your LOCAL browser opens it - - no IDE, no local setup; -- bridges a WebSocket at /dap to the debugpy adapter's DAP-over-TCP socket on loopback, - translating framing (one WebSocket text message per DAP JSON message <-> Content-Length - framed TCP) so the browser client stays trivial; -- rewrites `attach` requests in flight to inject the adapter's loopback connect address, - which the browser cannot know (and which is what makes debugpy accept re-attach after - a page reload); -- serves /source?path=... so the remote frontend can display files that only exist on the - container's disk (a browser can never fetch file:// URLs from the container); -- serves /config with the working directory so the frontend can group project files apart - from library code. -""" - -from __future__ import annotations - -import json -import mimetypes -import os -import socket -import threading -import time -from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer -from urllib.parse import parse_qs, urlparse - -from .websocket import WebSocketConnection, WebSocketClosed, accept_key - -UI_PREFIX = "/ui/" - - -def _read_dap_messages(sock: socket.socket): - """Yield DAP message bodies (bytes) from a Content-Length framed TCP stream.""" - buf = b"" - while True: - while b"\r\n\r\n" not in buf: - chunk = sock.recv(65536) - if not chunk: - return - buf += chunk - header, buf = buf.split(b"\r\n\r\n", 1) - length = None - for line in header.split(b"\r\n"): - name, _, value = line.partition(b":") - if name.strip().lower() == b"content-length": - length = int(value.strip()) - if length is None: - return - while len(buf) < length: - chunk = sock.recv(65536) - if not chunk: - return - buf += chunk - body, buf = buf[:length], buf[length:] - yield body - - -class DebugRequestHandler(BaseHTTPRequestHandler): - protocol_version = "HTTP/1.1" - - # Injected by start_debug_server(): - frontend_dir: str = "" - adapter_host: str = "127.0.0.1" - adapter_port: int = 5678 - brk: bool = False - entry_path: str = "" - - def log_message(self, format, *args): # noqa: A002 - stdlib signature - pass # keep the Actor log clean - - # -- HTTP --------------------------------------------------------------- - - def do_GET(self): # noqa: N802 - stdlib naming - parsed = urlparse(self.path) - if parsed.path == "/dap" and "websocket" in self.headers.get("Upgrade", "").lower(): - self._handle_websocket() - return - if parsed.path in ("", "/"): - self.send_response(302) - self.send_header("Location", f"{UI_PREFIX}") - self.send_header("Content-Length", "0") - self.end_headers() - return - if parsed.path == "/config": - self._send_json({"cwd": os.getcwd(), "brk": self.brk, "entryPath": self.entry_path}) - return - if parsed.path == "/source": - self._serve_source(parsed) - return - self._serve_static(parsed.path) - - def _send_json(self, payload: dict) -> None: - body = json.dumps(payload).encode() - self.send_response(200) - self.send_header("Content-Type", "application/json; charset=utf-8") - self.send_header("Content-Length", str(len(body))) - self.end_headers() - self.wfile.write(body) - - def _send_plain(self, status: int, text: str) -> None: - body = text.encode() - self.send_response(status) - self.send_header("Content-Type", "text/plain; charset=utf-8") - self.send_header("Content-Length", str(len(body))) - self.end_headers() - self.wfile.write(body) - - def _serve_source(self, parsed) -> None: - path = (parse_qs(parsed.query).get("path") or [""])[0] - if not path or not os.path.isabs(path) or not os.path.isfile(path): - self._send_plain(404, "not found") - return - try: - with open(path, "rb") as f: - body = f.read(4 * 1024 * 1024) - except OSError: - self._send_plain(404, "not found") - return - self.send_response(200) - self.send_header("Content-Type", "text/plain; charset=utf-8") - self.send_header("Content-Length", str(len(body))) - self.end_headers() - self.wfile.write(body) - - def _serve_static(self, url_path: str) -> None: - rel = url_path[len(UI_PREFIX):] if url_path.startswith(UI_PREFIX) else url_path.lstrip("/") - rel = rel or "index.html" - file_path = os.path.normpath(os.path.join(self.frontend_dir, rel)) - if not file_path.startswith(os.path.abspath(self.frontend_dir)): - self._send_plain(403, "forbidden") - return - if not os.path.isfile(file_path): - self._send_plain(404, "not found") - return - ctype = mimetypes.guess_type(file_path)[0] or "application/octet-stream" - with open(file_path, "rb") as f: - body = f.read() - self.send_response(200) - self.send_header("Content-Type", ctype) - self.send_header("Content-Length", str(len(body))) - self.end_headers() - self.wfile.write(body) - - # -- WebSocket <-> DAP bridge -------------------------------------------- - - def _handle_websocket(self) -> None: - key = self.headers.get("Sec-WebSocket-Key") - if not key: - self._send_plain(400, "bad websocket handshake") - return - upstream = None - deadline = time.monotonic() + 15 - while upstream is None: - try: - upstream = socket.create_connection((self.adapter_host, self.adapter_port), timeout=5) - except OSError: - if time.monotonic() >= deadline: - self._send_plain(502, "debug adapter unavailable") - return - time.sleep(0.5) - # The connect timeout must not linger on the established socket: it would make the - # idle recv() in the pump thread time out and tear the session down after 5 quiet - # seconds. A debug session idles for minutes while the developer reads code. - upstream.settimeout(None) - self.connection.sendall( - b"HTTP/1.1 101 Switching Protocols\r\n" - b"Upgrade: websocket\r\n" - b"Connection: Upgrade\r\n" - b"Sec-WebSocket-Accept: " + accept_key(key).encode() + b"\r\n\r\n" - ) - self.close_connection = True - ws = WebSocketConnection(self.connection) - - def pump_adapter_to_browser(): - try: - for body in _read_dap_messages(upstream): - ws.send_text(body.decode("utf-8", errors="replace")) - except (OSError, WebSocketClosed): - pass - finally: - ws.close() - - pump = threading.Thread(target=pump_adapter_to_browser, daemon=True) - pump.start() - saw_disconnect = False - try: - while True: - text = ws.recv_text() - if '"disconnect"' in text: - try: - parsed = json.loads(text) - if parsed.get("type") == "request" and parsed.get("command") == "disconnect": - saw_disconnect = True - except ValueError: - pass - body = self._rewrite_attach(text).encode() - upstream.sendall(b"Content-Length: %d\r\n\r\n%b" % (len(body), body)) - except (OSError, WebSocketClosed): - pass - finally: - if not saw_disconnect: - # The browser vanished (tab closed, page reloaded) without a DAP disconnect. - # An abruptly dropped client makes debugpy's adapter stop listening, killing - # re-attach - so say goodbye on the browser's behalf. - goodbye = json.dumps({ - "seq": 2_000_000_000, - "type": "request", - "command": "disconnect", - "arguments": {"terminateDebuggee": False}, - }).encode() - try: - upstream.sendall(b"Content-Length: %d\r\n\r\n%b" % (len(goodbye), goodbye)) - time.sleep(0.5) - except OSError: - pass - try: - upstream.close() - except OSError: - pass - ws.close() - - def _rewrite_attach(self, text: str) -> str: - """Inject the adapter's connect address into `attach` requests. - - debugpy treats an attach with `connect` as a socket attach and then accepts later - re-attaches on the same listen port - which is what makes browser reloads work. - The browser cannot know the loopback port, so the bridge fills it in. - """ - try: - message = json.loads(text) - except ValueError: - return text - if message.get("type") == "request" and message.get("command") == "attach": - arguments = message.setdefault("arguments", {}) - arguments["connect"] = {"host": self.adapter_host, "port": self.adapter_port} - return json.dumps(message) - return text - - -def start_debug_server( - listen_port: int, - adapter_port: int, - frontend_dir: str, - brk: bool = False, - entry_path: str = "", -) -> ThreadingHTTPServer: - handler = type( - "BoundDebugRequestHandler", - (DebugRequestHandler,), - { - "frontend_dir": os.path.abspath(frontend_dir), - "adapter_port": adapter_port, - "brk": brk, - "entry_path": entry_path, - }, - ) - server = ThreadingHTTPServer(("0.0.0.0", listen_port), handler) - server.daemon_threads = True - thread = threading.Thread(target=server.serve_forever, daemon=True) - thread.start() - return server diff --git a/example-actor-python/actor_debugger/frontend/index.html b/example-actor-python/actor_debugger/frontend/index.html deleted file mode 100644 index 942d7a6..0000000 --- a/example-actor-python/actor_debugger/frontend/index.html +++ /dev/null @@ -1,598 +0,0 @@ - - - - - -Actor Debugger - - - - -
-
- 🐛 Actor Debugger -
connecting…
- - - - - - - - -
- -

Sources

- -
Waiting for the debug session…

- Set breakpoints by clicking line numbers once a file is open. Files appear on the left as the - program loads them, or when it pauses.
- -
-

Call stack

not paused
-

Variables

-

Breakpoints

-
- - -
-
-
- -
-
-
› -
-
-
- - - - diff --git a/example-actor-python/actor_debugger/websocket.py b/example-actor-python/actor_debugger/websocket.py deleted file mode 100644 index e729d4f..0000000 --- a/example-actor-python/actor_debugger/websocket.py +++ /dev/null @@ -1,124 +0,0 @@ -"""Minimal server-side RFC 6455 WebSocket implementation (stdlib only). - -Just enough for the debugger bridge: handshake, text frames, fragmentation, -ping/pong, and close. Server-to-client frames are never masked (per spec); -client-to-server frames must be masked and are rejected otherwise. -""" - -from __future__ import annotations - -import base64 -import hashlib -import socket -import struct -import threading - -_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" - -OP_CONT = 0x0 -OP_TEXT = 0x1 -OP_BINARY = 0x2 -OP_CLOSE = 0x8 -OP_PING = 0x9 -OP_PONG = 0xA - - -def accept_key(sec_websocket_key: str) -> str: - digest = hashlib.sha1((sec_websocket_key + _GUID).encode()).digest() - return base64.b64encode(digest).decode() - - -class WebSocketClosed(Exception): - pass - - -class WebSocketConnection: - """A WebSocket over an already-upgraded socket.""" - - def __init__(self, sock: socket.socket) -> None: - self._sock = sock - self._send_lock = threading.Lock() - self._closed = False - - def _read_exact(self, n: int) -> bytes: - chunks = b"" - while len(chunks) < n: - chunk = self._sock.recv(n - len(chunks)) - if not chunk: - raise WebSocketClosed() - chunks += chunk - return chunks - - def _read_frame(self) -> tuple[bool, int, bytes]: - header = self._read_exact(2) - fin = bool(header[0] & 0x80) - opcode = header[0] & 0x0F - masked = bool(header[1] & 0x80) - length = header[1] & 0x7F - if length == 126: - length = struct.unpack(">H", self._read_exact(2))[0] - elif length == 127: - length = struct.unpack(">Q", self._read_exact(8))[0] - if not masked: - raise WebSocketClosed() # client frames must be masked - mask = self._read_exact(4) - payload = bytearray(self._read_exact(length)) - for i in range(length): - payload[i] ^= mask[i % 4] - return fin, opcode, bytes(payload) - - def recv_text(self) -> str: - """Block until a full text/binary message arrives; handle control frames inline. - - Raises WebSocketClosed when the peer closes or the socket drops. - """ - message = b"" - message_opcode = None - while True: - fin, opcode, payload = self._read_frame() - if opcode == OP_PING: - self._send_frame(OP_PONG, payload) - continue - if opcode == OP_PONG: - continue - if opcode == OP_CLOSE: - self.close() - raise WebSocketClosed() - if opcode in (OP_TEXT, OP_BINARY): - message_opcode = opcode - message = payload - elif opcode == OP_CONT and message_opcode is not None: - message += payload - else: - raise WebSocketClosed() - if fin: - return message.decode("utf-8", errors="replace") - - def _send_frame(self, opcode: int, payload: bytes) -> None: - length = len(payload) - if length < 126: - header = struct.pack(">BB", 0x80 | opcode, length) - elif length < 1 << 16: - header = struct.pack(">BBH", 0x80 | opcode, 126, length) - else: - header = struct.pack(">BBQ", 0x80 | opcode, 127, length) - with self._send_lock: - self._sock.sendall(header + payload) - - def send_text(self, text: str) -> None: - if self._closed: - raise WebSocketClosed() - self._send_frame(OP_TEXT, text.encode()) - - def close(self) -> None: - if self._closed: - return - self._closed = True - try: - self._send_frame(OP_CLOSE, struct.pack(">H", 1000)) - except OSError: - pass - try: - self._sock.close() - except OSError: - pass diff --git a/example-actor-python/requirements.txt b/example-actor-python/requirements.txt deleted file mode 100644 index d8557ca..0000000 --- a/example-actor-python/requirements.txt +++ /dev/null @@ -1,2 +0,0 @@ -apify>=4.0.0,<5.0.0 -debugpy>=1.8 diff --git a/example-actor-python/src/__init__.py b/example-actor-python/src/__init__.py deleted file mode 100644 index e69de29..0000000 diff --git a/example-actor-python/src/__main__.py b/example-actor-python/src/__main__.py deleted file mode 100644 index a66fc48..0000000 --- a/example-actor-python/src/__main__.py +++ /dev/null @@ -1,5 +0,0 @@ -import asyncio - -from .main import main - -asyncio.run(main()) diff --git a/example-actor-python/src/main.py b/example-actor-python/src/main.py deleted file mode 100644 index c849246..0000000 --- a/example-actor-python/src/main.py +++ /dev/null @@ -1,65 +0,0 @@ -"""Demo Actor for the browser debugger - deliberately structured to be fun to step through. - -Try this once the debugger UI is open: - * breakpoint inside `score_page` and watch `features` build up across ticks, - * step INTO `extract_features` / OUT again, - * evaluate `sum(features.values())` or mutate `page['depth']` in the console, - * tick "break on raised exceptions" - `flaky_lookup` raises (and handles) KeyError - every 5th tick, so the debugger will drop you on the raise site. -""" - -import asyncio -import random - -from apify import Actor - -FAKE_PAGES = [ - {'url': 'https://example.com/', 'depth': 0, 'html': '

Welcome

'}, - {'url': 'https://example.com/a', 'depth': 1, 'html': '

Products

'}, - {'url': 'https://example.com/b', 'depth': 1, 'html': '

Blog

hello world

'}, - {'url': 'https://example.com/a/1', 'depth': 2, 'html': '

Item 1

$42'}, -] - - -def extract_features(page: dict) -> dict: - html = page['html'] - return { - 'headings': sum(html.count(tag) for tag in (' str: - try: - return catalog[key] # raises KeyError for unknown keys - nice exception-breakpoint demo - except KeyError: - return 'uncategorized' - - -def score_page(page: dict, tick: int) -> dict: - features = extract_features(page) - catalog = {'https://example.com/a': 'catalog', 'https://example.com/a/1': 'product'} - category = flaky_lookup(catalog, page['url']) - score = features['headings'] * 10 + features['links'] * 5 + features['length'] // 10 - if features['has_price']: - score *= 2 - return {'url': page['url'], 'tick': tick, 'category': category, 'score': score, **features} - - -async def main() -> None: - async with Actor: - actor_input = await Actor.get_input() or {} - ticks = int(actor_input.get('ticks', 120)) - Actor.log.info(f'Demo Actor started - will process pages for {ticks} ticks.') - Actor.log.info('Open the debugger URL printed above by [actor-debugger].') - - for tick in range(1, ticks + 1): - page = random.choice(FAKE_PAGES) - result = score_page(page, tick) - await Actor.push_data(result) - Actor.log.info(f"tick {tick}: {result['url']} -> score {result['score']} ({result['category']})") - await asyncio.sleep(2) - - Actor.log.info('Demo Actor finished.') diff --git a/python/README.md b/python/README.md index 928d798..f2007c5 100644 --- a/python/README.md +++ b/python/README.md @@ -1,93 +1,99 @@ # actor-debugger (Python) -Drop-in remote debugging for **any Apify Python Actor** with a one-line Dockerfile change. It -launches your Actor under **debugpy** (the standard Python debugger that powers VS Code) **and +Drop-in remote debugging for **any Apify Python Actor** with a two-line Dockerfile change. It +launches your Actor under **debugpy** (the debugger that powers VS Code's Python debugging) **and serves a full browser debugger UI over the run's container URL** — so you open one link in your -own browser and debug. No IDE, no wstunnel, no local setup, no rebuild of your source, and **no -browser or IDE in the Actor**. +own browser and debug. No IDE, no tunnel, no local setup, no rebuild of your source, and **no +browser or IDE inside the Actor**. + +This is the Python sibling of the Node/TS [`actor-debugger`](../README.md) npm package, injected +the same way: ```dockerfile # Get the package RUN pip install actor-debugger -# pause on the first executable line until you attach (for short-lived Actors): -CMD ["python3", "-m", "actor_debugger", "--brk"] - -# or point at a specific module / file: -CMD ["python3", "-m", "actor_debugger", "-m", "src"] -CMD ["python3", "-m", "actor_debugger", "main.py"] +# Swap the entrypoint for the debug launcher (revert this line to disable debugging): +CMD ["python3", "-m", "actor_debugger", "--brk"] # was e.g.: CMD ["python3", "-m", "src"] ``` -## How it works +That's the entire integration — no code changes, no new ports, no platform configuration. Build, +run, and the run log prints one URL: -This is the Python sibling of the Node/TS `actor-debugger` npm package in the repository root, -built on the same architecture — swap `node --inspect` for debugpy and Chrome DevTools for a -served DAP client: +``` +[actor-debugger] OPEN THIS in your local browser for a full debugger UI (no local setup): +[actor-debugger] https://.runs.apify.net/ui/ +``` -1. It resolves your Actor's entrypoint (see detection order below) and launches it as - `python -m debugpy --listen 127.0.0.1:5678 ` — so the debugger runs on **your** code, - in its own process, exactly as it normally runs. debugpy speaks the - [Debug Adapter Protocol](https://microsoft.github.io/debug-adapter-protocol/) (DAP), the same - protocol VS Code uses to debug Python. -2. It runs one HTTP server on `ACTOR_WEB_SERVER_PORT` that: - - **serves a browser debugger UI** (static files, bundled with the package) so your *local* - browser opens it — the DAP *client* is JavaScript running in your browser; - - **bridges a WebSocket at `/dap`** to debugpy's DAP-over-TCP socket on loopback, translating - framing (one WebSocket message per DAP JSON message ↔ `Content-Length` framed TCP) and - injecting the adapter's loopback address into `attach` requests — which is also what makes - re-attach after a browser reload work; - - serves `/source?path=...` so the UI can display files that exist only on the container's - disk (a remote frontend can never fetch `file://` URLs — the same problem the Node version - solves by inlining source maps). -3. The run log prints one URL. Open it in your browser → a debugger attached to your Actor: - click line numbers to set breakpoints, step, inspect the stack and variables, evaluate - expressions in any frame, break on exceptions. +Open it: click line numbers to set breakpoints, step, inspect the call stack and variables, +evaluate expressions in the paused frame, break on exceptions. `--brk` pauses the Actor on its +first line until you attach — drop it to let the Actor run and attach mid-flight instead. -Because the container side is a dumb byte bridge, all protocol intelligence lives in the served -frontend — the Actor image gains only `debugpy` (~3 MB) and ~30 KB of static files. There is no -IDE, browser, Xvfb, or VNC in the image, and the Apify platform is used exactly as-is: the -standard container web server port is the only channel. +Until the package is published to PyPI, install it from the repository instead: -## Activation +```dockerfile +RUN pip install "actor-debugger @ git+https://github.com/apify/actor-debugger.git@master#subdirectory=python" +``` -Running the Actor through `python3 -m actor_debugger` **is** the switch — debugging is on -whenever the `CMD` line above is in place. To turn it off, revert the `CMD` to the Actor's normal -entrypoint (e.g. `CMD ["python3", "-m", "src"]`) and rebuild. Pass `--brk` to pause on the first -executable line of the entry file until you attach — useful for Actors that would otherwise -finish before you connect. (`--brk` maps to debugpy's `--wait-for-client` plus a synthetic -first-line breakpoint that the UI sets before resuming and removes after the first stop.) +## CLI forms -## Connect +```dockerfile +CMD ["python3", "-m", "actor_debugger"] # auto-detect the Actor's entrypoint +CMD ["python3", "-m", "actor_debugger", "--brk"] # pause on the first line until attached +CMD ["python3", "-m", "actor_debugger", "-m", "src"] # explicit module +CMD ["python3", "-m", "actor_debugger", "main.py"] # explicit file +``` -Open the URL the run log prints, in your own browser: +(`actor-debugger` also works as a console command; the `python3 -m` form is immune to PATH +surprises in slim images.) -``` -https://.runs.apify.net/ui/ -``` +Entrypoint detection order: explicit `-m `/`` argument → `src/__main__.py` +(the Apify template's `python3 -m src`) → `src/main.py` → `main.py` / `__main__.py` / `app.py`. -That page is the debugger; it connects back to the same host over `wss://…/dap`. Breakpoints are -kept in `localStorage`, so they survive page reloads and re-attach automatically. Program stdout -stays in the Actor run log as usual; the UI console is a REPL into the paused frame (plus -debugger events and exception details). +## How it works -Prefer a raw channel? `wss://.runs.apify.net/dap` speaks DAP directly, one JSON message per -WebSocket text frame — any DAP client can drive it. +1. It launches your Actor as `python -m debugpy --listen 127.0.0.1:5678 ` — the debugger + runs on **your** code, in its own process, exactly as it normally runs. debugpy speaks the + [Debug Adapter Protocol](https://microsoft.github.io/debug-adapter-protocol/) (DAP), the same + protocol VS Code uses. +2. It runs one HTTP server on `ACTOR_WEB_SERVER_PORT` that: + - **serves a browser debugger UI** (~30 KB of static files bundled with the package) — the DAP + *client* is JavaScript running in your browser; + - **bridges a WebSocket at `/dap`** to debugpy's DAP-over-TCP socket, translating framing (one + WebSocket message per DAP JSON message ↔ `Content-Length`-framed TCP), injecting the + adapter's loopback address into `attach` requests, and sending a `disconnect` on the + browser's behalf when a tab closes abruptly — which is what makes page reloads re-attach + cleanly; + - serves `/source?path=...` so the UI can display files that exist only on the container's + disk (the same problem the Node version solves by inlining source maps). +3. The run log prints the one URL above. Program stdout keeps flowing to the Actor run log as + usual; breakpoints live in the browser's `localStorage`, so they survive reloads and + re-attach automatically. + +Because the container side is a dumb byte bridge, all protocol intelligence lives in the served +frontend — the Actor image gains only `debugpy` (~3 MB) plus the static files. The platform is +used exactly as-is: the standard container web-server port is the only channel. Prefer a raw +channel? `wss://.runs.apify.net/dap` speaks DAP directly, one JSON message per WebSocket +text frame — any DAP client can drive it. -## Entrypoint detection order +## Verified on the Apify platform -1. An explicit `-m ` or `` argument, if given. -2. `src/__main__.py` → runs `-m src` (the Apify Python Actor template convention). -3. `src/main.py` → runs `-m src.main`. -4. Conventional files: `main.py`, `__main__.py`, `app.py`. +The full loop has been exercised against a real platform run: the UI served over +`https://.runs.apify.net/ui/`, the `wss://…/dap` WebSocket passed the platform ingress, and +a live run was attached, paused, and driven from a plain browser. Automated coverage in this +repo: a protocol test drives a complete DAP session through the bridge (breakpoint, stack, +scopes, variables, evaluate, resume, re-attach after disconnect), and a headless-Chromium test +drives the real UI end to end (entry pause under `--brk`, gutter breakpoints, stepping, +variables, in-frame evaluate, resume, re-attach after page reload), including idle sessions. ## Security The debug endpoint is **unauthenticated** — anyone who reaches the container URL can execute code in your run (and read its env, including `APIFY_TOKEN`); `/source` additionally serves any file -readable in the container. Keep the `actor_debugger` `CMD` only in builds you are actively -debugging, prefer a restricted run/token, never ship it in a published Actor, and gate the -endpoint (owner-only) before any non-prototype use. This matches the security posture of the Node -version; both need the same hardening pass. +readable in the container. Keep the debug `CMD` only in builds you are actively debugging, prefer +a restricted run/token, never ship it in a published Actor, and gate the endpoint (owner-only) +before any non-prototype use. This matches the security posture of the Node version; both need +the same hardening pass. ## Design notes: the routes considered @@ -98,34 +104,38 @@ the one exposed web-server port. Options considered: | Approach | Verdict | | --- | --- | | **debugpy + served browser DAP client** (this package) | CPython unchanged, actor code and deps run exactly as in production; debugpy is the canonical Python debugger; image cost ~3 MB. The UI is ours to grow. **Chosen.** | -| **GraalPy `--inspect`** — GraalVM's Python speaks the Chrome DevTools Protocol natively, so the Node version's chii DevTools frontend + CDP proxy would work *unchanged* | Elegant symmetry (identical UI and proxy for JS and Python), but it swaps the runtime under the Actor: a different base image, slower startup/warmup, and C-extension compatibility risk for real-world scraping deps (`lxml`, `pydantic`, `cryptography`, …). Debugging a different interpreter than production undermines the point. Worth a spike for pure-Python Actors; not the default. | -| **DAP→CDP translation shim** — keep CPython + debugpy, translate DAP into the Chrome DevTools Protocol in the bridge, and serve the same chii DevTools frontend the Node version uses | Best possible UI for free, one frontend for both languages. The mapping (breakpoints, stepping, stack, scopes, evaluate) is tractable, but the DevTools frontend is picky about `Debugger`/`Runtime` domain lifecycle, script IDs, and object groups — a meaningful project on its own. The `/dap` WebSocket this package exposes is exactly where such a shim would slot in later. | -| **code-server / openvscode-server in the container** — full VS Code in a browser tab, attach to debugpy | Great UX, zero custom code, but adds ~300 MB and a Node runtime to every Python Actor image, needs auth wiring, and its multiplexed WebSockets are more ingress-sensitive. Overkill for "set a breakpoint in a run". | -| **`web-pdb` / xterm.js + pdb over WebSocket** | Tiny and dependency-free, but a terminal `pdb` UX (no gutter breakpoints, no variable tree) and pdb can't attach to a running program the way debugpy can. Good fallback, not the goal. | -| **Jupyter server in the container** (JupyterLab's visual debugger also speaks debugpy) | The kernel model doesn't fit debugging an already-running script/Actor process; heavyweight. | -| **[`dap-python`](https://pypi.org/project/dap-python/)** — a typed Python DAP *client* library | Sits on the wrong side of this design's wire: the DAP client here is the browser, and the container side is a protocol-agnostic byte bridge. It would fit a server-orchestrated variant (Python translating DAP to a custom browser protocol), at the cost of a Pydantic dependency in every Actor image and a Python ≥3.12 floor. Noted as a building block if the bridge ever needs server-side smarts. | - -## Verified - -Tested end-to-end against a generic sample Python Actor (`example-actor-python/` in the repo): - -- The one-line `CMD` detects the entrypoint (`-m src`) and runs the Actor under debugpy; explicit - module/file and auto-detect modes all pass. -- `/ui/` and `/config` are served; `/source` returns container files; the `/dap` WebSocket - round-trips the full DAP handshake (`initialize` → `attach` → `setBreakpoints` → - `configurationDone`), hits a breakpoint, walks the stack/scopes/variables, evaluates in-frame, - and resumes — through the bridge with browser framing. -- **Headless Chromium loaded the served UI and drove a real session**: attached, auto-paused on - the entry line under `--brk`, rendered sources, stepped, hit a gutter-set breakpoint, showed - stack + variables, evaluated in the paused frame, and resumed. Re-attach after page reload - works. - -One thing to confirm on real infrastructure (shared with the Node version): the `wss://` -connection through Apify's ingress when it negotiates HTTP/2. +| **GraalPy `--inspect`** — GraalVM's Python speaks the Chrome DevTools Protocol natively, so the Node version's chii DevTools frontend + CDP proxy would work *unchanged* | Elegant symmetry, but it swaps the runtime under the Actor: a different base image, slower startup, and C-extension compatibility risk for real-world deps (`lxml`, `pydantic`, `cryptography`, …). Debugging a different interpreter than production undermines the point. | +| **DAP→CDP translation shim** — keep CPython + debugpy, translate DAP into the Chrome DevTools Protocol and serve the same chii DevTools frontend the Node version uses | Best possible UI for free, one frontend for both languages, but the DevTools frontend is picky about `Debugger`/`Runtime` lifecycle — a meaningful project on its own. The `/dap` WebSocket this package exposes is where such a shim would slot in later. | +| **code-server / openvscode-server in the container** | Great UX, zero custom code, but ~300 MB and a Node runtime in every Python Actor image, plus auth wiring. Overkill for "set a breakpoint in a run". | +| **`web-pdb` / xterm.js + pdb over WebSocket** | Tiny, but a terminal `pdb` UX (no gutter breakpoints, no variable tree) and pdb can't attach to a running program the way debugpy can. | +| **Jupyter server in the container** (JupyterLab's debugger also speaks debugpy) | The kernel model doesn't fit debugging an already-running script; heavyweight. | +| **[`dap-python`](https://pypi.org/project/dap-python/)** — a typed Python DAP *client* library | The DAP client here is the browser; the container side is a protocol-agnostic byte bridge. Useful only for a server-orchestrated variant, at the cost of a Pydantic dependency and a Python ≥3.12 floor. | + +The same architecture extends beyond Python: any language with a DAP adapter (Node via js-debug, +Go via Delve — which speaks DAP natively — Rust via lldb-dap, …) can sit behind the identical +bridge and frontend; only the adapter spawn command and entrypoint detection differ. + +## Releasing + +Publishing to PyPI is automated by +[`.github/workflows/publish-python.yml`](../.github/workflows/publish-python.yml), which fires on +`py-v*` tags. To cut a release: + +```bash +# bump version in python/pyproject.toml and python/src/actor_debugger/__init__.py, commit, then: +git tag py-v0.1.0 +git push --tags +``` + +One-time setup: create a PyPI API token for the project (or an account-scoped token for the very +first upload, which claims the `actor-debugger` name) and save it as the repository secret +`PYPI_TOKEN`. The publish job runs in the `pypi` GitHub environment, so you can add required +reviewers there to gate releases. ## Notes -- Only runtime dependency is `debugpy`; the server and WebSocket implementation are stdlib. -- `--wait-for-client` means a crashed debug UI never blocks a *running* Actor — the process only - waits at startup with `--brk`. -- The frontend applies "just my code" on (re)connect; untick it to step into library code. +- Only runtime dependency is `debugpy`; the HTTP server and the RFC 6455 WebSocket implementation + are pure stdlib (Python ≥3.9). +- A crashed or closed debugger tab never blocks a *running* Actor — the process only waits at + startup, and only with `--brk`. +- "Just my code" is applied on (re)connect; untick it in the UI header to step into library code. diff --git a/python/pyproject.toml b/python/pyproject.toml index 39be015..0bcba79 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -19,7 +19,7 @@ classifiers = [ ] [project.urls] -Homepage = "https://github.com/Pijukatel/actor-debugger" +Homepage = "https://github.com/apify/actor-debugger" [project.scripts] actor-debugger = "actor_debugger.cli:main" From ce54d3b47f819b9bbe99684d34f67988b95e65f5 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 8 Sep 2026 11:08:15 +0000 Subject: [PATCH 5/6] Switch PyPI publishing to Trusted Publishing via publish_to_pypi.yml Replaces the token-based publish-python.yml with publish_to_pypi.yml, matching the trusted publisher configured on PyPI (project actor-debugger, repository apify/actor-debugger, workflow publish_to_pypi.yml - the file name must match exactly). The publish job authenticates with OIDC (id-token: write) through pypa/gh-action-pypi-publish, so no PYPI_TOKEN secret is needed. The pre-publish checks are unchanged: tag/version match plus CLI and served-UI smoke tests. Docs updated. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0194oFPjsTxztgsPhyB4Wq7G --- ...publish-python.yml => publish_to_pypi.yml} | 32 ++++++++----------- python/README.md | 14 ++++---- 2 files changed, 21 insertions(+), 25 deletions(-) rename .github/workflows/{publish-python.yml => publish_to_pypi.yml} (81%) diff --git a/.github/workflows/publish-python.yml b/.github/workflows/publish_to_pypi.yml similarity index 81% rename from .github/workflows/publish-python.yml rename to .github/workflows/publish_to_pypi.yml index 23d12e6..b910fdb 100644 --- a/.github/workflows/publish-python.yml +++ b/.github/workflows/publish_to_pypi.yml @@ -1,11 +1,14 @@ # Publishes the Python actor-debugger package to PyPI when a py-v* tag is pushed. # +# Uses PyPI Trusted Publishing (OIDC) - no API token or secret anywhere. The trusted +# publisher on PyPI is configured as: +# project: actor-debugger +# repository: apify/actor-debugger +# workflow: publish_to_pypi.yml (this file name must match exactly) +# # Release flow: # (bump version in python/pyproject.toml and python/src/actor_debugger/__init__.py, commit) # git tag py-vX.Y.Z && git push --tags -# -# Setup (one-time): create a PyPI API token scoped to this project (or an account token for the -# first upload) and save it as the repository secret PYPI_TOKEN. name: Publish to PyPI @@ -62,18 +65,11 @@ jobs: curl -sf http://127.0.0.1:4321/ui/ | grep -q 'Actor Debugger' kill $CLI_PID - - name: Build distributions - run: | - pip install build - python -m build - publish: needs: check runs-on: ubuntu-latest - environment: pypi - defaults: - run: - working-directory: python + permissions: + id-token: write # required for PyPI Trusted Publishing (OIDC) steps: - uses: actions/checkout@v4 @@ -82,14 +78,12 @@ jobs: python-version: '3.11' - name: Build distributions + working-directory: python run: | pip install build python -m build - - name: Publish - run: | - pip install twine - twine upload dist/* - env: - TWINE_USERNAME: __token__ - TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} + - name: Publish to PyPI (trusted publishing) + uses: pypa/gh-action-pypi-publish@release/v1 + with: + packages-dir: python/dist diff --git a/python/README.md b/python/README.md index f2007c5..9f06ab5 100644 --- a/python/README.md +++ b/python/README.md @@ -118,8 +118,11 @@ bridge and frontend; only the adapter spawn command and entrypoint detection dif ## Releasing Publishing to PyPI is automated by -[`.github/workflows/publish-python.yml`](../.github/workflows/publish-python.yml), which fires on -`py-v*` tags. To cut a release: +[`.github/workflows/publish_to_pypi.yml`](../.github/workflows/publish_to_pypi.yml), which fires +on `py-v*` tags and publishes via **PyPI Trusted Publishing** (OIDC) — no API token or repository +secret. The trusted publisher configured on PyPI is: project `actor-debugger`, repository +`apify/actor-debugger`, workflow `publish_to_pypi.yml` (the workflow file name must stay exactly +that). To cut a release: ```bash # bump version in python/pyproject.toml and python/src/actor_debugger/__init__.py, commit, then: @@ -127,10 +130,9 @@ git tag py-v0.1.0 git push --tags ``` -One-time setup: create a PyPI API token for the project (or an account-scoped token for the very -first upload, which claims the `actor-debugger` name) and save it as the repository secret -`PYPI_TOKEN`. The publish job runs in the `pypi` GitHub environment, so you can add required -reviewers there to gate releases. +The workflow first installs the package, checks the tag matches the pyproject version, and smoke +tests the CLI and the served debugger UI; only then does the publish job build sdist+wheel and +upload. ## Notes From 87a079fac4d36f93aac92b0e93b89267cfc639fb Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 8 Sep 2026 11:08:41 +0000 Subject: [PATCH 6/6] Allow manual runs of the PyPI publish workflow Adds workflow_dispatch so the release can be started from the Actions tab (e.g. for the first upload that claims the actor-debugger name). The tag/version check only runs on tag pushes; a manual run publishes the version pyproject.toml carries on the chosen ref, and PyPI's refusal of re-uploads makes repeated runs safe. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0194oFPjsTxztgsPhyB4Wq7G --- .github/workflows/publish_to_pypi.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/publish_to_pypi.yml b/.github/workflows/publish_to_pypi.yml index b910fdb..66fceb7 100644 --- a/.github/workflows/publish_to_pypi.yml +++ b/.github/workflows/publish_to_pypi.yml @@ -9,6 +9,10 @@ # Release flow: # (bump version in python/pyproject.toml and python/src/actor_debugger/__init__.py, commit) # git tag py-vX.Y.Z && git push --tags +# +# Can also be started manually (Actions -> Publish to PyPI -> Run workflow); a manual run +# skips the tag/version check and publishes whatever version pyproject.toml carries on the +# chosen ref (PyPI rejects re-uploads of an existing version, so re-runs are safe). name: Publish to PyPI @@ -16,6 +20,7 @@ on: push: tags: - 'py-v*' + workflow_dispatch: jobs: check: @@ -34,6 +39,7 @@ jobs: run: pip install . - name: Verify tag matches pyproject version + if: github.ref_type == 'tag' run: | PKG_VERSION="$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")" TAG_VERSION="${GITHUB_REF_NAME#py-v}"