Skip to content

feat(displayhook): add register_hook/unregister_hook to ZMQShellDisplayHook#1522

Open
rgbkrk wants to merge 2 commits into
ipython:mainfrom
rgbkrk:feat/displayhook-hook-chain
Open

feat(displayhook): add register_hook/unregister_hook to ZMQShellDisplayHook#1522
rgbkrk wants to merge 2 commits into
ipython:mainfrom
rgbkrk:feat/displayhook-hook-chain

Conversation

@rgbkrk
Copy link
Copy Markdown
Member

@rgbkrk rgbkrk commented May 13, 2026

Summary

Mirrors the register_hook / unregister_hook / _hooks chain that ZMQDisplayPublisher already has, but on ZMQShellDisplayHook — the displayhook used for the bare-last-expression execute_result path. finish_displayhook now runs the chain before session.send, with the same contract as ZMQDisplayPublisher.publish:

  • hook(msg) -> msg — pass through, continue the chain, eventually session.send.
  • hook(msg) -> None — hook handled the send itself; suppress the default send.

The hook list lives on a threading.local() so registrations don't leak across threads, matching the existing implementation on ZMQDisplayPublisher.

Motivation

DisplayFormatter is the documented extension point for mime formatting, but inside ipykernel display_data and execute_result take different paths after formatting:

  • ZMQDisplayPublisher.publish (used by display() / display_data) builds the message and walks self._hooks before calling session.send. That's where buffer-attaching transforms, redirectors, and similar extensions live today.
  • ZMQShellDisplayHook.finish_displayhook (used for the bare-last-expression execute_result path) calls session.send directly with no hook chain.

So an extension that wants identical behavior on both paths — for example, attaching ZMQ buffers to the outbound message — has to subclass ZMQShellDisplayHook and reimplement finish_displayhook from scratch. This PR makes the two paths symmetric so a single hook function can register_hook on both seats.

Concrete use case

Streaming Arrow IPC bytes alongside a small JSON envelope on display_data / execute_result, without going through comms or widgets. The kernel emits a normal display message whose data carries a structured payload (mimetype + ref) and attaches the Arrow bytes as ZMQ buffers; the frontend reassembles. This pattern composes naturally with __arrow_c_stream__, so any dataframe library that implements the protocol gets a high-fidelity representation without per-library shims.

Doing this on display_data works today via ZMQDisplayPublisher.register_hook. Doing it on execute_result (the bare df on the last line of a cell) requires the same hook to fire on ZMQShellDisplayHook, which it can't today. nteract currently carries a subclass to bridge the gap; with this PR upstream, the subclass goes away.

Compatibility

  • No behavior change when no hooks are registered: the existing guards (self.msg, content.data non-empty, self.session configured) are preserved, and session.send is called exactly as before.
  • The hook contract is identical to ZMQDisplayPublisher's, so a hook authored against display_data works unchanged on execute_result.
  • Thread-local storage means existing single-threaded extensions see no difference; multi-threaded callers get the same isolation ZMQDisplayPublisher already provides.

rgbkrk added 2 commits May 12, 2026 23:58
…ayHook

Mirrors the hook chain on ZMQDisplayPublisher so the same transform
function can register on both display_data and execute_result. Lets
extensions attach ZMQ buffers (or otherwise mutate/suppress) the
bare-last-expression execute_result message without subclassing
ZMQShellDisplayHook and reimplementing finish_displayhook.

Hook contract matches ZMQDisplayPublisher.register_hook:
  hook(msg) -> msg   continue chain
  hook(msg) -> None  suppress send
…playHook

Mirrors the ZMQDisplayPublisher hook tests (NoReturnHook / ReturnHook,
thread-local isolation, unregister) and adds:

- a mutation test that attaches buffers to the outbound message — the
  motivating use case for execute_result hooks
- short-circuit semantics in a multi-hook chain
- threading semantics: each hook receives the previous hook's return value
- the existing empty-data guard still suppresses both hooks and send
@rgbkrk rgbkrk added the feature label May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant