goodboy
/
tractor
1
0
Fork 2
Code Issues 5 Pull Requests 10 Packages Projects Releases Wiki Activity

Add initial repl_fixture support, enter/exit hooks around the debugger sys! #28

Open
goodboy wants to merge 12 commits from repl_fixture into pytest_pluginize
pull from: repl_fixture
merge into: goodboy:pytest_pluginize
Conversation 0 Commits 12 Files Changed 31
goodboy commented 2025-05-12 04:47:28 +00:00
Owner

It turns out to be fairly useful to allow hooking into a given actor’s entry-and-exit around .devx._debug._pause/._post_mortem() calls which engage the pdbp.Pdb REPL (really our ._debug.PdbREPL but yeah).

Some very handy use cases include,

  • swapping out-of-band (config) state that may otherwise halt the user’s app since the actor normally handles kb&mouse input, in thread, which means that the handler will be blocked while the REPL is in use.

  • (remotely) reporting actor-runtime state for monitoring purposes around crashes or pauses in normal operation.

  • allowing for crash-handling to be hard-disabled via ._state._runtime_vars say for when you never want a debugger to be entered in a production instance where you’re not-sure-if/don’t-want per-actor debug_mode: bool settings to always be unset, say bc you’re still debugging some edge cases that ow you’d normally want to REPL up.

Impl deats for repl_fxture,

  • add a new optional ._state._runtime_vars['repl_fixture'] field which for now can be manually set; i saw no reason for a formal API yet since we want to convert the dict to a struct anyway (first).

  • augment both .devx._debug._pause()/._post_mortem() with a new optional repl_fixture: AbstractContextManager[bool] kwarg which when provided is opened as with repl_fixture() around the low-level-sync PdbREPL interaction calls; if the enter-result, an expected bool, is False then the interaction is hard-bypassed.

    • a new @cm helper to contain the activation logic, .devx.debug._post_mortem._maybe_open_repl_fixture() which:

      • when the new runtime-var repl_fixture is overridden, (only possible manually rn) use it but only whenever the repl_fixture-kwarg is left null.

    • for the ._pause() case the @cm is opened around the entire body of the embedded _enter_repl_sync() closure (for now though, since ideally longer term this entire routine is factored to be a lot less “nested”)

    • for _post_mortem() the entire fn body is embedded and now also excepts an optional boxed_maybe_exc: BoxedMaybeException; it is only passed in the open_crash_handler() caller case.

Other related refinements,

  • add a BoxedMaybeException.pformat() = __repr__() which when a .value: Exception is set renders a more “objecty” repr of the exc.

Reorg of .devx._debug to new .devx.debug sub-pkg,

  • split up the current .devx._debug components into various .debug.* submods, like:
    • [maybe_]open_crash_handler() and friends into a _post_mortem.
    • all the pdbp specific customizations into a ._repl.
    • all the root actor TTY locking into its own ._tty_lock.
    • all pausing/pdb.set_trace()-like apis into a ._trace
    • our shielding SIGINT handler into ._sigint
    • various (unrefined and higher level) syncing apis which parts of the runtime use mostly to avoid REPL/TTY clobbering in a ._sync
  • move .devx.debug.hide_runtime_frames() -> .devx._frame_stack

Tests

  • as repl_fixture input to sync code’s use of [maybe_]_open_crash_handler() (which is currently implemented the same as the async APIs since async def post_mortem() just calls def _post_mortem())
    • the same but using the global rtvars override
      • which we should add an pub api for.. question is now or later?
  • tests for all async uses which are already implemented via the augmentation of _pause()’s internal _enter_repl_sync() closure.
    • from root actor.
    • from any (nested) sub.
    • from an infected aio task?
  • support for pause_from_sync()/breakpoint() ??
    • we need to break up the provided repl_fixture().__enter__() and .__exit__() calls with the latter invoked as part of the PdbREPL per method calls to DebugaStatus.release() to ensure the fixture’s teardown isn’t invoked until after the user releases their repl interaction.
      • likely impl will be to add an DebugStatus._fixture_stack: ExitStack and have every _pause() call .enter_context() prior to interaction and then have DebugStatus/PdbREPL call .fixture_stack.close() from .set_continue/quit()?
  • support for an @acm impls of replx_fixture?
    • obvi this can only be supported from already async APIs or we have to somehow schedule the async code in the trio/runtime thread, so hardest to do for sync-code crashes/pauses.

Surrounding changes,

  • fix the stackscope-using shield-pause test by adding typing_extenions sub-dep it needs.

  • maybe we can also include some (stringent) tests around similar usage of the undocumented Actor.lifetime_stack: ExitStack ??

It turns out to be fairly useful to allow hooking into a given actor's entry-and-exit around `.devx._debug._pause/._post_mortem()` calls which engage the `pdbp.Pdb` REPL (really our `._debug.PdbREPL` but yeah). Some very handy use cases include, - swapping out-of-band (config) state that may otherwise halt the user's app since the actor normally handles kb&mouse input, in thread, which means that the handler will be blocked while the REPL is in use. - (remotely) reporting actor-runtime state for monitoring purposes around crashes or pauses in normal operation. - allowing for crash-handling to be hard-disabled via `._state._runtime_vars` say for when you never want a debugger to be entered in a production instance where you're not-sure-if/don't-want per-actor `debug_mode: bool` settings to always be unset, say bc you're still debugging some edge cases that ow you'd normally want to REPL up. --- #### Impl deats for `repl_fxture`, - add a new optional `._state._runtime_vars['repl_fixture']` field which for now can be manually set; i saw no reason for a formal API yet since we want to convert the `dict` to a struct anyway (first). - [x] augment both `.devx._debug._pause()/._post_mortem()` with a new optional `repl_fixture: AbstractContextManager[bool]` kwarg which when provided is opened as `with repl_fixture()` around the low-level-sync `PdbREPL` interaction calls; if the enter-result, an expected `bool`, is `False` then the interaction is hard-bypassed. * a new `@cm` helper to contain the activation logic, `.devx.debug._post_mortem._maybe_open_repl_fixture()` which: - when the new runtime-var `repl_fixture` is overridden, (only possible manually rn) use it but only whenever the `repl_fixture`-kwarg is left null. * for the `._pause()` case the `@cm` is opened around the entire body of the embedded `_enter_repl_sync()` closure (for now though, since ideally longer term this entire routine is factored to be a lot less "nested") * for `_post_mortem()` the entire fn body is embedded and now also excepts an optional `boxed_maybe_exc: BoxedMaybeException`; it is only passed in the `open_crash_handler()` caller case. --- #### Other related refinements, - add a `BoxedMaybeException.pformat() = __repr__()` which when a `.value: Exception` is set renders a more "objecty" repr of the exc. --- #### Reorg of `.devx._debug` to new `.devx.debug` sub-pkg, - [x] split up the current `.devx._debug` components into various `.debug.*` submods, like: - [x] `[maybe_]open_crash_handler()` and friends into a `_post_mortem`. - [x] all the `pdbp` specific customizations into a `._repl`. - [x] all the root actor TTY locking into its own `._tty_lock`. - [x] all pausing/`pdb.set_trace()`-like apis into a `._trace` - [x] our shielding SIGINT handler into `._sigint` - [x] various (unrefined and higher level) syncing apis which parts of the runtime use mostly to avoid REPL/TTY clobbering in a `._sync` - [x] move `.devx.debug.hide_runtime_frames()` -> `.devx._frame_stack` --- #### Tests - [ ] as `repl_fixture` input to sync code's use of `[maybe_]_open_crash_handler()` (which is currently implemented the same as the async APIs since `async def post_mortem()` just calls `def _post_mortem()`) - [ ] the same but using the global rtvars override - [ ] which we should add an pub api for.. question is now or later? - [ ] tests for all async uses which are already implemented via the augmentation of `_pause()`'s internal `_enter_repl_sync()` closure. - [ ] from root actor. - [ ] from any (nested) sub. - [ ] from an infected aio task? - support for `pause_from_sync()`/`breakpoint()` ?? - [ ] we need to break up the provided `repl_fixture().__enter__()` and `.__exit__()` calls with the latter invoked as part of the `PdbREPL` per method calls to `DebugaStatus.release()` to ensure the fixture's teardown isn't invoked until **after** the user releases their repl interaction. - [ ] likely impl will be to add an `DebugStatus._fixture_stack: ExitStack` and have every `_pause()` call `.enter_context()` prior to interaction and then have `DebugStatus`/`PdbREPL` call `.fixture_stack.close()` from `.set_continue/quit()`? - [ ] support for an `@acm` impls of `replx_fixture`? - obvi this can *only* be supported from already async APIs or we have to somehow schedule the async code in the trio/runtime thread, so hardest to do for sync-code crashes/pauses. Surrounding changes, - [x] fix the `stackscope`-using shield-pause test by adding `typing_extenions` sub-dep it needs. - [ ] maybe we can also include some (stringent) tests around similar usage of the undocumented `Actor.lifetime_stack: ExitStack` ??
goodboy added 1 commit 2025-05-12 04:47:29 +00:00
f604c8836d Add initial `repl_fixture` support B) 
It turns out to be fairly useful to allow hooking into a given actor's
entry-and-exit around `.devx._debug._pause/._post_mortem()` calls which
engage the `pdbp.Pdb` REPL (really our `._debug.PdbREPL` but yeah).

Some very handy use cases include,
- swapping out-of-band (config) state that may otherwise halt the
  user's app since the actor normally handles kb&mouse input, in thread,
  which means that the handler will be blocked while the REPL is in use.
- (remotely) reporting actor-runtime state for monitoring purposes
  around crashes or pauses in normal operation.
- allowing for crash-handling to be hard-disabled via
  `._state._runtime_vars` say for when you never want a debugger to be
  entered in a production instance where you're not-sure-if/don't-want
  per-actor `debug_mode: bool` settings to always be unset, say bc
  you're still debugging some edge cases that ow you'd normally want to
  REPL up.

Impl details,
- add a new optional `._state._runtime_vars['repl_fixture']` field which
  for now can be manually set; i saw no reason for a formal API yet
  since we want to convert the `dict` to a struct anyway (first).
- augment both `.devx._debug._pause()/._post_mortem()` with a new
  optional `repl_fixture: AbstractContextManager[bool]` kwarg which
  when provided is `with repl_fixture()` opened around the lowlevel
  REPL interaction calls; if the enter-result, an expected `bool`, is
  `False` then the interaction is hard-bypassed.
  * for the `._pause()` case the `@cm` is opened around the entire body
    of the embedded `_enter_repl_sync()` closure (for now) though
    ideally longer term this entire routine is factored to be a lot less
    "nested" Bp
  * in `_post_mortem()` the entire previous body is wrapped similarly
    and also now excepts an optional `boxed_maybe_exc: BoxedMaybeException`
    only passed in the `open_crash_handler()` caller case.
- when the new runtime-var is overridden, (only manually atm) it is used
  instead but only whenever the above `repl_fixture` kwarg is left null.
- add a `BoxedMaybeException.pformat() = __repr__()` which when
  a `.value: Exception` is set renders a more "objecty" repr of the exc.

Obviously tests for all this should be coming soon!
goodboy added 1 commit 2025-05-13 00:43:10 +00:00
09a61dbd8a Add exc suppression to `open_crash_handler()` 
By supporting a new optional param to `open_crash_handler()`,
`raise_on_exit: bool|Sequence[Type[BaseException]] = True` which
determines whether, after the REPL interaction completes, the handled
exception is raised upward. This is **very** handy for writing bits of
"debug-able but resilient code" as is the case in (many) dependent
projects/apps.

Impl,
- `raise_on_exit` can be a `bool` or (set) sequence of types which will
  always be raised.
- also add a `BoxedMaybeException.raise_on_exit` equiv which (for now)
  we check matches (in case down the road we want to offer dynamic ctls).
- rename both crash-handler cm's `tb_hide` -> `hide_tb`.
goodboy added 8 commits 2025-05-15 19:00:30 +00:00
18ae7b0048 Mk `.devx._debug` a sub-pkg `.devx.debug` 
With plans for much factoring of the original module into sub-mods!
Adjust all imports and refs throughout to match.
712450869b Add `_maybe_open_repl_fixture()` 
Factoring the (basically duplicate) content from both use spots into
a common `@cm` which delivers a `bool` signalling whether the REPL
should be engaged. Fixes a lingering bug with `nullcontext()` calling
btw..
cb31a330b3 Start splitting into `devx.debug.` sub-mods 
From what was originall the `.devx._debug` monolith module, since that
file was way out of ctl in terms of LoC!

New modules so far include,
- ._repl: our `pdb[p]` ext type/lowlevel-APIs and `mk_pdb()` factory.
- ._sigint: just our REPL-interaction shield-handler.
- ._tty_lock: containing all the root-actor TTY mutex machinery
  including the `Lock`/`DebugStatus` primitives/APIs as well as the
  inter-tree IPC context eps:
  * the server-side `lock_stdio_for_peer()` which pairs with the,
  * client-(subactor)-side `request_root_stdio_lock()` via the,
  * pld-msg-spec of `LockStatus/LockRelease`.
  AND the `any_connected_locker_child()` predicate.
006ee0752c Be explicit with `SpawnSpec` processing in subs 
As per the outstanding TODO just above the redic `setattr()` loop in
`Actor._from_parent()`!!

Instead of all that risk-ay monkeying, add detailed comment-sections
around each explicit assignment of each `SpawnSpec` field, including
those that were already being explicitly set.

Those and other deats,
- ONLY enable the `.devx.debug._tty_lock` module from `Actor.__init__()`
  in the root actor.
- add a new `get_mod_nsps2fps()` to replace the loop in init and assign
  the initial `.enable_modules: dict[str, str]` from it.
- do `self.enable_modules.update(spawnspec.enable_modules)` instead of
  an overwrite and assert the table is by default empty in all
  subs.
69267ae656 Mv `.hide_runtime_frames()` -> `.devx._frame_stack` 
A much more relevant module for a call-stack-frame hider ;)
f1e9926b79 Reorg `.devx.debug` into sub-mods! 
Which cleans out the pkg-mod to just the expected exports with (its
longstanding todo comment list) and thus a separation-of-concerns
and smaller mod-file sizes via the following new sub-mods:
- `._trace` for the `.pause()`/`breakpoint()`/`pdb.set_trace()`-style
  APIs including all sync-caller variants.
- `._post_mortem` to contain our async `.post_mortem()` and all other
  public crash handling APIs for use from sync callers.
- `._sync` for the high-level syncing helper-routines used throughout the
  runtime to avoid multi-proc TTY use collisions.

And also,
- remove `hide_runtime_frames()` since moved to `.devx._frame_stack`.
goodboy added 1 commit 2025-05-15 19:20:20 +00:00
4fca7317ea Add `typing_extensions`, it's a sub-dep of `stackscope`? 
Oddly my env was borked bc this (apparently missed by `uv`?) sub-dep
wasn't installed and then `stackscope` was silently failing import and
caused the shield-pause test to also fail (since it couldn't match the
expected `log.devx()` on console). The import failure is not very
explanatory due to the `log.warning()`; change it to `.error()` level.

Also, explicitly import `_sync_pause_from_builtin` in
`examples/debugging/restore_builtin_breakpoint.py` to ensure the ref is
exported properly from `.devx.debug` (which it wasn't during dev of the
prior commit Bp).
goodboy force-pushed repl_fixture from 4fca7317ea to d716c57234 2025-05-16 04:25:36 +00:00 Compare
This pull request can be merged automatically.
You are not authorized to merge this pull request.
You can also view command line instructions.

Step 1:

From your project repository, check out a new branch and test the changes.
git checkout -b repl_fixture pytest_pluginize
git pull origin repl_fixture

Step 2:

Merge the changes and update on Gitea.
git checkout pytest_pluginize
git merge --no-ff repl_fixture
git push origin pytest_pluginize
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: goodboy/tractor#28
There is no content yet.
Delete Branch "repl_fixture"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?