goodboy
/
tractor
1
0
Fork 2
Code Issues 5 Pull Requests 8 Packages Projects Releases Wiki Activity

Add tractor.pause_from_sync() using the amazing greenback! #1

Closed
goodboy wants to merge 185 commits from pause_from_sync_w_greenback into main
pull from: pause_from_sync_w_greenback
merge into: goodboy:main
Conversation 1 Commits 185 Files Changed 67
goodboy commented 2024-03-21 21:43:09 +00:00
Owner

NOTE THIS WAS ALREADY LANDED ON GITHUB!

Already merged in PR here, https://github.com/goodboy/tractor/pull/374

Namely since most of the work was before we moved the core dev process to self-hosted.

Note that there is more refinement work coming, particularly for better infected-asyncio-mode support, in new downstream PRs,

  • #7 as a dep of,
  • #2 which solves py3.13’s asyncio madness with SIGINT as well as various other oddities and benefits to latest the stdlib interfaces..

It’s been a super long outstanding task (with multiple “attempts” on my part) to support engaging the pdbp REPL from sync code whilst keeping all the benefits of our normal await tractor.pause() version.

Finally that PR has come brethren, and you shall debug from whatever func you so desire.

Feats landed with this (via GH PR),

Stuff moved to a downstream PR,

Namely the new #2

Bo

ToDos for there from this .pause_from_sync() effort,

See #2 for follow up of all the below!

  • from asyncio tasks when using .to_asyncio:
    • using both .pause_from_sync() and breakpoint()
    • ensure crashes in asyncio tasks engage correctly!
  • decide on greenback bootstrapping:

    • should it be an optional dep? => NO, instead we include it with the --dev optional deps

    • always override the breakpoint() hook when it is installed? => YES, and we also raise a RTE on any breakpoint() usage from code that both does not spec debug_mode=True to open_roo_actor() as well as if no greenback is avail!

    • when greenback is not installed how should we guard against debug_mode=True usage from sync code? => we raise an explicit RTE always around breakpoint() usage!

    • where is the best place to call devx._debug.maybe_init_greenback()? => _root and in _invoke() tasks

  • support for infected-asyncio-mode in a root actor via the tractor.to_asyncio.run_as_asyncio_guest() entrypoint which can be now used over the std trio.run() from the first main/root process to open the runtime via either .open_root_actor() or .open_nursery().
### NOTE THIS WAS ALREADY LANDED ON GITHUB! Already merged in PR here, https://github.com/goodboy/tractor/pull/374 Namely since most of the work was before we moved the core dev process to self-hosted. Note that there is more refinement work coming, particularly for better infected-`asyncio`-mode support, in new downstream PRs, - https://pikers.dev/goodboy/tractor/pulls/7 as a dep of, - https://pikers.dev/goodboy/tractor/pulls/2 which solves py3.13's `asyncio` madness with SIGINT as well as various other oddities and benefits to latest the stdlib interfaces.. --- It's been a super long outstanding task (with multiple "attempts" on my part) to support engaging the `pdbp` REPL from sync code whilst keeping all the benefits of our normal `await tractor.pause()` version. *Finally that PR has come brethren, and you shall debug from whatever func you so desire.* --- #### Feats landed with this (via GH PR), - [x] basic `tractor.pause_from_sync()` support in the root actor. - [x] also in (nested) subactors.. => see the first draft of `examples/debugging/sync_bp.py` - [x] use `greenback.with_portal_run[_sync]()` in/around `pause_from_sync()` ? (https://greenback.readthedocs.io/en/latest/reference.html#greenback.with_portal_run) - ideally we don't have to do any manual call to `.ensure_portal()` since we only are using this all for `.pause_from_sync()`? - [x] main thread usage was the first thing working from commit b38ff36e04bee92beb7007e0e9c832f272cde6cb B) - [x] **with non-main-thread support** such that you can leverage all the benefits of a multi-core debugger even from another `threading.Thread` in any subactor-process. => Only works via `trio.from_thread.run_sync()` spawned bg threads from commit 8731ab3134832efd3d477ccb791c18edf688a478 - previously this didn't work since we always override the `SIGINT` handler in sub-actors with a shielder func and in python this is only allowed from the "main thread": https://docs.python.org/3/library/signal.html#signals-and-threads - [x] prolly implement this by checking the current thread's id and then using `trio.from_thread.run_sync()` to call `Lock.shield_sigint()` only when the current thread is not the main thread? - https://docs.python.org/3/library/threading.html#threading.get_ident - https://docs.python.org/3/library/threading.html#threading.main_thread - https://trio.readthedocs.io/en/stable/reference-core.html#trio.from_thread.run_sync - [x] planning for **future** infected-`asyncio`-support by first, - [x] preventing usage in the wild via dedicated error in `.devx._debug.pause_from_sync()` for now via an RTE if called from any infected-`asyncio.Task`. - [x] draft an xfail test suite to keep it on the radar for later. - [x] add `_debug.maybe_init_greenback()` call in the `run_as_asyncio_guest()` ep. --- #### Stuff moved to a downstream PR, Namely the new #2 Bo ##### ToDos for there from this `.pause_from_sync()` effort, See #2 for follow up of all the below! - [ ] from `asyncio` tasks when using `.to_asyncio`: - [ ] using both `.pause_from_sync()` and `breakpoint()` - [ ] ensure crashes in `asyncio` tasks engage correctly! - [x] decide on `greenback` bootstrapping: - [x] should it be an optional dep? => NO, instead we include it with the `--dev` optional deps - [x] always override the `breakpoint()` hook when it **is** installed? => YES, and we also raise a RTE on any `breakpoint()` usage from code that both does not spec `debug_mode=True` to `open_roo_actor()` as well as if no `greenback` is avail! - [x] when `greenback` is **not** installed how should we guard against `debug_mode=True` usage from sync code? => we raise an explicit RTE always around `breakpoint()` usage! - [x] where is the best place to call `devx._debug.maybe_init_greenback()`? => `_root` and in `_invoke()` tasks - [ ] support for infected-asyncio-mode in a root actor via the `tractor.to_asyncio.run_as_asyncio_guest()` entrypoint which can be now used over the std `trio.run()` from the first main/root process to open the runtime via either `.open_root_actor()` or `.open_nursery()`.
goodboy added 1 commit 2024-03-21 21:43:10 +00:00
goodboy added 2 commits 2024-03-24 20:47:40 +00:00
4f863a6989 Refine and test `tractor.pause_from_sync()` 
Now supports use from any `trio` task, any sync thread started with
`trio.to_thread.run_sync()` AND also via `breakpoint()` builtin API!
The only bit missing now is support for `asyncio` tasks when in infected
mode.. Bo

`greenback` setup/API adjustments:
- move `._rpc.maybe_import_gb()` to -> `devx._debug` and factor out the cached
  import checking into a sync func whilst placing the async `.ensure_portal()`
  bootstrapping into a new async `maybe_init_greenback()`.
- use the new init-er func inside `open_root_actor()` with the output
  predicating whether we override the `breakpoint()` hook.

core `devx._debug` implementation deatz:
- make `mk_mpdb()` only return the `pdp.Pdb` subtype instance since
  the sigint unshielding func is now accessible from the `Lock`
  singleton from anywhere.

- add non-main thread support (at least for `trio.to_thread` use cases)
  to our `Lock` with a new `.is_trio_thread()` predicate that delegates
  directly to `trio`'s internal version.

- do `Lock.is_trio_thread()` checks inside any methods which require
  special provisions when invoked from a non-main `trio` thread:
  - `.[un]shield_sigint()` methods since `signal.signal` usage is only
    allowed from cpython's main thread.
  - `.release()` since `trio.StrictFIFOLock` can only be called from
    a `trio` task.

- rework `.pause_from_sync()` itself to directly call `._set_trace()`
  and don't bother with `greenback._await()` when we're already calling
  it from a `.to_thread.run_sync()` thread, oh and try to use the
  thread/task name when setting `Lock.local_task_in_debug`.

- make it an RTE for now if you try to use `.pause_from_sync()` from any
  infected-`asyncio` task, but support is (hopefully) coming soon!

For testing we add a new `test_debugger.py::test_pause_from_sync()`
which includes a ctrl-c parametrization around the
`examples/debugging/sync_bp.py` script which includes all currently
supported/working usages:
- `tractor.pause_from_sync()`.
- via `breakpoint()` overload.
- from a `trio.to_thread.run_sync()` spawn.
0055c1d954 Tweak main thread predicate to ensure `trio.run()` 
Change the name to `Lock.is_main_trio_thread()` indicating that when
`True` the thread is both the main one **and** the one that called
`trio.run()`. Add a todo for just copying the
`trio._util.is_main_thread()` impl (since it's private / may change) and
some brief notes about potential usage of
`trio.from_thread.check_cancelled()` to detect non-`.to_thread` thread
spawns.
goodboy added 3 commits 2025-03-13 19:12:51 +00:00
2588e54867 Add todo-notes for hiding `@acm` frames 
In the particular case of the `Portal.open_context().__aexit__()` frame,
due to usage of `contextlib.asynccontextmanager`, we can't easily hook
into monkeypatching a `__tracebackhide__` set nor catch-n-reraise around
the block exit without defining our own `.__aexit__()` impl. Thus, it's
prolly most sane to do something with an override of
`contextlib._AsyncGeneratorContextManager` or the public exposed
`AsyncContextDecorator` (which uses the former internally right?).

Also fixup some old `._invoke` mod paths in comments and just show
`str(eoc)` in `.open_stream().__aexit__()` terminated-by-EoC log msg
since the `repr()` form won't pprint the IPC msg nicely..
goodboy changed title from Add `tractor.pause_from_sync()` using the amazing `greenback`! to Add `tractor.pause_from_sync()` using the amazing `greenback`! 2025-03-22 21:32:09 +00:00
goodboy changed target branch from mv_to_new_trio_py3.11 to main 2025-03-22 21:32:10 +00:00
goodboy commented 2025-03-22 22:39:37 +00:00
Poster
Owner

As mentioned..

Already merged in PR here, https://github.com/goodboy/tractor/pull/374

As mentioned.. Already merged in PR here, https://github.com/goodboy/tractor/pull/374
goodboy closed this pull request 2025-03-22 22:39:37 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: goodboy/tractor#1
There is no content yet.
Delete Branch "pause_from_sync_w_greenback"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?