Prevent asyncio
from abandoning guest-runs, .pause_from_sync()
support via .to_asyncio
#2
Loading…
Reference in New Issue
There is no content yet.
Delete Branch "aio_abandons"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
On
asyncio
being super lovely and abandoning our guest-run..Took me a while to figure out what the heck was going on but, turns out
asyncio
changed their SIGINT handling in 3.11 as per:https://docs.python.org/3/library/asyncio-runner.html#handling-keyboard-interruption
I’m not entirely sure if it’s the 3.11 changes or possibly wtv further updates were made in 3.12 but more or less due to the way our current main task was written the
trio
guest-run was getting abandoned on SIGINTs sent from the OS to the infected child proc..Note that much of the bug and soln cases are layed out in very detailed comment-notes both in the new test and
run_as_asyncio_guest()
, right above the final “fix” lines.The (seemingly working) “fix” required 2 lines of code to be run inside a
asyncio.CancelledError
handler around the call toawait trio_done_fut
:Actor.cancel_soon()
which schedules the actor runtime to cancel on the nexttrio
runner cycle and results in a “self cancellation” of the actor.asyncio
event loop” with a non-0.sleep(0.1)
XD |_ seems that a “shielded” pump with some actualdelay: float >= 0
did the trick to getasyncio
to allow thetrio
runner/loop to fully complete its guest-run without abandonment.Much improved
asyncio
-mode support driven by upcoming py3.13 support,better handling of simultaneous but “independent”
trio
vs.asyncio.Task
errors such that they are raised in an eg per1ff79f86b7
support for infected-asyncio-mode in a root actor via the
tractor.to_asyncio.run_as_asyncio_guest()
entrypoint which can be now used over the stdtrio.run()
from the first main/root process to open the runtime via either.open_root_actor()
or.open_nursery()
(delivered in commit4a195eef4c
)New test suites/extensions introduced here,
a new
tests/devx
subpkg for all tooling affiliated testsofficially testing the proto-ed
stackscope
integrationa new
examples/debugging/asyncio_bp.py
to verifyto_asyncio
based support when usingtractor.pause_from_sync()
/breakpoint()
fromasyncio.Task
s.extensions/reworks to existing
test_infected_asyncio
suite generally as part of72035a20d7
,test_trio_prestarted_task_bubbles
test_trio_closes_early_and_channel_exits
test_aio_cancelled_from_aio_causes_trio_cancelled
adding a new dedicated
tests/test_root_infect_asyncio
suite inf26d487000
to pair with new root actor support (see below).a new
examples/debugging/restore_builtin_breakpoint.py
andtest_breakpoint_hook_restored
to verifybreakpoint()
restoration in9af6271e99
a356233b47
but was able to catch that muck-up thanks to this very bullet list! XD now brought back to life ine8111e40f9
History of outstandings from the original
.pause_from_sync()
and debug REPL from
asyncio.Task
s effort:There chronology started on github in a WIP PR,
but then was further followed up by adding a new
tractor.devx
sub-pkg again via github in PR,There was also an original PR proposed in (gitea) #1 but i (by old habbit/accident) landed it via a new GH PR,
TODO list from GH #374
To be solved here obvi!
trio.to_thread.run_sync()
32e12c8b03
5cdd012417
d9662d9b34
and7443e387b5
.asyncio
tasks when using our.to_asyncio
subsys:.pause_from_sync()
andbreakpoint()
which are verified supported as of commita356233b47
BUT ALSO shows (via test) that crashes inside anasyncio.Task
engage crash-handling correctly (at least mostly)!greenback
bootstrapping:uv install --dev
optional depsbreakpoint()
hook when it is installed?breakpoint()
usage from code that both does not specdebug_mode=True
toopen_root_actor()
as well as if nogreenback
is avail! as of test in9af6271e99
greenback
is not installed how should we guard againstdebug_mode=True
usage from sync code?breakpoint()
usage as per test from9af6271e99
asyncio.Task
” tries to call breakpoint as per1afef149d4
b7aa72465d
a newtest_sync_pause_from_non_greenbacked_aio_task
for maybe attempting to support this somehow in the future?devx._debug.maybe_init_greenback()
?_root
and in_invoke()
tasks supported from commit9811db9ac5
which (will) land(s) in the upstream #7Unrelated improvements thrown in,
Stuff that was deemed (historically) necessary enough to land alongside all the above,
72fc6fce24
Support for passing pre-conf-edLogger
; super handy for gettingtractor
-styled console pretty-formatting around an external sys/lib’slogging
usage/config.deliver a new a boxed-maybe-error from
open_crash_handler()
for post crash introspection (often for testing) purposes ina60837550e
example “raise-from-
finally:
” intrio
nursery test which demos a footgun2bd4cc9727
with a potential “holster” solution for unmasking the underlying suppressed errors in such cases in1075ea3687
Maybe to cherry from
py313_support
andext_type_plds
branches?py313_support
,8573cd3
Tweak some test asserts to betteris
style4de4897
Unset$PYTHON_COLORS
for test debugger suite..1f951a9
Anotheris
fix..08fa266
Add per-side graceful-exit/cancel excs-as-signals985c5a4
Moredebug_mode
test support, better nursery var names60eca81
Be extra sure to re-raise EoCs from translator5ff2740
Add a mark topytest.xfail()
questionably conc py stuff (ur mam.xfail()
s bish!)e313cb5
Repair/updatestackscope
testext_type_plds
,90287b9
Fix anaio_err
ref bug3d54885
Continue supporting py3.11+.to_asyncio
anyway?a66caa2
Dropasyncio
-canc error from._exceptions
47ec7e7
Add equiv ofAsyncioCancelled
for aio side5ed30dec40
to284fa0340e
284fa0340e
toa870df68c0
547b957bbf
tof7469442e3
a69bc00593
to71cf9e7bd3
71cf9e7bd3
to3b39cce741
9002f60
howtorelease.md filea1d75625e4
tob7aa72465d
Hack `asyncio` to not abandon a guest-mode run?to Hack `asyncio` to not abandon a guest-mode run, `.pause_from_sync()` support via `.to_asyncio`Hack `asyncio` to not abandon a guest-mode run, `.pause_from_sync()` support via `.to_asyncio`to Prevent `asyncio` from abandoning guest-runs, `.pause_from_sync()` support via `.to_asyncio`e646ce5c0d
to15f99c313e
I mean anyone wanting to click approve (since they already built a buncha sh#! on top of this ;) would allow us to conduct the (normal) formal [boom, rhyme time] protocol to all things “community” and “foss”..
@guille
Prevent `asyncio` from abandoning guest-runs, `.pause_from_sync()` support via `.to_asyncio`to Prevent `asyncio` from abandoning guest-runs, `.pause_from_sync()` support via `.to_asyncio`010d75248e
toc91373148a
@ -82,6 +82,48 @@ class InternalError(RuntimeError):
'''
class AsyncioCancelled(Exception):
This is the main error-translation-semantics that changed, more or less being more pedantic about which side errored/cancelled/exited-gracefully and whether it was independent of the other side.
@ -40,0 +39,4 @@
from tractor._exceptions import (
InternalError,
is_multi_cancelled,
TrioTaskExited,
For back-lookers (from the future) these new excs drove the improved error translation semantics throughout the cancel and exit handling machinery.
This file’s diff consists of the bulk of the changes described by the PR description.
@ -73,0 +162,4 @@
# self._final_result_is_set()
# )
async def wait_for_result(
Replicating the same outcome waiting API as
Context
.@ -442,0 +987,4 @@
# TODO? factor the next 2 branches into a func like
# `try_terminate_aio_task()` and use it for the taskc
# case above as well?
fut: asyncio.Future|None = aio_task._fut_waiter
This is one of the critical-yet-questionable changes;
asyncio.Task.cancel()
seems to never work reliably and can often cause full guest-run abandonment, so instead we take the approach of touching any internalFuture
first and hoping for the best (which seems to work in practise!).@ -516,0 +1218,4 @@
# a `Return`-msg for IPC ctxs)
aio_task: asyncio.Task = chan._aio_task
if not aio_task.done():
fut: asyncio.Future|None = aio_task._fut_waiter
Same as mentioned above; appears to be the best/most-reliable hack for the moment..
@ -516,0 +1242,4 @@
'''
def run_trio_task_in_future(
Much thanks to @oremanj (from core
trio
team on GH) for this fn!@ -1,8 +1,16 @@
'''
This is a pretty important step forward for the debugger REPL tooling since now you can definitely get multi-actor safe pausing from infected-
asyncio
actors including crash handling B)