ai/conc-anal/trio_033_cancel_cascade_slowdown_depth3_issue.md

@@ -1,102 +0,0 @@
-# `trio` 0.29 -> 0.33 slows the depth=3 cancel-cascade
-
-## Symptom
-
-After locking to `trio==0.33.0` (commit `c7741bba`, was
-`0.29.0`), this test reliably trips its `fail_after`
-deadline on the **`trio`** backend:
-
-```
-FAILED tests/test_cancellation.py::test_nested_multierrors[start_method=trio-depth=3]
-  - AssertionError: assert False
-    where False = isinstance(
-      Cancelled(source='deadline', source_task=None, reason=None),
-      tractor.RemoteActorError,
-    )
-```
-
-A `fail_after_w_trace` hang-snapshot is captured for the
-test each run (deadline-injected `Cancelled` wrapped into
-the actor-nursery `BaseExceptionGroup`).
-
-## Root cause (immediate)
-
-The test budgets `fail_after(6)` for the `trio` backend.
-That 6s was chosen (commit `32955db0`, while `trio==0.29`)
-with the assertion that trio finishes "well under" 6s.
-The `trio` 0.29 -> 0.33 bump slowed the depth=3 cascade
-past that budget, so the 6s deadline now fires mid-cascade.
-
-trio 0.33 added **cancel-reason tracking** — every
-`Cancelled` now carries `(source=, reason=, source_task=)`.
-The injected exc is `Cancelled(source='deadline')`, i.e.
-trio itself naming our `fail_after(6)` scope as the cancel
-origin. When that `Cancelled` collapses one branch of the
-nursery BEG, the test's `isinstance(subexc,
-RemoteActorError)` assertion fails. The healthy outcome is
-`BEG = [RemoteActorError, RemoteActorError]`; the
-`Cancelled` is purely an artifact of the deadline cutting
-the cascade short.
-
-## Measurements (standalone, this machine)
-
-```
-depth=1  trio   ~3.15s   PASS  (keeps 6s budget)
-depth=3  trio   ~6.8-8.2s  FAIL @ 6s  (now bumped to 12s)
-```
-
-depth=1 still fits comfortably; only depth=3 (deeper
-recursive spawn-and-error tree => more actors to reap)
-exceeds the old budget. The ~2s/depth-level cost looks
-like serialized per-actor reap / `terminate_after` waits.
-
-## Mitigation applied
-
-`test_nested_multierrors` now splits the `trio` budget:
-
-```python
-case ('trio', 1):
-    timeout = 6
-case ('trio', 3):
-    timeout = 12   # was 6; see this doc
-```
-
-This stops the deadline from firing so the cascade
-completes naturally to `[RAE, RAE]`.
-
-## Also affected — same root cause, different test
-
-`test_echoserver_detailed_mechanics[trio-raise_error=KeyboardInterrupt]`
-(`tests/test_infected_asyncio.py`) tripped the *same*
-slowdown via its much tighter `trio` budget of `1s`. The
-single-aio-subactor teardown now takes ~1s, so the `1s`
-`fail_after` raced the deadline (PASS at 0.99s / FAIL at
-1.03s across back-to-back standalone runs). On a deadline-
-fire the injected `Cancelled(source='deadline')` wraps the
-mid-stream `KeyboardInterrupt` into a `BaseExceptionGroup`,
-which is NOT a `KeyboardInterrupt` so the bare
-`pytest.raises(KeyboardInterrupt)` fails. (The sibling
-`raise_error=Exception` variant only "passes" by accident:
-an `ExceptionGroup` *is-a* `Exception`, so its
-`pytest.raises(Exception)` still matches even when wrapped.)
-
-Mitigation: bump that `trio` budget `1 -> 4s` (matching the
-forking-spawner case). Without a deadline-fire the KBI
-propagates bare and the assertion passes.
-
-## Open follow-up (the actual regression)
-
-The budget bump is a band-aid — the underlying question is
-**why** the depth=3 `trio` cancel-cascade went from <6s to
-~7-8s across `trio` 0.29 -> 0.33. Candidate avenues:
-
-- which scope owns the per-actor `terminate_after` wait,
-  and are the tree's reaps concurrent or serialized?
-- did trio 0.33's abort/reschedule or cancel-reason
-  bookkeeping change checkpoint timing on the cancel path?
-
-If/when the cascade speeds back up under-budget, depth=3
-will start completing well under 12s — at which point the
-budget can be tightened back toward 6s as a regression
-tripwire. Related (different backend, same cascade class):
-`cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.

tests/devx/conftest.py

@@ -79,8 +79,7 @@ def spawn(
     }
     if start_method not in supported_spawners:
         pytest.skip(
-            f'`pexpect` based tests NOT supported on spawning-backend: {start_method!r}\n'
+            '`pexpect` based tests only supported on `trio` backend'
-            f'supported-spawners: {supported_spawners!r}'
         )
 
     def unset_colors():
@@ -212,38 +211,21 @@ def spawn(
 def ctlc(
     request: pytest.FixtureRequest,
     ci_env: bool,
-    start_method: str,
+
 ) -> bool:
-    '''
-    Parametrize and optionally skip tests which handle
-    ctlc-in-`pdbp`-REPL testing scenarios; certain spawners and actor-tree depths
-    cope very poorly with this..
 
-    In particular the spawning backends from `multiprocessing` are
-    fragile, as can be the default `trio` spawner under certain
-    conditions where SIGINT is relayed down the entire subproc tree.
-
-    '''
     use_ctlc: bool = request.param
     node = request.node
     markers = node.own_markers
     for mark in markers:
-        if (
+        if mark.name == 'has_nested_actors':
-            mark.name == 'has_nested_actors'
-            and
-            start_method not in {
-                # TODO, any spawners we should try again?
-                # - [ ] 'trio' but WITHOUT the SIGINT handler setup
-                #      per subproc?
-                # 'main_thread_forkserver',
-            }
-        ):
             pytest.skip(
                 f'Test {node} has nested actors and fails with Ctrl-C.\n'
                 f'The test can sometimes run fine locally but until'
                 ' we solve' 'this issue this CI test will be xfail:\n'
                 'https://github.com/goodboy/tractor/issues/320'
             )
+
         if (
             mark.name == 'ctlcs_bish'
             and

tests/test_advanced_streaming.py

@@ -10,10 +10,6 @@ from typing import Type
 import pytest
 import trio
 import tractor
-from tractor._testing.trace import (
-    AfkAlarmWTraceFactory,
-    FailAfterWTraceFactory,
-)
 
 
 def is_win():
@@ -154,9 +150,6 @@ def test_dynamic_pub_sub(
 
     is_forking_spawner: bool,
     set_fork_aware_capture,
-
-    fail_after_w_trace: FailAfterWTraceFactory,
-    afk_alarm_w_trace: AfkAlarmWTraceFactory,
 ):
     failed_to_raise_report: str = (
         f'Never got a {expect_cancel_exc!r} ??'
@@ -174,36 +167,42 @@ def test_dynamic_pub_sub(
     # a per-spawn cost (forkserver round-trip + IPC peer-handshake)
     # that can stack up over `cpus - 1` sequential `n.run_in_actor()`
     # calls — especially on UDS under cross-pytest contention
-    # (#451 / #452). 4s was flaking right at the edge under fork
+    # (#451 / #452). Empirically a flat 15s flakes on
-    # backends — bumped to 8s with diag-snapshot-on-timeout via
+    # `main_thread_forkserver` for many-cpu hosts (a single bad
-    # `fail_after_w_trace` so a borderline run still fails loud
+    # spawn-stack puts total run-time at ~15.5s, just over);
-    # but lands a ptree/wchan/py-spy dump in
+    # 30s gives plenty of headroom while still failing-loud on
-    # `$XDG_CACHE_HOME/tractor/hung-dumps/` for inspection.
+    # a real hang.
     #
-    # XXX caveat: this is an *inner* trio cancel — its `Cancelled`
+    # XXX caveat: this is an *inner* `trio.fail_after` — its
-    # cannot reach a task parked in a shielded `await` (e.g. inside
+    # `Cancelled` cannot reach a task parked in a shielded `await`
-    # actor-nursery teardown). When the in-band cancel path is
+    # (e.g. inside actor-nursery teardown). When the in-band cancel
-    # itself buggy (the bug-class-3 `raise KBI` swallow we're
+    # path is itself buggy (the bug-class-3 `raise KBI` swallow we're
-    # currently chasing) this guard does NOT fire and the test
+    # currently chasing) this guard does NOT fire and the test sits
-    # sits forever until external SIGINT. The `afk_alarm_w_trace`
+    # forever until external SIGINT. The `_DIAG_CAP_S` outer guard
-    # outer guard below is the AFK-safety counterpart (SIGALRM
+    # below is the AFK-safety counterpart.
-    # raises in the main thread regardless of trio scope state).
     fail_after_s: int = (
-        8
+        4
         if is_forking_spawner
-        else 20
+        else 12
     )
 
+    # outer guard: when the inner fail_after fails to fire because of
+    # a shielded-await deadlock, this cap *aborts the trio run via
+    # signal.alarm → KBI* so AFK runs don't sit for >20min on the
+    # bug-class-3 hang. Slightly larger than `fail_after_s` so the
+    # trio-native path always wins when it works.
+    _DIAG_CAP_S: int = fail_after_s + 5
+
     async def main():
         # bug-class-3 breadcrumb: tag each level of the cancel path
         # so when the run hangs and we capture cancel-level logs, the
         # *last* breadcrumb that fired names the swallow point.
         test_log.cancel('test_dynamic_pub_sub: enter main()')
         try:
-            async with fail_after_w_trace(fail_after_s):
+            with trio.fail_after(fail_after_s):
                 test_log.cancel(
                     f'test_dynamic_pub_sub: '
-                    f'enter `fail_after_w_trace({fail_after_s})` scope'
+                    f'enter `trio.fail_after({fail_after_s})` scope'
                 )
                 try:
                     async with tractor.open_nursery(
@@ -259,7 +258,15 @@ def test_dynamic_pub_sub(
                 'test_dynamic_pub_sub: leaving `main()`'
             )
 
-    def _run_and_match():
+    # outer signal-based guard — survives a shielded-await deadlock
+    # since `signal.alarm` raises in the main thread regardless of
+    # trio's scope state. ONLY armed under fork-based backends since
+    # the bug we're chasing is MTF-specific.
+    import signal
+    armed_alarm: bool = bool(is_forking_spawner)
+    if armed_alarm:
+        signal.alarm(_DIAG_CAP_S)
+    try:
         try:
             trio.run(main)
             pytest.fail(failed_to_raise_report)
@@ -280,19 +287,11 @@ def test_dynamic_pub_sub(
                 pytest.fail(failed_to_raise_report)
 
             test_log.exception('Got user-cancel exc AS EXPECTED')
-
+    finally:
-    # outer SIGALRM-based guard — survives a shielded-await
+        # always disarm so a passing test doesn't get killed
-    # deadlock since `signal.alarm` raises in the main thread
+        # post-trio.run by a stale alarm.
-    # regardless of trio's scope state, AND captures a full diag
+        if armed_alarm:
-    # snapshot to `$XDG_CACHE_HOME/tractor/hung-dumps/` before
+            signal.alarm(0)
-    # re-raising. ONLY armed under fork-based backends since the
-    # bug we're chasing is MTF-specific. Cap = `fail_after_s + 5`
-    # so the trio-native path always wins when it works.
-    if is_forking_spawner:
-        with afk_alarm_w_trace(fail_after_s + 5):
-            _run_and_match()
-    else:
-        _run_and_match()
 
 
 @tractor.context

tests/test_cancellation.py

@@ -7,7 +7,6 @@ import signal
 import platform
 import time
 from itertools import repeat
-from typing  import Type
 
 import pytest
 import trio
@@ -15,7 +14,6 @@ import tractor
 from tractor._testing import (
     tractor_test,
 )
-from tractor._testing.trace import FailAfterWTraceFactory
 from .conftest import no_windows
 
 
@@ -23,44 +21,14 @@ _non_linux: bool = platform.system() != 'Linux'
 _friggin_windows: bool = platform.system() == 'Windows'
 
 
-pytestmark = [
+pytestmark = pytest.mark.skipon_spawn_backend(
-    # Multi-actor cancel cascades under
+    'subint',
-    # `--spawn-backend=subint` trip the abandoned-subint
+    reason=(
-    # GIL-hostage class — a stuck subint can starve the
+        'XXX SUBINT HANGING TEST XXX\n'
-    # parent's trio loop and block cancel-delivery.
+        'See oustanding issue(s)\n'
-    # Apply the skip module-wide rather than per-test
+        # TODO, put issue link!
-    # since every test here exercises the same cascade.
+    )
-    pytest.mark.skipon_spawn_backend(
+)
-        'subint',
-        reason=(
-            'XXX SUBINT GIL-CONTENTION HANGING TEST XXX\n'
-            'Cancel cascades under '
-            '`--spawn-backend=subint` trip the abandoned-subint '
-            'GIL-hostage class — see\n'
-            '  - `ai/conc-anal/subint_sigint_starvation_issue.md` '
-            '(GIL-hostage, SIGINT-unresponsive)\n'
-            '  - `ai/conc-anal/subint_cancel_delivery_hang_issue.md` '
-            '(sibling: parent parks on dead chan)\n'
-            '  - https://github.com/goodboy/tractor/issues/379 '
-            '(subint umbrella)\n'
-        )
-    ),
-    pytest.mark.usefixtures(
-        'reap_subactors_per_test',
-        # NOTE, cancellation tests stress the SIGKILL
-        # `hard_kill` path which leaks UDS sock-files when
-        # the subactor's IPC server `finally:` cleanup
-        # doesn't run. Track per-test for blame attribution.
-        'track_orphaned_uds_per_test',
-        # NOTE, cancel-cascade timing races (see
-        # `test_nested_multierrors`) can also leave a
-        # subactor spinning at 100% CPU when its cancel
-        # signal got swallowed mid-handshake. Catches the
-        # runaway-loop class that doesn't leak UDS socks
-        # but burns the box.
-        'detect_runaway_subactors_per_test',
-    ),
-]
 
 
 async def assert_err(delay=0):
@@ -87,11 +55,7 @@ async def do_nuthin():
     ],
     ids=['no_args', 'unexpected_args'],
 )
-def test_remote_error(
+def test_remote_error(reg_addr, args_err):
-    reg_addr: tuple,
-    args_err: tuple[dict, Type[Exception]],
-    set_fork_aware_capture,
-):
     '''
     Verify an error raised in a subactor that is propagated
     to the parent nursery, contains the underlying boxed builtin
@@ -156,10 +120,17 @@ def test_remote_error(
             assert exc.boxed_type == errtype
 
 
+# @pytest.mark.skipon_spawn_backend(
+#     'subint',
+#     reason=(
+#         'XXX SUBINT HANGING TEST XXX\n'
+#         'See oustanding issue(s)\n'
+#         # TODO, put issue link!
+#     )
+# )
 def test_multierror(
     reg_addr: tuple[str, int],
-    start_method: str,  # parametrized
+    start_method: str,
-    set_fork_aware_capture, #: Callable,
 ):
     '''
     Verify we raise a ``BaseExceptionGroup`` out of a nursery where
@@ -204,8 +175,6 @@ def test_multierror_fast_nursery(
     start_method: str,
     num_subactors: int,
     delay: float,
-    set_fork_aware_capture,
-    fail_after_w_trace: FailAfterWTraceFactory,
 ):
     '''
     Verify we raise a ``BaseExceptionGroup`` out of a nursery where
@@ -214,43 +183,21 @@ def test_multierror_fast_nursery(
 
     '''
     async def main():
-        # budget = 2× natural trio-backend cascade time for
+        async with tractor.open_nursery(
-        # 25 errorer subactors (~14s observed). on-timeout
+            registry_addrs=[reg_addr],
-        # diag snapshot → if the cancel cascade hangs
+        ) as nursery:
-        # (observed under MTF backend with N>=14 errorer
-        # subactors) we get a fresh ptree/wchan/py-spy dump
-        # on disk INSTEAD of an opaque pytest timeout-kill.
-        # See `tractor/_testing/trace.py` for the helper.
-        async with fail_after_w_trace(30.0):
-            async with tractor.open_nursery(
-                registry_addrs=[reg_addr],
-            ) as nursery:
 
-                for i in range(num_subactors):
+            for i in range(num_subactors):
-                    await nursery.run_in_actor(
+                await nursery.run_in_actor(
-                        assert_err,
+                    assert_err,
-                        name=f'errorer{i}',
+                    name=f'errorer{i}',
-                        delay=delay
+                    delay=delay
-                    )
+                )
 
     # with pytest.raises(trio.MultiError) as exc_info:
-    # NOTE, `trio.TooSlowError` from `fail_after_w_trace`
+    with pytest.raises(BaseExceptionGroup) as exc_info:
-    # bubbles UN-wrapped if `open_nursery.__aexit__` never
-    # gets re-entered; wrapped inside a `BaseExceptionGroup`
-    # if it did. Accept both shapes so the matcher itself
-    # doesn't lie about *what* failed.
-    with pytest.raises(
-        (BaseExceptionGroup, trio.TooSlowError),
-    ) as exc_info:
         trio.run(main)
 
-    if isinstance(exc_info.value, trio.TooSlowError):
-        pytest.fail(
-            f'cancel cascade hung past 12s '
-            f'(num_subactors={num_subactors}, delay={delay}); '
-            f'see stderr for `fail_after_w_trace` snapshot path'
-        )
-
     assert exc_info.type == ExceptionGroup
     err = exc_info.value
     exceptions = err.exceptions
@@ -330,7 +277,6 @@ async def stream_forever():
 async def test_cancel_infinite_streamer(
     reg_addr: tuple,
     start_method: str,
-    set_fork_aware_capture,
 ):
     # stream for at most 1 seconds
     with (
@@ -354,6 +300,14 @@ async def test_cancel_infinite_streamer(
     assert n.cancelled
 
 
+# @pytest.mark.skipon_spawn_backend(
+#     'subint',
+#     reason=(
+#         'XXX SUBINT HANGING TEST XXX\n'
+#         'See oustanding issue(s)\n'
+#         # TODO, put issue link!
+#     )
+# )
 @pytest.mark.parametrize(
     'num_actors_and_errs',
     [
@@ -391,7 +345,6 @@ async def test_some_cancels_all(
     reg_addr: tuple,
     start_method: str,
     loglevel: str,
-    set_fork_aware_capture, #: Callable,
 ):
     '''
     Verify a subset of failed subactors causes all others in
@@ -509,130 +462,32 @@ async def spawn_and_error(
 #     10,
 #     method='thread',
 # )
-@pytest.mark.parametrize(
+@tractor_test
-    'depth',
-    [1, 3],
-    ids='depth={}'.format,
-)
-@tractor_test(
-    # bumped from the 30s default to cover fork-based
-    # cancel-cascade flakes; 2 spawners × 2 errorers × depth 1+
-    # cascade through 6 portal-wait_for_result paths each
-    # paying `terminate_after=1.6s` + UDS sock-unlink under
-    # MTF/UDS contention can easily blow past 30s.
-    # Trio backend is fast and won't notice the extra budget.
-    # See `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
-    timeout=10,
-)
 async def test_nested_multierrors(
     reg_addr: tuple,
     loglevel: str,
     start_method: str,
-    set_fork_aware_capture,
-    fail_after_w_trace: FailAfterWTraceFactory,
-    request: pytest.FixtureRequest,
-    depth: int,
 ):
     '''
-    Test that failed actor sets are wrapped in `BaseExceptionGroup`s.
+    Test that failed actor sets are wrapped in `BaseExceptionGroup`s. This
-
+    test goes only 2 nurseries deep but we should eventually have tests
-    Parametrized over recursion `depth ∈ {1, 3}`:
+    for arbitrary n-depth actor trees.
-
-      - `depth=1`: shallow tree (2 spawners × 2 errorers, 2
-        levels). Cascade completes well within budget on ALL
-        backends including MTF — regression-safety green case.
-
-      - `depth=3`: deep tree (2 spawners × recursive depth-3
-        spawn-and-error). On `main_thread_forkserver` this
-        trips the cancel-cascade shape-mismatch bug class
-        (see `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`)
-        — xfailed below.
 
     '''
-    # XXX: `multiprocessing.forkserver` can't handle nested
+    if start_method == 'trio':
-    # spawning at any depth — hangs / broken-pipes. Pre-existing
+        depth = 3
-    # backend limitation, NOT depth-specific.
+        subactor_breadth = 2
-    if start_method == 'forkserver':
+    else:
-        pytest.skip("Forksever sux hard at nested spawning...")
+        # XXX: multiprocessing can't seem to handle any more then 2 depth
+        # process trees for whatever reason.
+        # Any more process levels then this and we see bugs that cause
+        # hangs and broken pipes all over the place...
+        if start_method == 'forkserver':
+            pytest.skip("Forksever sux hard at nested spawning...")
+        depth = 1  # means an additional actor tree of spawning (2 levels deep)
+        subactor_breadth = 2
 
-    subactor_breadth = 2
+    with trio.fail_after(120):
-
-    # MTF backend trips a probabilistic timing race in the
-    # cancel-cascade — NOT depth-gated; depth amplifies the
-    # variance so depth=3 misses nearly every run while
-    # depth=1 misses occasionally. Both get the xfail mark
-    # (with `strict=False`) since the bug class can fire at
-    # either depth.
-    #
-    # The scenario in detail:
-    #
-    #     T=0      spawn spawner_0 + spawner_1 in parallel
-    #     T=t1     spawner_0's child errors →
-    #              RemoteActorError reaches root nursery
-    #     T=t1+ε   root nursery starts cancelling
-    #              spawner_1's portal-wait
-    #     T=t2     spawner_1's child errors → tries to send
-    #              RemoteActorError back
-    #
-    #     if t2 < t1+ε:  BEG = [RAE, RAE]        ← clean (xpass)
-    #     if t2 > t1+ε:  BEG = [RAE, Cancelled]  ← race tripped (xfail)
-    #
-    # i.e. the assertion below (`isinstance(_, RemoteActorError)`)
-    # fails iff cancel-delivery beats the other tree's natural
-    # error-propagation. Depth amplifies `t2-t1` variance
-    # (longer per-tree paths = more skew); under MTF the
-    # fork-spawn jitter + UDS-contention widens both `t1` and
-    # `t2` further.
-    #
-    # With `strict=False` the clean-cascade cases (most
-    # depth=1 runs, rare depth=3 runs) report as `xpassed`
-    # while the race-tripped cases report as `xfailed` —
-    # neither flakes `--lf`. When MTF cancel-cascade
-    # eventually speeds up enough to close the race even at
-    # depth=3, BOTH variants will reliably `xpass` and
-    # pytest will yell — our signal to drop the marker. See
-    # `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
-    if start_method == 'main_thread_forkserver':
-        request.node.add_marker(
-            pytest.mark.xfail(
-                strict=False,
-                reason=(
-                    f'MTF cancel-cascade shape-mismatch at '
-                    f'depth={depth} (Cancelled races '
-                    f'RemoteActorError in BEG); see conc-anal/'
-                    'cancel_cascade_too_slow_under_main_thread_forkserver_issue.md'
-                ),
-            )
-        )
-
-    # Per-backend/-depth budgets: in the non-hang case the
-    # whole spawn + cancel-cascade should complete in well
-    # under these. On the borderline hang case the
-    # `fail_after_w_trace` fires `TooSlowError` AND captures a
-    # ptree/wchan/py-spy snapshot to
-    # `$XDG_CACHE_HOME/tractor/hung-dumps/` for offline
-    # inspection. See
-    # `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
-    #
-    # NOTE: the `trio` depth=3 budget was bumped 6 -> 12s after
-    # the `trio` 0.29 -> 0.33 lock bump (commit c7741bba) slowed
-    # the depth-3 cancel-cascade from <6s to ~7-8s; the 6s
-    # deadline was firing and its `Cancelled(source='deadline')`
-    # (trio 0.33 cancel-reason metadata) collapsed a BEG branch,
-    # breaking the `RemoteActorError` assertion below. depth=1
-    # still finishes in ~3s so keeps the 6s budget. See
-    # `ai/conc-anal/trio_033_cancel_cascade_slowdown_depth3_issue.md`.
-    match (start_method, depth):
-        case ('trio', 1):
-            timeout = 6
-        case ('trio', 3):
-            timeout = 12
-        case ('main_thread_forkserver', 1):
-            timeout = 16
-        case ('main_thread_forkserver', 3):
-            timeout = 30
-
-    async with fail_after_w_trace(timeout):
         try:
             async with tractor.open_nursery() as nursery:
                 for i in range(subactor_breadth):
@@ -803,6 +658,14 @@ async def spawn_sub_with_sync_blocking_task():
         print('exiting first subactor layer..\n')
 
 
+# @pytest.mark.skipon_spawn_backend(
+#     'subint',
+#     reason=(
+#         'XXX SUBINT HANGING TEST XXX\n'
+#         'See oustanding issue(s)\n'
+#         # TODO, put issue link!
+#     )
+# )
 @pytest.mark.parametrize(
     'man_cancel_outer',
     [
@@ -822,7 +685,7 @@ async def spawn_sub_with_sync_blocking_task():
 def test_cancel_while_childs_child_in_sync_sleep(
     loglevel: str,
     start_method: str,
-    is_forking_spawner: bool,
+    spawn_backend: str,
     debug_mode: bool,
     reg_addr: tuple,
     man_cancel_outer: bool,
@@ -838,10 +701,7 @@ def test_cancel_while_childs_child_in_sync_sleep(
 
     '''
     if start_method == 'forkserver':
-        pytest.skip(
+        pytest.skip("Forksever sux hard at resuming from sync sleep...")
-            "`multiprocessing`'s forkserver sux hard at "
-            "resuming from sync sleep..."
-        )
 
     async def main():
         #
@@ -884,11 +744,7 @@ def test_cancel_while_childs_child_in_sync_sleep(
         # delay = 2  # is AssertionError in eg AND no TooSlowError !?
         # is AssertionError in eg AND no _cs cancellation.
         delay = (
-            6 if (
+            6 if _non_linux
-                _non_linux
-                or
-                is_forking_spawner
-            )
             else 4 
         )
 

tests/test_clustering.py

@@ -77,7 +77,6 @@ async def worker(
 @tractor_test
 async def test_streaming_to_actor_cluster(
     tpt_proto: str,
-    is_forking_spawner: bool,
 ):
     '''
     Open an actor "cluster" using the (experimental) `._clustering`
@@ -89,11 +88,7 @@ async def test_streaming_to_actor_cluster(
             f'Test currently fails with tpt-proto={tpt_proto!r}\n'
         )
 
-    delay: float = (
+    with trio.fail_after(6):
-        10 if is_forking_spawner
-        else 6
-    )
-    with trio.fail_after(delay):
         async with (
             open_actor_cluster(modules=[__name__]) as portals,
 

tests/test_infected_asyncio.py

@@ -30,10 +30,6 @@ from tractor import (
 from tractor.runtime import _state
 from tractor.trionics import BroadcastReceiver
 from tractor._testing import expect_ctxc
-from tractor._testing.trace import (
-    AfkAlarmWTraceFactory,
-    FailAfterWTraceFactory,
-)
 
 
 # Per-test zombie-subactor reaper. Opt-in (NOT autouse) —
@@ -49,12 +45,6 @@ from tractor._testing.trace import (
 # `test_legacy_one_way_streaming`, etc.).
 pytestmark = pytest.mark.usefixtures(
     'reap_subactors_per_test',
-    # NOTE, asyncio cancel cascade has historically
-    # triggered both UDS sockfile leaks (SIGKILL path)
-    # AND the trio `WakeupSocketpair.drain()` busy-loop
-    # — see `test_aio_simple_error`'s history.
-    'track_orphaned_uds_per_test',
-    'detect_runaway_subactors_per_test',
 )
 
 
@@ -836,45 +826,15 @@ async def trio_to_aio_echo_server(
 
 @pytest.mark.parametrize(
     'raise_error_mid_stream',
-    [
+    [False, Exception, KeyboardInterrupt],
-        False,
-        Exception,
-        KeyboardInterrupt,
-    ],
     ids='raise_error={}'.format,
 )
 def test_echoserver_detailed_mechanics(
     reg_addr: tuple[str, int],
     debug_mode: bool,
     raise_error_mid_stream,
-
-    is_forking_spawner: bool,
-    fail_after_w_trace: FailAfterWTraceFactory,
 ):
-    # NOTE: under fork-based backends the cancel-cascade
+    async def main():
-    # path is structurally slower than `trio`'s subproc-exec
-    # (per-spawn forkserver-handshake compounds during
-    # teardown). Bump the cap so cross-test contamination
-    # doesn't flake this — see
-    # `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
-    timeout: float = (
-        999 if tractor.debug_mode()
-        else 4 if is_forking_spawner
-        # was 1; the `trio` 0.29 -> 0.33 bump slowed the
-        # cancel-cascade so a 1s budget raced the ~1s teardown
-        # deadline. On a deadline-fire the injected
-        # `Cancelled(source='deadline')` wraps the mid-stream
-        # KBI in a `BaseExceptionGroup`, breaking the bare
-        # `pytest.raises(KeyboardInterrupt)` below. See
-        # `ai/conc-anal/trio_033_cancel_cascade_slowdown_depth3_issue.md`.
-        else 4
-    )
-
-    # body factored out so the `fail_after_w_trace`-wrapping
-    # `main()` stays a 2-liner — keeps the deep `open_nursery`
-    # /`open_context`/`open_stream` block at its natural indent
-    # level instead of pushing it under yet another `async with`.
-    async def _body():
         async with tractor.open_nursery(
             registry_addrs=[reg_addr],
             debug_mode=debug_mode,
@@ -920,15 +880,6 @@ def test_echoserver_detailed_mechanics(
             # is cancelled by kbi or out of task cancellation
             await p.cancel_actor()
 
-    async def main():
-        # on-timeout diag snapshot via `fail_after_w_trace`
-        # — when the cancel cascade hangs under MTF we get a
-        # fresh `ptree`/`wchan`/`py-spy` dump on disk INSTEAD
-        # of an opaque pytest timeout-kill. See
-        # `tractor/_testing/trace.py`.
-        async with fail_after_w_trace(timeout):
-            await _body()
-
     if raise_error_mid_stream:
         with pytest.raises(raise_error_mid_stream):
             trio.run(main)
@@ -1087,8 +1038,7 @@ def test_sigint_closes_lifetime_stack(
     bg_aio_task: bool,
     trio_side_is_shielded: bool,
     send_sigint_to: str,
-    is_forking_spawner: bool,
+    start_method: str,
-    afk_alarm_w_trace: AfkAlarmWTraceFactory,
 ):
     '''
     Ensure that an infected child can use the `Actor.lifetime_stack`
@@ -1103,14 +1053,6 @@ def test_sigint_closes_lifetime_stack(
             if debug_mode
             else 1
         )
-        # pre-init so the `except (KeyboardInterrupt, ContextCancelled)`
-        # handler below doesn't `UnboundLocalError` if KBI fires BEFORE
-        # we ever enter the `as (ctx, first)` body (e.g. when
-        # `p.open_context().__aenter__` is hung waiting for the
-        # subactor's `StartAck` due to a fork-child IPC race —
-        # see `dynamic_pub_sub_spawn_time_transport_close_under_mtf_issue.md`).
-        tmp_file: Path|None = None
-        ctx: tractor.Context|None = None
         try:
             an: tractor.ActorNursery
             async with tractor.open_nursery(
@@ -1136,7 +1078,7 @@ def test_sigint_closes_lifetime_stack(
                 ) as (ctx, first):
 
                     path_str, cpid = first
-                    tmp_file = Path(path_str)
+                    tmp_file: Path = Path(path_str)
                     assert tmp_file.exists()
 
                     # XXX originally to simulate what (hopefully)
@@ -1187,7 +1129,7 @@ def test_sigint_closes_lifetime_stack(
                             if (
                                 send_sigint_to == 'child'
                                 and
-                                is_forking_spawner
+                                start_method == 'main_thread_forkserver'
                             ):
                                 pytest.xfail(
                                     reason=(
@@ -1214,21 +1156,6 @@ def test_sigint_closes_lifetime_stack(
             KeyboardInterrupt,
             ContextCancelled,
         ):
-            # If we got here BEFORE entering the ctx body (e.g.
-            # spawn-time IPC race hung `open_context.__aenter__` and
-            # the AFK-guard `signal.alarm` fired KBI from outside the
-            # trio loop), `tmp_file`/`ctx` are still `None` — surface
-            # that fact directly instead of `UnboundLocalError`.
-            if tmp_file is None:
-                pytest.fail(
-                    'KBI/ctxc fired BEFORE `p.open_context()` returned '
-                    "the child's `started` value — likely fork-child "
-                    'IPC race; see '
-                    '`ai/conc-anal/'
-                    'dynamic_pub_sub_spawn_time_transport_close_'
-                    'under_mtf_issue.md`'
-                )
-
             # XXX CASE 2: without the bug fixed, in the
             # KBI-raised-in-parent case, the actor teardown should
             # never get run (silently abaondoned by `asyncio`..) and
@@ -1236,26 +1163,7 @@ def test_sigint_closes_lifetime_stack(
             assert not tmp_file.exists()
             assert ctx.maybe_error
 
-    # outer hard wall-clock backstop via `afk_alarm_w_trace`:
+    trio.run(main)
-    # when the in-band trio cancel path doesn't fire (e.g.
-    # parent is parked in a shielded `await` inside actor-
-    # nursery teardown, or `open_context.__aenter__` hangs
-    # waiting for a child's `StartAck` that never comes), the
-    # `signal.alarm` inside the CM raises `AFKAlarmTimeout`
-    # in the main thread regardless of trio's scope state —
-    # AND captures a full diag snapshot to
-    # `$XDG_CACHE_HOME/tractor/hung-dumps/` before re-raising.
-    # Only armed under fork-based backends since this hang-
-    # class is MTF-specific.
-    if (
-        not debug_mode
-        and
-        is_forking_spawner
-    ):
-        with afk_alarm_w_trace(10):
-            trio.run(main)
-    else:
-        trio.run(main)
 
 
 

tests/test_inter_peer_cancellation.py

@@ -26,30 +26,14 @@ from tractor._testing import (
 
 from .conftest import cpu_scaling_factor
 
-pytestmark = [
+pytestmark = pytest.mark.skipon_spawn_backend(
-    pytest.mark.skipon_spawn_backend(
+    'subint',
-        'subint',
+    reason=(
-        reason=(
+        'XXX SUBINT GIL-CONTENTION HANGING TEST XXX\n'
-            'XXX SUBINT GIL-CONTENTION HANGING TEST XXX\n'
+        'See oustanding issue(s)\n'
-            'Inter-peer cancel cascades under '
+        # TODO, put issue link!
-            '`--spawn-backend=subint` trip the abandoned-subint '
+    )
-            'GIL-hostage class — see\n'
+)
-            '  - `ai/conc-anal/subint_sigint_starvation_issue.md` '
-            '(GIL-hostage, SIGINT-unresponsive)\n'
-            '  - `ai/conc-anal/subint_cancel_delivery_hang_issue.md` '
-            '(sibling: parent parks on dead chan)\n'
-            '  - https://github.com/goodboy/tractor/issues/379 '
-            '(subint umbrella)\n'
-        )
-    ),
-    # NOTE, inter-peer cancellation tests stress the
-    # multi-actor cancel cascade which under SIGKILL
-    # leaves UDS sock-files orphaned. Track per-test
-    # for blame attribution.
-    pytest.mark.usefixtures(
-        'track_orphaned_uds_per_test',
-    ),
-]
 
 # XXX TODO cases:
 # - [x] WE cancelled the peer and thus should not see any raised

tractor/_root.py

@@ -155,6 +155,7 @@ async def maybe_block_bp(
             os.environ.pop('PYTHONBREAKPOINT', None)
 
 
+
 @acm
 async def open_root_actor(
     *,
@@ -185,7 +186,6 @@ async def open_root_actor(
     # enables the multi-process debugger support
     debug_mode: bool = False,
     maybe_enable_greenback: bool = False,  # `.pause_from_sync()/breakpoint()` support
-
     # ^XXX NOTE^ the perf implications of use,
     # https://greenback.readthedocs.io/en/latest/principle.html#performance
     enable_stack_on_sig: bool = False,

tractor/runtime/_state.py

@@ -140,7 +140,7 @@ _RUNTIME_VARS_DEFAULTS: dict[str, Any] = {
     # `debug_mode: bool` settings
     '_debug_mode': False,  # bool
     'repl_fixture': False,  # |AbstractContextManager[bool]
-
+    
     'use_greenback': False,  # `.pause_from_sync()`/`breakpoint()`
     'use_stackscope': False,  # trio-task-stack dumps on SIGUSR1