Restore spawner-aware `ctlc` skip + richer skip-msg

Bring back two `tests/devx/conftest.py` refinements
whose carrying commit was otherwise superseded during
the sub-branch factoring:
- `ctlc` fixture takes `start_method` and documents the
  per-spawner fragility of ctl-c-in-REPL scenarios via
  a docstring + an (empty for now) allow-set for the
  `has_nested_actors` skip.
- `spawn` fixture's unsupported-backend skip msg now
  reports the requested backend + the supported set.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code

ai/conc-anal/trio_033_cancel_cascade_slowdown_depth3_issue.md

@@ -0,0 +1,102 @@
+# `trio` 0.29 -> 0.33 slows the depth=3 cancel-cascade
+
+## Symptom
+
+After locking to `trio==0.33.0` (commit `c7741bba`, was
+`0.29.0`), this test reliably trips its `fail_after`
+deadline on the **`trio`** backend:
+
+```
+FAILED tests/test_cancellation.py::test_nested_multierrors[start_method=trio-depth=3]
+  - AssertionError: assert False
+    where False = isinstance(
+      Cancelled(source='deadline', source_task=None, reason=None),
+      tractor.RemoteActorError,
+    )
+```
+
+A `fail_after_w_trace` hang-snapshot is captured for the
+test each run (deadline-injected `Cancelled` wrapped into
+the actor-nursery `BaseExceptionGroup`).
+
+## Root cause (immediate)
+
+The test budgets `fail_after(6)` for the `trio` backend.
+That 6s was chosen (commit `32955db0`, while `trio==0.29`)
+with the assertion that trio finishes "well under" 6s.
+The `trio` 0.29 -> 0.33 bump slowed the depth=3 cascade
+past that budget, so the 6s deadline now fires mid-cascade.
+
+trio 0.33 added **cancel-reason tracking** — every
+`Cancelled` now carries `(source=, reason=, source_task=)`.
+The injected exc is `Cancelled(source='deadline')`, i.e.
+trio itself naming our `fail_after(6)` scope as the cancel
+origin. When that `Cancelled` collapses one branch of the
+nursery BEG, the test's `isinstance(subexc,
+RemoteActorError)` assertion fails. The healthy outcome is
+`BEG = [RemoteActorError, RemoteActorError]`; the
+`Cancelled` is purely an artifact of the deadline cutting
+the cascade short.
+
+## Measurements (standalone, this machine)
+
+```
+depth=1  trio   ~3.15s   PASS  (keeps 6s budget)
+depth=3  trio   ~6.8-8.2s  FAIL @ 6s  (now bumped to 12s)
+```
+
+depth=1 still fits comfortably; only depth=3 (deeper
+recursive spawn-and-error tree => more actors to reap)
+exceeds the old budget. The ~2s/depth-level cost looks
+like serialized per-actor reap / `terminate_after` waits.
+
+## Mitigation applied
+
+`test_nested_multierrors` now splits the `trio` budget:
+
+```python
+case ('trio', 1):
+    timeout = 6
+case ('trio', 3):
+    timeout = 12   # was 6; see this doc
+```
+
+This stops the deadline from firing so the cascade
+completes naturally to `[RAE, RAE]`.
+
+## Also affected — same root cause, different test
+
+`test_echoserver_detailed_mechanics[trio-raise_error=KeyboardInterrupt]`
+(`tests/test_infected_asyncio.py`) tripped the *same*
+slowdown via its much tighter `trio` budget of `1s`. The
+single-aio-subactor teardown now takes ~1s, so the `1s`
+`fail_after` raced the deadline (PASS at 0.99s / FAIL at
+1.03s across back-to-back standalone runs). On a deadline-
+fire the injected `Cancelled(source='deadline')` wraps the
+mid-stream `KeyboardInterrupt` into a `BaseExceptionGroup`,
+which is NOT a `KeyboardInterrupt` so the bare
+`pytest.raises(KeyboardInterrupt)` fails. (The sibling
+`raise_error=Exception` variant only "passes" by accident:
+an `ExceptionGroup` *is-a* `Exception`, so its
+`pytest.raises(Exception)` still matches even when wrapped.)
+
+Mitigation: bump that `trio` budget `1 -> 4s` (matching the
+forking-spawner case). Without a deadline-fire the KBI
+propagates bare and the assertion passes.
+
+## Open follow-up (the actual regression)
+
+The budget bump is a band-aid — the underlying question is
+**why** the depth=3 `trio` cancel-cascade went from <6s to
+~7-8s across `trio` 0.29 -> 0.33. Candidate avenues:
+
+- which scope owns the per-actor `terminate_after` wait,
+  and are the tree's reaps concurrent or serialized?
+- did trio 0.33's abort/reschedule or cancel-reason
+  bookkeeping change checkpoint timing on the cancel path?
+
+If/when the cascade speeds back up under-budget, depth=3
+will start completing well under 12s — at which point the
+budget can be tightened back toward 6s as a regression
+tripwire. Related (different backend, same cascade class):
+`cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.

tests/devx/conftest.py

@@ -79,7 +79,8 @@ def spawn(
     }
     if start_method not in supported_spawners:
         pytest.skip(
-            '`pexpect` based tests only supported on `trio` backend'
+            f'`pexpect` based tests NOT supported on spawning-backend: {start_method!r}\n'
+            f'supported-spawners: {supported_spawners!r}'
         )
 
     def unset_colors():
@@ -211,21 +212,38 @@ def spawn(
 def ctlc(
     request: pytest.FixtureRequest,
     ci_env: bool,
-
+    start_method: str,
 ) -> bool:
+    '''
+    Parametrize and optionally skip tests which handle
+    ctlc-in-`pdbp`-REPL testing scenarios; certain spawners and actor-tree depths
+    cope very poorly with this..
 
+    In particular the spawning backends from `multiprocessing` are
+    fragile, as can be the default `trio` spawner under certain
+    conditions where SIGINT is relayed down the entire subproc tree.
+
+    '''
     use_ctlc: bool = request.param
     node = request.node
     markers = node.own_markers
     for mark in markers:
-        if mark.name == 'has_nested_actors':
+        if (
+            mark.name == 'has_nested_actors'
+            and
+            start_method not in {
+                # TODO, any spawners we should try again?
+                # - [ ] 'trio' but WITHOUT the SIGINT handler setup
+                #      per subproc?
+                # 'main_thread_forkserver',
+            }
+        ):
             pytest.skip(
                 f'Test {node} has nested actors and fails with Ctrl-C.\n'
                 f'The test can sometimes run fine locally but until'
                 ' we solve' 'this issue this CI test will be xfail:\n'
                 'https://github.com/goodboy/tractor/issues/320'
             )
-
         if (
             mark.name == 'ctlcs_bish'
             and

tests/test_advanced_streaming.py

@@ -10,6 +10,10 @@ from typing import Type
 import pytest
 import trio
 import tractor
+from tractor._testing.trace import (
+    AfkAlarmWTraceFactory,
+    FailAfterWTraceFactory,
+)
 
 
 def is_win():
@@ -150,6 +154,9 @@ def test_dynamic_pub_sub(
 
     is_forking_spawner: bool,
     set_fork_aware_capture,
+
+    fail_after_w_trace: FailAfterWTraceFactory,
+    afk_alarm_w_trace: AfkAlarmWTraceFactory,
 ):
     failed_to_raise_report: str = (
         f'Never got a {expect_cancel_exc!r} ??'
@@ -167,42 +174,36 @@ def test_dynamic_pub_sub(
     # a per-spawn cost (forkserver round-trip + IPC peer-handshake)
     # that can stack up over `cpus - 1` sequential `n.run_in_actor()`
     # calls — especially on UDS under cross-pytest contention
-    # (#451 / #452). Empirically a flat 15s flakes on
+    # (#451 / #452). 4s was flaking right at the edge under fork
-    # `main_thread_forkserver` for many-cpu hosts (a single bad
+    # backends — bumped to 8s with diag-snapshot-on-timeout via
-    # spawn-stack puts total run-time at ~15.5s, just over);
+    # `fail_after_w_trace` so a borderline run still fails loud
-    # 30s gives plenty of headroom while still failing-loud on
+    # but lands a ptree/wchan/py-spy dump in
-    # a real hang.
+    # `$XDG_CACHE_HOME/tractor/hung-dumps/` for inspection.
     #
-    # XXX caveat: this is an *inner* `trio.fail_after` — its
+    # XXX caveat: this is an *inner* trio cancel — its `Cancelled`
-    # `Cancelled` cannot reach a task parked in a shielded `await`
+    # cannot reach a task parked in a shielded `await` (e.g. inside
-    # (e.g. inside actor-nursery teardown). When the in-band cancel
+    # actor-nursery teardown). When the in-band cancel path is
-    # path is itself buggy (the bug-class-3 `raise KBI` swallow we're
+    # itself buggy (the bug-class-3 `raise KBI` swallow we're
-    # currently chasing) this guard does NOT fire and the test sits
+    # currently chasing) this guard does NOT fire and the test
-    # forever until external SIGINT. The `_DIAG_CAP_S` outer guard
+    # sits forever until external SIGINT. The `afk_alarm_w_trace`
-    # below is the AFK-safety counterpart.
+    # outer guard below is the AFK-safety counterpart (SIGALRM
+    # raises in the main thread regardless of trio scope state).
     fail_after_s: int = (
-        4
+        8
         if is_forking_spawner
-        else 12
+        else 20
     )
 
-    # outer guard: when the inner fail_after fails to fire because of
-    # a shielded-await deadlock, this cap *aborts the trio run via
-    # signal.alarm → KBI* so AFK runs don't sit for >20min on the
-    # bug-class-3 hang. Slightly larger than `fail_after_s` so the
-    # trio-native path always wins when it works.
-    _DIAG_CAP_S: int = fail_after_s + 5
-
     async def main():
         # bug-class-3 breadcrumb: tag each level of the cancel path
         # so when the run hangs and we capture cancel-level logs, the
         # *last* breadcrumb that fired names the swallow point.
         test_log.cancel('test_dynamic_pub_sub: enter main()')
         try:
-            with trio.fail_after(fail_after_s):
+            async with fail_after_w_trace(fail_after_s):
                 test_log.cancel(
                     f'test_dynamic_pub_sub: '
-                    f'enter `trio.fail_after({fail_after_s})` scope'
+                    f'enter `fail_after_w_trace({fail_after_s})` scope'
                 )
                 try:
                     async with tractor.open_nursery(
@@ -258,15 +259,7 @@ def test_dynamic_pub_sub(
                 'test_dynamic_pub_sub: leaving `main()`'
             )
 
-    # outer signal-based guard — survives a shielded-await deadlock
+    def _run_and_match():
-    # since `signal.alarm` raises in the main thread regardless of
-    # trio's scope state. ONLY armed under fork-based backends since
-    # the bug we're chasing is MTF-specific.
-    import signal
-    armed_alarm: bool = bool(is_forking_spawner)
-    if armed_alarm:
-        signal.alarm(_DIAG_CAP_S)
-    try:
         try:
             trio.run(main)
             pytest.fail(failed_to_raise_report)
@@ -287,11 +280,19 @@ def test_dynamic_pub_sub(
                 pytest.fail(failed_to_raise_report)
 
             test_log.exception('Got user-cancel exc AS EXPECTED')
-    finally:
+
-        # always disarm so a passing test doesn't get killed
+    # outer SIGALRM-based guard — survives a shielded-await
-        # post-trio.run by a stale alarm.
+    # deadlock since `signal.alarm` raises in the main thread
-        if armed_alarm:
+    # regardless of trio's scope state, AND captures a full diag
-            signal.alarm(0)
+    # snapshot to `$XDG_CACHE_HOME/tractor/hung-dumps/` before
+    # re-raising. ONLY armed under fork-based backends since the
+    # bug we're chasing is MTF-specific. Cap = `fail_after_s + 5`
+    # so the trio-native path always wins when it works.
+    if is_forking_spawner:
+        with afk_alarm_w_trace(fail_after_s + 5):
+            _run_and_match()
+    else:
+        _run_and_match()
 
 
 @tractor.context

tests/test_cancellation.py

@@ -7,6 +7,7 @@ import signal
 import platform
 import time
 from itertools import repeat
+from typing  import Type
 
 import pytest
 import trio
@@ -14,6 +15,7 @@ import tractor
 from tractor._testing import (
     tractor_test,
 )
+from tractor._testing.trace import FailAfterWTraceFactory
 from .conftest import no_windows
 
 
@@ -21,14 +23,44 @@ _non_linux: bool = platform.system() != 'Linux'
 _friggin_windows: bool = platform.system() == 'Windows'
 
 
-pytestmark = pytest.mark.skipon_spawn_backend(
+pytestmark = [
+    # Multi-actor cancel cascades under
+    # `--spawn-backend=subint` trip the abandoned-subint
+    # GIL-hostage class — a stuck subint can starve the
+    # parent's trio loop and block cancel-delivery.
+    # Apply the skip module-wide rather than per-test
+    # since every test here exercises the same cascade.
+    pytest.mark.skipon_spawn_backend(
         'subint',
         reason=(
-        'XXX SUBINT HANGING TEST XXX\n'
+            'XXX SUBINT GIL-CONTENTION HANGING TEST XXX\n'
-        'See oustanding issue(s)\n'
+            'Cancel cascades under '
-        # TODO, put issue link!
+            '`--spawn-backend=subint` trip the abandoned-subint '
-    )
+            'GIL-hostage class — see\n'
+            '  - `ai/conc-anal/subint_sigint_starvation_issue.md` '
+            '(GIL-hostage, SIGINT-unresponsive)\n'
+            '  - `ai/conc-anal/subint_cancel_delivery_hang_issue.md` '
+            '(sibling: parent parks on dead chan)\n'
+            '  - https://github.com/goodboy/tractor/issues/379 '
+            '(subint umbrella)\n'
         )
+    ),
+    pytest.mark.usefixtures(
+        'reap_subactors_per_test',
+        # NOTE, cancellation tests stress the SIGKILL
+        # `hard_kill` path which leaks UDS sock-files when
+        # the subactor's IPC server `finally:` cleanup
+        # doesn't run. Track per-test for blame attribution.
+        'track_orphaned_uds_per_test',
+        # NOTE, cancel-cascade timing races (see
+        # `test_nested_multierrors`) can also leave a
+        # subactor spinning at 100% CPU when its cancel
+        # signal got swallowed mid-handshake. Catches the
+        # runaway-loop class that doesn't leak UDS socks
+        # but burns the box.
+        'detect_runaway_subactors_per_test',
+    ),
+]
 
 
 async def assert_err(delay=0):
@@ -55,7 +87,11 @@ async def do_nuthin():
     ],
     ids=['no_args', 'unexpected_args'],
 )
-def test_remote_error(reg_addr, args_err):
+def test_remote_error(
+    reg_addr: tuple,
+    args_err: tuple[dict, Type[Exception]],
+    set_fork_aware_capture,
+):
     '''
     Verify an error raised in a subactor that is propagated
     to the parent nursery, contains the underlying boxed builtin
@@ -120,17 +156,10 @@ def test_remote_error(reg_addr, args_err):
             assert exc.boxed_type == errtype
 
 
-# @pytest.mark.skipon_spawn_backend(
-#     'subint',
-#     reason=(
-#         'XXX SUBINT HANGING TEST XXX\n'
-#         'See oustanding issue(s)\n'
-#         # TODO, put issue link!
-#     )
-# )
 def test_multierror(
     reg_addr: tuple[str, int],
-    start_method: str,
+    start_method: str,  # parametrized
+    set_fork_aware_capture, #: Callable,
 ):
     '''
     Verify we raise a ``BaseExceptionGroup`` out of a nursery where
@@ -175,6 +204,8 @@ def test_multierror_fast_nursery(
     start_method: str,
     num_subactors: int,
     delay: float,
+    set_fork_aware_capture,
+    fail_after_w_trace: FailAfterWTraceFactory,
 ):
     '''
     Verify we raise a ``BaseExceptionGroup`` out of a nursery where
@@ -183,6 +214,14 @@ def test_multierror_fast_nursery(
 
     '''
     async def main():
+        # budget = 2× natural trio-backend cascade time for
+        # 25 errorer subactors (~14s observed). on-timeout
+        # diag snapshot → if the cancel cascade hangs
+        # (observed under MTF backend with N>=14 errorer
+        # subactors) we get a fresh ptree/wchan/py-spy dump
+        # on disk INSTEAD of an opaque pytest timeout-kill.
+        # See `tractor/_testing/trace.py` for the helper.
+        async with fail_after_w_trace(30.0):
             async with tractor.open_nursery(
                 registry_addrs=[reg_addr],
             ) as nursery:
@@ -195,9 +234,23 @@ def test_multierror_fast_nursery(
                     )
 
     # with pytest.raises(trio.MultiError) as exc_info:
-    with pytest.raises(BaseExceptionGroup) as exc_info:
+    # NOTE, `trio.TooSlowError` from `fail_after_w_trace`
+    # bubbles UN-wrapped if `open_nursery.__aexit__` never
+    # gets re-entered; wrapped inside a `BaseExceptionGroup`
+    # if it did. Accept both shapes so the matcher itself
+    # doesn't lie about *what* failed.
+    with pytest.raises(
+        (BaseExceptionGroup, trio.TooSlowError),
+    ) as exc_info:
         trio.run(main)
 
+    if isinstance(exc_info.value, trio.TooSlowError):
+        pytest.fail(
+            f'cancel cascade hung past 12s '
+            f'(num_subactors={num_subactors}, delay={delay}); '
+            f'see stderr for `fail_after_w_trace` snapshot path'
+        )
+
     assert exc_info.type == ExceptionGroup
     err = exc_info.value
     exceptions = err.exceptions
@@ -277,6 +330,7 @@ async def stream_forever():
 async def test_cancel_infinite_streamer(
     reg_addr: tuple,
     start_method: str,
+    set_fork_aware_capture,
 ):
     # stream for at most 1 seconds
     with (
@@ -300,14 +354,6 @@ async def test_cancel_infinite_streamer(
     assert n.cancelled
 
 
-# @pytest.mark.skipon_spawn_backend(
-#     'subint',
-#     reason=(
-#         'XXX SUBINT HANGING TEST XXX\n'
-#         'See oustanding issue(s)\n'
-#         # TODO, put issue link!
-#     )
-# )
 @pytest.mark.parametrize(
     'num_actors_and_errs',
     [
@@ -345,6 +391,7 @@ async def test_some_cancels_all(
     reg_addr: tuple,
     start_method: str,
     loglevel: str,
+    set_fork_aware_capture, #: Callable,
 ):
     '''
     Verify a subset of failed subactors causes all others in
@@ -462,32 +509,130 @@ async def spawn_and_error(
 #     10,
 #     method='thread',
 # )
-@tractor_test
+@pytest.mark.parametrize(
+    'depth',
+    [1, 3],
+    ids='depth={}'.format,
+)
+@tractor_test(
+    # bumped from the 30s default to cover fork-based
+    # cancel-cascade flakes; 2 spawners × 2 errorers × depth 1+
+    # cascade through 6 portal-wait_for_result paths each
+    # paying `terminate_after=1.6s` + UDS sock-unlink under
+    # MTF/UDS contention can easily blow past 30s.
+    # Trio backend is fast and won't notice the extra budget.
+    # See `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
+    timeout=10,
+)
 async def test_nested_multierrors(
     reg_addr: tuple,
     loglevel: str,
     start_method: str,
+    set_fork_aware_capture,
+    fail_after_w_trace: FailAfterWTraceFactory,
+    request: pytest.FixtureRequest,
+    depth: int,
 ):
     '''
-    Test that failed actor sets are wrapped in `BaseExceptionGroup`s. This
+    Test that failed actor sets are wrapped in `BaseExceptionGroup`s.
-    test goes only 2 nurseries deep but we should eventually have tests
+
-    for arbitrary n-depth actor trees.
+    Parametrized over recursion `depth ∈ {1, 3}`:
+
+      - `depth=1`: shallow tree (2 spawners × 2 errorers, 2
+        levels). Cascade completes well within budget on ALL
+        backends including MTF — regression-safety green case.
+
+      - `depth=3`: deep tree (2 spawners × recursive depth-3
+        spawn-and-error). On `main_thread_forkserver` this
+        trips the cancel-cascade shape-mismatch bug class
+        (see `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`)
+        — xfailed below.
 
     '''
-    if start_method == 'trio':
+    # XXX: `multiprocessing.forkserver` can't handle nested
-        depth = 3
+    # spawning at any depth — hangs / broken-pipes. Pre-existing
-        subactor_breadth = 2
+    # backend limitation, NOT depth-specific.
-    else:
-        # XXX: multiprocessing can't seem to handle any more then 2 depth
-        # process trees for whatever reason.
-        # Any more process levels then this and we see bugs that cause
-        # hangs and broken pipes all over the place...
     if start_method == 'forkserver':
         pytest.skip("Forksever sux hard at nested spawning...")
-        depth = 1  # means an additional actor tree of spawning (2 levels deep)
+
     subactor_breadth = 2
 
-    with trio.fail_after(120):
+    # MTF backend trips a probabilistic timing race in the
+    # cancel-cascade — NOT depth-gated; depth amplifies the
+    # variance so depth=3 misses nearly every run while
+    # depth=1 misses occasionally. Both get the xfail mark
+    # (with `strict=False`) since the bug class can fire at
+    # either depth.
+    #
+    # The scenario in detail:
+    #
+    #     T=0      spawn spawner_0 + spawner_1 in parallel
+    #     T=t1     spawner_0's child errors →
+    #              RemoteActorError reaches root nursery
+    #     T=t1+ε   root nursery starts cancelling
+    #              spawner_1's portal-wait
+    #     T=t2     spawner_1's child errors → tries to send
+    #              RemoteActorError back
+    #
+    #     if t2 < t1+ε:  BEG = [RAE, RAE]        ← clean (xpass)
+    #     if t2 > t1+ε:  BEG = [RAE, Cancelled]  ← race tripped (xfail)
+    #
+    # i.e. the assertion below (`isinstance(_, RemoteActorError)`)
+    # fails iff cancel-delivery beats the other tree's natural
+    # error-propagation. Depth amplifies `t2-t1` variance
+    # (longer per-tree paths = more skew); under MTF the
+    # fork-spawn jitter + UDS-contention widens both `t1` and
+    # `t2` further.
+    #
+    # With `strict=False` the clean-cascade cases (most
+    # depth=1 runs, rare depth=3 runs) report as `xpassed`
+    # while the race-tripped cases report as `xfailed` —
+    # neither flakes `--lf`. When MTF cancel-cascade
+    # eventually speeds up enough to close the race even at
+    # depth=3, BOTH variants will reliably `xpass` and
+    # pytest will yell — our signal to drop the marker. See
+    # `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
+    if start_method == 'main_thread_forkserver':
+        request.node.add_marker(
+            pytest.mark.xfail(
+                strict=False,
+                reason=(
+                    f'MTF cancel-cascade shape-mismatch at '
+                    f'depth={depth} (Cancelled races '
+                    f'RemoteActorError in BEG); see conc-anal/'
+                    'cancel_cascade_too_slow_under_main_thread_forkserver_issue.md'
+                ),
+            )
+        )
+
+    # Per-backend/-depth budgets: in the non-hang case the
+    # whole spawn + cancel-cascade should complete in well
+    # under these. On the borderline hang case the
+    # `fail_after_w_trace` fires `TooSlowError` AND captures a
+    # ptree/wchan/py-spy snapshot to
+    # `$XDG_CACHE_HOME/tractor/hung-dumps/` for offline
+    # inspection. See
+    # `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
+    #
+    # NOTE: the `trio` depth=3 budget was bumped 6 -> 12s after
+    # the `trio` 0.29 -> 0.33 lock bump (commit c7741bba) slowed
+    # the depth-3 cancel-cascade from <6s to ~7-8s; the 6s
+    # deadline was firing and its `Cancelled(source='deadline')`
+    # (trio 0.33 cancel-reason metadata) collapsed a BEG branch,
+    # breaking the `RemoteActorError` assertion below. depth=1
+    # still finishes in ~3s so keeps the 6s budget. See
+    # `ai/conc-anal/trio_033_cancel_cascade_slowdown_depth3_issue.md`.
+    match (start_method, depth):
+        case ('trio', 1):
+            timeout = 6
+        case ('trio', 3):
+            timeout = 12
+        case ('main_thread_forkserver', 1):
+            timeout = 16
+        case ('main_thread_forkserver', 3):
+            timeout = 30
+
+    async with fail_after_w_trace(timeout):
         try:
             async with tractor.open_nursery() as nursery:
                 for i in range(subactor_breadth):
@@ -658,14 +803,6 @@ async def spawn_sub_with_sync_blocking_task():
         print('exiting first subactor layer..\n')
 
 
-# @pytest.mark.skipon_spawn_backend(
-#     'subint',
-#     reason=(
-#         'XXX SUBINT HANGING TEST XXX\n'
-#         'See oustanding issue(s)\n'
-#         # TODO, put issue link!
-#     )
-# )
 @pytest.mark.parametrize(
     'man_cancel_outer',
     [
@@ -685,7 +822,7 @@ async def spawn_sub_with_sync_blocking_task():
 def test_cancel_while_childs_child_in_sync_sleep(
     loglevel: str,
     start_method: str,
-    spawn_backend: str,
+    is_forking_spawner: bool,
     debug_mode: bool,
     reg_addr: tuple,
     man_cancel_outer: bool,
@@ -701,7 +838,10 @@ def test_cancel_while_childs_child_in_sync_sleep(
 
     '''
     if start_method == 'forkserver':
-        pytest.skip("Forksever sux hard at resuming from sync sleep...")
+        pytest.skip(
+            "`multiprocessing`'s forkserver sux hard at "
+            "resuming from sync sleep..."
+        )
 
     async def main():
         #
@@ -744,7 +884,11 @@ def test_cancel_while_childs_child_in_sync_sleep(
         # delay = 2  # is AssertionError in eg AND no TooSlowError !?
         # is AssertionError in eg AND no _cs cancellation.
         delay = (
-            6 if _non_linux
+            6 if (
+                _non_linux
+                or
+                is_forking_spawner
+            )
             else 4 
         )
 

tests/test_clustering.py

@@ -77,6 +77,7 @@ async def worker(
 @tractor_test
 async def test_streaming_to_actor_cluster(
     tpt_proto: str,
+    is_forking_spawner: bool,
 ):
     '''
     Open an actor "cluster" using the (experimental) `._clustering`
@@ -88,7 +89,11 @@ async def test_streaming_to_actor_cluster(
             f'Test currently fails with tpt-proto={tpt_proto!r}\n'
         )
 
-    with trio.fail_after(6):
+    delay: float = (
+        10 if is_forking_spawner
+        else 6
+    )
+    with trio.fail_after(delay):
         async with (
             open_actor_cluster(modules=[__name__]) as portals,
 

tests/test_infected_asyncio.py

@@ -30,6 +30,10 @@ from tractor import (
 from tractor.runtime import _state
 from tractor.trionics import BroadcastReceiver
 from tractor._testing import expect_ctxc
+from tractor._testing.trace import (
+    AfkAlarmWTraceFactory,
+    FailAfterWTraceFactory,
+)
 
 
 # Per-test zombie-subactor reaper. Opt-in (NOT autouse) —
@@ -45,6 +49,12 @@ from tractor._testing import expect_ctxc
 # `test_legacy_one_way_streaming`, etc.).
 pytestmark = pytest.mark.usefixtures(
     'reap_subactors_per_test',
+    # NOTE, asyncio cancel cascade has historically
+    # triggered both UDS sockfile leaks (SIGKILL path)
+    # AND the trio `WakeupSocketpair.drain()` busy-loop
+    # — see `test_aio_simple_error`'s history.
+    'track_orphaned_uds_per_test',
+    'detect_runaway_subactors_per_test',
 )
 
 
@@ -826,15 +836,45 @@ async def trio_to_aio_echo_server(
 
 @pytest.mark.parametrize(
     'raise_error_mid_stream',
-    [False, Exception, KeyboardInterrupt],
+    [
+        False,
+        Exception,
+        KeyboardInterrupt,
+    ],
     ids='raise_error={}'.format,
 )
 def test_echoserver_detailed_mechanics(
     reg_addr: tuple[str, int],
     debug_mode: bool,
     raise_error_mid_stream,
+
+    is_forking_spawner: bool,
+    fail_after_w_trace: FailAfterWTraceFactory,
 ):
-    async def main():
+    # NOTE: under fork-based backends the cancel-cascade
+    # path is structurally slower than `trio`'s subproc-exec
+    # (per-spawn forkserver-handshake compounds during
+    # teardown). Bump the cap so cross-test contamination
+    # doesn't flake this — see
+    # `ai/conc-anal/cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
+    timeout: float = (
+        999 if tractor.debug_mode()
+        else 4 if is_forking_spawner
+        # was 1; the `trio` 0.29 -> 0.33 bump slowed the
+        # cancel-cascade so a 1s budget raced the ~1s teardown
+        # deadline. On a deadline-fire the injected
+        # `Cancelled(source='deadline')` wraps the mid-stream
+        # KBI in a `BaseExceptionGroup`, breaking the bare
+        # `pytest.raises(KeyboardInterrupt)` below. See
+        # `ai/conc-anal/trio_033_cancel_cascade_slowdown_depth3_issue.md`.
+        else 4
+    )
+
+    # body factored out so the `fail_after_w_trace`-wrapping
+    # `main()` stays a 2-liner — keeps the deep `open_nursery`
+    # /`open_context`/`open_stream` block at its natural indent
+    # level instead of pushing it under yet another `async with`.
+    async def _body():
         async with tractor.open_nursery(
             registry_addrs=[reg_addr],
             debug_mode=debug_mode,
@@ -880,6 +920,15 @@ def test_echoserver_detailed_mechanics(
             # is cancelled by kbi or out of task cancellation
             await p.cancel_actor()
 
+    async def main():
+        # on-timeout diag snapshot via `fail_after_w_trace`
+        # — when the cancel cascade hangs under MTF we get a
+        # fresh `ptree`/`wchan`/`py-spy` dump on disk INSTEAD
+        # of an opaque pytest timeout-kill. See
+        # `tractor/_testing/trace.py`.
+        async with fail_after_w_trace(timeout):
+            await _body()
+
     if raise_error_mid_stream:
         with pytest.raises(raise_error_mid_stream):
             trio.run(main)
@@ -1038,7 +1087,8 @@ def test_sigint_closes_lifetime_stack(
     bg_aio_task: bool,
     trio_side_is_shielded: bool,
     send_sigint_to: str,
-    start_method: str,
+    is_forking_spawner: bool,
+    afk_alarm_w_trace: AfkAlarmWTraceFactory,
 ):
     '''
     Ensure that an infected child can use the `Actor.lifetime_stack`
@@ -1053,6 +1103,14 @@ def test_sigint_closes_lifetime_stack(
             if debug_mode
             else 1
         )
+        # pre-init so the `except (KeyboardInterrupt, ContextCancelled)`
+        # handler below doesn't `UnboundLocalError` if KBI fires BEFORE
+        # we ever enter the `as (ctx, first)` body (e.g. when
+        # `p.open_context().__aenter__` is hung waiting for the
+        # subactor's `StartAck` due to a fork-child IPC race —
+        # see `dynamic_pub_sub_spawn_time_transport_close_under_mtf_issue.md`).
+        tmp_file: Path|None = None
+        ctx: tractor.Context|None = None
         try:
             an: tractor.ActorNursery
             async with tractor.open_nursery(
@@ -1078,7 +1136,7 @@ def test_sigint_closes_lifetime_stack(
                 ) as (ctx, first):
 
                     path_str, cpid = first
-                    tmp_file: Path = Path(path_str)
+                    tmp_file = Path(path_str)
                     assert tmp_file.exists()
 
                     # XXX originally to simulate what (hopefully)
@@ -1129,7 +1187,7 @@ def test_sigint_closes_lifetime_stack(
                             if (
                                 send_sigint_to == 'child'
                                 and
-                                start_method == 'main_thread_forkserver'
+                                is_forking_spawner
                             ):
                                 pytest.xfail(
                                     reason=(
@@ -1156,6 +1214,21 @@ def test_sigint_closes_lifetime_stack(
             KeyboardInterrupt,
             ContextCancelled,
         ):
+            # If we got here BEFORE entering the ctx body (e.g.
+            # spawn-time IPC race hung `open_context.__aenter__` and
+            # the AFK-guard `signal.alarm` fired KBI from outside the
+            # trio loop), `tmp_file`/`ctx` are still `None` — surface
+            # that fact directly instead of `UnboundLocalError`.
+            if tmp_file is None:
+                pytest.fail(
+                    'KBI/ctxc fired BEFORE `p.open_context()` returned '
+                    "the child's `started` value — likely fork-child "
+                    'IPC race; see '
+                    '`ai/conc-anal/'
+                    'dynamic_pub_sub_spawn_time_transport_close_'
+                    'under_mtf_issue.md`'
+                )
+
             # XXX CASE 2: without the bug fixed, in the
             # KBI-raised-in-parent case, the actor teardown should
             # never get run (silently abaondoned by `asyncio`..) and
@@ -1163,6 +1236,25 @@ def test_sigint_closes_lifetime_stack(
             assert not tmp_file.exists()
             assert ctx.maybe_error
 
+    # outer hard wall-clock backstop via `afk_alarm_w_trace`:
+    # when the in-band trio cancel path doesn't fire (e.g.
+    # parent is parked in a shielded `await` inside actor-
+    # nursery teardown, or `open_context.__aenter__` hangs
+    # waiting for a child's `StartAck` that never comes), the
+    # `signal.alarm` inside the CM raises `AFKAlarmTimeout`
+    # in the main thread regardless of trio's scope state —
+    # AND captures a full diag snapshot to
+    # `$XDG_CACHE_HOME/tractor/hung-dumps/` before re-raising.
+    # Only armed under fork-based backends since this hang-
+    # class is MTF-specific.
+    if (
+        not debug_mode
+        and
+        is_forking_spawner
+    ):
+        with afk_alarm_w_trace(10):
+            trio.run(main)
+    else:
         trio.run(main)
 
 

tests/test_inter_peer_cancellation.py

@@ -26,14 +26,30 @@ from tractor._testing import (
 
 from .conftest import cpu_scaling_factor
 
-pytestmark = pytest.mark.skipon_spawn_backend(
+pytestmark = [
+    pytest.mark.skipon_spawn_backend(
         'subint',
         reason=(
             'XXX SUBINT GIL-CONTENTION HANGING TEST XXX\n'
-        'See oustanding issue(s)\n'
+            'Inter-peer cancel cascades under '
-        # TODO, put issue link!
+            '`--spawn-backend=subint` trip the abandoned-subint '
-    )
+            'GIL-hostage class — see\n'
+            '  - `ai/conc-anal/subint_sigint_starvation_issue.md` '
+            '(GIL-hostage, SIGINT-unresponsive)\n'
+            '  - `ai/conc-anal/subint_cancel_delivery_hang_issue.md` '
+            '(sibling: parent parks on dead chan)\n'
+            '  - https://github.com/goodboy/tractor/issues/379 '
+            '(subint umbrella)\n'
         )
+    ),
+    # NOTE, inter-peer cancellation tests stress the
+    # multi-actor cancel cascade which under SIGKILL
+    # leaves UDS sock-files orphaned. Track per-test
+    # for blame attribution.
+    pytest.mark.usefixtures(
+        'track_orphaned_uds_per_test',
+    ),
+]
 
 # XXX TODO cases:
 # - [x] WE cancelled the peer and thus should not see any raised

tractor/_root.py

@@ -155,7 +155,6 @@ async def maybe_block_bp(
             os.environ.pop('PYTHONBREAKPOINT', None)
 
 
-
 @acm
 async def open_root_actor(
     *,
@@ -186,6 +185,7 @@ async def open_root_actor(
     # enables the multi-process debugger support
     debug_mode: bool = False,
     maybe_enable_greenback: bool = False,  # `.pause_from_sync()/breakpoint()` support
+
     # ^XXX NOTE^ the perf implications of use,
     # https://greenback.readthedocs.io/en/latest/principle.html#performance
     enable_stack_on_sig: bool = False,