ai/conc-anal/ria_nursery_removal_plan.md

@@ -1,187 +0,0 @@
-# `_ria_nursery` removal plan (issue #477 follow-up)
-
-Goal: drop the secondary "run-in-actor" spawn nursery (and
-friends) from `ActorNursery`/spawn internals, now that
-`tractor.to_actor.run()` delivers one-shot semantics purely on
-the daemon-spawn + portal primitives.
-
-## Verified machinery map (2026-07-02, wkt @ a34aaf98)
-
-The entire mechanism is 4 files:
-
-- `runtime/_supervise.py`
-  - `ActorNursery.__init__(.., ria_nursery, ..)` stores
-    `._ria_nursery` (:202, :238); sole read is
-    `run_in_actor()` passing `nursery=self._ria_nursery`
-    (:442) into `start_actor()`'s `nursery:
-    trio.Nursery|None` escape-hatch param (:305, :367).
-  - `._cancel_after_result_on_exit: set` (:244) marks ria
-    portals (:457).
-  - `_open_and_supervise_one_cancels_all_nursery()` nests
-    `da_nursery` (:609) around `ria_nursery` (:622); the
-    `finally:` at the ria->da boundary (:747-766) raises
-    collected `errors` (single exc or BEG).
-- `runtime/_portal.py`
-  - `._expect_result_ctx` (:112) set by `_submit_for_result()`
-    (:142, sole caller `run_in_actor()`); consumed by
-    `wait_for_result()` (:167) + deprecated `result()` (:220).
-    The `None` branch (:184-196) returns the `NoResult`
-    sentinel (`_exceptions.py:1164`).
-- `spawn/_spawn.py`
-  - `exhaust_portal()` (:129): awaits
-    `portal.wait_for_result()`, CATCHES+RETURNS any exc
-    (never raises).
-  - `cancel_on_completion()` (:177): `exhaust_portal()` ->
-    on exc-result stash `errors[uid] = result` (:203) ->
-    ALWAYS `portal.cancel_actor()` (:218).
-- `spawn/_trio.py` (:195-222) + `spawn/_mp.py` (:187-213),
-  identical shape: after shielded
-  `await an._join_procs.wait()`, open a per-child local
-  nursery; IFF `portal in an._cancel_after_result_on_exit`
-  start `cancel_on_completion` alongside `soft_kill()`; when
-  `soft_kill` returns first, `nursery.cancel_scope.cancel()`
-  reaps the result-waiter.
-
-## The load-bearing semantic (already-deferred errors)
-
-Remote ria-child errors NEVER raise into `ria_nursery`:
-
-1. reaper tasks only START after `_join_procs.set()` (block
-   exit or the inner error handler),
-2. `exhaust_portal` swallows the exc into a return value,
-3. `cancel_on_completion` stashes it in `errors` + cancels
-   that child,
-4. the ria->da `finally:` re-raises collected `errors` (and
-   `an.cancel()`s any daemon stragglers).
-
-So mid-block there is NO error propagation from ria children
-(unless user code explicitly `await portal.wait_for_result()`s)
-— the two-nursery nesting only sequences "reap ria results
-BEFORE blocking on daemon join". A single-nursery impl only
-needs to preserve that sequencing, not any ASAP-cancel
-behavior.
-
-## Target design
-
-### step A: single-nursery `run_in_actor()` (mechanical)
-
-- `run_in_actor()` spawns via the DEFAULT (`_da_nursery`)
-  path — drop `nursery=self._ria_nursery`.
-- rename `._cancel_after_result_on_exit` ->
-  `._ria_portals: dict[portal, Actor]` (need the subactor ref
-  for `cancel_on_completion`).
-- move reaper start-up OUT of the backends into
-  `_open_and_supervise...`: immediately after EACH
-  `an._join_procs.set()` call-site (happy path :642, inner
-  error handler :661), start one
-  `cancel_on_completion(portal, subactor, errors)` task per
-  ria portal into `da_nursery`, then (happy path only)
-  `await` their completion BEFORE falling out of the
-  `try:`/`finally:` that raises `errors` — e.g. gather in a
-  dedicated inner `trio.open_nursery()` block replacing
-  today's `ria_nursery` join point.
-- delete the membership branch + local reaper nursery from
-  `_trio.py`/`_mp.py` (keep the `soft_kill()` call; the
-  per-child local nursery collapses to just `soft_kill`).
-- `_trio.py:310` `_children.pop()` etc. unchanged.
-
-### step B: delete the plumbing
-
-- `_open_and_supervise...`: drop the inner
-  `ria_nursery` + merge its `except BaseException` classify
-  logic into ONE handler on the (now single) nursery scope;
-  `ActorNursery.__init__` loses the `ria_nursery` param.
-- `start_actor()` loses the `nursery:` escape-hatch param
-  (the :302-304 TODO).
-- backends: no more `_cancel_after_result_on_exit` refs.
-
-### step C: (separate PRs) deprecate + migrate + excise
-
-- migrate in-repo `.run_in_actor()` usage to
-  `to_actor.run()`: tests 46 hits/9 files (test_cancellation
-  15, test_infected_asyncio 10, test_spawning 8, registrar 3,
-  adv_streaming 4, pubsub 2, rpc 1, runtime 1), examples 28
-  hits/13 files (debugging/* dominate), docs 20 hits/8 rst
-  files. NOTE: many sites also use deprecated
-  `Portal.result()`/`wait_for_result()` — these die with
-  `_expect_result_ctx`, so migration must land FIRST.
-- add `DeprecationWarning` to `run_in_actor()` (+
-  `_submit_for_result`/`wait_for_result`).
-- final excision: `run_in_actor()`, `_submit_for_result`,
-  `_expect_result_ctx`, `wait_for_result`/`result`,
-  `exhaust_portal`, `cancel_on_completion`, `NoResult`.
-
-## Risk register
-
-1. hard-killed ria child: today the backend-local
-   `nursery.cancel_scope.cancel()` discards a still-parked
-   reaper when the proc dies first; a da_nursery-hosted
-   reaper instead sees the transport break ->
-   `exhaust_portal` returns a `TransportClosed`-ish exc ->
-   NEW entry in `errors` that today gets discarded. Guard:
-   reap-gather block must cancel remaining reapers once all
-   ria procs are dead, or filter transport-death excs for
-   already-`cancel_called` children.
-2. error-path ordering: inner handler today sets
-   `_join_procs` THEN `an.cancel()`; reapers race the
-   cancel-RPC. Keep that ordering when moving reaper spawn.
-3. debugger interplay: `maybe_wait_for_debugger()` calls
-   (:654, :730) must stay BEFORE any reap/cancel issuance.
-4. `errors` double-entry: local body error (:646) + child's
-   relayed exc (via reaper) can both land for the same
-   scenario -> BEG shape changes vs today? (today has the
-   same dual-write sites; keep behavior identical.)
-5. mp backend parity: mirror every `_trio.py` edit in
-   `_mp.py` (identical block).
-
-## Step-A first-probe findings (2026-07-02, WIP in tree)
-
-Step A is IMPLEMENTED (uncommitted):
-`run_in_actor()` spawns via da_nursery; new
-`_supervise._reap_ria_portals()` helper; reap awaited after
-happy-path `_join_procs.set()`; error-path runs reap
-CONCURRENT with `an.cancel()` in the shielded block;
-backends stripped of the membership branch + per-child
-reaper nursery (+ dead imports).
-
-Probe history (trio backend):
-- `tests/test_to_actor.py` + `tests/test_spawning.py`:
-  20/20 PASS — incl. all `run_in_actor()` result
-  round-trips + `test_remote_error` (single erroring child,
-  body re-raise -> inner error path).
-- FIRST attempt ran the error-path reap CONCURRENT with
-  `an.cancel()` (mimicking the old backend-side race):
-  `test_cancellation.py::test_multierror` (2 erroring ria
-  children, body re-raises one) DEADLOCKED. Root cause per
-  the sequencing fix below: reap + cancel must NOT race at
-  this layer (suspected `._children` pop-during-iteration
-  and/or double-cancel RPC wedge; not fully root-caused
-  since the fix removes the race wholesale).
-- FIX (2nd attempt, current impl): error path SEQUENCES:
-  (1) snapshot ria `(portal, subactor)` pairs (backend
-  `finally`s pop `._children` as procs reap), (2)
-  `await an.cancel()`, (3) bounded reap over the snapshot.
-  Bound was first 3s -> blew the `fail_after` deadline in
-  `test_cancel_while_childs_child_in_sync_sleep` (hard-
-  killed grandchild never relays => reaper parks the full
-  bound). Tightened to 0.5s: anything collectable is
-  already queued in the local ctx (relayed BEFORE the
-  cancel); a parked reaper self-cleans (`trio.Cancelled`
-  results are never stashed).
-- RESULT: `tests/test_cancellation.py` FULLY GREEN
-  (20 passed, 1 xfailed, 77s); full-suite gate run kicked
-  off same session (see final report/next session).
-
-Remaining risk: on slow CI a relayed-but-undelivered error
-racing the 0.5s bound could drop an `errors` entry
-(BEG-shape flake); if observed, scale the bound via the
-`cpu_perf_headroom()`-style approach or peek
-`Portal._final_result_msg`/ctx queue state instead of
-time-bounding.
-
-## Verification gate
-
-- `tests/test_cancellation.py test_spawning.py test_local.py
-  test_rpc.py` on `trio` + `mp_spawn` + `mp_forkserver`
-  backends, then full suite; `tests/devx/test_debugger.py`
-  for risk 3.

ai/prompt-io/claude/20260702T165806Z_a34aaf98_prompt_io.md

@@ -1,78 +0,0 @@
----
-model: claude-fable-5
-service: claude
-session: f6c84722-471a-4458-9a80-e453fea9029f
-timestamp: 2026-07-02T16:58:06Z
-git_ref: a34aaf98
-scope: code
-substantive: true
-raw_file: 20260702T165806Z_a34aaf98_prompt_io.raw.md
----
-
-## Prompt
-
-Follow-up round in the same session as the
-`tractor.to_actor` landing (see
-`20260702T154255Z_65bf9df5_prompt_io.md`). After
-committing that work the user green-lit the deferred
-items:
-
-> go go go on this with what time you have left, in
-> particular see if you can get the _ria_nursery
-> removal going!
-
-then extended the deadline twice to iterate on the
-discovered hang:
-
-> continue on this up until a 12:58:30 deadline
-
-and finally chose "Commit step A now" from the
-next-steps prompt.
-
-## Response summary
-
-Step A of the `._ria_nursery` removal (issue #477): hoist
-`.run_in_actor()` result-reaping out of the spawn
-backends into the `ActorNursery` machinery so ria
-children spawn via the default daemon nursery,
-
-- new `_supervise._reap_ria_portals()` runs one
-  `_spawn.cancel_on_completion()` task per ria child
-  AFTER `._join_procs` is set; happy path awaits it
-  right after `._join_procs.set()`.
-- error path SEQUENCES: snapshot ria
-  `(portal, subactor)` pairs -> `await an.cancel()` ->
-  0.5s-bounded reap. Two failed intermediates informed
-  this: a concurrent reap+cancel DEADLOCKED
-  `test_multierror`; a 3s bound blew
-  `test_cancel_while_childs_child_in_sync_sleep`'s
-  `fail_after` deadline.
-- backends (`spawn/_trio.py`, `spawn/_mp.py`) lose the
-  `._cancel_after_result_on_exit` membership branch,
-  per-child reaper nursery + dead imports.
-- design/probe-history doc:
-  `ai/conc-anal/ria_nursery_removal_plan.md` (from an
-  agent-verified machinery map).
-
-Verification: `test_cancellation.py` fully green
-(20 passed, 1 xfailed) incl. the previously-hung
-`test_multierror`; `test_to_actor`+`test_spawning`
-20/20; bounded full-suite gate SIGINT'd ~30s early at
-303 passed / 0 failures (user opted to commit on that
-signal, deferring the unbounded re-run to step-B
-verification).
-
-## Files changed
-
-- `tractor/runtime/_supervise.py` — `_reap_ria_portals()`
-  + two call-sites; `run_in_actor()` off the ria nursery
-- `tractor/spawn/_trio.py` — reaper branch + import drop
-- `tractor/spawn/_mp.py` — same as `_trio.py`
-- `ai/conc-anal/ria_nursery_removal_plan.md` — plan +
-  probe history
-
-## Human edits
-
-None yet — committed via the drafted
-`.claude/git_commit_msg_ria_step_a.md` (user-driven
-`git commit --edit`).

ai/prompt-io/claude/20260702T165806Z_a34aaf98_prompt_io.raw.md

@@ -1,55 +0,0 @@
----
-model: claude-fable-5
-service: claude
-timestamp: 2026-07-02T16:58:06Z
-git_ref: a34aaf98
-diff_cmd: git diff a34aaf98..wkt/to_actor_subpkg
----
-
-# Raw AI output (diff-ref mode)
-
-Step-A code is committed on `wkt/to_actor_subpkg`
-directly after `a34aaf98`; per diff-ref mode the verbatim
-content is reachable via the pointers below.
-
-## Generated files
-
-> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/runtime/_supervise.py`
-
-New `_reap_ria_portals(an, errors, ria_children=None)`
-helper (one `_spawn.cancel_on_completion()` task per ria
-child under `collapse_eg()` + a local nursery);
-`run_in_actor()` drops `nursery=self._ria_nursery`; happy
-path awaits the reap right after `._join_procs.set()`;
-inner error handler snapshots ria pairs, runs
-`await an.cancel()` then a `move_on_after(0.5)`-bounded
-reap over the snapshot.
-
-> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/spawn/_trio.py`
-> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/spawn/_mp.py`
-
-Both backends: the post-`_join_procs` block collapses to
-a bare `soft_kill()` (membership branch, per-child reaper
-nursery, reaper-cancel logging and the now-unused
-`cancel_on_completion` imports all removed).
-
-> `git diff a34aaf98..wkt/to_actor_subpkg -- ai/conc-anal/ria_nursery_removal_plan.md`
-
-Agent-verified machinery map, 3-step design (A/B/C),
-probe history (deadlock -> sequencing fix -> bound
-tightening) and risk register.
-
-## Test runs (verbatim)
-
-```
-tests/test_cancellation.py: 20 passed, 1 xfailed in 77.28s
-tests/test_to_actor.py + tests/test_spawning.py: 20 passed
-full-suite (SIGINT at 540s bound, ~30s short):
-303 passed, 9 skipped, 1 xfailed, 2 xpassed in 539.42s
-```
-
-Failed intermediates (not in final diff): concurrent
-reap+`an.cancel()` deadlocked `test_multierror`; a 3s
-reap bound blew
-`test_cancel_while_childs_child_in_sync_sleep`'s
-`fail_after` deadline.

tractor/runtime/_supervise.py

@@ -438,6 +438,8 @@ class ActorNursery:
             ),
             bind_addrs=bind_addrs,
             loglevel=loglevel,
+            # use the run_in_actor nursery
+            nursery=self._ria_nursery,
             infect_asyncio=infect_asyncio,
             inherit_parent_main=inherit_parent_main,
             proc_kwargs=proc_kwargs
@@ -578,51 +580,6 @@ class ActorNursery:
         self._join_procs.set()
 
 
-async def _reap_ria_portals(
-    an: ActorNursery,
-    errors: dict[tuple[str, str], BaseException],
-    ria_children: list[tuple[Portal, Actor]]|None = None,
-) -> None:
-    '''
-    Wait on and stash the final result/error from every
-    `.run_in_actor()`-spawned child then cancel its actor
-    runtime, one `_spawn.cancel_on_completion()` task per
-    child.
-
-    Replaces the per-child reaper task formerly spawned by
-    the spawn backends (keyed off
-    `._cancel_after_result_on_exit` membership) which
-    required routing such children into the (now removable)
-    `._ria_nursery`. Only call AFTER `._join_procs` is set
-    so user code inside the nursery block retains exclusive
-    result-await access; see the "manually await results"
-    note in `spawn._mp.mp_proc()`.
-
-    '''
-    if ria_children is None:
-        ria_children: list[tuple[Portal, Actor]] = [
-            (portal, subactor)
-            for subactor, _, portal in an._children.values()
-            if portal in an._cancel_after_result_on_exit
-        ]
-    if not ria_children:
-        return
-
-    async with (
-        collapse_eg(),
-        trio.open_nursery() as tn,
-    ):
-        portal: Portal
-        subactor: Actor
-        for portal, subactor in ria_children:
-            tn.start_soon(
-                _spawn.cancel_on_completion,
-                portal,
-                subactor,
-                errors,
-            )
-
-
 @acm
 async def _open_and_supervise_one_cancels_all_nursery(
     actor: Actor,
@@ -684,11 +641,6 @@ async def _open_and_supervise_one_cancels_all_nursery(
                     )
                     an._join_procs.set()
 
-                    # collect results (and errors) from all
-                    # `.run_in_actor()` children then cancel
-                    # each, one reaper task per child.
-                    await _reap_ria_portals(an, errors)
-
                 except BaseException as _inner_err:
                     inner_err = _inner_err
                     errors[actor.aid.uid] = inner_err
@@ -752,39 +704,9 @@ async def _open_and_supervise_one_cancels_all_nursery(
                                 # '------ - ------'
                             )
 
-                        # snapshot `.run_in_actor()` children
-                        # BEFORE cancelling: each backend
-                        # spawn-task pops its `._children`
-                        # entry as the proc gets reaped.
-                        ria_children: list = [
-                            (portal, subactor)
-                            for subactor, _, portal
-                            in an._children.values()
-                            if portal in
-                            an._cancel_after_result_on_exit
-                        ]
                         # cancel all subactors
                         await an.cancel()
 
-                        # then collect any already-relayed
-                        # results/errors from ria children.
-                        # Tightly bounded: anything
-                        # collectable is already queued in
-                        # the local ctx (relayed BEFORE the
-                        # cancel above); a child hard-killed
-                        # without relaying just parks its
-                        # reaper which then self-cleans (a
-                        # `trio.Cancelled` result is never
-                        # stashed), mirroring the old
-                        # backend-side reaper-vs-`soft_kill`
-                        # cancel race.
-                        with trio.move_on_after(0.5):
-                            await _reap_ria_portals(
-                                an,
-                                errors,
-                                ria_children=ria_children,
-                            )
-
             # ria_nursery scope end
 
         # TODO: this is the handler around the ``.run_in_actor()``

tractor/spawn/_mp.py

@@ -48,6 +48,7 @@ from ._entry import _mp_main
 # by `try_set_start_method()` after module load time.
 from . import _spawn
 from ._spawn import (
+    cancel_on_completion,
     proc_waiter,
     soft_kill,
 )
@@ -186,16 +187,30 @@ async def mp_proc(
         with trio.CancelScope(shield=True):
             await actor_nursery._join_procs.wait()
 
-        # This is a "soft" (cancellable) join/reap which
+        async with trio.open_nursery() as nursery:
-        # will remote cancel the actor on a ``trio.Cancelled``
+            if portal in actor_nursery._cancel_after_result_on_exit:
-        # condition. Any `.run_in_actor()` result-reaping
+                nursery.start_soon(
-        # happens up in the `ActorNursery` machinery (see
+                    cancel_on_completion,
-        # `_supervise._reap_ria_portals()`), NOT here.
+                    portal,
-        await soft_kill(
+                    subactor,
-            proc,
+                    errors
-            proc_waiter,
+                )
-            portal
+
-        )
+            # This is a "soft" (cancellable) join/reap which
+            # will remote cancel the actor on a ``trio.Cancelled``
+            # condition.
+            await soft_kill(
+                proc,
+                proc_waiter,
+                portal
+            )
+
+            # cancel result waiter that may have been spawned in
+            # tandem if not done already
+            log.warning(
+                "Cancelling existing result waiter task for "
+                f"{subactor.aid.uid}")
+            nursery.cancel_scope.cancel()
 
     finally:
         # hard reap sequence

tractor/spawn/_trio.py

@@ -50,6 +50,7 @@ from tractor.msg import (
     pretty_struct,
 )
 from ._spawn import (
+    cancel_on_completion,
     hard_kill,
     soft_kill,
 )
@@ -194,16 +195,31 @@ async def trio_proc(
         with trio.CancelScope(shield=True):
             await actor_nursery._join_procs.wait()
 
-        # This is a "soft" (cancellable) join/reap which
+        async with trio.open_nursery() as nursery:
-        # will remote cancel the actor on a ``trio.Cancelled``
+            if portal in actor_nursery._cancel_after_result_on_exit:
-        # condition. Any `.run_in_actor()` result-reaping
+                nursery.start_soon(
-        # happens up in the `ActorNursery` machinery (see
+                    cancel_on_completion,
-        # `_supervise._reap_ria_portals()`), NOT here.
+                    portal,
-        await soft_kill(
+                    subactor,
-            proc,
+                    errors
-            trio.Process.wait,  # XXX, uses `pidfd_open()` below.
+                )
-            portal
+
-        )
+            # This is a "soft" (cancellable) join/reap which
+            # will remote cancel the actor on a ``trio.Cancelled``
+            # condition.
+            await soft_kill(
+                proc,
+                trio.Process.wait,  # XXX, uses `pidfd_open()` below.
+                portal
+            )
+
+            # cancel result waiter that may have been spawned in
+            # tandem if not done already
+            log.cancel(
+                'Cancelling portal result reaper task\n'
+                f'c)> {subactor.aid.reprol()!r}\n'
+            )
+            nursery.cancel_scope.cancel()
 
     finally:
         # XXX NOTE XXX: The "hard" reap since no actor zombies are