Compare commits

...

45 Commits

Author SHA1 Message Date
Gud Boi 3798aa9c9a Clean broadcast cancellation diagnostics
Bound `BroadcastState.cancelled` entries to receiver progress,
terminal state and resource lifetime instead of retaining completed
`Task`s indefinitely.

Deats,
- make EOC durable so awakened peers never re-enter a closed source.
- close root broadcasters during explicit `MsgStream` and
  `LinkedTaskChannel` teardown without re-entrant EOC closure or
  breaking `MsgStream.aclose()` overrides.
- reject non-positive fan-out retention capacity before constructing
  an unusable zero-length queue.
- cover child/root cancellation cleanup, terminal peer wakeups,
  wrapper teardown, subclass compatibility and zero-buffer rejection.

Prompt-IO: ai/prompt-io/opencode/20260813T181901Z_a2e0df4b_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 13:43:40 -04:00
Gud Boi b881512d65 Expose stream subscriber lag policy
`MsgStream.subscribe()` and `LinkedTaskChannel.subscribe()` omitted
`BroadcastReceiver.raise_on_lag`, forcing downstream consumers to
mutate a private receiver attribute when overruns were acceptable.

Add `raise_on_lag` to both public wrappers. The first subscription
sets the irreversible root broadcaster's policy, while every child
selects its own strict or warn/drop/resume behavior independently.

Document both fan-out APIs. Cover policy forwarding plus real IPC
and infected-asyncio paths.

Prompt-IO: ai/prompt-io/opencode/20260812T213117Z_51185487_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 13:43:40 -04:00
Gud Boi 933374de2c Isolate `BroadcastReceiver.aclose()` wakeups
Closing any subscriber set the shared `recv_ready` event, even when
another receiver owned the source read. Waiting peers then looped
until an idle source produced another value.

Give each receiver private source-read and peer-wait cancellation
scopes. Closing a waiting peer interrupts only that peer. Closing
the source owner discards post-close source outcomes and then wakes
peers for a clean ownership handoff.

Keep outer task cancellation as `trio.Cancelled`; only explicit
receiver close maps either private scope's cancellation to
`ClosedResourceError`. Assert that scope cancellation implies the
receiver is closed and document the source-owner key check.

Prompt-IO: ai/prompt-io/opencode/20260812T150027Z_c2a6ccef_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 13:43:40 -04:00
Gud Boi 08dfd834a6 Wake broadcast peers on shared receive failures
Only `EndOfChannel` and direct cancellation woke tasks waiting
behind the subscriber which owned the underlying receive. Any other
failure cleared `BroadcastState.recv_ready` while peers remained
blocked on its unreachable event.

Publish ordinary receive exceptions as terminal broadcast state.
The owner keeps the original failure while peers drain retained
values and then raise `BroadcastReceiveError` from that cause. Also
wake peers on process-control exits without retaining them as state.

Document the public owner/peer contract and cover current, late and
control-flow subscribers with deterministic bounded regressions.

Prompt-IO: ai/prompt-io/opencode/20260812T030608Z_1095e7f7_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 13:43:40 -04:00
Gud Boi 8ab6281a68 Fix `BroadcastState.statistics()` queue counts
`BroadcastState.subs` stores each receiver's next unread deque
index, but `.statistics()` exposed that index as a queue length. A
caught-up receiver looked correct by accident while every queued
count was one short.

Convert cursors to retained, receivable counts and clamp lagged
receivers to the current queue length. Also avoid deprecated
`trio.Event` truthiness when reporting waiter counts.

Cover caught-up, queued, lagged and real-event states using actual
broadcast sends and receives.

Prompt-IO: ai/prompt-io/opencode/20260812T012324Z_06c4af17_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 13:43:40 -04:00
Gud Boi 8db13375fc Fix `BroadcastReceiver` lag counts
`BroadcastReceiver.receive_nowait()` treated `seq` as a deque index
but subtracted `BroadcastState.maxlen` without counting the first
invalid index. A one-slot queue thus claimed it dropped zero values
after its subscriber missed one.

Include that first displaced value in the count. Preserve the
existing Tokio-style reset to the oldest retained item.

Also, cover exact loss reporting and recovery for one- and
three-slot retention windows.

Prompt-IO: ai/prompt-io/opencode/20260811T233833Z_7cbd64ee_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 13:43:40 -04:00
Bd e90f2f224a
Merge pull request #484 from goodboy/drop_ria_nursery
Drop `run_in_actor()` + the ria reap-cluster
2026-08-26 22:48:32 -04:00
Gud Boi 1ad6281348 Clarify recursive one-shot spawning test
Handle the child RPC leaf as an early return, then leave the root-only
runtime and recursive one-shot flow unindented. Rename the helper for
that behavior and document why RPC namespace lookup requires it to
remain import-addressable at module scope.

Review: PR #484 (goodboy)
https://github.com/goodboy/tractor/pull/484#discussion_r3859956375

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-26 18:27:11 -04:00
Gud Boi b3e8ed18d1 Link asyncio cancellation regression provenance
Point `test_tractor_cancels_aio()`'s anti-hang guard at the original
fix commit and the detailed ria-removal analysis plan.

Review: PR #484 (goodboy)
https://github.com/goodboy/tractor/pull/484#discussion_r3859956368

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-26 17:00:54 -04:00
Gud Boi d5e66dffaa Restore high-fanout startup cancellation stress
Release 25 concurrent `to_actor.run()` callers through one local
barrier so their implicit child starts race the first remote error.
Exercise one, five and 25 deterministically placed errorers without
overloading actor-nursery internals.

Validate bounded cancel-on-first teardown, only boxed assertion
relays and empty actor-nursery child/reap maps across Trio and
multiprocessing backends. Clarify this successor's distinction from
`test_nested_multierrors()`, diagram flat-pool error propagation and
align the nearby expected-error comment with its handler.

Review: PR #484 (goodboy)
https://github.com/goodboy/tractor/pull/484#discussion_r3858891778

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-26 17:00:44 -04:00
Gud Boi cf56f33be6 Correct cancellation and asyncio test prose
Fix five spelling errors in comments and failure text touched by the
one-shot migration: one `propagate`, one `Daemon` and three `directly`
corrections.

Review: PR #484 (GitHub Copilot)
https://github.com/goodboy/tractor/pull/484#pullrequestreview-5025348921

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 23:35:43 -04:00
Gud Boi 8739c5fadb Document legacy one-shot API removals
Add the PR #484 towncrier fragment for removing
`ActorNursery.run_in_actor()`, `Portal.wait_for_result()` and
`Portal.result()`.

Point callers to `to_actor.run()`, `Portal.run()` or
`Portal.open_context()` according to task ownership and dialog shape.

Review: PR #484 (OpenCode)
https://github.com/goodboy/tractor/pull/484#pullrequestreview-5025383596

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 23:03:20 -04:00
Gud Boi dfdaf2b1c1 Validate `test_multierror()` relay shapes
Inspect the exception emitted by the concurrent error fan-out instead
of accepting any `RemoteActorError` or `BaseExceptionGroup`.

Require each non-cancellation leaf to box `AssertionError`, allow one
or two relays for cancel-on-first timing and require both child relays
when no cancellation leaf accompanies the group.

Review: PR #484 (OpenCode)
https://github.com/goodboy/tractor/pull/484#pullrequestreview-5025383596

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 23:00:13 -04:00
Gud Boi 37eeb7bab6 Exercise active RPC tasks in SIGINT test
Open one linked sleeping context in each daemon and wait for every
`StartAck` before `test_cancel_via_SIGINT_other_task()` reports
startup.

This restores the legacy test's active remote-task cancellation
target instead of proving SIGINT teardown only against idle actor
runtimes.

Review: PR #484 (OpenCode)
https://github.com/goodboy/tractor/pull/484#pullrequestreview-5025383596

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 22:53:18 -04:00
Gud Boi 1d59f1963c Propagate unexpected `@pub` subscriber errors
Give each background subscriber runner its own teardown event and
suppress only `ContextCancelled` relayed by the root actor after that
portal's explicit cancellation begins.

Let generic remote errors, foreign cancellation and cancellation
before teardown escape the local task nursery so the test cannot pass
after a subscriber fails unexpectedly.

Review: PR #484 (GitHub Copilot and OpenCode)
https://github.com/goodboy/tractor/pull/484#discussion_r3858426546

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 20:38:57 -04:00
Gud Boi dd3e7482bf Wait for active pub/sub cancellation targets
Replace `test_dynamic_pub_sub()`'s fixed startup sleep with an RPC
activity probe in the publisher actor. Track the publisher task and
wait until every launched consumer has installed its first
subscription before raising the user cancellation exception.

This keeps slow spawn backends from passing the regression by
cancelling actors which never reached the streaming workload.

Review: PR #484 (OpenCode)
https://github.com/goodboy/tractor/pull/484#pullrequestreview-5025383596

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 20:23:53 -04:00
Gud Boi 266073cb69 Reap the `@pub` example daemon on stream exit
Wrap the documented publisher stream in `try/finally` and explicitly
cancel its `start_actor()` daemon. Closing `open_stream_from()` owns
only the remote stream task, so actor-nursery exit otherwise waits on
the still-running actor indefinitely.

Review: PR #484 (OpenCode)
https://github.com/goodboy/tractor/pull/484#pullrequestreview-5025383596

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 20:03:18 -04:00
Gud Boi 4134726ec4 Pass `pub_service` directly to stream RPC
Keep the `@pub` docstring example's remote target namespace
addressable by passing the module-level function directly to
`Portal.open_stream_from()`.

Forward the topic and task-name inputs as RPC kwargs instead of
wrapping the target in a `functools.partial` object that resolves to
the wrong namespace path.

Review: PR #484 (OpenCode)
https://github.com/goodboy/tractor/pull/484#pullrequestreview-5025383596

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 19:59:17 -04:00
Gud Boi 09e78ad087 Bind named `to_actor.run()` inputs with partials
PR #481 made target inputs positional and reserved keywords for
actor placement/runtime controls. PR #484 still forwarded target
kwargs, so tests and examples failed local signature binding after
the rebase.

Deats,
- bind named target inputs with `functools.partial()`
- keep placement, naming and runtime controls as direct keywords
- reject invalid target calls locally before actor startup
- require linked one-shots to raise one direct `RemoteActorError`
- doc linked context execution and per-child process reaping

Prompt-IO: ai/prompt-io/opencode/20260819T184640Z_481ba003_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-25 19:20:43 -04:00
Gud Boi dd91195377 Doc the #477 migration outcome + one-shot-acm sketch
Fold the endeavour's resolution into the plan doc + log the
session per prompt-io policy,

- `ria_nursery_removal_plan.md`: RESOLVED section — migrate
  everything, remove the API; the migration-pattern table
  (blocking / fire-and-forget / fan-out / collect-don't-cancel
  / mutual-rendezvous), the semantic deltas (cancel-on-first +
  `collapse_eg()` chain collapse vs the old teardown-reap BEG),
  the excision inventory and the structural dissolution of the
  reap-hang class.
- adds the `to_actor.open_one_shot()` follow-up sketch: an
  `@acm` + private task-nursery over the existing blocking
  `run()` — done-`trio.Event` as a result memo (NOT a
  cancel-relay), no `Portal` in the iface, errors always
  propagate at scope exit; zero `_supervise` coupling.
- prompt-io entry `20260706T172818Z_ad42871e` (+ raw diff-ref
  companion) covering commits `d01a2123..ad42871e`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 19:19:28 -04:00
Gud Boi e7f5968850 Name every `ActorNursery` binding `an` in tests/examples
Convention sweep (user req): all `tractor.open_nursery()`
bindings in test + example code use `an: ActorNursery` (`n`,
`nursery` + several tractor-nurseries confusingly named `tn`
are renamed); `trio.open_nursery()` bindings stay `tn` (incl.
`concurrent_actors_primes.py`'s inner trio nursery, renamed
`n` -> `tn` to match).

Purely mechanical, function-scoped renames — prose "nursery"/
"an" in docstrings/comments untouched; func-arg kwargs like
`portal.run(func, n=value)` untouched.

Gate: renamed test modules green on `trio`; full debugger suite
(28p/6s) + example-runner (21p) green.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 19:19:28 -04:00
Gud Boi 551090d129 Fix mutual-rendezvous premature-reap race (#477)
The `test_trynamic_trio` + `a_trynamic_first_scene.py` migration
to paired `to_actor.run()` one-shots carries a race the legacy
`run_in_actor()` shape never had: donny + gretchen each
`wait_for_actor()` (then DIAL) the *other*, but a one-shot is
reaped the instant its own hello returns — so the slower peer
can resolve the winner's registry entry and connect to an
already-dead sockaddr -> `ConnectionRefusedError` boxed as a
`RemoteActorError` (or a reg-wait `TooSlowError`), flaking
~1-in-3 standalone runs.

Mutual-rendezvous peers must OUTLIVE both dialogs, so pin the
lifetimes explicitly: `start_actor()` both as daemons, run both
hellos concurrently via bg `Portal.run()` tasks, then reap with
`an.cancel()` only after the task-nursery joins. (The legacy
teardown-reap provided this pinning implicitly — one of the
few places its semantics were ever actually relied upon.)

Gate: `-k trynamic` standalone x8 green (was flaking); full
`test_registrar` module + the example-runner green.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 19:19:23 -04:00
Gud Boi bb0a9b3c93 Remove `run_in_actor()` + the ria reap cluster
The final excision of #477: with zero in-repo callers left (all
tests/examples/docs migrated to `to_actor.run()` et al) the
entire legacy one-shot machinery drops out,

- `runtime/_supervise.py`: `ActorNursery.run_in_actor()`, the
  `._cancel_after_result_on_exit` portal-set and the
  `_reap_ria_portals()` teardown-reaper (both its happy-path
  block-exit call AND the error-path snapshot + 0.5s-bounded
  collection) are deleted — one-shot result-waiting now lives
  entirely in the caller's task via `to_actor.run()`, whose
  enclosing cancel-scope bounds the wait by construction (the
  correct-scoping fix for the unbounded-reap hang class; the
  `d1fb4a1a` guard test now passes structurally).
- `runtime/_portal.py`: `Portal._submit_for_result()`,
  `._expect_result_ctx`, `._final_result_msg/_pld`,
  `.wait_for_result()` + the deprecated `.result()` alias are
  gone — a `Portal` no longer has any "main result" notion.
  NB `Context.wait_for_result()` is a different (very alive)
  API and is untouched.
- `spawn/_spawn.py`: `exhaust_portal()` +
  `cancel_on_completion()` (the reaper tasks) deleted; backend
  comment sweeps in `_trio.py`/`_mp.py`.
- `_exceptions.py`: the `NoResult` sentinel dies with its lone
  reader.
- `tests/test_ringbuf.py`: drop a daemon-portal `.result()`
  call that was already a warn + `NoResult` no-op (the ctx-acm
  exit does the real result-wait); unshadow the 2nd `sctx` as
  `rctx`.
- comment/docstring x-ref sweeps: `msg/types.py`,
  `_context.py`, `to_actor/`, `tests/test_to_actor.py`.

Gate: `test_to_actor test_spawning test_cancellation
test_infected_asyncio test_local test_rpc` = 81 passed,
3 xfailed on `trio`; +`test_ringbuf` = 70 passed, 3 skipped,
3 xfailed on `mp_spawn`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 19:16:32 -04:00
Gud Boi 5d70959a2b Fix stale `@pub` docstring example in `experimental`
The `_pubsub.pub` decorator's usage example predates several API
generations: ancient positional-arg-order `run_in_actor()` (a
missing `await` too) plus the deprecated `portal.result()` — and
`run_in_actor()` never allowed streaming funcs anyway. Show the
canonical `start_actor()` + `Portal.open_stream_from()`
consumption instead (#477 removal sweep).

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 19:16:32 -04:00
Gud Boi e8f636ddbb Port docs off `run_in_actor` + `Portal.wait_for_result`
The 8-page docs sweep of the #477 removal, ahead of the API's
excision,

- `start/quickstart.rst`: the first-actor-tree walkthrough now
  narrates the (migrated) `to_actor.run()` example — no portal
  in hand until the daemon section introduces `start_actor()`.
- `guide/spawning.rst`: the one-shot section becomes
  `to_actor.run()` (blocking call, placement opts, "built on the
  primitives" note); lifetime/teardown rules update — one-shots
  never make it to nursery exit since each is reaped inside its
  own call.
- `guide/rpc.rst`: the `wait_for_result()` section (an API that
  dies with the reap cluster, incl. the `NoResult` sentinel)
  becomes a `to_actor.run()` one-shot section.
- `api/core.rst`: drop `run_in_actor`/`wait_for_result` from the
  autodoc member lists, drop the `Portal.result()` deprecation
  note, add a "One-shot task actors" `tractor.to_actor.run`
  autodoc section.
- `guide/{asyncio,context,cancellation,parallelism}.rst`:
  mention swaps to the successor API.

Gate: `make -C docs html` builds clean; `to_actor.run` autodoc
renders in `api/core.html`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 19:16:21 -04:00
Gud Boi 4eca8d7a10 Port `test_dynamic_pub_sub` off `run_in_actor`
The known-flaky dynamic pubsub test's 3 fire-and-forget spawn
sites (#477 removal),

- the forever-streaming `publisher` + N `consumer` one-shots now
  bg-schedule as `to_actor.run(fn, an=n)` tasks in a local `trio`
  task-nursery (`publisher`'s rendezvous name still derives from
  `fn.__name__`).
- the simulated user-cancel raise (`KeyboardInterrupt` /
  `TooSlowError` params) cancels the task-nursery, each one-shot
  reaping its subactor via `to_actor.run()`'s shielded
  `Portal.cancel_actor()`; `_run_and_match()`'s existing
  `BaseExceptionGroup.split()` walk covers the (possibly nested)
  relay shapes unchanged.
- spawns now issue concurrently rather than sequentially —
  comment on the fork-backend budget updated to match.

Gate: both params x4 runs green on `trio` + x1 on `mp_spawn`;
full module green.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 3fcc4ee713 Port SIGINT + sync-sleep cancel tests off `run_in_actor`
Final `test_cancellation.py` group of the `run_in_actor` removal
(#477) — cancel-mechanics tests, so clean conversions,

- `test_cancel_via_SIGINT_other_task`: the 3 keep-alive
  `run_in_actor(sleep_forever)` one-shots become plain
  `start_actor()` daemons (an idle daemon needs no "main" task,
  and no longer shares a single dup'd `namesucka` name).
- `spawn_sub_with_sync_blocking_task`: the middle layer's spawn
  becomes a blocking `to_actor.run(spin_for, an=an)` which parks
  awaiting the sync-sleeping grandchild's result until cancelled
  from above.
- `test_cancel_while_childs_child_in_sync_sleep`: the
  fire-and-forget middle-actor spawn becomes a bg
  `to_actor.run()` task in a local task-nursery; the root's
  `assert 0` cancels it, driving the same
  graceful-cancel-then-zombie-reap cascade on the sync-blocked
  grandchild. The `man_cancel_outer` xfail param is unchanged.

Zero live `run_in_actor()` call-sites remain in this suite.

Gate: full `test_cancellation.py` module green on both `trio`
(18p/1xf) + `mp_spawn` (18p/1xf).

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi de3dc2ded0 Port `test_nested_multierrors` off `run_in_actor`
Third `test_cancellation.py` group of the `run_in_actor` removal
(#477),

- `spawn_and_error` fans out each level's erroring one-shots as
  concurrent `to_actor.run(fn, an=an)` tasks in a local `trio`
  task-nursery (recursing per spawner subactor), as does the
  test-body's top-level spawner loop.
- the deterministic exact-breadth nested-BEG shape dies with the
  legacy teardown-reap: each level now groups whatever subset of
  sub-tree errors relay before the first one's cancel wins, and
  a single-member group gets unwrapped by the runtime's own
  `collapse_eg()` at every actor boundary — so a fully-raced
  tree relays a bare `RemoteActorError` chain.
- loosen the shape walk accordingly: accept a lone
  `RemoteActorError` or a 1..breadth group whose members box
  `ExceptionGroup` (multi-relay), `AssertionError` (collapsed
  leaf chain), `RemoteActorError` (re-boxed collapsed chain) or
  `BaseExceptionGroup` (runtime reap-deadline `Cancelled`
  upgrade); fold the windows-only tolerances into the same walk.
- raced sibling `trio.Cancelled`s are now ABSORBED by the
  task-nursery instead of landing in the group, so the MTF
  shape-mismatch xfail should consistently xpass — note added to
  drop the marker once CI confirms.
- add an `else: pytest.fail()` so a silently-clean tree can no
  longer pass.

Gate: both depths green on `trio` (10 consecutive runs) +
`mp_spawn`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi f81f7d40c3 Fix unbound `timeout` under non-trio/MTF backends
`test_nested_multierrors`'s backend/depth budget `match` only
carries arms for the `trio` + `main_thread_forkserver` spawn
backends, so running under any other (e.g. `mp_spawn`) leaves
`timeout` unbound and crashes with an `UnboundLocalError` at the
headroom-scaling below. Add default per-depth arms riding the MTF
budgets (same per-spawn round-trip cost class).

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi f4f3034555 Port `test_some_cancels_all` off `run_in_actor`
Second `test_cancellation.py` group of the `run_in_actor` removal
(#477),

- one-shot subactors now run as concurrent `to_actor.run(fn,
  an=an)` tasks in a local `trio` task-nursery, so their errors
  raise WHILE the actor-nursery block is open (vs the legacy
  teardown-reap) and the first error cancels sibling one-shots.
- wrap the task-nursery in `collapse_eg()` so the deterministic
  single-error cases still surface a bare `RemoteActorError`.
- loosen the group-shape assertion: the relay-vs-cancel race
  populates anywhere from 1 to `num_actors` `RemoteActorError`s
  (the exact-`num_actors` BEG was `run_in_actor`'s
  reap-all-at-teardown); group members are always
  `RemoteActorError` now since sibling `trio.Cancelled`s are
  absorbed by the task-nursery.
- move the daemon-portal call loop inside the task-nursery body
  so the sleep-forever one-shot case is cancelled by the daemon
  error raise.
- rename the `*run_in_actor*` param ids to `*one_shot*`.

Gate: 6 passed on both `trio` + `mp_spawn` backends.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 5420b13482 Doc ria-reap hang fix + paused reaper re-scope
Append two sections to the ria-removal plan capturing the
2026-07-02 hang episode + the resulting design pivot.

Regression writeup: the full-suite hang on
`test_tractor_cancels_aio` root-caused to the step-A reaper
hoist (`5cd190c5`), not the B2 handler merge. The happy-path
`_reap_ria_portals()` parks unbounded on `wait_for_result()`
after a user `portal.cancel_actor()`; the old spawn-backend
reaper raced `soft_kill()`'s scope-cancel, the hoist dropped
it. Records the `proc.poll()` death-watch fix + why poll (not
the event `wait_func`) bc `soft_kill` already awaits
`proc.sentinel` (a 2nd `wait_readable` -> `BusyResourceError`).

Pause writeup: user's insight that the hoist landed in the
wrong scope — result-waiting belongs in the `to_actor`
one-shot scope (`_invoke_in_subactor()`), beside `an` + a
local task-nursery + cancel-scope, where bounding the wait is
trivial + the hang dissolves. So the poll fix is likely
SUPERSEDED (flagged do-not-land); the anti-hang guard commit
(`d1fb4a1a`) stays red-first per the failing-test convention.

(this patch was generated in some part by `claude-code` using
`claude-opus-4-8` (`anthropic`))
2026-08-25 17:57:12 -04:00
Gud Boi e824e6b768 Port `test_cancellation` multierror cluster off `run_in_actor`
First group of the `test_cancellation.py` `run_in_actor` removal
(#477),

- `test_remote_error` -> blocking `to_actor.run()` (single erroring
  one-shot; a bad-arg `TypeError` still relays as a
  `RemoteActorError`).
- `test_multierror` -> concurrent fan-out via
  `gather_contexts([p.open_context(assert_err_ctx) ...])` over
  `start_actor()` portals. NB `gather_contexts` is cancel-on-first
  so the 2nd errorer is usually cancelled before relaying its own
  exc and the pair collapses to a single `RemoteActorError` (vs the
  legacy reap-all-at-teardown `BEG`-of-N) — the assertion now
  accepts either shape.
- delete `test_multierror_fast_nursery` — a 25-actor stress test of
  `run_in_actor`'s teardown-reap; no analogous surface under the
  `to_actor` fan-out.
- add an `assert_err_ctx` `@context` shim for the `open_context`
  fan-out.

Remaining `test_cancellation` groups (some_cancels_all, nested,
SIGINT, sync-blocking) still on `run_in_actor` — ported next.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 0835963fa2 Port `test_registrar` off `run_in_actor`
Two sites migrated (#477 removal),

- `test_trynamic_trio`: donny + gretchen each wait on the *other*
  to register, so they must run CONCURRENTLY — was two
  non-blocking `run_in_actor()`s awaited after; now two
  `to_actor.run()` one-shots scheduled into a local `trio`
  task-nursery.
- the unregister-on-cancel cluster test: its non-streaming branch
  spawned `run_in_actor(trio.sleep_forever)` purely to keep each
  subactor alive + registered — a `start_actor()` daemon does that
  without a "main" task, so the spawn loop collapses to the same
  `start_actor()` the streaming branch already used.

Suite: 16 passed.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 4e4191701c Port `test_pubsub` off `run_in_actor`
`test_multi_actor_subs_arbiter_pub` used `run_in_actor()` to spawn
two forever-ish subscriber actors and hold their portals for a
later `cancel_actor()` (its `.result()` was commented out exactly
because `subs()` never cleanly returns). That deferred-spawn +
cancel shape isn't a blocking `to_actor.run()`, so convert to the
successor primitives (#477 removal),

- `start_actor()` per subscriber — keeps the portal for the
  existing `cancel_actor()` teardown,
- run `subs()` on each via a background `Portal.run()` task in a
  local `trio` nursery so both subscribe concurrently with the
  test's `wait_for_actor` / topic checks,
- each bg runner swallows the `RemoteActorError`/`ContextCancelled`
  that `cancel_actor()` relays; a trailing `tn.cancel_scope.cancel()`
  drops any lingering runner.

Suite: 8 passed.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 19d0e9cb78 Port `test_spawning` off `run_in_actor`
Migrate all 4 sites to blocking `tractor.to_actor.run()` (#477
removal),

- rename the two API-named tests to `test_to_actor_run_*`
  (`same_func_in_child`, `can_skip_parent_main_inheritance`) —
  they exercise the same spawn / `inherit_parent_main` path via
  the successor API.
- the recursive `spawn()` helper drops its white-box
  `an._children` / portal-`_peers` asserts (which probed
  `run_in_actor`'s portal + nursery-tracking internals);
  `to_actor.run()` returns the result and reaps internally, so
  keep the user-facing `result == 10` check.
- `test_most_beautiful_word` drops the 2nd `wait_for_result()`
  (the legacy result-cache re-fetch) — `to_actor.run()` delivers
  the value once, no cache.

Suite: 9 passed.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 8df118dde8 Port `test_rpc` off `run_in_actor`
Sole call-site: `run_in_actor(sleep_back_actor, ...)` ->
blocking `tractor.to_actor.run(..., an=n, ...)` (#477 removal).
The RPC-callback subactor is awaited in-caller instead of
reaped at nursery teardown; `name=`/`enable_modules=` map to
`to_actor.run()`'s same-named params, the rest to `**fn_kwargs`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi e328b4d729 Port `test_runtime` off `run_in_actor`
Sole call-site: an inlined `run_in_actor(...).result()` ->
blocking `tractor.to_actor.run(fn, an=an, ...)` (#477 removal).
Behaviour identical — the one-shot's result/error is awaited
in the caller's task rather than reaped at nursery teardown;
the enclosing `move_on_after` still cancels the sub in the
`error_in_child=False` case.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 3a6bff0b6e Port `test_infected_asyncio` off `run_in_actor`
First test-file of the #477 `.run_in_actor()` removal (blocking
`to_actor.run()` is the successor; the legacy non-blocking one-shot
is dropped, not replaced). All 9 call-sites migrated,

- blocking result/error/streaming-result tests -> `to_actor.run(fn,
  an=an, ...)`; the "streaming" ones stream aio<->trio INSIDE the
  subactor so the caller only awaits the final result.
- forever-task + cancel tests (`test_tractor_cancels_aio`,
  `test_trio_cancels_aio`) -> `start_actor()` +
  `Portal.open_context()` + cancel — can't block on a
  never-returning task. Adds a small `sleep_forever_aio_ctx`
  `@context` shim.
- greens the red `test_tractor_cancels_aio` anti-hang guard from
  the prior commit: under the correctly-scoped API the wait is
  bounded by the caller's cancel scope, so the hang is structurally
  gone — not patched.

Suite: 34 passed, 2 xfailed (trio backend).

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 61ad5bd158 Add anti-hang `fail_after` cap to aio-cancel test
Wrap `test_tractor_cancels_aio`'s `main()` in a
`trio.fail_after(9 * cpu_perf_headroom())` so a wedged remote
runtime can't hang the test forever. This is the blessed
anti-hang guard here bc `pytest-timeout`'s global cap is
intentionally off (it breaks `trio` under the fork backends,
per the `pyproject` NOTE).

The cap is generous + CPU-headroom-scaled bc it's an anti-hang
guard, not a perf assertion. Motivated by the
`._ria_nursery`-removal regression where a wedged ria-reaper
once hung this exact test indefinitely.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 34e638863e Merge the supervise error handlers into one
Step B2 of the `._ria_nursery` removal (issue #477; see
`ai/conc-anal/ria_nursery_removal_plan.md`). With the 2ndary
nursery gone (step B), the two nested error handlers in
`_open_and_supervise_one_cancels_all_nursery` collapse to one,

- the outer `except (Exception, BaseExceptionGroup,
  trio.Cancelled)` existed to catch errors bubbling from the
  old `._ria_nursery.__aexit__` reaper-group; that nursery no
  longer exists.
- trace shows the outer handler's `raise` was already DEAD: the
  inner handler records `errors[uid]` as its first action, so
  `errors` is always non-empty by the time anything could reach
  the outer handler, and the `finally`'s raise-from-`errors`
  always superseded the outer `raise`.
- so fold both into a single `except BaseException as
  _scope_err` guarding the lone daemon nursery; the `finally`
  (unchanged) still raises the collected `errors` as a single
  exc or `BaseExceptionGroup`.
- drop the now-unused `outer_err`/`inner_err` locals.

Behaviour-preserving (net ~30 lines lighter); the big diff is
the one-level de-indent of the handler body. The two remaining
`maybe_wait_for_debugger()` guards collapse to the single
pre-teardown wait.

Prompt-IO: ai/prompt-io/claude/20260702T222544Z_9201a2ed_prompt_io.md

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi d6da42984f Doc step-B2 handler-merge + prompt-io
Split from the step-B2 code commit to keep the runtime diff
free of `ai/` meta noise,

- `ai/conc-anal/ria_nursery_removal_plan.md`: add a "Step-B2
  outcome" section — the dead-outer-`raise` trace, why the
  merge is behavior-preserving, and the gate results.
- `ai/prompt-io/claude/20260702T222544Z_9201a2ed_*`: NLNet
  provenance (log + unedited raw) for the step-B2 work.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 916655996c Drop the vestigial `._ria_nursery`
Step B of the `._ria_nursery` removal (issue #477; see
`ai/conc-anal/ria_nursery_removal_plan.md`). With step A having
rerouted `.run_in_actor()` children onto the daemon nursery,
the 2ndary "run-in-actor" nursery spawns nothing and its stored
ref is never read — pure dead weight,

- collapse the inner `async with trio.open_nursery() as
  ria_nursery` layer in
  `_open_and_supervise_one_cancels_all_nursery`; `da_nursery` is
  now the single nursery for ALL subactors.
- `ActorNursery.__init__` loses the `ria_nursery` param + the
  `self._ria_nursery` attr; `start_actor()` loses its `nursery=`
  escape-hatch (spawns via `self._da_nursery` directly).
- `._cancel_after_result_on_exit` stays — still the ria-child
  discriminator for `_reap_ria_portals()`.

Behavior-preserving: a zero-task `trio.open_nursery()` only adds
a checkpoint. The two error handlers are KEPT (now nested under
the single nursery); merging them changes error/cancel
propagation and is deferred to its own PR (TODO left at the
outer `except`).

Prompt-IO: ai/prompt-io/claude/20260702T172233Z_5cd190c5_prompt_io.md

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 601d92eb4a Doc step-B outcome + prompt-io
Split from the step-B code commit to keep the runtime diff
free of `ai/` meta noise,

- `ai/conc-anal/ria_nursery_removal_plan.md`: add a "Step-B
  outcome" section — the empty-nursery collapse, why it's
  behavior-preserving, the deliberate handler-merge deferral,
  and the targeted-gate result.
- `ai/prompt-io/claude/20260702T172233Z_5cd190c5_*`: NLNet
  provenance (log + unedited raw) for the step-B work.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi b7298e6507 Hoist ria-reaping out of the spawn backends
Step A of the `._ria_nursery` removal (issue #477 follow-up, see
`ai/conc-anal/ria_nursery_removal_plan.md`): `.run_in_actor()`
children now spawn via the default daemon nursery and their
result-reaping moves up into the `ActorNursery` machinery,

- new `_supervise._reap_ria_portals()`: one
  `_spawn.cancel_on_completion()` task per ria child, run AFTER
  `._join_procs` is set — replacing the per-child reaper task the
  backends formerly spawned (keyed off
  `._cancel_after_result_on_exit` membership) which required
  routing such children into `._ria_nursery`.
- happy path: reap awaited right after `._join_procs.set()`,
  preserving "collect ria results before daemon join" sequencing.
- error path: snapshot ria `(portal, subactor)` pairs (backend
  `finally`s pop `._children` as procs reap), `await an.cancel()`,
  THEN a 0.5s-bounded reap over the snapshot; anything collectable
  is already queued in the local ctx and a parked reaper
  self-cleans (`trio.Cancelled` results are never stashed). NB: a
  concurrent reap+cancel variant deadlocked `test_multierror` and
  a 3s bound blew `test_cancel_while_childs_child_in_sync_sleep`'s
  deadline — deats in the plan doc's probe history.
- `spawn/_trio.py` + `spawn/_mp.py`: drop the membership branch,
  per-child reaper nursery + now-unused `cancel_on_completion`
  imports; the join phase is a bare `soft_kill()`.

`._ria_nursery` is now vestigial (zero spawn users): step B
deletes it + `start_actor()`'s `nursery=` escape hatch and merges
the supervisor's two error handlers.

Prompt-IO: ai/prompt-io/claude/20260702T165806Z_a34aaf98_prompt_io.md

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
Gud Boi 5f49544cf3 Add `_ria_nursery` removal plan + step-A prompt-io
Split from the step-A code commit to keep the runtime diff
free of `ai/` meta noise,

- `ai/conc-anal/ria_nursery_removal_plan.md`: agent-verified
  machinery map + 3-step (A/B/C) design + probe history
  (reap-relocation deadlock -> sequencing fix -> bound
  tighten) + risk register for the `._ria_nursery` excision.
- `ai/prompt-io/claude/20260702T165806Z_a34aaf98_*`: NLNet
  provenance (log + unedited raw) for the step-A work.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-25 17:57:12 -04:00
65 changed files with 3700 additions and 1046 deletions

View File

@ -0,0 +1,463 @@
# `_ria_nursery` removal plan (issue #477 follow-up)
Goal: drop the secondary "run-in-actor" spawn nursery (and
friends) from `ActorNursery`/spawn internals, now that
`tractor.to_actor.run()` delivers one-shot semantics purely on
the daemon-spawn + portal primitives.
## Verified machinery map (2026-07-02, wkt @ a34aaf98)
The entire mechanism is 4 files:
- `runtime/_supervise.py`
- `ActorNursery.__init__(.., ria_nursery, ..)` stores
`._ria_nursery` (:202, :238); sole read is
`run_in_actor()` passing `nursery=self._ria_nursery`
(:442) into `start_actor()`'s `nursery:
trio.Nursery|None` escape-hatch param (:305, :367).
- `._cancel_after_result_on_exit: set` (:244) marks ria
portals (:457).
- `_open_and_supervise_one_cancels_all_nursery()` nests
`da_nursery` (:609) around `ria_nursery` (:622); the
`finally:` at the ria->da boundary (:747-766) raises
collected `errors` (single exc or BEG).
- `runtime/_portal.py`
- `._expect_result_ctx` (:112) set by `_submit_for_result()`
(:142, sole caller `run_in_actor()`); consumed by
`wait_for_result()` (:167) + deprecated `result()` (:220).
The `None` branch (:184-196) returns the `NoResult`
sentinel (`_exceptions.py:1164`).
- `spawn/_spawn.py`
- `exhaust_portal()` (:129): awaits
`portal.wait_for_result()`, CATCHES+RETURNS any exc
(never raises).
- `cancel_on_completion()` (:177): `exhaust_portal()` ->
on exc-result stash `errors[uid] = result` (:203) ->
ALWAYS `portal.cancel_actor()` (:218).
- `spawn/_trio.py` (:195-222) + `spawn/_mp.py` (:187-213),
identical shape: after shielded
`await an._join_procs.wait()`, open a per-child local
nursery; IFF `portal in an._cancel_after_result_on_exit`
start `cancel_on_completion` alongside `soft_kill()`; when
`soft_kill` returns first, `nursery.cancel_scope.cancel()`
reaps the result-waiter.
## The load-bearing semantic (already-deferred errors)
Remote ria-child errors NEVER raise into `ria_nursery`:
1. reaper tasks only START after `_join_procs.set()` (block
exit or the inner error handler),
2. `exhaust_portal` swallows the exc into a return value,
3. `cancel_on_completion` stashes it in `errors` + cancels
that child,
4. the ria->da `finally:` re-raises collected `errors` (and
`an.cancel()`s any daemon stragglers).
So mid-block there is NO error propagation from ria children
(unless user code explicitly `await portal.wait_for_result()`s)
— the two-nursery nesting only sequences "reap ria results
BEFORE blocking on daemon join". A single-nursery impl only
needs to preserve that sequencing, not any ASAP-cancel
behavior.
## Target design
### step A: single-nursery `run_in_actor()` (mechanical)
- `run_in_actor()` spawns via the DEFAULT (`_da_nursery`)
path — drop `nursery=self._ria_nursery`.
- rename `._cancel_after_result_on_exit` ->
`._ria_portals: dict[portal, Actor]` (need the subactor ref
for `cancel_on_completion`).
- move reaper start-up OUT of the backends into
`_open_and_supervise...`: immediately after EACH
`an._join_procs.set()` call-site (happy path :642, inner
error handler :661), start one
`cancel_on_completion(portal, subactor, errors)` task per
ria portal into `da_nursery`, then (happy path only)
`await` their completion BEFORE falling out of the
`try:`/`finally:` that raises `errors` — e.g. gather in a
dedicated inner `trio.open_nursery()` block replacing
today's `ria_nursery` join point.
- delete the membership branch + local reaper nursery from
`_trio.py`/`_mp.py` (keep the `soft_kill()` call; the
per-child local nursery collapses to just `soft_kill`).
- `_trio.py:310` `_children.pop()` etc. unchanged.
### step B: delete the plumbing
- `_open_and_supervise...`: drop the inner
`ria_nursery` + merge its `except BaseException` classify
logic into ONE handler on the (now single) nursery scope;
`ActorNursery.__init__` loses the `ria_nursery` param.
- `start_actor()` loses the `nursery:` escape-hatch param
(the :302-304 TODO).
- backends: no more `_cancel_after_result_on_exit` refs.
### step C: (separate PRs) deprecate + migrate + excise
- migrate in-repo `.run_in_actor()` usage to
`to_actor.run()`: tests 46 hits/9 files (test_cancellation
15, test_infected_asyncio 10, test_spawning 8, registrar 3,
adv_streaming 4, pubsub 2, rpc 1, runtime 1), examples 28
hits/13 files (debugging/* dominate), docs 20 hits/8 rst
files. NOTE: many sites also use deprecated
`Portal.result()`/`wait_for_result()` — these die with
`_expect_result_ctx`, so migration must land FIRST.
- add `DeprecationWarning` to `run_in_actor()` (+
`_submit_for_result`/`wait_for_result`).
- final excision: `run_in_actor()`, `_submit_for_result`,
`_expect_result_ctx`, `wait_for_result`/`result`,
`exhaust_portal`, `cancel_on_completion`, `NoResult`.
## Risk register
1. hard-killed ria child: today the backend-local
`nursery.cancel_scope.cancel()` discards a still-parked
reaper when the proc dies first; a da_nursery-hosted
reaper instead sees the transport break ->
`exhaust_portal` returns a `TransportClosed`-ish exc ->
NEW entry in `errors` that today gets discarded. Guard:
reap-gather block must cancel remaining reapers once all
ria procs are dead, or filter transport-death excs for
already-`cancel_called` children.
2. error-path ordering: inner handler today sets
`_join_procs` THEN `an.cancel()`; reapers race the
cancel-RPC. Keep that ordering when moving reaper spawn.
3. debugger interplay: `maybe_wait_for_debugger()` calls
(:654, :730) must stay BEFORE any reap/cancel issuance.
4. `errors` double-entry: local body error (:646) + child's
relayed exc (via reaper) can both land for the same
scenario -> BEG shape changes vs today? (today has the
same dual-write sites; keep behavior identical.)
5. mp backend parity: mirror every `_trio.py` edit in
`_mp.py` (identical block).
## Step-A first-probe findings (2026-07-02, WIP in tree)
Step A is IMPLEMENTED (uncommitted):
`run_in_actor()` spawns via da_nursery; new
`_supervise._reap_ria_portals()` helper; reap awaited after
happy-path `_join_procs.set()`; error-path runs reap
CONCURRENT with `an.cancel()` in the shielded block;
backends stripped of the membership branch + per-child
reaper nursery (+ dead imports).
Probe history (trio backend):
- `tests/test_to_actor.py` + `tests/test_spawning.py`:
20/20 PASS — incl. all `run_in_actor()` result
round-trips + `test_remote_error` (single erroring child,
body re-raise -> inner error path).
- FIRST attempt ran the error-path reap CONCURRENT with
`an.cancel()` (mimicking the old backend-side race):
`test_cancellation.py::test_multierror` (2 erroring ria
children, body re-raises one) DEADLOCKED. Root cause per
the sequencing fix below: reap + cancel must NOT race at
this layer (suspected `._children` pop-during-iteration
and/or double-cancel RPC wedge; not fully root-caused
since the fix removes the race wholesale).
- FIX (2nd attempt, current impl): error path SEQUENCES:
(1) snapshot ria `(portal, subactor)` pairs (backend
`finally`s pop `._children` as procs reap), (2)
`await an.cancel()`, (3) bounded reap over the snapshot.
Bound was first 3s -> blew the `fail_after` deadline in
`test_cancel_while_childs_child_in_sync_sleep` (hard-
killed grandchild never relays => reaper parks the full
bound). Tightened to 0.5s: anything collectable is
already queued in the local ctx (relayed BEFORE the
cancel); a parked reaper self-cleans (`trio.Cancelled`
results are never stashed).
- RESULT: `tests/test_cancellation.py` FULLY GREEN
(20 passed, 1 xfailed, 77s); full-suite gate run kicked
off same session (see final report/next session).
Remaining risk: on slow CI a relayed-but-undelivered error
racing the 0.5s bound could drop an `errors` entry
(BEG-shape flake); if observed, scale the bound via the
`cpu_perf_headroom()`-style approach or peek
`Portal._final_result_msg`/ctx queue state instead of
time-bounding.
## Step-B outcome (2026-07-02, done in tree)
Step A landed as `5cd190c5` (code) + `99310269` (docs).
Step B implemented on top (uncommitted):
- `._ria_nursery` is GONE — the inner
`async with (collapse_eg(), trio.open_nursery() as
ria_nursery)` layer in
`_open_and_supervise_one_cancels_all_nursery` is deleted;
`da_nursery` is now the single nursery for ALL subactors.
- `ActorNursery.__init__` drops the `ria_nursery` param +
the `self._ria_nursery` attr; `start_actor()` drops its
`nursery=` escape-hatch param (uses `self._da_nursery`
directly).
- `._cancel_after_result_on_exit` STAYS — it's the
ria-child discriminator for `_reap_ria_portals()`.
Deliberately NOT done (deferred to its own higher-risk PR,
flagged with a TODO at the outer `except`): merging the two
error handlers into one. Rationale — collapsing the empty
nursery is provably behavior-preserving (a zero-task
`trio.open_nursery()` only adds a checkpoint), whereas the
inner `except BaseException` (swallow-into-`errors`) and
outer `except (...)` (re-raise, safety-net for the inner
handler's own non-shielded awaits) have DIFFERENT
semantics; merging changes error/cancel propagation and
wants isolated review + its own gate. Both handlers are
kept, now nested directly under the single nursery.
Why the collapse is safe: post-step-A NOTHING spawns into
`ria_nursery` (its only reader, `run_in_actor`'s
`nursery=self._ria_nursery`, was removed in A; the stored
attr was never read again). So the layer was pure dead
weight.
Gate (trio backend, all 0-failure):
- targeted set (`test_cancellation test_spawning test_local
test_rpc test_to_actor`) = 49 passed, 1 xfailed.
- tail set (`test_reg_err_types remote_exc_relay
resource_cache ringbuf root_infect_asyncio root_runtime
runtime shm task_broadcasting trioisms trionics/`) = 63
passed, 1 skipped, 5 xfailed.
- full-suite head ~73% (subdirs + `test_2way`..`test_pubsub`)
= 303 passed before the known-flaky `test_dynamic_pub_sub`
TooSlowError stall (pre-existing; same hang in the step-A
full run). Suite ran slow this session (~13min vs 555s
cold, likely thermal from back-to-back runs), never
completing within an 800s bound — but split across the
above three runs EVERY module passed under step B.
## Step-B2 outcome (2026-07-02, done in tree)
Step B committed as `9201a2ed` (code) + `d2e812fb` (docs), then
branched to `drop_ria_nursery`. Step B2 (the deferred
handler-merge) implemented on top (uncommitted):
- the two nested handlers in
`_open_and_supervise_one_cancels_all_nursery` collapse to
ONE `except BaseException as _scope_err` + the existing
`finally`. The `outer_err`/`inner_err` locals go away.
Why it's safe (trace, not hope): the OLD inner handler records
`errors[actor.aid.uid]` as its FIRST statement (before any
await). So whenever an error path runs, `errors` is non-empty.
The OLD outer handler was only reachable via leakage from the
inner handler (it catches `BaseException`, so nothing from the
`yield` scope bypasses it) — and by then `errors` is already
populated, so the `finally`'s `raise` from `errors` ALWAYS
superseded the outer handler's own `raise`. i.e. the outer
`raise` was dead. The outer handler's other effects
(`_scope_error`, a 2nd debugger-wait, child-cancel) are
redundant with the merged handler + `finally`. So one handler
+ `finally` is observably equivalent.
Residual nuance (accepted): in the rare "`trio.Cancelled`
delivered during the non-shielded `maybe_wait_for_debugger`"
path, the merged form may leave `_cancel_called` False (cancel
happens after the wait), so `open_nursery`'s tb-hiding guard
(`not cancel_called and _scope_error`) can show a tb it
previously hid. More informative, not less; no test asserts on
it.
Gate (box ran ~2.7x slow this session, load-induced
`TooSlowError` flakiness on timing tests — NOT code; see
[[env_cpu_throttle_masquerades_as_regression]]):
- baseline (pre-B2 tip `9201a2ed`) full suite
(`-k 'not dynamic_pub_sub'`) = 300 passed + 1
`test_ext_types_over_ipc` `TooSlowError` that passes 6/6 in
isolation (4.89s).
- B2 error/cancel gate (`test_cancellation remote_exc_relay
inter_peer_cancellation advanced_faults oob_cancellation
to_actor spawning local rpc`) = 71 passed, 1 xfailed
(125s).
- B2 full-suite run: see `b2_full.log` (result appended on
completion). RECOMMEND a clean full-suite run on a
normal-speed box before this merges.
## Regression + fix: ria-reap hang (2026-07-02)
Human hit a full-suite hang on
`test_infected_asyncio.py::test_tractor_cancels_aio`. Bisected:
passes at pre-ria `a34aaf98` (0.59s), hangs at B2 `e617b498`
(90s+). Root-caused to the STEP-A reaper hoist (`5cd190c5`),
NOT B2 (`_reap_ria_portals` is byte-identical A->B2).
Bug: the test does `run_in_actor(asyncio_actor)` then a USER
`portal.cancel_actor()` and exits the block cleanly -> the
happy path's `await _reap_ria_portals()`, which waits UNBOUNDED
on `cancel_on_completion -> wait_for_result()`. The child was
cancelled out-of-band so no final result is relayed -> parked
forever. The OLD spawn-backend reaper was raced against
`soft_kill()` (per-child nursery `cancel_scope.cancel()` on
subproc death); the hoist dropped that race.
Fix: `_reap_ria_portals()` runs each `cancel_on_completion()`
in a local nursery alongside a `proc.poll()` death-watch that
cancels the parked reaper once the subproc exits — restoring
the old race, backend-agnostic (guarded by
`hasattr(proc, 'poll')` for a future `subint` handle).
Why POLL (`proc.poll()`) not the event-driven `wait_func`:
the mp waiter (`_spawn.proc_waiter`) does
`wait_readable(proc.sentinel)`, and `soft_kill()` is ALREADY
awaiting that same fd concurrently in the daemon nursery — a
2nd `wait_readable` on one fd raises `trio.BusyResourceError`.
(`trio.Process.wait()` IS multi-waiter-safe, but mp has no
async equivalent.) `proc.poll()` — the same liveness check
`soft_kill` itself falls back to — is the conflict-free common
denominator. Verified: poll-fix passes on BOTH trio and
mp_spawn.
Also added a per-test anti-hang guard: wrapped
`test_tractor_cancels_aio`'s `main()` in
`with trio.fail_after(9 * cpu_perf_headroom())` — the blessed
pattern (`pytest-timeout`'s global cap is intentionally off;
breaks trio under fork backends, see `pyproject` NOTE). So a
future recurrence FAILS FAST instead of hanging the suite.
(Several other tests in the file are still guardless —
`test_aio_simple_error`, `test_trio_error_cancels_intertask_chan`,
`test_aio_errors_and_channel_propagates_and_closes` — candidate
follow-up sweep.)
Lesson: the B2 focused gate OMITTED `test_infected_asyncio`
(and the full runs were clipped/slow), so the step-A hang
slipped through. Any future ria-touching change MUST gate
`test_infected_asyncio` explicitly.
Gate: `test_tractor_cancels_aio` green (trio 1.53s, mp 3.98s);
fix gate (`test_infected_asyncio test_cancellation test_to_actor
test_spawning`) = 74 passed, 3 xfailed, 0 failures.
## PAUSED (2026-07-02): re-assess the reaper's SCOPE
User's insight (compelling — likely the real root cause of
the hang, not just the missing proc-death race):
> the "hoisting" of 5cd190c5 was just not really done right
> — the hoist should have been into the `to_actor` scope,
> not `_supervise`.
The argument: `.run_in_actor()`'s result-waiting/reaping got
hoisted into `_supervise._reap_ria_portals` (nursery-machinery
scope), which has NO natural cancel-scope to bound a parked
`wait_for_result()` — hence the awkward proc-death race +
the poll-vs-`proc_waiter` dilemma. If the result-wait instead
lived in the `to_actor` one-shot scope
(`to_actor._invoke_in_subactor()`), it would sit right next to
the caller's `an` + a local `trio` task-nursery + cancel-scope
(the `trio.to_thread`-style model #477 actually wants) — so
bounding/cancelling the wait is trivial and the hang
dissolves from correct scoping rather than a bolt-on race.
Follow-on to re-evaluate on resume:
- should `_reap_ria_portals` exist AT ALL, or should
result-waiting move entirely into
`to_actor._invoke_in_subactor()`?
- reimplement legacy `run_in_actor()` on top of
`to_actor.run()` so `_reap_ria_portals` +
`_cancel_after_result_on_exit` can be DROPPED from
`_supervise` entirely (the true #477 simplification)?
- the poll-vs-event decision is MOOT under this re-scoping.
State at pause: `test_infected_asyncio` anti-hang guard
COMMITTED (`d1fb4a1a`, intentionally red w/o the fix — the
user's failing-test-first convention). The poll-based reap
fix in `_supervise.py` is UNCOMMITTED and likely SUPERSEDED
by the re-scoping — do NOT land it as-is.
## RESOLVED (2026-07-06): migrate everything, remove the API
The PAUSED re-assessment concluded decisively: rather than
re-scope `_reap_ria_portals` (or bolt any hack onto it), the
`run_in_actor()` API itself was REMOVED — its non-blocking
"result at teardown" semantic predates streaming and confused
more than it served. Every in-repo caller was migrated
per-file/-group (each its own commit, each gated):
- tests: `test_infected_asyncio` `test_runtime` `test_rpc`
`test_spawning` `test_pubsub` `test_registrar`
`test_cancellation` (3 groups) `test_advanced_streaming`.
- examples: 4 non-debugging + all 8 `debugging/` REPL scripts
(debugger suite byte-identical green, 28p/6s).
- docs: 8 rst pages + the `experimental/_pubsub` docstring.
Migration patterns (the `run_in_actor` shape -> successor):
- blocking result -> `to_actor.run(fn, an=an, ...)`
- fire-&-forget/forever -> bg `to_actor.run()` task in a local
`trio` task-nursery (or `start_actor`
+ bg `Portal.run()` when a portal
handle is needed)
- concurrent fan-out -> N bg `to_actor.run()` tasks / or
`gather_contexts([p.open_context(..)])`
- reap-all-error-collect -> the "collect don't cancel" pattern:
each one-shot catches + stashes its
`RemoteActorError`, group raised
after the task-nursery joins (see
`examples/debugging/multi_subactors.py`)
- mutual-rendezvous -> peers must OUTLIVE both dialogs:
`start_actor()` daemons + concurrent
`Portal.run()`s + explicit
`an.cancel()` (eager one-shot reap
races the slower peer's dial of the
winner's dead sockaddr; found via
`test_trynamic_trio` flake).
Semantic deltas (tests loosened accordingly):
- teardown-reap-all BEG-of-N is GONE: local task-nurseries are
cancel-on-first, raced siblings' `Cancelled`s are absorbed,
and the runtime's `collapse_eg()` unwraps every single-member
group at each actor boundary — a fully-raced nested tree
relays a bare (annotated) `RemoteActorError` chain.
- `test_multierror_fast_nursery`'s obsolete BEG-of-25 assertion
deleted; `test_concurrent_start_error_reaps_all` retains its
high-fan-out startup/cancel/reap stress under caller-scoped
semantics.
- `test_nested_multierrors` re-purposed separately as deep-tree
cancel-cascade stress w/ a race-tolerant shape walk.
Final excision (after zero callers remained): `run_in_actor()`,
`._cancel_after_result_on_exit`, `_reap_ria_portals()`,
`Portal._submit_for_result/._expect_result_ctx/
.wait_for_result()/.result()`, `exhaust_portal()`,
`cancel_on_completion()`, `NoResult` — net -402 lines. The
reap-hang class (unbounded `wait_for_result` in machinery
scope) dissolves structurally: the only result-wait left lives
in the caller's task inside its own cancel-scope; the
`d1fb4a1a` anti-hang guard test passes by construction. The
poll-vs-`proc_waiter` debate is moot as predicted.
## Follow-up sketch: `to_actor.open_one_shot()` (run-async parity)
If deferred-result parity is ever wanted, the design that needs
NO runtime coupling, NO returned `Portal` and NO cancel-relay
`trio.Event` machinery:
async with to_actor.open_one_shot(
fn, an=an, **kws,
) as one_shot:
... # concurrent caller work
val = await one_shot.wait() # optional; errors always
# propagate at scope exit
an `@acm` that opens a private task-nursery, `start_soon`s ONE
task running the existing blocking `run()` and stashes the
value in a slot + sets a done-`trio.Event` (a memo, not a
cancel relay). Cancellation = plain scope-cancel of the acm's
nursery (the parked `Portal.run()` unwinds via `Cancelled`, the
shielded `cancel_actor()` reap still runs); a child error
raises into the acm scope so an un-`wait()`ed one-shot can
never silently drop its error. i.e. the old reaper's job is
done by scoping, not machinery. ~40 lines, all in
`to_actor/_api.py`, zero `_supervise` involvement.
## Verification gate
- per-migration-commit module gates on `trio` (+ `mp_spawn`
spot-gates incl. `test_infected_asyncio` per the B2 lesson);
`tests/devx/test_debugger.py` for the REPL flows.
- full suite on `trio` + `mp_spawn` at branch tip + CI matrix
via draft PR #484.

View File

@ -0,0 +1,78 @@
---
model: claude-fable-5
service: claude
session: f6c84722-471a-4458-9a80-e453fea9029f
timestamp: 2026-07-02T16:58:06Z
git_ref: a34aaf98
scope: code
substantive: true
raw_file: 20260702T165806Z_a34aaf98_prompt_io.raw.md
---
## Prompt
Follow-up round in the same session as the
`tractor.to_actor` landing (see
`20260702T154255Z_65bf9df5_prompt_io.md`). After
committing that work the user green-lit the deferred
items:
> go go go on this with what time you have left, in
> particular see if you can get the _ria_nursery
> removal going!
then extended the deadline twice to iterate on the
discovered hang:
> continue on this up until a 12:58:30 deadline
and finally chose "Commit step A now" from the
next-steps prompt.
## Response summary
Step A of the `._ria_nursery` removal (issue #477): hoist
`.run_in_actor()` result-reaping out of the spawn
backends into the `ActorNursery` machinery so ria
children spawn via the default daemon nursery,
- new `_supervise._reap_ria_portals()` runs one
`_spawn.cancel_on_completion()` task per ria child
AFTER `._join_procs` is set; happy path awaits it
right after `._join_procs.set()`.
- error path SEQUENCES: snapshot ria
`(portal, subactor)` pairs -> `await an.cancel()` ->
0.5s-bounded reap. Two failed intermediates informed
this: a concurrent reap+cancel DEADLOCKED
`test_multierror`; a 3s bound blew
`test_cancel_while_childs_child_in_sync_sleep`'s
`fail_after` deadline.
- backends (`spawn/_trio.py`, `spawn/_mp.py`) lose the
`._cancel_after_result_on_exit` membership branch,
per-child reaper nursery + dead imports.
- design/probe-history doc:
`ai/conc-anal/ria_nursery_removal_plan.md` (from an
agent-verified machinery map).
Verification: `test_cancellation.py` fully green
(20 passed, 1 xfailed) incl. the previously-hung
`test_multierror`; `test_to_actor`+`test_spawning`
20/20; bounded full-suite gate SIGINT'd ~30s early at
303 passed / 0 failures (user opted to commit on that
signal, deferring the unbounded re-run to step-B
verification).
## Files changed
- `tractor/runtime/_supervise.py``_reap_ria_portals()`
+ two call-sites; `run_in_actor()` off the ria nursery
- `tractor/spawn/_trio.py` — reaper branch + import drop
- `tractor/spawn/_mp.py` — same as `_trio.py`
- `ai/conc-anal/ria_nursery_removal_plan.md` — plan +
probe history
## Human edits
None yet — committed via the drafted
`.claude/git_commit_msg_ria_step_a.md` (user-driven
`git commit --edit`).

View File

@ -0,0 +1,55 @@
---
model: claude-fable-5
service: claude
timestamp: 2026-07-02T16:58:06Z
git_ref: a34aaf98
diff_cmd: git diff a34aaf98..wkt/to_actor_subpkg
---
# Raw AI output (diff-ref mode)
Step-A code is committed on `wkt/to_actor_subpkg`
directly after `a34aaf98`; per diff-ref mode the verbatim
content is reachable via the pointers below.
## Generated files
> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/runtime/_supervise.py`
New `_reap_ria_portals(an, errors, ria_children=None)`
helper (one `_spawn.cancel_on_completion()` task per ria
child under `collapse_eg()` + a local nursery);
`run_in_actor()` drops `nursery=self._ria_nursery`; happy
path awaits the reap right after `._join_procs.set()`;
inner error handler snapshots ria pairs, runs
`await an.cancel()` then a `move_on_after(0.5)`-bounded
reap over the snapshot.
> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/spawn/_trio.py`
> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/spawn/_mp.py`
Both backends: the post-`_join_procs` block collapses to
a bare `soft_kill()` (membership branch, per-child reaper
nursery, reaper-cancel logging and the now-unused
`cancel_on_completion` imports all removed).
> `git diff a34aaf98..wkt/to_actor_subpkg -- ai/conc-anal/ria_nursery_removal_plan.md`
Agent-verified machinery map, 3-step design (A/B/C),
probe history (deadlock -> sequencing fix -> bound
tightening) and risk register.
## Test runs (verbatim)
```
tests/test_cancellation.py: 20 passed, 1 xfailed in 77.28s
tests/test_to_actor.py + tests/test_spawning.py: 20 passed
full-suite (SIGINT at 540s bound, ~30s short):
303 passed, 9 skipped, 1 xfailed, 2 xpassed in 539.42s
```
Failed intermediates (not in final diff): concurrent
reap+`an.cancel()` deadlocked `test_multierror`; a 3s
reap bound blew
`test_cancel_while_childs_child_in_sync_sleep`'s
`fail_after` deadline.

View File

@ -0,0 +1,60 @@
---
model: claude-opus-4-8
service: claude
session: c7860302-6a33-4d92-821c-0807d68be464
timestamp: 2026-07-02T17:22:33Z
git_ref: 5cd190c5
scope: code
substantive: true
raw_file: 20260702T172233Z_5cd190c5_prompt_io.raw.md
---
## Prompt
Branched session (`to_actor_subpkg_POST_WEEKLY_CUTOFF_2OPUS`,
now on Opus 4.8) continuing the `._ria_nursery` removal after
step A committed + pushed (`5cd190c5`/`99310269`):
> ok continue where your superior left off please
then, after the commit-split discussion:
> yup committed and pushed so take a look ma fren.
i.e. green-lighting step-B prep against the committed step-A
tree.
## Response summary
Step B of the `._ria_nursery` removal (issue #477): delete
the now-vestigial secondary nursery,
- collapse the inner `async with (collapse_eg(),
trio.open_nursery() as ria_nursery)` layer in
`_open_and_supervise_one_cancels_all_nursery``da_nursery`
becomes the single nursery for ALL subactors.
- `ActorNursery.__init__` drops the `ria_nursery` param + the
`self._ria_nursery` attr; `start_actor()` drops its
`nursery=` escape-hatch param.
- `._cancel_after_result_on_exit` kept (ria-child
discriminator for `_reap_ria_portals()`).
Verified behavior-preserving via a first-principles argument
(zero-task nursery = a bare checkpoint) + the targeted gate
(`test_cancellation test_spawning test_local test_rpc
test_to_actor` = 49 passed, 1 xfailed on trio). The two
error handlers were deliberately NOT merged — that changes
propagation semantics and is deferred to its own PR (TODO
left at the outer `except`).
## Files changed
- `tractor/runtime/_supervise.py` — collapse the ria nursery
layer + drop the ctor/`start_actor` params + refresh the
now-stale nursery comments
## Human edits
None yet — committed via the drafted
`.claude/git_commit_msg_ria_step_b.md` (user-driven
`git commit --edit`).

View File

@ -0,0 +1,51 @@
---
model: claude-opus-4-8
service: claude
timestamp: 2026-07-02T17:22:33Z
git_ref: 5cd190c5
diff_cmd: git diff 5cd190c5..wkt/to_actor_subpkg
---
# Raw AI output (diff-ref mode)
Step-B code lives on `wkt/to_actor_subpkg` after `5cd190c5`;
per diff-ref mode the verbatim content is reachable via the
pointer below.
## Generated files
> `git diff 5cd190c5..wkt/to_actor_subpkg -- tractor/runtime/_supervise.py`
- `ActorNursery.__init__`: `ria_nursery` param removed;
`self._ria_nursery = ria_nursery` block deleted;
`_cancel_after_result_on_exit` comment refreshed.
- `start_actor()`: `nursery=` param removed; body uses
`self._da_nursery.start(...)` directly.
- `_open_and_supervise_one_cancels_all_nursery()`: the inner
`async with (collapse_eg(), trio.open_nursery() as
ria_nursery)` layer removed; `an = ActorNursery(actor,
da_nursery, errors)` constructed once under the single
`da_nursery`; the inner-try body de-indented one level;
both error handlers retained; the da-nursery lead comment
and the outer-`except` TODO refreshed to describe the
single-nursery reality + flag the (deferred) handler-merge.
> `git diff 5cd190c5..wkt/to_actor_subpkg -- ai/conc-anal/ria_nursery_removal_plan.md`
Added a "Step-B outcome" section (collapse rationale,
handler-merge deferral, safety argument, gate result).
## Test runs (verbatim)
```
targeted gate (trio):
tests/test_cancellation.py tests/test_spawning.py
tests/test_local.py tests/test_rpc.py tests/test_to_actor.py
-> 49 passed, 1 xfailed in 88.62s
signature checks:
ActorNursery.__init__ params: ['self', 'actor', 'da_nursery', 'errors']
start_actor has nursery param: False
full-suite gate: in progress (see stepB_full.log)
```

View File

@ -0,0 +1,68 @@
---
model: claude-opus-4-8
service: claude
session: c7860302-6a33-4d92-821c-0807d68be464
timestamp: 2026-07-02T22:25:44Z
git_ref: 9201a2ed
scope: code
substantive: true
raw_file: 20260702T222544Z_9201a2ed_prompt_io.raw.md
---
## Prompt
Continuing on branch `drop_ria_nursery` after step B
(`9201a2ed`). User green-lit the deferred error-handler merge:
> yes, i checked us out on a new branch for the ria-tn removal;
> work off the new wkt branch drop_ria_nursery.
and (re the migration) "nope, later we'll do this", plus:
> do ensure the test suite is green in the current state and
> those going forward ya?
and the standing reminder: no commits without a human ack.
## Response summary
Step B2 of the `._ria_nursery` removal (issue #477): merge the
two nested error handlers in
`_open_and_supervise_one_cancels_all_nursery` into one, now
that the 2ndary nursery (whose `__aexit__` the outer handler
originally guarded) is gone.
Trace-driven justification (see
`ai/conc-anal/ria_nursery_removal_plan.md` Step-B2 section):
the old inner handler records `errors[uid]` as its FIRST
action, so `errors` is always non-empty by the time control
could reach the outer handler — making the outer handler's
`raise` DEAD (always superseded by the `finally`'s
raise-from-`errors`). Collapsing to a single
`except BaseException as _scope_err` + the existing `finally`
is therefore behavior-preserving.
- removed the outer `try` + the
`except (Exception, BaseExceptionGroup, trio.Cancelled)`
handler; de-indented the inner handler one level.
- dropped the now-unused `outer_err`/`inner_err` locals
(single `_scope_err`).
- net ~30 lines lighter.
Gated hard: baseline (pre-B2 tip) = 300 passed + 1
load-induced `TooSlowError` flake (passes 6/6 in isolation);
B2 error/cancel gate (`test_cancellation remote_exc_relay
inter_peer_cancellation advanced_faults oob_cancellation
to_actor spawning local rpc`) = 71 passed, 1 xfailed; B2
full-suite run following.
## Files changed
- `tractor/runtime/_supervise.py` — collapse the two handlers
into one; drop `outer_err`/`inner_err`
## Human edits
None yet — committed via the drafted
`.claude/git_commit_msg_ria_b2.md` (user-driven
`git commit --edit`).

View File

@ -0,0 +1,55 @@
---
model: claude-opus-4-8
service: claude
timestamp: 2026-07-02T22:25:44Z
git_ref: 9201a2ed
diff_cmd: git diff 9201a2ed..drop_ria_nursery
---
# Raw AI output (diff-ref mode)
Step-B2 code lives on `drop_ria_nursery` after `9201a2ed`; per
diff-ref mode the verbatim content is reachable via the pointer
below.
## Generated files
> `git diff 9201a2ed..drop_ria_nursery -- tractor/runtime/_supervise.py`
`_open_and_supervise_one_cancels_all_nursery`:
- removed the outer `try:` wrapper and the
`except (Exception, BaseExceptionGroup, trio.Cancelled) as
_outer_err:` safety-net handler.
- the former inner `except BaseException` is now THE handler,
renamed local `_inner_err` -> `_scope_err`, de-indented one
level; it sets `an._scope_error`, records `errors[uid]`,
waits on the debugger, `_join_procs.set()`, then a shielded
classify/log + snapshot-ria + `an.cancel()` + 0.5s-bounded
`_reap_ria_portals()`. No re-raise (the `finally` raises
from `errors`).
- `finally` block unchanged.
- dropped the `outer_err`/`inner_err` local decls at fn top.
(The diff is large — ~119+/149- — because de-indenting the
handler body one level rewrites every line in the block; the
logic delta is just "two handlers -> one".)
## Test runs (verbatim)
```
baseline (pre-B2, step-B tip 9201a2ed), full suite
(dynamic_pub_sub deselected):
1 failed, 300 passed, 9 skipped, 2 deselected, 1 xfailed,
2 xpassed in 1499.49s
-> the 1 failure = test_ext_types_over_ipc[...] trio.TooSlowError
(load-induced; passes 6/6 in isolation in 4.89s)
B2 error/cancel gate:
tests/test_cancellation test_remote_exc_relay
test_inter_peer_cancellation test_advanced_faults
test_oob_cancellation test_to_actor test_spawning test_local
test_rpc
-> 71 passed, 1 xfailed in 125.26s
B2 full-suite run: see b2_full.log
```

View File

@ -0,0 +1,81 @@
---
model: claude-fable-5
service: claude
session: 6db64ac6-6986-4505-9343-df4ee31e67db
timestamp: 2026-07-06T17:28:18Z
git_ref: ad42871e
scope: code
substantive: true
raw_file: 20260706T172818Z_ad42871e_prompt_io.raw.md
---
## Prompt
Resuming the `drop_ria_nursery` (wkt) branch after the human's
review pause at `ba4af0ad`, with the directive:
> we need to resolve the remaining outstandings ... to finish
> this endeavour we ideally,
> - finish the ria-tn hoist into the machinery in `to_actor` if
> possible.
> - avoid supporting the `.run_in_actor()` non-blocking API
> going forward other then if we can re-wrap `to_actor.run()`
> per prior discussion with opus4.8.
> - get the test harness completely green locally and in CI -
> AND ESPECIALLY without any hangs due to the "the old
> spawn-backend reaper raced `soft_kill()`'s scope-cancel"
> - update all docs to reflect these changes once we're happy.
plus mid-flight additions:
> make sure both all the test and example code uses the naming
> `an: ActorNursery` for variable names
> where we at in terms of a potential hackaround design for
> supporting a to_actor.run_async() ... without requiring the
> same malarky ... with a `trio.Event` relay ... i'm totes fine
> with rejigging the iface to NOT return a Portal as well.
## Response summary
Completed the entire #477 `run_in_actor` removal on
`drop_ria_nursery` (16 commits this session), gated per-commit:
- ported the remaining `test_cancellation` groups
(`some_cancels_all`, `nested_multierrors`+`spawn_and_error`,
SIGINT + sync-sleep) — one-shots become concurrent
`to_actor.run()` tasks in local task-nurseries; BEG-shape
assertions loosened for cancel-on-first + the runtime's
`collapse_eg()` single-member unwrap (a fully-raced nested
tree relays a bare annotated `RemoteActorError` chain).
- fixed a pre-existing `UnboundLocalError` (`timeout` `match`
had no default arm for non-trio/MTF backends).
- ported `test_dynamic_pub_sub`, 4 non-debugging examples, all
8 `debugging/` examples (debugger suite byte-identical,
28p/6s; `multi_subactors` introduces the "collect don't
cancel" reap-all replacement pattern), 8 docs pages + the
`experimental/_pubsub` docstring.
- EXCISED the API + cluster: `run_in_actor`,
`_reap_ria_portals`, `_cancel_after_result_on_exit`,
`Portal._submit_for_result/_expect_result_ctx/
wait_for_result/result`, `exhaust_portal`,
`cancel_on_completion`, `NoResult` — net -402 lines. The
reap-hang class dissolves structurally (result-waits now only
in caller task-scope).
- found + fixed a real migration race: mutual-rendezvous peers
(`test_trynamic_trio`, `a_trynamic_first_scene.py`) flaked
because an eagerly-reaped one-shot dies while its peer still
dials the registry-resolved (dead) sockaddr — such peers now
pin lifetimes via `start_actor()` + concurrent `Portal.run()`
+ explicit `an.cancel()`.
- `an: ActorNursery` naming sweep across tests/examples (±82
lines, scoped renames, prose untouched).
- parked a `to_actor.open_one_shot()` design sketch (acm +
private task-nursery over blocking `run()`; done-Event as
memo not cancel-relay; no Portal) in the plan doc.
## Files changed
See commits `d01a2123..ad42871e` on `drop_ria_nursery`
(tests, examples, docs, `tractor/{runtime,spawn,to_actor,msg}`
+ `_exceptions/_context/experimental`).

View File

@ -0,0 +1,39 @@
---
model: claude-fable-5
service: claude
timestamp: 2026-07-06T17:28:18Z
git_ref: ad42871e
diff_cmd: git diff ba4af0ad..ad42871e
---
# Raw AI output (diff-ref mode)
This session's output spans the 16 migration/excision commits
`d01a2123..ad42871e` on `drop_ria_nursery`; per diff-ref mode
the verbatim content is reachable via the pointer below.
## Generated files
> `git diff ba4af0ad..ad42871e`
Commit-wise (each `Gate:`-footed msg documents its own module
gate):
- `d01a2123` port `test_some_cancels_all`
- `697c6152` fix unbound `timeout` (non-trio/MTF `match` arm)
- `fa8799d5` port `test_nested_multierrors`
- `f11754ce` port SIGINT + sync-sleep cancel tests
- `cb6202e3` port `test_dynamic_pub_sub`
- `d8af5f12` port non-debugging examples
- `a3057cb2` port debugging examples (+ `test_debugger`
nested-nurseries final-shape expectations)
- `d6bed7c4` port docs (8 rst pages)
- `07e1669e` fix stale `@pub` docstring example
- `2a59cefb` REMOVE `run_in_actor()` + the ria reap cluster
(net -402 lines)
- `a297a32a` fix mutual-rendezvous premature-reap race
- `ad42871e` `an: ActorNursery` naming sweep
Plan/design record updated in
`ai/conc-anal/ria_nursery_removal_plan.md` (RESOLVED section +
the `to_actor.open_one_shot()` follow-up sketch).

View File

@ -0,0 +1,30 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-11T23:38:33Z
git_ref: 7cbd64ee
scope: code
substantive: true
raw_file: 20260811T233833Z_7cbd64ee_prompt_io.raw.md
---
## Prompt
Open a new isolated worktree in the local Tractor repository and draft a fix
for `BroadcastReceiver` reporting that a lagged one-slot consumer dropped
zero values when one value had actually been displaced.
## Response summary
Corrected the off-by-one lag count while preserving cursor recovery and added
deterministic narrow- and wider-window regressions for exact loss reporting.
## Files changed
- `tractor/trionics/_broadcast.py` - exact broadcast overrun count.
- `tests/test_task_broadcasting.py` - lag count and recovery regression.
## Human edits
None - generated output follows the user's diagnosed edge case.

View File

@ -0,0 +1,50 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-11T23:38:33Z
git_ref: 7cbd64ee
diff_cmd: git diff HEAD~1..HEAD
---
The user asked for a Tractor fix in a new isolated worktree after a live piker
failure reported:
```text
tractor.trionics._broadcast.Lagged:
Task `piker.brokers.ib.broker.handle_order_requests` overrun and
dropped `0` values
```
Inspection showed the lag exception was valid but its count was off by one.
`BroadcastReceiver.receive_nowait()` treats `seq` as a deque index. With a
one-entry queue, index zero is the only retained value and `seq == 1` already
means one value was displaced. The old `seq - maxlen` calculation therefore
reported zero instead of one.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Adjusted the lag count to `seq - maxlen + 1` and documented why the first
invalid deque index must be included. The existing Tokio-style cursor reset
remains unchanged.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added a deterministic parameterized regression covering a one-slot queue
with one dropped value and a three-slot queue with two dropped values. The
test keeps the root receiver idle while a child subscriber synchronously
drains each produced value, asserts the exact `Lagged` message, and proves the
next receive resumes at the oldest retained item.
Verification output:
```text
.. [100%]
2 passed in 0.04s
.......... [100%]
10 passed in 4.65s
```
The targeted import resolved to the new Tractor worktree. Python compilation
and `git diff --check` passed. Adversarial review found no actionable issues;
zero-capacity channels remain a pre-existing untested edge outside this fix.

View File

@ -0,0 +1,31 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T01:23:24Z
git_ref: 06c4af17
scope: code
substantive: true
raw_file: 20260812T012324Z_06c4af17_prompt_io.raw.md
---
## Prompt
Iteratively refine Tractor PR 490. For item one, correct broadcast statistics
queue counts and Trio event checks, verify and review the exact change, then
return a complete commit plan before proceeding.
## Response summary
Converted subscriber cursor indexes into clamped retained queue counts,
removed deprecated event truthiness, and added deterministic state and
deprecation regressions.
## Files changed
- `tractor/trionics/_broadcast.py` - accurate queue and waiter statistics.
- `tests/test_task_broadcasting.py` - retained-count and event regression.
## Human edits
None - generated output follows the requested first iterative item.

View File

@ -0,0 +1,37 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T01:23:24Z
git_ref: 06c4af17
diff_cmd: git diff HEAD~1..HEAD
---
After opening draft Tractor PR 490, the user requested an iterative pass over
additional broadcast subsystem findings. The first item was to correct
`BroadcastState.statistics()` queued counts and its deprecated Trio event
truthiness check, then stop for a complete commit plan.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Changed `queued_len_by_task` from raw deque cursor indexes to retained,
receivable counts. Caught-up `-1` reports zero, valid indexes report index plus
one, and lagged cursors clamp to the current retained queue length. Replaced
`trio.Event` truthiness with an explicit `is not None` branch.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added a deterministic statistics regression using actual sends and receives.
It verifies caught-up and one-queued states, drives a root receiver beyond a
three-slot retention window to prove clamping, and installs a real
`trio.Event` while treating deprecations as errors.
Verification output:
```text
........... [100%]
11 passed in 5.76s
```
Python compilation and `git diff --check` passed. Initial adversarial review
caught unclamped lagged cursors and an ineffective event test; both were
fixed. Final review found no actionable issues.

View File

@ -0,0 +1,33 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T03:06:08Z
git_ref: 1095e7f7
scope: code
substantive: true
raw_file: 20260812T030608Z_1095e7f7_prompt_io.raw.md
---
## Prompt
For Tractor PR 490 item two, make shared underlying receive failures wake and
terminate every broadcast subscriber without losing retained values. Review,
verify and return a complete commit plan before continuing.
## Response summary
Published ordinary receive failures as terminal broadcast state, introduced
a public chained peer exception, kept control-flow exits transient while
waking peers, documented the contract, and added deterministic regressions.
## Files changed
- `tractor/trionics/_broadcast.py` - terminal failure and peer wake protocol.
- `tractor/trionics/__init__.py` - public peer exception export.
- `docs/api/trionics.rst` - failure-delivery API contract.
- `tests/test_task_broadcasting.py` - terminal and transient failure tests.
## Human edits
None - generated output follows the requested second iterative item.

View File

@ -0,0 +1,49 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T03:06:08Z
git_ref: 1095e7f7
diff_cmd: git diff HEAD~1..HEAD
---
The user requested the second iterative refinement for Tractor PR 490: ensure
non-EOC failures from a shared underlying broadcast receiver do not leave peer
subscribers blocked forever, then stop for a complete commit plan.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Added shared terminal failure publication for ordinary `Exception` values.
The receive owner gets the original exception; peers may drain retained
values and then get a fresh `BroadcastReceiveError` chained from the original.
Late subscribers observe the same terminal state without retrying the failed
underlying receiver. Process-control and cancellation-like `BaseException`
values wake peers but are re-raised without becoming durable channel state.
> `git diff HEAD~1..HEAD -- tractor/trionics/__init__.py`
Exported `BroadcastReceiveError` as the public peer-delivery exception.
> `git diff HEAD~1..HEAD -- docs/api/trionics.rst`
Documented `BroadcastReceiveError` and the owner-versus-peer delivery
contract, including retained-value draining and late subscribers.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added deterministic bounded regressions. One scripts a successful receive
followed by `RuntimeError`, proving the root drains retained data, all current
and late receivers observe terminal failure, and the source is not retried.
The second scripts a custom `BaseException`, proving peers wake and take over
the next source receive without retaining control-flow state.
Verification output:
```text
............. [100%]
13 passed in 5.62s
```
Python compilation and `git diff --check` passed. Iterative adversarial review
drove independent peer exception wrappers, ordinary-versus-control-flow
classification, bounded test completion, public docs, and the final catch-all
peer wake. Final review found no issues.

View File

@ -0,0 +1,33 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T15:00:27Z
git_ref: c2a6ccef
scope: code
substantive: true
raw_file: 20260812T150027Z_c2a6ccef_prompt_io.raw.md
---
## Prompt
For Tractor PR 490 item three, prevent subscriber closure from waking another
receiver's shared event or stranding peers. Preserve close-safe source-read
ownership handoff, then review, verify and return a complete commit plan.
## Response summary
Introduced receiver-local wait/source cancellation, owner-specific handoff,
and close precedence over shielded source values/errors/EOC. Clarified that
private scope cancellation means explicit close while outer task cancellation
remains `trio.Cancelled` for both source-read and peer-wait scopes, with
deterministic receiver-close regressions.
## Files changed
- `tractor/trionics/_broadcast.py` - receiver-local close and ownership scopes.
- `tests/test_task_broadcasting.py` - peer close and owner handoff regressions.
## Human edits
None - generated output follows the requested third iterative item.

View File

@ -0,0 +1,51 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T15:00:27Z
git_ref: c2a6ccef
diff_cmd: git diff HEAD~1..HEAD
---
The user requested the third iterative refinement for Tractor PR 490: closing
one broadcast subscriber must not set another receiver owner's shared event
and create a runnable hot loop, then stop for a complete commit plan.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Added receiver-local wait cancellation and source-read ownership scopes.
Closing a non-owner waiting behind another source reader cancels only that
receiver's private wait and maps it to `ClosedResourceError`; the shared event
remains untouched. Closing the active source owner cancels only its private
source-read scope, wakes peers after cleanup, and lets one peer take ownership.
Source outcomes are captured inside the owner scope and classified only after
checking close/cancel state. A cancellation-shielding source therefore cannot
publish a returned value, ordinary error, or EOC after its owner was closed.
The private scope's `cancel_called` bit is asserted to imply receiver closure;
outer task cancellation remains `trio.Cancelled` and is not translated into
`ClosedResourceError`. Owner-key comments document that only the receiver
identified by `recv_ready[0]` may cancel the shared source-read scope. The
same explicit-close invariant is enforced symmetrically for private peer-wait
scope cancellation.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added deterministic bounded regressions for both close positions. The
non-owner test places two peers behind an active source read, closes one and
proves only that peer exits while the shared event stays unset. The owner test
closes a source owner whose receive shields cancellation and parameterizes a
returned value, `RuntimeError`, and `EndOfChannel`; each discarded outcome
hands the next source receive to the waiting root without terminal-state or
EOC publication.
Verification output:
```text
................. [100%]
17 passed in 5.84s
```
Python compilation and `git diff --check` passed. Iterative adversarial review
caught a waiting non-owner hang, cancellation-shielded source returns, and
shielded source exceptions. All were fixed. Final review found no actionable
issues.

View File

@ -0,0 +1,34 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T21:31:17Z
git_ref: 51185487
scope: code
substantive: true
raw_file: 20260812T213117Z_51185487_prompt_io.raw.md
---
## Prompt
For Tractor PR 490 item four, expose `raise_on_lag` through the public IPC and
asyncio linked-channel subscription wrappers. Review, verify and return a
complete commit plan before applying the API downstream in piker.
## Response summary
Added public per-subscription lag policy to both wrappers, preserved first-call
root policy, documented the semantics, and covered forwarding plus real fan-out
paths.
## Files changed
- `tractor/_streaming.py` - `MsgStream` lag policy forwarding.
- `tractor/to_asyncio.py` - linked-channel lag policy forwarding.
- `docs/guide/streaming.rst` - IPC fan-out policy docs.
- `docs/guide/asyncio.rst` - linked-channel fan-out policy docs.
- `tests/test_task_broadcasting.py` - wrapper policy regression.
## Human edits
None - generated output follows the requested fourth iterative item.

View File

@ -0,0 +1,53 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T21:31:17Z
git_ref: 51185487
diff_cmd: git diff HEAD~1..HEAD
---
The user requested the fourth iterative refinement for Tractor PR 490: expose
subscriber lag policy through the public `MsgStream.subscribe()` and
`LinkedTaskChannel.subscribe()` wrappers, then stop for a complete commit
plan. This enables piker to replace private receiver mutation.
> `git diff HEAD~1..HEAD -- tractor/_streaming.py`
Added `raise_on_lag: bool = True` to `MsgStream.subscribe()`. The first call
passes the policy to the irreversibly allocated root broadcaster and its
child; later calls configure each child independently while retaining the
root's first-call policy.
> `git diff HEAD~1..HEAD -- tractor/to_asyncio.py`
Added equivalent lag-policy forwarding to `LinkedTaskChannel.subscribe()`.
> `git diff HEAD~1..HEAD -- docs/guide/streaming.rst`
> `git diff HEAD~1..HEAD -- docs/guide/asyncio.rst`
Documented strict versus warn/drop/resume behavior, independent child policy,
and first-call root policy for both wrapper types.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added a parameterized wrapper-level regression using minimal receive-compatible
handles. It verifies a first non-raising subscription configures root and
child, then a later strict child does not mutate the sticky root policy.
Verification output:
```text
................... [100%]
19 passed in 5.71s
.................... [100%]
20 passed in 6.88s
. [100%]
1 passed in 0.86s
```
The second and third runs cover actual `MsgStream` and infected-asyncio
`LinkedTaskChannel` fan-out respectively. Python compilation and
`git diff --check` passed. Adversarial review found no actionable issues.

View File

@ -0,0 +1,36 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: unavailable
timestamp: 2026-08-13T18:19:01Z
git_ref: a2e0df4b
scope: code
substantive: true
raw_file: 20260813T181901Z_a2e0df4b_prompt_io.raw.md
---
## Prompt
Continue Tractor PR 490 after the paired piker EMS consumer commit. Clean
cancelled-task diagnostics, define or reject zero-buffer broadcast behavior,
review and verify the change, then stop at a complete commit plan.
## Response summary
Bound cancellation diagnostics to receiver progress, terminal state and
resource lifetime; made EOC durable across peers; released wrapper-owned root
broadcasters without breaking graceful EOC or subclass overrides; and rejected
non-positive fan-out retention capacity.
## Files changed
- `tractor/trionics/_broadcast.py` - diagnostic lifecycle, durable EOC and
buffer validation.
- `tractor/_streaming.py` - safe `MsgStream` root broadcaster cleanup.
- `tractor/to_asyncio.py` - linked-channel root broadcaster cleanup.
- `tests/test_task_broadcasting.py` - cancellation, EOC, wrapper and capacity
regressions.
## Human edits
None - generated output follows the requested fifth iterative item.

View File

@ -0,0 +1,58 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-13T18:19:01Z
git_ref: a2e0df4b
diff_cmd: git diff HEAD~1..HEAD
---
The user asked to continue after committing the paired piker EMS consumer
fix. The next isolated Tractor PR 490 item was to clean cancelled-task
diagnostics and define zero-buffer broadcast behavior, then review, test and
stop at a complete commit plan.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Made `BroadcastState.cancelled` transient: receiver progress and close clear
that receiver's diagnostic, terminal EOC and shared receive failure clear all
stale cancelled tasks, and durable EOC prevents peers from re-entering the
closed source. `broadcast_receiver()` now rejects non-positive retention
capacity before creating an unusable zero-length deque.
> `git diff HEAD~1..HEAD -- tractor/_streaming.py`
Made explicit `MsgStream.aclose()` release its internally allocated root
broadcaster while preserving graceful receive-internal EOC teardown. Used a
task-local marker so the public zero-argument `aclose()` signature and valid
subclass overrides remain compatible.
> `git diff HEAD~1..HEAD -- tractor/to_asyncio.py`
Made `LinkedTaskChannel.aclose()` release its internally allocated root
broadcaster before closing the underlying Trio receive channel.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added synchronized regressions for transient child cancellation diagnostics,
cross-receiver terminal cleanup, durable EOC peer wakeups, root broadcaster
cleanup through both public wrappers, `MsgStream.aclose()` subclass
compatibility, and zero-buffer rejection.
Verification output:
```text
........................... [100%]
27 passed in 5.89s
. [100%]
1 passed in 1.22s
. [100%]
1 passed in 0.88s
```
The integration runs cover real `MsgStream` actor fan-out and infected-asyncio
`LinkedTaskChannel` fan-out. Ruff, Python compilation and `git diff --check`
passed. Repeated adversarial review found and resolved root close re-entrancy,
cross-receiver terminal retention, durable-EOC and subclass-compatibility
issues; final review reported no findings.

View File

@ -0,0 +1,56 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: 76c5d31c-5a2f-4503-9b16-410ee7f4fab3
timestamp: 2026-08-19T18:46:40Z
git_ref: 481ba003
scope: code
substantive: true
raw_file: 20260819T184640Z_481ba003_prompt_io.raw.md
---
## Prompt
Rebase PR #484 onto final PR #481, migrate every affected one-shot call
to the new positional target API and continue through downstream tests,
examples and documentation review.
## Response summary
Converted stale target keyword calls to target partials so previously
named inputs remain explicit while placement/runtime controls stay
direct. Updated error expectations for local signature validation and
linked remote error propagation, then corrected docs which still
described the removed one-shot implementation. Linked spawning and
context lifecycle prose to the corresponding API methods and detailed
context guide.
## Files changed
- `docs/api/core.rst` - describe linked one-shot context execution.
- `docs/guide/rpc.rst` - update placement and target call semantics.
- `docs/guide/spawning.rst` - document positional target inputs.
- `examples/debugging/multi_nested_subactors_error_up_through_nurseries.py` - migrate nested actor target inputs.
- `examples/debugging/root_cancelled_but_child_is_in_tty_lock.py` - preserve named recursive target inputs with partials.
- `tests/test_advanced_streaming.py` - migrate streaming target inputs.
- `tests/test_cancellation.py` - migrate calls and tighten errors.
- `tests/test_infected_asyncio.py` - bind asyncio target options.
- `tests/test_rpc.py` - migrate RPC target argument binding.
- `tests/test_runtime.py` - preserve named runtime target inputs.
- `tests/test_spawning.py` - preserve named spawning target inputs.
## Human edits
The human selected the stack order and final PR #481 base, asked the
agent to continue after each diagnostic step and required a complete
commit plan after independently force-pushing the rebased history.
After reviewing the migration, the human required every formerly named
target input to remain visibly named through `functools.partial()`
rather than becoming positional. These were human-directed agent edits;
the human also required plain `start_actor()` and `open_context()`
references in the spawning and RPC guides to link to their API methods
and the detailed context guide, then clarified that `to_actor.run()`
already uses the full context API while `Portal.run()` should share
linked lifecycle machinery without necessarily delegating through
`Portal.open_context()` or adding a `Started` message. The human made
no direct source-line edits.

View File

@ -0,0 +1,30 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-19T18:46:40Z
git_ref: 481ba003
diff_cmd: git diff HEAD~1..HEAD
---
Migrate PR #484's downstream one-shot calls to PR #481's final
`tractor.to_actor.run()` contract after the stack rebase.
> `git diff HEAD~1..HEAD -- docs examples tests`
Pass target arguments positionally and bind target keyword-only inputs
with `functools.partial()`. Keep placement and runtime controls as
direct `to_actor.run()` keywords. Update the invalid-target-argument
test to expect local signature binding before actor startup and require
direct `RemoteActorError` propagation from linked one-shots.
Update API and guide prose to describe positional target inputs,
linked `Portal.open_context()` execution and per-child reaping instead
of the removed `Portal.run()` and target-`**kwargs` conventions.
Verification:
- core and migrated runtime batches: `97 passed`
- discovery and related lifecycle batch: `33 passed, 1 skipped`
- changed executable examples: `9 passed`
- mapped debugger cases: `12 passed, 6 skipped`
- Ruff, compilation and `git diff --check`: clean

View File

@ -62,8 +62,8 @@ One-shot task actors
``portal=`` it owns only the linked task and leaves the existing
actor's lifetime to the portal owner; that actor must expose both
the target module and ``tractor.to_actor.MODULE``. It supersedes
the legacy, non-blocking ``ActorNursery.run_in_actor()`` retained
only for compatibility until its removal in PR #484.
the removed (legacy, non-blocking)
``ActorNursery.run_in_actor()``.
.. deprecated:: 0.1.0a6

View File

@ -40,6 +40,9 @@ Broadcast fan-out
.. autoexception:: Lagged
:show-inheritance:
.. autoexception:: BroadcastReceiveError
:show-inheritance:
A single-producer, many-consumer broadcast layer over any
``trio``-style receive channel: non-lossy for the *fastest*
consumer while slower consumers raise :class:`Lagged` (a
@ -48,6 +51,13 @@ internal ring. This is exactly the machinery behind
:meth:`tractor.MsgStream.subscribe` — see
``examples/streaming_broadcast_fanout.py``.
If the shared underlying receiver raises an ordinary exception, the
subscriber which owned that receive gets the original failure.
Waiting peers drain their retained values and then raise
:class:`BroadcastReceiveError`, with the original failure available
as ``__cause__``. Later subscribers observe the same terminal state
without retrying the failed underlying receiver.
ExceptionGroup helpers
----------------------

View File

@ -209,6 +209,11 @@ The underlying broadcast machinery is lazily allocated on first
use and is *not* reversible for the channel's remaining lifetime,
so only reach for it when you actually want the fan-out.
As with ``MsgStream``, pass ``raise_on_lag=False`` for a consumer
which may warn, drop old values and resume from the retained window.
Each child chooses independently; the first subscription also fixes
the linked channel's root receive policy.
One-shot calls with ``run_task()``
----------------------------------
When you just want a single ``asyncio`` result and no streaming

View File

@ -171,6 +171,12 @@ keeps pace with the *fastest* subscriber; a task falling more
than the buffered window behind has its next receive raise
``tractor.trionics.Lagged`` to say it lost data.
Pass ``raise_on_lag=False`` when a consumer may drop old values and
resume from the oldest retained item instead. The receiver logs the
overrun rather than raising. Each child subscription chooses its own
policy; the first call also fixes the policy of the stream's root
receive handle because broadcaster allocation is irreversible.
The broadcast handle stays duplex btw: it proxies ``send()``
through to the underlying stream, so each subscriber task can
keep talking upstream while consuming its fan-out copy.

View File

@ -12,9 +12,9 @@ async def movie_theatre_question():
async def main():
"""The main ``tractor`` routine.
"""
async with tractor.open_nursery() as n:
async with tractor.open_nursery() as an:
portal = await n.start_actor(
portal = await an.start_actor(
'frank',
# enable the actor to run funcs from this current module
enable_modules=[__name__],

View File

@ -15,9 +15,9 @@ async def stream_forever() -> AsyncIterator[int]:
async def main():
async with tractor.open_nursery() as n:
async with tractor.open_nursery() as an:
portal = await n.start_actor(
portal = await an.start_actor(
'donny',
enable_modules=[__name__],
)

View File

@ -21,8 +21,8 @@ async def main() -> None:
async with tractor.open_nursery(
debug_mode=True,
) as n:
portal = await n.start_actor(
) as an:
portal = await an.start_actor(
'ctx_child',
# XXX: we don't enable the current module in order

View File

@ -6,14 +6,14 @@ async def die():
async def main():
async with tractor.open_nursery() as tn:
async with tractor.open_nursery() as an:
debug_actor = await tn.start_actor(
debug_actor = await an.start_actor(
'debugged_boi',
enable_modules=[__name__],
debug_mode=True,
)
crash_boi = await tn.start_actor(
crash_boi = await an.start_actor(
'crash_boi',
enable_modules=[__name__],
# debug_mode=True,

View File

@ -58,8 +58,8 @@ async def main():
debug_mode=True,
enable_transports=[tpt],
loglevel='devx',
) as n:
p = await n.start_actor(
) as an:
p = await an.start_actor(
'bp_boi',
enable_modules=[__name__],
)

View File

@ -50,8 +50,8 @@ async def trio_to_aio_echo_server(
async def main():
async with tractor.open_nursery() as n:
p = await n.start_actor(
async with tractor.open_nursery() as an:
p = await an.start_actor(
'aio_server',
enable_modules=[__name__],
infect_asyncio=True,

View File

@ -29,9 +29,9 @@ async def main() -> None:
))
await proc.wait()
# await trio.sleep_forever()
# async with tractor.open_nursery() as n:
# async with tractor.open_nursery() as an:
# portal = await n.start_actor(
# portal = await an.start_actor(
# 'rpc_server',
# enable_modules=[__name__],
# )

View File

@ -55,7 +55,7 @@ async def worker_pool(workers=4):
Yes, the workers stay alive (and ready for work) until you close
the context.
"""
async with tractor.open_nursery() as tn:
async with tractor.open_nursery() as an:
portals = []
snd_chan, recv_chan = trio.open_memory_channel(len(PRIMES))
@ -65,7 +65,7 @@ async def worker_pool(workers=4):
# this starts a new sub-actor (process + trio runtime) and
# stores it's "portal" for later use to "submit jobs" (ugh).
portals.append(
await tn.start_actor(
await an.start_actor(
f'worker_{i}',
enable_modules=[__name__],
)
@ -80,10 +80,10 @@ async def worker_pool(workers=4):
async def send_result(func, value, portal):
await snd_chan.send((value, await portal.run(func, n=value)))
async with trio.open_nursery() as n:
async with trio.open_nursery() as tn:
for value, portal in zip(sequence, itertools.cycle(portals)):
n.start_soon(
tn.start_soon(
send_result,
worker_func,
value,
@ -98,7 +98,7 @@ async def worker_pool(workers=4):
yield _map
# tear down all "workers" on pool close
await tn.cancel()
await an.cancel()
async def main():

View File

@ -31,9 +31,9 @@ async def simple_rpc(
async def main() -> None:
async with tractor.open_nursery() as n:
async with tractor.open_nursery() as an:
portal = await n.start_actor(
portal = await an.start_actor(
'rpc_server',
enable_modules=[__name__],
)

View File

@ -0,0 +1,5 @@
Remove legacy ``ActorNursery.run_in_actor()``,
``Portal.wait_for_result()`` and ``Portal.result()``. Use
``tractor.to_actor.run()`` for caller-owned one-shot tasks,
``Portal.run()`` for daemon RPC results or ``Portal.open_context()``
for linked task dialogs.

View File

@ -46,9 +46,9 @@ async def test_reg_then_unreg(
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as n:
) as an:
portal = await n.start_actor('actor', enable_modules=[__name__])
portal = await an.start_actor('actor', enable_modules=[__name__])
uid = portal.channel.aid.uid
async with tractor.get_registry(reg_addr) as aportal:
@ -62,7 +62,7 @@ async def test_reg_then_unreg(
# XXX: can we figure out what the listen addr will be?
assert sockaddrs
await n.cancel() # tear down nursery
await an.cancel() # tear down nursery
await trio.sleep(0.1)
assert uid not in aportal.actor._registry
@ -89,9 +89,9 @@ async def test_reg_then_unreg_maddr(
async with tractor.open_nursery(
registry_addrs=[maddr_str],
) as n:
) as an:
portal = await n.start_actor(
portal = await an.start_actor(
'actor_maddr',
enable_modules=[__name__],
)
@ -105,7 +105,7 @@ async def test_reg_then_unreg_maddr(
sockaddrs = actor._registry[uid]
assert sockaddrs
await n.cancel()
await an.cancel()
await trio.sleep(0.1)
assert uid not in aportal.actor._registry
@ -152,23 +152,37 @@ async def test_trynamic_trio(
for the directed subs.
'''
async with tractor.open_nursery() as n:
async with tractor.open_nursery() as an:
print("Alright... Action!")
donny = await n.run_in_actor(
ria_fn,
other_actor='gretchen',
reg_addr=reg_addr,
name='donny',
# donny + gretchen each wait on (then dial!) the *other*, so
# both actors must OUTLIVE both hellos: spawn as daemons and
# only reap after both tasks complete. NB a pair of eagerly
# reaped `to_actor.run()` one-shots races: the first to
# finish dies while the other may still be dialing its
# registry-resolved (now dead) sockaddr -> conn-refused.
portals: dict[str, tractor.Portal] = {
name: await an.start_actor(
name,
enable_modules=[__name__],
)
gretchen = await n.run_in_actor(
for name in ('donny', 'gretchen')
}
async def _direct(this_name: str, other_actor: str):
res = await portals[this_name].run(
ria_fn,
other_actor='donny',
other_actor=other_actor,
reg_addr=reg_addr,
name='gretchen',
)
print(await gretchen.result())
print(await donny.result())
print(res)
async with trio.open_nursery() as tn:
tn.start_soon(_direct, 'donny', 'gretchen')
tn.start_soon(_direct, 'gretchen', 'donny')
# both hellos have completed; reap the thespians.
await an.cancel()
print("CUTTTT CUUTT CUT!!?! Donny!! You're supposed to say...")
@ -270,13 +284,15 @@ async def spawn_and_check_registry(
portals = {}
for i in range(3):
name = f'a{i}'
if with_streaming:
# a daemon subactor is alive + registered
# without a "main" task; the streaming
# branch below uses the module funcs, the
# non-streaming case just needs it up (was
# `run_in_actor(trio.sleep_forever)`).
portals[name] = await an.start_actor(
name=name, enable_modules=[__name__])
else: # no streaming
portals[name] = await an.run_in_actor(
trio.sleep_forever, name=name)
name=name,
enable_modules=[__name__],
)
# wait on last actor to come up
async with tractor.wait_for_actor(name):

View File

@ -3,6 +3,7 @@ Advanced streaming patterns using bidirectional streams and contexts.
'''
from collections import Counter
from functools import partial
import itertools
import platform
from typing import Type
@ -24,6 +25,7 @@ _registry: dict[str, set[tractor.MsgStream]] = {
'even': set(),
'odd': set(),
}
_publisher_started: bool = False
async def publisher(
@ -32,11 +34,13 @@ async def publisher(
) -> None:
global _registry
global _publisher_started, _registry
def is_even(i):
return i % 2 == 0
_publisher_started = True
try:
for val in itertools.count(seed):
sub = 'even' if is_even(val) else 'odd'
@ -48,6 +52,26 @@ async def publisher(
# making it readable to a human user
await trio.sleep(1/1000)
finally:
_publisher_started = False
async def pubsub_active(
expected_subs: int,
) -> bool:
'''
Report whether the publisher and all subscriber tasks are active.
Runs as an RPC task in the publisher actor, where `_registry` is
mutated by each `subscribe()` context after its consumer sends the
first subscription.
'''
return (
_publisher_started
and sum(map(len, _registry.values())) >= expected_subs
)
@tractor.context
async def subscribe(
@ -173,8 +197,8 @@ def test_dynamic_pub_sub(
# test. Picked backend-aware: under `trio` backend spawn is
# cheap (~1s for `cpus` actors) but fork-based backends pay
# a per-spawn cost (forkserver round-trip + IPC peer-handshake)
# that can stack up over `cpus - 1` sequential `n.run_in_actor()`
# calls — especially on UDS under cross-pytest contention
# that can stack up over the `cpus - 1` one-shot
# (`to_actor.run()`) spawns — especially on UDS under cross-pytest contention
# (#451 / #452). 4s was flaking right at the edge under fork
# backends — bumped to 8s with diag-snapshot-on-timeout via
# `fail_after_w_trace` so a borderline run still fails loud
@ -214,37 +238,71 @@ def test_dynamic_pub_sub(
f'enter `fail_after_w_trace({fail_after_s})` scope'
)
try:
async with tractor.open_nursery(
async with (
tractor.open_nursery(
registry_addrs=[reg_addr],
debug_mode=debug_mode,
) as n:
) as an,
# bg-schedules the forever-streaming
# one-shots below; the user-cancel raise
# cancels them all, each reaping its
# subactor via `to_actor.run()`'s
# (shielded) `Portal.cancel_actor()`.
trio.open_nursery() as tn,
):
test_log.cancel(
'test_dynamic_pub_sub: '
'actor nursery opened'
)
# name of this actor will be same as target func
await n.run_in_actor(publisher)
tn.start_soon(
partial(
tractor.to_actor.run,
publisher,
an=an,
)
)
for i, sub in zip(
range(cpus - 2),
itertools.cycle(_registry.keys())
):
await n.run_in_actor(
tn.start_soon(
partial(
tractor.to_actor.run,
partial(
consumer,
name=f'consumer_{sub}',
subs=[sub],
),
an=an,
name=f'consumer_{sub}',
)
)
# make one dynamic subscriber
await n.run_in_actor(
tn.start_soon(
partial(
tractor.to_actor.run,
partial(
consumer,
name='consumer_dynamic',
subs=list(_registry.keys()),
),
an=an,
name='consumer_dynamic',
)
)
# block until "cancelled by user"
await trio.sleep(3)
expected_subs: int = max(cpus - 2, 0) + 1
async with tractor.wait_for_actor(
'publisher',
) as portal:
while not await portal.run(
pubsub_active,
expected_subs=expected_subs,
):
await trio.sleep(0.01)
test_log.warning(
f'Raising user cancel exc: '
f'{expect_cancel_exc!r}'
@ -347,10 +405,10 @@ def test_reqresp_ontopof_streaming():
timeout = 4
with trio.move_on_after(timeout):
async with tractor.open_nursery() as n:
async with tractor.open_nursery() as an:
# name of this actor will be same as target func
portal = await n.start_actor(
portal = await an.start_actor(
'dual_tasks',
enable_modules=[__name__]
)
@ -413,9 +471,9 @@ def test_sigint_both_stream_types():
async def main():
with trio.fail_after(timeout):
async with tractor.open_nursery() as n:
async with tractor.open_nursery() as an:
# name of this actor will be same as target func
portal = await n.start_actor(
portal = await an.start_actor(
'2_way',
enable_modules=[__name__]
)
@ -528,8 +586,8 @@ def test_local_task_fanout_from_stream(
async with tractor.open_nursery(
debug_mode=debug_mode,
) as tn:
p: tractor.Portal = await tn.start_actor(
) as an:
p: tractor.Portal = await an.start_actor(
'inf_streamer',
enable_modules=[__name__],
)

View File

@ -2,6 +2,7 @@
Cancellation and error propagation
"""
from functools import partial
import os
import signal
import platform
@ -16,6 +17,10 @@ from tractor._testing import (
tractor_test,
)
from tractor._testing.trace import FailAfterWTraceFactory
from tractor.trionics import (
collapse_eg,
gather_contexts,
)
from .conftest import no_windows
@ -68,10 +73,39 @@ async def assert_err(delay=0):
assert 0
@tractor.context
async def assert_err_ctx(
ctx: tractor.Context,
delay: float = 0,
) -> None:
'''
`@context` shim around `assert_err()` so the multi-actor error
tests can fan-out one-shot erroring subactors via
`Portal.open_context()` + `gather_contexts()` instead of the
removed `ActorNursery.run_in_actor()` (#477).
'''
await ctx.started()
await trio.sleep(delay)
assert 0
async def sleep_forever():
await trio.sleep_forever()
@tractor.context
async def sleep_forever_ctx(
ctx: tractor.Context,
) -> None:
'''
Signal task startup before sleeping until context cancellation.
'''
await ctx.started()
await sleep_forever()
async def do_nuthin():
# just nick the scheduler
await trio.sleep(0)
@ -82,7 +116,7 @@ async def do_nuthin():
[
# expected to be thrown in assert_err
({}, AssertionError),
# argument mismatch raised in _invoke()
# argument mismatch rejected locally before spawn
({'unexpected': 10}, TypeError)
],
ids=['no_args', 'unexpected_args'],
@ -104,57 +138,38 @@ def test_remote_error(
async def main():
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as nursery:
) as an:
# on a remote type error caused by bad input args
# this should raise directly which means we **don't** get
# an exception group outside the nursery since the error
# here and the far end task error are one in the same?
portal = await nursery.run_in_actor(
assert_err,
name='errorer',
**args
)
# get result(s) from main task
# `to_actor.run()` blocks on the one-shot's result and
# raises the remote error directly here in the caller's
# task. Invalid target args fail local signature binding
# before any one-shot actor is spawned.
try:
# this means the root actor will also raise a local
# parent task error and thus an eg will propagate out
# of this actor nursery.
await portal.result()
await tractor.to_actor.run(
partial(assert_err, **args),
an=an,
name='errorer',
)
except tractor.RemoteActorError as err:
assert err.boxed_type == errtype
print("Look Maa that actor failed hard, hehh")
raise
# ensure boxed errors
# Invalid args never cross the process boundary.
if args:
with pytest.raises(tractor.RemoteActorError) as excinfo:
with pytest.raises(errtype):
trio.run(main)
else:
# The linked one-shot raises the child's boxed error
# directly in this caller task.
with pytest.raises(
tractor.RemoteActorError,
) as excinfo:
trio.run(main)
assert excinfo.value.boxed_type == errtype
else:
# the root task will also error on the `Portal.result()`
# call so we expect an error from there AND the child.
# |_ tho seems like on new `trio` this doesn't always
# happen?
with pytest.raises((
BaseExceptionGroup,
tractor.RemoteActorError,
)) as excinfo:
trio.run(main)
# ensure boxed errors are `errtype`
err: BaseException = excinfo.value
if isinstance(err, BaseExceptionGroup):
suberrs: list[BaseException] = err.exceptions
else:
suberrs: list[BaseException] = [err]
for exc in suberrs:
assert exc.boxed_type == errtype
def test_multierror(
reg_addr: tuple[str, int],
@ -162,113 +177,202 @@ def test_multierror(
set_fork_aware_capture, #: Callable,
):
'''
Verify we raise a ``BaseExceptionGroup`` out of a nursery where
more then one actor errors.
Verify concurrent one-shot subactors erroring propagate a remote
error out of the `gather_contexts()` fan-out grouped as a
`BaseExceptionGroup`, or (under cancel-on-first, where the 2nd
errorer is cancelled before relaying its own exc) collapsed to a
single `RemoteActorError`.
NB the legacy `run_in_actor()` reaped *all* children at nursery
teardown so this always yielded a BEG-of-N; the `to_actor`
fan-out is cancel-on-first, so accept either shape.
'''
async def main():
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as nursery:
) as an:
await nursery.run_in_actor(assert_err, name='errorer1')
portal2 = await nursery.run_in_actor(assert_err, name='errorer2')
portals = [
await an.start_actor(
f'errorer{i}',
enable_modules=[__name__],
)
for i in range(2)
]
# get result(s) from main task
try:
await portal2.result()
except tractor.RemoteActorError as err:
assert err.boxed_type is AssertionError
print("Look Maa that first actor failed hard, hehh")
raise
# both one-shot subactors error concurrently, so the
# `gather_contexts()` task-nursery collects them into a
# `BaseExceptionGroup` (was two non-blocking
# `run_in_actor()`s reaped at nursery teardown).
async with gather_contexts(
mngrs=[
p.open_context(assert_err_ctx)
for p in portals
],
):
pass
# here we should get a ``BaseExceptionGroup`` containing exceptions
# from both subactors
with pytest.raises(BaseExceptionGroup):
with pytest.raises((
BaseExceptionGroup,
tractor.RemoteActorError,
)) as excinfo:
trio.run(main)
exc = excinfo.value
if isinstance(exc, tractor.RemoteActorError):
assert exc.boxed_type is AssertionError
return
def iter_group_leaves(
group: BaseExceptionGroup,
):
for subexc in group.exceptions:
if isinstance(subexc, BaseExceptionGroup):
yield from iter_group_leaves(subexc)
else:
yield subexc
assertion_errors: list[tractor.RemoteActorError] = []
cancellations: list[BaseException] = []
for leaf in iter_group_leaves(exc):
if isinstance(leaf, tractor.ContextCancelled):
cancellations.append(leaf)
elif isinstance(leaf, trio.Cancelled):
cancellations.append(leaf)
else:
assert isinstance(leaf, tractor.RemoteActorError)
assert leaf.boxed_type is AssertionError
assertion_errors.append(leaf)
assert len(assertion_errors) in (1, 2)
if not cancellations:
assert len(assertion_errors) == 2
@pytest.mark.parametrize(
'delay',
(0, 0.5),
ids='delays={}'.format,
'errorer_count',
(1, 5, 25),
ids='errorers={}'.format,
)
@pytest.mark.parametrize(
'num_subactors',
range(25, 26),
ids= 'num_subs={}'.format,
)
def test_multierror_fast_nursery(
reg_addr: tuple,
def test_concurrent_start_error_reaps_all(
reg_addr: tuple[str, int],
start_method: str,
num_subactors: int,
delay: float,
errorer_count: int,
set_fork_aware_capture,
fail_after_w_trace: FailAfterWTraceFactory,
):
'''
Verify we raise a ``BaseExceptionGroup`` out of a nursery where
more then one actor errors and also with a delay before failure
to test failure during an ongoing spawning.
Reap high-fan-out children cancelled during concurrent startup.
The removed `test_multierror_fast_nursery()` launched 25 legacy
one-shots while earlier children could already be failing. Its
exact 25-error group depended on deferred nursery-exit result
collection, but the startup/cancellation load remains valuable.
This replacement schedules 25 blocking `to_actor.run()` calls and
holds all local caller tasks before any actor process starts.
Releasing one barrier makes the flat pool race through
`to_actor.run()` together; that API implicitly calls
`ActorNursery.start_actor()` for each child. Backend and Trio
scheduling then vary which children are spawning, handshaking or
running when the first remote error arrives.
Parameterizing one, five and all 25 errorers covers sparse through
saturated failure. Unlike the old nursery-owned deferred result
collection, the local Trio task nursery is the OCA supervisor: its
first remote error cancels sibling callers, and each
`to_actor.run()` shield-reaps any child process it accepted.
Bounded completion, only boxed assertion relays and empty
child/reap maps prove the entire flat pool remained supervised.
'''
async def main():
# budget = 2× natural trio-backend cascade time for
# 25 errorer subactors (~14s observed). on-timeout
# diag snapshot → if the cancel cascade hangs
# (observed under MTF backend with N>=14 errorer
# subactors) we get a fresh ptree/wchan/py-spy dump
# on disk INSTEAD of an opaque pytest timeout-kill.
# See `tractor/_testing/trace.py` for the helper.
async with fail_after_w_trace(30.0):
child_count: int = 25
async def main() -> None:
callers_ready: int = 0
all_callers_ready = trio.Event()
async with fail_after_w_trace(40):
with pytest.raises((
BaseExceptionGroup,
tractor.RemoteActorError,
)) as excinfo:
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as nursery:
) as an:
async def run_child(
i: int,
) -> None:
nonlocal callers_ready
for i in range(num_subactors):
await nursery.run_in_actor(
assert_err,
name=f'errorer{i}',
delay=delay
# Hold every local caller before ANY actor
# process starts. Arrival publication through
# `.set()` has no checkpoint; releasing the
# barrier maximizes scheduler/backend variation
# across the implicit `start_actor()` calls.
callers_ready += 1
if callers_ready == child_count:
all_callers_ready.set()
await all_callers_ready.wait()
is_errorer: bool = (
i >= child_count - errorer_count
)
fn = (
assert_err
if is_errorer
else sleep_forever
)
name = (
f'errorer_{i}'
if is_errorer
else f'waiter_{i}'
)
await tractor.to_actor.run(
fn,
an=an,
name=name,
)
# with pytest.raises(trio.MultiError) as exc_info:
# NOTE, `trio.TooSlowError` from `fail_after_w_trace`
# bubbles UN-wrapped if `open_nursery.__aexit__` never
# gets re-entered; wrapped inside a `BaseExceptionGroup`
# if it did. Accept both shapes so the matcher itself
# doesn't lie about *what* failed.
with pytest.raises(
(BaseExceptionGroup, trio.TooSlowError),
) as exc_info:
async with trio.open_nursery() as tn:
for i in range(child_count):
tn.start_soon(
run_child,
i,
)
assert callers_ready == child_count
assert all_callers_ready.is_set()
assert not an._children
assert not an._child_reap_requests
assert not an._child_reaped
def iter_leaves(
exc: BaseException,
):
if isinstance(exc, BaseExceptionGroup):
for subexc in exc.exceptions:
yield from iter_leaves(subexc)
else:
yield exc
assertion_errors: list[tractor.RemoteActorError] = []
for leaf in iter_leaves(excinfo.value):
if isinstance(leaf, (
trio.Cancelled,
tractor.ContextCancelled,
)):
continue
assert isinstance(leaf, tractor.RemoteActorError)
assert leaf.boxed_type is AssertionError
assertion_errors.append(leaf)
assert 1 <= len(assertion_errors) <= errorer_count
trio.run(main)
if isinstance(exc_info.value, trio.TooSlowError):
pytest.fail(
f'cancel cascade hung past 12s '
f'(num_subactors={num_subactors}, delay={delay}); '
f'see stderr for `fail_after_w_trace` snapshot path'
)
assert exc_info.type == ExceptionGroup
err = exc_info.value
exceptions = err.exceptions
if len(exceptions) == 2:
# sometimes oddly now there's an embedded BrokenResourceError ?
for exc in exceptions:
excs = getattr(exc, 'exceptions', None)
if excs:
exceptions = excs
break
assert len(exceptions) == num_subactors
for exc in exceptions:
assert isinstance(exc, tractor.RemoteActorError)
assert exc.boxed_type is AssertionError
async def do_nothing():
pass
@ -296,16 +400,16 @@ def test_cancel_single_subactor(
'''
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as nursery:
) as an:
portal = await nursery.start_actor(
portal = await an.start_actor(
'nothin', enable_modules=[__name__],
)
assert (await portal.run(do_nothing)) is None
if mechanism == 'nursery_cancel':
# would hang otherwise
await nursery.cancel()
await an.cancel()
else:
raise mechanism
@ -337,8 +441,8 @@ async def test_cancel_infinite_streamer(
trio.fail_after(4),
trio.move_on_after(1) as cancel_scope
):
async with tractor.open_nursery() as n:
portal = await n.start_actor(
async with tractor.open_nursery() as an:
portal = await an.start_actor(
'donny',
enable_modules=[__name__],
)
@ -351,36 +455,36 @@ async def test_cancel_infinite_streamer(
# we support trio's cancellation system
assert cancel_scope.cancelled_caught
assert n.cancel_called
assert an.cancel_called
@pytest.mark.parametrize(
'num_actors_and_errs',
[
# daemon actors sit idle while single task actors error out
# daemon actors sit idle while one-shot task actors error out
(1, tractor.RemoteActorError, AssertionError, (assert_err, {}), None),
(2, BaseExceptionGroup, AssertionError, (assert_err, {}), None),
(3, BaseExceptionGroup, AssertionError, (assert_err, {}), None),
# 1 daemon actor errors out while single task actors sleep forever
# 1 daemon actor errors out while one-shot task actors sleep forever
(3, tractor.RemoteActorError, AssertionError, (sleep_forever, {}),
(assert_err, {}, True)),
# daemon actors error out after brief delay while single task
# daemon actors error out after brief delay while one-shot task
# actors complete quickly
(3, tractor.RemoteActorError, AssertionError,
(do_nuthin, {}), (assert_err, {'delay': 1}, True)),
# daemon complete quickly delay while single task
# daemon complete quickly delay while one-shot task
# actors error after brief delay
(3, BaseExceptionGroup, AssertionError,
(assert_err, {'delay': 1}), (do_nuthin, {}, False)),
],
ids=[
'1_run_in_actor_fails',
'2_run_in_actors_fail',
'3_run_in_actors_fail',
'1_one_shot_fails',
'2_one_shots_fail',
'3_one_shots_fail',
'1_daemon_actors_fail',
'1_daemon_actors_fail_all_run_in_actors_dun_quick',
'no_daemon_actors_fail_all_run_in_actors_sleep_then_fail',
'1_daemon_actors_fail_all_one_shots_dun_quick',
'no_daemon_actors_fail_all_one_shots_sleep_then_fail',
],
)
@tractor_test(
@ -399,12 +503,34 @@ async def test_some_cancels_all(
This is the first and only supervisory strategy at the moment.
One-shot subactors run as concurrent `to_actor.run()` tasks
in a local task-nursery so their errors raise WHILE the
actor-nursery block is still open (vs the legacy
`run_in_actor()` teardown-reap); the first error cancels the
sibling one-shots (whose `trio.Cancelled`s the task-nursery
absorbs) so the group shape is 1..num_actors
`RemoteActorError`s depending on relay-vs-cancel timing
with `collapse_eg()` unwrapping the deterministic
single-error cases to a bare `RemoteActorError`.
Supervision and error flow (`RAE` is `RemoteActorError`):
root actor task
`-- ActorNursery an (owns child processes)
+-- daemon actor_i
| `-- Portal.run() -------- RAE --+
+-- one-shot actor_i |
| `-- remote target -- RAE --> run() caller
`-- local trio.Nursery tn <----------+
`-- first RAE cancels sibling callers
`-- escapes -> an cancels/reaps all children
'''
(
num_actors,
first_err,
err_type,
ria_func,
one_shot_func,
da_func,
) = num_actors_and_errs
try:
@ -418,29 +544,35 @@ async def test_some_cancels_all(
enable_modules=[__name__],
))
func, kwargs = ria_func
riactor_portals = []
func, kwargs = one_shot_func
async with (
collapse_eg(),
trio.open_nursery() as tn,
):
for i in range(num_actors):
# start actor(s) that will fail immediately
riactor_portals.append(
await an.run_in_actor(
func,
# schedule one-shot task actor(s); errors
# raise into this task-nursery scope.
tn.start_soon(
partial(
tractor.to_actor.run,
partial(func, **kwargs),
an=an,
name=f'actor_{i}',
**kwargs
)
)
if da_func:
func, kwargs, expect_error = da_func
for portal in dactor_portals:
# if this function fails then we should error here
# and the nursery should teardown all other actors
# if this function fails then we should error
# here and the nursery should teardown all
# other actors
try:
await portal.run(func, **kwargs)
except tractor.RemoteActorError as err:
assert err.boxed_type == err_type
# we only expect this first error to propogate
# we only expect this first error to propagate
# (all other daemons are cancelled before they
# can be scheduled)
num_actors = 1
@ -449,20 +581,26 @@ async def test_some_cancels_all(
else:
if expect_error:
pytest.fail(
"Deamon call should fail at checkpoint?")
"Daemon call should fail at checkpoint?")
# should error here with a ``RemoteActorError`` or ``MultiError``
# should error here with a `RemoteActorError` or a beg of them
except first_err as _err:
except (
BaseExceptionGroup,
tractor.RemoteActorError,
) as _err:
err = _err
if isinstance(err, BaseExceptionGroup):
assert len(err.exceptions) == num_actors
# only the concurrent multi-error cases can group; the
# relay-vs-cancel race means anywhere from 1 (all
# siblings cancelled before relaying) up to all
# `num_actors` errors may populate the group.
assert first_err is BaseExceptionGroup
assert 1 <= len(err.exceptions) <= num_actors
for exc in err.exceptions:
if isinstance(exc, tractor.RemoteActorError):
assert isinstance(exc, tractor.RemoteActorError)
assert exc.boxed_type == err_type
else:
assert isinstance(exc, trio.Cancelled)
elif isinstance(err, tractor.RemoteActorError):
assert err.boxed_type == err_type
assert an.cancel_called is True
@ -475,19 +613,33 @@ async def spawn_and_error(
breadth: int,
depth: int,
) -> None:
'''
Recursively spawn a breadth-wide level of erroring one-shot
subactors as concurrent `to_actor.run()` tasks; the leaf level
errors ~simultaneously and each level's task-nursery groups
whatever `RemoteActorError`s relay before the first one's
cancel wins, boxing the (`ExceptionGroup`-shaped) group into
this actor's own relayed error.
'''
name = tractor.current_actor().name
async with tractor.open_nursery() as nursery:
async with (
tractor.open_nursery() as an,
trio.open_nursery() as tn,
):
for i in range(breadth):
if depth > 0:
args = (
partial(
spawn_and_error,
breadth=breadth,
depth=depth - 1,
),
)
kwargs = {
'name': f'spawner_{i}_depth_{depth}',
'breadth': breadth,
'depth': depth - 1,
}
else:
args = (
@ -496,7 +648,14 @@ async def spawn_and_error(
kwargs = {
'name': f'{name}_errorer_{i}',
}
await nursery.run_in_actor(*args, **kwargs)
tn.start_soon(
partial(
tractor.to_actor.run,
*args,
an=an,
**kwargs,
)
)
# NOTE: `main_thread_forkserver` capture-fd hang class is no
@ -538,7 +697,11 @@ async def test_nested_multierrors(
depth: int,
):
'''
Test that failed actor sets are wrapped in `BaseExceptionGroup`s.
Test that a nested tree of concurrently failing one-shot
subactors tears down cleanly, relaying (whatever subset of)
the leaf `AssertionError`s (that win the per-level
relay-vs-cancel race) re-boxed/grouped at each actor
boundary.
Parametrized over recursion `depth {1, 3}`:
@ -588,6 +751,13 @@ async def test_nested_multierrors(
# fork-spawn jitter + UDS-contention widens both `t1` and
# `t2` further.
#
# NB post-#477 (`to_actor.run()` fan-out in a local
# task-nursery) a race-tripped sibling's `Cancelled` is
# ABSORBED by the task-nursery instead of landing in the
# group — the raced case now shows as a *smaller* BEG, so
# this marker should consistently `xpass`; drop it once CI
# confirms.
#
# With `strict=False` the clean-cascade cases (most
# depth=1 runs, rare depth=3 runs) report as `xpassed`
# while the race-tripped cases report as `xfailed` —
@ -672,6 +842,14 @@ async def test_nested_multierrors(
timeout = 16
case ('main_thread_forkserver', 3):
timeout = 30
# any other fork-based backend (`mp_spawn` et al) pays
# the same per-spawn round-trip costs as MTF so rides
# its budgets; without a default arm `timeout` is left
# unbound -> `UnboundLocalError` at the scaling below.
case (_, 1):
timeout = 16
case (_, 3):
timeout = 30
# inflate the budget by the throttle headroom probed above so
# a slow box doesn't masquerade as a deadline regression.
@ -684,66 +862,83 @@ async def test_nested_multierrors(
async with fail_after_w_trace(timeout):
try:
async with tractor.open_nursery() as nursery:
async with (
tractor.open_nursery() as an,
trio.open_nursery() as tn,
):
for i in range(subactor_breadth):
await nursery.run_in_actor(
tn.start_soon(
partial(
tractor.to_actor.run,
partial(
spawn_and_error,
name=f'spawner_{i}',
breadth=subactor_breadth,
depth=depth,
),
an=an,
name=f'spawner_{i}',
)
except BaseExceptionGroup as err:
assert len(err.exceptions) == subactor_breadth
for subexc in err.exceptions:
# verify first level actor errors are wrapped as remote
if _friggin_windows:
)
except (
BaseExceptionGroup,
tractor.RemoteActorError,
) as err:
# group membership is bounded by the relay-vs-cancel
# race: the first spawner-tree's error cancels its
# siblings, whose own errors only group when relayed
# first; a fully-raced tree even collapses (via the
# runtime's own `collapse_eg()` unwrapping each level's
# single-member group) to a bare `RemoteActorError`
# re-boxing the leaf `AssertionError` at every actor
# boundary. The deterministic exact-breadth nested-BEG
# was the legacy `run_in_actor()` reap-all-at-teardown.
subexcs: list[BaseException] = (
err.exceptions
if isinstance(err, BaseExceptionGroup)
else [err]
)
assert 1 <= len(subexcs) <= subactor_breadth
for subexc in subexcs:
if (
_friggin_windows
and
isinstance(subexc, trio.Cancelled)
):
# windows is often too slow and cancellation seems
# to happen before an actor is spawned
if isinstance(subexc, trio.Cancelled):
continue
elif isinstance(subexc, tractor.RemoteActorError):
# on windows it seems we can't exactly be sure wtf
# will happen..
assert subexc.boxed_type in (
tractor.RemoteActorError,
trio.Cancelled,
BaseExceptionGroup,
)
elif isinstance(subexc, BaseExceptionGroup):
for subsub in subexc.exceptions:
if subsub in (tractor.RemoteActorError,):
subsub = subsub.boxed_type
assert type(subsub) in (
trio.Cancelled,
BaseExceptionGroup,
)
else:
assert isinstance(subexc, tractor.RemoteActorError)
if depth > 0 and subactor_breadth > 1:
# XXX not sure what's up with this..
# on windows sometimes spawning is just too slow and
# we get back the (sent) cancel signal instead
if _friggin_windows:
if isinstance(subexc, tractor.RemoteActorError):
assert subexc.boxed_type in (
BaseExceptionGroup,
tractor.RemoteActorError
)
else:
assert isinstance(subexc, BaseExceptionGroup)
else:
assert subexc.boxed_type is ExceptionGroup
else:
assert subexc.boxed_type in (
accepted: tuple[Type[BaseException], ...] = (
# ≥2 sub-tree errors relayed before the
# cancel-cascade won → grouped per-level.
ExceptionGroup,
# every level collapsed down to its lone
# relayed (leaf) error.
AssertionError,
# a mid-level spawner relays an
# already-boxed (collapsed) leaf chain,
# re-boxing the `RemoteActorError` itself.
tractor.RemoteActorError,
trio.Cancelled
# under heavy load a runtime-internal reap
# deadline can inject a `trio.Cancelled`
# into a child's group before relay (the
# same class the depth=3 throttle-xfail
# covers) upgrading it from an
# `ExceptionGroup`.
BaseExceptionGroup,
)
if _friggin_windows:
# on windows it seems we can't exactly be
# sure wtf will happen..
accepted += (
trio.Cancelled,
)
assert subexc.boxed_type in accepted
else:
pytest.fail(
'Should have raised a (grouped) `RemoteActorError`?'
)
@ -764,8 +959,8 @@ def test_cancel_via_SIGINT(
with trio.fail_after(2):
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as tn:
await tn.start_actor('sucka')
) as an:
await an.start_actor('sucka')
if 'mp' in start_method:
time.sleep(0.1)
os.kill(pid, signal.SIGINT)
@ -809,12 +1004,24 @@ def test_cancel_via_SIGINT_other_task(
):
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as tn:
for i in range(3):
await tn.run_in_actor(
sleep_forever,
name='namesucka',
) as an:
portals = [
await an.start_actor(
f'namesucka_{i}',
enable_modules=[__name__],
)
for i in range(3)
]
# Keep one linked RPC task active in every daemon before
# reporting startup, preserving the original
# `run_in_actor(sleep_forever)` cancellation target.
async with gather_contexts(
mngrs=[
portal.open_context(sleep_forever_ctx)
for portal in portals
],
):
task_status.started()
await trio.sleep_forever()
@ -854,8 +1061,11 @@ async def spin_for(period=3):
async def spawn_sub_with_sync_blocking_task():
async with tractor.open_nursery() as an:
print('starting sync blocking subactor..\n')
await an.run_in_actor(
# one-shot: parks HERE awaiting the sync-sleeping
# grandchild's result until cancelled from above.
await tractor.to_actor.run(
spin_for,
an=an,
name='sleeper',
)
print('exiting first subactor layer..\n')
@ -961,11 +1171,19 @@ def test_cancel_while_childs_child_in_sync_sleep(
debug_mode=debug_mode,
registry_addrs=[reg_addr],
) as an,
trio.open_nursery() as tn,
):
await an.run_in_actor(
# bg one-shot: parks on the middle actor's result
# (itself parked on the sync-sleeping grandchild)
# until the `assert 0` below cancels this scope.
tn.start_soon(
partial(
tractor.to_actor.run,
spawn_sub_with_sync_blocking_task,
an=an,
name='sync_blocking_sub',
)
)
await trio.sleep(1)
if man_cancel_outer:
@ -1013,8 +1231,8 @@ def test_fast_graceful_cancel_when_spawn_task_in_soft_proc_wait_for_daemon(
start = time.time()
try:
async with trio.open_nursery() as nurse:
async with tractor.open_nursery() as tn:
p = await tn.start_actor(
async with tractor.open_nursery() as an:
p = await an.start_actor(
'fast_boi',
enable_modules=[__name__],
)

View File

@ -156,8 +156,8 @@ def test_actor_managed_trio_nursery_task_error_cancels_aio(
async def main():
# cancel the nursery shortly after boot
async with tractor.open_nursery() as n:
p = await n.start_actor(
async with tractor.open_nursery() as an:
p = await an.start_actor(
'nursery_mngr',
infect_asyncio=asyncio_mode, # TODO, is this enabling debug mode?
enable_modules=[__name__],

View File

@ -5,7 +5,7 @@ The hipster way to force SC onto the stdlib's "async": 'infection mode'.
import asyncio
import builtins
from contextlib import ExitStack
# from functools import partial
from functools import partial
import itertools
import importlib
import os
@ -36,6 +36,7 @@ from tractor import (
current_actor,
Actor,
to_asyncio,
to_actor,
RemoteActorError,
ContextCancelled,
)
@ -122,8 +123,9 @@ def test_trio_cancels_aio_on_actor_side(
registry_addrs=[reg_addr],
debug_mode=debug_mode,
) as an:
await an.run_in_actor(
await to_actor.run(
trio_cancels_single_aio_task,
an=an,
infect_asyncio=True,
)
@ -169,6 +171,28 @@ async def asyncio_actor(
raise
@tractor.context
async def sleep_forever_aio_ctx(
ctx: tractor.Context,
expect_err: str = 'trio.Cancelled',
) -> None:
'''
`@context` shim so a parent can spawn a forever-sleeping
infected-`asyncio` task via `Portal.open_context()` and cancel it
(via `Portal.cancel_actor()` or an enclosing `trio` cancel scope),
asserting the graceful `trio.Cancelled` teardown.
Replaces the legacy `ActorNursery.run_in_actor()` spawn the
aio-cancel tests below used to rely on (removed with #477).
'''
await ctx.started()
await asyncio_actor(
target='aio_sleep_forever',
expect_err=expect_err,
)
def test_aio_simple_error(
reg_addr: tuple[str, int],
debug_mode: bool,
@ -184,10 +208,13 @@ def test_aio_simple_error(
registry_addrs=[reg_addr],
debug_mode=debug_mode,
) as an:
await an.run_in_actor(
await to_actor.run(
partial(
asyncio_actor,
target='sleep_and_err',
expect_err='AssertionError',
),
an=an,
infect_asyncio=True,
)
@ -219,18 +246,36 @@ def test_tractor_cancels_aio(
'''
async def main():
# anti-hang wall-clock cap: a per-test `trio.fail_after`
# is the blessed guard here since `pytest-timeout`'s
# global cap is intentionally off (see the `pyproject`
# NOTE — it breaks trio under fork backends). Generous +
# CPU-headroom-scaled bc this is an anti-hang guard, not
# a perf assertion; a wedged ria-reaper once hung this
# test forever (the `._ria_nursery`-removal regression).
# Regression fix: commit `d1fb4a1a`, documented in
# `ai/conc-anal/ria_nursery_removal_plan.md`.
from .conftest import cpu_perf_headroom
with trio.fail_after(9 * cpu_perf_headroom()):
async with tractor.open_nursery(
debug_mode=debug_mode,
registry_addrs=[reg_addr],
) as an:
portal = await an.run_in_actor(
asyncio_actor,
target='aio_sleep_forever',
expect_err='trio.Cancelled',
p: tractor.Portal = await an.start_actor(
'aio_daemon',
enable_modules=[__name__],
infect_asyncio=True,
)
# cancel the entire remote runtime
await portal.cancel_actor()
async with (
# `.cancel_actor()` below tears the ctx down
expect_ctxc(yay=True),
p.open_context(
sleep_forever_aio_ctx,
) as (ctx, first),
):
# cancel the entire remote runtime while its
# infected-`asyncio` task sleeps forever
await p.cancel_actor()
trio.run(main)
@ -248,13 +293,19 @@ def test_trio_cancels_aio(
with trio.move_on_after(1):
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as tn:
await tn.run_in_actor(
asyncio_actor,
target='aio_sleep_forever',
expect_err='trio.Cancelled',
) as an:
p: tractor.Portal = await an.start_actor(
'aio_daemon',
enable_modules=[__name__],
infect_asyncio=True,
)
async with p.open_context(
sleep_forever_aio_ctx,
) as (ctx, first):
# block until the enclosing `move_on_after`
# cancels this `trio` scope, tearing down the
# infected-aio task via ctx cancellation
await trio.sleep_forever()
trio.run(main)
@ -404,17 +455,20 @@ def test_aio_cancelled_from_aio_causes_trio_cancelled(
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as an:
p: tractor.Portal = await an.run_in_actor(
# `to_actor.run()` blocks on the one-shot's result and
# relays the remote error here in the caller's task.
with trio.fail_after(1 + delay):
await to_actor.run(
partial(
asyncio_actor,
target='aio_cancel',
expect_err='tractor.to_asyncio.AsyncioCancelled',
expect_err=(
'tractor.to_asyncio.AsyncioCancelled'
),
),
an=an,
infect_asyncio=True,
)
# NOTE: normally the `an.__aexit__()` waits on the
# portal's result but we do it explicitly here
# to avoid indent levels.
with trio.fail_after(1 + delay):
await p.wait_for_result()
with pytest.raises(
expected_exception=(RemoteActorError, ExceptionGroup),
@ -615,13 +669,15 @@ def test_basic_interloop_channel_stream(
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as an:
portal = await an.run_in_actor(
# should raise RAE directly
await to_actor.run(
partial(
stream_from_aio,
infect_asyncio=True,
fan_out=fan_out,
),
an=an,
infect_asyncio=True,
)
# should raise RAE diectly
await portal.result()
trio.run(main)
@ -634,13 +690,15 @@ def test_trio_error_cancels_intertask_chan(
async with tractor.open_nursery(
registry_addrs=[reg_addr],
) as an:
portal = await an.run_in_actor(
# should trigger remote actor error
await to_actor.run(
partial(
stream_from_aio,
trio_raise_err=True,
),
an=an,
infect_asyncio=True,
)
# should trigger remote actor error
await portal.result()
with pytest.raises(RemoteActorError) as excinfo:
trio.run(main)
@ -670,14 +728,16 @@ def test_trio_closes_early_causes_aio_checkpoint_raise(
# enable_stack_on_sig=True,
registry_addrs=[reg_addr],
) as an:
portal = await an.run_in_actor(
# should raise RAE directly
print('waiting on final infected subactor result..')
res: None = await to_actor.run(
partial(
stream_from_aio,
trio_exit_early=True,
),
an=an,
infect_asyncio=True,
)
# should raise RAE diectly
print('waiting on final infected subactor result..')
res: None = await portal.wait_for_result()
assert res is None
print(f'infected subactor returned result: {res!r}\n')
@ -721,15 +781,17 @@ def test_aio_exits_early_relays_AsyncioTaskExited(
debug_mode=debug_mode,
# enable_stack_on_sig=True,
) as an:
portal = await an.run_in_actor(
# should raise RAE directly
print('waiting on final infected subactor result..')
res: None = await to_actor.run(
partial(
stream_from_aio,
infect_asyncio=True,
trio_exit_early=False,
aio_exit_early=True,
),
an=an,
infect_asyncio=True,
)
# should raise RAE diectly
print('waiting on final infected subactor result..')
res: None = await portal.wait_for_result()
assert res is None
print(f'infected subactor returned result: {res!r}\n')
@ -761,17 +823,21 @@ def test_aio_errors_and_channel_propagates_and_closes(
registry_addrs=[reg_addr],
debug_mode=debug_mode,
) as an:
portal = await an.run_in_actor(
# should trigger RAE directly, not an eg.
await to_actor.run(
partial(
stream_from_aio,
aio_raise_err=True,
),
an=an,
infect_asyncio=True,
)
# should trigger RAE directly, not an eg.
await portal.result()
with pytest.raises(
# NOTE: bc we directly wait on `Portal.result()` instead
# of capturing it inside the `ActorNursery` machinery.
# NOTE: bc `to_actor.run()` blocks on + relays the result
# in the caller's task (not captured inside the
# `ActorNursery` teardown machinery) we get a direct RAE,
# not an eg.
expected_exception=RemoteActorError,
) as excinfo:
trio.run(main)

View File

@ -163,12 +163,12 @@ def test_do_not_swallow_error_before_started_by_remote_contextcancelled(
async def main():
async with tractor.open_nursery(
debug_mode=debug_mode,
) as n:
portal = await n.start_actor(
) as an:
portal = await an.start_actor(
'errorer',
enable_modules=[__name__],
)
await n.start_actor(
await an.start_actor(
'sleeper',
enable_modules=[__name__],
)

View File

@ -139,9 +139,9 @@ async def test_required_args(callwith_expecterror):
with pytest.raises(err):
await func(**kwargs)
else:
async with tractor.open_nursery() as n:
async with tractor.open_nursery() as an:
portal = await n.start_actor(
portal = await an.start_actor(
name='pubber',
enable_modules=[__name__],
)
@ -176,32 +176,73 @@ def test_multi_actor_subs_arbiter_pub(
async def main():
async with tractor.open_nursery(
async with (
tractor.open_nursery(
registry_addrs=[reg_addr],
enable_modules=[__name__],
) as n:
) as an,
trio.open_nursery() as tn,
):
name = 'root'
if pub_actor == 'streamer':
# start the publisher as a daemon
master_portal = await n.start_actor(
master_portal = await an.start_actor(
'streamer',
enable_modules=[__name__],
)
name = 'streamer'
even_portal = await n.run_in_actor(
root_uid = tractor.current_actor().aid.uid
# spawn the two subscriber actors as daemons and run
# `subs()` on each as a background task (was the legacy
# `run_in_actor()`); keep the portals for the explicit
# `cancel_actor()` teardown below. Each runner swallows
# only the cancellation relayed after its own teardown
# starts; every earlier or unrelated failure propagates.
async def _run_subs(
portal: tractor.Portal,
which: list[str],
teardown_started: trio.Event,
) -> None:
try:
await portal.run(
subs,
which=['even'],
name='evens',
pub_actor_name=name
which=which,
pub_actor_name=name,
)
odd_portal = await n.run_in_actor(
subs,
which=['odd'],
name='odds',
pub_actor_name=name
except tractor.ContextCancelled as ctxc:
if not (
teardown_started.is_set()
and
ctxc.canceller == root_uid
):
raise
even_teardown_started = trio.Event()
odd_teardown_started = trio.Event()
even_portal = await an.start_actor(
'evens',
enable_modules=[__name__],
)
odd_portal = await an.start_actor(
'odds',
enable_modules=[__name__],
)
tn.start_soon(
_run_subs,
even_portal,
['even'],
even_teardown_started,
)
tn.start_soon(
_run_subs,
odd_portal,
['odd'],
odd_teardown_started,
)
async with tractor.wait_for_actor('evens'):
@ -241,12 +282,14 @@ def test_multi_actor_subs_arbiter_pub(
# await even_portal.result()
await trio.sleep(0.5)
even_teardown_started.set()
await even_portal.cancel_actor()
await trio.sleep(1)
if pub_actor == 'arbiter':
assert 'even' not in get_topics()
odd_teardown_started.set()
await odd_portal.cancel_actor()
if pub_actor == 'arbiter':
@ -257,6 +300,9 @@ def test_multi_actor_subs_arbiter_pub(
else:
await master_portal.cancel_actor()
# drop the bg `subs()` runners now the subs are cancelled
tn.cancel_scope.cancel()
trio.run(main)
@ -269,9 +315,9 @@ def test_single_subactor_pub_multitask_subs(
async with tractor.open_nursery(
registry_addrs=[reg_addr],
enable_modules=[__name__],
) as n:
) as an:
portal = await n.start_actor(
portal = await an.start_actor(
'streamer',
enable_modules=[__name__],
)

View File

@ -143,9 +143,12 @@ def test_ringbuf(
child_read_shm,
**common_kwargs,
total_bytes=total_bytes,
) as (sctx, _sent),
) as (rctx, _sent),
):
await recv_p.result()
# ctx-acm exits await each child task's
# `Return` (the prior `recv_p.result()` here
# was a daemon-portal no-op).
pass
await send_p.cancel_actor()
await recv_p.cancel_actor()

View File

@ -4,6 +4,7 @@ related API and error checks.
'''
import itertools
from functools import partial
from unittest.mock import (
AsyncMock,
Mock,
@ -234,23 +235,26 @@ def test_rpc_errors(
# do that if actually debugging subactor but keep it
# disabled for the test.
# debug_mode=True,
) as n:
) as an:
actor = tractor.current_actor()
assert actor.is_registrar
await n.run_in_actor(
await tractor.to_actor.run(
partial(
sleep_back_actor,
actor_name=subactor_requests_to,
func_name=funcname,
func_defined=bool(func_defined),
exposed_mods=exposed_mods,
reg_addr=reg_addr,
),
an=an,
name='subactor',
# function from the local exposed module space
# the subactor will invoke when it RPCs back to this actor
func_name=funcname,
exposed_mods=exposed_mods,
func_defined=True if func_defined else False,
# Function from the local exposed module space the
# subactor invokes when it RPCs back to this actor.
enable_modules=subactor_exposed_mods,
reg_addr=reg_addr,
)
def run():

View File

@ -2,6 +2,7 @@
Verifying internal runtime state and undocumented extras.
"""
from functools import partial
import os
import pytest
@ -83,13 +84,14 @@ async def test_lifetime_stack_wipes_tmpfile(
async with tractor.open_nursery(
loglevel=loglevel,
) as an:
await ( # inlined `tractor.Portal`
await an.run_in_actor(
await tractor.to_actor.run(
partial(
crash_and_clean_tmpdir,
tmp_file_path=path,
error=error_in_child,
),
an=an,
)
).result()
except (
tractor.RemoteActorError,
BaseExceptionGroup,

View File

@ -26,17 +26,31 @@ data_to_pass_down = {
}
async def spawn(
async def run_same_func_in_child(
should_be_root: bool,
data: dict,
reg_addr: tuple[str, int],
debug_mode: bool = False,
):
'''
Invoke this same module-scoped RPC target in a child actor.
RPC targets cross IPC as `module:name` namespace paths, so this
helper must remain import-addressable at module scope instead of
being nested inside the test. The root invocation boots a runtime
and recursively calls this function as a one-shot child endpoint;
the child branch returns the result.
'''
await trio.sleep(0.1)
actor = tractor.current_actor(err_on_no_runtime=False)
if should_be_root:
if not should_be_root:
assert actor is not None
assert actor.is_registrar == should_be_root
return 10
assert actor is None # no runtime yet
async with (
tractor.open_root_actor(
@ -48,44 +62,33 @@ async def spawn(
actor: tractor.Actor = tractor.current_actor()
assert actor.is_registrar == should_be_root
# spawns subproc here
portal: tractor.Portal = await an.run_in_actor(
fn=spawn,
# recursively spawn this same function as the lone
# task of a one-shot child subactor and get its result.
result = await tractor.to_actor.run(
partial(
run_same_func_in_child,
should_be_root=False,
data=data_to_pass_down,
reg_addr=reg_addr,
),
an=an,
# spawning args
name='sub-actor',
enable_modules=[__name__],
# passed to a subactor-recursive RPC invoke
# of this same `spawn()` fn.
should_be_root=False,
data=data_to_pass_down,
reg_addr=reg_addr,
)
assert len(an._children) == 1
assert (
portal.channel.aid.uid
in
tractor.current_actor().ipc_server._peers
)
# get result from child subactor
result = await portal.result()
assert result == 10
return result
else:
assert actor.is_registrar == should_be_root
return 10
def test_run_in_actor_same_func_in_child(
def test_to_actor_run_same_func_in_child(
reg_addr: tuple,
debug_mode: bool,
):
result = trio.run(
partial(
spawn,
run_same_func_in_child,
should_be_root=True,
data=data_to_pass_down,
reg_addr=reg_addr,
@ -159,21 +162,18 @@ async def test_most_beautiful_word(
async with tractor.open_nursery(
debug_mode=debug_mode,
) as an:
portal = await an.run_in_actor(
res: Any = await tractor.to_actor.run(
partial(
cellar_door,
return_value=return_value,
),
an=an,
name='some_linguist',
)
res: Any = await portal.wait_for_result()
assert res == return_value
# The ``async with`` will unblock here since the 'some_linguist'
# actor has completed its main task ``cellar_door``.
# this should pull the cached final result already captured during
# the nursery block exit.
res: Any = await portal.wait_for_result()
assert res == return_value
# The ``async with`` unblocks here — the 'some_linguist'
# one-shot actor completed its lone task ``cellar_door`` and
# was reaped by `to_actor.run()`.
print(res)
@ -215,11 +215,14 @@ def test_loglevel_propagated_to_subactor(
start_method=start_method,
registry_addrs=[reg_addr],
) as tn:
await tn.run_in_actor(
) as an:
await tractor.to_actor.run(
partial(
check_loglevel,
loglevel=level,
level=level,
),
an=an,
loglevel=level,
)
trio.run(main)
@ -267,11 +270,11 @@ async def check_parent_main_inheritance(
return has_data
def test_run_in_actor_can_skip_parent_main_inheritance(
def test_to_actor_run_can_skip_parent_main_inheritance(
start_method: str, # <- only support on `trio` backend rn.
):
'''
Verify ``inherit_parent_main=False`` on ``run_in_actor()``
Verify ``inherit_parent_main=False`` on ``to_actor.run()``
prevents parent ``__main__`` data from reaching the child.
'''
@ -284,21 +287,25 @@ def test_run_in_actor_can_skip_parent_main_inheritance(
async with tractor.open_nursery(start_method='trio') as an:
# Default: child receives parent __main__ bootstrap data
replaying = await an.run_in_actor(
await tractor.to_actor.run(
partial(
check_parent_main_inheritance,
name='replaying-parent-main',
expect_inherited=True,
),
an=an,
name='replaying-parent-main',
)
await replaying.result()
# Opt-out: child gets no parent __main__ data
isolated = await an.run_in_actor(
await tractor.to_actor.run(
partial(
check_parent_main_inheritance,
expect_inherited=False,
),
an=an,
name='isolated-parent-main',
inherit_parent_main=False,
expect_inherited=False,
)
await isolated.result()
trio.run(main)

View File

@ -8,14 +8,18 @@ from contextlib import (
from functools import partial
from itertools import cycle
import time
from types import SimpleNamespace
from typing import Optional
import warnings
import pytest
import trio
from trio.lowlevel import current_task
import tractor
from tractor.to_asyncio import LinkedTaskChannel
from tractor.trionics import (
broadcast_receiver,
BroadcastReceiveError,
Lagged,
collapse_eg,
)
@ -307,6 +311,801 @@ def test_subscribe_errors_after_close():
trio.run(main)
@pytest.mark.parametrize(
('size', 'sent', 'dropped'),
[
(1, 2, 1),
(3, 5, 2),
],
)
def test_lagged_reports_exact_drop_count(
size: int,
sent: int,
dropped: int,
) -> None:
'''
`Lagged` must report every value outside the retained window.
`BroadcastReceiver.receive_nowait()` previously subtracted the
queue length from an already-invalid deque index without counting
that first displaced value. A one-slot queue therefore claimed it
dropped zero values after two sends. Keep one root receiver idle
while a child subscriber drains every produced value, then prove
the lag error reports the exact overrun and positions the root at
the oldest value still retained by `BroadcastState.queue`.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(size)
brx = broadcast_receiver(rx, size)
async with brx.subscribe() as fast:
for value in range(sent):
await tx.send(value)
assert await fast.receive() == value
match = rf'dropped `{dropped}` values'
with pytest.raises(Lagged, match=match):
await brx.receive()
assert await brx.receive() == sent - size
trio.run(main)
def test_broadcast_statistics_report_queued_counts() -> None:
'''
`BroadcastState.statistics()` must report counts, not indexes.
Each `BroadcastState.subs` value is the deque index of a
receiver's next unread value, with `-1` meaning caught up. The
statistics API returned these indexes directly, so one queued
value appeared as zero and every positive count was one short.
Keep one root receiver idle while a child synchronously receives
four produced values. Prove the root count advances through one
and three retained values, then remains clamped to the three-slot
retention window after lagging.
Finally install an actual unwaited `trio.Event` in
`BroadcastState.recv_ready` while treating deprecations as errors.
This proves statistics checks `None` explicitly instead of using
deprecated `trio.Event` truthiness.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(3)
brx = broadcast_receiver(rx, 3)
async with brx.subscribe() as child:
state = brx._state
assert state.statistics()['queued_len_by_task'] == {
brx.key: 0,
child.key: 0,
}
await tx.send(0)
assert await child.receive() == 0
assert state.statistics()['queued_len_by_task'] == {
brx.key: 1,
child.key: 0,
}
for value in range(1, 4):
await tx.send(value)
assert await child.receive() == value
state.recv_ready = (child.key, trio.Event())
with warnings.catch_warnings():
warnings.simplefilter('error', DeprecationWarning)
stats = state.statistics()
assert stats['queued_len_by_task'] == {
brx.key: 3,
child.key: 0,
}
assert stats['tasks_waiting'] == 0
state.recv_ready = None
trio.run(main)
def test_cancelled_reader_diagnostics_are_transient() -> None:
'''
Cancelled-reader diagnostics must not retain stale `Task`s.
`BroadcastState.cancelled` previously accumulated every source
owner cancelled during `BroadcastReceiver.receive()`. Even after
that receiver successfully read again or its subscription closed,
`BroadcastState.statistics()` retained the old `Task`, reporting
stale state and keeping the completed task alive.
Cancel one child's source read under a receiver-local scope and
verify its task is reported. Reuse that same receiver for one
successful read to prove progress clears the entry. Cancel it once
more, then leave the subscription and prove close also removes the
diagnostic while the root receiver remains registered.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(1)
brx = broadcast_receiver(rx, 1)
cancel_scope = trio.CancelScope()
child_key: int
child_task = None
async with brx.subscribe() as child:
child_key = child.key
async def cancel_source_read() -> None:
nonlocal child_task
child_task = current_task()
with cancel_scope:
await child.receive()
assert cancel_scope.cancelled_caught
async with trio.open_nursery() as nursery:
nursery.start_soon(cancel_source_read)
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
cancel_scope.cancel()
stats = brx._state.statistics()
assert child_task is not None
assert stats['tasks_cancelled'] == {
child_key: child_task,
}
await tx.send(1)
assert await child.receive() == 1
assert not brx._state.cancelled
cancel_scope = trio.CancelScope()
async with trio.open_nursery() as nursery:
nursery.start_soon(cancel_source_read)
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
cancel_scope.cancel()
assert child_key in brx._state.cancelled
assert child_key not in brx._state.cancelled
assert brx.key in brx._state.subs
trio.run(main)
@pytest.mark.parametrize(
'terminal_exc',
[
trio.EndOfChannel(),
RuntimeError('terminal source failure'),
],
ids=['end-of-channel', 'receive-error'],
)
def test_terminal_broadcast_clears_cancelled_tasks(
terminal_exc: Exception,
) -> None:
'''
Terminal broadcast state must release every cancelled `Task`.
A receiver which owned and cancelled a source read can leave its
task in `BroadcastState.cancelled`. If another receiver later gets
EOC or a terminal source failure, no subscriber can make source
progress to clear that stale diagnostic. Clearing only the terminal
owner's key therefore retained the first receiver's completed task.
Cancel a child during the first controlled source read, then let
the root own a second read which raises EOC or `RuntimeError`.
Prove each terminal path clears the other receiver's diagnostic
before propagating its exact source outcome.
'''
class TerminalReceiver:
'''
Block one cancellable read, then raise a terminal outcome.
'''
def __init__(self) -> None:
self.calls = 0
self.first_started = trio.Event()
async def receive(self) -> None:
'''
Drive cancellation followed by terminal source state.
'''
self.calls += 1
if self.calls == 1:
self.first_started.set()
await trio.sleep_forever()
raise terminal_exc
async def main() -> None:
source = TerminalReceiver()
brx = broadcast_receiver(source, 1)
cancel_scope = trio.CancelScope()
async with brx.subscribe() as child:
async def cancel_child_read() -> None:
with cancel_scope:
await child.receive()
assert cancel_scope.cancelled_caught
async with trio.open_nursery() as nursery:
nursery.start_soon(cancel_child_read)
await source.first_started.wait()
cancel_scope.cancel()
assert child.key in brx._state.cancelled
with pytest.raises(type(terminal_exc)) as exc_info:
await brx.receive()
assert exc_info.value is terminal_exc
assert not brx._state.cancelled
trio.run(main)
def test_end_of_channel_is_terminal_for_waiting_peer() -> None:
'''
EOC must not let an awakened peer re-enter the closed source.
`BroadcastState.eoc` was set when one source owner received EOC,
but neither receive path consulted it. A peer waiting behind that
owner therefore woke, saw no queued value, and started a second
source read. Cancellation at that checkpoint could repopulate
`BroadcastState.cancelled` after the broadcast became terminal.
Block one child in the sole source read while the root waits on its
event, then release EOC. Both receivers must terminate from that
one source call, and the root's later receive must replay EOC
immediately without retaining cancellation diagnostics.
'''
class EOCReceiver:
'''
Publish one controlled EOC and reject any second source read.
'''
def __init__(self) -> None:
self.calls = 0
self.started = trio.Event()
self.release = trio.Event()
async def receive(self) -> None:
'''
Block the only valid source read until EOC release.
'''
self.calls += 1
assert self.calls == 1
self.started.set()
await self.release.wait()
raise trio.EndOfChannel
async def main() -> None:
source = EOCReceiver()
brx = broadcast_receiver(source, 1)
outcomes: list[str] = []
async with brx.subscribe() as child:
async def receive_eoc(
receiver,
name: str,
) -> None:
with pytest.raises(trio.EndOfChannel):
await receiver.receive()
outcomes.append(name)
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_eoc, child, 'child')
await source.started.wait()
nursery.start_soon(receive_eoc, brx, 'root')
_, event = brx._state.recv_ready
while not event.statistics().tasks_waiting:
await trio.lowlevel.checkpoint()
source.release.set()
with pytest.raises(trio.EndOfChannel):
await brx.receive()
assert sorted(outcomes) == ['child', 'root']
assert source.calls == 1
assert not brx._state.cancelled
trio.run(main)
def test_msgstream_eoc_close_preserves_aclose_override() -> None:
'''
Internal EOC cleanup must preserve the public `aclose()` contract.
Passing a new private keyword from `MsgStream.receive()` to
`self.aclose()` broke subclasses whose compatible override kept
the original zero-argument signature. Use a minimal subclass which
records virtual dispatch and delegates to the base implementation.
Drive graceful EOC through the real root broadcaster and prove the
override runs without closing that active root re-entrantly.
'''
class Stream(tractor.MsgStream):
'''
Record public close dispatch with the established signature.
'''
close_calls = 0
async def aclose(self):
'''
Delegate closure without accepting private arguments.
'''
self.close_calls += 1
return await super().aclose()
class PldRx:
'''
Delegate source receive and terminate the close drain.
'''
def __init__(self, rx) -> None:
self._rx = rx
async def recv_pld(self, **kwargs):
'''
Receive directly from the test source channel.
'''
return await self._rx.receive()
def recv_msg_nowait(self, **kwargs):
'''
Report EOC to finish `MsgStream.aclose()` draining.
'''
raise trio.EndOfChannel
async def main() -> None:
tx, rx = trio.open_memory_channel(1)
ctx = SimpleNamespace(
cid='test-context',
_pld_rx=PldRx(rx),
send_stop=lambda: trio.lowlevel.checkpoint(),
side='caller',
peer_side='callee',
maybe_raise=lambda **kwargs: None,
)
stream = Stream(ctx, rx)
async with stream.subscribe():
await tx.aclose()
with pytest.raises(trio.EndOfChannel):
await stream.receive()
assert stream.close_calls == 1
assert not stream._broadcaster._closed
trio.run(main)
@pytest.mark.parametrize(
'close_wrapper',
[
tractor.MsgStream.aclose,
LinkedTaskChannel.aclose,
],
ids=['msg-stream', 'linked-task-channel'],
)
def test_wrapper_close_clears_root_cancelled_task(
close_wrapper,
) -> None:
'''
Public stream close must release root cancellation diagnostics.
Root broadcasters allocated by `MsgStream.subscribe()` and
`LinkedTaskChannel.subscribe()` are private implementation state.
If their source receive was cancelled, callers had no public way
to close the root, so wrapper teardown retained the completed
`Task` in `BroadcastState.cancelled` indefinitely.
Cancel a root source read, attach that broadcaster to a minimal
public wrapper, and close it through each real `aclose()` method.
The root receiver and its task diagnostic must both be removed;
for `MsgStream`, pre-close the source to cover its idempotent early
return path.
'''
async def main() -> None:
_, rx = trio.open_memory_channel(1)
brx = broadcast_receiver(rx, 1)
cancel_scope = trio.CancelScope()
async def cancel_source_read() -> None:
with cancel_scope:
await brx.receive()
assert cancel_scope.cancelled_caught
async with trio.open_nursery() as nursery:
nursery.start_soon(cancel_source_read)
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
cancel_scope.cancel()
assert brx.key in brx._state.cancelled
if close_wrapper is tractor.MsgStream.aclose:
ctx = SimpleNamespace(cid='test-context')
wrapper = tractor.MsgStream(ctx, rx)
wrapper._broadcaster = brx
await rx.aclose()
else:
wrapper = SimpleNamespace(
_broadcaster=brx,
_from_aio=rx,
)
await close_wrapper(wrapper)
assert brx.key not in brx._state.subs
assert brx.key not in brx._state.cancelled
trio.run(main)
def test_broadcast_rejects_zero_buffer_size() -> None:
'''
A broadcaster must retain at least one value for peer fan-out.
`collections.deque(maxlen=0)` silently discards every appended
value, so `broadcast_receiver(..., 0)` allowed the source owner to
receive while peer cursors advanced into an always-empty queue.
Their lag recovery then reset to index `-1` and recursively retried
without any retained value to consume.
Construct a rendezvous memory channel and prove broadcaster setup
rejects its zero capacity synchronously with a clear public error,
before any receiver is registered or source receive can begin.
'''
_, rx = trio.open_memory_channel(0)
with pytest.raises(
ValueError,
match='`max_buffer_size` must be greater than zero',
):
broadcast_receiver(rx, 0)
def test_underlying_receive_failure_wakes_all_subscribers() -> None:
'''
A shared receive failure must terminate every broadcast receiver.
Previously, only `EndOfChannel` and receiver cancellation woke
peer tasks waiting on `BroadcastState.recv_ready`. If the shared
underlying receiver raised another error, its owner propagated
the failure and cleared the event while every peer remained
blocked forever.
Script one successful receive followed by a controlled
`RuntimeError`. Let a fast child own both underlying receives
while the root first drains its retained value and then waits on
the child's second receive. Release the failure only after both
tasks have reached those positions. Both exact errors prove the
peer was awakened without losing buffered data. A later
subscriber proves the terminal failure remains published for new
receivers instead of retrying the failed underlying channel.
'''
class FailingReceiver:
'''
Return one value, then fail after deterministic release.
'''
def __init__(self) -> None:
self.calls: int = 0
self.failure_started = trio.Event()
self.release_failure = trio.Event()
async def receive(self) -> int:
'''
Drive the scripted success-then-failure sequence.
'''
self.calls += 1
if self.calls == 1:
return 1
self.failure_started.set()
await self.release_failure.wait()
raise RuntimeError('underlying receive failed')
async def main() -> None:
source = FailingReceiver()
brx = broadcast_receiver(source, 3)
child_error: list[RuntimeError] = []
root_error: list[BroadcastReceiveError] = []
late_error: list[BroadcastReceiveError] = []
root_drained = trio.Event()
async def receive_child() -> None:
async with brx.subscribe() as child:
assert await child.receive() == 1
try:
await child.receive()
except RuntimeError as exc:
child_error.append(exc)
async def receive_root() -> None:
assert await brx.receive() == 1
root_drained.set()
try:
await brx.receive()
except BroadcastReceiveError as exc:
root_error.append(exc)
with trio.fail_after(1):
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_child)
await source.failure_started.wait()
nursery.start_soon(receive_root)
await root_drained.wait()
source.release_failure.set()
assert source.calls == 2
assert [str(exc) for exc in child_error] == [
'underlying receive failed',
]
assert [str(exc) for exc in root_error] == [
'Shared broadcast receiver failed',
]
assert child_error[0] is not root_error[0]
assert root_error[0].__cause__ is child_error[0]
async with brx.subscribe() as late:
with pytest.raises(
BroadcastReceiveError,
match='Shared broadcast receiver failed',
) as exc_info:
await late.receive()
late_error.append(exc_info.value)
assert late_error[0] is not child_error[0]
assert late_error[0] is not root_error[0]
assert late_error[0].__cause__ is child_error[0]
assert source.calls == 2
trio.run(main)
def test_control_flow_exit_wakes_broadcast_peer() -> None:
'''
Non-terminal control flow must wake peers without being retained.
Process-control and cancellation-like `BaseException` values
should remain local to the task which receives them, but the old
owner still has to wake subscribers blocked on its shared event.
Make one child own a controlled `BaseException` receive while the
root waits behind it. After release, prove the child gets that
exact exit and the root takes ownership of the next underlying
receive instead of hanging or replaying the control-flow event.
'''
class ReceiveExit(BaseException):
'''
Model a non-terminal process-control receive exit.
'''
class ControlFlowReceiver:
'''
Raise one controlled exit, then return a value.
'''
def __init__(self) -> None:
self.calls: int = 0
self.exit_started = trio.Event()
self.release_exit = trio.Event()
async def receive(self) -> int:
'''
Drive the scripted control-flow-then-value sequence.
'''
self.calls += 1
if self.calls == 1:
self.exit_started.set()
await self.release_exit.wait()
raise ReceiveExit
return 2
async def main() -> None:
source = ControlFlowReceiver()
brx = broadcast_receiver(source, 3)
child_exit: list[ReceiveExit] = []
root_value: list[int] = []
async def receive_child() -> None:
async with brx.subscribe() as child:
try:
await child.receive()
except ReceiveExit as exc:
child_exit.append(exc)
async def receive_root() -> None:
root_value.append(await brx.receive())
with trio.fail_after(1):
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_child)
await source.exit_started.wait()
nursery.start_soon(receive_root)
while True:
_, event = brx._state.recv_ready
if event.statistics().tasks_waiting:
break
await trio.lowlevel.checkpoint()
source.release_exit.set()
assert len(child_exit) == 1
assert root_value == [2]
assert source.calls == 2
assert brx._state.receive_exc is None
trio.run(main)
def test_closing_non_owner_preserves_source_wait() -> None:
'''
Closing one subscriber must not wake another receiver's peers.
`BroadcastReceiver.aclose()` previously set the one shared
`BroadcastState.recv_ready` event even when a different receiver
owned the source read. Waiting peers then repeatedly awaited an
already-set event until the source produced another value,
creating a runnable hot loop on idle streams.
Block one child in the source receive, then place both the root
and a closing child behind its event. Close only that waiting
child and prove it gets `ClosedResourceError` without setting the
shared event. Both remaining receivers must still get the same
value after the source is released.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(1)
brx = broadcast_receiver(rx, 3)
owner_value: list[int] = []
root_value: list[int] = []
closing_closed = trio.Event()
async with (
brx.subscribe() as owner,
brx.subscribe() as closing,
):
async def receive_owner() -> None:
owner_value.append(await owner.receive())
async def receive_root() -> None:
root_value.append(await brx.receive())
async def receive_closing() -> None:
with pytest.raises(trio.ClosedResourceError):
await closing.receive()
closing_closed.set()
with trio.fail_after(1):
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_owner)
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
nursery.start_soon(receive_root)
nursery.start_soon(receive_closing)
_, event = brx._state.recv_ready
while event.statistics().tasks_waiting < 2:
await trio.lowlevel.checkpoint()
await closing.aclose()
await closing_closed.wait()
assert not event.is_set()
await tx.send(1)
assert owner_value == [1]
assert root_value == [1]
trio.run(main)
@pytest.mark.parametrize(
'first_outcome',
[
1,
RuntimeError('discarded source error'),
trio.EndOfChannel(),
],
)
def test_closing_source_owner_hands_read_to_peer(
first_outcome: int|Exception,
) -> None:
'''
Closing the source-read owner must transfer ownership to a peer.
Merely suppressing the old shared-event wake would leave peers
blocked behind an externally closed receiver that still owned an
idle source read. Script a first receive which blocks until its
private scope is cancelled and a second which returns immediately.
Close that owner only after the root is waiting behind it. Cover
a shielded value, ordinary error and EOC from the cancelled source
read. The owner must always get `ClosedResourceError`, while the
awakened root takes the second source read without publishing the
discarded source outcome.
'''
class HandoffReceiver:
'''
Block the first source read and satisfy the second.
'''
def __init__(self) -> None:
self.calls: int = 0
self.first_started = trio.Event()
self.release_first = trio.Event()
async def receive(self) -> int:
'''
Drive one cancelled read followed by one value.
'''
self.calls += 1
if self.calls == 1:
self.first_started.set()
with trio.CancelScope(shield=True):
await self.release_first.wait()
if isinstance(first_outcome, BaseException):
raise first_outcome
return first_outcome
return 2
async def main() -> None:
source = HandoffReceiver()
brx = broadcast_receiver(source, 3)
owner_closed = trio.Event()
root_value: list[int] = []
async with brx.subscribe() as owner:
async def receive_owner() -> None:
with pytest.raises(trio.ClosedResourceError):
await owner.receive()
owner_closed.set()
async def receive_root() -> None:
root_value.append(await brx.receive())
with trio.fail_after(1):
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_owner)
await source.first_started.wait()
nursery.start_soon(receive_root)
_, event = brx._state.recv_ready
while not event.statistics().tasks_waiting:
await trio.lowlevel.checkpoint()
await owner.aclose()
source.release_first.set()
await owner_closed.wait()
assert source.calls == 2
assert root_value == [2]
assert brx._state.receive_exc is None
assert not brx._state.eoc
trio.run(main)
def test_ensure_slow_consumers_lag_out(
reg_addr,
start_method,
@ -448,6 +1247,7 @@ def test_first_recver_is_cancelled():
async with brx.subscribe() as bc:
async for value in bc:
print(value)
assert cs.cancelled_caught
async def cancel_and_send():
await trio.sleep(0.2)
@ -519,3 +1319,74 @@ def test_no_raise_on_lag():
with pytest.raises(KeyboardInterrupt):
trio.run(main)
@pytest.mark.parametrize(
('subscribe', 'chan_attr'),
[
(tractor.MsgStream.subscribe, '_rx_chan'),
(LinkedTaskChannel.subscribe, '_from_aio'),
],
ids=['msg-stream', 'linked-task-channel'],
)
def test_stream_subscribe_forwards_lag_policy(
subscribe,
chan_attr: str,
) -> None:
'''
Stream wrappers must expose per-subscriber lag policy.
`MsgStream.subscribe()` and `LinkedTaskChannel.subscribe()`
previously omitted `BroadcastReceiver.raise_on_lag`, forcing
downstream users to mutate a private receiver attribute. Invoke
each public wrapper against a minimal receive-compatible handle.
Prove the first non-raising subscription configures both the
irreversible root broadcaster and its child, while a later strict
child selects its own policy without changing that root.
'''
class StreamHandle:
'''
Provide the wrapper fields needed for local fan-out.
'''
def __init__(self) -> None:
self._broadcaster = None
setattr(
self,
chan_attr,
SimpleNamespace(
_state=SimpleNamespace(max_buffer_size=1),
),
)
async def receive(self):
'''
Block if a regression unexpectedly enters source receive.
'''
await trio.sleep_forever()
async def send(self, value) -> None:
'''
Satisfy `MsgStream` duplex-handle patching.
'''
async def main() -> None:
stream = StreamHandle()
async with subscribe(
stream,
raise_on_lag=False,
) as first:
assert not stream._broadcaster._raise_on_lag
assert not first._raise_on_lag
async with subscribe(
stream,
raise_on_lag=True,
) as second:
assert not stream._broadcaster._raise_on_lag
assert second._raise_on_lag
trio.run(main)

View File

@ -1,8 +1,8 @@
'''
`tractor.to_actor`: one-shot single-remote-task API suite.
Verifies the "spiritual successor" to (and eventual
replacement of) `ActorNursery.run_in_actor()`; see
Verifies the "spiritual successor" to (and replacement of)
the removed legacy `ActorNursery.run_in_actor()`; see
https://github.com/goodboy/tractor/issues/477
'''
@ -160,7 +160,7 @@ async def test_remote_error_relayed_to_caller_task(
A remote task error is raised directly in the
caller's task as a boxed `RemoteActorError` instead
of surfacing at actor-nursery teardown as with the
legacy `.run_in_actor()` API.
removed legacy `.run_in_actor()` API.
'''
with pytest.raises(RemoteActorError) as excinfo:

View File

@ -780,7 +780,7 @@ class Context:
# `Portal.open_context()` has been opened since it's
# assumed that other portal APIs like,
# - `Portal.run()`,
# - `ActorNursery.run_in_actor()`
# - `to_actor.run()`
# do their own error checking at their own call points and
# result processing.

View File

@ -1161,10 +1161,6 @@ class TransportClosed(Exception):
)
class NoResult(RuntimeError):
"No final result is expected for this actor"
class ModuleNotExposed(ModuleNotFoundError):
"The requested module is not exposed for RPC"

View File

@ -103,6 +103,12 @@ class MsgStream(trio.abc.Channel):
self._eoc: bool|trio.EndOfChannel = False
self._closed: bool|trio.ClosedResourceError = False
# `MsgStream.receive()` sets this while it calls
# `MsgStream.aclose()` after source EOC. That close is
# re-entrant from the root `BroadcastReceiver._recv`, so it
# must not cancel the same receiver before EOC propagates.
self._eoc_close_task: trio.lowlevel.Task|None = None
@property
def ctx(self) -> Context:
'''
@ -256,7 +262,16 @@ class MsgStream(trio.abc.Channel):
# when the send is closed we assume the stream has
# terminated and signal this local iterator to stop
#
# Preserve virtual dispatch through the public zero-argument
# `MsgStream.aclose()` API. The task marker lets the base
# implementation distinguish this receive-internal close from
# an explicit caller or `MsgStream.__aexit__()` close.
self._eoc_close_task = trio.lowlevel.current_task()
try:
drained: list[Exception|dict] = await self.aclose()
finally:
self._eoc_close_task = None
if drained:
# ^^^^^^^^TODO? pass these to the `._ctx._drained_msgs:
# deque` and then iterate them as part of any
@ -335,6 +350,20 @@ class MsgStream(trio.abc.Channel):
# `.__aexit__()` as well!!!
# => SO ENSURE WE CATCH ALL TERMINATION STATES in this
# block including the EoC..
# `MsgStream.subscribe()` stores its hidden root broadcaster
# on `self._broadcaster`. Explicit teardown owns that root and
# must close it to release its subscriber and cancelled-task
# diagnostic. Skip only the receive-internal EOC close above:
# cancelling the active root's source-read scope there would
# turn graceful EOC into `trio.ClosedResourceError`.
if (
trio.lowlevel.current_task() is not self._eoc_close_task
and
(broadcaster := self._broadcaster) is not None
):
await broadcaster.aclose()
if self.closed:
# this stream has already been closed so silently succeed as
# per ``trio.AsyncResource`` semantics.
@ -512,6 +541,7 @@ class MsgStream(trio.abc.Channel):
@acm
async def subscribe(
self,
raise_on_lag: bool = True,
) -> AsyncIterator[BroadcastReceiver]:
'''
@ -526,6 +556,11 @@ class MsgStream(trio.abc.Channel):
value from the far end via the internally created broudcast
receiver wrapper.
``raise_on_lag=False`` makes this subscription warn and resume
at the oldest retained value after an overrun. The first call
also sets that policy for this stream's root receive handle;
later child subscriptions choose their policy independently.
'''
# NOTE: This operation is indempotent and non-reversible, so be
# sure you can deal with any (theoretical) overhead of the the
@ -541,6 +576,7 @@ class MsgStream(trio.abc.Channel):
# TODO: can remove this kwarg right since
# by default behaviour is to do this anyway?
receive_afunc=self.receive,
raise_on_lag=raise_on_lag,
)
# NOTE: we override the original stream instance's receive
@ -552,7 +588,9 @@ class MsgStream(trio.abc.Channel):
# seems there's no graceful way to type this with ``mypy``?
# https://github.com/python/mypy/issues/708
async with self._broadcaster.subscribe() as bstream:
async with self._broadcaster.subscribe(
raise_on_lag=raise_on_lag,
) as bstream:
assert bstream.key != self._broadcaster.key
assert bstream._recv == self._broadcaster._recv

View File

@ -217,20 +217,23 @@ def pub(
.. code:: python
from functools import partial
import tractor
async with tractor.open_nursery() as n:
portal = n.run_in_actor(
portal = await n.start_actor(
'publisher', # actor name
partial( # func to execute in it
pub_service,
enable_modules=[__name__],
)
try:
async with portal.open_stream_from(
pub_service, # func to execute in it
topics=('clicks', 'users'),
task_name='source1',
)
)
async for value in await portal.result():
) as stream:
async for value in stream:
print(f"Subscriber received {value}")
finally:
await portal.cancel_actor()
Here, you don't need to provide the ``ctx`` argument since the

View File

@ -306,12 +306,11 @@ class Start(
It is called by all the following public APIs:
- `ActorNursery.run_in_actor()`
- `to_actor.run()`
- `Portal.run()`
`|_.run_from_ns()`
`|_.open_stream_from()`
`|_._submit_for_result()`
- `Context.open_context()`

View File

@ -50,13 +50,11 @@ from ..ipc import Channel
from ..log import get_logger
from ..msg import (
# Error,
PayloadMsg,
NamespacePath,
Return,
)
from .._exceptions import (
ActorTooSlowError,
NoResult,
TransportClosed,
)
from .._context import (
@ -102,14 +100,6 @@ class Portal:
) -> None:
self._chan: Channel = channel
# during the portal's lifetime
self._final_result_pld: Any|None = None
self._final_result_msg: PayloadMsg|None = None
# When set to a ``Context`` (when _submit_for_result is called)
# it is expected that ``result()`` will be awaited at some
# point.
self._expect_result_ctx: Context|None = None
self._streams: set[MsgStream] = set()
# TODO, this should be PRIVATE (and never used publicly)! since it's just
@ -137,102 +127,6 @@ class Portal:
)
return self.chan
# TODO: factor this out into a `.highlevel` API-wrapper that uses
# a single `.open_context()` call underneath.
async def _submit_for_result(
self,
ns: str,
func: str,
**kwargs
) -> None:
if self._expect_result_ctx is not None:
raise RuntimeError(
'A pending main result has already been submitted'
)
self._expect_result_ctx: Context = await self.actor.start_remote_task(
self.channel,
nsf=NamespacePath(f'{ns}:{func}'),
kwargs=kwargs,
portal=self,
)
# TODO: we should deprecate this API right? since if we remove
# `.run_in_actor()` (and instead move it to a `.highlevel`
# wrapper api (around a single `.open_context()` call) we don't
# really have any notion of a "main" remote task any more?
#
# @api_frame
async def wait_for_result(
self,
hide_tb: bool = True,
) -> Any:
'''
Return the final result delivered by a `Return`-msg from the
remote peer actor's "main" task's `return` statement.
'''
__tracebackhide__: bool = hide_tb
# Check for non-rpc errors slapped on the
# channel for which we always raise
exc = self.channel._exc
if exc:
raise exc
# not expecting a "main" result
if self._expect_result_ctx is None:
peer_id: str = f'{self.channel.aid.reprol()!r}'
log.warning(
f'Portal to peer {peer_id} will not deliver a final result?\n'
f'\n'
f'Context.result() can only be called by the parent of '
f'a sub-actor when it was spawned with '
f'`ActorNursery.run_in_actor()`'
f'\n'
f'Further this `ActorNursery`-method-API will deprecated in the'
f'near fututre!\n'
)
return NoResult
# expecting a "main" result
assert self._expect_result_ctx
if self._final_result_msg is None:
try:
(
self._final_result_msg,
self._final_result_pld,
) = await self._expect_result_ctx._pld_rx.recv_msg(
ipc=self._expect_result_ctx,
expect_msg=Return,
)
except BaseException as err:
# TODO: wrap this into `@api_frame` optionally with
# some kinda filtering mechanism like log levels?
__tracebackhide__: bool = False
raise err
return self._final_result_pld
# TODO: factor this out into a `.highlevel` API-wrapper that uses
# a single `.open_context()` call underneath.
async def result(
self,
*args,
**kwargs,
) -> Any|Exception:
typname: str = type(self).__name__
log.warning(
f'`{typname}.result()` is DEPRECATED!\n'
f'\n'
f'Use `{typname}.wait_for_result()` instead!\n'
)
return await self.wait_for_result(
*args,
**kwargs,
)
async def _cancel_streams(self):
# terminate all locally running async generator
# IPC calls

View File

@ -20,7 +20,6 @@
"""
from contextlib import asynccontextmanager as acm
from functools import partial
import inspect
from typing import (
TYPE_CHECKING,
)
@ -225,7 +224,6 @@ class ActorNursery:
self,
# TODO: maybe def these as fields of a struct looking type?
actor: Actor,
ria_nursery: trio.Nursery,
da_nursery: trio.Nursery,
errors: dict[tuple[str, str], BaseException],
@ -267,16 +265,6 @@ class ActorNursery:
# and syncing purposes to any actor opened nurseries.
self._implicit_runtime_started: bool = False
# TODO: remove the `.run_in_actor()` API and thus this 2ndary
# nursery when that API get's moved outside this primitive!
self._ria_nursery = ria_nursery
# TODO, factor this into a .hilevel api!
#
# portals spawned with ``run_in_actor()`` are
# cancelled when their "main" result arrives
self._cancel_after_result_on_exit: set = set()
# trio.Nursery-like cancel (request) statuses
self._cancelled_caught: bool = False
self._cancel_called: bool = False
@ -438,11 +426,6 @@ class ActorNursery:
debug_mode: bool|None = None,
infect_asyncio: bool = False,
inherit_parent_main: bool = True,
# TODO: ideally we can rm this once we no longer have
# a `._ria_nursery` since the dependent APIs have been
# removed!
nursery: trio.Nursery|None = None,
proc_kwargs: dict[str, typing.Any] | None = None,
) -> Portal:
@ -510,10 +493,8 @@ class ActorNursery:
# start a task to spawn a process
# blocks until process has been started and a portal setup
nursery: trio.Nursery = nursery or self._da_nursery
# XXX: the type ignore is actually due to a `mypy` bug
return await nursery.start( # type: ignore
return await self._da_nursery.start( # type: ignore
partial(
_spawn.new_proc,
name,
@ -528,86 +509,6 @@ class ActorNursery:
)
)
# TODO: DEPRECATE THIS:
# -[x] impl instead as a hilevel wrapper on top of
# the lower level daemon-spawn + portal APIs
# |_ see `.to_actor.run()` (issue #477) which does
# `.start_actor()` + `Portal.run()` + a one-shot
# reap via `Portal.cancel_actor()`.
# -[ ] emit a `DeprecationWarning` here (requires
# migrating all in-repo usage first!)
# -[ ] use @api_frame on the wrapper
async def run_in_actor(
self,
fn: typing.Callable,
*,
name: str | None = None,
bind_addrs: UnwrappedAddress|None = None,
rpc_module_paths: list[str] | None = None,
enable_modules: list[str] | None = None,
loglevel: str | None = None, # set log level per subactor
infect_asyncio: bool = False,
inherit_parent_main: bool = True,
proc_kwargs: dict[str, typing.Any] | None = None,
**kwargs, # explicit args to ``fn``
) -> Portal:
'''
Spawn a new actor, run a lone task, then terminate the actor and
return its result.
Actors spawned using this method are kept alive at nursery teardown
until the task spawned by executing ``fn`` completes at which point
the actor is terminated.
NOTE: prefer the (eventual) replacement API
`tractor.to_actor.run()` which delivers the same
one-shot semantics decoupled from this nursery's
internal spawn machinery; see issue #477.
'''
__runtimeframe__: int = 1 # noqa
mod_path: str = fn.__module__
if name is None:
# use the explicit function name if not provided
name = fn.__name__
proc_kwargs = dict(proc_kwargs or {})
portal: Portal = await self.start_actor(
name,
enable_modules=[mod_path] + (
enable_modules or rpc_module_paths or []
),
bind_addrs=bind_addrs,
loglevel=loglevel,
# use the run_in_actor nursery
nursery=self._ria_nursery,
infect_asyncio=infect_asyncio,
inherit_parent_main=inherit_parent_main,
proc_kwargs=proc_kwargs
)
# XXX: don't allow stream funcs
if not (
inspect.iscoroutinefunction(fn) and
not getattr(fn, '_tractor_stream_function', False)
):
raise TypeError(f'{fn} must be an async function!')
# this marks the actor to be cancelled after its portal result
# is retreived, see logic in `open_nursery()` below.
self._cancel_after_result_on_exit.add(portal)
await portal._submit_for_result(
mod_path,
fn.__name__,
**kwargs
)
return portal
# @api_frame
async def cancel(
self,
@ -738,40 +639,20 @@ async def _open_and_supervise_one_cancels_all_nursery(
# normally don't need to show user by default
__tracebackhide__: bool = hide_tb
outer_err: BaseException|None = None
inner_err: BaseException|None = None
# the collection of errors retreived from spawned sub-actors
errors: dict[tuple[str, str], BaseException] = {}
# This is the outermost level "deamon actor" nursery. It is awaited
# **after** the below inner "run in actor nursery". This allows for
# handling errors that are generated by the inner nursery in
# a supervisor strategy **before** blocking indefinitely to wait for
# actors spawned in "daemon mode" (aka started using
# `ActorNursery.start_actor()`).
# errors from this daemon actor nursery bubble up to caller
# The single "daemon actor" nursery into which ALL subactors
# are spawned; one-shot (`to_actor.run()`) subactors are
# result-waited and reaped in their caller's own task-scope
# (see the #477 `.run_in_actor()`/`._ria_nursery` removal);
# errors from this nursery bubble up to the caller.
async with (
collapse_eg(),
trio.open_nursery() as da_nursery,
):
try:
# This is the inner level "run in actor" nursery. It is
# awaited first since actors spawned in this way (using
# `ActorNusery.run_in_actor()`) are expected to only
# return a single result and then complete (i.e. be canclled
# gracefully). Errors collected from these actors are
# immediately raised for handling by a supervisor strategy.
# As such if the strategy propagates any error(s) upwards
# the above "daemon actor" nursery will be notified.
async with (
collapse_eg(),
trio.open_nursery() as ria_nursery,
):
an = ActorNursery(
actor,
ria_nursery,
da_nursery,
errors
)
@ -789,9 +670,19 @@ async def _open_and_supervise_one_cancels_all_nursery(
)
an._request_reap_all()
except BaseException as _inner_err:
inner_err = _inner_err
errors[actor.aid.uid] = inner_err
# Single one-cancels-all handler for the (now single)
# daemon nursery. Pre-#477 a 2ndary `._ria_nursery`
# required a separate *outer* handler to catch errors
# bubbling from its task-reaping `__aexit__`; with that
# nursery gone this lone handler covers every scope
# error. NB: we deliberately do NOT re-raise here — the
# `finally` below raises the collected `errors` (as a
# single exc or `BaseExceptionGroup`), which already
# superseded the old outer handler's `raise` anyway
# since `errors` is populated (below) before any await.
except BaseException as _scope_err:
an._scope_error = _scope_err
errors[actor.aid.uid] = _scope_err
# If we error in the root but the debugger is
# engaged we don't want to prematurely kill (and
@ -814,12 +705,12 @@ async def _open_and_supervise_one_cancels_all_nursery(
# block here might not complete? For now,
# shield both.
with trio.CancelScope(shield=True):
etype: type = type(inner_err)
etype: type = type(_scope_err)
if etype in (
trio.Cancelled,
KeyboardInterrupt,
) or (
is_multi_cancelled(inner_err)
is_multi_cancelled(_scope_err)
):
log.cancel(
f'Actor-nursery cancelled by {etype}\n\n'
@ -836,7 +727,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
log.cancel(
'Actor-nursery caught remote cancellation\n'
'\n'
f'{inner_err.tb_str}'
f'{_scope_err.tb_str}'
)
else:
log.exception(
@ -855,50 +746,11 @@ async def _open_and_supervise_one_cancels_all_nursery(
# cancel all subactors
await an.cancel()
# ria_nursery scope end
# TODO: this is the handler around the ``.run_in_actor()``
# nursery. Ideally we can drop this entirely in the future as
# the whole ``.run_in_actor()`` API should be built "on top of"
# this lower level spawn-request-cancel "daemon actor" API where
# a local in-actor task nursery is used with one-to-one task
# + `await Portal.run()` calls and the results/errors are
# handled directly (inline) and errors by the local nursery.
except (
Exception,
BaseExceptionGroup,
trio.Cancelled
) as _outer_err:
outer_err = _outer_err
an._scope_error = outer_err or inner_err
# XXX: yet another guard before allowing the cancel
# sequence in case a (single) child is in debug.
await debug.maybe_wait_for_debugger(
child_in_debug=an._at_least_one_child_in_debug
)
# If actor-local error was raised while waiting on
# ".run_in_actor()" actors then we also want to cancel all
# remaining sub-actors (due to our lone strategy:
# one-cancels-all).
if an._children:
log.cancel(
'Actor-nursery cancelling due error type:\n'
f'{outer_err}\n'
)
with trio.CancelScope(shield=True):
await an.cancel()
raise
finally:
# No errors were raised while awaiting ".run_in_actor()"
# actors but those actors may have returned remote errors as
# results (meaning they errored remotely and have relayed
# those errors back to this parent actor). The errors are
# collected in ``errors`` so cancel all actors, summarize
# all errors and re-raise.
# an error was stashed by the handler above (or by
# a spawn task via the shared `errors` dict) so
# cancel any remaining subactors, summarize and
# re-raise.
if errors:
if an._children:
with trio.CancelScope(shield=True):
@ -942,15 +794,13 @@ async def open_nursery(
Create and yield a new ``ActorNursery`` to be used for spawning
structured concurrent subactors.
When an actor is spawned a new trio task is started which
invokes one of the process spawning backends to create and start
a new subprocess. These tasks are started by one of two nurseries
detailed below. The reason for spawning processes from within
a new task is because ``trio_run_in_process`` itself creates a new
internal nursery and the same task that opens a nursery **must**
close it. It turns out this approach is probably more correct
anyway since it is more clear from the following nested nurseries
which cancellation scopes correspond to each spawned subactor set.
When an actor is spawned a new trio task invokes one of the
process spawning backends to create and start a new subprocess.
These tasks are started in the supervisor's process nursery.
Spawning from a task is required because ``trio_run_in_process``
creates an internal nursery which the opening task **must** close;
this also makes each task's cancellation scope correspond to its
spawned subactor.
'''
__tracebackhide__: bool = hide_tb

View File

@ -48,7 +48,6 @@ from ._entry import _mp_main
# by `try_set_start_method()` after module load time.
from . import _spawn
from ._spawn import (
cancel_on_completion,
proc_waiter,
soft_kill,
)
@ -199,15 +198,6 @@ async def mp_proc(
with trio.CancelScope(shield=True):
await reap_request.wait()
async with trio.open_nursery() as nursery:
if portal in actor_nursery._cancel_after_result_on_exit:
nursery.start_soon(
cancel_on_completion,
portal,
subactor,
errors
)
# This is a "soft" (cancellable) join/reap which
# will remote cancel the actor on a ``trio.Cancelled``
# condition.
@ -217,13 +207,6 @@ async def mp_proc(
portal
)
# cancel result waiter that may have been spawned in
# tandem if not done already
log.warning(
"Cancelling existing result waiter task for "
f"{subactor.aid.uid}")
nursery.cancel_scope.cancel()
finally:
# hard reap sequence
if proc.is_alive():

View File

@ -126,98 +126,6 @@ def try_set_start_method(
return _ctx
async def exhaust_portal(
portal: Portal,
actor: Actor
) -> Any:
'''
Pull final result from portal (assuming it has one).
If the main task is an async generator do our best to consume
what's left of it.
'''
__tracebackhide__ = True
try:
log.debug(
f'Waiting on final result from {actor.aid.uid}'
)
# XXX: streams should never be reaped here since they should
# always be established and shutdown using a context manager api
final: Any = await portal.wait_for_result()
except (
Exception,
BaseExceptionGroup,
) as err:
# we reraise in the parent task via a ``BaseExceptionGroup``
return err
except trio.Cancelled as err:
# lol, of course we need this too ;P
# TODO: merge with above?
log.warning(
'Cancelled portal result waiter task:\n'
f'uid: {portal.channel.aid}\n'
f'error: {err}\n'
)
return err
else:
log.debug(
f'Returning final result from portal:\n'
f'uid: {portal.channel.aid}\n'
f'result: {final}\n'
)
return final
async def cancel_on_completion(
portal: Portal,
actor: Actor,
errors: dict[tuple[str, str], Exception],
) -> None:
'''
Cancel actor gracefully once its "main" portal's
result arrives.
Should only be called for actors spawned via the
`Portal.run_in_actor()` API.
=> and really this API will be deprecated and should be
re-implemented as a `.hilevel.one_shot_task_nursery()`..)
'''
# if this call errors we store the exception for later
# in ``errors`` which will be reraised inside
# an exception group and we still send out a cancel request
result: Any|Exception = await exhaust_portal(
portal,
actor,
)
if isinstance(result, Exception):
errors[actor.aid.uid]: Exception = result
log.cancel(
'Cancelling subactor runtime due to error:\n\n'
f'Portal.cancel_actor() => {portal.channel.aid}\n\n'
f'error: {result}\n'
)
else:
log.runtime(
'Cancelling subactor gracefully:\n\n'
f'Portal.cancel_actor() => {portal.channel.aid}\n\n'
f'result: {result}\n'
)
# cancel the process now that we have a final result
await portal.cancel_actor()
async def hard_kill(
proc: trio.Process,
@ -464,8 +372,8 @@ async def new_proc(
# NOTE: bottom-of-module to avoid a circular import since the
# backend submodules pull `cancel_on_completion`/`soft_kill`/
# `hard_kill`/`proc_waiter` from this module.
# backend submodules pull `soft_kill`/`hard_kill`/`proc_waiter`
# from this module.
from ._trio import trio_proc
from ._mp import mp_proc

View File

@ -49,7 +49,6 @@ from tractor.msg import (
pretty_struct,
)
from ._spawn import (
cancel_on_completion,
hard_kill,
soft_kill,
)
@ -215,15 +214,6 @@ async def trio_proc(
with trio.CancelScope(shield=True):
await reap_request.wait()
async with trio.open_nursery() as nursery:
if portal in actor_nursery._cancel_after_result_on_exit:
nursery.start_soon(
cancel_on_completion,
portal,
subactor,
errors
)
# This is a "soft" (cancellable) join/reap which
# will remote cancel the actor on a ``trio.Cancelled``
# condition.
@ -233,14 +223,6 @@ async def trio_proc(
portal
)
# cancel result waiter that may have been spawned in
# tandem if not done already
log.cancel(
'Cancelling portal result reaper task\n'
f'c)> {subactor.aid.reprol()!r}\n'
)
nursery.cancel_scope.cancel()
finally:
# XXX NOTE XXX: The "hard" reap since no actor zombies are
# allowed! Do this **after** cancellation/teardown to avoid

View File

@ -25,8 +25,8 @@ its result and (when the call owns the subactor) reap it.
Target arguments follow Trio's positional convention; use
`functools.partial()` to bind target keyword arguments.
The "spiritual successor" to (and eventual replacement of)
the `ActorNursery.run_in_actor()` API; see
The "spiritual successor" to (and replacement of) the removed
legacy `ActorNursery.run_in_actor()` API; see
https://github.com/goodboy/tractor/issues/477
'''

View File

@ -31,7 +31,7 @@ the lower level daemon-actor spawn + portal APIs,
such that error collection and propagation happens in the
*caller's task* (and thus whatever `trio` nursery/scope
encloses it) instead of inside the actor-nursery's
spawn-machinery nurseries as with the (to be deprecated)
spawn-machinery nurseries as with the (now removed) legacy
`ActorNursery.run_in_actor()` API.
'''
@ -282,8 +282,8 @@ async def run(
module in its `enable_modules` list. Calls that spawn their own
actor add the trampoline module automatically.
Unlike `ActorNursery.run_in_actor()` (which returns
a `Portal` whose result is only collected at
Unlike the removed legacy `ActorNursery.run_in_actor()` (which
returned a `Portal` whose result was only collected at
actor-nursery teardown) this is a plain "call and
wait" primitive: any remote error is raised HERE, in
the caller's task. Concurrency is composed the usual

View File

@ -213,6 +213,14 @@ class LinkedTaskChannel(
_broadcaster: BroadcastReceiver|None = None
async def aclose(self) -> None:
# `LinkedTaskChannel.subscribe()` lazily allocates and retains
# this root receiver. Close it first so its receiver-local
# source-read scope and cancellation diagnostics are released
# before `self._from_aio` becomes inaccessible; child
# subscriptions retain their own independent close lifetimes.
if (broadcaster := self._broadcaster) is not None:
await broadcaster.aclose()
await self._from_aio.aclose()
# ?TODO? async version of this?
@ -324,6 +332,7 @@ class LinkedTaskChannel(
@acm
async def subscribe(
self,
raise_on_lag: bool = True,
) -> AsyncIterator[BroadcastReceiver]:
'''
@ -335,6 +344,11 @@ class LinkedTaskChannel(
See ``tractor._streaming.MsgStream.subscribe()`` for further
similar details.
``raise_on_lag=False`` makes this subscription warn and resume
at the oldest retained value after an overrun. The first call
also sets that policy for this channel's root receive handle;
later child subscriptions choose their policy independently.
'''
if self._broadcaster is None:
@ -343,11 +357,14 @@ class LinkedTaskChannel(
# use memory channel size by default
self._from_aio._state.max_buffer_size, # type: ignore
receive_afunc=self.receive,
raise_on_lag=raise_on_lag,
)
self.receive = bcast.receive # type: ignore
async with self._broadcaster.subscribe() as bstream:
async with self._broadcaster.subscribe(
raise_on_lag=raise_on_lag,
) as bstream:
assert bstream.key != self._broadcaster.key
assert bstream._recv == self._broadcaster._recv
yield bstream

View File

@ -26,6 +26,7 @@ from ._mngrs import (
from ._broadcast import (
AsyncReceiver as AsyncReceiver,
broadcast_receiver as broadcast_receiver,
BroadcastReceiveError as BroadcastReceiveError,
BroadcastReceiver as BroadcastReceiver,
Lagged as Lagged,
)

View File

@ -100,6 +100,20 @@ class Lagged(trio.TooSlowError):
'''
class BroadcastReceiveError(Exception):
'''
A shared underlying receiver failed in another subscriber task.
'''
class _BroadcastReceiverClosed(Exception):
'''
An active receiver was closed while owning the source read.
'''
class BroadcastState(Struct):
'''
Common state to all receivers of a broadcast.
@ -115,6 +129,7 @@ class BroadcastState(Struct):
# broadcast event to wake up all sleeping consumer tasks
# on a newly produced value from the sender.
recv_ready: tuple[int, trio.Event]|None = None
recv_scope: trio.CancelScope|None = None
# if a ``trio.EndOfChannel`` is received on any
# consumer all consumers should be placed in this state
@ -122,7 +137,13 @@ class BroadcastState(Struct):
# For now, this is solely for testing/debugging purposes.
eoc: bool = False
# If the broadcaster was cancelled, we might as well track it
# Any non-EOC failure from the shared underlying receiver is
# terminal for every subscriber. Retained values remain readable
# before this failure is replayed at each receiver's boundary.
receive_exc: Exception | None = None
# Retain the latest interrupted source-reader task until its
# receiver next makes progress or closes.
cancelled: dict[int, Task] = {}
def statistics(self) -> dict[str, Any]:
@ -142,13 +163,20 @@ class BroadcastState(Struct):
qlens: dict[int, int] = {}
for tid, sz in subs.items():
qlens[tid] = sz if sz != -1 else 0
qlens[tid] = min(
sz + 1,
len(self.queue),
)
return {
'open_consumers': len(subs),
'queued_len_by_task': qlens,
'max_buffer_size': self.maxlen,
'tasks_waiting': ev.statistics().tasks_waiting if ev else 0,
'tasks_waiting': (
ev.statistics().tasks_waiting
if ev is not None
else 0
),
'tasks_cancelled': self.cancelled,
'next_value_receiver_id': key,
}
@ -190,6 +218,7 @@ class BroadcastReceiver(ReceiveChannel):
self._recv = receive_afunc or rx_chan.receive
self._closed: bool = False
self._raise_on_lag = raise_on_lag
self._wait_scope: trio.CancelScope|None = None
def receive_nowait(
self,
@ -237,7 +266,10 @@ class BroadcastReceiver(ReceiveChannel):
# https://docs.rs/tokio/1.11.0/tokio/sync/broadcast/index.html#lagging
mxln = state.maxlen
lost = seq - mxln
# `seq == mxln` is already one past the final
# valid deque index, so include that first
# displaced value in the loss count.
lost = seq - mxln + 1
# decrement to the last value and expect
# consumer to either handle the ``Lagged`` and come back
@ -255,8 +287,21 @@ class BroadcastReceiver(ReceiveChannel):
return self.receive_nowait(_key, _state)
state.subs[key] -= 1
state.cancelled.pop(key, None)
return value
receive_exc = state.receive_exc
if receive_exc is not None:
# Re-raising one shared exception mutates its traceback on
# every delivery. Give each receiver a stable wrapper while
# retaining the original failure as its cause.
raise BroadcastReceiveError(
'Shared broadcast receiver failed'
) from receive_exc
if state.eoc:
raise trio.EndOfChannel
raise trio.WouldBlock
async def _receive_from_underlying(
@ -270,14 +315,34 @@ class BroadcastReceiver(ReceiveChannel):
raise trio.ClosedResourceError
event = trio.Event()
recv_scope = trio.CancelScope()
assert state.recv_ready is None
assert state.recv_scope is None
state.recv_ready = key, event
state.recv_scope = recv_scope
try:
# if we're cancelled here it should be
# fine to bail without affecting any other consumers
# right?
receive_exc: BaseException|None = None
with recv_scope:
try:
value = await self._recv()
except BaseException as exc:
receive_exc = exc
# Only this receiver's `aclose()` cancels its private
# source-read scope, and it marks the receiver closed
# first without a checkpoint. Outer task cancellation does
# not set `recv_scope.cancel_called`; it remains a real
# `trio.Cancelled` and follows the handler below.
if recv_scope.cancel_called:
assert self._closed
if self._closed:
raise _BroadcastReceiverClosed
if receive_exc is not None:
raise receive_exc
# items with lower indices are "newer"
# NOTE: ``collections.deque`` implicitly takes care of
@ -303,6 +368,8 @@ class BroadcastReceiver(ReceiveChannel):
):
state.subs[sub_key] += 1
state.cancelled.pop(key, None)
# NOTE: this should ONLY be set if the above task was *NOT*
# cancelled on the `._recv()` call.
event.set()
@ -312,11 +379,20 @@ class BroadcastReceiver(ReceiveChannel):
# if any one consumer gets an EOC from the underlying
# receiver we need to unblock and send that signal to
# all other consumers.
state.cancelled.clear()
self._state.eoc = True
if event.statistics().tasks_waiting:
event.set()
raise
except _BroadcastReceiverClosed:
# `aclose()` cancelled this receiver's source-read scope.
# Wake peers so one of them can take ownership after this
# task clears `recv_ready` in `finally`.
if event.statistics().tasks_waiting:
event.set()
raise trio.ClosedResourceError
except (
trio.Cancelled,
):
@ -329,12 +405,33 @@ class BroadcastReceiver(ReceiveChannel):
event.set()
raise
except Exception as receive_exc:
# The underlying receiver is shared by every subscriber,
# so any non-EOC failure terminates the entire broadcast.
# Publish it before waking peers so they can drain their
# retained values and then observe the same failure.
state.cancelled.clear()
state.receive_exc = receive_exc
if event.statistics().tasks_waiting:
event.set()
raise
except BaseException:
# Process-control and cancellation-like exceptions must
# not become durable broadcast state, but peers still
# need waking before `recv_ready` is cleared.
state.cancelled.pop(key, None)
if event.statistics().tasks_waiting:
event.set()
raise
finally:
# Reset receiver waiter task event for next blocking condition.
# this MUST be reset even if the above ``.recv()`` call
# was cancelled to avoid the next consumer from blocking on
# an event that won't be set!
state.recv_ready = None
state.recv_scope = None
async def receive(self) -> ReceiveType:
key = self.key
@ -362,7 +459,23 @@ class BroadcastReceiver(ReceiveChannel):
# seq = state.subs[key]
# assert seq == -1 # sanity
_, ev = state.recv_ready
wait_scope = trio.CancelScope()
self._wait_scope = wait_scope
try:
with wait_scope:
await ev.wait()
# As with `recv_scope`, only this receiver's
# `aclose()` cancels its private peer-wait scope
# after marking the receiver closed. Outer task
# cancellation remains `trio.Cancelled`.
if wait_scope.cancel_called:
assert self._closed
if self._closed:
raise trio.ClosedResourceError
finally:
self._wait_scope = None
try:
return self.receive_nowait(
_key=key,
@ -440,18 +553,35 @@ class BroadcastReceiver(ReceiveChannel):
if self._closed:
return
# if there are sleeping consumers wake
# them on closure.
rr = self._state.recv_ready
if rr:
_, event = rr
event.set()
# XXX: leaving it like this consumers can still get values
# up to the last received that still reside in the queue.
self._state.subs.pop(self.key)
state = self._state
state.subs.pop(self.key)
state.cancelled.pop(self.key, None)
self._closed = True
# A non-owner close must not wake peers waiting behind some
# other receiver's source read. If this receiver owns that
# read, cancel only its private scope; the owner task wakes
# peers after cancellation is delivered and state is ready
# for a clean ownership handoff.
rr = state.recv_ready
if (
rr is not None
# `recv_ready[0]` identifies the receiver which currently
# owns the one shared source read. Only that receiver may
# cancel `BroadcastState.recv_scope`; closing any other
# subscriber must not disturb the owner or its peer tasks.
and
rr[0] == self.key
):
recv_scope = state.recv_scope
assert recv_scope is not None
recv_scope.cancel()
elif (wait_scope := self._wait_scope) is not None:
wait_scope.cancel()
def broadcast_receiver(
@ -462,6 +592,11 @@ def broadcast_receiver(
) -> BroadcastReceiver:
if max_buffer_size < 1:
raise ValueError(
'`max_buffer_size` must be greater than zero'
)
return BroadcastReceiver(
recv_chan,
state=BroadcastState(