Compare commits
75 Commits
6e2ce2ba2f
...
38d6688d2a
| Author | SHA1 | Date |
|---|---|---|
|
|
38d6688d2a | |
|
|
38aa92bc61 | |
|
|
7e9e39c5f8 | |
|
|
0a1431c933 | |
|
|
3146409baf | |
|
|
dd9d890921 | |
|
|
0b926210a3 | |
|
|
7c6bff50c5 | |
|
|
17d6879706 | |
|
|
2af5acde88 | |
|
|
5c374b183e | |
|
|
29ac9067ce | |
|
|
efce12e145 | |
|
|
0c3b21e9fe | |
|
|
6a7e06e39d | |
|
|
aa9c9a97ff | |
|
|
07326d2b3e | |
|
|
e876430787 | |
|
|
0402994572 | |
|
|
fa748980d5 | |
|
|
5b5235962b | |
|
|
d8962f2a8a | |
|
|
d73c3b55e0 | |
|
|
2029a154ed | |
|
|
194ce5e392 | |
|
|
64ee2eeb75 | |
|
|
62b574a3d4 | |
|
|
93227086b0 | |
|
|
4151b9569a | |
|
|
8d04af0d73 | |
|
|
a3cd448612 | |
|
|
4e27dcde48 | |
|
|
93322405a1 | |
|
|
359fe75ced | |
|
|
40be587ce2 | |
|
|
ca4582b003 | |
|
|
75385e448d | |
|
|
34d81adef3 | |
|
|
1691be96fd | |
|
|
36ad1f3dd0 | |
|
|
72441124c0 | |
|
|
0cbb850645 | |
|
|
8f0df0ff6f | |
|
|
f75c9cdeab | |
|
|
75cda1933c | |
|
|
f454cefe56 | |
|
|
d0cc06815f | |
|
|
ebf2258b4f | |
|
|
ac61d7a5bf | |
|
|
584ea4e9ad | |
|
|
16dd876b0c | |
|
|
1a9ce915f3 | |
|
|
0b63af020e | |
|
|
6e424d4696 | |
|
|
5724c0516a | |
|
|
0d6d7c2a63 | |
|
|
bd38204fde | |
|
|
340d506940 | |
|
|
64e820e18e | |
|
|
0a3b0efcc6 | |
|
|
634d914161 | |
|
|
451e0acf8a | |
|
|
3d02a8569e | |
|
|
bceca74eb7 | |
|
|
18faffcff7 | |
|
|
7554f90e59 | |
|
|
3705bbe594 | |
|
|
66ac7863b5 | |
|
|
089e158da9 | |
|
|
7987f6b8d7 | |
|
|
70c7e334a7 | |
|
|
62b729a106 | |
|
|
213a298aad | |
|
|
d1d3fc58bb | |
|
|
a2c8af558e |
|
|
@ -91,6 +91,10 @@ jobs:
|
|||
name: '${{ matrix.os }} Python${{ matrix.python-version }} spawn_backend=${{ matrix.spawn_backend }} tpt_proto=${{ matrix.tpt_proto }}'
|
||||
timeout-minutes: 16
|
||||
runs-on: ${{ matrix.os }}
|
||||
# Windows support is nascent: its full test suite remains
|
||||
# informational, while setup and the `import tractor` smoke below
|
||||
# are hard signals. Promote the test step to required once the
|
||||
# suite is green.
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
|
|
@ -98,6 +102,7 @@ jobs:
|
|||
os: [
|
||||
ubuntu-latest,
|
||||
macos-latest,
|
||||
windows-latest,
|
||||
]
|
||||
python-version: [
|
||||
'3.13',
|
||||
|
|
@ -118,10 +123,10 @@ jobs:
|
|||
'tcp',
|
||||
'uds',
|
||||
]
|
||||
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
|
||||
exclude:
|
||||
# don't do UDS run on macOS (for now)
|
||||
- os: macos-latest
|
||||
# UDS is POSIX-only; Windows has no `AF_UNIX` so the
|
||||
# backend is intentionally unavailable there.
|
||||
- os: windows-latest
|
||||
tpt_proto: 'uds'
|
||||
|
||||
steps:
|
||||
|
|
@ -150,7 +155,14 @@ jobs:
|
|||
- name: List deps tree
|
||||
run: uv tree
|
||||
|
||||
# hard signal for the Windows import-safety fix: `import
|
||||
# tractor` must succeed everywhere, and `HAS_UDS` reflects
|
||||
# platform capability (False on Windows, True on POSIX).
|
||||
- name: 'Smoke: import tractor'
|
||||
run: uv run python -c "import sys; import tractor; from tractor.ipc._uds import HAS_UDS; assert sys.platform != 'win32' or not HAS_UDS; print('import tractor OK | HAS_UDS=', HAS_UDS)"
|
||||
|
||||
- name: Run tests
|
||||
continue-on-error: ${{ matrix.os == 'windows-latest' }}
|
||||
run: >
|
||||
uv run
|
||||
pytest
|
||||
|
|
|
|||
|
|
@ -0,0 +1,460 @@
|
|||
# `_ria_nursery` removal plan (issue #477 follow-up)
|
||||
|
||||
Goal: drop the secondary "run-in-actor" spawn nursery (and
|
||||
friends) from `ActorNursery`/spawn internals, now that
|
||||
`tractor.to_actor.run()` delivers one-shot semantics purely on
|
||||
the daemon-spawn + portal primitives.
|
||||
|
||||
## Verified machinery map (2026-07-02, wkt @ a34aaf98)
|
||||
|
||||
The entire mechanism is 4 files:
|
||||
|
||||
- `runtime/_supervise.py`
|
||||
- `ActorNursery.__init__(.., ria_nursery, ..)` stores
|
||||
`._ria_nursery` (:202, :238); sole read is
|
||||
`run_in_actor()` passing `nursery=self._ria_nursery`
|
||||
(:442) into `start_actor()`'s `nursery:
|
||||
trio.Nursery|None` escape-hatch param (:305, :367).
|
||||
- `._cancel_after_result_on_exit: set` (:244) marks ria
|
||||
portals (:457).
|
||||
- `_open_and_supervise_one_cancels_all_nursery()` nests
|
||||
`da_nursery` (:609) around `ria_nursery` (:622); the
|
||||
`finally:` at the ria->da boundary (:747-766) raises
|
||||
collected `errors` (single exc or BEG).
|
||||
- `runtime/_portal.py`
|
||||
- `._expect_result_ctx` (:112) set by `_submit_for_result()`
|
||||
(:142, sole caller `run_in_actor()`); consumed by
|
||||
`wait_for_result()` (:167) + deprecated `result()` (:220).
|
||||
The `None` branch (:184-196) returns the `NoResult`
|
||||
sentinel (`_exceptions.py:1164`).
|
||||
- `spawn/_spawn.py`
|
||||
- `exhaust_portal()` (:129): awaits
|
||||
`portal.wait_for_result()`, CATCHES+RETURNS any exc
|
||||
(never raises).
|
||||
- `cancel_on_completion()` (:177): `exhaust_portal()` ->
|
||||
on exc-result stash `errors[uid] = result` (:203) ->
|
||||
ALWAYS `portal.cancel_actor()` (:218).
|
||||
- `spawn/_trio.py` (:195-222) + `spawn/_mp.py` (:187-213),
|
||||
identical shape: after shielded
|
||||
`await an._join_procs.wait()`, open a per-child local
|
||||
nursery; IFF `portal in an._cancel_after_result_on_exit`
|
||||
start `cancel_on_completion` alongside `soft_kill()`; when
|
||||
`soft_kill` returns first, `nursery.cancel_scope.cancel()`
|
||||
reaps the result-waiter.
|
||||
|
||||
## The load-bearing semantic (already-deferred errors)
|
||||
|
||||
Remote ria-child errors NEVER raise into `ria_nursery`:
|
||||
|
||||
1. reaper tasks only START after `_join_procs.set()` (block
|
||||
exit or the inner error handler),
|
||||
2. `exhaust_portal` swallows the exc into a return value,
|
||||
3. `cancel_on_completion` stashes it in `errors` + cancels
|
||||
that child,
|
||||
4. the ria->da `finally:` re-raises collected `errors` (and
|
||||
`an.cancel()`s any daemon stragglers).
|
||||
|
||||
So mid-block there is NO error propagation from ria children
|
||||
(unless user code explicitly `await portal.wait_for_result()`s)
|
||||
— the two-nursery nesting only sequences "reap ria results
|
||||
BEFORE blocking on daemon join". A single-nursery impl only
|
||||
needs to preserve that sequencing, not any ASAP-cancel
|
||||
behavior.
|
||||
|
||||
## Target design
|
||||
|
||||
### step A: single-nursery `run_in_actor()` (mechanical)
|
||||
|
||||
- `run_in_actor()` spawns via the DEFAULT (`_da_nursery`)
|
||||
path — drop `nursery=self._ria_nursery`.
|
||||
- rename `._cancel_after_result_on_exit` ->
|
||||
`._ria_portals: dict[portal, Actor]` (need the subactor ref
|
||||
for `cancel_on_completion`).
|
||||
- move reaper start-up OUT of the backends into
|
||||
`_open_and_supervise...`: immediately after EACH
|
||||
`an._join_procs.set()` call-site (happy path :642, inner
|
||||
error handler :661), start one
|
||||
`cancel_on_completion(portal, subactor, errors)` task per
|
||||
ria portal into `da_nursery`, then (happy path only)
|
||||
`await` their completion BEFORE falling out of the
|
||||
`try:`/`finally:` that raises `errors` — e.g. gather in a
|
||||
dedicated inner `trio.open_nursery()` block replacing
|
||||
today's `ria_nursery` join point.
|
||||
- delete the membership branch + local reaper nursery from
|
||||
`_trio.py`/`_mp.py` (keep the `soft_kill()` call; the
|
||||
per-child local nursery collapses to just `soft_kill`).
|
||||
- `_trio.py:310` `_children.pop()` etc. unchanged.
|
||||
|
||||
### step B: delete the plumbing
|
||||
|
||||
- `_open_and_supervise...`: drop the inner
|
||||
`ria_nursery` + merge its `except BaseException` classify
|
||||
logic into ONE handler on the (now single) nursery scope;
|
||||
`ActorNursery.__init__` loses the `ria_nursery` param.
|
||||
- `start_actor()` loses the `nursery:` escape-hatch param
|
||||
(the :302-304 TODO).
|
||||
- backends: no more `_cancel_after_result_on_exit` refs.
|
||||
|
||||
### step C: (separate PRs) deprecate + migrate + excise
|
||||
|
||||
- migrate in-repo `.run_in_actor()` usage to
|
||||
`to_actor.run()`: tests 46 hits/9 files (test_cancellation
|
||||
15, test_infected_asyncio 10, test_spawning 8, registrar 3,
|
||||
adv_streaming 4, pubsub 2, rpc 1, runtime 1), examples 28
|
||||
hits/13 files (debugging/* dominate), docs 20 hits/8 rst
|
||||
files. NOTE: many sites also use deprecated
|
||||
`Portal.result()`/`wait_for_result()` — these die with
|
||||
`_expect_result_ctx`, so migration must land FIRST.
|
||||
- add `DeprecationWarning` to `run_in_actor()` (+
|
||||
`_submit_for_result`/`wait_for_result`).
|
||||
- final excision: `run_in_actor()`, `_submit_for_result`,
|
||||
`_expect_result_ctx`, `wait_for_result`/`result`,
|
||||
`exhaust_portal`, `cancel_on_completion`, `NoResult`.
|
||||
|
||||
## Risk register
|
||||
|
||||
1. hard-killed ria child: today the backend-local
|
||||
`nursery.cancel_scope.cancel()` discards a still-parked
|
||||
reaper when the proc dies first; a da_nursery-hosted
|
||||
reaper instead sees the transport break ->
|
||||
`exhaust_portal` returns a `TransportClosed`-ish exc ->
|
||||
NEW entry in `errors` that today gets discarded. Guard:
|
||||
reap-gather block must cancel remaining reapers once all
|
||||
ria procs are dead, or filter transport-death excs for
|
||||
already-`cancel_called` children.
|
||||
2. error-path ordering: inner handler today sets
|
||||
`_join_procs` THEN `an.cancel()`; reapers race the
|
||||
cancel-RPC. Keep that ordering when moving reaper spawn.
|
||||
3. debugger interplay: `maybe_wait_for_debugger()` calls
|
||||
(:654, :730) must stay BEFORE any reap/cancel issuance.
|
||||
4. `errors` double-entry: local body error (:646) + child's
|
||||
relayed exc (via reaper) can both land for the same
|
||||
scenario -> BEG shape changes vs today? (today has the
|
||||
same dual-write sites; keep behavior identical.)
|
||||
5. mp backend parity: mirror every `_trio.py` edit in
|
||||
`_mp.py` (identical block).
|
||||
|
||||
## Step-A first-probe findings (2026-07-02, WIP in tree)
|
||||
|
||||
Step A is IMPLEMENTED (uncommitted):
|
||||
`run_in_actor()` spawns via da_nursery; new
|
||||
`_supervise._reap_ria_portals()` helper; reap awaited after
|
||||
happy-path `_join_procs.set()`; error-path runs reap
|
||||
CONCURRENT with `an.cancel()` in the shielded block;
|
||||
backends stripped of the membership branch + per-child
|
||||
reaper nursery (+ dead imports).
|
||||
|
||||
Probe history (trio backend):
|
||||
- `tests/test_to_actor.py` + `tests/test_spawning.py`:
|
||||
20/20 PASS — incl. all `run_in_actor()` result
|
||||
round-trips + `test_remote_error` (single erroring child,
|
||||
body re-raise -> inner error path).
|
||||
- FIRST attempt ran the error-path reap CONCURRENT with
|
||||
`an.cancel()` (mimicking the old backend-side race):
|
||||
`test_cancellation.py::test_multierror` (2 erroring ria
|
||||
children, body re-raises one) DEADLOCKED. Root cause per
|
||||
the sequencing fix below: reap + cancel must NOT race at
|
||||
this layer (suspected `._children` pop-during-iteration
|
||||
and/or double-cancel RPC wedge; not fully root-caused
|
||||
since the fix removes the race wholesale).
|
||||
- FIX (2nd attempt, current impl): error path SEQUENCES:
|
||||
(1) snapshot ria `(portal, subactor)` pairs (backend
|
||||
`finally`s pop `._children` as procs reap), (2)
|
||||
`await an.cancel()`, (3) bounded reap over the snapshot.
|
||||
Bound was first 3s -> blew the `fail_after` deadline in
|
||||
`test_cancel_while_childs_child_in_sync_sleep` (hard-
|
||||
killed grandchild never relays => reaper parks the full
|
||||
bound). Tightened to 0.5s: anything collectable is
|
||||
already queued in the local ctx (relayed BEFORE the
|
||||
cancel); a parked reaper self-cleans (`trio.Cancelled`
|
||||
results are never stashed).
|
||||
- RESULT: `tests/test_cancellation.py` FULLY GREEN
|
||||
(20 passed, 1 xfailed, 77s); full-suite gate run kicked
|
||||
off same session (see final report/next session).
|
||||
|
||||
Remaining risk: on slow CI a relayed-but-undelivered error
|
||||
racing the 0.5s bound could drop an `errors` entry
|
||||
(BEG-shape flake); if observed, scale the bound via the
|
||||
`cpu_perf_headroom()`-style approach or peek
|
||||
`Portal._final_result_msg`/ctx queue state instead of
|
||||
time-bounding.
|
||||
|
||||
## Step-B outcome (2026-07-02, done in tree)
|
||||
|
||||
Step A landed as `5cd190c5` (code) + `99310269` (docs).
|
||||
Step B implemented on top (uncommitted):
|
||||
|
||||
- `._ria_nursery` is GONE — the inner
|
||||
`async with (collapse_eg(), trio.open_nursery() as
|
||||
ria_nursery)` layer in
|
||||
`_open_and_supervise_one_cancels_all_nursery` is deleted;
|
||||
`da_nursery` is now the single nursery for ALL subactors.
|
||||
- `ActorNursery.__init__` drops the `ria_nursery` param +
|
||||
the `self._ria_nursery` attr; `start_actor()` drops its
|
||||
`nursery=` escape-hatch param (uses `self._da_nursery`
|
||||
directly).
|
||||
- `._cancel_after_result_on_exit` STAYS — it's the
|
||||
ria-child discriminator for `_reap_ria_portals()`.
|
||||
|
||||
Deliberately NOT done (deferred to its own higher-risk PR,
|
||||
flagged with a TODO at the outer `except`): merging the two
|
||||
error handlers into one. Rationale — collapsing the empty
|
||||
nursery is provably behavior-preserving (a zero-task
|
||||
`trio.open_nursery()` only adds a checkpoint), whereas the
|
||||
inner `except BaseException` (swallow-into-`errors`) and
|
||||
outer `except (...)` (re-raise, safety-net for the inner
|
||||
handler's own non-shielded awaits) have DIFFERENT
|
||||
semantics; merging changes error/cancel propagation and
|
||||
wants isolated review + its own gate. Both handlers are
|
||||
kept, now nested directly under the single nursery.
|
||||
|
||||
Why the collapse is safe: post-step-A NOTHING spawns into
|
||||
`ria_nursery` (its only reader, `run_in_actor`'s
|
||||
`nursery=self._ria_nursery`, was removed in A; the stored
|
||||
attr was never read again). So the layer was pure dead
|
||||
weight.
|
||||
|
||||
Gate (trio backend, all 0-failure):
|
||||
- targeted set (`test_cancellation test_spawning test_local
|
||||
test_rpc test_to_actor`) = 49 passed, 1 xfailed.
|
||||
- tail set (`test_reg_err_types remote_exc_relay
|
||||
resource_cache ringbuf root_infect_asyncio root_runtime
|
||||
runtime shm task_broadcasting trioisms trionics/`) = 63
|
||||
passed, 1 skipped, 5 xfailed.
|
||||
- full-suite head ~73% (subdirs + `test_2way`..`test_pubsub`)
|
||||
= 303 passed before the known-flaky `test_dynamic_pub_sub`
|
||||
TooSlowError stall (pre-existing; same hang in the step-A
|
||||
full run). Suite ran slow this session (~13min vs 555s
|
||||
cold, likely thermal from back-to-back runs), never
|
||||
completing within an 800s bound — but split across the
|
||||
above three runs EVERY module passed under step B.
|
||||
|
||||
## Step-B2 outcome (2026-07-02, done in tree)
|
||||
|
||||
Step B committed as `9201a2ed` (code) + `d2e812fb` (docs), then
|
||||
branched to `drop_ria_nursery`. Step B2 (the deferred
|
||||
handler-merge) implemented on top (uncommitted):
|
||||
|
||||
- the two nested handlers in
|
||||
`_open_and_supervise_one_cancels_all_nursery` collapse to
|
||||
ONE `except BaseException as _scope_err` + the existing
|
||||
`finally`. The `outer_err`/`inner_err` locals go away.
|
||||
|
||||
Why it's safe (trace, not hope): the OLD inner handler records
|
||||
`errors[actor.aid.uid]` as its FIRST statement (before any
|
||||
await). So whenever an error path runs, `errors` is non-empty.
|
||||
The OLD outer handler was only reachable via leakage from the
|
||||
inner handler (it catches `BaseException`, so nothing from the
|
||||
`yield` scope bypasses it) — and by then `errors` is already
|
||||
populated, so the `finally`'s `raise` from `errors` ALWAYS
|
||||
superseded the outer handler's own `raise`. i.e. the outer
|
||||
`raise` was dead. The outer handler's other effects
|
||||
(`_scope_error`, a 2nd debugger-wait, child-cancel) are
|
||||
redundant with the merged handler + `finally`. So one handler
|
||||
+ `finally` is observably equivalent.
|
||||
|
||||
Residual nuance (accepted): in the rare "`trio.Cancelled`
|
||||
delivered during the non-shielded `maybe_wait_for_debugger`"
|
||||
path, the merged form may leave `_cancel_called` False (cancel
|
||||
happens after the wait), so `open_nursery`'s tb-hiding guard
|
||||
(`not cancel_called and _scope_error`) can show a tb it
|
||||
previously hid. More informative, not less; no test asserts on
|
||||
it.
|
||||
|
||||
Gate (box ran ~2.7x slow this session, load-induced
|
||||
`TooSlowError` flakiness on timing tests — NOT code; see
|
||||
[[env_cpu_throttle_masquerades_as_regression]]):
|
||||
- baseline (pre-B2 tip `9201a2ed`) full suite
|
||||
(`-k 'not dynamic_pub_sub'`) = 300 passed + 1
|
||||
`test_ext_types_over_ipc` `TooSlowError` that passes 6/6 in
|
||||
isolation (4.89s).
|
||||
- B2 error/cancel gate (`test_cancellation remote_exc_relay
|
||||
inter_peer_cancellation advanced_faults oob_cancellation
|
||||
to_actor spawning local rpc`) = 71 passed, 1 xfailed
|
||||
(125s).
|
||||
- B2 full-suite run: see `b2_full.log` (result appended on
|
||||
completion). RECOMMEND a clean full-suite run on a
|
||||
normal-speed box before this merges.
|
||||
|
||||
## Regression + fix: ria-reap hang (2026-07-02)
|
||||
|
||||
Human hit a full-suite hang on
|
||||
`test_infected_asyncio.py::test_tractor_cancels_aio`. Bisected:
|
||||
passes at pre-ria `a34aaf98` (0.59s), hangs at B2 `e617b498`
|
||||
(90s+). Root-caused to the STEP-A reaper hoist (`5cd190c5`),
|
||||
NOT B2 (`_reap_ria_portals` is byte-identical A->B2).
|
||||
|
||||
Bug: the test does `run_in_actor(asyncio_actor)` then a USER
|
||||
`portal.cancel_actor()` and exits the block cleanly -> the
|
||||
happy path's `await _reap_ria_portals()`, which waits UNBOUNDED
|
||||
on `cancel_on_completion -> wait_for_result()`. The child was
|
||||
cancelled out-of-band so no final result is relayed -> parked
|
||||
forever. The OLD spawn-backend reaper was raced against
|
||||
`soft_kill()` (per-child nursery `cancel_scope.cancel()` on
|
||||
subproc death); the hoist dropped that race.
|
||||
|
||||
Fix: `_reap_ria_portals()` runs each `cancel_on_completion()`
|
||||
in a local nursery alongside a `proc.poll()` death-watch that
|
||||
cancels the parked reaper once the subproc exits — restoring
|
||||
the old race, backend-agnostic (guarded by
|
||||
`hasattr(proc, 'poll')` for a future `subint` handle).
|
||||
|
||||
Why POLL (`proc.poll()`) not the event-driven `wait_func`:
|
||||
the mp waiter (`_spawn.proc_waiter`) does
|
||||
`wait_readable(proc.sentinel)`, and `soft_kill()` is ALREADY
|
||||
awaiting that same fd concurrently in the daemon nursery — a
|
||||
2nd `wait_readable` on one fd raises `trio.BusyResourceError`.
|
||||
(`trio.Process.wait()` IS multi-waiter-safe, but mp has no
|
||||
async equivalent.) `proc.poll()` — the same liveness check
|
||||
`soft_kill` itself falls back to — is the conflict-free common
|
||||
denominator. Verified: poll-fix passes on BOTH trio and
|
||||
mp_spawn.
|
||||
|
||||
Also added a per-test anti-hang guard: wrapped
|
||||
`test_tractor_cancels_aio`'s `main()` in
|
||||
`with trio.fail_after(9 * cpu_perf_headroom())` — the blessed
|
||||
pattern (`pytest-timeout`'s global cap is intentionally off;
|
||||
breaks trio under fork backends, see `pyproject` NOTE). So a
|
||||
future recurrence FAILS FAST instead of hanging the suite.
|
||||
(Several other tests in the file are still guardless —
|
||||
`test_aio_simple_error`, `test_trio_error_cancels_intertask_chan`,
|
||||
`test_aio_errors_and_channel_propagates_and_closes` — candidate
|
||||
follow-up sweep.)
|
||||
|
||||
Lesson: the B2 focused gate OMITTED `test_infected_asyncio`
|
||||
(and the full runs were clipped/slow), so the step-A hang
|
||||
slipped through. Any future ria-touching change MUST gate
|
||||
`test_infected_asyncio` explicitly.
|
||||
|
||||
Gate: `test_tractor_cancels_aio` green (trio 1.53s, mp 3.98s);
|
||||
fix gate (`test_infected_asyncio test_cancellation test_to_actor
|
||||
test_spawning`) = 74 passed, 3 xfailed, 0 failures.
|
||||
|
||||
## PAUSED (2026-07-02): re-assess the reaper's SCOPE
|
||||
|
||||
User's insight (compelling — likely the real root cause of
|
||||
the hang, not just the missing proc-death race):
|
||||
|
||||
> the "hoisting" of 5cd190c5 was just not really done right
|
||||
> — the hoist should have been into the `to_actor` scope,
|
||||
> not `_supervise`.
|
||||
|
||||
The argument: `.run_in_actor()`'s result-waiting/reaping got
|
||||
hoisted into `_supervise._reap_ria_portals` (nursery-machinery
|
||||
scope), which has NO natural cancel-scope to bound a parked
|
||||
`wait_for_result()` — hence the awkward proc-death race +
|
||||
the poll-vs-`proc_waiter` dilemma. If the result-wait instead
|
||||
lived in the `to_actor` one-shot scope
|
||||
(`to_actor._invoke_in_subactor()`), it would sit right next to
|
||||
the caller's `an` + a local `trio` task-nursery + cancel-scope
|
||||
(the `trio.to_thread`-style model #477 actually wants) — so
|
||||
bounding/cancelling the wait is trivial and the hang
|
||||
dissolves from correct scoping rather than a bolt-on race.
|
||||
|
||||
Follow-on to re-evaluate on resume:
|
||||
- should `_reap_ria_portals` exist AT ALL, or should
|
||||
result-waiting move entirely into
|
||||
`to_actor._invoke_in_subactor()`?
|
||||
- reimplement legacy `run_in_actor()` on top of
|
||||
`to_actor.run()` so `_reap_ria_portals` +
|
||||
`_cancel_after_result_on_exit` can be DROPPED from
|
||||
`_supervise` entirely (the true #477 simplification)?
|
||||
- the poll-vs-event decision is MOOT under this re-scoping.
|
||||
|
||||
State at pause: `test_infected_asyncio` anti-hang guard
|
||||
COMMITTED (`d1fb4a1a`, intentionally red w/o the fix — the
|
||||
user's failing-test-first convention). The poll-based reap
|
||||
fix in `_supervise.py` is UNCOMMITTED and likely SUPERSEDED
|
||||
by the re-scoping — do NOT land it as-is.
|
||||
|
||||
## RESOLVED (2026-07-06): migrate everything, remove the API
|
||||
|
||||
The PAUSED re-assessment concluded decisively: rather than
|
||||
re-scope `_reap_ria_portals` (or bolt any hack onto it), the
|
||||
`run_in_actor()` API itself was REMOVED — its non-blocking
|
||||
"result at teardown" semantic predates streaming and confused
|
||||
more than it served. Every in-repo caller was migrated
|
||||
per-file/-group (each its own commit, each gated):
|
||||
|
||||
- tests: `test_infected_asyncio` `test_runtime` `test_rpc`
|
||||
`test_spawning` `test_pubsub` `test_registrar`
|
||||
`test_cancellation` (3 groups) `test_advanced_streaming`.
|
||||
- examples: 4 non-debugging + all 8 `debugging/` REPL scripts
|
||||
(debugger suite byte-identical green, 28p/6s).
|
||||
- docs: 8 rst pages + the `experimental/_pubsub` docstring.
|
||||
|
||||
Migration patterns (the `run_in_actor` shape -> successor):
|
||||
|
||||
- blocking result -> `to_actor.run(fn, an=an, ...)`
|
||||
- fire-&-forget/forever -> bg `to_actor.run()` task in a local
|
||||
`trio` task-nursery (or `start_actor`
|
||||
+ bg `Portal.run()` when a portal
|
||||
handle is needed)
|
||||
- concurrent fan-out -> N bg `to_actor.run()` tasks / or
|
||||
`gather_contexts([p.open_context(..)])`
|
||||
- reap-all-error-collect -> the "collect don't cancel" pattern:
|
||||
each one-shot catches + stashes its
|
||||
`RemoteActorError`, group raised
|
||||
after the task-nursery joins (see
|
||||
`examples/debugging/multi_subactors.py`)
|
||||
- mutual-rendezvous -> peers must OUTLIVE both dialogs:
|
||||
`start_actor()` daemons + concurrent
|
||||
`Portal.run()`s + explicit
|
||||
`an.cancel()` (eager one-shot reap
|
||||
races the slower peer's dial of the
|
||||
winner's dead sockaddr; found via
|
||||
`test_trynamic_trio` flake).
|
||||
|
||||
Semantic deltas (tests loosened accordingly):
|
||||
|
||||
- teardown-reap-all BEG-of-N is GONE: local task-nurseries are
|
||||
cancel-on-first, raced siblings' `Cancelled`s are absorbed,
|
||||
and the runtime's `collapse_eg()` unwraps every single-member
|
||||
group at each actor boundary — a fully-raced nested tree
|
||||
relays a bare (annotated) `RemoteActorError` chain.
|
||||
- `test_multierror_fast_nursery` deleted (pure reap-stress);
|
||||
`test_nested_multierrors` re-purposed as deep-tree
|
||||
cancel-cascade stress w/ a race-tolerant shape walk.
|
||||
|
||||
Final excision (after zero callers remained): `run_in_actor()`,
|
||||
`._cancel_after_result_on_exit`, `_reap_ria_portals()`,
|
||||
`Portal._submit_for_result/._expect_result_ctx/
|
||||
.wait_for_result()/.result()`, `exhaust_portal()`,
|
||||
`cancel_on_completion()`, `NoResult` — net -402 lines. The
|
||||
reap-hang class (unbounded `wait_for_result` in machinery
|
||||
scope) dissolves structurally: the only result-wait left lives
|
||||
in the caller's task inside its own cancel-scope; the
|
||||
`d1fb4a1a` anti-hang guard test passes by construction. The
|
||||
poll-vs-`proc_waiter` debate is moot as predicted.
|
||||
|
||||
## Follow-up sketch: `to_actor.open_one_shot()` (run-async parity)
|
||||
|
||||
If deferred-result parity is ever wanted, the design that needs
|
||||
NO runtime coupling, NO returned `Portal` and NO cancel-relay
|
||||
`trio.Event` machinery:
|
||||
|
||||
async with to_actor.open_one_shot(
|
||||
fn, an=an, **kws,
|
||||
) as one_shot:
|
||||
... # concurrent caller work
|
||||
val = await one_shot.wait() # optional; errors always
|
||||
# propagate at scope exit
|
||||
|
||||
an `@acm` that opens a private task-nursery, `start_soon`s ONE
|
||||
task running the existing blocking `run()` and stashes the
|
||||
value in a slot + sets a done-`trio.Event` (a memo, not a
|
||||
cancel relay). Cancellation = plain scope-cancel of the acm's
|
||||
nursery (the parked `Portal.run()` unwinds via `Cancelled`, the
|
||||
shielded `cancel_actor()` reap still runs); a child error
|
||||
raises into the acm scope so an un-`wait()`ed one-shot can
|
||||
never silently drop its error. i.e. the old reaper's job is
|
||||
done by scoping, not machinery. ~40 lines, all in
|
||||
`to_actor/_api.py`, zero `_supervise` involvement.
|
||||
|
||||
## Verification gate
|
||||
|
||||
- per-migration-commit module gates on `trio` (+ `mp_spawn`
|
||||
spot-gates incl. `test_infected_asyncio` per the B2 lesson);
|
||||
`tests/devx/test_debugger.py` for the REPL flows.
|
||||
- full suite on `trio` + `mp_spawn` at branch tip + CI matrix
|
||||
via draft PR #484.
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
session: f6c84722-471a-4458-9a80-e453fea9029f
|
||||
timestamp: 2026-07-02T15:42:55Z
|
||||
git_ref: 65bf9df5
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260702T154255Z_65bf9df5_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Driver prompt file `ai/prompt-io/prompts/issue_477.md`:
|
||||
|
||||
> attempt to resolve
|
||||
> https://github.com/goodboy/tractor/issues/477
|
||||
> do it with /open-wkt.
|
||||
|
||||
(plus a hard stop-for-human-review deadline of 12:50PM
|
||||
EST the same day)
|
||||
|
||||
Issue #477 asks to factor `ActorNursery.run_in_actor()`
|
||||
(and possibly `Portal.run()`) out of the nursery
|
||||
internals into a new `tractor.to_actor` wrapper
|
||||
subpackage of "higher level one shot" single-remote-task
|
||||
APIs, adopting the `trio.to_thread`/`anyio.to_process`
|
||||
parlance, so that error collection/propagation moves up
|
||||
into the caller's local `trio` scope and the nursery's
|
||||
spawn machinery can eventually drop the
|
||||
`._ria_nursery` coupling.
|
||||
|
||||
## Response summary
|
||||
|
||||
First-cut `tractor.to_actor` subpkg delivering the
|
||||
one-shot API composed purely from the existing
|
||||
daemon-spawn + portal primitives (`start_actor()` +
|
||||
`Portal.run()` + `Portal.cancel_actor()`), leaving the
|
||||
legacy `.run_in_actor()` machinery untouched (formal
|
||||
deprecation deferred until in-repo usage migrates):
|
||||
|
||||
- `to_actor.run(fn, **fn_kwargs) -> Any`: spawn a
|
||||
subactor, schedule `fn` as its lone remote task, wait
|
||||
on and return its result, ALWAYS reaping the subactor
|
||||
(shield-safe `finally`). Remote errors raise in the
|
||||
caller's task as boxed `RemoteActorError`s.
|
||||
- placement variants: `portal=` reuses a running actor
|
||||
(no spawn/reap), `an=` spawns from a caller-managed
|
||||
actor-nursery, neither opens a call-scoped private
|
||||
`open_nursery()` (implicitly booting the runtime,
|
||||
configurable via `runtime_kwargs`).
|
||||
- fail-fast validation before any spawn: non-streaming
|
||||
async fn required; `portal=`/`an=` mutually
|
||||
exclusive; `runtime_kwargs` rejected alongside any
|
||||
placement opt.
|
||||
- `run_in_actor()` TODO/docstring now cross-reference
|
||||
the successor API.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/to_actor/__init__.py` — new subpkg,
|
||||
re-exports `run`
|
||||
- `tractor/to_actor/_api.py` — `run()` +
|
||||
`_invoke_in_subactor()` + `_validate_one_shot_fn()`
|
||||
- `tractor/__init__.py` — top-level `to_actor`
|
||||
re-export
|
||||
- `tractor/runtime/_supervise.py` — comment/docstring
|
||||
pointers from `run_in_actor()` to the successor
|
||||
- `tests/test_to_actor.py` — 11-test suite covering
|
||||
all placement variants, error relay, the concurrent
|
||||
worker-pool-ish pattern and arg validation
|
||||
- `examples/parallelism/to_actor_one_shots.py` —
|
||||
runnable demo (auto-collected by
|
||||
`test_docs_examples.py`)
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — pending human review (work paused before the
|
||||
12:50PM EST deadline per the driver prompt).
|
||||
|
|
@ -0,0 +1,100 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
timestamp: 2026-07-02T15:42:55Z
|
||||
git_ref: 65bf9df5
|
||||
diff_cmd: git diff main..wkt/to_actor_subpkg
|
||||
---
|
||||
|
||||
# Raw AI output (diff-ref mode)
|
||||
|
||||
All generated code is committed on the
|
||||
`wkt/to_actor_subpkg` branch; per diff-ref mode each
|
||||
file's verbatim content is reachable via the pointers
|
||||
below rather than duplicated here.
|
||||
|
||||
## Generated files
|
||||
|
||||
> `git diff main..wkt/to_actor_subpkg -- tractor/to_actor/__init__.py`
|
||||
|
||||
New subpackage init: module docstring establishing the
|
||||
`trio.to_thread`/`anyio.to_process` "run it over there"
|
||||
parlance for actors, plus the single public re-export
|
||||
`run as run` from `._api`.
|
||||
|
||||
> `git diff main..wkt/to_actor_subpkg -- tractor/to_actor/_api.py`
|
||||
|
||||
The one-shot invocation impl, composed entirely from the
|
||||
lower level daemon-spawn + portal primitives as
|
||||
prescribed by issue #477:
|
||||
|
||||
- `_validate_one_shot_fn()`: the `Portal.run()`
|
||||
non-streaming-async-fn constraint checked up-front,
|
||||
before any subactor is spawned.
|
||||
- `_invoke_in_subactor()`: `an.start_actor()` ->
|
||||
`Portal.run()` -> always-reap via
|
||||
`Portal.cancel_actor()` in a `finally` (the cancel
|
||||
req's bounded wait is internally shielded so the reap
|
||||
also runs under caller-scope cancellation).
|
||||
- `run()`: the public API. Placement options:
|
||||
`portal=` (reuse a running actor, no spawn/reap),
|
||||
`an=` (spawn from a caller-managed nursery), or
|
||||
neither (private `open_nursery()` scoped to the call,
|
||||
implicitly booting the runtime when needed, tunable
|
||||
via pass-through `runtime_kwargs`). Spawn opts mirror
|
||||
`ActorNursery.start_actor()`; `**fn_kwargs` are
|
||||
relayed to the remote task. Errors raise in the
|
||||
caller's task as boxed `RemoteActorError`s.
|
||||
`runtime_kwargs` alongside any placement opt is a
|
||||
hard `ValueError`, never silently ignored.
|
||||
|
||||
> `git diff main..wkt/to_actor_subpkg -- tractor/__init__.py`
|
||||
|
||||
Top-level `from . import to_actor as to_actor`
|
||||
re-export.
|
||||
|
||||
> `git diff main..wkt/to_actor_subpkg -- tractor/runtime/_supervise.py`
|
||||
|
||||
Comment/docstring-only: the `run_in_actor()` deprecation
|
||||
TODO now points at the implemented `.to_actor.run()`
|
||||
successor (checkbox ticked) and the method docstring
|
||||
gains a NOTE steering users to the new API; remaining
|
||||
TODO items are the `DeprecationWarning` emission +
|
||||
in-repo usage migration.
|
||||
|
||||
> `git diff main..wkt/to_actor_subpkg -- tests/test_to_actor.py`
|
||||
|
||||
11-test suite: private-nursery one-shot, implicit
|
||||
runtime boot via `runtime_kwargs`, remote-error relay to
|
||||
the caller's task (bare + caller-managed nursery),
|
||||
caller-nursery spawn, portal reuse w/o implicit reap,
|
||||
the concurrent worker-pool-ish pattern (local `trio`
|
||||
nursery x shared `an`), and the four validation
|
||||
rejections (sync fn, async-gen fn, `portal`+`an`
|
||||
combo, `runtime_kwargs`+placement combo).
|
||||
|
||||
> `git diff main..wkt/to_actor_subpkg -- examples/parallelism/to_actor_one_shots.py`
|
||||
|
||||
Runnable example (auto-collected by
|
||||
`test_docs_examples.py`): the fully-implicit one-shot
|
||||
plus the concurrent worker-pool-ish prime-check pattern
|
||||
against a shared caller-managed actor-nursery.
|
||||
|
||||
## Test runs (verbatim)
|
||||
|
||||
```
|
||||
tests/test_to_actor.py .......... [100%]
|
||||
============= 10 passed in 4.29s =============
|
||||
```
|
||||
|
||||
Regression subset for touched modules
|
||||
(`test_local.py test_rpc.py test_spawning.py
|
||||
test_cancellation.py`):
|
||||
|
||||
```
|
||||
38 passed, 1 xfailed, 24 warnings in 80.71s (0:01:20)
|
||||
```
|
||||
|
||||
(warnings are pre-existing stdlib `os.fork()`
|
||||
DeprecationWarnings from the mp spawn backends, not
|
||||
introduced by this change)
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
session: b8ec2ffe-1e47-4db5-8bc3-5bebc5feaaf6
|
||||
timestamp: 2026-07-02T15:50:06Z
|
||||
git_ref: 65bf9df5
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260702T155006Z_65bf9df5_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
From `ai/prompt-io/prompts/issue_473.md`:
|
||||
|
||||
> NOTE: you MUST pause this work at 12:50PM EST (BEFORE
|
||||
> your weekly limit reset) for review by a human!
|
||||
>
|
||||
> resolve GH #473: https://github.com/goodboy/tractor/issues/474
|
||||
> do it with /open-wkt.
|
||||
|
||||
(The prompt's issue-number/URL mismatch was resolved in
|
||||
favor of #473 — file name, session name and prompt text
|
||||
all say #473; the `/474` URL is a typo.)
|
||||
|
||||
GH #473: "Root-cause UDS-on-macOS —
|
||||
`uds_transport_actor_tree` example fails on CI".
|
||||
|
||||
## Response summary
|
||||
|
||||
Worked all four #473 next-step checkboxes in wkt
|
||||
`uds_macos_473`:
|
||||
|
||||
1. **stderr surfacing**: `test_docs_examples.py` now
|
||||
raises with the full subproc stderr (+stdout) on ANY
|
||||
non-zero exit. The old last-line-'Error' check could
|
||||
never fire for `tractor` crashes since root-actor
|
||||
stderr always ends with the strict-EG collapse note
|
||||
`( ^^^ this exc was collapsed from a group ^^^ )` —
|
||||
proven against the real PR #460 macOS CI log (bare
|
||||
`assert 1 == 0`, no traceback).
|
||||
2. **root-cause (linux-provable layer)**: macOS-only
|
||||
addr corruption in
|
||||
`MsgpackUDSStream.get_stream_addrs()` — no
|
||||
`SO_PASSCRED`/autobind on darwin means the accept
|
||||
side's `getpeername()` is `''`, and the
|
||||
`(str(), str())` arm took `peername` unconditionally →
|
||||
`Path('')` garbage addrs on every accepted conn.
|
||||
Proven + fixed via linux no-autobind simulation.
|
||||
Possibly not the final macOS crasher (non-fatal on
|
||||
linux-sim); the diagnostic patch guarantees the next
|
||||
macOS CI run shows any remaining layer.
|
||||
3. **CI matrix**: removed the `macos-latest`+`uds`
|
||||
exclude.
|
||||
4. **un-skip**: dropped the macOS+CI skip of the example.
|
||||
|
||||
Also: `start_listener()` bindspace mkdir hardened
|
||||
(`parents=True, exist_ok=True`), example docstring
|
||||
peer-pid mechanism corrected for macOS.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tests/test_docs_examples.py` — surface full stderr on
|
||||
non-zero exit; remove macOS skip of the UDS example
|
||||
- `tractor/ipc/_uds.py` — fix no-autobind
|
||||
`get_stream_addrs()` arm; harden bindspace mkdir;
|
||||
document autobind semantics
|
||||
- `.github/workflows/ci.yml` — add macOS UDS leg (remove
|
||||
matrix exclude)
|
||||
- `examples/uds_transport_actor_tree.py` — platform-
|
||||
correct peer-pid docs
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — pending human review (hard-stop review
|
||||
checkpoint at 12:50PM EST per prompt); commit staging and
|
||||
any edits are the human's.
|
||||
|
|
@ -0,0 +1,101 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
timestamp: 2026-07-02T15:50:06Z
|
||||
git_ref: 65bf9df5
|
||||
diff_cmd: git diff main..wkt/uds_macos_473
|
||||
---
|
||||
|
||||
# Raw output — GH #473 UDS-on-macOS root-cause session
|
||||
|
||||
NOTE: code output is diff-referenced (not copied) per
|
||||
prompt-io diff-ref mode; all generated code is on branch
|
||||
`wkt/uds_macos_473` relative to `main` (65bf9df5).
|
||||
|
||||
## Diagnostic narrative (verbatim)
|
||||
|
||||
Root-cause work proceeded by linux-side simulation since no
|
||||
macOS host is available:
|
||||
|
||||
1. Pulled the actual macOS CI failure log from PR #460 run
|
||||
28343878933 (job 84115107992): the failure is a bare
|
||||
`assert 1 == 0` on `proc.returncode` with NO stderr
|
||||
surfaced — the harness only re-raised when the LAST
|
||||
stderr line contained 'Error'.
|
||||
|
||||
2. Reproduced the stderr-swallow mechanism on linux: any
|
||||
`tractor` root-actor crash ends stderr with the
|
||||
strict-EG collapse note
|
||||
`( ^^^ this exc was collapsed from a group ^^^ )` which
|
||||
never matches 'Error' — so EVERY possible crash was
|
||||
swallowed. (Verified by sabotaging the runtime dir via
|
||||
an over-long `XDG_RUNTIME_DIR` → `OSError: AF_UNIX path
|
||||
too long` → rc=1 + swallowed.)
|
||||
|
||||
3. Found + proved a macOS-only addr-corruption bug in
|
||||
`MsgpackUDSStream.get_stream_addrs()`: the
|
||||
`(str(), str())` match-arm unconditionally took
|
||||
`peername`, but on no-autobind platforms (macOS lacks
|
||||
linux's `SO_PASSCRED`-triggered autobind) the accept
|
||||
side's `getpeername()` is `''` → `Path('')` garbage
|
||||
laddr/raddr on EVERY accepted UDS conn. Simulated on
|
||||
linux by nulling `SO_PASSCRED` (no autobind → same `''`
|
||||
shape): pre-fix the example printed
|
||||
`listener sock file: .`; post-fix it prints the real
|
||||
registry sockpath. Non-fatal on linux-sim (rc=0), so
|
||||
possibly not the final macOS crasher — the diagnostic
|
||||
patch guarantees the next macOS CI run reveals any
|
||||
remaining layer.
|
||||
|
||||
4. Falsified the missing-parent-dir theory:
|
||||
`get_rt_dir()` already `mkdir(parents=True,
|
||||
exist_ok=True)`s at import (and macOS TCP CI passes),
|
||||
so `~/Library/Caches/TemporaryItems` absence cannot be
|
||||
the crasher. Hardened `start_listener()`'s bindspace
|
||||
mkdir anyway (custom `filedir` case + racing actors).
|
||||
|
||||
## Generated changes (diff pointers)
|
||||
|
||||
> `git diff main..wkt/uds_macos_473 -- tests/test_docs_examples.py`
|
||||
|
||||
- always raise with FULL subproc stderr (+stdout) on any
|
||||
non-zero example exit; keep legacy last-line 'Error'
|
||||
check for zero-rc cases; drop the macOS+CI skip of
|
||||
`uds_transport_actor_tree.py` (GH #473 next-step).
|
||||
|
||||
> `git diff main..wkt/uds_macos_473 -- tractor/ipc/_uds.py`
|
||||
|
||||
- `get_stream_addrs()`: document the autobind semantics
|
||||
(bytes = linux abstract-ns autobind artifact), add
|
||||
no-autobind `(str, str)` arm picking the non-empty name
|
||||
(`peername` connect-side, `sockname` accept-side) with
|
||||
an empty-pair `ValueError` guard.
|
||||
- `start_listener()`: `bs.mkdir(parents=True,
|
||||
exist_ok=True)`.
|
||||
|
||||
> `git diff main..wkt/uds_macos_473 -- .github/workflows/ci.yml`
|
||||
|
||||
- remove the `macos-latest`+`uds` matrix exclude so
|
||||
UDS-on-macOS is exercised by CI (GH #473 next-step).
|
||||
|
||||
> `git diff main..wkt/uds_macos_473 -- examples/uds_transport_actor_tree.py`
|
||||
|
||||
- docs nit: peer-pid mechanism is `SO_PEERCRED` on linux,
|
||||
`LOCAL_PEERPID` on macOS.
|
||||
|
||||
## Verification (verbatim summary)
|
||||
|
||||
- macOS-shape sim (no autobind) + fix: example rc=0 with
|
||||
correct listener sockpath.
|
||||
- native linux post-fix: example rc=0, autobind arms
|
||||
unchanged.
|
||||
- sabotage smoke-test: diagnostic patch surfaces the full
|
||||
EG traceback incl. collapse-note line.
|
||||
- `pytest tests/ipc/ tests/test_2way.py tests/discovery/
|
||||
--tpt-proto uds`: 61 passed, 1 `TooSlowError` flake
|
||||
(test_simple_rpc) that passes solo in 2.2s — attributed
|
||||
to a concurrent full-suite run (other session) loading
|
||||
the host.
|
||||
- full `tests/test_docs_examples.py` run pending at
|
||||
raw-file write time (queued behind the concurrent
|
||||
session's suite).
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
session: b6b42e23-0454-4d9b-be1c-0a89d65aed1b
|
||||
timestamp: 2026-07-02T15:56:26Z
|
||||
git_ref: 65bf9df5
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260702T155626Z_65bf9df5_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
From `ai/prompt-io/prompts/issue_470.md`:
|
||||
|
||||
> attempt to resolve
|
||||
> https://github.com/goodboy/tractor/issues/470
|
||||
> do it with /open-wkt, ensure you /run-tests on all
|
||||
> changes
|
||||
|
||||
(plus a hard pause-for-human-review deadline of
|
||||
12:50PM EST.)
|
||||
|
||||
Issue #470 asks to trim the ~0.42s `import tractor`
|
||||
cost — which dominates per-actor spawn latency on the
|
||||
`trio` backend — by lazy-importing heavy/optional deps
|
||||
(`pdbp`, `stackscope`, `multiaddr`, + audit of
|
||||
`colorlog`/`bidict`/`wrapt`/`setproctitle`).
|
||||
|
||||
## Response summary
|
||||
|
||||
Profiling showed the issue's dep-list only accounted
|
||||
for ~20ms; the dominant cost (~244ms) was
|
||||
`log.get_logger()`'s `get_caller_mod()` calling
|
||||
`inspect.stack()` at module level in ~39 modules —
|
||||
each call walks every stack frame (deep during nested
|
||||
imports) and scans `sys.modules` per frame via
|
||||
`inspect.getmodule()`.
|
||||
|
||||
Changes, in impact order:
|
||||
|
||||
1. `get_caller_mod()` -> `sys._getframe()` +
|
||||
`f_globals['__name__']` `sys.modules` lookup
|
||||
(~240ms saved).
|
||||
2. Issue's lazy-import checklist: `bidict`,
|
||||
`multiaddr`, `colorlog`, `wrapt` moved to
|
||||
`TYPE_CHECKING`/function-local imports;
|
||||
`platformdirs` function-local; `asyncio` +
|
||||
`.to_asyncio` deferred out of the `devx.debug` +
|
||||
`spawn._entry` eager paths (~15ms saved).
|
||||
3. PEP 562 `__getattr__` on `tractor/__init__.py`
|
||||
preserving public `tractor.to_asyncio` attr access.
|
||||
|
||||
Results: `import tractor` 0.42s -> ~0.145s (~65%);
|
||||
sequential `start_actor` latency 0.40-0.44s ->
|
||||
~0.179s/actor. `pdbp` (needs `_repl.py` class-base
|
||||
restructure) + `platformdirs` (needs
|
||||
`UDSAddress.def_bindspace` protocol rework) documented
|
||||
as follow-ups.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/log.py` — `get_caller_mod()` perf fix +
|
||||
lazy `colorlog`
|
||||
- `tractor/__init__.py` — PEP 562 lazy `to_asyncio`
|
||||
- `tractor/discovery/_addr.py` — `bidict` ->
|
||||
`TYPE_CHECKING`
|
||||
- `tractor/discovery/_multiaddr.py` — lazy `multiaddr`
|
||||
- `tractor/ipc/_tcp.py`, `tractor/ipc/_uds.py` —
|
||||
`Multiaddr` -> `TYPE_CHECKING`
|
||||
- `tractor/runtime/_state.py` — lazy `platformdirs`
|
||||
- `tractor/devx/_frame_stack.py` — lazy `pdbp` +
|
||||
`wrapt`
|
||||
- `tractor/devx/debug/_trace.py`,
|
||||
`tractor/devx/debug/_tty_lock.py` — lazy `asyncio` +
|
||||
`.to_asyncio`
|
||||
- `tractor/spawn/_entry.py` — lazy
|
||||
`run_as_asyncio_guest`
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — pending user review at the 12:50PM EST
|
||||
pause gate (test-suite results reported in-session).
|
||||
|
|
@ -0,0 +1,122 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
timestamp: 2026-07-02T15:56:26Z
|
||||
git_ref: 65bf9df5
|
||||
diff_cmd: git diff main..wkt/boot_latency_470
|
||||
---
|
||||
|
||||
# Raw AI output — gh #470 `import tractor` latency trim
|
||||
|
||||
All generated code is committed on the
|
||||
`wkt/boot_latency_470` branch; per diff-ref mode each
|
||||
file's content is referenced via its diff instead of
|
||||
copied verbatim.
|
||||
|
||||
## Profiling findings (verbatim analysis output)
|
||||
|
||||
Baseline: `import tractor` ~0.39-0.42s wall.
|
||||
|
||||
`python -X importtime` + `cProfile` traced the cost NOT
|
||||
primarily to third-party deps (the issue's hypothesis)
|
||||
but to `tractor/log.py:get_logger()` calling
|
||||
`get_caller_mod()` -> `inspect.stack()` at module level
|
||||
in ~39 tractor modules:
|
||||
|
||||
- `inspect.stack()` builds `FrameInfo` (incl. src-file
|
||||
and line-context resolution) for EVERY frame on the
|
||||
stack; during nested imports the stack is dozens of
|
||||
importlib frames deep.
|
||||
- each `FrameInfo` resolution calls
|
||||
`inspect.getmodule()` which scans all of
|
||||
`sys.modules` per frame (1.4M `ismodule()` calls in
|
||||
one profiled import).
|
||||
- aggregate: ~244ms of tractor-own module "self" time
|
||||
vs ~20ms for ALL the issue-listed third-party deps
|
||||
(`pdbp` ~10ms, `bidict` ~4.5ms, `multiaddr` ~3.5ms,
|
||||
`wrapt`/`colorlog` ~1ms each); `trio` itself is
|
||||
~70-100ms and unavoidable.
|
||||
|
||||
## Generated changes
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/log.py`
|
||||
|
||||
`get_caller_mod()` rewritten from `inspect.stack()` +
|
||||
`inspect.getmodule()` to `sys._getframe(frames_up)` +
|
||||
`frame.f_globals['__name__']` -> `sys.modules` lookup
|
||||
(O(1) vs O(stack x sys.modules)). Unused `inspect`
|
||||
imports dropped; `FrameType` imported from `types`.
|
||||
Also `colorlog` lazy-imported inside
|
||||
`get_console_log()`.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/discovery/_addr.py`
|
||||
|
||||
`bidict` import moved under `TYPE_CHECKING`
|
||||
(annotation-only use; `_address_types` is a plain dict
|
||||
literal).
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/discovery/_multiaddr.py`
|
||||
|
||||
`from __future__ import annotations` added; `multiaddr`
|
||||
import moved under `TYPE_CHECKING` + function-local
|
||||
imports in `mk_maddr()`/`parse_maddr()`.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/ipc/_tcp.py tractor/ipc/_uds.py`
|
||||
|
||||
`Multiaddr` imports moved under `TYPE_CHECKING`
|
||||
(annotation-only in both transports).
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/runtime/_state.py`
|
||||
|
||||
`platformdirs` lazy-imported inside `get_rt_dir()`
|
||||
(NOTE: still imported eagerly via
|
||||
`UDSAddress.def_bindspace` class-var eval; see
|
||||
follow-ups).
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/devx/_frame_stack.py`
|
||||
|
||||
`pdbp` + `wrapt` lazy-imported inside
|
||||
`hide_runtime_frames()` / `api_frame()` respectively.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/devx/debug/_trace.py tractor/devx/debug/_tty_lock.py`
|
||||
|
||||
`asyncio` moved to `TYPE_CHECKING` + call-site local
|
||||
imports (`asyncio.current_task()` sites);
|
||||
`tractor.to_asyncio.run_trio_task_in_future` imports
|
||||
moved into the infected-aio runtime branches.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/spawn/_entry.py`
|
||||
|
||||
`run_as_asyncio_guest` import moved into the
|
||||
`infect_asyncio=True` branches of `_mp_main()` /
|
||||
`_trio_main()`.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/__init__.py`
|
||||
|
||||
PEP 562 module `__getattr__` added so
|
||||
`tractor.to_asyncio` attr-access still works (required
|
||||
by `tests/test_child_manages_service_nursery.py` and
|
||||
any downstream user) while keeping `asyncio` off the
|
||||
eager import path.
|
||||
|
||||
## Measured results (verbatim)
|
||||
|
||||
- `import tractor`: 0.39-0.42s -> ~0.145s (~65% cut)
|
||||
- `start_actor` spawn+boot+reg+cancel: ~0.40-0.44s ->
|
||||
~0.179s/actor (n=5 sequential, warm parent)
|
||||
- post-change eager-module check: only `pdbp` +
|
||||
`platformdirs` of the issue's list remain eager.
|
||||
|
||||
## Known follow-ups (not implemented, deadline-bound)
|
||||
|
||||
- `pdbp` (~10ms): still eager via
|
||||
`devx/debug/_repl.py` class bases
|
||||
(`class PdbREPL(pdbp.Pdb)`) + `_tty_lock.py`
|
||||
module-level `@pdbp.hideframe`; needs `_repl`
|
||||
restructure + PEP 562 in `devx.debug.__init__`.
|
||||
- `platformdirs` (~1.5ms): eager via
|
||||
`UDSAddress.def_bindspace: ClassVar = get_rt_dir()`
|
||||
class-body call; needs `Address`-protocol rework of
|
||||
`def_bindspace` to a lazy accessor.
|
||||
- `stackscope` + `setproctitle`: already lazy/absent —
|
||||
no change needed.
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
session: f6c84722-471a-4458-9a80-e453fea9029f
|
||||
timestamp: 2026-07-02T16:58:06Z
|
||||
git_ref: a34aaf98
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260702T165806Z_a34aaf98_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Follow-up round in the same session as the
|
||||
`tractor.to_actor` landing (see
|
||||
`20260702T154255Z_65bf9df5_prompt_io.md`). After
|
||||
committing that work the user green-lit the deferred
|
||||
items:
|
||||
|
||||
> go go go on this with what time you have left, in
|
||||
> particular see if you can get the _ria_nursery
|
||||
> removal going!
|
||||
|
||||
then extended the deadline twice to iterate on the
|
||||
discovered hang:
|
||||
|
||||
> continue on this up until a 12:58:30 deadline
|
||||
|
||||
and finally chose "Commit step A now" from the
|
||||
next-steps prompt.
|
||||
|
||||
## Response summary
|
||||
|
||||
Step A of the `._ria_nursery` removal (issue #477): hoist
|
||||
`.run_in_actor()` result-reaping out of the spawn
|
||||
backends into the `ActorNursery` machinery so ria
|
||||
children spawn via the default daemon nursery,
|
||||
|
||||
- new `_supervise._reap_ria_portals()` runs one
|
||||
`_spawn.cancel_on_completion()` task per ria child
|
||||
AFTER `._join_procs` is set; happy path awaits it
|
||||
right after `._join_procs.set()`.
|
||||
- error path SEQUENCES: snapshot ria
|
||||
`(portal, subactor)` pairs -> `await an.cancel()` ->
|
||||
0.5s-bounded reap. Two failed intermediates informed
|
||||
this: a concurrent reap+cancel DEADLOCKED
|
||||
`test_multierror`; a 3s bound blew
|
||||
`test_cancel_while_childs_child_in_sync_sleep`'s
|
||||
`fail_after` deadline.
|
||||
- backends (`spawn/_trio.py`, `spawn/_mp.py`) lose the
|
||||
`._cancel_after_result_on_exit` membership branch,
|
||||
per-child reaper nursery + dead imports.
|
||||
- design/probe-history doc:
|
||||
`ai/conc-anal/ria_nursery_removal_plan.md` (from an
|
||||
agent-verified machinery map).
|
||||
|
||||
Verification: `test_cancellation.py` fully green
|
||||
(20 passed, 1 xfailed) incl. the previously-hung
|
||||
`test_multierror`; `test_to_actor`+`test_spawning`
|
||||
20/20; bounded full-suite gate SIGINT'd ~30s early at
|
||||
303 passed / 0 failures (user opted to commit on that
|
||||
signal, deferring the unbounded re-run to step-B
|
||||
verification).
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/runtime/_supervise.py` — `_reap_ria_portals()`
|
||||
+ two call-sites; `run_in_actor()` off the ria nursery
|
||||
- `tractor/spawn/_trio.py` — reaper branch + import drop
|
||||
- `tractor/spawn/_mp.py` — same as `_trio.py`
|
||||
- `ai/conc-anal/ria_nursery_removal_plan.md` — plan +
|
||||
probe history
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — committed via the drafted
|
||||
`.claude/git_commit_msg_ria_step_a.md` (user-driven
|
||||
`git commit --edit`).
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
timestamp: 2026-07-02T16:58:06Z
|
||||
git_ref: a34aaf98
|
||||
diff_cmd: git diff a34aaf98..wkt/to_actor_subpkg
|
||||
---
|
||||
|
||||
# Raw AI output (diff-ref mode)
|
||||
|
||||
Step-A code is committed on `wkt/to_actor_subpkg`
|
||||
directly after `a34aaf98`; per diff-ref mode the verbatim
|
||||
content is reachable via the pointers below.
|
||||
|
||||
## Generated files
|
||||
|
||||
> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/runtime/_supervise.py`
|
||||
|
||||
New `_reap_ria_portals(an, errors, ria_children=None)`
|
||||
helper (one `_spawn.cancel_on_completion()` task per ria
|
||||
child under `collapse_eg()` + a local nursery);
|
||||
`run_in_actor()` drops `nursery=self._ria_nursery`; happy
|
||||
path awaits the reap right after `._join_procs.set()`;
|
||||
inner error handler snapshots ria pairs, runs
|
||||
`await an.cancel()` then a `move_on_after(0.5)`-bounded
|
||||
reap over the snapshot.
|
||||
|
||||
> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/spawn/_trio.py`
|
||||
> `git diff a34aaf98..wkt/to_actor_subpkg -- tractor/spawn/_mp.py`
|
||||
|
||||
Both backends: the post-`_join_procs` block collapses to
|
||||
a bare `soft_kill()` (membership branch, per-child reaper
|
||||
nursery, reaper-cancel logging and the now-unused
|
||||
`cancel_on_completion` imports all removed).
|
||||
|
||||
> `git diff a34aaf98..wkt/to_actor_subpkg -- ai/conc-anal/ria_nursery_removal_plan.md`
|
||||
|
||||
Agent-verified machinery map, 3-step design (A/B/C),
|
||||
probe history (deadlock -> sequencing fix -> bound
|
||||
tightening) and risk register.
|
||||
|
||||
## Test runs (verbatim)
|
||||
|
||||
```
|
||||
tests/test_cancellation.py: 20 passed, 1 xfailed in 77.28s
|
||||
tests/test_to_actor.py + tests/test_spawning.py: 20 passed
|
||||
full-suite (SIGINT at 540s bound, ~30s short):
|
||||
303 passed, 9 skipped, 1 xfailed, 2 xpassed in 539.42s
|
||||
```
|
||||
|
||||
Failed intermediates (not in final diff): concurrent
|
||||
reap+`an.cancel()` deadlocked `test_multierror`; a 3s
|
||||
reap bound blew
|
||||
`test_cancel_while_childs_child_in_sync_sleep`'s
|
||||
`fail_after` deadline.
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
model: claude-opus-4-8
|
||||
service: claude
|
||||
session: c7860302-6a33-4d92-821c-0807d68be464
|
||||
timestamp: 2026-07-02T17:22:33Z
|
||||
git_ref: 5cd190c5
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260702T172233Z_5cd190c5_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Branched session (`to_actor_subpkg_POST_WEEKLY_CUTOFF_2OPUS`,
|
||||
now on Opus 4.8) continuing the `._ria_nursery` removal after
|
||||
step A committed + pushed (`5cd190c5`/`99310269`):
|
||||
|
||||
> ok continue where your superior left off please
|
||||
|
||||
then, after the commit-split discussion:
|
||||
|
||||
> yup committed and pushed so take a look ma fren.
|
||||
|
||||
i.e. green-lighting step-B prep against the committed step-A
|
||||
tree.
|
||||
|
||||
## Response summary
|
||||
|
||||
Step B of the `._ria_nursery` removal (issue #477): delete
|
||||
the now-vestigial secondary nursery,
|
||||
|
||||
- collapse the inner `async with (collapse_eg(),
|
||||
trio.open_nursery() as ria_nursery)` layer in
|
||||
`_open_and_supervise_one_cancels_all_nursery` — `da_nursery`
|
||||
becomes the single nursery for ALL subactors.
|
||||
- `ActorNursery.__init__` drops the `ria_nursery` param + the
|
||||
`self._ria_nursery` attr; `start_actor()` drops its
|
||||
`nursery=` escape-hatch param.
|
||||
- `._cancel_after_result_on_exit` kept (ria-child
|
||||
discriminator for `_reap_ria_portals()`).
|
||||
|
||||
Verified behavior-preserving via a first-principles argument
|
||||
(zero-task nursery = a bare checkpoint) + the targeted gate
|
||||
(`test_cancellation test_spawning test_local test_rpc
|
||||
test_to_actor` = 49 passed, 1 xfailed on trio). The two
|
||||
error handlers were deliberately NOT merged — that changes
|
||||
propagation semantics and is deferred to its own PR (TODO
|
||||
left at the outer `except`).
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/runtime/_supervise.py` — collapse the ria nursery
|
||||
layer + drop the ctor/`start_actor` params + refresh the
|
||||
now-stale nursery comments
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — committed via the drafted
|
||||
`.claude/git_commit_msg_ria_step_b.md` (user-driven
|
||||
`git commit --edit`).
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
model: claude-opus-4-8
|
||||
service: claude
|
||||
timestamp: 2026-07-02T17:22:33Z
|
||||
git_ref: 5cd190c5
|
||||
diff_cmd: git diff 5cd190c5..wkt/to_actor_subpkg
|
||||
---
|
||||
|
||||
# Raw AI output (diff-ref mode)
|
||||
|
||||
Step-B code lives on `wkt/to_actor_subpkg` after `5cd190c5`;
|
||||
per diff-ref mode the verbatim content is reachable via the
|
||||
pointer below.
|
||||
|
||||
## Generated files
|
||||
|
||||
> `git diff 5cd190c5..wkt/to_actor_subpkg -- tractor/runtime/_supervise.py`
|
||||
|
||||
- `ActorNursery.__init__`: `ria_nursery` param removed;
|
||||
`self._ria_nursery = ria_nursery` block deleted;
|
||||
`_cancel_after_result_on_exit` comment refreshed.
|
||||
- `start_actor()`: `nursery=` param removed; body uses
|
||||
`self._da_nursery.start(...)` directly.
|
||||
- `_open_and_supervise_one_cancels_all_nursery()`: the inner
|
||||
`async with (collapse_eg(), trio.open_nursery() as
|
||||
ria_nursery)` layer removed; `an = ActorNursery(actor,
|
||||
da_nursery, errors)` constructed once under the single
|
||||
`da_nursery`; the inner-try body de-indented one level;
|
||||
both error handlers retained; the da-nursery lead comment
|
||||
and the outer-`except` TODO refreshed to describe the
|
||||
single-nursery reality + flag the (deferred) handler-merge.
|
||||
|
||||
> `git diff 5cd190c5..wkt/to_actor_subpkg -- ai/conc-anal/ria_nursery_removal_plan.md`
|
||||
|
||||
Added a "Step-B outcome" section (collapse rationale,
|
||||
handler-merge deferral, safety argument, gate result).
|
||||
|
||||
## Test runs (verbatim)
|
||||
|
||||
```
|
||||
targeted gate (trio):
|
||||
tests/test_cancellation.py tests/test_spawning.py
|
||||
tests/test_local.py tests/test_rpc.py tests/test_to_actor.py
|
||||
-> 49 passed, 1 xfailed in 88.62s
|
||||
|
||||
signature checks:
|
||||
ActorNursery.__init__ params: ['self', 'actor', 'da_nursery', 'errors']
|
||||
start_actor has nursery param: False
|
||||
|
||||
full-suite gate: in progress (see stepB_full.log)
|
||||
```
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
---
|
||||
model: claude-opus-4-8
|
||||
service: claude
|
||||
session: c7860302-6a33-4d92-821c-0807d68be464
|
||||
timestamp: 2026-07-02T22:25:44Z
|
||||
git_ref: 9201a2ed
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260702T222544Z_9201a2ed_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Continuing on branch `drop_ria_nursery` after step B
|
||||
(`9201a2ed`). User green-lit the deferred error-handler merge:
|
||||
|
||||
> yes, i checked us out on a new branch for the ria-tn removal;
|
||||
> work off the new wkt branch drop_ria_nursery.
|
||||
|
||||
and (re the migration) "nope, later we'll do this", plus:
|
||||
|
||||
> do ensure the test suite is green in the current state and
|
||||
> those going forward ya?
|
||||
|
||||
and the standing reminder: no commits without a human ack.
|
||||
|
||||
## Response summary
|
||||
|
||||
Step B2 of the `._ria_nursery` removal (issue #477): merge the
|
||||
two nested error handlers in
|
||||
`_open_and_supervise_one_cancels_all_nursery` into one, now
|
||||
that the 2ndary nursery (whose `__aexit__` the outer handler
|
||||
originally guarded) is gone.
|
||||
|
||||
Trace-driven justification (see
|
||||
`ai/conc-anal/ria_nursery_removal_plan.md` Step-B2 section):
|
||||
the old inner handler records `errors[uid]` as its FIRST
|
||||
action, so `errors` is always non-empty by the time control
|
||||
could reach the outer handler — making the outer handler's
|
||||
`raise` DEAD (always superseded by the `finally`'s
|
||||
raise-from-`errors`). Collapsing to a single
|
||||
`except BaseException as _scope_err` + the existing `finally`
|
||||
is therefore behavior-preserving.
|
||||
|
||||
- removed the outer `try` + the
|
||||
`except (Exception, BaseExceptionGroup, trio.Cancelled)`
|
||||
handler; de-indented the inner handler one level.
|
||||
- dropped the now-unused `outer_err`/`inner_err` locals
|
||||
(single `_scope_err`).
|
||||
- net ~30 lines lighter.
|
||||
|
||||
Gated hard: baseline (pre-B2 tip) = 300 passed + 1
|
||||
load-induced `TooSlowError` flake (passes 6/6 in isolation);
|
||||
B2 error/cancel gate (`test_cancellation remote_exc_relay
|
||||
inter_peer_cancellation advanced_faults oob_cancellation
|
||||
to_actor spawning local rpc`) = 71 passed, 1 xfailed; B2
|
||||
full-suite run following.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/runtime/_supervise.py` — collapse the two handlers
|
||||
into one; drop `outer_err`/`inner_err`
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — committed via the drafted
|
||||
`.claude/git_commit_msg_ria_b2.md` (user-driven
|
||||
`git commit --edit`).
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
---
|
||||
model: claude-opus-4-8
|
||||
service: claude
|
||||
timestamp: 2026-07-02T22:25:44Z
|
||||
git_ref: 9201a2ed
|
||||
diff_cmd: git diff 9201a2ed..drop_ria_nursery
|
||||
---
|
||||
|
||||
# Raw AI output (diff-ref mode)
|
||||
|
||||
Step-B2 code lives on `drop_ria_nursery` after `9201a2ed`; per
|
||||
diff-ref mode the verbatim content is reachable via the pointer
|
||||
below.
|
||||
|
||||
## Generated files
|
||||
|
||||
> `git diff 9201a2ed..drop_ria_nursery -- tractor/runtime/_supervise.py`
|
||||
|
||||
`_open_and_supervise_one_cancels_all_nursery`:
|
||||
- removed the outer `try:` wrapper and the
|
||||
`except (Exception, BaseExceptionGroup, trio.Cancelled) as
|
||||
_outer_err:` safety-net handler.
|
||||
- the former inner `except BaseException` is now THE handler,
|
||||
renamed local `_inner_err` -> `_scope_err`, de-indented one
|
||||
level; it sets `an._scope_error`, records `errors[uid]`,
|
||||
waits on the debugger, `_join_procs.set()`, then a shielded
|
||||
classify/log + snapshot-ria + `an.cancel()` + 0.5s-bounded
|
||||
`_reap_ria_portals()`. No re-raise (the `finally` raises
|
||||
from `errors`).
|
||||
- `finally` block unchanged.
|
||||
- dropped the `outer_err`/`inner_err` local decls at fn top.
|
||||
|
||||
(The diff is large — ~119+/149- — because de-indenting the
|
||||
handler body one level rewrites every line in the block; the
|
||||
logic delta is just "two handlers -> one".)
|
||||
|
||||
## Test runs (verbatim)
|
||||
|
||||
```
|
||||
baseline (pre-B2, step-B tip 9201a2ed), full suite
|
||||
(dynamic_pub_sub deselected):
|
||||
1 failed, 300 passed, 9 skipped, 2 deselected, 1 xfailed,
|
||||
2 xpassed in 1499.49s
|
||||
-> the 1 failure = test_ext_types_over_ipc[...] trio.TooSlowError
|
||||
(load-induced; passes 6/6 in isolation in 4.89s)
|
||||
|
||||
B2 error/cancel gate:
|
||||
tests/test_cancellation test_remote_exc_relay
|
||||
test_inter_peer_cancellation test_advanced_faults
|
||||
test_oob_cancellation test_to_actor test_spawning test_local
|
||||
test_rpc
|
||||
-> 71 passed, 1 xfailed in 125.26s
|
||||
|
||||
B2 full-suite run: see b2_full.log
|
||||
```
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
session: 6db64ac6-6986-4505-9343-df4ee31e67db
|
||||
timestamp: 2026-07-06T17:28:18Z
|
||||
git_ref: ad42871e
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260706T172818Z_ad42871e_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Resuming the `drop_ria_nursery` (wkt) branch after the human's
|
||||
review pause at `ba4af0ad`, with the directive:
|
||||
|
||||
> we need to resolve the remaining outstandings ... to finish
|
||||
> this endeavour we ideally,
|
||||
> - finish the ria-tn hoist into the machinery in `to_actor` if
|
||||
> possible.
|
||||
> - avoid supporting the `.run_in_actor()` non-blocking API
|
||||
> going forward other then if we can re-wrap `to_actor.run()`
|
||||
> per prior discussion with opus4.8.
|
||||
> - get the test harness completely green locally and in CI -
|
||||
> AND ESPECIALLY without any hangs due to the "the old
|
||||
> spawn-backend reaper raced `soft_kill()`'s scope-cancel"
|
||||
> - update all docs to reflect these changes once we're happy.
|
||||
|
||||
plus mid-flight additions:
|
||||
|
||||
> make sure both all the test and example code uses the naming
|
||||
> `an: ActorNursery` for variable names
|
||||
|
||||
> where we at in terms of a potential hackaround design for
|
||||
> supporting a to_actor.run_async() ... without requiring the
|
||||
> same malarky ... with a `trio.Event` relay ... i'm totes fine
|
||||
> with rejigging the iface to NOT return a Portal as well.
|
||||
|
||||
## Response summary
|
||||
|
||||
Completed the entire #477 `run_in_actor` removal on
|
||||
`drop_ria_nursery` (16 commits this session), gated per-commit:
|
||||
|
||||
- ported the remaining `test_cancellation` groups
|
||||
(`some_cancels_all`, `nested_multierrors`+`spawn_and_error`,
|
||||
SIGINT + sync-sleep) — one-shots become concurrent
|
||||
`to_actor.run()` tasks in local task-nurseries; BEG-shape
|
||||
assertions loosened for cancel-on-first + the runtime's
|
||||
`collapse_eg()` single-member unwrap (a fully-raced nested
|
||||
tree relays a bare annotated `RemoteActorError` chain).
|
||||
- fixed a pre-existing `UnboundLocalError` (`timeout` `match`
|
||||
had no default arm for non-trio/MTF backends).
|
||||
- ported `test_dynamic_pub_sub`, 4 non-debugging examples, all
|
||||
8 `debugging/` examples (debugger suite byte-identical,
|
||||
28p/6s; `multi_subactors` introduces the "collect don't
|
||||
cancel" reap-all replacement pattern), 8 docs pages + the
|
||||
`experimental/_pubsub` docstring.
|
||||
- EXCISED the API + cluster: `run_in_actor`,
|
||||
`_reap_ria_portals`, `_cancel_after_result_on_exit`,
|
||||
`Portal._submit_for_result/_expect_result_ctx/
|
||||
wait_for_result/result`, `exhaust_portal`,
|
||||
`cancel_on_completion`, `NoResult` — net -402 lines. The
|
||||
reap-hang class dissolves structurally (result-waits now only
|
||||
in caller task-scope).
|
||||
- found + fixed a real migration race: mutual-rendezvous peers
|
||||
(`test_trynamic_trio`, `a_trynamic_first_scene.py`) flaked
|
||||
because an eagerly-reaped one-shot dies while its peer still
|
||||
dials the registry-resolved (dead) sockaddr — such peers now
|
||||
pin lifetimes via `start_actor()` + concurrent `Portal.run()`
|
||||
+ explicit `an.cancel()`.
|
||||
- `an: ActorNursery` naming sweep across tests/examples (±82
|
||||
lines, scoped renames, prose untouched).
|
||||
- parked a `to_actor.open_one_shot()` design sketch (acm +
|
||||
private task-nursery over blocking `run()`; done-Event as
|
||||
memo not cancel-relay; no Portal) in the plan doc.
|
||||
|
||||
## Files changed
|
||||
|
||||
See commits `d01a2123..ad42871e` on `drop_ria_nursery`
|
||||
(tests, examples, docs, `tractor/{runtime,spawn,to_actor,msg}`
|
||||
+ `_exceptions/_context/experimental`).
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
timestamp: 2026-07-06T17:28:18Z
|
||||
git_ref: ad42871e
|
||||
diff_cmd: git diff ba4af0ad..ad42871e
|
||||
---
|
||||
|
||||
# Raw AI output (diff-ref mode)
|
||||
|
||||
This session's output spans the 16 migration/excision commits
|
||||
`d01a2123..ad42871e` on `drop_ria_nursery`; per diff-ref mode
|
||||
the verbatim content is reachable via the pointer below.
|
||||
|
||||
## Generated files
|
||||
|
||||
> `git diff ba4af0ad..ad42871e`
|
||||
|
||||
Commit-wise (each `Gate:`-footed msg documents its own module
|
||||
gate):
|
||||
|
||||
- `d01a2123` port `test_some_cancels_all`
|
||||
- `697c6152` fix unbound `timeout` (non-trio/MTF `match` arm)
|
||||
- `fa8799d5` port `test_nested_multierrors`
|
||||
- `f11754ce` port SIGINT + sync-sleep cancel tests
|
||||
- `cb6202e3` port `test_dynamic_pub_sub`
|
||||
- `d8af5f12` port non-debugging examples
|
||||
- `a3057cb2` port debugging examples (+ `test_debugger`
|
||||
nested-nurseries final-shape expectations)
|
||||
- `d6bed7c4` port docs (8 rst pages)
|
||||
- `07e1669e` fix stale `@pub` docstring example
|
||||
- `2a59cefb` REMOVE `run_in_actor()` + the ria reap cluster
|
||||
(net -402 lines)
|
||||
- `a297a32a` fix mutual-rendezvous premature-reap race
|
||||
- `ad42871e` `an: ActorNursery` naming sweep
|
||||
|
||||
Plan/design record updated in
|
||||
`ai/conc-anal/ria_nursery_removal_plan.md` (RESOLVED section +
|
||||
the `to_actor.open_one_shot()` follow-up sketch).
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: pr475-review-fixes-20260817
|
||||
timestamp: 2026-08-17T23:18:25Z
|
||||
git_ref: 359fe75c
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260817T231825Z_359fe75c_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Continue the `/code-review-changes` pass for PR #475 in its isolated
|
||||
worktree. Address the seven accepted manual-review findings in
|
||||
`tractor/ipc/_types.py` and `tractor/ipc/_uds.py`, preserve the existing
|
||||
Windows capability behavior, verify the result, and prepare the work for
|
||||
human-controlled commit and review-reply steps. Do not publish replies,
|
||||
stage, commit, or push without the required explicit authorization.
|
||||
|
||||
## Response summary
|
||||
|
||||
Restored project quote, docstring, multiline-expression, and
|
||||
`match/case` conventions while retaining the Windows-safe UDS guard.
|
||||
Removed unnecessary structural and comment churn, then verified the
|
||||
focused transport, discovery, and lazy-import paths plus the missing
|
||||
`AF_UNIX` behavior.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/ipc/_types.py` - restore project style and guarded
|
||||
socket-family dispatch.
|
||||
- `tractor/ipc/_uds.py` - format the UDS capability gate
|
||||
consistently.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - the generated patch remains uncommitted and awaits human
|
||||
review.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-17T23:18:25Z
|
||||
git_ref: 359fe75c
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
Applied the seven accepted manual-review fixes for PR #475 while
|
||||
preserving the Windows transport capability behavior.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tractor/ipc/_types.py`
|
||||
|
||||
The generated changes restore the project's single-quote docstring and
|
||||
string conventions, remove the unnecessary helper divider, simplify the
|
||||
transport-registry comments, and restore `match/case` socket-family
|
||||
dispatch. The UDS case retains a `HAS_UDS` guard that short-circuits
|
||||
before `socket.AF_UNIX` is evaluated on unsupported hosts. Nearby error
|
||||
messages are wrapped without changing their content.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tractor/ipc/_uds.py`
|
||||
|
||||
The generated change reformats the `HAS_UDS` conjunction according to
|
||||
the project's multiline boolean-expression convention and simplifies
|
||||
the adjacent capability comment.
|
||||
|
||||
Verification:
|
||||
|
||||
`/home/goodboy/repos/tractor/py313/bin/pytest -q tests/test_lazy_imports.py tests/discovery tests/ipc/test_server.py`
|
||||
|
||||
Result: `66 passed, 2 xpassed in 60.62s`.
|
||||
|
||||
`ruff check --no-cache --output-format=json tractor/ipc/_types.py tractor/ipc/_uds.py`
|
||||
|
||||
Result: no findings.
|
||||
|
||||
`git diff --check`
|
||||
|
||||
Result: no whitespace errors.
|
||||
|
||||
An explicit missing-`AF_UNIX` probe set `HAS_UDS = False`, removed the
|
||||
socket constant, and exercised an unsupported socket family. It raised
|
||||
the expected `NotImplementedError` instead of `AttributeError`.
|
||||
|
||||
No review replies, commits, or pushes were published.
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
NOTE: you MUST pause this work at 12:50PM EST (BEFORE your weekly
|
||||
limit reset) for review by a human!
|
||||
|
||||
---
|
||||
|
||||
attempt to resolve https://github.com/goodboy/tractor/issues/477
|
||||
do it with /open-wkt.
|
||||
|
|
@ -37,7 +37,6 @@ Spawning actors
|
|||
|
||||
.. autoclass:: ActorNursery
|
||||
:members: start_actor,
|
||||
run_in_actor,
|
||||
cancel,
|
||||
cancel_called,
|
||||
cancelled_caught
|
||||
|
|
@ -47,10 +46,22 @@ Spawning actors
|
|||
:meth:`ActorNursery.start_actor` (daemon actor + portal) is the
|
||||
blessed spawning primitive; pair it with
|
||||
``Portal.open_context()`` for SC-linked remote tasks.
|
||||
:meth:`ActorNursery.run_in_actor` is a *convenience* one-shot —
|
||||
spawn, run a single task, auto-cancel after the result — slated
|
||||
to be rebuilt as a high-level wrapper, so don't design around
|
||||
it as the core model.
|
||||
|
||||
One-shot task actors
|
||||
--------------------
|
||||
|
||||
.. autofunction:: tractor.to_actor.run
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`tractor.to_actor.run` (parlance of
|
||||
``trio.to_thread.run_sync()`` and friends) is the
|
||||
*convenience* one-shot — spawn, run a single task, block on
|
||||
its result, reap — built entirely on
|
||||
:meth:`ActorNursery.start_actor` + :meth:`Portal.run` +
|
||||
:meth:`Portal.cancel_actor`, so don't design around it as the
|
||||
core model. It supersedes the removed (legacy, non-blocking)
|
||||
``ActorNursery.run_in_actor()``.
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
|
|
@ -71,14 +82,12 @@ flowing back `exactly like trio`_.
|
|||
:members: run,
|
||||
run_from_ns,
|
||||
open_stream_from,
|
||||
wait_for_result,
|
||||
cancel_actor,
|
||||
chan
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
``Portal.result()`` warns; use :meth:`Portal.wait_for_result`.
|
||||
The str-form ``Portal.run('mod.path', 'fn_name')`` also warns;
|
||||
The str-form ``Portal.run('mod.path', 'fn_name')`` warns;
|
||||
pass a function *object* whose module is listed in the target's
|
||||
``enable_modules``. ``Portal.channel`` is the legacy spelling
|
||||
of :attr:`Portal.chan`.
|
||||
|
|
|
|||
|
|
@ -130,9 +130,10 @@ UDS: same-host, creds included
|
|||
|
||||
Pass ``enable_transports=['uds']`` and actors instead talk over
|
||||
unix-domain sockets, with socket files placed in the per-user
|
||||
runtime dir (``$XDG_RUNTIME_DIR/tractor/`` on linux, the
|
||||
``platformdirs`` equivalent elsewhere). Two perks over tcp on a
|
||||
single host:
|
||||
runtime dir: ``$XDG_RUNTIME_DIR/tractor/`` on linux, a short
|
||||
owner-only ``/tmp/tractor-<uid>`` dir on Darwin, and the
|
||||
``platformdirs`` equivalent elsewhere. Two perks over tcp on a single
|
||||
host:
|
||||
|
||||
- no ports to fight over; addrs are just file paths,
|
||||
- the kernel snitches on your peer for free: the listening side
|
||||
|
|
|
|||
|
|
@ -76,8 +76,8 @@ Just flip the flag on :meth:`tractor.ActorNursery.start_actor`:
|
|||
infect_asyncio=True,
|
||||
)
|
||||
|
||||
The one-shot convenience ``ActorNursery.run_in_actor()`` accepts
|
||||
the same flag. The ``to_asyncio`` APIs may **only** be called from
|
||||
The one-shot convenience ``tractor.to_actor.run()`` accepts the
|
||||
same flag. The ``to_asyncio`` APIs may **only** be called from
|
||||
tasks inside an infected actor; calling them anywhere else raises
|
||||
a loud ``RuntimeError``. You can introspect at runtime with
|
||||
``tractor.current_actor().is_infected_aio()``.
|
||||
|
|
@ -229,7 +229,7 @@ dialog, skip the channel ceremony and use
|
|||
|
||||
It schedules the fn as an ``asyncio.Task``, waits for completion
|
||||
and hands the return value back to ``trio``; think of it as the
|
||||
cross-loop sibling of ``ActorNursery.run_in_actor()``. Errors and
|
||||
cross-loop sibling of ``tractor.to_actor.run()``. Errors and
|
||||
cancellation are translated exactly as for channels.
|
||||
|
||||
Cross-loop errors and cancellation
|
||||
|
|
|
|||
|
|
@ -64,11 +64,13 @@ What's going on here?
|
|||
- three healthy actors are spawned as daemons via
|
||||
:meth:`tractor.ActorNursery.start_actor`; left alone they'd
|
||||
happily idle forever,
|
||||
- a fourth actor runs ``assert_err()`` via ``.run_in_actor()`` and
|
||||
promptly trips its ``assert 0``,
|
||||
- a fourth actor runs ``assert_err()`` via a blocking
|
||||
``tractor.to_actor.run()`` one-shot and promptly trips its
|
||||
``assert 0``,
|
||||
- the resulting ``AssertionError`` ships back over IPC as a
|
||||
serialized error msg and re-raises *boxed* inside the nursery
|
||||
block as a :class:`tractor.RemoteActorError`,
|
||||
serialized error msg and re-raises *boxed* right at the call
|
||||
inside the nursery block as a
|
||||
:class:`tractor.RemoteActorError`,
|
||||
- the nursery reacts like any ``trio`` nursery would: it cancels
|
||||
the three healthy siblings (graceful runtime-cancel requests,
|
||||
acks awaited), reaps all four processes, then re-raises,
|
||||
|
|
|
|||
|
|
@ -15,8 +15,8 @@ a single `structured concurrency`_ (SC) scope over IPC.
|
|||
:alt: sequence diagram of the context handshake msg flow
|
||||
|
||||
Pretty much everything else is (or is slated to be) built on this
|
||||
one primitive: ``ActorNursery.run_in_actor()`` is a convenience
|
||||
for "spawn, open a context, await the result, tear down"; plain
|
||||
one primitive: ``tractor.to_actor.run()`` is a convenience for
|
||||
"spawn, run the lone task, await the result, tear down"; plain
|
||||
``Portal.run()`` RPC is planned to be re-implemented on top of it;
|
||||
the multi-process debugger's tree-wide REPL lock rides one. Grok
|
||||
this page and the rest of the library reads as convenience
|
||||
|
|
|
|||
|
|
@ -44,8 +44,9 @@ clan shares one registry with zero config on your part.
|
|||
The bootstrap rule inside ``open_root_actor()`` is delightfully
|
||||
simple:
|
||||
|
||||
- on boot, ping every socket addr in ``registry_addrs``; when none
|
||||
are passed the per-transport defaults are used: for TCP the
|
||||
- on boot, probe every addr in ``registry_addrs`` with a bounded
|
||||
Tractor ``Aid`` handshake; when none are passed the per-transport
|
||||
defaults are used: for TCP the
|
||||
loopback ``('127.0.0.1', 1616)``, for UDS a
|
||||
``registry@1616.sock`` file,
|
||||
|
||||
|
|
@ -53,9 +54,11 @@ simple:
|
|||
actor and register with the *existing* registry; your own IPC
|
||||
server binds random same-transport addrs instead,
|
||||
|
||||
- if **nothing answers, congratulations: you just became the
|
||||
registrar**. Your transport server binds the registry addrs
|
||||
themselves and you start serving lookups for everyone else.
|
||||
- if every address is absent, congratulations: you just became the
|
||||
registrar. Your transport server binds the registry addrs
|
||||
themselves and you start serving lookups for everyone else,
|
||||
- if no registrar answers but an address is occupied by a foreign or
|
||||
non-responsive endpoint, startup fails instead of binding over it.
|
||||
|
||||
Pass ``ensure_registry=True`` when your program *requires* being
|
||||
the one-and-only registrar; boot then fails loudly with a
|
||||
|
|
@ -196,9 +199,10 @@ the existing registrar:
|
|||
|
||||
trio.run(main)
|
||||
|
||||
Per the bootstrap rules above, if the registrar at those addrs is
|
||||
*not* reachable this process simply becomes its own (registrar)
|
||||
root — so the same code works standalone and as a tree-joiner.
|
||||
Per the bootstrap rules above, if those addrs are absent this process
|
||||
becomes its own registrar root, so the same code works standalone and
|
||||
as a tree-joiner. An occupied address that does not complete a Tractor
|
||||
registrar handshake fails startup instead of being rebound.
|
||||
|
||||
"Arbiter"? A legacy naming note
|
||||
-------------------------------
|
||||
|
|
|
|||
|
|
@ -119,15 +119,16 @@ Run a func in a process
|
|||
|
||||
Even a pool can be overkill; "run this one async func in a
|
||||
subprocess and give me the result" is a one-liner via
|
||||
:meth:`tractor.ActorNursery.run_in_actor`,
|
||||
:func:`tractor.to_actor.run`,
|
||||
|
||||
.. literalinclude:: ../../examples/parallelism/single_func.py
|
||||
:caption: examples/parallelism/single_func.py
|
||||
:language: python
|
||||
|
||||
``run_in_actor()`` is a *convenience wrapper* — spawn an actor, run
|
||||
exactly one task in it, reap on result — not the core spawning
|
||||
model (that's :meth:`tractor.ActorNursery.start_actor` plus
|
||||
``to_actor.run()`` is a *convenience wrapper* — spawn an actor,
|
||||
run exactly one task in it, block on and return its result, reap
|
||||
— not the core spawning model (that's
|
||||
:meth:`tractor.ActorNursery.start_actor` plus
|
||||
:meth:`tractor.Portal.open_context`; see :doc:`/guide/context`).
|
||||
But for this fire-and-collect shape it's exactly the right amount
|
||||
of typing.
|
||||
|
|
|
|||
|
|
@ -80,28 +80,30 @@ One special namespace exists: ``'self'`` resolves to the remote
|
|||
how internal machinery (cancel requests, registry ops) travels;
|
||||
don't build your app on it.
|
||||
|
||||
One-shot results: ``wait_for_result()``
|
||||
---------------------------------------
|
||||
A portal returned from
|
||||
:meth:`~tractor.ActorNursery.run_in_actor` has exactly one
|
||||
"main" task running remotely; that task's ``return`` value is
|
||||
delivered as the portal's *final result*:
|
||||
One-shot subactors: ``to_actor.run()``
|
||||
--------------------------------------
|
||||
When a subactor's *entire job* is a single function call, skip
|
||||
the portal plumbing with :func:`tractor.to_actor.run`: spawn,
|
||||
run the lone task, return its result and reap the process — all
|
||||
in one blocking call:
|
||||
|
||||
.. code:: python
|
||||
|
||||
portal = await an.run_in_actor(fib, n=10)
|
||||
final = await portal.wait_for_result()
|
||||
final = await tractor.to_actor.run(fib, an=an, n=10)
|
||||
|
||||
Semantics worth knowing:
|
||||
|
||||
- it blocks until the remote task returns, re-raising any
|
||||
remote error in the usual boxed form.
|
||||
- once resolved it's idempotent: later calls return the same
|
||||
cached value.
|
||||
- a *daemon* portal (from ``start_actor()``) has no main task,
|
||||
so there's no final result to wait for: you'll get a warning
|
||||
plus a ``NoResult`` sentinel. Results of individual daemon
|
||||
calls come straight back from each ``await portal.run()``.
|
||||
remote error in the usual boxed form right in the calling
|
||||
task.
|
||||
- "placement" is composable: ``an=`` spawns from an existing
|
||||
actor-nursery, ``portal=`` reuses an already-running actor
|
||||
(no spawn/reap, just a ``Portal.run()``), and passing
|
||||
neither opens a private call-scoped nursery (booting the
|
||||
runtime if needed).
|
||||
- concurrency composes the plain ``trio`` way: schedule
|
||||
multiple ``run()`` calls into a local task nursery (see
|
||||
``examples/parallelism/to_actor_one_shots.py``).
|
||||
|
||||
Pure RPC daemons: ``run_daemon()``
|
||||
----------------------------------
|
||||
|
|
|
|||
|
|
@ -103,19 +103,22 @@ What's going on here?
|
|||
on him **forever**. Daemon lifetimes are *yours* to end; that
|
||||
explicitness is the point.
|
||||
|
||||
``run_in_actor()``: quick one-shot parallelism
|
||||
``to_actor.run()``: quick one-shot parallelism
|
||||
----------------------------------------------
|
||||
:meth:`~tractor.ActorNursery.run_in_actor` is the convenience
|
||||
wrapper: spawn an actor, run exactly one async function in it,
|
||||
then reap the process as soon as the result arrives.
|
||||
:func:`tractor.to_actor.run` is the convenience wrapper: spawn
|
||||
an actor, run exactly one async function in it, block on the
|
||||
result, then reap the process — the distributed sibling of
|
||||
``trio.to_thread.run_sync()``.
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with tractor.open_nursery() as an:
|
||||
portal = await an.run_in_actor(burn_cpu)
|
||||
async with (
|
||||
tractor.open_nursery() as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
# burn rubber in the parent too...
|
||||
await burn_cpu()
|
||||
total = await portal.wait_for_result()
|
||||
tn.start_soon(burn_cpu)
|
||||
total = await tractor.to_actor.run(burn_cpu, an=an)
|
||||
|
||||
A few details worth knowing:
|
||||
|
||||
|
|
@ -124,18 +127,21 @@ A few details worth knowing:
|
|||
- the function's module is auto-added to the child's
|
||||
``enable_modules`` allowlist.
|
||||
- extra ``**kwargs`` are forwarded to the function itself.
|
||||
- the child is *auto-cancelled* once its "main" result lands;
|
||||
at nursery exit these run-once children are always reaped
|
||||
first (causality_ is paramount!).
|
||||
- the call blocks until the result (or error) lands and the
|
||||
child is *auto-cancelled* (reaped) right after — so remote
|
||||
errors raise directly in your calling task (causality_ is
|
||||
paramount!).
|
||||
- "placement" composes: ``an=`` spawns from a caller-managed
|
||||
actor-nursery, ``portal=`` reuses an already-running actor
|
||||
(no spawn/reap), and passing neither opens a private
|
||||
call-scoped nursery (booting the runtime if needed).
|
||||
|
||||
.. note::
|
||||
|
||||
``run_in_actor()`` is a convenience, **not** the core model.
|
||||
The source literally marks it for an eventual rebuild as
|
||||
a thin "hilevel" wrapper on top of
|
||||
:meth:`~tractor.Portal.open_context` (the modern inter-actor
|
||||
task API). Teach your fingers to use it for quick
|
||||
fire-and-collect parallelism — think a per-function
|
||||
``to_actor.run()`` is a convenience, **not** the core model —
|
||||
it's built *entirely* on ``start_actor()`` + ``Portal.run()``
|
||||
+ ``Portal.cancel_actor()``. Teach your fingers to use it for
|
||||
quick fire-and-collect parallelism — think a per-function
|
||||
trio-parallel_ style one-shot — and reach for
|
||||
``start_actor()`` + ``open_context()`` for anything
|
||||
long-lived, stateful or streaming
|
||||
|
|
@ -145,9 +151,9 @@ Actor lifetimes and teardown order
|
|||
----------------------------------
|
||||
So we have two lifetime flavors:
|
||||
|
||||
- **run-once** (``run_in_actor()``): lives exactly as long as
|
||||
- **one-shot** (``to_actor.run()``): lives exactly as long as
|
||||
its single task; reaped the moment its result (or error)
|
||||
arrives.
|
||||
arrives back in the (blocking) call.
|
||||
- **daemon** (``start_actor()``): lives until *someone* cancels
|
||||
it — an explicit ``await portal.cancel_actor()``, a bulk
|
||||
``await an.cancel()``, or the one-cancels-all strategy kicking
|
||||
|
|
@ -155,11 +161,12 @@ So we have two lifetime flavors:
|
|||
|
||||
On a clean exit of the nursery block the teardown order is:
|
||||
|
||||
1. the nursery waits on every run-once actor's final result;
|
||||
any errors from these are raised immediately so your code
|
||||
(acting as supervisor) gets first crack at handling them.
|
||||
2. then it waits on daemon actors — **indefinitely**. If you
|
||||
spawned a daemon, you own its lifetime.
|
||||
1. one-shot actors never make it to nursery exit: each is
|
||||
reaped inside its own ``to_actor.run()`` call, any error
|
||||
raising immediately in the calling task so your code
|
||||
(acting as supervisor) gets first crack at handling it.
|
||||
2. the nursery then waits on daemon actors — **indefinitely**.
|
||||
If you spawned a daemon, you own its lifetime.
|
||||
|
||||
When a child *is* cancelled, teardown is graceful-first per SC
|
||||
discipline: the runtime sends an IPC cancel request and gives
|
||||
|
|
|
|||
|
|
@ -185,8 +185,10 @@ first with a bounded grace window — so actor runtimes can run
|
|||
their ``trio`` teardown paths — escalating to ``SIGKILL`` only as
|
||||
a last resort. The ``--shm`` sweep unlinks ``/dev/shm/`` segments
|
||||
that no live process has open (it leans on psutil_, already in
|
||||
your dev venv, to check live mappings and fds) and ``--uds``
|
||||
clears socket files whose binder pid is dead.
|
||||
your dev venv, to check live mappings and fds) and ``--uds`` clears
|
||||
dead-binder sockets from Tractor's platform-specific runtime dir. It
|
||||
also unconditionally removes ``registry@1616.sock``; do not run the UDS
|
||||
sweep while a live registrar is serving from that default address.
|
||||
|
||||
Testing your own ``tractor`` app
|
||||
--------------------------------
|
||||
|
|
|
|||
|
|
@ -43,24 +43,20 @@ Run it::
|
|||
What's going on here?
|
||||
|
||||
- ``trio.run(main)`` starts the **root actor**; the ``tractor``
|
||||
runtime boots *implicitly* inside ``tractor.open_nursery()``
|
||||
runtime boots *implicitly* inside ``tractor.to_actor.run()``
|
||||
whenever it isn't already up. No special entrypoint, no
|
||||
framework takeover - it's just a ``trio`` app,
|
||||
- inside ``main()`` a *subactor* is spawned via
|
||||
``ActorNursery.run_in_actor()`` and told to run exactly one
|
||||
``tractor.to_actor.run()`` and told to run exactly one
|
||||
function: ``cellar_door()``,
|
||||
- you get back a ``Portal``: your handle for invoking tasks in
|
||||
the new process's (separate!) memory domain. We lean on it
|
||||
much harder in the next section,
|
||||
- the subactor, *some_linguist*, boots a fresh ``trio.run()`` in
|
||||
a **new process** and executes ``cellar_door()`` as its *main
|
||||
task* (note the child proving it is *not* the root with
|
||||
``tractor.is_root_process()``), then ships the return value
|
||||
back over IPC,
|
||||
- the parent grabs that *final result* with
|
||||
``await portal.wait_for_result()``, much like you'd expect
|
||||
from a "future" - except causality is preserved: the nursery
|
||||
block only exits once the child is *done*, dead, and reaped.
|
||||
- the call *blocks* until that final result arrives, then
|
||||
returns it - causality is preserved: your task only proceeds
|
||||
once the child is *done*, dead, and reaped.
|
||||
|
||||
.. margin:: Just need a worker pool?
|
||||
|
||||
|
|
@ -71,19 +67,22 @@ What's going on here?
|
|||
|
||||
.. note::
|
||||
|
||||
``run_in_actor()`` is the *convenience* wrapper: one-shot
|
||||
``to_actor.run()`` (parlance of ``trio.to_thread`` and
|
||||
friends) is the *convenience* wrapper: one-shot
|
||||
spawn-run-reap semantics for when a subactor's entire job is
|
||||
a single function call. The core primitives are
|
||||
``ActorNursery.start_actor()`` (next up) paired with
|
||||
``ActorNursery.start_actor()`` (next up) — which hands you
|
||||
a ``Portal``, your handle for invoking tasks in the new
|
||||
process's (separate!) memory domain — paired with
|
||||
``Portal.open_context()`` for full, SC-linked cross-actor
|
||||
dialogs - see :doc:`/guide/context`.
|
||||
|
||||
Daemon actors and RPC
|
||||
---------------------
|
||||
A ``run_in_actor()``-spawned actor terminates when its main task
|
||||
returns. But often you want long-lived *daemon* actors instead:
|
||||
spawned once, then serving (allowlisted) RPC requests until told
|
||||
otherwise. That's ``start_actor()``:
|
||||
A ``to_actor.run()`` one-shot subactor terminates when its lone
|
||||
task returns. But often you want long-lived *daemon* actors
|
||||
instead: spawned once, then serving (allowlisted) RPC requests
|
||||
until told otherwise. That's ``start_actor()``:
|
||||
|
||||
.. literalinclude:: ../../examples/actor_spawning_and_causality_with_daemon.py
|
||||
:caption: examples/actor_spawning_and_causality_with_daemon.py
|
||||
|
|
@ -91,9 +90,9 @@ otherwise. That's ``start_actor()``:
|
|||
|
||||
Two lifetime rules to internalize:
|
||||
|
||||
- a ``run_in_actor()`` actor lives exactly as long as its main
|
||||
task; the nursery waits for that function (and thus the
|
||||
process) to complete before unblocking,
|
||||
- a ``to_actor.run()`` one-shot actor lives exactly as long as
|
||||
its lone task; the call blocks until that function (and thus
|
||||
the process) completes,
|
||||
- a ``start_actor()`` actor *lives forever* - an RPC daemon the
|
||||
nursery will happily wait on **indefinitely** - until some
|
||||
task explicitly cancels it via ``Portal.cancel_actor()`` (as
|
||||
|
|
|
|||
|
|
@ -21,22 +21,34 @@ async def main():
|
|||
"""Main tractor entry point, the "master" process (for now
|
||||
acts as the "director").
|
||||
"""
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
print("Alright... Action!")
|
||||
|
||||
donny = await n.run_in_actor(
|
||||
say_hello,
|
||||
name='donny',
|
||||
# arguments are always named
|
||||
other_actor='gretchen',
|
||||
# both actors wait on (then dial!) the *other*, so each
|
||||
# must outlive both hellos: spawn as daemons, run the
|
||||
# hellos concurrently, reap only once both complete.
|
||||
portals: dict[str, tractor.Portal] = {
|
||||
name: await an.start_actor(
|
||||
name,
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
gretchen = await n.run_in_actor(
|
||||
for name in ('donny', 'gretchen')
|
||||
}
|
||||
|
||||
async def run_and_print(name: str, other_actor: str):
|
||||
print(
|
||||
await portals[name].run(
|
||||
say_hello,
|
||||
name='gretchen',
|
||||
other_actor='donny',
|
||||
other_actor=other_actor,
|
||||
)
|
||||
print(await gretchen.wait_for_result())
|
||||
print(await donny.wait_for_result())
|
||||
)
|
||||
|
||||
async with trio.open_nursery() as tn:
|
||||
tn.start_soon(run_and_print, 'donny', 'gretchen')
|
||||
tn.start_soon(run_and_print, 'gretchen', 'donny')
|
||||
|
||||
await an.cancel()
|
||||
|
||||
print("CUTTTT CUUTT CUT!!! Donny!! You're supposed to say...")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -10,17 +10,14 @@ async def cellar_door():
|
|||
async def main():
|
||||
"""The main ``tractor`` routine.
|
||||
"""
|
||||
async with tractor.open_nursery() as n:
|
||||
|
||||
portal = await n.run_in_actor(
|
||||
# spawn a subactor, run ``cellar_door()`` as its lone task,
|
||||
# block until its result arrives and the subactor is reaped.
|
||||
print(
|
||||
await tractor.to_actor.run(
|
||||
cellar_door,
|
||||
name='some_linguist',
|
||||
)
|
||||
|
||||
# The ``async with`` will unblock here since the 'some_linguist'
|
||||
# actor has completed its main task ``cellar_door``.
|
||||
|
||||
print(await portal.wait_for_result())
|
||||
)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -12,9 +12,9 @@ async def movie_theatre_question():
|
|||
async def main():
|
||||
"""The main ``tractor`` routine.
|
||||
"""
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
|
||||
portal = await n.start_actor(
|
||||
portal = await an.start_actor(
|
||||
'frank',
|
||||
# enable the actor to run funcs from this current module
|
||||
enable_modules=[__name__],
|
||||
|
|
|
|||
|
|
@ -15,9 +15,9 @@ async def stream_forever() -> AsyncIterator[int]:
|
|||
|
||||
async def main():
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
|
||||
portal = await n.start_actor(
|
||||
portal = await an.start_actor(
|
||||
'donny',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
from functools import partial
|
||||
|
||||
import trio
|
||||
import tractor
|
||||
|
||||
|
|
@ -21,25 +23,36 @@ async def breakpoint_forever():
|
|||
async def spawn_until(depth=0):
|
||||
""""A nested nursery that triggers another ``NameError``.
|
||||
"""
|
||||
async with tractor.open_nursery() as n:
|
||||
async with (
|
||||
tractor.open_nursery() as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
if depth < 1:
|
||||
|
||||
await n.run_in_actor(breakpoint_forever)
|
||||
|
||||
p = await n.run_in_actor(
|
||||
name_error,
|
||||
name='name_error'
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
breakpoint_forever,
|
||||
an=an,
|
||||
)
|
||||
)
|
||||
|
||||
await trio.sleep(0.5)
|
||||
# rx and propagate error from child
|
||||
await p.result()
|
||||
await tractor.to_actor.run(
|
||||
name_error,
|
||||
an=an,
|
||||
name='name_error',
|
||||
)
|
||||
|
||||
else:
|
||||
# recusrive call to spawn another process branching layer of
|
||||
# the tree
|
||||
# the tree; blocks (up) each level until the leaf's
|
||||
# `name_error` relays through.
|
||||
depth -= 1
|
||||
await n.run_in_actor(
|
||||
await tractor.to_actor.run(
|
||||
spawn_until,
|
||||
an=an,
|
||||
depth=depth,
|
||||
name=f'spawn_until_{depth}',
|
||||
)
|
||||
|
|
@ -65,34 +78,33 @@ async def main():
|
|||
└─ python -m tractor._child --uid ('spawn_until_0', 'de918e6d ...)
|
||||
|
||||
"""
|
||||
async with tractor.open_nursery(
|
||||
async with (
|
||||
tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
loglevel='pdb',
|
||||
) as n:
|
||||
|
||||
# spawn both actors
|
||||
portal = await n.run_in_actor(
|
||||
) as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
# spawn both spawner trees as concurrent one-shots; the
|
||||
# first tree's (relayed) error cancels the other.
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
spawn_until,
|
||||
an=an,
|
||||
depth=3,
|
||||
name='spawner0',
|
||||
)
|
||||
portal1 = await n.run_in_actor(
|
||||
)
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
spawn_until,
|
||||
an=an,
|
||||
depth=4,
|
||||
name='spawner1',
|
||||
)
|
||||
|
||||
# TODO: test this case as well where the parent don't see
|
||||
# the sub-actor errors by default and instead expect a user
|
||||
# ctrl-c to kill the root.
|
||||
with trio.move_on_after(3):
|
||||
await trio.sleep_forever()
|
||||
|
||||
# gah still an issue here.
|
||||
await portal.result()
|
||||
|
||||
# should never get here
|
||||
await portal1.result()
|
||||
)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -15,12 +15,12 @@ async def name_error():
|
|||
async def spawn_error():
|
||||
""""A nested nursery that triggers another ``NameError``.
|
||||
"""
|
||||
async with tractor.open_nursery() as n:
|
||||
portal = await n.run_in_actor(
|
||||
async with tractor.open_nursery() as an:
|
||||
return await tractor.to_actor.run(
|
||||
name_error,
|
||||
an=an,
|
||||
name='name_error_1',
|
||||
)
|
||||
return await portal.result()
|
||||
|
||||
|
||||
async def main():
|
||||
|
|
@ -38,29 +38,36 @@ async def main():
|
|||
- root actor should then fail on assert
|
||||
- program termination
|
||||
"""
|
||||
async with tractor.open_nursery(
|
||||
async with (
|
||||
tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
loglevel='devx',
|
||||
) as n:
|
||||
) as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
# spawn both actors..
|
||||
portal = await an.start_actor(
|
||||
'name_error',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
portal1 = await an.start_actor(
|
||||
'spawn_error',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
||||
# spawn both actors
|
||||
portal = await n.run_in_actor(
|
||||
name_error,
|
||||
name='name_error',
|
||||
)
|
||||
portal1 = await n.run_in_actor(
|
||||
spawn_error,
|
||||
name='spawn_error',
|
||||
)
|
||||
# ..and bg-schedule their erroring tasks.
|
||||
tn.start_soon(portal.run, name_error)
|
||||
tn.start_soon(portal1.run, spawn_error)
|
||||
|
||||
# yield to the bg tasks so both RPC requests are
|
||||
# submitted (and start crashing) before the root's own
|
||||
# error below (the legacy `run_in_actor()` submitted
|
||||
# in-line with each spawn).
|
||||
await trio.sleep(0.5)
|
||||
|
||||
# trigger a root actor error
|
||||
assert 0
|
||||
|
||||
# attempt to collect results (which raises error in parent)
|
||||
# still has some issues where the parent seems to get stuck
|
||||
await portal.result()
|
||||
await portal1.result()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
trio.run(main)
|
||||
|
|
|
|||
|
|
@ -17,12 +17,12 @@ async def name_error():
|
|||
async def spawn_error():
|
||||
""""A nested nursery that triggers another ``NameError``.
|
||||
"""
|
||||
async with tractor.open_nursery() as n:
|
||||
portal = await n.run_in_actor(
|
||||
async with tractor.open_nursery() as an:
|
||||
return await tractor.to_actor.run(
|
||||
name_error,
|
||||
an=an,
|
||||
name='name_error_1',
|
||||
)
|
||||
return await portal.result()
|
||||
|
||||
|
||||
async def main():
|
||||
|
|
@ -36,17 +36,39 @@ async def main():
|
|||
`-python -m tractor._child --uid ('spawn_error', '52ee14a5 ...)
|
||||
`-python -m tractor._child --uid ('name_error', '3391222c ...)
|
||||
"""
|
||||
errors: list[BaseException] = []
|
||||
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
# loglevel='runtime',
|
||||
) as n:
|
||||
) as an:
|
||||
|
||||
# Spawn both actors, don't bother with collecting results
|
||||
# (would result in a different debugger outcome due to parent's
|
||||
# cancellation).
|
||||
await n.run_in_actor(breakpoint_forever)
|
||||
await n.run_in_actor(name_error)
|
||||
await n.run_in_actor(spawn_error)
|
||||
async def run_and_collect(fn):
|
||||
'''
|
||||
One-shot whose (boxed) error is stashed instead of
|
||||
raised so a sibling's crash never cancels the others
|
||||
before they've had their own debugger sessions (the
|
||||
"collect all errors" the legacy `run_in_actor()` API
|
||||
did implicitly at nursery teardown).
|
||||
|
||||
'''
|
||||
try:
|
||||
await tractor.to_actor.run(fn, an=an)
|
||||
except tractor.RemoteActorError as rae:
|
||||
errors.append(rae)
|
||||
|
||||
# Spawn all one-shot task actors, collecting (vs.
|
||||
# raising) their errors.
|
||||
async with trio.open_nursery() as tn:
|
||||
tn.start_soon(run_and_collect, breakpoint_forever)
|
||||
tn.start_soon(run_and_collect, name_error)
|
||||
tn.start_soon(run_and_collect, spawn_error)
|
||||
|
||||
if errors:
|
||||
raise BaseExceptionGroup(
|
||||
'multi_subactors errored!',
|
||||
errors,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -21,8 +21,8 @@ async def main() -> None:
|
|||
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
) as n:
|
||||
portal = await n.start_actor(
|
||||
) as an:
|
||||
portal = await an.start_actor(
|
||||
'ctx_child',
|
||||
|
||||
# XXX: we don't enable the current module in order
|
||||
|
|
|
|||
|
|
@ -6,14 +6,14 @@ async def die():
|
|||
|
||||
|
||||
async def main():
|
||||
async with tractor.open_nursery() as tn:
|
||||
async with tractor.open_nursery() as an:
|
||||
|
||||
debug_actor = await tn.start_actor(
|
||||
debug_actor = await an.start_actor(
|
||||
'debugged_boi',
|
||||
enable_modules=[__name__],
|
||||
debug_mode=True,
|
||||
)
|
||||
crash_boi = await tn.start_actor(
|
||||
crash_boi = await an.start_actor(
|
||||
'crash_boi',
|
||||
enable_modules=[__name__],
|
||||
# debug_mode=True,
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
from functools import partial
|
||||
|
||||
import trio
|
||||
import tractor
|
||||
|
||||
|
|
@ -10,14 +12,14 @@ async def name_error():
|
|||
async def spawn_until(depth=0):
|
||||
""""A nested nursery that triggers another ``NameError``.
|
||||
"""
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
if depth < 1:
|
||||
# await n.run_in_actor('breakpoint_forever', breakpoint_forever)
|
||||
await n.run_in_actor(name_error)
|
||||
await tractor.to_actor.run(name_error, an=an)
|
||||
else:
|
||||
depth -= 1
|
||||
await n.run_in_actor(
|
||||
await tractor.to_actor.run(
|
||||
spawn_until,
|
||||
an=an,
|
||||
depth=depth,
|
||||
name=f'spawn_until_{depth}',
|
||||
)
|
||||
|
|
@ -37,28 +39,33 @@ async def main():
|
|||
└─ python -m tractor._child --uid ('name_error', '6c2733b8 ...)
|
||||
|
||||
'''
|
||||
async with tractor.open_nursery(
|
||||
async with (
|
||||
tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
enable_transports=['uds'], # TODO, apss this via osenv?
|
||||
loglevel='devx', # XXX, required for test!
|
||||
) as n:
|
||||
|
||||
# spawn both actors
|
||||
portal = await n.run_in_actor(
|
||||
spawn_until,
|
||||
depth=0,
|
||||
name='spawner0',
|
||||
)
|
||||
portal1 = await n.run_in_actor(
|
||||
) as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
# spawn the deeper tree in the bg..
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
spawn_until,
|
||||
an=an,
|
||||
depth=1,
|
||||
name='spawner1',
|
||||
)
|
||||
)
|
||||
|
||||
# nursery cancellation should be triggered due to propagated
|
||||
# error from child.
|
||||
await portal.result()
|
||||
await portal1.result()
|
||||
# ..while blocking on the shallow (faster to fail) tree
|
||||
# whose propagated error triggers nursery cancellation.
|
||||
await tractor.to_actor.run(
|
||||
spawn_until,
|
||||
an=an,
|
||||
depth=0,
|
||||
name='spawner0',
|
||||
)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -13,17 +13,24 @@ async def main():
|
|||
simultaneously.
|
||||
|
||||
'''
|
||||
async with tractor.open_nursery(
|
||||
async with (
|
||||
tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
# loglevel='debug' # ?XXX required?
|
||||
) as n:
|
||||
|
||||
# spawn both actors
|
||||
portal = await n.run_in_actor(key_error)
|
||||
) as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
# spawn the actor..
|
||||
portal = await an.start_actor(
|
||||
'key_error',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
print(
|
||||
f'Child is up @ {portal.chan.aid.reprol()}'
|
||||
)
|
||||
|
||||
# ..then schedule its erroring task in the bg while the
|
||||
# root blocks below.
|
||||
tn.start_soon(portal.run, key_error)
|
||||
|
||||
# XXX: originally a bug caused by this is where root would enter
|
||||
# the debugger and clobber the tty used by the repl even though
|
||||
|
|
|
|||
|
|
@ -74,11 +74,11 @@ async def cancelled_before_pause(
|
|||
async def main():
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
) as n:
|
||||
portal: tractor.Portal = await n.run_in_actor(
|
||||
) as an:
|
||||
await tractor.to_actor.run(
|
||||
cancelled_before_pause,
|
||||
an=an,
|
||||
)
|
||||
await portal.wait_for_result()
|
||||
|
||||
# ensure the same works in the root actor!
|
||||
await pm_on_cancelled()
|
||||
|
|
|
|||
|
|
@ -58,8 +58,8 @@ async def main():
|
|||
debug_mode=True,
|
||||
enable_transports=[tpt],
|
||||
loglevel='devx',
|
||||
) as n:
|
||||
p = await n.start_actor(
|
||||
) as an:
|
||||
p = await an.start_actor(
|
||||
'bp_boi',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -17,12 +17,14 @@ async def main():
|
|||
async with tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
loglevel='cancel',
|
||||
) as n:
|
||||
) as an:
|
||||
|
||||
portal = await n.run_in_actor(
|
||||
# parks awaiting a result which only arrives once the
|
||||
# user quits (`BdbQuit`s) the child's REPL loop.
|
||||
await tractor.to_actor.run(
|
||||
breakpoint_forever,
|
||||
an=an,
|
||||
)
|
||||
await portal.wait_for_result()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -12,16 +12,12 @@ async def main():
|
|||
) as an:
|
||||
|
||||
# TODO: ideally the REPL arrives at this frame in the parent,
|
||||
# ABOVE the @api_frame of `Portal.run_in_actor()` (which
|
||||
# should eventually not even be a portal method ... XD)
|
||||
# ABOVE the @api_frame of `to_actor.run()` ..
|
||||
# await tractor.pause()
|
||||
p: tractor.Portal = await an.run_in_actor(name_error)
|
||||
|
||||
# with this style, should raise on this line
|
||||
await p.wait_for_result()
|
||||
|
||||
# with this alt style should raise at `open_nusery()`
|
||||
# return await p.wait_for_result()
|
||||
# the one-shot blocks on the subactor's result so the
|
||||
# boxed `NameError` raises right here.
|
||||
await tractor.to_actor.run(name_error, an=an)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -90,7 +90,7 @@ async def main() -> None:
|
|||
|
||||
# TODO: 3 sub-actor usage cases:
|
||||
# -[x] via a `.open_context()`
|
||||
# -[ ] via a `.run_in_actor()` call
|
||||
# -[ ] via a `to_actor.run()` call
|
||||
# -[ ] via a `.run()`
|
||||
# -[ ] via a `.to_thread.run_sync()` in subactor
|
||||
async with p.open_context(
|
||||
|
|
|
|||
|
|
@ -50,8 +50,8 @@ async def trio_to_aio_echo_server(
|
|||
|
||||
async def main():
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
p = await n.start_actor(
|
||||
async with tractor.open_nursery() as an:
|
||||
p = await an.start_actor(
|
||||
'aio_server',
|
||||
enable_modules=[__name__],
|
||||
infect_asyncio=True,
|
||||
|
|
|
|||
|
|
@ -29,9 +29,9 @@ async def main() -> None:
|
|||
))
|
||||
await proc.wait()
|
||||
# await trio.sleep_forever()
|
||||
# async with tractor.open_nursery() as n:
|
||||
# async with tractor.open_nursery() as an:
|
||||
|
||||
# portal = await n.start_actor(
|
||||
# portal = await an.start_actor(
|
||||
# 'rpc_server',
|
||||
# enable_modules=[__name__],
|
||||
# )
|
||||
|
|
|
|||
|
|
@ -55,7 +55,7 @@ async def worker_pool(workers=4):
|
|||
Yes, the workers stay alive (and ready for work) until you close
|
||||
the context.
|
||||
"""
|
||||
async with tractor.open_nursery() as tn:
|
||||
async with tractor.open_nursery() as an:
|
||||
|
||||
portals = []
|
||||
snd_chan, recv_chan = trio.open_memory_channel(len(PRIMES))
|
||||
|
|
@ -65,7 +65,7 @@ async def worker_pool(workers=4):
|
|||
# this starts a new sub-actor (process + trio runtime) and
|
||||
# stores it's "portal" for later use to "submit jobs" (ugh).
|
||||
portals.append(
|
||||
await tn.start_actor(
|
||||
await an.start_actor(
|
||||
f'worker_{i}',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
@ -80,10 +80,10 @@ async def worker_pool(workers=4):
|
|||
async def send_result(func, value, portal):
|
||||
await snd_chan.send((value, await portal.run(func, n=value)))
|
||||
|
||||
async with trio.open_nursery() as n:
|
||||
async with trio.open_nursery() as tn:
|
||||
|
||||
for value, portal in zip(sequence, itertools.cycle(portals)):
|
||||
n.start_soon(
|
||||
tn.start_soon(
|
||||
send_result,
|
||||
worker_func,
|
||||
value,
|
||||
|
|
@ -98,7 +98,7 @@ async def worker_pool(workers=4):
|
|||
yield _map
|
||||
|
||||
# tear down all "workers" on pool close
|
||||
await tn.cancel()
|
||||
await an.cancel()
|
||||
|
||||
|
||||
async def main():
|
||||
|
|
|
|||
|
|
@ -25,17 +25,15 @@ async def burn_cpu():
|
|||
|
||||
async def main():
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
|
||||
portal = await n.run_in_actor(burn_cpu)
|
||||
async with trio.open_nursery() as tn:
|
||||
|
||||
# burn rubber in the parent too
|
||||
await burn_cpu()
|
||||
tn.start_soon(burn_cpu)
|
||||
|
||||
# wait on result from target function
|
||||
pid = await portal.wait_for_result()
|
||||
# run the same func as the lone task in a subactor,
|
||||
# block on (and collect) its result
|
||||
pid = await tractor.to_actor.run(burn_cpu)
|
||||
|
||||
# end of nursery block
|
||||
print(f"Collected subproc {pid}")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,83 @@
|
|||
'''
|
||||
`tractor.to_actor.run()`: one-shot single-task subactor
|
||||
invocation, the SC-parallelism sibling of
|
||||
`trio.to_thread.run_sync()` (and `anyio.to_process`).
|
||||
|
||||
Each call spawns a subactor, schedules the async fn as
|
||||
its lone remote task, waits on the result and reaps the
|
||||
subactor. Concurrency composes the plain `trio` way:
|
||||
schedule multiple one-shot calls in a local task nursery
|
||||
against a shared actor-nursery; any remote error raises
|
||||
directly in the task which scheduled it.
|
||||
|
||||
'''
|
||||
import math
|
||||
|
||||
import tractor
|
||||
import trio
|
||||
|
||||
|
||||
async def is_prime(
|
||||
n: int,
|
||||
) -> bool:
|
||||
if n < 2:
|
||||
return False
|
||||
if n == 2:
|
||||
return True
|
||||
if n % 2 == 0:
|
||||
return False
|
||||
|
||||
sqrt_n = int(math.floor(math.sqrt(n)))
|
||||
for i in range(3, sqrt_n + 1, 2):
|
||||
if n % i == 0:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
|
||||
# fully implicit one-shot: boots the actor-runtime,
|
||||
# spawns a subactor, runs the task, reaps the
|
||||
# subactor, tears the runtime back down.
|
||||
assert await tractor.to_actor.run(
|
||||
is_prime,
|
||||
n=2,
|
||||
)
|
||||
|
||||
# the "worker-pool-ish" pattern from the original
|
||||
# `concurrent.futures` example: one subactor per
|
||||
# input, all concurrent, results and errors
|
||||
# collected by caller-side tasks.
|
||||
results: dict[int, bool] = {}
|
||||
|
||||
async def check(
|
||||
an: tractor.ActorNursery,
|
||||
n: int,
|
||||
i: int,
|
||||
) -> None:
|
||||
results[n] = await tractor.to_actor.run(
|
||||
is_prime,
|
||||
an=an,
|
||||
name=f'prime_checker_{i}',
|
||||
n=n,
|
||||
)
|
||||
|
||||
inputs: list[int] = [
|
||||
7,
|
||||
8,
|
||||
3691,
|
||||
3693,
|
||||
]
|
||||
async with (
|
||||
tractor.open_nursery() as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
for i, n in enumerate(inputs):
|
||||
tn.start_soon(check, an, n, i)
|
||||
|
||||
for n, prime in sorted(results.items()):
|
||||
print(f'{n} is prime: {prime}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
trio.run(main)
|
||||
|
|
@ -23,18 +23,10 @@ async def endpoint(
|
|||
await trio.sleep_forever()
|
||||
|
||||
|
||||
async def spawn_and_open_ep(
|
||||
an: tractor.ActorNursery,
|
||||
async def open_ep(
|
||||
ptl: tractor.Portal,
|
||||
i: int,
|
||||
) -> None:
|
||||
'''
|
||||
Spawn a subactor, start a remote `endpoint()`-task in it.
|
||||
|
||||
'''
|
||||
ptl: tractor.Portal = await an.start_actor(
|
||||
name=f'worker_{i}',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
ctx: tractor.Context
|
||||
async with ptl.open_context(endpoint) as (
|
||||
ctx,
|
||||
|
|
@ -47,7 +39,33 @@ async def spawn_and_open_ep(
|
|||
await ctx.wait_for_result()
|
||||
|
||||
|
||||
async def main():
|
||||
async def spawn_and_open_ep(
|
||||
an: tractor.ActorNursery,
|
||||
i: int,
|
||||
maybe_ptl: tractor.Portal|None = None,
|
||||
) -> None:
|
||||
'''
|
||||
Spawn a subactor, start a remote `endpoint()`-task in it.
|
||||
|
||||
'''
|
||||
if maybe_ptl is None:
|
||||
maybe_ptl: tractor.Portal = await an.start_actor(
|
||||
name=f'worker_{i}',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
await open_ep(
|
||||
ptl=maybe_ptl,
|
||||
i=i,
|
||||
)
|
||||
|
||||
|
||||
async def main(
|
||||
# spawn subs concurrently (in bg `trio.Task`s) so each
|
||||
# actor's cold `import tractor` (~0.4s, see #470) overlaps
|
||||
# instead of stacking; once forkserver (#463) lands, spawn
|
||||
# is cheap enough to just loop sequentially.
|
||||
spawn_subs_in_bg_tasks: bool = True,
|
||||
):
|
||||
'''
|
||||
Spawn a subactor-per-CPU then self-destruct the cluster.
|
||||
|
||||
|
|
@ -60,17 +78,21 @@ async def main():
|
|||
# https://github.com/goodboy/tractor/pull/463
|
||||
# start_method='main_thread_forkserver',
|
||||
) as an,
|
||||
# spawn subs concurrently (in bg `trio.Task`s) so each
|
||||
# actor's cold `import tractor` (~0.4s, see #470) overlaps
|
||||
# instead of stacking; once forkserver (#463) lands, spawn
|
||||
# is cheap enough to just loop sequentially.
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
for i in range(cpu_count()):
|
||||
|
||||
maybe_ptl: tractor.Portal|None = None
|
||||
if not spawn_subs_in_bg_tasks:
|
||||
maybe_ptl: tractor.Portal = await an.start_actor(
|
||||
name=f'worker_{i}',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
tn.start_soon(
|
||||
spawn_and_open_ep,
|
||||
an,
|
||||
i,
|
||||
maybe_ptl,
|
||||
)
|
||||
destruct_in: int = 2
|
||||
print(
|
||||
|
|
|
|||
|
|
@ -7,19 +7,20 @@ async def assert_err():
|
|||
|
||||
|
||||
async def main():
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
real_actors = []
|
||||
for i in range(3):
|
||||
real_actors.append(await n.start_actor(
|
||||
real_actors.append(await an.start_actor(
|
||||
f'actor_{i}',
|
||||
enable_modules=[__name__],
|
||||
))
|
||||
|
||||
# start one actor that will fail immediately
|
||||
await n.run_in_actor(assert_err)
|
||||
# run one one-shot task actor that will fail immediately;
|
||||
# its error raises right here in the caller's task..
|
||||
await tractor.to_actor.run(assert_err, an=an)
|
||||
|
||||
# should error here with a ``RemoteActorError`` containing
|
||||
# an ``AssertionError`` and all the other actors have been cancelled
|
||||
# ..as a ``RemoteActorError`` containing an ``AssertionError``
|
||||
# and all the other actors have been cancelled
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -31,9 +31,9 @@ async def simple_rpc(
|
|||
|
||||
async def main() -> None:
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
|
||||
portal = await n.start_actor(
|
||||
portal = await an.start_actor(
|
||||
'rpc_server',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -6,7 +6,8 @@ subactor inherits the preference.
|
|||
|
||||
Every channel address is a filesystem socket path (no TCP port
|
||||
in sight!) and, as a kernel-provided bonus, the peer's pid is
|
||||
exchanged for free via `SO_PEERCRED`.
|
||||
exchanged for free via `SO_PEERCRED` on linux,
|
||||
`LOCAL_PEERPID` on macOS.
|
||||
|
||||
'''
|
||||
import os
|
||||
|
|
@ -42,7 +43,7 @@ async def main() -> None:
|
|||
# (named for the root registrar) this channel rode in
|
||||
# on, NOT a per-child path; the child-specific identity
|
||||
# we get for free is the kernel-reported peer pid (via
|
||||
# `SO_PEERCRED`).
|
||||
# `SO_PEERCRED` on linux, `LOCAL_PEERPID` on macOS).
|
||||
print(
|
||||
f'portal chan tpt proto: {raddr.proto_key!r}\n'
|
||||
f'listener sock file: {raddr.sockpath}\n'
|
||||
|
|
|
|||
|
|
@ -0,0 +1,4 @@
|
|||
Fix Unix-domain-socket actor trees and registrar discovery on macOS.
|
||||
Runtime sockets now use a short, owner-only runtime directory,
|
||||
generated socket names remain within platform limits, and transient
|
||||
or reset pre-handshake connections no longer destabilize discovery.
|
||||
|
|
@ -23,10 +23,10 @@ Two cleanup phases (run in order when both are enabled):
|
|||
hard-crashing actor leaves leaked segments that
|
||||
nothing else GCs.
|
||||
|
||||
3. **UDS sweep** (`--uds` / `--uds-only`) — unlinks
|
||||
`${XDG_RUNTIME_DIR}/tractor/<name>@<pid>.sock` files
|
||||
whose binder pid is dead (or the `1616` registry
|
||||
sentinel). Needed because the IPC server's
|
||||
3. **UDS sweep** (`--uds` / `--uds-only`) — unlinks socket
|
||||
files from Tractor's platform-specific default bindspace whose
|
||||
binder pid is dead (or the `1616` registry sentinel). Needed
|
||||
because the IPC server's
|
||||
`os.unlink()` cleanup lives in a `finally:` block
|
||||
that doesn't always run on hard exits (SIGKILL,
|
||||
escaped `KeyboardInterrupt`, etc.) — see issue #452.
|
||||
|
|
@ -137,8 +137,8 @@ def main() -> int:
|
|||
action='store_true',
|
||||
help=(
|
||||
'after process reap, also unlink orphaned '
|
||||
'${XDG_RUNTIME_DIR}/tractor/*.sock files '
|
||||
'whose binder pid is dead (or the 1616 '
|
||||
'sockets from Tractor\'s platform default '
|
||||
'bindspace whose binder pid is dead (or the 1616 '
|
||||
'registry sentinel). See issue #452.'
|
||||
),
|
||||
)
|
||||
|
|
@ -212,7 +212,9 @@ def main() -> int:
|
|||
|
||||
# --- phase 3: UDS sweep (opt-in) ---
|
||||
if args.uds or args.uds_only:
|
||||
leaked_uds: list[str] = find_orphaned_uds()
|
||||
leaked_uds: list[str] = find_orphaned_uds(
|
||||
include_registry_sentinel=True,
|
||||
)
|
||||
if not leaked_uds:
|
||||
print(
|
||||
'[tractor-reap] no orphaned UDS sock-files '
|
||||
|
|
|
|||
|
|
@ -849,43 +849,36 @@ def test_multi_nested_subactors_error_through_nurseries(
|
|||
break
|
||||
|
||||
# boxed source errors
|
||||
#
|
||||
# NB post-#477 (`to_actor.run()` one-shots in local
|
||||
# task-nurseries) the final relay is the LAST-released
|
||||
# (leaf) REPL's error chain: it wins each level's
|
||||
# relay-vs-cancel race so every level's single-member
|
||||
# group gets unwrapped by the runtime's `collapse_eg()`
|
||||
# (annotated at each actor boundary) while the sibling
|
||||
# tree ('spawner1') is cancelled + absorbed. The legacy
|
||||
# `run_in_actor()` teardown-reap instead grouped BOTH the
|
||||
# `name_error` and bp-quit chains into the final dump
|
||||
# (the previously-unexplained "extra" patterns).
|
||||
expect_patts: list[str] = [
|
||||
"NameError: name 'doggypants' is not defined",
|
||||
"tractor._exceptions.RemoteActorError:",
|
||||
"('name_error'",
|
||||
|
||||
# first level subtrees
|
||||
# "tractor._exceptions.RemoteActorError: ('spawner0'",
|
||||
"src_uid=('spawner0'",
|
||||
|
||||
# "tractor._exceptions.RemoteActorError: ('spawner1'",
|
||||
|
||||
# propagation of errors up through nested subtrees
|
||||
# "tractor._exceptions.RemoteActorError: ('spawn_until_0'",
|
||||
# "tractor._exceptions.RemoteActorError: ('spawn_until_1'",
|
||||
# "tractor._exceptions.RemoteActorError: ('spawn_until_2'",
|
||||
# ^-NOTE-^ old RAE repr, new one is below with a field
|
||||
# showing the src actor's uid.
|
||||
"src_uid=('spawn_until_2'",
|
||||
# each level's unwrapped-single-member-group
|
||||
# annotation + the first-level subtree's boundary
|
||||
# footer.
|
||||
"( ^^^ this exc was collapsed from a group ^^^ )",
|
||||
"------ ('spawner0'",
|
||||
]
|
||||
# XXX, I HAVE NO IDEA why these patts only show on the
|
||||
# `trio`-spawner but it seems to have something to do with
|
||||
# what gets dumped in prior-prompt latches somehow??
|
||||
# TODO for claude, explain and or work through how this is
|
||||
# happening but ONLY WHEN RUN FROM THE TEST, bc when i try to
|
||||
# run the test script manually the correct output ALWAYS seems
|
||||
# to be in the last `str(child.before.decode())` output !?!?
|
||||
if (
|
||||
not is_forking_spawner
|
||||
and
|
||||
last_send_char == 'q'
|
||||
):
|
||||
expect_patts += [
|
||||
# expect the pdb-quit exc.
|
||||
# expect the pdb-quit exc relayed from the leaf's
|
||||
# bp-loop child.
|
||||
"bdb.BdbQuit",
|
||||
# BUT WHY these dude!?
|
||||
"src_uid=('spawn_until_0'",
|
||||
"relay_uid=('spawn_until_1'",
|
||||
"src_uid=('breakpoint_forever'",
|
||||
]
|
||||
|
||||
assert_before(
|
||||
|
|
@ -1307,19 +1300,31 @@ def test_ctxep_pauses_n_maybe_ipc_breaks(
|
|||
if _non_linux:
|
||||
tpt: str = 'TCP'
|
||||
|
||||
assert_before(
|
||||
before: str = assert_before(
|
||||
child,
|
||||
['peer IPC channel closed abruptly?',
|
||||
'another task closed this fd',
|
||||
'Debug lock request was CANCELLED?',
|
||||
f"'Msgpack{tpt}Stream' was already closed locally?",
|
||||
f"TransportClosed: 'Msgpack{tpt}Stream' was already closed 'by peer'?",
|
||||
]
|
||||
|
||||
# XXX races on whether these show/hit?
|
||||
# 'Failed to REPl via `_pause()` You called `tractor.pause()` from an already cancelled scope!',
|
||||
# 'AssertionError',
|
||||
)
|
||||
|
||||
# Error shipment and peer receive race after local close.
|
||||
# Either diagnostic proves the transport was torn down.
|
||||
closed_locally: str = (
|
||||
f"'Msgpack{tpt}Stream' was already closed locally?"
|
||||
)
|
||||
closed_by_peer: str = (
|
||||
f"TransportClosed: 'Msgpack{tpt}Stream' was "
|
||||
f"already closed 'by peer'?"
|
||||
)
|
||||
assert (
|
||||
closed_locally in before
|
||||
or closed_by_peer in before
|
||||
)
|
||||
# OSc(ancel) the hanging tree
|
||||
do_ctlc(
|
||||
child=child,
|
||||
|
|
|
|||
|
|
@ -1,21 +1,18 @@
|
|||
'''
|
||||
Discovery-suite fixtures, including the `daemon`
|
||||
remote-registrar subprocess used by the multi-program
|
||||
discovery tests.
|
||||
Discovery-suite fixtures, including the `daemon` remote-registrar
|
||||
subprocess used by the multi-program discovery tests.
|
||||
|
||||
Lives here (vs. the parent `tests/conftest.py`)
|
||||
because `daemon` is a discovery-protocol primitive —
|
||||
boots a separate `tractor.run_daemon()` process whose
|
||||
sole purpose is to serve as a registrar peer for
|
||||
because `daemon` is a discovery-protocol primitive: it boots a child
|
||||
that enters `open_root_actor()` and waits as a registrar peer for
|
||||
discovery-roundtrip tests. Pytest fixtures inherit
|
||||
DOWNWARD through conftest hierarchy, so anything
|
||||
under `tests/discovery/` automatically picks this up.
|
||||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
|
@ -31,33 +28,27 @@ from ..conftest import (
|
|||
|
||||
|
||||
def _wait_for_daemon_ready(
|
||||
reg_addr: tuple,
|
||||
tpt_proto: str,
|
||||
ready_path: Path,
|
||||
*,
|
||||
deadline: float = 10.0,
|
||||
poll_interval: float = 0.05,
|
||||
proc: subprocess.Popen|None = None,
|
||||
) -> None:
|
||||
'''
|
||||
Active-poll the daemon's bind address until it
|
||||
accepts a connection (proving it has called
|
||||
`bind() + listen()` and is ready to handle IPC).
|
||||
Poll until the daemon reports completed actor startup.
|
||||
|
||||
Replaces the historical blind `time.sleep()` in the
|
||||
`daemon` fixture which was racy under load — see
|
||||
`ai/conc-anal/test_register_duplicate_name_daemon_connect_race_issue.md`.
|
||||
|
||||
Uses stdlib `socket` directly (no trio runtime
|
||||
bootstrap cost) — sufficient because
|
||||
`tractor.run_daemon()` doesn't return from
|
||||
bootstrap until the runtime is fully ready to
|
||||
accept IPC.
|
||||
The child writes `ready_path` only after entering
|
||||
`open_root_actor()`, which guarantees all transport listeners are
|
||||
serving without requiring a raw connection probe.
|
||||
|
||||
Raises `TimeoutError` on `deadline` exceeded. If
|
||||
`proc` is given, ALSO raises early if the daemon
|
||||
process exits non-zero before the deadline (catches
|
||||
daemon-startup-crash that the blind sleep used to
|
||||
silently mask).
|
||||
process exits before the deadline (catches a daemon startup crash
|
||||
that the blind sleep used to silently mask).
|
||||
|
||||
'''
|
||||
end: float = time.monotonic() + deadline
|
||||
|
|
@ -70,43 +61,25 @@ def _wait_for_daemon_ready(
|
|||
if proc is not None and proc.poll() is not None:
|
||||
raise RuntimeError(
|
||||
f'Daemon proc exited (rc={proc.returncode}) '
|
||||
f'before becoming ready to accept on '
|
||||
f'{reg_addr!r}'
|
||||
f'before reporting ready at {ready_path!r}'
|
||||
)
|
||||
try:
|
||||
if tpt_proto == 'tcp':
|
||||
# `socket.create_connection` does the
|
||||
# `socket() + connect()` dance with a
|
||||
# builtin timeout — perfect primitive
|
||||
# for a one-shot probe.
|
||||
with socket.create_connection(
|
||||
reg_addr,
|
||||
timeout=poll_interval,
|
||||
):
|
||||
if ready_path.is_file():
|
||||
if proc is not None and proc.poll() is not None:
|
||||
raise RuntimeError(
|
||||
f'Daemon proc exited (rc={proc.returncode}) '
|
||||
f'after reporting ready at {ready_path!r}'
|
||||
)
|
||||
return
|
||||
else:
|
||||
# UDS — `reg_addr` is a `(filedir, sockname)`
|
||||
# tuple per `tractor.ipc._uds.UDSAddress.unwrap`.
|
||||
sockpath: str = os.path.join(*reg_addr)
|
||||
sock = socket.socket(socket.AF_UNIX)
|
||||
try:
|
||||
sock.settimeout(poll_interval)
|
||||
sock.connect(sockpath)
|
||||
return
|
||||
finally:
|
||||
sock.close()
|
||||
except (
|
||||
ConnectionRefusedError,
|
||||
FileNotFoundError,
|
||||
OSError,
|
||||
socket.timeout,
|
||||
) as exc:
|
||||
last_exc = exc
|
||||
time.sleep(poll_interval)
|
||||
raise TimeoutError(
|
||||
f'Daemon never accepted on {reg_addr!r} within '
|
||||
f'{deadline}s (last connect-attempt exc: '
|
||||
f'{last_exc!r})'
|
||||
f'Daemon never reported ready at {ready_path!r} within '
|
||||
f'{deadline}s (last sentinel-state exc: {last_exc!r})'
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -136,18 +109,27 @@ def daemon(
|
|||
)
|
||||
loglevel: str = 'info'
|
||||
|
||||
ready_path: Path = (
|
||||
Path(str(testdir.tmpdir))
|
||||
/ 'daemon-ready'
|
||||
)
|
||||
ready_path.unlink(missing_ok=True)
|
||||
code: str = (
|
||||
"import tractor; "
|
||||
"tractor.run_daemon([], "
|
||||
"registry_addrs={reg_addrs}, "
|
||||
"enable_transports={enable_tpts}, "
|
||||
"debug_mode={debug_mode}, "
|
||||
"loglevel={ll})"
|
||||
).format(
|
||||
reg_addrs=str([reg_addr]),
|
||||
enable_tpts=str([tpt_proto]),
|
||||
ll="'{}'".format(loglevel) if loglevel else None,
|
||||
debug_mode=debug_mode,
|
||||
f'from pathlib import Path\n'
|
||||
f'import tractor\n'
|
||||
f'import trio\n'
|
||||
f'\n'
|
||||
f'async def main():\n'
|
||||
f' async with tractor.open_root_actor(\n'
|
||||
f' registry_addrs={[reg_addr]!r},\n'
|
||||
f' enable_transports={[tpt_proto]!r},\n'
|
||||
f' debug_mode={debug_mode!r},\n'
|
||||
f' loglevel={loglevel!r},\n'
|
||||
f' ):\n'
|
||||
f' Path({str(ready_path)!r}).touch()\n'
|
||||
f' await trio.sleep_forever()\n'
|
||||
f'\n'
|
||||
f'trio.run(main)\n'
|
||||
)
|
||||
cmd: list[str] = [
|
||||
sys.executable,
|
||||
|
|
@ -163,9 +145,9 @@ def daemon(
|
|||
**kwargs,
|
||||
)
|
||||
|
||||
# Active-poll the daemon's bind address until it's
|
||||
# ready to accept connections — replaces the legacy
|
||||
# blind `time.sleep(2.2)` which was racy under load
|
||||
# Poll the child's ready sentinel, published after actor startup,
|
||||
# instead of connecting to its transport socket. This replaces
|
||||
# the legacy blind `time.sleep(2.2)` which was racy under load
|
||||
# (see
|
||||
# `ai/conc-anal/test_register_duplicate_name_daemon_connect_race_issue.md`).
|
||||
#
|
||||
|
|
@ -176,20 +158,22 @@ def daemon(
|
|||
15.0 if (_non_linux and ci_env)
|
||||
else 10.0
|
||||
)
|
||||
try:
|
||||
_wait_for_daemon_ready(
|
||||
reg_addr=reg_addr,
|
||||
tpt_proto=tpt_proto,
|
||||
ready_path=ready_path,
|
||||
deadline=deadline,
|
||||
proc=proc,
|
||||
)
|
||||
|
||||
assert not proc.returncode
|
||||
yield proc
|
||||
finally:
|
||||
if proc.poll() is None:
|
||||
sig_prog(proc, _INT_SIGNAL)
|
||||
|
||||
# XXX! yeah.. just be reaaal careful with this bc
|
||||
# sometimes it can lock up on the `_io.BufferedReader`
|
||||
# and hang..
|
||||
# NOTE: these blocking reads can hang when descendants retain
|
||||
# inherited pipe descriptors. Keep teardown signaling above
|
||||
# them and avoid adding subprocesses outside the actor tree.
|
||||
#
|
||||
# NB, drain happens at TEARDOWN (post-yield), so the
|
||||
# test body has its chance to read `proc.stderr`
|
||||
|
|
@ -219,5 +203,4 @@ def daemon(
|
|||
)
|
||||
if rc < 0:
|
||||
raise RuntimeError(msg)
|
||||
|
||||
test_log.error(msg)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,76 @@
|
|||
'''
|
||||
Discovery daemon fixture regressions.
|
||||
|
||||
This module imports private helpers from the sibling
|
||||
`tests.discovery.conftest` plugin to exercise that fixture machinery
|
||||
directly, rather than testing a production `tractor` API.
|
||||
|
||||
'''
|
||||
from unittest.mock import (
|
||||
call,
|
||||
Mock,
|
||||
)
|
||||
|
||||
from .conftest import _wait_for_daemon_ready
|
||||
|
||||
|
||||
def test_daemon_ready_check_does_not_connect(
|
||||
monkeypatch,
|
||||
tmp_path,
|
||||
):
|
||||
'''
|
||||
Observe completed daemon startup without a raw connection.
|
||||
|
||||
The old UDS readiness helper connected and immediately closed. That
|
||||
entered Tractor's actor-handshake handler with no `Aid` payload and
|
||||
destabilized the remote registrar on macOS before discovery tests
|
||||
started. This test creates the child sentinel, forbids all socket
|
||||
construction and connection helpers, then proves readiness returns
|
||||
without touching the transport layer.
|
||||
|
||||
'''
|
||||
ready_path = tmp_path / 'daemon-ready'
|
||||
ready_path.touch()
|
||||
socket_ctor = Mock(side_effect=AssertionError('socket opened'))
|
||||
connect = Mock(side_effect=AssertionError('socket connected'))
|
||||
monkeypatch.setattr('socket.socket', socket_ctor)
|
||||
monkeypatch.setattr('socket.create_connection', connect)
|
||||
|
||||
_wait_for_daemon_ready(
|
||||
ready_path=ready_path,
|
||||
deadline=.1,
|
||||
poll_interval=.01,
|
||||
)
|
||||
|
||||
socket_ctor.assert_not_called()
|
||||
connect.assert_not_called()
|
||||
|
||||
|
||||
def test_daemon_ready_check_backs_off(monkeypatch):
|
||||
'''
|
||||
Back off while waiting for the child startup sentinel.
|
||||
|
||||
The sentinel may appear after several parent polling intervals.
|
||||
A deterministic false/false/true path sequence proves the helper
|
||||
sleeps between unsuccessful observations instead of hot-spinning
|
||||
and starving a booting daemon on constrained CI workers.
|
||||
|
||||
'''
|
||||
ready_path = Mock()
|
||||
ready_path.is_file.side_effect = [False, False, True]
|
||||
sleep = Mock()
|
||||
monotonic = Mock(side_effect=[0, 0, 0, 0])
|
||||
monkeypatch.setattr('time.sleep', sleep)
|
||||
monkeypatch.setattr('time.monotonic', monotonic)
|
||||
|
||||
_wait_for_daemon_ready(
|
||||
ready_path=ready_path,
|
||||
deadline=.2,
|
||||
poll_interval=.01,
|
||||
)
|
||||
|
||||
assert ready_path.is_file.call_count == 3
|
||||
assert sleep.call_args_list == [
|
||||
call(.01),
|
||||
call(.01),
|
||||
]
|
||||
|
|
@ -46,9 +46,9 @@ async def test_reg_then_unreg(
|
|||
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as n:
|
||||
) as an:
|
||||
|
||||
portal = await n.start_actor('actor', enable_modules=[__name__])
|
||||
portal = await an.start_actor('actor', enable_modules=[__name__])
|
||||
uid = portal.channel.aid.uid
|
||||
|
||||
async with tractor.get_registry(reg_addr) as aportal:
|
||||
|
|
@ -62,7 +62,7 @@ async def test_reg_then_unreg(
|
|||
# XXX: can we figure out what the listen addr will be?
|
||||
assert sockaddrs
|
||||
|
||||
await n.cancel() # tear down nursery
|
||||
await an.cancel() # tear down nursery
|
||||
|
||||
await trio.sleep(0.1)
|
||||
assert uid not in aportal.actor._registry
|
||||
|
|
@ -89,9 +89,9 @@ async def test_reg_then_unreg_maddr(
|
|||
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[maddr_str],
|
||||
) as n:
|
||||
) as an:
|
||||
|
||||
portal = await n.start_actor(
|
||||
portal = await an.start_actor(
|
||||
'actor_maddr',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
@ -105,7 +105,7 @@ async def test_reg_then_unreg_maddr(
|
|||
sockaddrs = actor._registry[uid]
|
||||
assert sockaddrs
|
||||
|
||||
await n.cancel()
|
||||
await an.cancel()
|
||||
|
||||
await trio.sleep(0.1)
|
||||
assert uid not in aportal.actor._registry
|
||||
|
|
@ -152,23 +152,37 @@ async def test_trynamic_trio(
|
|||
for the directed subs.
|
||||
|
||||
'''
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
print("Alright... Action!")
|
||||
|
||||
donny = await n.run_in_actor(
|
||||
ria_fn,
|
||||
other_actor='gretchen',
|
||||
reg_addr=reg_addr,
|
||||
name='donny',
|
||||
# donny + gretchen each wait on (then dial!) the *other*, so
|
||||
# both actors must OUTLIVE both hellos: spawn as daemons and
|
||||
# only reap after both tasks complete. NB a pair of eagerly
|
||||
# reaped `to_actor.run()` one-shots races: the first to
|
||||
# finish dies while the other may still be dialing its
|
||||
# registry-resolved (now dead) sockaddr -> conn-refused.
|
||||
portals: dict[str, tractor.Portal] = {
|
||||
name: await an.start_actor(
|
||||
name,
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
gretchen = await n.run_in_actor(
|
||||
for name in ('donny', 'gretchen')
|
||||
}
|
||||
|
||||
async def _direct(this_name: str, other_actor: str):
|
||||
res = await portals[this_name].run(
|
||||
ria_fn,
|
||||
other_actor='donny',
|
||||
other_actor=other_actor,
|
||||
reg_addr=reg_addr,
|
||||
name='gretchen',
|
||||
)
|
||||
print(await gretchen.result())
|
||||
print(await donny.result())
|
||||
print(res)
|
||||
|
||||
async with trio.open_nursery() as tn:
|
||||
tn.start_soon(_direct, 'donny', 'gretchen')
|
||||
tn.start_soon(_direct, 'gretchen', 'donny')
|
||||
|
||||
# both hellos have completed; reap the thespians.
|
||||
await an.cancel()
|
||||
print("CUTTTT CUUTT CUT!!?! Donny!! You're supposed to say...")
|
||||
|
||||
|
||||
|
|
@ -270,13 +284,15 @@ async def spawn_and_check_registry(
|
|||
portals = {}
|
||||
for i in range(3):
|
||||
name = f'a{i}'
|
||||
if with_streaming:
|
||||
# a daemon subactor is alive + registered
|
||||
# without a "main" task; the streaming
|
||||
# branch below uses the module funcs, the
|
||||
# non-streaming case just needs it up (was
|
||||
# `run_in_actor(trio.sleep_forever)`).
|
||||
portals[name] = await an.start_actor(
|
||||
name=name, enable_modules=[__name__])
|
||||
|
||||
else: # no streaming
|
||||
portals[name] = await an.run_in_actor(
|
||||
trio.sleep_forever, name=name)
|
||||
name=name,
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
||||
# wait on last actor to come up
|
||||
async with tractor.wait_for_actor(name):
|
||||
|
|
|
|||
|
|
@ -2,23 +2,211 @@
|
|||
`open_root_actor(tpt_bind_addrs=...)` test suite.
|
||||
|
||||
Verify all three runtime code paths for explicit IPC-server
|
||||
bind-address selection in `_root.py`:
|
||||
bind-address selection in `_root.py` and registry probing in
|
||||
`discovery._api`:
|
||||
|
||||
1. Non-registrar, no explicit bind -> random addrs from registry proto
|
||||
2. Registrar, no explicit bind -> binds to registry_addrs
|
||||
3. Explicit bind given -> wraps via `wrap_address()` and uses them
|
||||
|
||||
'''
|
||||
from contextlib import asynccontextmanager as acm
|
||||
from unittest.mock import (
|
||||
AsyncMock,
|
||||
call,
|
||||
Mock,
|
||||
)
|
||||
|
||||
import pytest
|
||||
import trio
|
||||
import tractor
|
||||
from tractor.discovery import _api
|
||||
from tractor.discovery._addr import (
|
||||
wrap_address,
|
||||
)
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
from tractor.ipc import _connect_chan
|
||||
from tractor._testing.addr import get_rando_addr
|
||||
|
||||
|
||||
def test_registry_probe_retries_transient_handshake(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
'''
|
||||
Retry a connected registrar after transient handshake timeout.
|
||||
|
||||
Loaded macOS runners can accept the transport while delaying the
|
||||
actor handshake beyond one second. Treating that first timeout as
|
||||
final makes a healthy remote daemon look occupied and cascades into
|
||||
discovery failures. This deterministic fake fails once, succeeds
|
||||
on the second complete handshake, and proves one bounded backoff.
|
||||
|
||||
'''
|
||||
async def stall_handshake(**kwargs):
|
||||
await trio.sleep_forever()
|
||||
|
||||
first_handshake = AsyncMock(side_effect=stall_handshake)
|
||||
second_handshake = AsyncMock(
|
||||
return_value=tractor.msg.Aid(
|
||||
name='registrar',
|
||||
uuid='registrar-uuid',
|
||||
pid=1234,
|
||||
is_registrar=True,
|
||||
),
|
||||
)
|
||||
chans = [
|
||||
Mock(_do_handshake=first_handshake),
|
||||
Mock(_do_handshake=second_handshake),
|
||||
]
|
||||
closed: list[object] = []
|
||||
|
||||
@acm
|
||||
async def connect_chan(addr, close_timeout):
|
||||
assert close_timeout == .2
|
||||
chan = chans[len(closed)]
|
||||
try:
|
||||
yield chan
|
||||
finally:
|
||||
closed.append(chan)
|
||||
|
||||
sleep = AsyncMock()
|
||||
monkeypatch.setattr(_api, '_connect_chan', connect_chan)
|
||||
monkeypatch.setattr(_api.trio, 'sleep', sleep)
|
||||
|
||||
async def main():
|
||||
status = await _api._probe_registry(
|
||||
addr=wrap_address(('127.0.0.1', 1616)),
|
||||
timeout=.3,
|
||||
attempt_timeout=.1,
|
||||
max_attempts=3,
|
||||
retry_delay=.01,
|
||||
)
|
||||
assert status == 'registrar'
|
||||
|
||||
trio.run(main)
|
||||
|
||||
first_handshake.assert_awaited_once()
|
||||
second_handshake.assert_awaited_once()
|
||||
assert first_handshake.await_args.kwargs['timeout'] == .1
|
||||
assert second_handshake.await_args.kwargs['timeout'] == .1
|
||||
assert closed == chans
|
||||
sleep.assert_has_awaits([call(.01)])
|
||||
|
||||
|
||||
def test_probe_channel_close_is_bounded(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
'''
|
||||
Bound shielded channel cleanup after a registry probe.
|
||||
|
||||
`_connect_chan()` shields `.aclose()` so cancellation cannot leak
|
||||
ordinary channels. A stalled close previously let registry probing
|
||||
exceed every connect and handshake deadline. This fake close never
|
||||
completes; the explicit cleanup allowance must still return control
|
||||
to the caller without cancelling its surrounding task.
|
||||
|
||||
'''
|
||||
chan = Mock()
|
||||
chan.aclose = AsyncMock(side_effect=trio.sleep_forever)
|
||||
monkeypatch.setattr(
|
||||
tractor.Channel,
|
||||
'from_addr',
|
||||
AsyncMock(return_value=chan),
|
||||
)
|
||||
|
||||
async def main():
|
||||
with trio.fail_after(.5):
|
||||
async with _connect_chan(
|
||||
('127.0.0.1', 1616),
|
||||
close_timeout=.01,
|
||||
):
|
||||
pass
|
||||
|
||||
trio.run(main)
|
||||
chan.aclose.assert_awaited_once()
|
||||
|
||||
|
||||
def test_transport_only_listener_is_not_registrar():
|
||||
'''
|
||||
Require a Tractor handshake before accepting a registry address.
|
||||
|
||||
The old election probe marked an address live after transport
|
||||
connect alone. A non-Tractor listener, or a registrar still
|
||||
failing its initial handshake, was therefore selected as the
|
||||
remote registry. This test accepts the probe and closes it without
|
||||
replying, then proves `open_root_actor()` rejects that occupied
|
||||
endpoint instead of selecting it or binding over it.
|
||||
|
||||
'''
|
||||
async def transport_only_handler(
|
||||
stream: trio.SocketStream,
|
||||
) -> None:
|
||||
await stream.aclose()
|
||||
|
||||
async def main():
|
||||
listeners = await trio.open_tcp_listeners(0)
|
||||
listener = listeners[0]
|
||||
sockname = listener.socket.getsockname()
|
||||
reg_addr: tuple[str, int] = (
|
||||
sockname[0],
|
||||
sockname[1],
|
||||
)
|
||||
|
||||
async with trio.open_nursery() as tn:
|
||||
tn.start_soon(
|
||||
trio.serve_listeners,
|
||||
transport_only_handler,
|
||||
listeners,
|
||||
)
|
||||
with pytest.raises(
|
||||
RuntimeError,
|
||||
match='occupied but did not answer',
|
||||
):
|
||||
async with tractor.open_root_actor(
|
||||
registry_addrs=[reg_addr],
|
||||
enable_transports=['tcp'],
|
||||
):
|
||||
pytest.fail('foreign listener selected as registrar')
|
||||
|
||||
tn.cancel_scope.cancel()
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
||||
def test_registry_probe_preserves_no_peers_state(
|
||||
reg_addr: tuple,
|
||||
tpt_proto: str,
|
||||
):
|
||||
'''
|
||||
Keep an idle registrar peer-free after an election probe.
|
||||
|
||||
Probe handshakes exchange registrar capability but must not enter
|
||||
`IPCServer._peers`. Resetting `_no_more_peers` before identifying a
|
||||
probe left an idle registrar reporting phantom peers and delayed
|
||||
shutdown. This test probes the live local registrar and proves its
|
||||
peer map and no-peers event remain unchanged afterward.
|
||||
|
||||
'''
|
||||
async def main():
|
||||
async with tractor.open_root_actor(
|
||||
registry_addrs=[reg_addr],
|
||||
enable_transports=[tpt_proto],
|
||||
):
|
||||
actor = tractor.current_actor()
|
||||
server = actor.ipc_server
|
||||
|
||||
probe_status = await _api._probe_registry(
|
||||
addr=wrap_address(reg_addr),
|
||||
)
|
||||
assert probe_status == 'registrar'
|
||||
|
||||
await trio.sleep(0)
|
||||
assert not server._peers
|
||||
assert server._no_more_peers.is_set()
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
|
|
|||
|
|
@ -3,14 +3,21 @@ Unit-ish tests for specific IPC transport protocol backends.
|
|||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import stat
|
||||
import sys
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
import trio
|
||||
import tractor
|
||||
from tractor import Actor
|
||||
from tractor.runtime import _state
|
||||
from tractor.discovery import _addr
|
||||
from tractor.runtime import _state
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
|
|
@ -31,6 +38,381 @@ def bindspace_dir_str() -> str:
|
|||
bs_dir.rmdir()
|
||||
|
||||
|
||||
def test_macos_rt_dir_fits_uds_path_limit(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Keep the default Darwin UDS bindpath below its 104-byte limit.
|
||||
|
||||
`platformdirs` normally places the runtime directory below the
|
||||
long `~/Library/Caches/TemporaryItems` path. Pytest also assigns
|
||||
a deeply nested temporary home, so appending a registry socket
|
||||
name made every macOS UDS listener fail with `AF_UNIX path too
|
||||
long`. This test simulates Darwin and an intentionally long
|
||||
platformdirs result, then proves `get_rt_dir()` uses the short
|
||||
system temporary directory and leaves room for the socket name.
|
||||
|
||||
'''
|
||||
long_rt_dir: Path = tmp_path / ('long' * 30)
|
||||
monkeypatch.setattr(sys, 'platform', 'darwin')
|
||||
monkeypatch.setattr(
|
||||
'platformdirs.user_runtime_dir',
|
||||
lambda appname: str(long_rt_dir / appname),
|
||||
)
|
||||
monkeypatch.setattr(_state, '_DARWIN_TMPDIR', tmp_path)
|
||||
rt_dir: Path = _state.get_rt_dir()
|
||||
sockpath: Path = (
|
||||
Path('/tmp')
|
||||
/ f'tractor-{os.getuid()}'
|
||||
/ 'registry@1616.sock'
|
||||
)
|
||||
|
||||
assert rt_dir == tmp_path / f'tractor-{os.getuid()}'
|
||||
assert len(os.fsencode(sockpath)) < 104
|
||||
assert stat.S_IMODE(rt_dir.stat().st_mode) == 0o700
|
||||
|
||||
|
||||
def test_macos_rt_dir_rejects_symlink(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Reject a pre-created symlink at the Darwin runtime path.
|
||||
|
||||
Darwin uses the predictable `/tmp/tractor-<uid>` path to stay
|
||||
below its `AF_UNIX` limit. A hostile local user could otherwise
|
||||
point that path at a victim-owned directory and make
|
||||
`get_rt_dir()` chmod or place sockets in the symlink target. The
|
||||
test replaces `/tmp` with a controlled directory, installs the
|
||||
malicious link, and proves non-following validation rejects it.
|
||||
|
||||
'''
|
||||
runtime_link: Path = tmp_path / f'tractor-{os.getuid()}'
|
||||
target_dir: Path = tmp_path / 'target'
|
||||
target_dir.mkdir(mode=0o755)
|
||||
runtime_link.symlink_to(target_dir, target_is_directory=True)
|
||||
monkeypatch.setattr(sys, 'platform', 'darwin')
|
||||
monkeypatch.setattr(_state, '_DARWIN_TMPDIR', tmp_path)
|
||||
|
||||
with pytest.raises(PermissionError, match='Unsafe Darwin'):
|
||||
_state.get_rt_dir()
|
||||
|
||||
assert stat.S_IMODE(target_dir.stat().st_mode) == 0o755
|
||||
|
||||
|
||||
def test_reaper_uses_default_uds_bindspace(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Sweep the same platform-specific bindspace used by UDS actors.
|
||||
|
||||
The reaper previously consulted only `XDG_RUNTIME_DIR`, missing
|
||||
Darwin sockets after the runtime moved to `/tmp/tractor-<uid>`.
|
||||
This test replaces `UDSAddress.def_bindspace` and proves the test
|
||||
harness resolves that shared transport default directly.
|
||||
|
||||
'''
|
||||
from tractor._testing import _reap
|
||||
from tractor.ipc._uds import UDSAddress
|
||||
|
||||
monkeypatch.setattr(
|
||||
UDSAddress,
|
||||
'def_bindspace',
|
||||
tmp_path,
|
||||
)
|
||||
|
||||
assert _reap.get_uds_dir() == str(tmp_path)
|
||||
|
||||
|
||||
def test_automatic_reaper_preserves_registry_sentinel(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
'''
|
||||
Reserve unconditional registry cleanup for the explicit CLI.
|
||||
|
||||
The `registry@1616.sock` suffix does not encode its binder PID, so
|
||||
automatic pytest cleanup cannot distinguish a leak from another
|
||||
live registrar. This test creates registry and actor sockets,
|
||||
proves the default sweep selects only the dead actor, then proves
|
||||
explicit sentinel inclusion retains the CLI's documented behavior.
|
||||
|
||||
'''
|
||||
from tractor._testing import _reap
|
||||
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix='tractor-reap-',
|
||||
dir='/tmp',
|
||||
) as tmpdir:
|
||||
bindspace: Path = Path(tmpdir)
|
||||
registry_path: Path = bindspace / 'registry@1616.sock'
|
||||
actor_path: Path = bindspace / 'worker@1234.sock'
|
||||
socks: list[socket.socket] = []
|
||||
for path in (registry_path, actor_path):
|
||||
sock = socket.socket(socket.AF_UNIX)
|
||||
sock.bind(str(path))
|
||||
socks.append(sock)
|
||||
|
||||
monkeypatch.setattr(_reap, '_is_alive', lambda pid: False)
|
||||
try:
|
||||
assert _reap.find_orphaned_uds(
|
||||
uds_dir=str(bindspace),
|
||||
) == [str(actor_path)]
|
||||
assert set(
|
||||
_reap.find_orphaned_uds(
|
||||
uds_dir=str(bindspace),
|
||||
include_registry_sentinel=True,
|
||||
)
|
||||
) == {
|
||||
str(registry_path),
|
||||
str(actor_path),
|
||||
}
|
||||
finally:
|
||||
for sock in socks:
|
||||
sock.close()
|
||||
|
||||
|
||||
def test_rt_dir_rejects_non_directory(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Preserve the non-Darwin runtime-directory type contract.
|
||||
|
||||
Replacing `Path.is_dir()` with unguarded `lstat()` briefly made
|
||||
existing files look like valid runtime directories on Linux.
|
||||
This test points `platformdirs` at a regular file and proves
|
||||
`get_rt_dir()` rejects it during initialization.
|
||||
|
||||
'''
|
||||
rt_file: Path = tmp_path / 'runtime-file'
|
||||
rt_file.touch()
|
||||
monkeypatch.setattr(sys, 'platform', 'linux')
|
||||
monkeypatch.setattr(
|
||||
'platformdirs.user_runtime_dir',
|
||||
lambda appname: str(rt_file),
|
||||
)
|
||||
|
||||
with pytest.raises(
|
||||
PermissionError,
|
||||
match='Unsafe POSIX',
|
||||
):
|
||||
_state.get_rt_dir()
|
||||
|
||||
new_rt_dir: Path = tmp_path / 'new-runtime-dir'
|
||||
monkeypatch.setattr(
|
||||
'platformdirs.user_runtime_dir',
|
||||
lambda appname: str(new_rt_dir),
|
||||
)
|
||||
assert _state.get_rt_dir() == new_rt_dir
|
||||
assert stat.S_IMODE(new_rt_dir.stat().st_mode) == 0o700
|
||||
|
||||
|
||||
def test_linux_rt_dir_secures_existing_path(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Enforce owner-only access on an existing Linux runtime directory.
|
||||
|
||||
Linux previously accepted any existing directory returned by
|
||||
`platformdirs`, without checking ownership or correcting a
|
||||
traversable mode. This test creates an owner-controlled `0o755`
|
||||
directory and proves `get_rt_dir()` normalizes the managed
|
||||
bindspace to `0o700` before returning it.
|
||||
|
||||
'''
|
||||
rt_dir: Path = tmp_path / 'tractor'
|
||||
rt_dir.mkdir(mode=0o755)
|
||||
monkeypatch.setattr(sys, 'platform', 'linux')
|
||||
monkeypatch.setattr(
|
||||
'platformdirs.user_runtime_dir',
|
||||
lambda appname: str(rt_dir),
|
||||
)
|
||||
|
||||
assert _state.get_rt_dir() == rt_dir
|
||||
assert stat.S_IMODE(rt_dir.stat().st_mode) == 0o700
|
||||
|
||||
|
||||
def test_linux_rt_dir_rejects_foreign_owner(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Reject an existing Linux runtime directory owned by another UID.
|
||||
|
||||
A pre-created bindspace must never be made private with `chmod`
|
||||
until ownership is verified. This test makes the current process
|
||||
appear to have a different UID and proves `get_rt_dir()` rejects
|
||||
the directory without changing its original mode.
|
||||
|
||||
'''
|
||||
rt_dir: Path = tmp_path / 'tractor'
|
||||
rt_dir.mkdir(mode=0o755)
|
||||
original_mode: int = stat.S_IMODE(rt_dir.stat().st_mode)
|
||||
monkeypatch.setattr(sys, 'platform', 'linux')
|
||||
monkeypatch.setattr(
|
||||
'platformdirs.user_runtime_dir',
|
||||
lambda appname: str(rt_dir),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
os,
|
||||
'getuid',
|
||||
lambda: rt_dir.stat().st_uid + 1,
|
||||
)
|
||||
|
||||
with pytest.raises(
|
||||
PermissionError,
|
||||
match='Unsafe POSIX',
|
||||
):
|
||||
_state.get_rt_dir()
|
||||
|
||||
assert stat.S_IMODE(rt_dir.stat().st_mode) == original_mode
|
||||
|
||||
|
||||
def test_macos_rt_dir_rejects_intermediate_symlink(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Reject symlinks in nested Darwin runtime subdirectories.
|
||||
|
||||
The earlier final-component check allowed `link/child` to follow
|
||||
an intermediate symlink and create `child` outside the secured
|
||||
runtime root. This test installs that link and proves traversal
|
||||
stops before anything is created in its target.
|
||||
|
||||
'''
|
||||
rt_root: Path = tmp_path / f'tractor-{os.getuid()}'
|
||||
target_dir: Path = tmp_path / 'target'
|
||||
rt_root.mkdir(mode=0o700)
|
||||
target_dir.mkdir()
|
||||
(rt_root / 'link').symlink_to(
|
||||
target_dir,
|
||||
target_is_directory=True,
|
||||
)
|
||||
monkeypatch.setattr(sys, 'platform', 'darwin')
|
||||
monkeypatch.setattr(_state, '_DARWIN_TMPDIR', tmp_path)
|
||||
|
||||
with pytest.raises(PermissionError, match='Unsafe Darwin'):
|
||||
_state.get_rt_dir(subdir='link/child')
|
||||
|
||||
assert not (target_dir / 'child').exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
('platform_name', 'path_limit'),
|
||||
[
|
||||
('darwin', 104),
|
||||
('linux', 108),
|
||||
],
|
||||
)
|
||||
def test_uds_sockname_compaction(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
platform_name: str,
|
||||
path_limit: int,
|
||||
):
|
||||
'''
|
||||
Keep generated actor sockets safe and below Darwin's byte limit.
|
||||
|
||||
Actor names are unrestricted identity strings. A long, multibyte,
|
||||
or path-like name previously produced overlong or escaping socket
|
||||
paths. These cases prove `UDSAddress.get_sockname()` preserves a
|
||||
short legacy name, deterministically compacts unsafe names, keeps
|
||||
the reaper's `@pid.sock` suffix, and stays within Darwin's byte
|
||||
limit.
|
||||
|
||||
'''
|
||||
from tractor.ipc._uds import UDSAddress
|
||||
|
||||
bindspace: Path = Path('/tmp/tractor-501')
|
||||
pid: int = 12345
|
||||
from tractor.ipc import _uds
|
||||
|
||||
monkeypatch.setattr(sys, 'platform', platform_name)
|
||||
monkeypatch.setattr(_uds, '_SUN_PATH_LIMIT', path_limit)
|
||||
|
||||
short: Path = UDSAddress.get_sockname(
|
||||
name='worker',
|
||||
pid=pid,
|
||||
bindspace=bindspace,
|
||||
)
|
||||
long_name: str = 'actor-' + ('\u00e9' * 100)
|
||||
compact: Path = UDSAddress.get_sockname(
|
||||
name=long_name,
|
||||
pid=pid,
|
||||
bindspace=bindspace,
|
||||
)
|
||||
unsafe: Path = UDSAddress.get_sockname(
|
||||
name='../worker',
|
||||
pid=pid,
|
||||
bindspace=bindspace,
|
||||
)
|
||||
|
||||
assert short == Path(f'worker@{pid}.sock')
|
||||
assert compact == UDSAddress.get_sockname(
|
||||
name=long_name,
|
||||
pid=pid,
|
||||
bindspace=bindspace,
|
||||
)
|
||||
assert compact.name.endswith(f'@{pid}.sock')
|
||||
assert unsafe.parent == Path('.')
|
||||
assert '..' not in unsafe.name
|
||||
assert len(os.fsencode(bindspace / compact)) < path_limit
|
||||
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
UDSAddress.get_sockname(
|
||||
name=long_name,
|
||||
pid=pid,
|
||||
bindspace=Path('/tmp') / ('x' * 90),
|
||||
)
|
||||
|
||||
errmsg: str = str(exc_info.value)
|
||||
assert 'leaves no room' in errmsg
|
||||
assert 'name was unsafe: False' in errmsg
|
||||
assert 'name was over budget: True' in errmsg
|
||||
assert f'AF_UNIX path limit: {path_limit}' in errmsg
|
||||
|
||||
|
||||
def test_uds_reaper_ignores_unreconstructable_path(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
'''
|
||||
Keep post-kill UDS cleanup best-effort on path overflow.
|
||||
|
||||
`unlink_uds_bind_addrs()` reconstructs a self-assigned socket from
|
||||
the dead actor's name and PID. An over-budget bindspace makes that
|
||||
naming helper raise before `os.unlink()`; propagating the error
|
||||
would replace the original supervision outcome after the child was
|
||||
already killed. This test forces overflow and proves cleanup skips
|
||||
reconstruction without attempting an unlink or raising.
|
||||
|
||||
'''
|
||||
from tractor.ipc import _uds
|
||||
from tractor.spawn import _reap
|
||||
|
||||
long_bindspace: Path = Path('/tmp') / ('x' * 120)
|
||||
proc = SimpleNamespace(pid=12345)
|
||||
subactor = SimpleNamespace(
|
||||
aid=SimpleNamespace(name='worker'),
|
||||
)
|
||||
unlink = Mock()
|
||||
monkeypatch.setattr(
|
||||
_uds.UDSAddress,
|
||||
'def_bindspace',
|
||||
long_bindspace,
|
||||
)
|
||||
monkeypatch.setattr(_reap.os, 'unlink', unlink)
|
||||
|
||||
_reap.unlink_uds_bind_addrs(
|
||||
proc=proc,
|
||||
subactor=subactor,
|
||||
)
|
||||
|
||||
unlink.assert_not_called()
|
||||
|
||||
|
||||
def test_uds_bindspace_created_implicitly(
|
||||
debug_mode: bool,
|
||||
bindspace_dir_str: str,
|
||||
|
|
|
|||
|
|
@ -3,7 +3,13 @@ High-level `.ipc._server` unit tests.
|
|||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import errno
|
||||
from unittest.mock import (
|
||||
AsyncMock,
|
||||
Mock,
|
||||
)
|
||||
|
||||
import msgspec
|
||||
import pytest
|
||||
import trio
|
||||
from tractor import (
|
||||
|
|
@ -14,6 +20,11 @@ from tractor import (
|
|||
from tractor._testing.addr import (
|
||||
get_rando_addr,
|
||||
)
|
||||
from tractor._exceptions import TransportClosed
|
||||
from tractor.ipc._chan import Channel
|
||||
from tractor.ipc import _server
|
||||
from tractor.ipc._transport import MsgpackTransport
|
||||
from tractor.msg.types import Aid
|
||||
# TODO, use/check-roundtripping with some of these wrapper types?
|
||||
#
|
||||
# from .._addr import Address
|
||||
|
|
@ -23,6 +34,165 @@ from tractor._testing.addr import (
|
|||
# from ._tcp import TCPAddress
|
||||
|
||||
|
||||
def test_send_normalizes_only_grouped_peer_resets():
|
||||
'''
|
||||
Normalize only all-peer-close grouped transport failures.
|
||||
|
||||
A UDS peer may disconnect before completing the actor handshake.
|
||||
Darwin can report the server's first handshake write as
|
||||
`ECONNRESET`, wrapped by `trio.BrokenResourceError` and potentially
|
||||
nested in an `ExceptionGroup`. This fake stream first groups reset
|
||||
and broken-pipe branches, proving `.send()` normalizes a complete
|
||||
peer-close tree to `TransportClosed`. It then groups a reset with
|
||||
an unrelated `ValueError`, proving the mixed failure remains a
|
||||
`trio.BrokenResourceError` instead of hiding the application error.
|
||||
|
||||
'''
|
||||
def broken_resource(err_no: int) -> trio.BrokenResourceError:
|
||||
try:
|
||||
raise OSError(
|
||||
err_no,
|
||||
'Peer closed',
|
||||
)
|
||||
except OSError as peer_err:
|
||||
try:
|
||||
raise trio.BrokenResourceError from peer_err
|
||||
except trio.BrokenResourceError as broken_err:
|
||||
return broken_err
|
||||
|
||||
class GroupedFailureStream:
|
||||
def __init__(self, exceptions: list[Exception]) -> None:
|
||||
self.exceptions = exceptions
|
||||
|
||||
async def send_all(self, data: bytes) -> None:
|
||||
grouped_err = ExceptionGroup(
|
||||
'concurrent send failures',
|
||||
self.exceptions,
|
||||
)
|
||||
raise trio.BrokenResourceError from grouped_err
|
||||
|
||||
async def main():
|
||||
transport = object.__new__(MsgpackTransport)
|
||||
transport.stream = GroupedFailureStream([
|
||||
broken_resource(errno.ECONNRESET),
|
||||
broken_resource(errno.EPIPE),
|
||||
])
|
||||
transport._send_lock = trio.StrictFIFOLock()
|
||||
transport._laddr = 'local'
|
||||
transport._raddr = 'remote'
|
||||
transport._task = trio.lowlevel.current_task()
|
||||
|
||||
with pytest.raises(TransportClosed) as exc_info:
|
||||
await transport.send(
|
||||
{'probe': True},
|
||||
strict_types=False,
|
||||
)
|
||||
|
||||
grouped_err = exc_info.value.src_exc.__cause__
|
||||
assert isinstance(grouped_err, ExceptionGroup)
|
||||
assert len(grouped_err.exceptions) == 2
|
||||
|
||||
transport.stream = GroupedFailureStream([
|
||||
ValueError('unrelated failure'),
|
||||
broken_resource(errno.ECONNRESET),
|
||||
])
|
||||
with pytest.raises(trio.BrokenResourceError) as exc_info:
|
||||
await transport.send(
|
||||
{'probe': True},
|
||||
strict_types=False,
|
||||
)
|
||||
|
||||
grouped_err = exc_info.value.__cause__
|
||||
assert isinstance(grouped_err, ExceptionGroup)
|
||||
assert isinstance(grouped_err.exceptions[0], ValueError)
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
||||
def test_handshake_normalizes_decode_error():
|
||||
'''
|
||||
Keep malformed pre-handshake frames out of the service nursery.
|
||||
|
||||
A non-msgpack peer can trigger `msgspec.DecodeError` before a
|
||||
remote `Aid` exists. Letting that decoder error escape the inbound
|
||||
handler cancels the actor's shared IPC nursery. This fake channel
|
||||
proves `_do_handshake()` presents only `TransportClosed` upward.
|
||||
|
||||
'''
|
||||
chan = object.__new__(Channel)
|
||||
chan.send = AsyncMock()
|
||||
chan.recv = AsyncMock(
|
||||
side_effect=msgspec.DecodeError('malformed handshake'),
|
||||
)
|
||||
|
||||
async def main():
|
||||
with pytest.raises(TransportClosed) as exc_info:
|
||||
await chan._do_handshake(
|
||||
aid=Aid(
|
||||
name='local',
|
||||
uuid='local-uuid',
|
||||
pid=1234,
|
||||
),
|
||||
timeout=.1,
|
||||
)
|
||||
|
||||
assert isinstance(
|
||||
exc_info.value.src_exc,
|
||||
msgspec.DecodeError,
|
||||
)
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
||||
def test_server_uses_independent_handshake_timeout(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
'''
|
||||
Give ordinary actor handshakes a distinct, generous deadline.
|
||||
|
||||
Registry probes use short retries, but ordinary portal and child
|
||||
connections do not retry. Applying the probe's one-second timeout
|
||||
in the server can terminate a valid delayed child and leave its
|
||||
parent blocked in `IPCServer.wait_for_peer()`. This handler fake
|
||||
proves the server uses its separate pre-registration budget.
|
||||
|
||||
'''
|
||||
handshake = AsyncMock(
|
||||
side_effect=TransportClosed(message='stop after assertion'),
|
||||
)
|
||||
chan = Mock(_do_handshake=handshake)
|
||||
actor = Mock(
|
||||
aid=Aid(
|
||||
name='local',
|
||||
uuid='local-uuid',
|
||||
pid=1234,
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
Channel,
|
||||
'from_stream',
|
||||
Mock(return_value=chan),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
_server._state,
|
||||
'current_actor',
|
||||
Mock(return_value=actor),
|
||||
)
|
||||
|
||||
async def main():
|
||||
await _server.handle_stream_from_peer(
|
||||
stream=Mock(),
|
||||
server=Mock(),
|
||||
)
|
||||
|
||||
trio.run(main)
|
||||
handshake.assert_awaited_once_with(
|
||||
aid=actor.aid,
|
||||
timeout=_server._PRE_REG_HANDSHAKE_TIMEOUT,
|
||||
)
|
||||
assert _server._PRE_REG_HANDSHAKE_TIMEOUT == 10
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'_tpt_proto',
|
||||
['uds', 'tcp']
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ Advanced streaming patterns using bidirectional streams and contexts.
|
|||
|
||||
'''
|
||||
from collections import Counter
|
||||
from functools import partial
|
||||
import itertools
|
||||
import platform
|
||||
from typing import Type
|
||||
|
|
@ -173,8 +174,8 @@ def test_dynamic_pub_sub(
|
|||
# test. Picked backend-aware: under `trio` backend spawn is
|
||||
# cheap (~1s for `cpus` actors) but fork-based backends pay
|
||||
# a per-spawn cost (forkserver round-trip + IPC peer-handshake)
|
||||
# that can stack up over `cpus - 1` sequential `n.run_in_actor()`
|
||||
# calls — especially on UDS under cross-pytest contention
|
||||
# that can stack up over the `cpus - 1` one-shot
|
||||
# (`to_actor.run()`) spawns — especially on UDS under cross-pytest contention
|
||||
# (#451 / #452). 4s was flaking right at the edge under fork
|
||||
# backends — bumped to 8s with diag-snapshot-on-timeout via
|
||||
# `fail_after_w_trace` so a borderline run still fails loud
|
||||
|
|
@ -214,34 +215,56 @@ def test_dynamic_pub_sub(
|
|||
f'enter `fail_after_w_trace({fail_after_s})` scope'
|
||||
)
|
||||
try:
|
||||
async with tractor.open_nursery(
|
||||
async with (
|
||||
tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
debug_mode=debug_mode,
|
||||
) as n:
|
||||
) as an,
|
||||
# bg-schedules the forever-streaming
|
||||
# one-shots below; the user-cancel raise
|
||||
# cancels them all, each reaping its
|
||||
# subactor via `to_actor.run()`'s
|
||||
# (shielded) `Portal.cancel_actor()`.
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
test_log.cancel(
|
||||
'test_dynamic_pub_sub: '
|
||||
'actor nursery opened'
|
||||
)
|
||||
|
||||
# name of this actor will be same as target func
|
||||
await n.run_in_actor(publisher)
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
publisher,
|
||||
an=an,
|
||||
)
|
||||
)
|
||||
|
||||
for i, sub in zip(
|
||||
range(cpus - 2),
|
||||
itertools.cycle(_registry.keys())
|
||||
):
|
||||
await n.run_in_actor(
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
consumer,
|
||||
an=an,
|
||||
name=f'consumer_{sub}',
|
||||
subs=[sub],
|
||||
)
|
||||
)
|
||||
|
||||
# make one dynamic subscriber
|
||||
await n.run_in_actor(
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
consumer,
|
||||
an=an,
|
||||
name='consumer_dynamic',
|
||||
subs=list(_registry.keys()),
|
||||
)
|
||||
)
|
||||
|
||||
# block until "cancelled by user"
|
||||
await trio.sleep(3)
|
||||
|
|
@ -347,10 +370,10 @@ def test_reqresp_ontopof_streaming():
|
|||
timeout = 4
|
||||
|
||||
with trio.move_on_after(timeout):
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
|
||||
# name of this actor will be same as target func
|
||||
portal = await n.start_actor(
|
||||
portal = await an.start_actor(
|
||||
'dual_tasks',
|
||||
enable_modules=[__name__]
|
||||
)
|
||||
|
|
@ -413,9 +436,9 @@ def test_sigint_both_stream_types():
|
|||
|
||||
async def main():
|
||||
with trio.fail_after(timeout):
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
# name of this actor will be same as target func
|
||||
portal = await n.start_actor(
|
||||
portal = await an.start_actor(
|
||||
'2_way',
|
||||
enable_modules=[__name__]
|
||||
)
|
||||
|
|
@ -528,8 +551,8 @@ def test_local_task_fanout_from_stream(
|
|||
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=debug_mode,
|
||||
) as tn:
|
||||
p: tractor.Portal = await tn.start_actor(
|
||||
) as an:
|
||||
p: tractor.Portal = await an.start_actor(
|
||||
'inf_streamer',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
Cancellation and error propagation
|
||||
|
||||
"""
|
||||
from functools import partial
|
||||
import os
|
||||
import signal
|
||||
import platform
|
||||
|
|
@ -16,6 +17,10 @@ from tractor._testing import (
|
|||
tractor_test,
|
||||
)
|
||||
from tractor._testing.trace import FailAfterWTraceFactory
|
||||
from tractor.trionics import (
|
||||
collapse_eg,
|
||||
gather_contexts,
|
||||
)
|
||||
from .conftest import no_windows
|
||||
|
||||
|
||||
|
|
@ -68,6 +73,23 @@ async def assert_err(delay=0):
|
|||
assert 0
|
||||
|
||||
|
||||
@tractor.context
|
||||
async def assert_err_ctx(
|
||||
ctx: tractor.Context,
|
||||
delay: float = 0,
|
||||
) -> None:
|
||||
'''
|
||||
`@context` shim around `assert_err()` so the multi-actor error
|
||||
tests can fan-out one-shot erroring subactors via
|
||||
`Portal.open_context()` + `gather_contexts()` instead of the
|
||||
removed `ActorNursery.run_in_actor()` (#477).
|
||||
|
||||
'''
|
||||
await ctx.started()
|
||||
await trio.sleep(delay)
|
||||
assert 0
|
||||
|
||||
|
||||
async def sleep_forever():
|
||||
await trio.sleep_forever()
|
||||
|
||||
|
|
@ -104,24 +126,19 @@ def test_remote_error(
|
|||
async def main():
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as nursery:
|
||||
) as an:
|
||||
|
||||
# on a remote type error caused by bad input args
|
||||
# this should raise directly which means we **don't** get
|
||||
# an exception group outside the nursery since the error
|
||||
# here and the far end task error are one in the same?
|
||||
portal = await nursery.run_in_actor(
|
||||
# `to_actor.run()` blocks on the one-shot's result and
|
||||
# raises the remote error directly here in the caller's
|
||||
# task (a bad-arg `TypeError` likewise relays as a
|
||||
# `RemoteActorError`).
|
||||
try:
|
||||
await tractor.to_actor.run(
|
||||
assert_err,
|
||||
an=an,
|
||||
name='errorer',
|
||||
**args
|
||||
)
|
||||
|
||||
# get result(s) from main task
|
||||
try:
|
||||
# this means the root actor will also raise a local
|
||||
# parent task error and thus an eg will propagate out
|
||||
# of this actor nursery.
|
||||
await portal.result()
|
||||
except tractor.RemoteActorError as err:
|
||||
assert err.boxed_type == errtype
|
||||
print("Look Maa that actor failed hard, hehh")
|
||||
|
|
@ -162,113 +179,48 @@ def test_multierror(
|
|||
set_fork_aware_capture, #: Callable,
|
||||
):
|
||||
'''
|
||||
Verify we raise a ``BaseExceptionGroup`` out of a nursery where
|
||||
more then one actor errors.
|
||||
Verify concurrent one-shot subactors erroring propagate a remote
|
||||
error out of the `gather_contexts()` fan-out — grouped as a
|
||||
`BaseExceptionGroup`, or (under cancel-on-first, where the 2nd
|
||||
errorer is cancelled before relaying its own exc) collapsed to a
|
||||
single `RemoteActorError`.
|
||||
|
||||
NB the legacy `run_in_actor()` reaped *all* children at nursery
|
||||
teardown so this always yielded a BEG-of-N; the `to_actor`
|
||||
fan-out is cancel-on-first, so accept either shape.
|
||||
|
||||
'''
|
||||
async def main():
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as nursery:
|
||||
) as an:
|
||||
|
||||
await nursery.run_in_actor(assert_err, name='errorer1')
|
||||
portal2 = await nursery.run_in_actor(assert_err, name='errorer2')
|
||||
|
||||
# get result(s) from main task
|
||||
try:
|
||||
await portal2.result()
|
||||
except tractor.RemoteActorError as err:
|
||||
assert err.boxed_type is AssertionError
|
||||
print("Look Maa that first actor failed hard, hehh")
|
||||
raise
|
||||
|
||||
# here we should get a ``BaseExceptionGroup`` containing exceptions
|
||||
# from both subactors
|
||||
|
||||
with pytest.raises(BaseExceptionGroup):
|
||||
trio.run(main)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'delay',
|
||||
(0, 0.5),
|
||||
ids='delays={}'.format,
|
||||
portals = [
|
||||
await an.start_actor(
|
||||
f'errorer{i}',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
@pytest.mark.parametrize(
|
||||
'num_subactors',
|
||||
range(25, 26),
|
||||
ids= 'num_subs={}'.format,
|
||||
)
|
||||
def test_multierror_fast_nursery(
|
||||
reg_addr: tuple,
|
||||
start_method: str,
|
||||
num_subactors: int,
|
||||
delay: float,
|
||||
set_fork_aware_capture,
|
||||
fail_after_w_trace: FailAfterWTraceFactory,
|
||||
for i in range(2)
|
||||
]
|
||||
|
||||
# both one-shot subactors error concurrently, so the
|
||||
# `gather_contexts()` task-nursery collects them into a
|
||||
# `BaseExceptionGroup` (was two non-blocking
|
||||
# `run_in_actor()`s reaped at nursery teardown).
|
||||
async with gather_contexts(
|
||||
mngrs=[
|
||||
p.open_context(assert_err_ctx)
|
||||
for p in portals
|
||||
],
|
||||
):
|
||||
'''
|
||||
Verify we raise a ``BaseExceptionGroup`` out of a nursery where
|
||||
more then one actor errors and also with a delay before failure
|
||||
to test failure during an ongoing spawning.
|
||||
pass
|
||||
|
||||
'''
|
||||
async def main():
|
||||
# budget = 2× natural trio-backend cascade time for
|
||||
# 25 errorer subactors (~14s observed). on-timeout
|
||||
# diag snapshot → if the cancel cascade hangs
|
||||
# (observed under MTF backend with N>=14 errorer
|
||||
# subactors) we get a fresh ptree/wchan/py-spy dump
|
||||
# on disk INSTEAD of an opaque pytest timeout-kill.
|
||||
# See `tractor/_testing/trace.py` for the helper.
|
||||
async with fail_after_w_trace(30.0):
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as nursery:
|
||||
|
||||
for i in range(num_subactors):
|
||||
await nursery.run_in_actor(
|
||||
assert_err,
|
||||
name=f'errorer{i}',
|
||||
delay=delay
|
||||
)
|
||||
|
||||
# with pytest.raises(trio.MultiError) as exc_info:
|
||||
# NOTE, `trio.TooSlowError` from `fail_after_w_trace`
|
||||
# bubbles UN-wrapped if `open_nursery.__aexit__` never
|
||||
# gets re-entered; wrapped inside a `BaseExceptionGroup`
|
||||
# if it did. Accept both shapes so the matcher itself
|
||||
# doesn't lie about *what* failed.
|
||||
with pytest.raises(
|
||||
(BaseExceptionGroup, trio.TooSlowError),
|
||||
) as exc_info:
|
||||
with pytest.raises((
|
||||
BaseExceptionGroup,
|
||||
tractor.RemoteActorError,
|
||||
)):
|
||||
trio.run(main)
|
||||
|
||||
if isinstance(exc_info.value, trio.TooSlowError):
|
||||
pytest.fail(
|
||||
f'cancel cascade hung past 12s '
|
||||
f'(num_subactors={num_subactors}, delay={delay}); '
|
||||
f'see stderr for `fail_after_w_trace` snapshot path'
|
||||
)
|
||||
|
||||
assert exc_info.type == ExceptionGroup
|
||||
err = exc_info.value
|
||||
exceptions = err.exceptions
|
||||
|
||||
if len(exceptions) == 2:
|
||||
# sometimes oddly now there's an embedded BrokenResourceError ?
|
||||
for exc in exceptions:
|
||||
excs = getattr(exc, 'exceptions', None)
|
||||
if excs:
|
||||
exceptions = excs
|
||||
break
|
||||
|
||||
assert len(exceptions) == num_subactors
|
||||
|
||||
for exc in exceptions:
|
||||
assert isinstance(exc, tractor.RemoteActorError)
|
||||
assert exc.boxed_type is AssertionError
|
||||
|
||||
|
||||
async def do_nothing():
|
||||
pass
|
||||
|
|
@ -296,16 +248,16 @@ def test_cancel_single_subactor(
|
|||
'''
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as nursery:
|
||||
) as an:
|
||||
|
||||
portal = await nursery.start_actor(
|
||||
portal = await an.start_actor(
|
||||
'nothin', enable_modules=[__name__],
|
||||
)
|
||||
assert (await portal.run(do_nothing)) is None
|
||||
|
||||
if mechanism == 'nursery_cancel':
|
||||
# would hang otherwise
|
||||
await nursery.cancel()
|
||||
await an.cancel()
|
||||
else:
|
||||
raise mechanism
|
||||
|
||||
|
|
@ -337,8 +289,8 @@ async def test_cancel_infinite_streamer(
|
|||
trio.fail_after(4),
|
||||
trio.move_on_after(1) as cancel_scope
|
||||
):
|
||||
async with tractor.open_nursery() as n:
|
||||
portal = await n.start_actor(
|
||||
async with tractor.open_nursery() as an:
|
||||
portal = await an.start_actor(
|
||||
'donny',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
@ -351,36 +303,36 @@ async def test_cancel_infinite_streamer(
|
|||
|
||||
# we support trio's cancellation system
|
||||
assert cancel_scope.cancelled_caught
|
||||
assert n.cancel_called
|
||||
assert an.cancel_called
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'num_actors_and_errs',
|
||||
[
|
||||
# daemon actors sit idle while single task actors error out
|
||||
# daemon actors sit idle while one-shot task actors error out
|
||||
(1, tractor.RemoteActorError, AssertionError, (assert_err, {}), None),
|
||||
(2, BaseExceptionGroup, AssertionError, (assert_err, {}), None),
|
||||
(3, BaseExceptionGroup, AssertionError, (assert_err, {}), None),
|
||||
|
||||
# 1 daemon actor errors out while single task actors sleep forever
|
||||
# 1 daemon actor errors out while one-shot task actors sleep forever
|
||||
(3, tractor.RemoteActorError, AssertionError, (sleep_forever, {}),
|
||||
(assert_err, {}, True)),
|
||||
# daemon actors error out after brief delay while single task
|
||||
# daemon actors error out after brief delay while one-shot task
|
||||
# actors complete quickly
|
||||
(3, tractor.RemoteActorError, AssertionError,
|
||||
(do_nuthin, {}), (assert_err, {'delay': 1}, True)),
|
||||
# daemon complete quickly delay while single task
|
||||
# daemon complete quickly delay while one-shot task
|
||||
# actors error after brief delay
|
||||
(3, BaseExceptionGroup, AssertionError,
|
||||
(assert_err, {'delay': 1}), (do_nuthin, {}, False)),
|
||||
],
|
||||
ids=[
|
||||
'1_run_in_actor_fails',
|
||||
'2_run_in_actors_fail',
|
||||
'3_run_in_actors_fail',
|
||||
'1_one_shot_fails',
|
||||
'2_one_shots_fail',
|
||||
'3_one_shots_fail',
|
||||
'1_daemon_actors_fail',
|
||||
'1_daemon_actors_fail_all_run_in_actors_dun_quick',
|
||||
'no_daemon_actors_fail_all_run_in_actors_sleep_then_fail',
|
||||
'1_daemon_actors_fail_all_one_shots_dun_quick',
|
||||
'no_daemon_actors_fail_all_one_shots_sleep_then_fail',
|
||||
],
|
||||
)
|
||||
@tractor_test(
|
||||
|
|
@ -399,12 +351,22 @@ async def test_some_cancels_all(
|
|||
|
||||
This is the first and only supervisory strategy at the moment.
|
||||
|
||||
One-shot subactors run as concurrent `to_actor.run()` tasks
|
||||
in a local task-nursery so their errors raise WHILE the
|
||||
actor-nursery block is still open (vs the legacy
|
||||
`run_in_actor()` teardown-reap); the first error cancels the
|
||||
sibling one-shots (whose `trio.Cancelled`s the task-nursery
|
||||
absorbs) so the group shape is 1..num_actors
|
||||
`RemoteActorError`s depending on relay-vs-cancel timing —
|
||||
with `collapse_eg()` unwrapping the deterministic
|
||||
single-error cases to a bare `RemoteActorError`.
|
||||
|
||||
'''
|
||||
(
|
||||
num_actors,
|
||||
first_err,
|
||||
err_type,
|
||||
ria_func,
|
||||
one_shot_func,
|
||||
da_func,
|
||||
) = num_actors_and_errs
|
||||
try:
|
||||
|
|
@ -418,23 +380,30 @@ async def test_some_cancels_all(
|
|||
enable_modules=[__name__],
|
||||
))
|
||||
|
||||
func, kwargs = ria_func
|
||||
riactor_portals = []
|
||||
func, kwargs = one_shot_func
|
||||
async with (
|
||||
collapse_eg(),
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
for i in range(num_actors):
|
||||
# start actor(s) that will fail immediately
|
||||
riactor_portals.append(
|
||||
await an.run_in_actor(
|
||||
# schedule one-shot task actor(s); errors
|
||||
# raise into this task-nursery scope.
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
func,
|
||||
an=an,
|
||||
name=f'actor_{i}',
|
||||
**kwargs
|
||||
**kwargs,
|
||||
)
|
||||
)
|
||||
|
||||
if da_func:
|
||||
func, kwargs, expect_error = da_func
|
||||
for portal in dactor_portals:
|
||||
# if this function fails then we should error here
|
||||
# and the nursery should teardown all other actors
|
||||
# if this function fails then we should error
|
||||
# here and the nursery should teardown all
|
||||
# other actors
|
||||
try:
|
||||
await portal.run(func, **kwargs)
|
||||
|
||||
|
|
@ -451,18 +420,24 @@ async def test_some_cancels_all(
|
|||
pytest.fail(
|
||||
"Deamon call should fail at checkpoint?")
|
||||
|
||||
# should error here with a ``RemoteActorError`` or ``MultiError``
|
||||
# should error here with a `RemoteActorError` or a beg of them
|
||||
|
||||
except first_err as _err:
|
||||
except (
|
||||
BaseExceptionGroup,
|
||||
tractor.RemoteActorError,
|
||||
) as _err:
|
||||
err = _err
|
||||
if isinstance(err, BaseExceptionGroup):
|
||||
assert len(err.exceptions) == num_actors
|
||||
# only the concurrent multi-error cases can group; the
|
||||
# relay-vs-cancel race means anywhere from 1 (all
|
||||
# siblings cancelled before relaying) up to all
|
||||
# `num_actors` errors may populate the group.
|
||||
assert first_err is BaseExceptionGroup
|
||||
assert 1 <= len(err.exceptions) <= num_actors
|
||||
for exc in err.exceptions:
|
||||
if isinstance(exc, tractor.RemoteActorError):
|
||||
assert isinstance(exc, tractor.RemoteActorError)
|
||||
assert exc.boxed_type == err_type
|
||||
else:
|
||||
assert isinstance(exc, trio.Cancelled)
|
||||
elif isinstance(err, tractor.RemoteActorError):
|
||||
assert err.boxed_type == err_type
|
||||
|
||||
assert an.cancel_called is True
|
||||
|
|
@ -475,8 +450,20 @@ async def spawn_and_error(
|
|||
breadth: int,
|
||||
depth: int,
|
||||
) -> None:
|
||||
'''
|
||||
Recursively spawn a breadth-wide level of erroring one-shot
|
||||
subactors as concurrent `to_actor.run()` tasks; the leaf level
|
||||
errors ~simultaneously and each level's task-nursery groups
|
||||
whatever `RemoteActorError`s relay before the first one's
|
||||
cancel wins, boxing the (`ExceptionGroup`-shaped) group into
|
||||
this actor's own relayed error.
|
||||
|
||||
'''
|
||||
name = tractor.current_actor().name
|
||||
async with tractor.open_nursery() as nursery:
|
||||
async with (
|
||||
tractor.open_nursery() as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
for i in range(breadth):
|
||||
|
||||
if depth > 0:
|
||||
|
|
@ -496,7 +483,14 @@ async def spawn_and_error(
|
|||
kwargs = {
|
||||
'name': f'{name}_errorer_{i}',
|
||||
}
|
||||
await nursery.run_in_actor(*args, **kwargs)
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
*args,
|
||||
an=an,
|
||||
**kwargs,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# NOTE: `main_thread_forkserver` capture-fd hang class is no
|
||||
|
|
@ -538,7 +532,11 @@ async def test_nested_multierrors(
|
|||
depth: int,
|
||||
):
|
||||
'''
|
||||
Test that failed actor sets are wrapped in `BaseExceptionGroup`s.
|
||||
Test that a nested tree of concurrently failing one-shot
|
||||
subactors tears down cleanly, relaying (whatever subset of)
|
||||
the leaf `AssertionError`s (that win the per-level
|
||||
relay-vs-cancel race) re-boxed/grouped at each actor
|
||||
boundary.
|
||||
|
||||
Parametrized over recursion `depth ∈ {1, 3}`:
|
||||
|
||||
|
|
@ -588,6 +586,13 @@ async def test_nested_multierrors(
|
|||
# fork-spawn jitter + UDS-contention widens both `t1` and
|
||||
# `t2` further.
|
||||
#
|
||||
# NB post-#477 (`to_actor.run()` fan-out in a local
|
||||
# task-nursery) a race-tripped sibling's `Cancelled` is
|
||||
# ABSORBED by the task-nursery instead of landing in the
|
||||
# group — the raced case now shows as a *smaller* BEG, so
|
||||
# this marker should consistently `xpass`; drop it once CI
|
||||
# confirms.
|
||||
#
|
||||
# With `strict=False` the clean-cascade cases (most
|
||||
# depth=1 runs, rare depth=3 runs) report as `xpassed`
|
||||
# while the race-tripped cases report as `xfailed` —
|
||||
|
|
@ -672,6 +677,14 @@ async def test_nested_multierrors(
|
|||
timeout = 16
|
||||
case ('main_thread_forkserver', 3):
|
||||
timeout = 30
|
||||
# any other fork-based backend (`mp_spawn` et al) pays
|
||||
# the same per-spawn round-trip costs as MTF so rides
|
||||
# its budgets; without a default arm `timeout` is left
|
||||
# unbound -> `UnboundLocalError` at the scaling below.
|
||||
case (_, 1):
|
||||
timeout = 16
|
||||
case (_, 3):
|
||||
timeout = 30
|
||||
|
||||
# inflate the budget by the throttle headroom probed above so
|
||||
# a slow box doesn't masquerade as a deadline regression.
|
||||
|
|
@ -684,66 +697,81 @@ async def test_nested_multierrors(
|
|||
|
||||
async with fail_after_w_trace(timeout):
|
||||
try:
|
||||
async with tractor.open_nursery() as nursery:
|
||||
async with (
|
||||
tractor.open_nursery() as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
for i in range(subactor_breadth):
|
||||
await nursery.run_in_actor(
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
spawn_and_error,
|
||||
an=an,
|
||||
name=f'spawner_{i}',
|
||||
breadth=subactor_breadth,
|
||||
depth=depth,
|
||||
)
|
||||
except BaseExceptionGroup as err:
|
||||
assert len(err.exceptions) == subactor_breadth
|
||||
for subexc in err.exceptions:
|
||||
|
||||
# verify first level actor errors are wrapped as remote
|
||||
if _friggin_windows:
|
||||
|
||||
)
|
||||
except (
|
||||
BaseExceptionGroup,
|
||||
tractor.RemoteActorError,
|
||||
) as err:
|
||||
# group membership is bounded by the relay-vs-cancel
|
||||
# race: the first spawner-tree's error cancels its
|
||||
# siblings, whose own errors only group when relayed
|
||||
# first; a fully-raced tree even collapses (via the
|
||||
# runtime's own `collapse_eg()` unwrapping each level's
|
||||
# single-member group) to a bare `RemoteActorError`
|
||||
# re-boxing the leaf `AssertionError` at every actor
|
||||
# boundary. The deterministic exact-breadth nested-BEG
|
||||
# was the legacy `run_in_actor()` reap-all-at-teardown.
|
||||
subexcs: list[BaseException] = (
|
||||
err.exceptions
|
||||
if isinstance(err, BaseExceptionGroup)
|
||||
else [err]
|
||||
)
|
||||
assert 1 <= len(subexcs) <= subactor_breadth
|
||||
for subexc in subexcs:
|
||||
if (
|
||||
_friggin_windows
|
||||
and
|
||||
isinstance(subexc, trio.Cancelled)
|
||||
):
|
||||
# windows is often too slow and cancellation seems
|
||||
# to happen before an actor is spawned
|
||||
if isinstance(subexc, trio.Cancelled):
|
||||
continue
|
||||
|
||||
elif isinstance(subexc, tractor.RemoteActorError):
|
||||
# on windows it seems we can't exactly be sure wtf
|
||||
# will happen..
|
||||
assert subexc.boxed_type in (
|
||||
tractor.RemoteActorError,
|
||||
trio.Cancelled,
|
||||
BaseExceptionGroup,
|
||||
)
|
||||
|
||||
elif isinstance(subexc, BaseExceptionGroup):
|
||||
for subsub in subexc.exceptions:
|
||||
|
||||
if subsub in (tractor.RemoteActorError,):
|
||||
subsub = subsub.boxed_type
|
||||
|
||||
assert type(subsub) in (
|
||||
trio.Cancelled,
|
||||
BaseExceptionGroup,
|
||||
)
|
||||
else:
|
||||
assert isinstance(subexc, tractor.RemoteActorError)
|
||||
|
||||
if depth > 0 and subactor_breadth > 1:
|
||||
# XXX not sure what's up with this..
|
||||
# on windows sometimes spawning is just too slow and
|
||||
# we get back the (sent) cancel signal instead
|
||||
if _friggin_windows:
|
||||
if isinstance(subexc, tractor.RemoteActorError):
|
||||
assert subexc.boxed_type in (
|
||||
BaseExceptionGroup,
|
||||
tractor.RemoteActorError
|
||||
)
|
||||
else:
|
||||
assert isinstance(subexc, BaseExceptionGroup)
|
||||
else:
|
||||
assert subexc.boxed_type is ExceptionGroup
|
||||
else:
|
||||
assert subexc.boxed_type in (
|
||||
accepted: tuple[Type[BaseException], ...] = (
|
||||
# ≥2 sub-tree errors relayed before the
|
||||
# cancel-cascade won → grouped per-level.
|
||||
ExceptionGroup,
|
||||
# every level collapsed down to its lone
|
||||
# relayed (leaf) error.
|
||||
AssertionError,
|
||||
# a mid-level spawner relays an
|
||||
# already-boxed (collapsed) leaf chain,
|
||||
# re-boxing the `RemoteActorError` itself.
|
||||
tractor.RemoteActorError,
|
||||
trio.Cancelled
|
||||
# under heavy load a runtime-internal reap
|
||||
# deadline can inject a `trio.Cancelled`
|
||||
# into a child's group before relay (the
|
||||
# same class the depth=3 throttle-xfail
|
||||
# covers) upgrading it from an
|
||||
# `ExceptionGroup`.
|
||||
BaseExceptionGroup,
|
||||
)
|
||||
if _friggin_windows:
|
||||
# on windows it seems we can't exactly be
|
||||
# sure wtf will happen..
|
||||
accepted += (
|
||||
trio.Cancelled,
|
||||
)
|
||||
assert subexc.boxed_type in accepted
|
||||
else:
|
||||
pytest.fail(
|
||||
'Should have raised a (grouped) `RemoteActorError`?'
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -764,8 +792,8 @@ def test_cancel_via_SIGINT(
|
|||
with trio.fail_after(2):
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as tn:
|
||||
await tn.start_actor('sucka')
|
||||
) as an:
|
||||
await an.start_actor('sucka')
|
||||
if 'mp' in start_method:
|
||||
time.sleep(0.1)
|
||||
os.kill(pid, signal.SIGINT)
|
||||
|
|
@ -809,11 +837,13 @@ def test_cancel_via_SIGINT_other_task(
|
|||
):
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as tn:
|
||||
) as an:
|
||||
# just keep a set of (daemon) subactors alive for the
|
||||
# SIGINT to cancel (was 3 `run_in_actor(sleep_forever)`
|
||||
# one-shots — a daemon needs no "main" task to idle).
|
||||
for i in range(3):
|
||||
await tn.run_in_actor(
|
||||
sleep_forever,
|
||||
name='namesucka',
|
||||
await an.start_actor(
|
||||
f'namesucka_{i}',
|
||||
)
|
||||
task_status.started()
|
||||
await trio.sleep_forever()
|
||||
|
|
@ -854,8 +884,11 @@ async def spin_for(period=3):
|
|||
async def spawn_sub_with_sync_blocking_task():
|
||||
async with tractor.open_nursery() as an:
|
||||
print('starting sync blocking subactor..\n')
|
||||
await an.run_in_actor(
|
||||
# one-shot: parks HERE awaiting the sync-sleeping
|
||||
# grandchild's result until cancelled from above.
|
||||
await tractor.to_actor.run(
|
||||
spin_for,
|
||||
an=an,
|
||||
name='sleeper',
|
||||
)
|
||||
print('exiting first subactor layer..\n')
|
||||
|
|
@ -961,11 +994,19 @@ def test_cancel_while_childs_child_in_sync_sleep(
|
|||
debug_mode=debug_mode,
|
||||
registry_addrs=[reg_addr],
|
||||
) as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
await an.run_in_actor(
|
||||
# bg one-shot: parks on the middle actor's result
|
||||
# (itself parked on the sync-sleeping grandchild)
|
||||
# until the `assert 0` below cancels this scope.
|
||||
tn.start_soon(
|
||||
partial(
|
||||
tractor.to_actor.run,
|
||||
spawn_sub_with_sync_blocking_task,
|
||||
an=an,
|
||||
name='sync_blocking_sub',
|
||||
)
|
||||
)
|
||||
await trio.sleep(1)
|
||||
|
||||
if man_cancel_outer:
|
||||
|
|
@ -1013,8 +1054,8 @@ def test_fast_graceful_cancel_when_spawn_task_in_soft_proc_wait_for_daemon(
|
|||
start = time.time()
|
||||
try:
|
||||
async with trio.open_nursery() as nurse:
|
||||
async with tractor.open_nursery() as tn:
|
||||
p = await tn.start_actor(
|
||||
async with tractor.open_nursery() as an:
|
||||
p = await an.start_actor(
|
||||
'fast_boi',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -156,8 +156,8 @@ def test_actor_managed_trio_nursery_task_error_cancels_aio(
|
|||
async def main():
|
||||
|
||||
# cancel the nursery shortly after boot
|
||||
async with tractor.open_nursery() as n:
|
||||
p = await n.start_actor(
|
||||
async with tractor.open_nursery() as an:
|
||||
p = await an.start_actor(
|
||||
'nursery_mngr',
|
||||
infect_asyncio=asyncio_mode, # TODO, is this enabling debug mode?
|
||||
enable_modules=[__name__],
|
||||
|
|
|
|||
|
|
@ -5,11 +5,13 @@ Let's make sure them docs work yah?
|
|||
from contextlib import contextmanager
|
||||
import itertools
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import subprocess
|
||||
import platform
|
||||
import shutil
|
||||
from typing import Callable
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
import tractor
|
||||
|
|
@ -21,6 +23,184 @@ _non_linux: bool = platform.system() != 'Linux'
|
|||
_friggin_macos: bool = platform.system() == 'Darwin'
|
||||
|
||||
|
||||
def _kill_proc_tree(proc: subprocess.Popen) -> None:
|
||||
'''
|
||||
Terminate an example process and its POSIX descendants.
|
||||
|
||||
'''
|
||||
try:
|
||||
if platform.system() == 'Windows':
|
||||
proc.kill()
|
||||
else:
|
||||
os.killpg(proc.pid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
|
||||
|
||||
def _reap_killed_proc(
|
||||
proc: subprocess.Popen,
|
||||
) -> tuple[bytes, bytes]:
|
||||
'''
|
||||
Reap a killed process without waiting on Windows descendants.
|
||||
|
||||
'''
|
||||
if platform.system() != 'Windows':
|
||||
return proc.communicate()
|
||||
|
||||
proc.wait(timeout=5)
|
||||
if proc.stdin:
|
||||
proc.stdin.close()
|
||||
if proc.stdout:
|
||||
proc.stdout.close()
|
||||
if proc.stderr:
|
||||
proc.stderr.close()
|
||||
return b'', b''
|
||||
|
||||
|
||||
def _wait_for_proc(
|
||||
proc: subprocess.Popen,
|
||||
timeout: float,
|
||||
test_log: tractor.log.StackLevelAdapter,
|
||||
) -> None:
|
||||
'''
|
||||
Wait for an example process and surface its captured output.
|
||||
|
||||
'''
|
||||
try:
|
||||
out, err = proc.communicate(timeout=timeout)
|
||||
|
||||
except subprocess.TimeoutExpired as timeout_exc:
|
||||
test_log.exception(
|
||||
f'Example failed to finish within {timeout}s ??\n'
|
||||
)
|
||||
_kill_proc_tree(proc)
|
||||
out, err = _reap_killed_proc(proc)
|
||||
if platform.system() == 'Windows':
|
||||
out = timeout_exc.output or b''
|
||||
err = timeout_exc.stderr or b''
|
||||
|
||||
errmsg: str = err.decode(errors='replace')
|
||||
|
||||
# NOTE: always include captured stdout and stderr for a non-zero
|
||||
# exit. Depending on the final stderr line previously hid grouped
|
||||
# exception diagnostics; see GH #473.
|
||||
#
|
||||
# The prior impl only raised when the LAST stderr
|
||||
# line contained 'Error', swallowing any crash whose
|
||||
# traceback ends in a non-`XxxError:` line; in
|
||||
# particular EVERY `tractor` root-actor crash ends
|
||||
# with the strict-EG collapse note,
|
||||
# '( ^^^ this exc was collapsed from a group ^^^ )',
|
||||
# so ALL such failures were reduced to a bare
|
||||
# `assert 1 == 0` in CI logs.. see GH #473.
|
||||
rc: int|None = proc.returncode
|
||||
if rc:
|
||||
outmsg: str = out.decode(errors='replace')
|
||||
raise Exception(
|
||||
f'Example script exited with rc={rc} !?\n'
|
||||
f'\n'
|
||||
f'stdout:\n'
|
||||
f'{outmsg}\n'
|
||||
f'\n'
|
||||
f'stderr:\n'
|
||||
f'{errmsg}\n'
|
||||
)
|
||||
|
||||
# if we get some gnarly output let's aggregate and raise
|
||||
if errmsg:
|
||||
errlines = errmsg.splitlines()
|
||||
last_error = errlines[-1]
|
||||
if (
|
||||
'Error' in last_error
|
||||
|
||||
# XXX: currently we print this to console, but maybe
|
||||
# shouldn't eventually once we figure out what's
|
||||
# a better way to be explicit about aio side
|
||||
# cancels?
|
||||
and
|
||||
'asyncio.exceptions.CancelledError' not in last_error
|
||||
):
|
||||
raise Exception(errmsg)
|
||||
|
||||
assert proc.returncode == 0
|
||||
|
||||
|
||||
def test_wait_for_failed_example_captures_output():
|
||||
'''
|
||||
Preserve diagnostics from a subprocess which already exited.
|
||||
|
||||
The previous `poll()` guard skipped `communicate()` when a fast
|
||||
failure returned a non-zero status before the parent checked it.
|
||||
Its stdout and stderr were therefore reported as empty. This
|
||||
fake process begins with `returncode=1` and returns non-UTF-8
|
||||
output, proving the helper always drains both pipes and replaces
|
||||
undecodable bytes without hiding the original process failure.
|
||||
|
||||
'''
|
||||
proc = Mock()
|
||||
proc.returncode = 1
|
||||
proc.communicate.return_value = (
|
||||
b'stdout\xff',
|
||||
b'stderr\xff',
|
||||
)
|
||||
|
||||
with pytest.raises(Exception) as exc_info:
|
||||
_wait_for_proc(
|
||||
proc=proc,
|
||||
timeout=1,
|
||||
test_log=Mock(),
|
||||
)
|
||||
|
||||
proc.communicate.assert_called_once_with(timeout=1)
|
||||
errmsg: str = str(exc_info.value)
|
||||
assert 'stdout\ufffd' in errmsg
|
||||
assert 'stderr\ufffd' in errmsg
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
platform.system() == 'Windows',
|
||||
reason='POSIX process groups are unavailable on Windows',
|
||||
)
|
||||
def test_wait_for_timed_out_example_reaps_group(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
'''
|
||||
Kill the example process group and reap its leader on timeout.
|
||||
|
||||
The old timeout branch killed only the immediate process and
|
||||
never drained it. Actor descendants could retain the capture
|
||||
pipes while the leader remained unreaped, hanging CI until its
|
||||
job timeout. This fake process raises `TimeoutExpired` on the
|
||||
timed wait and completes on the second `communicate()` call;
|
||||
the assertions prove group-directed `SIGKILL` precedes that
|
||||
final drain and leaves a concrete non-zero return code.
|
||||
|
||||
'''
|
||||
proc = Mock()
|
||||
proc.pid = 1234
|
||||
|
||||
def communicate(timeout=None):
|
||||
if timeout is not None:
|
||||
raise subprocess.TimeoutExpired('example', timeout)
|
||||
proc.returncode = -signal.SIGKILL
|
||||
return b'', b'timed out'
|
||||
|
||||
proc.communicate.side_effect = communicate
|
||||
killpg = Mock()
|
||||
monkeypatch.setattr(os, 'killpg', killpg)
|
||||
|
||||
with pytest.raises(Exception, match='timed out'):
|
||||
_wait_for_proc(
|
||||
proc=proc,
|
||||
timeout=.01,
|
||||
test_log=Mock(),
|
||||
)
|
||||
|
||||
killpg.assert_called_once_with(1234, signal.SIGKILL)
|
||||
assert proc.communicate.call_count == 2
|
||||
assert proc.returncode == -signal.SIGKILL
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def run_example_in_subproc(
|
||||
loglevel: str,
|
||||
|
|
@ -61,14 +241,14 @@ def run_example_in_subproc(
|
|||
]
|
||||
else:
|
||||
script_file = testdir.makefile('.py', script_code)
|
||||
kwargs['start_new_session'] = True
|
||||
cmdargs = [
|
||||
sys.executable,
|
||||
str(script_file),
|
||||
]
|
||||
|
||||
# XXX: BE FOREVER WARNED: if you enable lots of tractor logging
|
||||
# in the subprocess it may cause infinite blocking on the pipes
|
||||
# due to backpressure!!!
|
||||
# Captured pipes are drained by `_wait_for_proc()` while the
|
||||
# example runs.
|
||||
proc = testdir.popen(
|
||||
cmdargs,
|
||||
stdin=subprocess.PIPE,
|
||||
|
|
@ -77,9 +257,20 @@ def run_example_in_subproc(
|
|||
**kwargs,
|
||||
)
|
||||
assert not proc.returncode
|
||||
try:
|
||||
yield proc
|
||||
proc.wait()
|
||||
assert proc.returncode == 0
|
||||
except BaseException:
|
||||
if proc.poll() is None:
|
||||
try:
|
||||
_kill_proc_tree(proc)
|
||||
_reap_killed_proc(proc)
|
||||
except Exception:
|
||||
pass
|
||||
raise
|
||||
else:
|
||||
if proc.poll() is None:
|
||||
_kill_proc_tree(proc)
|
||||
_reap_killed_proc(proc)
|
||||
|
||||
yield run
|
||||
|
||||
|
|
@ -145,21 +336,6 @@ def test_example(
|
|||
'This test does run just fine "in person" however..'
|
||||
)
|
||||
|
||||
if (
|
||||
'uds_transport_actor_tree' in ex_file
|
||||
and
|
||||
_friggin_macos
|
||||
and
|
||||
ci_env
|
||||
):
|
||||
pytest.skip(
|
||||
'UDS-transport example reliably fails on macOS CI.\n'
|
||||
'UDS-on-macOS is otherwise un-exercised by the matrix\n'
|
||||
'(no `tpt_proto=uds` macOS job), so this new example is\n'
|
||||
'the first to surface it; the macOS UDS path needs\n'
|
||||
'root-causing. Passes on Linux.'
|
||||
)
|
||||
|
||||
from .conftest import cpu_perf_headroom
|
||||
|
||||
timeout: float = (
|
||||
|
|
@ -178,33 +354,8 @@ def test_example(
|
|||
code = ex.read()
|
||||
|
||||
with run_example_in_subproc(code) as proc:
|
||||
err = None
|
||||
try:
|
||||
if not proc.poll():
|
||||
_, err = proc.communicate(timeout=timeout)
|
||||
|
||||
except subprocess.TimeoutExpired as e:
|
||||
test_log.exception(
|
||||
f'Example failed to finish within {timeout}s ??\n'
|
||||
_wait_for_proc(
|
||||
proc=proc,
|
||||
timeout=timeout,
|
||||
test_log=test_log,
|
||||
)
|
||||
proc.kill()
|
||||
err = e.stderr
|
||||
|
||||
# if we get some gnarly output let's aggregate and raise
|
||||
if err:
|
||||
errmsg = err.decode()
|
||||
errlines = errmsg.splitlines()
|
||||
last_error = errlines[-1]
|
||||
if (
|
||||
'Error' in last_error
|
||||
|
||||
# XXX: currently we print this to console, but maybe
|
||||
# shouldn't eventually once we figure out what's
|
||||
# a better way to be explicit about aio side
|
||||
# cancels?
|
||||
and
|
||||
'asyncio.exceptions.CancelledError' not in last_error
|
||||
):
|
||||
raise Exception(errmsg)
|
||||
|
||||
assert proc.returncode == 0
|
||||
|
|
|
|||
|
|
@ -20,10 +20,23 @@ from typing import (
|
|||
import pytest
|
||||
import trio
|
||||
import tractor
|
||||
|
||||
# `infect_asyncio` mode is unsupported on Windows (asyncio's
|
||||
# `ProactorEventLoop` is incompatible with our `trio` guest-mode
|
||||
# interop and currently hangs/crashes the run). Skip the module on
|
||||
# Windows so the CI leg completes + reports the rest of the suite.
|
||||
import platform
|
||||
if platform.system() == 'Windows':
|
||||
pytest.skip(
|
||||
'infect_asyncio mode is unsupported on Windows',
|
||||
allow_module_level=True,
|
||||
)
|
||||
|
||||
from tractor import (
|
||||
current_actor,
|
||||
Actor,
|
||||
to_asyncio,
|
||||
to_actor,
|
||||
RemoteActorError,
|
||||
ContextCancelled,
|
||||
)
|
||||
|
|
@ -110,8 +123,9 @@ def test_trio_cancels_aio_on_actor_side(
|
|||
registry_addrs=[reg_addr],
|
||||
debug_mode=debug_mode,
|
||||
) as an:
|
||||
await an.run_in_actor(
|
||||
await to_actor.run(
|
||||
trio_cancels_single_aio_task,
|
||||
an=an,
|
||||
infect_asyncio=True,
|
||||
)
|
||||
|
||||
|
|
@ -157,6 +171,28 @@ async def asyncio_actor(
|
|||
raise
|
||||
|
||||
|
||||
@tractor.context
|
||||
async def sleep_forever_aio_ctx(
|
||||
ctx: tractor.Context,
|
||||
expect_err: str = 'trio.Cancelled',
|
||||
) -> None:
|
||||
'''
|
||||
`@context` shim so a parent can spawn a forever-sleeping
|
||||
infected-`asyncio` task via `Portal.open_context()` and cancel it
|
||||
(via `Portal.cancel_actor()` or an enclosing `trio` cancel scope),
|
||||
asserting the graceful `trio.Cancelled` teardown.
|
||||
|
||||
Replaces the legacy `ActorNursery.run_in_actor()` spawn the
|
||||
aio-cancel tests below used to rely on (removed with #477).
|
||||
|
||||
'''
|
||||
await ctx.started()
|
||||
await asyncio_actor(
|
||||
target='aio_sleep_forever',
|
||||
expect_err=expect_err,
|
||||
)
|
||||
|
||||
|
||||
def test_aio_simple_error(
|
||||
reg_addr: tuple[str, int],
|
||||
debug_mode: bool,
|
||||
|
|
@ -172,8 +208,9 @@ def test_aio_simple_error(
|
|||
registry_addrs=[reg_addr],
|
||||
debug_mode=debug_mode,
|
||||
) as an:
|
||||
await an.run_in_actor(
|
||||
await to_actor.run(
|
||||
asyncio_actor,
|
||||
an=an,
|
||||
target='sleep_and_err',
|
||||
expect_err='AssertionError',
|
||||
infect_asyncio=True,
|
||||
|
|
@ -207,18 +244,34 @@ def test_tractor_cancels_aio(
|
|||
|
||||
'''
|
||||
async def main():
|
||||
# anti-hang wall-clock cap: a per-test `trio.fail_after`
|
||||
# is the blessed guard here since `pytest-timeout`'s
|
||||
# global cap is intentionally off (see the `pyproject`
|
||||
# NOTE — it breaks trio under fork backends). Generous +
|
||||
# CPU-headroom-scaled bc this is an anti-hang guard, not
|
||||
# a perf assertion; a wedged ria-reaper once hung this
|
||||
# test forever (the `._ria_nursery`-removal regression).
|
||||
from .conftest import cpu_perf_headroom
|
||||
with trio.fail_after(9 * cpu_perf_headroom()):
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=debug_mode,
|
||||
registry_addrs=[reg_addr],
|
||||
) as an:
|
||||
portal = await an.run_in_actor(
|
||||
asyncio_actor,
|
||||
target='aio_sleep_forever',
|
||||
expect_err='trio.Cancelled',
|
||||
p: tractor.Portal = await an.start_actor(
|
||||
'aio_daemon',
|
||||
enable_modules=[__name__],
|
||||
infect_asyncio=True,
|
||||
)
|
||||
# cancel the entire remote runtime
|
||||
await portal.cancel_actor()
|
||||
async with (
|
||||
# `.cancel_actor()` below tears the ctx down
|
||||
expect_ctxc(yay=True),
|
||||
p.open_context(
|
||||
sleep_forever_aio_ctx,
|
||||
) as (ctx, first),
|
||||
):
|
||||
# cancel the entire remote runtime while its
|
||||
# infected-`asyncio` task sleeps forever
|
||||
await p.cancel_actor()
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
|
@ -236,13 +289,19 @@ def test_trio_cancels_aio(
|
|||
with trio.move_on_after(1):
|
||||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as tn:
|
||||
await tn.run_in_actor(
|
||||
asyncio_actor,
|
||||
target='aio_sleep_forever',
|
||||
expect_err='trio.Cancelled',
|
||||
) as an:
|
||||
p: tractor.Portal = await an.start_actor(
|
||||
'aio_daemon',
|
||||
enable_modules=[__name__],
|
||||
infect_asyncio=True,
|
||||
)
|
||||
async with p.open_context(
|
||||
sleep_forever_aio_ctx,
|
||||
) as (ctx, first):
|
||||
# block until the enclosing `move_on_after`
|
||||
# cancels this `trio` scope, tearing down the
|
||||
# infected-aio task via ctx cancellation
|
||||
await trio.sleep_forever()
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
|
@ -392,17 +451,16 @@ def test_aio_cancelled_from_aio_causes_trio_cancelled(
|
|||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as an:
|
||||
p: tractor.Portal = await an.run_in_actor(
|
||||
# `to_actor.run()` blocks on the one-shot's result and
|
||||
# relays the remote error here in the caller's task.
|
||||
with trio.fail_after(1 + delay):
|
||||
await to_actor.run(
|
||||
asyncio_actor,
|
||||
an=an,
|
||||
target='aio_cancel',
|
||||
expect_err='tractor.to_asyncio.AsyncioCancelled',
|
||||
infect_asyncio=True,
|
||||
)
|
||||
# NOTE: normally the `an.__aexit__()` waits on the
|
||||
# portal's result but we do it explicitly here
|
||||
# to avoid indent levels.
|
||||
with trio.fail_after(1 + delay):
|
||||
await p.wait_for_result()
|
||||
|
||||
with pytest.raises(
|
||||
expected_exception=(RemoteActorError, ExceptionGroup),
|
||||
|
|
@ -603,13 +661,13 @@ def test_basic_interloop_channel_stream(
|
|||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as an:
|
||||
portal = await an.run_in_actor(
|
||||
# should raise RAE diectly
|
||||
await to_actor.run(
|
||||
stream_from_aio,
|
||||
an=an,
|
||||
infect_asyncio=True,
|
||||
fan_out=fan_out,
|
||||
)
|
||||
# should raise RAE diectly
|
||||
await portal.result()
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
|
@ -622,13 +680,13 @@ def test_trio_error_cancels_intertask_chan(
|
|||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
) as an:
|
||||
portal = await an.run_in_actor(
|
||||
# should trigger remote actor error
|
||||
await to_actor.run(
|
||||
stream_from_aio,
|
||||
an=an,
|
||||
trio_raise_err=True,
|
||||
infect_asyncio=True,
|
||||
)
|
||||
# should trigger remote actor error
|
||||
await portal.result()
|
||||
|
||||
with pytest.raises(RemoteActorError) as excinfo:
|
||||
trio.run(main)
|
||||
|
|
@ -658,14 +716,14 @@ def test_trio_closes_early_causes_aio_checkpoint_raise(
|
|||
# enable_stack_on_sig=True,
|
||||
registry_addrs=[reg_addr],
|
||||
) as an:
|
||||
portal = await an.run_in_actor(
|
||||
# should raise RAE diectly
|
||||
print('waiting on final infected subactor result..')
|
||||
res: None = await to_actor.run(
|
||||
stream_from_aio,
|
||||
an=an,
|
||||
trio_exit_early=True,
|
||||
infect_asyncio=True,
|
||||
)
|
||||
# should raise RAE diectly
|
||||
print('waiting on final infected subactor result..')
|
||||
res: None = await portal.wait_for_result()
|
||||
assert res is None
|
||||
print(f'infected subactor returned result: {res!r}\n')
|
||||
|
||||
|
|
@ -709,15 +767,15 @@ def test_aio_exits_early_relays_AsyncioTaskExited(
|
|||
debug_mode=debug_mode,
|
||||
# enable_stack_on_sig=True,
|
||||
) as an:
|
||||
portal = await an.run_in_actor(
|
||||
# should raise RAE diectly
|
||||
print('waiting on final infected subactor result..')
|
||||
res: None = await to_actor.run(
|
||||
stream_from_aio,
|
||||
an=an,
|
||||
infect_asyncio=True,
|
||||
trio_exit_early=False,
|
||||
aio_exit_early=True,
|
||||
)
|
||||
# should raise RAE diectly
|
||||
print('waiting on final infected subactor result..')
|
||||
res: None = await portal.wait_for_result()
|
||||
assert res is None
|
||||
print(f'infected subactor returned result: {res!r}\n')
|
||||
|
||||
|
|
@ -749,17 +807,19 @@ def test_aio_errors_and_channel_propagates_and_closes(
|
|||
registry_addrs=[reg_addr],
|
||||
debug_mode=debug_mode,
|
||||
) as an:
|
||||
portal = await an.run_in_actor(
|
||||
# should trigger RAE directly, not an eg.
|
||||
await to_actor.run(
|
||||
stream_from_aio,
|
||||
an=an,
|
||||
aio_raise_err=True,
|
||||
infect_asyncio=True,
|
||||
)
|
||||
# should trigger RAE directly, not an eg.
|
||||
await portal.result()
|
||||
|
||||
with pytest.raises(
|
||||
# NOTE: bc we directly wait on `Portal.result()` instead
|
||||
# of capturing it inside the `ActorNursery` machinery.
|
||||
# NOTE: bc `to_actor.run()` blocks on + relays the result
|
||||
# in the caller's task (not captured inside the
|
||||
# `ActorNursery` teardown machinery) we get a direct RAE,
|
||||
# not an eg.
|
||||
expected_exception=RemoteActorError,
|
||||
) as excinfo:
|
||||
trio.run(main)
|
||||
|
|
|
|||
|
|
@ -163,12 +163,12 @@ def test_do_not_swallow_error_before_started_by_remote_contextcancelled(
|
|||
async def main():
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=debug_mode,
|
||||
) as n:
|
||||
portal = await n.start_actor(
|
||||
) as an:
|
||||
portal = await an.start_actor(
|
||||
'errorer',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
await n.start_actor(
|
||||
await an.start_actor(
|
||||
'sleeper',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,160 @@
|
|||
'''
|
||||
Regression tests for the cold package import surface.
|
||||
|
||||
'''
|
||||
import json
|
||||
import os
|
||||
from statistics import median
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import (
|
||||
Any,
|
||||
get_type_hints,
|
||||
)
|
||||
|
||||
from tractor.discovery import (
|
||||
_addr,
|
||||
_multiaddr,
|
||||
)
|
||||
from tractor.ipc import (
|
||||
_tcp,
|
||||
_uds,
|
||||
)
|
||||
|
||||
|
||||
def run_cold_import(code: str) -> dict[str, object]:
|
||||
result = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
'-c',
|
||||
code,
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
return json.loads(result.stdout)
|
||||
|
||||
|
||||
def test_lazy_to_asyncio_package_api():
|
||||
'''
|
||||
Keep the public lazy submodule discoverable without eagerly
|
||||
importing it.
|
||||
|
||||
Before the lazy conversion, package import side effects exposed
|
||||
`to_asyncio` to `dir()` and wildcard imports. Exercise those APIs
|
||||
in cold interpreters so this test proves normal `import tractor`
|
||||
leaves `asyncio` unloaded, while discovery and wildcard access
|
||||
still advertise and resolve the public submodule.
|
||||
|
||||
'''
|
||||
cold = run_cold_import(
|
||||
'import json, sys, tractor; '
|
||||
'print(json.dumps({'
|
||||
'"advertised": "to_asyncio" in dir(tractor), '
|
||||
'"asyncio_loaded": "asyncio" in sys.modules}))'
|
||||
)
|
||||
assert cold == {
|
||||
'advertised': True,
|
||||
'asyncio_loaded': False,
|
||||
}
|
||||
|
||||
wildcard = run_cold_import(
|
||||
'import json; '
|
||||
'from tractor import *; '
|
||||
'print(json.dumps({'
|
||||
'"module": to_asyncio.__name__}))'
|
||||
)
|
||||
assert wildcard == {
|
||||
'module': 'tractor.to_asyncio',
|
||||
}
|
||||
|
||||
|
||||
def test_cold_import_budget():
|
||||
'''
|
||||
Keep cold package import below the pre-optimization regression.
|
||||
|
||||
The original `inspect.stack()` caller lookup made a fresh
|
||||
`import tractor` take about 0.42s and dominate actor startup.
|
||||
Run seven independent interpreters and gate their median at a
|
||||
deliberately broad 0.35s: over twice the measured ~0.145s
|
||||
baseline, but low enough to catch restoration of that hot path.
|
||||
|
||||
Taking the median absorbs process-start and shared-runner noise.
|
||||
The child measures only its import, rather than parent-side
|
||||
process creation. `TRACTOR_IMPORT_BUDGET_S` provides an explicit,
|
||||
reviewable override for platforms that establish a different
|
||||
baseline instead of silently weakening the project default.
|
||||
|
||||
Each child also reports the modules whose eager loading this PR
|
||||
intentionally removes, proving a timing pass cannot hide a
|
||||
dependency-import regression.
|
||||
|
||||
'''
|
||||
budget_s = float(
|
||||
os.environ.get(
|
||||
'TRACTOR_IMPORT_BUDGET_S',
|
||||
'0.35',
|
||||
)
|
||||
)
|
||||
optional_mods = (
|
||||
'asyncio',
|
||||
'bidict',
|
||||
'colorlog',
|
||||
'multiaddr',
|
||||
'wrapt',
|
||||
)
|
||||
code = (
|
||||
'import json, sys, time; '
|
||||
'started = time.perf_counter(); '
|
||||
'import tractor; '
|
||||
'elapsed = time.perf_counter() - started; '
|
||||
f'optional = {optional_mods!r}; '
|
||||
'print(json.dumps({'
|
||||
'"elapsed": elapsed, '
|
||||
'"loaded": [name for name in optional '
|
||||
'if name in sys.modules]}))'
|
||||
)
|
||||
samples = [
|
||||
run_cold_import(code)
|
||||
for _ in range(7)
|
||||
]
|
||||
elapsed = [
|
||||
float(sample['elapsed'])
|
||||
for sample in samples
|
||||
]
|
||||
loaded = {
|
||||
name
|
||||
for sample in samples
|
||||
for name in sample['loaded']
|
||||
}
|
||||
|
||||
assert not loaded
|
||||
assert median(elapsed) < budget_s, (
|
||||
f'cold import median exceeded {budget_s:.3f}s budget: '
|
||||
f'{elapsed!r}'
|
||||
)
|
||||
|
||||
|
||||
def test_lazy_annotation_names_resolve():
|
||||
'''
|
||||
Resolve annotations without importing optional dependencies.
|
||||
|
||||
Moving annotation-only third-party names under `TYPE_CHECKING`
|
||||
left their runtime globals undefined, causing
|
||||
`typing.get_type_hints()` to raise `NameError`. Resolve every
|
||||
affected API and prove the lazy aliases retain import-free runtime
|
||||
introspection.
|
||||
|
||||
'''
|
||||
assert get_type_hints(_multiaddr.mk_maddr)['return'] is Any
|
||||
assert get_type_hints(_tcp.MsgpackTCPStream.maddr.fget)[
|
||||
'return'
|
||||
] is Any
|
||||
assert get_type_hints(_uds.MsgpackUDSStream.maddr.fget)[
|
||||
'return'
|
||||
] == Any|str
|
||||
assert get_type_hints(_addr.Address.get_random)[
|
||||
'current_actor'
|
||||
] is Any
|
||||
assert _addr.__annotations__['_address_types'].startswith('dict')
|
||||
|
|
@ -2,9 +2,11 @@
|
|||
`tractor.log`-wrapping unit tests.
|
||||
|
||||
'''
|
||||
import importlib
|
||||
import logging
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import sys
|
||||
from types import ModuleType
|
||||
|
||||
import pytest
|
||||
|
|
@ -165,6 +167,53 @@ def test_implicit_mod_name_applied_for_child(
|
|||
assert submod.log.logger in sub_logs
|
||||
|
||||
|
||||
def test_implicit_mod_name_from_unregistered_namespace(
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Preserve implicit logger naming for dynamic module namespaces.
|
||||
|
||||
The fast `sys.modules` caller lookup cannot resolve `runpy`,
|
||||
plugin-loader, or `exec()` namespaces that are not registered.
|
||||
Compile a real package file under an unregistered module name so
|
||||
the rare filename fallback must recover its imported package and
|
||||
retain the same package-level logger name.
|
||||
|
||||
'''
|
||||
pkg_name = 'dynamic_logger_pkg'
|
||||
pkg_dir = tmp_path / pkg_name
|
||||
pkg_dir.mkdir()
|
||||
init_path = pkg_dir / '__init__.py'
|
||||
init_path.write_text('')
|
||||
mod_path = pkg_dir / 'plugin.py'
|
||||
mod_path.write_text('')
|
||||
|
||||
sys.path.insert(0, str(tmp_path))
|
||||
try:
|
||||
importlib.import_module(pkg_name)
|
||||
namespace = {
|
||||
'__name__': f'{pkg_name}.unregistered',
|
||||
'__package__': pkg_name,
|
||||
'tractor': tractor,
|
||||
}
|
||||
exec(
|
||||
compile(
|
||||
'log = tractor.log.get_logger('
|
||||
f'pkg_name={pkg_name!r})',
|
||||
str(mod_path),
|
||||
'exec',
|
||||
),
|
||||
namespace,
|
||||
)
|
||||
dynamic_log = namespace.get('log')
|
||||
finally:
|
||||
sys.path.remove(str(tmp_path))
|
||||
sys.modules.pop(pkg_name, None)
|
||||
|
||||
assert dynamic_log is not None
|
||||
assert dynamic_log.name == pkg_name
|
||||
|
||||
|
||||
def test_io_custom_level_registered():
|
||||
'''
|
||||
The `IO`(21) level (registered via `add_log_level()` at
|
||||
|
|
|
|||
|
|
@ -139,9 +139,9 @@ async def test_required_args(callwith_expecterror):
|
|||
with pytest.raises(err):
|
||||
await func(**kwargs)
|
||||
else:
|
||||
async with tractor.open_nursery() as n:
|
||||
async with tractor.open_nursery() as an:
|
||||
|
||||
portal = await n.start_actor(
|
||||
portal = await an.start_actor(
|
||||
name='pubber',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
@ -176,33 +176,55 @@ def test_multi_actor_subs_arbiter_pub(
|
|||
|
||||
async def main():
|
||||
|
||||
async with tractor.open_nursery(
|
||||
async with (
|
||||
tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
enable_modules=[__name__],
|
||||
) as n:
|
||||
) as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
|
||||
name = 'root'
|
||||
|
||||
if pub_actor == 'streamer':
|
||||
# start the publisher as a daemon
|
||||
master_portal = await n.start_actor(
|
||||
master_portal = await an.start_actor(
|
||||
'streamer',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
name = 'streamer'
|
||||
|
||||
even_portal = await n.run_in_actor(
|
||||
# spawn the two subscriber actors as daemons and run
|
||||
# `subs()` on each as a background task (was the legacy
|
||||
# `run_in_actor()`); keep the portals for the explicit
|
||||
# `cancel_actor()` teardown below. Each runner swallows
|
||||
# the teardown error that `cancel_actor()` relays.
|
||||
async def _run_subs(
|
||||
portal: tractor.Portal,
|
||||
which: list[str],
|
||||
) -> None:
|
||||
try:
|
||||
await portal.run(
|
||||
subs,
|
||||
which=['even'],
|
||||
name='evens',
|
||||
pub_actor_name=name
|
||||
which=which,
|
||||
pub_actor_name=name,
|
||||
)
|
||||
odd_portal = await n.run_in_actor(
|
||||
subs,
|
||||
which=['odd'],
|
||||
name='odds',
|
||||
pub_actor_name=name
|
||||
except (
|
||||
tractor.RemoteActorError,
|
||||
tractor.ContextCancelled,
|
||||
):
|
||||
pass # expected once we `cancel_actor()` below
|
||||
|
||||
even_portal = await an.start_actor(
|
||||
'evens',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
odd_portal = await an.start_actor(
|
||||
'odds',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
tn.start_soon(_run_subs, even_portal, ['even'])
|
||||
tn.start_soon(_run_subs, odd_portal, ['odd'])
|
||||
|
||||
async with tractor.wait_for_actor('evens'):
|
||||
# block until 2nd actor is initialized
|
||||
|
|
@ -257,6 +279,9 @@ def test_multi_actor_subs_arbiter_pub(
|
|||
else:
|
||||
await master_portal.cancel_actor()
|
||||
|
||||
# drop the bg `subs()` runners now the subs are cancelled
|
||||
tn.cancel_scope.cancel()
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
||||
|
|
@ -269,9 +294,9 @@ def test_single_subactor_pub_multitask_subs(
|
|||
async with tractor.open_nursery(
|
||||
registry_addrs=[reg_addr],
|
||||
enable_modules=[__name__],
|
||||
) as n:
|
||||
) as an:
|
||||
|
||||
portal = await n.start_actor(
|
||||
portal = await an.start_actor(
|
||||
'streamer',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -1,10 +1,22 @@
|
|||
import time
|
||||
import platform
|
||||
|
||||
import trio
|
||||
import pytest
|
||||
|
||||
import tractor
|
||||
|
||||
# `tractor.ipc._ringbuf` is built on linux `eventfd(2)`; importing
|
||||
# it pulls in `tractor.ipc._linux` whose module-level
|
||||
# `ffi.dlopen(None)` raises on non-linux. Skip the whole module at
|
||||
# COLLECTION before that crashing import runs (a `pytestmark` skip
|
||||
# is too late — markers apply only after the import succeeds).
|
||||
if platform.system() != 'Linux':
|
||||
pytest.skip(
|
||||
'ringbuf (eventfd) IPC is linux-only',
|
||||
allow_module_level=True,
|
||||
)
|
||||
|
||||
# XXX `cffi` dun build on py3.14 yet..
|
||||
pytest.importorskip("cffi")
|
||||
|
||||
|
|
@ -131,9 +143,12 @@ def test_ringbuf(
|
|||
child_read_shm,
|
||||
**common_kwargs,
|
||||
total_bytes=total_bytes,
|
||||
) as (sctx, _sent),
|
||||
) as (rctx, _sent),
|
||||
):
|
||||
await recv_p.result()
|
||||
# ctx-acm exits await each child task's
|
||||
# `Return` (the prior `recv_p.result()` here
|
||||
# was a daemon-portal no-op).
|
||||
pass
|
||||
|
||||
await send_p.cancel_actor()
|
||||
await recv_p.cancel_actor()
|
||||
|
|
|
|||
|
|
@ -9,6 +9,17 @@ from functools import partial
|
|||
import pytest
|
||||
import trio
|
||||
import tractor
|
||||
|
||||
# `infect_asyncio` mode is unsupported on Windows (see
|
||||
# `test_infected_asyncio`); skip at COLLECTION before the
|
||||
# asyncio-interop imports below so the CI leg completes.
|
||||
import platform
|
||||
if platform.system() == 'Windows':
|
||||
pytest.skip(
|
||||
'infect_asyncio mode is unsupported on Windows',
|
||||
allow_module_level=True,
|
||||
)
|
||||
|
||||
from tractor import (
|
||||
to_asyncio,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -4,11 +4,18 @@ related API and error checks.
|
|||
|
||||
'''
|
||||
import itertools
|
||||
from unittest.mock import (
|
||||
AsyncMock,
|
||||
Mock,
|
||||
)
|
||||
|
||||
import pytest
|
||||
import tractor
|
||||
import trio
|
||||
|
||||
from tractor._exceptions import TransportClosed
|
||||
from tractor.runtime import _rpc
|
||||
|
||||
|
||||
async def sleep_back_actor(
|
||||
actor_name,
|
||||
|
|
@ -46,6 +53,126 @@ async def short_sleep():
|
|||
await trio.sleep(0)
|
||||
|
||||
|
||||
def test_rpc_runs_after_startack_disconnect():
|
||||
'''
|
||||
Complete an accepted RPC when its caller closes before `StartAck`.
|
||||
|
||||
Registrar teardown opens short-lived `unregister_actor` RPCs. A
|
||||
loaded caller can close its channel while the registrar sends the
|
||||
acknowledgement; normalized `TransportClosed` previously escaped
|
||||
into the shared service nursery before the already-created
|
||||
coroutine was awaited. This fake fails the first response send and
|
||||
proves the RPC side effect still runs with no later send attempt.
|
||||
|
||||
'''
|
||||
async def main():
|
||||
rpc_ran = trio.Event()
|
||||
|
||||
chan = Mock()
|
||||
chan.send = AsyncMock(
|
||||
side_effect=TransportClosed(
|
||||
message='caller closed before StartAck',
|
||||
),
|
||||
)
|
||||
chan.connected.return_value = False
|
||||
ctx = Mock(
|
||||
chan=chan,
|
||||
cid='rpc-cid',
|
||||
_scope=None,
|
||||
_task='rpc-task',
|
||||
)
|
||||
actor = Mock()
|
||||
actor.get_context.return_value = ctx
|
||||
actor._rpc_tasks = {}
|
||||
actor._ongoing_rpc_tasks = trio.Event()
|
||||
actor._ongoing_rpc_tasks.set()
|
||||
|
||||
async def rpc_func():
|
||||
assert (chan, ctx.cid) in actor._rpc_tasks
|
||||
rpc_ran.set()
|
||||
|
||||
async def invoke(task_status):
|
||||
await _rpc._invoke(
|
||||
actor=actor,
|
||||
cid=ctx.cid,
|
||||
chan=chan,
|
||||
func=rpc_func,
|
||||
kwargs={},
|
||||
task_status=task_status,
|
||||
)
|
||||
|
||||
async with trio.open_nursery() as nursery:
|
||||
started_ctx = await nursery.start(invoke)
|
||||
assert started_ctx is ctx
|
||||
await rpc_ran.wait()
|
||||
|
||||
assert rpc_ran.is_set()
|
||||
assert not actor._rpc_tasks
|
||||
assert actor._ongoing_rpc_tasks.is_set()
|
||||
chan.send.assert_awaited_once()
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
||||
def test_error_shipment_ignores_closed_response_channel(monkeypatch):
|
||||
'''
|
||||
Preserve an application error when its response channel is closed.
|
||||
|
||||
A caller can disconnect after submitting an RPC but before its
|
||||
error response. Normalized `TransportClosed` from that final send
|
||||
is terminal response failure, not a new actor-wide service error.
|
||||
This test proves error shipment logs and returns without replacing
|
||||
the original application exception.
|
||||
|
||||
'''
|
||||
chan = Mock()
|
||||
chan.send = AsyncMock(
|
||||
side_effect=[
|
||||
None,
|
||||
TransportClosed(
|
||||
message='caller closed before Error response',
|
||||
),
|
||||
],
|
||||
)
|
||||
error_msg = Mock(boxed_type_str='ValueError')
|
||||
monkeypatch.setattr(
|
||||
_rpc,
|
||||
'pack_error',
|
||||
Mock(return_value=error_msg),
|
||||
)
|
||||
ctx = Mock(
|
||||
chan=chan,
|
||||
cid='rpc-cid',
|
||||
_scope=None,
|
||||
_task='rpc-task',
|
||||
)
|
||||
actor = Mock()
|
||||
actor.get_context.return_value = ctx
|
||||
actor._rpc_tasks = {}
|
||||
actor._ongoing_rpc_tasks = trio.Event()
|
||||
actor._ongoing_rpc_tasks.set()
|
||||
|
||||
async def failing_rpc():
|
||||
raise ValueError('application failure')
|
||||
|
||||
async def main():
|
||||
async with trio.open_nursery() as nursery:
|
||||
started_ctx = await nursery.start(
|
||||
_rpc._invoke,
|
||||
actor,
|
||||
ctx.cid,
|
||||
chan,
|
||||
failing_rpc,
|
||||
{},
|
||||
)
|
||||
assert started_ctx is ctx
|
||||
|
||||
trio.run(main)
|
||||
assert chan.send.await_count == 2
|
||||
assert not actor._rpc_tasks
|
||||
assert actor._ongoing_rpc_tasks.is_set()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'to_call', [
|
||||
([], 'short_sleep', tractor.RemoteActorError),
|
||||
|
|
@ -107,12 +234,13 @@ def test_rpc_errors(
|
|||
# do that if actually debugging subactor but keep it
|
||||
# disabled for the test.
|
||||
# debug_mode=True,
|
||||
) as n:
|
||||
) as an:
|
||||
|
||||
actor = tractor.current_actor()
|
||||
assert actor.is_registrar
|
||||
await n.run_in_actor(
|
||||
await tractor.to_actor.run(
|
||||
sleep_back_actor,
|
||||
an=an,
|
||||
actor_name=subactor_requests_to,
|
||||
|
||||
name='subactor',
|
||||
|
|
|
|||
|
|
@ -73,18 +73,22 @@ async def test_lifetime_stack_wipes_tmpfile(
|
|||
1.6 if error_in_child
|
||||
else 1
|
||||
)
|
||||
# scale for slow/noisy CI (esp. macOS) so the child error
|
||||
# propagates before the deadline; otherwise `move_on_after`
|
||||
# cancels first and flips the `error_in_child=True` assert.
|
||||
from .conftest import cpu_perf_headroom
|
||||
timeout *= cpu_perf_headroom()
|
||||
try:
|
||||
with trio.move_on_after(timeout) as cs:
|
||||
async with tractor.open_nursery(
|
||||
loglevel=loglevel,
|
||||
) as an:
|
||||
await ( # inlined `tractor.Portal`
|
||||
await an.run_in_actor(
|
||||
await tractor.to_actor.run(
|
||||
crash_and_clean_tmpdir,
|
||||
an=an,
|
||||
tmp_file_path=path,
|
||||
error=error_in_child,
|
||||
)
|
||||
).result()
|
||||
except (
|
||||
tractor.RemoteActorError,
|
||||
BaseExceptionGroup,
|
||||
|
|
|
|||
|
|
@ -120,6 +120,13 @@ async def child_read_shm_list(
|
|||
print(f'(child): reading frame: {frame}')
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
platform.system() == 'Windows',
|
||||
reason=(
|
||||
'parent/child shm IPC deadlocks on Windows '
|
||||
'(frame-size dependent hang); nascent — see #404'
|
||||
),
|
||||
)
|
||||
@pytest.mark.parametrize(
|
||||
'use_str',
|
||||
[False, True],
|
||||
|
|
|
|||
|
|
@ -48,9 +48,11 @@ async def spawn(
|
|||
actor: tractor.Actor = tractor.current_actor()
|
||||
assert actor.is_registrar == should_be_root
|
||||
|
||||
# spawns subproc here
|
||||
portal: tractor.Portal = await an.run_in_actor(
|
||||
fn=spawn,
|
||||
# recursively spawn this same `spawn()` fn as the lone
|
||||
# task of a one-shot child subactor and get its result.
|
||||
result = await tractor.to_actor.run(
|
||||
spawn,
|
||||
an=an,
|
||||
|
||||
# spawning args
|
||||
name='sub-actor',
|
||||
|
|
@ -62,16 +64,6 @@ async def spawn(
|
|||
data=data_to_pass_down,
|
||||
reg_addr=reg_addr,
|
||||
)
|
||||
|
||||
assert len(an._children) == 1
|
||||
assert (
|
||||
portal.channel.aid.uid
|
||||
in
|
||||
tractor.current_actor().ipc_server._peers
|
||||
)
|
||||
|
||||
# get result from child subactor
|
||||
result = await portal.result()
|
||||
assert result == 10
|
||||
return result
|
||||
else:
|
||||
|
|
@ -79,7 +71,7 @@ async def spawn(
|
|||
return 10
|
||||
|
||||
|
||||
def test_run_in_actor_same_func_in_child(
|
||||
def test_to_actor_run_same_func_in_child(
|
||||
reg_addr: tuple,
|
||||
debug_mode: bool,
|
||||
):
|
||||
|
|
@ -159,21 +151,16 @@ async def test_most_beautiful_word(
|
|||
async with tractor.open_nursery(
|
||||
debug_mode=debug_mode,
|
||||
) as an:
|
||||
portal = await an.run_in_actor(
|
||||
res: Any = await tractor.to_actor.run(
|
||||
cellar_door,
|
||||
an=an,
|
||||
return_value=return_value,
|
||||
name='some_linguist',
|
||||
)
|
||||
|
||||
res: Any = await portal.wait_for_result()
|
||||
assert res == return_value
|
||||
# The ``async with`` will unblock here since the 'some_linguist'
|
||||
# actor has completed its main task ``cellar_door``.
|
||||
|
||||
# this should pull the cached final result already captured during
|
||||
# the nursery block exit.
|
||||
res: Any = await portal.wait_for_result()
|
||||
assert res == return_value
|
||||
# The ``async with`` unblocks here — the 'some_linguist'
|
||||
# one-shot actor completed its lone task ``cellar_door`` and
|
||||
# was reaped by `to_actor.run()`.
|
||||
print(res)
|
||||
|
||||
|
||||
|
|
@ -215,9 +202,10 @@ def test_loglevel_propagated_to_subactor(
|
|||
start_method=start_method,
|
||||
registry_addrs=[reg_addr],
|
||||
|
||||
) as tn:
|
||||
await tn.run_in_actor(
|
||||
) as an:
|
||||
await tractor.to_actor.run(
|
||||
check_loglevel,
|
||||
an=an,
|
||||
loglevel=level,
|
||||
level=level,
|
||||
)
|
||||
|
|
@ -267,11 +255,11 @@ async def check_parent_main_inheritance(
|
|||
return has_data
|
||||
|
||||
|
||||
def test_run_in_actor_can_skip_parent_main_inheritance(
|
||||
def test_to_actor_run_can_skip_parent_main_inheritance(
|
||||
start_method: str, # <- only support on `trio` backend rn.
|
||||
):
|
||||
'''
|
||||
Verify ``inherit_parent_main=False`` on ``run_in_actor()``
|
||||
Verify ``inherit_parent_main=False`` on ``to_actor.run()``
|
||||
prevents parent ``__main__`` data from reaching the child.
|
||||
|
||||
'''
|
||||
|
|
@ -284,21 +272,21 @@ def test_run_in_actor_can_skip_parent_main_inheritance(
|
|||
async with tractor.open_nursery(start_method='trio') as an:
|
||||
|
||||
# Default: child receives parent __main__ bootstrap data
|
||||
replaying = await an.run_in_actor(
|
||||
await tractor.to_actor.run(
|
||||
check_parent_main_inheritance,
|
||||
an=an,
|
||||
name='replaying-parent-main',
|
||||
expect_inherited=True,
|
||||
)
|
||||
await replaying.result()
|
||||
|
||||
# Opt-out: child gets no parent __main__ data
|
||||
isolated = await an.run_in_actor(
|
||||
await tractor.to_actor.run(
|
||||
check_parent_main_inheritance,
|
||||
an=an,
|
||||
name='isolated-parent-main',
|
||||
inherit_parent_main=False,
|
||||
expect_inherited=False,
|
||||
)
|
||||
await isolated.result()
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,271 @@
|
|||
'''
|
||||
`tractor.to_actor`: one-shot single-remote-task API suite.
|
||||
|
||||
Verifies the "spiritual successor" to (and replacement of)
|
||||
the removed legacy `ActorNursery.run_in_actor()`; see
|
||||
https://github.com/goodboy/tractor/issues/477
|
||||
|
||||
'''
|
||||
from functools import partial
|
||||
|
||||
import pytest
|
||||
import trio
|
||||
import tractor
|
||||
from tractor import (
|
||||
RemoteActorError,
|
||||
to_actor,
|
||||
)
|
||||
from tractor._testing import tractor_test
|
||||
|
||||
|
||||
async def add_one(
|
||||
n: int,
|
||||
) -> int:
|
||||
return n + 1
|
||||
|
||||
|
||||
async def raise_value_error() -> None:
|
||||
raise ValueError('kaboom')
|
||||
|
||||
|
||||
@tractor_test
|
||||
async def test_one_shot_in_private_nursery(
|
||||
start_method: str,
|
||||
debug_mode: bool,
|
||||
):
|
||||
'''
|
||||
No `an`/`portal` provided: a private actor-nursery
|
||||
is opened (and torn down) scoped to just the call.
|
||||
|
||||
'''
|
||||
assert await to_actor.run(
|
||||
add_one,
|
||||
n=1,
|
||||
) == 2
|
||||
|
||||
|
||||
def test_one_shot_boots_implicit_runtime(
|
||||
reg_addr: tuple,
|
||||
start_method: str,
|
||||
loglevel: str,
|
||||
):
|
||||
'''
|
||||
Outside any actor-runtime `to_actor.run()` boots one
|
||||
implicitly (just like bare `open_nursery()` usage)
|
||||
configured via pass-through `runtime_kwargs`.
|
||||
|
||||
'''
|
||||
async def main() -> None:
|
||||
assert tractor.current_actor(
|
||||
err_on_no_runtime=False,
|
||||
) is None
|
||||
result = await to_actor.run(
|
||||
add_one,
|
||||
n=41,
|
||||
runtime_kwargs=dict(
|
||||
registry_addrs=[reg_addr],
|
||||
start_method=start_method,
|
||||
loglevel=loglevel,
|
||||
),
|
||||
)
|
||||
assert result == 42
|
||||
|
||||
trio.run(main)
|
||||
|
||||
|
||||
@tractor_test
|
||||
async def test_remote_error_relayed_to_caller_task(
|
||||
start_method: str,
|
||||
debug_mode: bool,
|
||||
):
|
||||
'''
|
||||
A remote task error is raised directly in the
|
||||
caller's task as a boxed `RemoteActorError` instead
|
||||
of surfacing at actor-nursery teardown as with the
|
||||
removed legacy `.run_in_actor()` API.
|
||||
|
||||
'''
|
||||
with pytest.raises(RemoteActorError) as excinfo:
|
||||
await to_actor.run(raise_value_error)
|
||||
|
||||
assert excinfo.value.boxed_type is ValueError
|
||||
|
||||
|
||||
@tractor_test
|
||||
async def test_spawn_from_caller_nursery(
|
||||
start_method: str,
|
||||
debug_mode: bool,
|
||||
):
|
||||
'''
|
||||
Pass a caller-managed `an: ActorNursery` for the
|
||||
spawn; the subactor is still one-shot reaped by the
|
||||
time the call returns.
|
||||
|
||||
'''
|
||||
async with tractor.open_nursery() as an:
|
||||
assert await to_actor.run(
|
||||
add_one,
|
||||
an=an,
|
||||
n=10,
|
||||
) == 11
|
||||
|
||||
|
||||
@tractor_test
|
||||
async def test_remote_error_from_caller_nursery(
|
||||
start_method: str,
|
||||
debug_mode: bool,
|
||||
):
|
||||
'''
|
||||
With a caller-managed `an` the remote error also
|
||||
surfaces in the caller's task, INSIDE the nursery
|
||||
block, allowing inline (supervision-style) handling.
|
||||
|
||||
'''
|
||||
async with tractor.open_nursery() as an:
|
||||
with pytest.raises(RemoteActorError) as excinfo:
|
||||
await to_actor.run(
|
||||
raise_value_error,
|
||||
an=an,
|
||||
)
|
||||
|
||||
assert excinfo.value.boxed_type is ValueError
|
||||
|
||||
|
||||
@tractor_test
|
||||
async def test_reuse_existing_actor_via_portal(
|
||||
start_method: str,
|
||||
debug_mode: bool,
|
||||
):
|
||||
'''
|
||||
Pass `portal=` to schedule the one-shot task in an
|
||||
already-running actor; no spawn, no implicit reap.
|
||||
|
||||
'''
|
||||
async with tractor.open_nursery() as an:
|
||||
portal: tractor.Portal = await an.start_actor(
|
||||
'one_shot_worker',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
for i in range(3):
|
||||
assert await to_actor.run(
|
||||
add_one,
|
||||
portal=portal,
|
||||
n=i,
|
||||
) == i + 1
|
||||
|
||||
# still alive: caller owns the actor's lifetime.
|
||||
await portal.cancel_actor()
|
||||
|
||||
|
||||
@tractor_test
|
||||
async def test_concurrent_one_shots_from_task_nursery(
|
||||
start_method: str,
|
||||
debug_mode: bool,
|
||||
):
|
||||
'''
|
||||
The worker-pool-ish pattern from #477: concurrency
|
||||
is composed with a plain (caller-side) `trio` task
|
||||
nursery scheduling multiple one-shot calls against
|
||||
a shared caller-managed actor-nursery; error
|
||||
collection thus lives entirely in caller-code.
|
||||
|
||||
'''
|
||||
results: dict[int, int] = {}
|
||||
|
||||
async def one_shot(
|
||||
an: tractor.ActorNursery,
|
||||
i: int,
|
||||
) -> None:
|
||||
results[i] = await to_actor.run(
|
||||
add_one,
|
||||
an=an,
|
||||
name=f'one_shot_{i}',
|
||||
n=i,
|
||||
)
|
||||
|
||||
async with (
|
||||
tractor.open_nursery() as an,
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
for i in range(4):
|
||||
tn.start_soon(one_shot, an, i)
|
||||
|
||||
assert results == {
|
||||
i: i + 1 for i in range(4)
|
||||
}
|
||||
|
||||
|
||||
def test_rejects_sync_fn():
|
||||
'''
|
||||
Non-async callables error BEFORE any spawn (or even
|
||||
runtime-boot) happens.
|
||||
|
||||
'''
|
||||
def not_async() -> None:
|
||||
...
|
||||
|
||||
with pytest.raises(TypeError):
|
||||
trio.run(
|
||||
partial(
|
||||
to_actor.run,
|
||||
not_async,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def test_rejects_streaming_fn():
|
||||
'''
|
||||
Async-gen (streaming) fns are not one-shot-able,
|
||||
same constraint as `Portal.run()`.
|
||||
|
||||
'''
|
||||
async def agen():
|
||||
yield 1
|
||||
|
||||
with pytest.raises(TypeError):
|
||||
trio.run(
|
||||
partial(
|
||||
to_actor.run,
|
||||
agen,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def test_rejects_portal_and_an_combo():
|
||||
'''
|
||||
`portal=` and `an=` are mutually exclusive
|
||||
placement options.
|
||||
|
||||
'''
|
||||
with pytest.raises(ValueError):
|
||||
trio.run(
|
||||
partial(
|
||||
to_actor.run,
|
||||
add_one,
|
||||
portal=object(),
|
||||
an=object(),
|
||||
n=1,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def test_rejects_runtime_kwargs_with_placement():
|
||||
'''
|
||||
`runtime_kwargs` only applies when the call opens
|
||||
its own private actor-nursery; passing it alongside
|
||||
a placement opt is an error, never silently
|
||||
ignored.
|
||||
|
||||
'''
|
||||
with pytest.raises(ValueError):
|
||||
trio.run(
|
||||
partial(
|
||||
to_actor.run,
|
||||
add_one,
|
||||
an=object(),
|
||||
runtime_kwargs=dict(
|
||||
loglevel='cancel',
|
||||
),
|
||||
n=1,
|
||||
)
|
||||
)
|
||||
|
|
@ -62,6 +62,7 @@ from .devx import (
|
|||
post_mortem as post_mortem,
|
||||
)
|
||||
from . import msg as msg
|
||||
from . import to_actor as to_actor
|
||||
from ._root import (
|
||||
run_daemon as run_daemon,
|
||||
open_root_actor as open_root_actor,
|
||||
|
|
@ -74,3 +75,39 @@ from .discovery._registry import (
|
|||
Arbiter as Arbiter,
|
||||
)
|
||||
# from . import hilevel as hilevel
|
||||
|
||||
|
||||
__all__: tuple[str, ...] = tuple(
|
||||
name
|
||||
for name in globals()
|
||||
if not name.startswith('_')
|
||||
) + (
|
||||
'to_asyncio',
|
||||
)
|
||||
|
||||
|
||||
def __dir__() -> list[str]:
|
||||
return sorted(set(globals()) | set(__all__))
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
'''
|
||||
PEP 562 lazy sub-module loading, presently only for
|
||||
`.to_asyncio` which (transitively) imports `asyncio`
|
||||
itself: a non-trivial multi-ms chunk of the eager
|
||||
`import tractor` cost (gh #470) unneeded by
|
||||
`trio`-only apps.
|
||||
|
||||
Any `tractor.to_asyncio.<attr>` access (or a
|
||||
`from tractor import to_asyncio`) still works, the
|
||||
sub-mod is simply imported on first-access instead
|
||||
of at pkg-import time.
|
||||
|
||||
'''
|
||||
if name == 'to_asyncio':
|
||||
from importlib import import_module
|
||||
return import_module('.to_asyncio', __name__)
|
||||
|
||||
raise AttributeError(
|
||||
f'module {__name__!r} has no attribute {name!r}'
|
||||
)
|
||||
|
|
|
|||
|
|
@ -780,7 +780,7 @@ class Context:
|
|||
# `Portal.open_context()` has been opened since it's
|
||||
# assumed that other portal APIs like,
|
||||
# - `Portal.run()`,
|
||||
# - `ActorNursery.run_in_actor()`
|
||||
# - `to_actor.run()`
|
||||
# do their own error checking at their own call points and
|
||||
# result processing.
|
||||
|
||||
|
|
|
|||
|
|
@ -1161,10 +1161,6 @@ class TransportClosed(Exception):
|
|||
)
|
||||
|
||||
|
||||
class NoResult(RuntimeError):
|
||||
"No final result is expected for this actor"
|
||||
|
||||
|
||||
class ModuleNotExposed(ModuleNotFoundError):
|
||||
"The requested module is not exposed for RPC"
|
||||
|
||||
|
|
|
|||
|
|
@ -47,9 +47,7 @@ from .devx import (
|
|||
from .spawn import _spawn
|
||||
from .runtime import _state
|
||||
from . import log
|
||||
from .ipc import (
|
||||
_connect_chan,
|
||||
)
|
||||
from .discovery._api import _probe_registry_addrs
|
||||
from .discovery._addr import (
|
||||
Address,
|
||||
UnwrappedAddress,
|
||||
|
|
@ -451,49 +449,22 @@ async def open_root_actor(
|
|||
from .devx._stackscope import enable_stack_on_sig
|
||||
enable_stack_on_sig()
|
||||
|
||||
# closed into below ping task-func
|
||||
ponged_addrs: list[Address] = []
|
||||
ponged_addrs: list[Address]
|
||||
occupied_addrs: list[Address]
|
||||
(
|
||||
ponged_addrs,
|
||||
occupied_addrs,
|
||||
) = await _probe_registry_addrs(uw_reg_addrs)
|
||||
|
||||
async def ping_tpt_socket(
|
||||
addr: Address,
|
||||
timeout: float = 1,
|
||||
) -> None:
|
||||
'''
|
||||
Attempt temporary connection to see if a registry is
|
||||
listening at the requested address by a tranport layer
|
||||
ping.
|
||||
|
||||
If a connection can't be made quickly we assume none no
|
||||
server is listening at that addr.
|
||||
|
||||
'''
|
||||
try:
|
||||
# TODO: this connect-and-bail forces us to have to
|
||||
# carefully rewrap TCP 104-connection-reset errors as
|
||||
# EOF so as to avoid propagating cancel-causing errors
|
||||
# to the channel-msg loop machinery. Likely it would
|
||||
# be better to eventually have a "discovery" protocol
|
||||
# with basic handshake instead?
|
||||
with trio.move_on_after(timeout):
|
||||
async with _connect_chan(addr.unwrap()):
|
||||
ponged_addrs.append(addr)
|
||||
|
||||
except OSError:
|
||||
# ?TODO, make this a "discovery" log level?
|
||||
logger.info(
|
||||
f'No root-actor registry found @ {addr!r}\n'
|
||||
)
|
||||
|
||||
# !TODO, this is basically just another (abstract)
|
||||
# happy-eyeballs, so we should try for formalize it somewhere
|
||||
# in a `.[_]discovery` ya?
|
||||
#
|
||||
async with trio.open_nursery() as tn:
|
||||
for uw_addr in uw_reg_addrs:
|
||||
addr: Address = wrap_address(uw_addr)
|
||||
tn.start_soon(
|
||||
ping_tpt_socket,
|
||||
addr,
|
||||
if (
|
||||
not ponged_addrs
|
||||
and
|
||||
occupied_addrs
|
||||
):
|
||||
raise RuntimeError(
|
||||
f'Registry address(es) are occupied but did not '
|
||||
f'answer as Tractor registrars!\n'
|
||||
f'occupied_addrs: {occupied_addrs!r}\n'
|
||||
)
|
||||
|
||||
if tpt_bind_addrs is None:
|
||||
|
|
|
|||
|
|
@ -111,14 +111,13 @@ SHM_DIR: str = '/dev/shm'
|
|||
|
||||
# UDS-socket leak sweep — see `find_orphaned_uds()` /
|
||||
# `reap_uds()` below. Tractor's UDS transport
|
||||
# (`tractor.ipc._uds`) creates sock files under
|
||||
# `${XDG_RUNTIME_DIR}/tractor/<name>@<pid>.sock`; a
|
||||
# (`tractor.ipc._uds`) creates sock files in its platform-specific
|
||||
# default bindspace; a
|
||||
# crash / SIGKILL / mid-cancel teardown can leave the
|
||||
# file behind because `os.unlink()` lives in the
|
||||
# `_serve_ipc_eps` `finally:` block which doesn't always
|
||||
# get to run on hard exits. The reaper here is best-effort
|
||||
# cleanup for the test harness + the `tractor-reap` CLI.
|
||||
_UDS_SUBDIR: str = 'tractor'
|
||||
# `<actor-name>@<pid>.sock` — pid is the binder's pid at
|
||||
# creation time. Special sentinel: `registry@1616.sock`
|
||||
# uses the magic `1616` not a real pid (the root
|
||||
|
|
@ -738,19 +737,16 @@ def reap_shm(
|
|||
|
||||
def get_uds_dir() -> str|None:
|
||||
'''
|
||||
Path of tractor's per-user UDS sock-file dir
|
||||
(`${XDG_RUNTIME_DIR}/tractor/`).
|
||||
Path of Tractor's platform-specific default UDS bindspace.
|
||||
|
||||
Returns `None` when `XDG_RUNTIME_DIR` is unset (e.g.
|
||||
non-systemd hosts, or inside a container without the
|
||||
var plumbed through). Caller should treat that as
|
||||
"no UDS leaks possible to detect — skip".
|
||||
Returns `None` only when the bindspace cannot be resolved.
|
||||
|
||||
'''
|
||||
xdg: str|None = os.environ.get('XDG_RUNTIME_DIR')
|
||||
if not xdg:
|
||||
try:
|
||||
from tractor.ipc._uds import UDSAddress
|
||||
return str(UDSAddress.def_bindspace)
|
||||
except Exception:
|
||||
return None
|
||||
return os.path.join(xdg, _UDS_SUBDIR)
|
||||
|
||||
|
||||
def _parse_uds_name(filename: str) -> tuple[str, int]|None:
|
||||
|
|
@ -768,16 +764,16 @@ def _parse_uds_name(filename: str) -> tuple[str, int]|None:
|
|||
def find_orphaned_uds(
|
||||
*,
|
||||
uds_dir: str|None = None,
|
||||
include_registry_sentinel: bool = False,
|
||||
) -> list[str]:
|
||||
'''
|
||||
`<uds_dir>/*.sock` paths whose binder pid is no
|
||||
longer alive (orphaned). Includes the
|
||||
`registry@1616.sock` sentinel — `1616` is a magic
|
||||
sentinel pid (not a real one) so the file's
|
||||
presence alone signals a leak from a dead session.
|
||||
longer alive (orphaned). Explicit callers may include the
|
||||
`registry@1616.sock` sentinel; automatic pytest cleanup excludes
|
||||
it because binder liveness cannot be inferred from magic `1616`.
|
||||
|
||||
Returns `[]` on platforms without `XDG_RUNTIME_DIR`
|
||||
or when the dir doesn't exist. Files whose name
|
||||
Returns `[]` when the platform bindspace cannot be resolved or the
|
||||
dir doesn't exist. Files whose name
|
||||
doesn't match the `<name>@<pid>.sock` pattern are
|
||||
skipped (we don't unlink things we don't recognize).
|
||||
|
||||
|
|
@ -811,9 +807,7 @@ def find_orphaned_uds(
|
|||
continue
|
||||
_name, pid = parsed
|
||||
if pid == _UDS_REGISTRY_SENTINEL_PID:
|
||||
# sentinel — never a real pid; if the file
|
||||
# exists nobody live is "owning" it via
|
||||
# /proc lookup, so always orphaned
|
||||
if include_registry_sentinel:
|
||||
leaked.append(path)
|
||||
continue
|
||||
if not _is_alive(pid):
|
||||
|
|
@ -933,8 +927,8 @@ def track_orphaned_uds_per_test():
|
|||
teardown that flakifies sibling tests via
|
||||
sock-file rebind races).
|
||||
|
||||
Snapshots `${XDG_RUNTIME_DIR}/tractor/` before and
|
||||
after each test; any `<name>@<pid>.sock` files
|
||||
Snapshots Tractor's platform-specific default UDS bindspace before
|
||||
and after each test; any `<name>@<pid>.sock` files
|
||||
created during the test that survive teardown AND
|
||||
whose creator pid is dead are surfaced as a loud
|
||||
warning AND reaped, so the next test starts with a
|
||||
|
|
@ -950,8 +944,8 @@ def track_orphaned_uds_per_test():
|
|||
it (vs. blanket session-end sweep) makes blame
|
||||
obvious + prevents cascade flakiness.
|
||||
|
||||
Cheap: 2x `os.listdir` + a few `os.stat`s per test.
|
||||
Skips silently when `XDG_RUNTIME_DIR` isn't set.
|
||||
Cheap: 2x `os.listdir` + a few `os.stat`s per test. Skips silently
|
||||
when the platform bindspace cannot be resolved.
|
||||
|
||||
'''
|
||||
uds_dir: str|None = get_uds_dir()
|
||||
|
|
|
|||
|
|
@ -39,14 +39,15 @@ from typing import (
|
|||
Type,
|
||||
)
|
||||
|
||||
import pdbp
|
||||
# NOTE, `pdbp` + `wrapt` are lazy-imported at their
|
||||
# single use-sites below to keep them off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
from tractor.log import get_logger
|
||||
import trio
|
||||
from tractor.msg import (
|
||||
pretty_struct,
|
||||
NamespacePath,
|
||||
)
|
||||
import wrapt
|
||||
|
||||
|
||||
log = get_logger()
|
||||
|
|
@ -257,6 +258,7 @@ def api_frame(
|
|||
caller_frames_up: int = 1,
|
||||
|
||||
) -> Callable:
|
||||
import wrapt
|
||||
|
||||
# handle the decorator called WITHOUT () case,
|
||||
# i.e. just @api_frame, NOT @api_frame(extra=<blah>)
|
||||
|
|
@ -320,6 +322,8 @@ def hide_runtime_frames() -> dict[FunctionType, CodeType]:
|
|||
as possible, particularly from inside a `PdbREPL`.
|
||||
|
||||
'''
|
||||
import pdbp
|
||||
|
||||
# XXX HACKZONE XXX
|
||||
# hide exit stack frames on nurseries and cancel-scopes!
|
||||
# |_ so avoid seeing it when the `pdbp` REPL is first engaged from
|
||||
|
|
|
|||
|
|
@ -31,12 +31,21 @@ from threading import (
|
|||
RLock,
|
||||
)
|
||||
import multiprocessing as mp
|
||||
|
||||
import platform
|
||||
|
||||
from signal import (
|
||||
signal,
|
||||
getsignal,
|
||||
SIGUSR1,
|
||||
SIGINT,
|
||||
)
|
||||
|
||||
|
||||
if platform.system() != "Windows":
|
||||
from signal import SIGUSR1
|
||||
else:
|
||||
SIGUSR1 = None
|
||||
|
||||
# import traceback
|
||||
from types import ModuleType
|
||||
from typing import (
|
||||
|
|
@ -347,8 +356,8 @@ def dump_tree_on_sig(
|
|||
|
||||
|
||||
def enable_stack_on_sig(
|
||||
sig: int = SIGUSR1,
|
||||
) -> ModuleType:
|
||||
sig: int|None = SIGUSR1,
|
||||
) -> ModuleType|None:
|
||||
'''
|
||||
Enable `stackscope` tracing on reception of a signal; by
|
||||
default this is SIGUSR1.
|
||||
|
|
@ -367,6 +376,16 @@ def enable_stack_on_sig(
|
|||
>> pkill --signal SIGUSR1 -f <part-of-cmd: str>
|
||||
|
||||
'''
|
||||
# no `SIGUSR1` on this platform (e.g. Windows) -> nothing to
|
||||
# wire up; degrade gracefully instead of crashing callers that
|
||||
# only guard against a missing `stackscope` (`ImportError`).
|
||||
if sig is None:
|
||||
log.warning(
|
||||
'No `SIGUSR1` on this platform;\n'
|
||||
'skipping `stackscope` trace-on-signal setup!\n'
|
||||
)
|
||||
return None
|
||||
|
||||
try:
|
||||
# NOTE, `stackscope._glue` does intentional async-gen type
|
||||
# introspection at import-time which trips
|
||||
|
|
|
|||
|
|
@ -24,7 +24,6 @@ mult-process support within a single actor tree.
|
|||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import asyncio
|
||||
import bdb
|
||||
from contextlib import (
|
||||
AbstractContextManager,
|
||||
|
|
@ -53,7 +52,6 @@ from trio import (
|
|||
)
|
||||
import tractor
|
||||
from tractor.log import get_logger
|
||||
from tractor.to_asyncio import run_trio_task_in_future
|
||||
from tractor._context import Context
|
||||
from tractor.runtime import _state
|
||||
from tractor._exceptions import (
|
||||
|
|
@ -85,6 +83,10 @@ from ..pformat import (
|
|||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# NOTE, `asyncio` (and `.to_asyncio`) are
|
||||
# lazy-imported at their use-sites to keep them off
|
||||
# the eager `import tractor` path (gh #470).
|
||||
import asyncio
|
||||
from trio.lowlevel import Task
|
||||
from threading import Thread
|
||||
from tractor.runtime._runtime import (
|
||||
|
|
@ -164,6 +166,7 @@ async def _pause(
|
|||
'An `asyncio` task should not be calling this!?'
|
||||
) from rte
|
||||
else:
|
||||
import asyncio
|
||||
task = asyncio.current_task()
|
||||
|
||||
if debug_func is not None:
|
||||
|
|
@ -946,6 +949,7 @@ def pause_from_sync(
|
|||
|
||||
asyncio_task: asyncio.Task|None = None
|
||||
if is_infected_aio:
|
||||
import asyncio
|
||||
asyncio_task = asyncio.current_task()
|
||||
|
||||
# TODO: we could also check for a non-`.to_thread` context
|
||||
|
|
@ -1059,6 +1063,9 @@ def pause_from_sync(
|
|||
greenback: ModuleType = maybe_import_greenback()
|
||||
|
||||
if greenback.has_portal():
|
||||
from tractor.to_asyncio import (
|
||||
run_trio_task_in_future,
|
||||
)
|
||||
DebugStatus.shield_sigint()
|
||||
fute: asyncio.Future = run_trio_task_in_future(
|
||||
partial(
|
||||
|
|
|
|||
|
|
@ -20,7 +20,6 @@ Root-actor TTY mutex-locking machinery.
|
|||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import asyncio
|
||||
from contextlib import (
|
||||
AbstractContextManager,
|
||||
asynccontextmanager as acm,
|
||||
|
|
@ -52,7 +51,6 @@ from trio import (
|
|||
TaskStatus,
|
||||
)
|
||||
import tractor
|
||||
from tractor.to_asyncio import run_trio_task_in_future
|
||||
from tractor.log import get_logger
|
||||
from tractor._context import Context
|
||||
from tractor.runtime import _state
|
||||
|
|
@ -66,6 +64,10 @@ from tractor.runtime._state import (
|
|||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# NOTE, `asyncio` (and `.to_asyncio`) are
|
||||
# lazy-imported at their use-sites to keep them off
|
||||
# the eager `import tractor` path (gh #470).
|
||||
import asyncio
|
||||
from trio.lowlevel import Task
|
||||
from threading import Thread
|
||||
from tractor.ipc import (
|
||||
|
|
@ -910,6 +912,9 @@ class DebugStatus:
|
|||
async def _set_repl_release():
|
||||
repl_release.set()
|
||||
|
||||
from tractor.to_asyncio import (
|
||||
run_trio_task_in_future,
|
||||
)
|
||||
fute: asyncio.Future = run_trio_task_in_future(
|
||||
_set_repl_release
|
||||
)
|
||||
|
|
|
|||
|
|
@ -16,13 +16,13 @@
|
|||
from __future__ import annotations
|
||||
from uuid import uuid4
|
||||
from typing import (
|
||||
Any,
|
||||
Protocol,
|
||||
ClassVar,
|
||||
Type,
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
from bidict import bidict
|
||||
from trio import (
|
||||
SocketListener,
|
||||
)
|
||||
|
|
@ -32,14 +32,20 @@ from ..runtime._state import (
|
|||
_def_tpt_proto,
|
||||
)
|
||||
from ..ipc._tcp import TCPAddress
|
||||
from ..ipc._uds import UDSAddress
|
||||
from ..ipc._uds import (
|
||||
UDSAddress,
|
||||
HAS_UDS,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ONLY type-annots, the eager import costs ~4.5ms
|
||||
# of `import tractor` wall-time (gh #470).
|
||||
from ..runtime._runtime import Actor
|
||||
else:
|
||||
Actor = Any
|
||||
|
||||
log = get_logger()
|
||||
|
||||
|
||||
# TODO, maybe breakout the netns key to a struct?
|
||||
# class NetNs(Struct)[str, int]:
|
||||
# ...
|
||||
|
|
@ -170,25 +176,36 @@ class Address(Protocol):
|
|||
...
|
||||
|
||||
|
||||
_address_types: bidict[str, Type[Address]] = {
|
||||
'tcp': TCPAddress,
|
||||
'uds': UDSAddress
|
||||
# the address types available on this host: TCP always, UDS only
|
||||
# where usable (`HAS_UDS`). Both registries derive from this single
|
||||
# list via each type's `proto_key`.
|
||||
_address_protos: list[Type[Address]] = [TCPAddress]
|
||||
if HAS_UDS:
|
||||
_address_protos.append(UDSAddress)
|
||||
|
||||
_address_types: dict[str, Type[Address]] = {
|
||||
cls.proto_key: cls
|
||||
for cls in _address_protos
|
||||
}
|
||||
|
||||
|
||||
# TODO! really these are discovery sys default addrs ONLY useful for
|
||||
# when none is provided to a root actor on first boot.
|
||||
_default_lo_addrs: dict[
|
||||
str,
|
||||
UnwrappedAddress
|
||||
] = {
|
||||
'tcp': TCPAddress.get_root().unwrap(),
|
||||
'uds': UDSAddress.get_root().unwrap(),
|
||||
_default_lo_addrs: dict[str, UnwrappedAddress] = {
|
||||
cls.proto_key: cls.get_root().unwrap()
|
||||
for cls in _address_protos
|
||||
}
|
||||
|
||||
|
||||
def get_address_cls(name: str) -> Type[Address]:
|
||||
try:
|
||||
return _address_types[name]
|
||||
except KeyError:
|
||||
raise NotImplementedError(
|
||||
f'No IPC transport backend for {name!r} on this '
|
||||
f'platform!\n'
|
||||
f'(available: {list(_address_types)})\n'
|
||||
)
|
||||
|
||||
|
||||
def is_wrapped_addr(addr: any) -> bool:
|
||||
|
|
@ -286,7 +303,14 @@ def default_lo_addrs(
|
|||
for an input transport key set.
|
||||
|
||||
'''
|
||||
return [
|
||||
_default_lo_addrs[transport]
|
||||
for transport in transports
|
||||
]
|
||||
lo_addrs: list[UnwrappedAddress] = []
|
||||
for transport in transports:
|
||||
try:
|
||||
lo_addrs.append(_default_lo_addrs[transport])
|
||||
except KeyError:
|
||||
raise NotImplementedError(
|
||||
f'No default loopback addr for transport '
|
||||
f'{transport!r} on this platform!\n'
|
||||
f'(available: {list(_default_lo_addrs)})\n'
|
||||
)
|
||||
return lo_addrs
|
||||
|
|
|
|||
|
|
@ -21,14 +21,18 @@ management of (service) actors.
|
|||
"""
|
||||
from __future__ import annotations
|
||||
import ipaddress
|
||||
import os
|
||||
import socket
|
||||
from typing import (
|
||||
AsyncGenerator,
|
||||
AsyncContextManager,
|
||||
Literal,
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
from contextlib import asynccontextmanager as acm
|
||||
|
||||
import trio
|
||||
|
||||
from tractor.log import get_logger
|
||||
from ..trionics import (
|
||||
gather_contexts,
|
||||
|
|
@ -40,6 +44,7 @@ from ..ipc._uds import UDSAddress
|
|||
from ._addr import (
|
||||
UnwrappedAddress,
|
||||
Address,
|
||||
mk_uuid,
|
||||
wrap_address,
|
||||
)
|
||||
from ..runtime._portal import (
|
||||
|
|
@ -52,6 +57,7 @@ from ..runtime._state import (
|
|||
_runtime_vars,
|
||||
_def_tpt_proto,
|
||||
)
|
||||
from ..msg.types import Aid
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..runtime._runtime import Actor
|
||||
|
|
@ -60,6 +66,115 @@ if TYPE_CHECKING:
|
|||
log = get_logger()
|
||||
|
||||
|
||||
async def _probe_registry(
|
||||
addr: Address,
|
||||
timeout: float = 3,
|
||||
attempt_timeout: float = 1,
|
||||
max_attempts: int = 3,
|
||||
retry_delay: float = .05,
|
||||
close_timeout: float = .2,
|
||||
) -> Literal[
|
||||
'absent',
|
||||
'occupied',
|
||||
'registrar',
|
||||
]:
|
||||
'''
|
||||
Confirm an address serves the Tractor actor handshake.
|
||||
|
||||
Connection and handshake work share `timeout`; each attempt gets
|
||||
`attempt_timeout`. Shielded cleanup may add up to `close_timeout`
|
||||
per attempted channel.
|
||||
|
||||
'''
|
||||
from .._exceptions import TransportClosed
|
||||
|
||||
connected_once: bool = False
|
||||
with trio.move_on_after(timeout):
|
||||
for attempt in range(max_attempts):
|
||||
try:
|
||||
with trio.move_on_after(attempt_timeout) as attempt_cs:
|
||||
async with _connect_chan(
|
||||
addr.unwrap(),
|
||||
close_timeout=close_timeout,
|
||||
) as chan:
|
||||
connected_once = True
|
||||
peer_aid: Aid = await chan._do_handshake(
|
||||
aid=Aid(
|
||||
name='registry-probe',
|
||||
uuid=mk_uuid(),
|
||||
pid=os.getpid(),
|
||||
is_probe=True,
|
||||
),
|
||||
timeout=attempt_timeout,
|
||||
)
|
||||
if peer_aid.is_registrar is not False:
|
||||
return 'registrar'
|
||||
return 'occupied'
|
||||
|
||||
if attempt_cs.cancelled_caught:
|
||||
if not connected_once:
|
||||
return 'absent'
|
||||
|
||||
except OSError:
|
||||
return (
|
||||
'occupied'
|
||||
if connected_once
|
||||
else 'absent'
|
||||
)
|
||||
except TransportClosed:
|
||||
pass
|
||||
|
||||
if attempt + 1 < max_attempts:
|
||||
await trio.sleep(retry_delay * (attempt + 1))
|
||||
|
||||
return 'occupied'
|
||||
|
||||
|
||||
async def _probe_registry_addrs(
|
||||
addrs: list[UnwrappedAddress],
|
||||
timeout: float = 3,
|
||||
) -> tuple[
|
||||
list[Address],
|
||||
list[Address],
|
||||
]:
|
||||
'''
|
||||
Concurrently classify candidate registrar addresses.
|
||||
|
||||
Return confirmed registrar addresses followed by addresses occupied
|
||||
by non-registrar or unresponsive Tractor peers.
|
||||
|
||||
'''
|
||||
registrar_addrs: list[Address] = []
|
||||
occupied_addrs: list[Address] = []
|
||||
|
||||
async def probe_addr(addr: Address) -> None:
|
||||
probe_status = await _probe_registry(
|
||||
addr=addr,
|
||||
timeout=timeout,
|
||||
)
|
||||
if probe_status == 'registrar':
|
||||
registrar_addrs.append(addr)
|
||||
elif probe_status == 'occupied':
|
||||
occupied_addrs.append(addr)
|
||||
else:
|
||||
# ?TODO, make this a "discovery" log level?
|
||||
log.info(
|
||||
f'No root-actor registry found @ {addr!r}\n'
|
||||
)
|
||||
|
||||
async with trio.open_nursery() as nursery:
|
||||
for unwrapped_addr in addrs:
|
||||
nursery.start_soon(
|
||||
probe_addr,
|
||||
wrap_address(unwrapped_addr),
|
||||
)
|
||||
|
||||
return (
|
||||
registrar_addrs,
|
||||
occupied_addrs,
|
||||
)
|
||||
|
||||
|
||||
def _is_local_addr(addr: Address) -> bool:
|
||||
'''
|
||||
Determine whether `addr` is reachable on the
|
||||
|
|
|
|||
|
|
@ -24,14 +24,23 @@ Multiaddress support using the upstream `py-multiaddr` lib
|
|||
- https://github.com/multiformats/multiaddr/blob/master/protocols/unix.md
|
||||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import ipaddress
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from multiaddr import Multiaddr
|
||||
from typing import (
|
||||
Any,
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# NOTE, `multiaddr` is lazy-imported at first use
|
||||
# (in the fns below) to keep it off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.discovery._addr import Address
|
||||
else:
|
||||
Multiaddr = Any
|
||||
Address = Any
|
||||
|
||||
# map from tractor-internal `proto_key` identifiers
|
||||
# to the standard multiaddr protocol name strings.
|
||||
|
|
@ -56,6 +65,8 @@ def mk_maddr(
|
|||
multiaddr-spec-compliant protocol path.
|
||||
|
||||
'''
|
||||
from multiaddr import Multiaddr
|
||||
|
||||
proto_key: str = addr.proto_key
|
||||
maddr_proto: str|None = _tpt_proto_to_maddr.get(proto_key)
|
||||
if maddr_proto is None:
|
||||
|
|
@ -98,6 +109,7 @@ def parse_maddr(
|
|||
|
||||
'''
|
||||
# lazy imports to avoid circular deps
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.ipc._tcp import TCPAddress
|
||||
from tractor.ipc._uds import UDSAddress
|
||||
|
||||
|
|
|
|||
|
|
@ -221,15 +221,18 @@ def pub(
|
|||
import tractor
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
portal = n.run_in_actor(
|
||||
portal = await n.start_actor(
|
||||
'publisher', # actor name
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
async with portal.open_stream_from(
|
||||
partial( # func to execute in it
|
||||
pub_service,
|
||||
topics=('clicks', 'users'),
|
||||
task_name='source1',
|
||||
)
|
||||
)
|
||||
async for value in await portal.result():
|
||||
) as stream:
|
||||
async for value in stream:
|
||||
print(f"Subscriber received {value}")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ from typing import (
|
|||
)
|
||||
import warnings
|
||||
|
||||
import msgspec
|
||||
import trio
|
||||
|
||||
from ._types import (
|
||||
|
|
@ -495,6 +496,7 @@ class Channel:
|
|||
async def _do_handshake(
|
||||
self,
|
||||
aid: Aid,
|
||||
timeout: float|None = None,
|
||||
|
||||
) -> Aid:
|
||||
'''
|
||||
|
|
@ -505,8 +507,28 @@ class Channel:
|
|||
"actor model" parlance.
|
||||
|
||||
'''
|
||||
try:
|
||||
with trio.fail_after(
|
||||
timeout if timeout is not None else float('inf')
|
||||
):
|
||||
await self.send(aid)
|
||||
peer_aid: Aid = await self.recv()
|
||||
if not isinstance(peer_aid, Aid):
|
||||
raise TypeError(
|
||||
f'Expected {Aid!r}, received {peer_aid!r}'
|
||||
)
|
||||
except (
|
||||
MsgTypeError,
|
||||
msgspec.DecodeError,
|
||||
TypeError,
|
||||
UnicodeDecodeError,
|
||||
trio.TooSlowError,
|
||||
) as handshake_err:
|
||||
raise TransportClosed(
|
||||
message='Peer sent an invalid actor handshake!\n',
|
||||
src_exc=handshake_err,
|
||||
loglevel='warning',
|
||||
) from handshake_err
|
||||
log.runtime(
|
||||
f'Received hanshake with peer\n'
|
||||
f'<= {peer_aid.reprol(sin_uuid=False)}\n'
|
||||
|
|
@ -518,7 +540,8 @@ class Channel:
|
|||
|
||||
@acm
|
||||
async def _connect_chan(
|
||||
addr: UnwrappedAddress
|
||||
addr: UnwrappedAddress,
|
||||
close_timeout: float|None = None,
|
||||
) -> typing.AsyncGenerator[Channel, None]:
|
||||
'''
|
||||
Create and connect a `Channel` to the provided `addr`, disconnect
|
||||
|
|
@ -529,6 +552,18 @@ async def _connect_chan(
|
|||
|
||||
'''
|
||||
chan = await Channel.from_addr(addr)
|
||||
try:
|
||||
yield chan
|
||||
finally:
|
||||
with trio.CancelScope(shield=True):
|
||||
if close_timeout is None:
|
||||
await chan.aclose()
|
||||
else:
|
||||
with trio.move_on_after(close_timeout) as close_cs:
|
||||
await chan.aclose()
|
||||
if close_cs.cancelled_caught:
|
||||
log.warning(
|
||||
f'Timed out closing channel after '
|
||||
f'{close_timeout}s\n'
|
||||
f'|_{chan}\n'
|
||||
)
|
||||
|
|
|
|||
|
|
@ -62,16 +62,19 @@ from .. import log
|
|||
from ..discovery._addr import Address
|
||||
from ._chan import Channel
|
||||
from ._transport import MsgTransport
|
||||
from ._uds import UDSAddress
|
||||
from ._tcp import TCPAddress
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..runtime._runtime import Actor
|
||||
from ..runtime._supervise import ActorNursery
|
||||
|
||||
|
||||
from ._tcp import TCPAddress
|
||||
from ._uds import UDSAddress
|
||||
|
||||
log = log.get_logger()
|
||||
|
||||
_PRE_REG_HANDSHAKE_TIMEOUT: float = 10
|
||||
|
||||
async def maybe_wait_on_canced_subs(
|
||||
uid: tuple[str, str],
|
||||
|
|
@ -316,8 +319,6 @@ async def handle_stream_from_peer(
|
|||
)
|
||||
|
||||
'''
|
||||
server._no_more_peers = trio.Event() # unset by making new
|
||||
|
||||
# TODO, debug_mode tooling for when hackin this lower layer?
|
||||
# with debug.maybe_open_crash_handler(
|
||||
# pdb=True,
|
||||
|
|
@ -335,6 +336,7 @@ async def handle_stream_from_peer(
|
|||
if actor := _state.current_actor():
|
||||
peer_aid: msgtypes.Aid = await chan._do_handshake(
|
||||
aid=actor.aid,
|
||||
timeout=_PRE_REG_HANDSHAKE_TIMEOUT,
|
||||
)
|
||||
except (
|
||||
TransportClosed,
|
||||
|
|
@ -351,12 +353,9 @@ async def handle_stream_from_peer(
|
|||
# "kinda-error" that we expect to tolerate during
|
||||
# discovery-sys related pings, queires, DoS etc.
|
||||
):
|
||||
# XXX: This may propagate up from `Channel._aiter_recv()`
|
||||
# and `MsgpackStream._inter_packets()` on a read from the
|
||||
# stream particularly when the runtime is first starting up
|
||||
# inside `open_root_actor()` where there is a check for
|
||||
# a bound listener on the registrar addr. the reset will be
|
||||
# because the handshake was never meant took place.
|
||||
# `TransportClosed` is expected when a peer disconnects or
|
||||
# fails the initial typed handshake, including foreign clients
|
||||
# and probes racing shutdown.
|
||||
log.runtime(
|
||||
con_status
|
||||
+
|
||||
|
|
@ -364,6 +363,13 @@ async def handle_stream_from_peer(
|
|||
)
|
||||
return
|
||||
|
||||
# Registry election probes need only the server's `Aid` capability
|
||||
# response; never register them as ordinary RPC peers.
|
||||
if peer_aid.is_probe:
|
||||
return
|
||||
|
||||
server._no_more_peers = trio.Event() # unset by making new
|
||||
|
||||
uid: tuple[str, str] = (
|
||||
peer_aid.name,
|
||||
peer_aid.uuid,
|
||||
|
|
|
|||
|
|
@ -20,7 +20,9 @@ TCP implementation of tractor.ipc._transport.MsgTransport protocol
|
|||
from __future__ import annotations
|
||||
import ipaddress
|
||||
from typing import (
|
||||
Any,
|
||||
ClassVar,
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
# from contextlib import (
|
||||
# asynccontextmanager as acm,
|
||||
|
|
@ -33,7 +35,6 @@ from trio import (
|
|||
open_tcp_listeners,
|
||||
)
|
||||
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.msg import MsgCodec
|
||||
from tractor.log import get_logger
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
|
|
@ -42,6 +43,13 @@ from tractor.ipc._transport import (
|
|||
MsgpackTransport,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ONLY type-annots, the eager import costs
|
||||
# `import tractor` wall-time (gh #470).
|
||||
from multiaddr import Multiaddr
|
||||
else:
|
||||
Multiaddr = Any
|
||||
|
||||
|
||||
log = get_logger()
|
||||
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ from collections.abc import (
|
|||
AsyncGenerator,
|
||||
AsyncIterator,
|
||||
)
|
||||
import errno
|
||||
import struct
|
||||
|
||||
import trio
|
||||
|
|
@ -61,6 +62,68 @@ if TYPE_CHECKING:
|
|||
log = get_logger()
|
||||
|
||||
|
||||
def _peer_closed_errno(exc: BaseException) -> int|None:
|
||||
'''
|
||||
Classify a complete transport exception tree as peer closure.
|
||||
|
||||
Follow explicit cause/context links. For a `BaseExceptionGroup`,
|
||||
require every child branch to resolve to a peer-close errno so an
|
||||
unrelated concurrent failure is never hidden as `TransportClosed`.
|
||||
|
||||
'''
|
||||
def find_peer_errno(
|
||||
current_exc: BaseException,
|
||||
ancestors: set[int],
|
||||
) -> int|None:
|
||||
exc_id: int = id(current_exc)
|
||||
if exc_id in ancestors:
|
||||
return None
|
||||
|
||||
ancestors = ancestors | {exc_id}
|
||||
if (
|
||||
isinstance(current_exc, OSError)
|
||||
and
|
||||
current_exc.errno in {
|
||||
errno.ECONNRESET,
|
||||
errno.EPIPE,
|
||||
}
|
||||
):
|
||||
return current_exc.errno
|
||||
|
||||
if isinstance(current_exc, BaseExceptionGroup):
|
||||
child_errnos: list[int|None] = [
|
||||
find_peer_errno(
|
||||
child_exc,
|
||||
ancestors,
|
||||
)
|
||||
for child_exc in current_exc.exceptions
|
||||
]
|
||||
if all(
|
||||
child_errno is not None
|
||||
for child_errno in child_errnos
|
||||
):
|
||||
return child_errnos[0]
|
||||
return None
|
||||
|
||||
chained_exc: BaseException|None = (
|
||||
current_exc.__cause__
|
||||
or
|
||||
current_exc.__context__
|
||||
)
|
||||
if chained_exc is not None:
|
||||
return find_peer_errno(
|
||||
chained_exc,
|
||||
ancestors,
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
return find_peer_errno(
|
||||
exc,
|
||||
set(),
|
||||
)
|
||||
|
||||
|
||||
# (codec, transport)
|
||||
MsgTransportKey = tuple[str, str]
|
||||
|
||||
|
|
@ -443,23 +506,23 @@ class MsgpackTransport(MsgTransport):
|
|||
trans_err = _re
|
||||
tpt_name: str = f'{type(self).__name__!r}'
|
||||
|
||||
trans_err_msg: str = trans_err.args[0]
|
||||
trans_err_msg: str = (
|
||||
str(trans_err.args[0])
|
||||
if trans_err.args
|
||||
else ''
|
||||
)
|
||||
by_whom: str = {
|
||||
'another task closed this fd': 'locally',
|
||||
'this socket was already closed': 'by peer',
|
||||
}.get(trans_err_msg)
|
||||
match trans_err:
|
||||
|
||||
# XXX, specifc to UDS transport and its,
|
||||
# well, "speediness".. XD
|
||||
# |_ likely todo with races related to how fast
|
||||
# the socket is setup/torn-down on linux
|
||||
# as it pertains to rando pings from the
|
||||
# `.discovery` subsys and protos.
|
||||
# UDS peers can disconnect before handshake.
|
||||
# Linux normally reports `EPIPE`; Darwin reports
|
||||
# `ECONNRESET` for the same expected closure.
|
||||
case trio.BrokenResourceError() if (
|
||||
'[Errno 32] Broken pipe'
|
||||
in
|
||||
trans_err_msg
|
||||
_peer_closed_errno(trans_err)
|
||||
is not None
|
||||
):
|
||||
tpt_closed = TransportClosed.from_src_exc(
|
||||
message=(
|
||||
|
|
|
|||
|
|
@ -18,17 +18,13 @@
|
|||
IPC subsys type-lookup helpers?
|
||||
|
||||
'''
|
||||
from typing import (
|
||||
Type,
|
||||
# TYPE_CHECKING,
|
||||
)
|
||||
|
||||
import trio
|
||||
from typing import Type
|
||||
import socket
|
||||
import trio
|
||||
|
||||
from tractor.ipc._transport import (
|
||||
MsgTransportKey,
|
||||
MsgTransport
|
||||
MsgTransport,
|
||||
)
|
||||
from tractor.ipc._tcp import (
|
||||
TCPAddress,
|
||||
|
|
@ -37,37 +33,33 @@ from tractor.ipc._tcp import (
|
|||
from tractor.ipc._uds import (
|
||||
UDSAddress,
|
||||
MsgpackUDSStream,
|
||||
HAS_UDS,
|
||||
)
|
||||
|
||||
# if TYPE_CHECKING:
|
||||
# from tractor._addr import Address
|
||||
|
||||
|
||||
# the UDS backend is importable everywhere but only *usable* when
|
||||
# `HAS_UDS` is `True`; otherwise the runtime registers TCP only.
|
||||
Address = TCPAddress|UDSAddress
|
||||
|
||||
# manually updated list of all supported msg transport types
|
||||
_msg_transports = [
|
||||
# the available msg-transport backends on this host: TCP always,
|
||||
# UDS only where usable (`HAS_UDS`). The lookup maps below derive
|
||||
# from this single list via each backend's `codec_key` and
|
||||
# `address_type`: register a backend here and every map picks it up.
|
||||
_msg_transports: list[Type[MsgTransport]] = [
|
||||
MsgpackTCPStream,
|
||||
MsgpackUDSStream
|
||||
]
|
||||
if HAS_UDS:
|
||||
_msg_transports.append(MsgpackUDSStream)
|
||||
|
||||
|
||||
# convert a MsgTransportKey to the corresponding transport type
|
||||
_key_to_transport: dict[
|
||||
MsgTransportKey,
|
||||
Type[MsgTransport],
|
||||
] = {
|
||||
('msgpack', 'tcp'): MsgpackTCPStream,
|
||||
('msgpack', 'uds'): MsgpackUDSStream,
|
||||
# map a `MsgTransportKey` -> `MsgTransport` type
|
||||
_key_to_transport: dict[MsgTransportKey, Type[MsgTransport]] = {
|
||||
(t.codec_key, t.address_type.proto_key): t
|
||||
for t in _msg_transports
|
||||
}
|
||||
|
||||
# convert an Address wrapper to its corresponding transport type
|
||||
_addr_to_transport: dict[
|
||||
Type[TCPAddress|UDSAddress],
|
||||
Type[MsgTransport]
|
||||
] = {
|
||||
TCPAddress: MsgpackTCPStream,
|
||||
UDSAddress: MsgpackUDSStream,
|
||||
# map an `Address`-wrapper -> `MsgTransport` type
|
||||
_addr_to_transport: dict[Type[Address], Type[MsgTransport]] = {
|
||||
t.address_type: t
|
||||
for t in _msg_transports
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -81,41 +73,51 @@ def transport_from_addr(
|
|||
|
||||
'''
|
||||
try:
|
||||
return _addr_to_transport[type(addr)]
|
||||
addr_type = type(addr)
|
||||
return _addr_to_transport[addr_type]
|
||||
|
||||
except KeyError:
|
||||
raise NotImplementedError(
|
||||
f'No known transport for address {repr(addr)}'
|
||||
f'No known transport for address '
|
||||
f'{addr!r}'
|
||||
)
|
||||
|
||||
|
||||
def transport_from_stream(
|
||||
stream: trio.abc.Stream,
|
||||
codec_key: str = 'msgpack'
|
||||
codec_key: str = 'msgpack',
|
||||
) -> Type[MsgTransport]:
|
||||
'''
|
||||
Given an arbitrary `trio.abc.Stream` and a desired codec,
|
||||
find the corresponding `MsgTransport` type.
|
||||
|
||||
'''
|
||||
transport = None
|
||||
transport: str|None = None
|
||||
|
||||
if isinstance(stream, trio.SocketStream):
|
||||
sock: socket.socket = stream.socket
|
||||
match sock.family:
|
||||
case socket.AF_INET | socket.AF_INET6:
|
||||
transport = 'tcp'
|
||||
|
||||
case socket.AF_UNIX:
|
||||
# `HAS_UDS` short-circuits before `socket.AF_UNIX` on
|
||||
# hosts where that constant is absent.
|
||||
case fam if (
|
||||
HAS_UDS
|
||||
and
|
||||
fam == socket.AF_UNIX
|
||||
):
|
||||
transport = 'uds'
|
||||
|
||||
case _:
|
||||
case fam:
|
||||
raise NotImplementedError(
|
||||
f'Unsupported socket family: {sock.family}'
|
||||
f'Unsupported socket family: {fam}'
|
||||
)
|
||||
|
||||
if not transport:
|
||||
raise NotImplementedError(
|
||||
f'Could not figure out transport type for stream type {type(stream)}'
|
||||
f'Could not figure out transport type for stream type '
|
||||
f'{type(stream)}'
|
||||
)
|
||||
|
||||
key = (codec_key, transport)
|
||||
|
|
|
|||
|
|
@ -21,17 +21,29 @@ from __future__ import annotations
|
|||
from contextlib import (
|
||||
contextmanager as cm,
|
||||
)
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import os
|
||||
import sys
|
||||
from socket import (
|
||||
AF_UNIX,
|
||||
SOCK_STREAM,
|
||||
SOL_SOCKET,
|
||||
error as socket_error,
|
||||
)
|
||||
# NOTE, `AF_UNIX` is absent on Windows / any CPython built without
|
||||
# unix-domain-socket support. Keep this module importable
|
||||
# everywhere (so `UDSAddress` stays referenceable for type and
|
||||
# `isinstance()` checks plus registry lookups); the `AF_UNIX`-using
|
||||
# code paths below are runtime-only and are never reached when the
|
||||
# UDS backend is unusable (gated on `trio`'s `has_unix`, see
|
||||
# `HAS_UDS`).
|
||||
try:
|
||||
from socket import AF_UNIX
|
||||
except ImportError:
|
||||
AF_UNIX = None
|
||||
import struct
|
||||
from typing import (
|
||||
Any,
|
||||
Type,
|
||||
TYPE_CHECKING,
|
||||
ClassVar,
|
||||
|
|
@ -49,7 +61,6 @@ from trio._highlevel_open_unix_stream import (
|
|||
has_unix,
|
||||
)
|
||||
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.msg import MsgCodec
|
||||
from tractor.log import get_logger
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
|
|
@ -63,7 +74,13 @@ from tractor.runtime._state import (
|
|||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ONLY type-annots, the eager import costs
|
||||
# `import tractor` wall-time (gh #470).
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.runtime._runtime import Actor
|
||||
else:
|
||||
Multiaddr = Any
|
||||
Actor = Any
|
||||
|
||||
|
||||
# Platform-specific credential passing constants
|
||||
|
|
@ -90,6 +107,22 @@ else:
|
|||
|
||||
log = get_logger()
|
||||
|
||||
_SUN_PATH_LIMIT: int = (
|
||||
108
|
||||
if sys.platform == 'linux'
|
||||
else 104
|
||||
)
|
||||
|
||||
|
||||
# single source of truth for whether the UDS backend is usable on this
|
||||
# host. Windows can expose `AF_UNIX`, but this backend remains
|
||||
# POSIX-only until its credential and lifecycle paths are supported.
|
||||
HAS_UDS: bool = (
|
||||
sys.platform != 'win32'
|
||||
and
|
||||
has_unix
|
||||
)
|
||||
|
||||
|
||||
def unwrap_sockpath(
|
||||
sockpath: Path,
|
||||
|
|
@ -190,7 +223,7 @@ class UDSAddress(
|
|||
err_on_no_runtime=False,
|
||||
)
|
||||
if actor:
|
||||
sockname: str = f'{actor.aid.name}@{pid}'
|
||||
sockname: str = actor.aid.name
|
||||
# XXX, orig version which broke both macOS (file-name
|
||||
# length) and `multiaddrs` ('::' invalid separator).
|
||||
# sockname: str = '::'.join(actor.uid) + f'@{pid}'
|
||||
|
|
@ -216,15 +249,72 @@ class UDSAddress(
|
|||
# `(?P<name>.+)@(?P<pid>\d+)\.sock` regex, and the
|
||||
# `spawn._reap` `{name}@{pid}.sock` reconstruction.
|
||||
token: str = uuid4().hex[:8]
|
||||
sockname: str = f'{prefix}.{token}@{pid}'
|
||||
sockname = f'{prefix}.{token}'
|
||||
|
||||
sockpath: Path = Path(f'{sockname}.sock')
|
||||
sockpath: Path = cls.get_sockname(
|
||||
name=sockname,
|
||||
pid=pid,
|
||||
bindspace=filedir,
|
||||
)
|
||||
return UDSAddress(
|
||||
filedir=filedir,
|
||||
filename=sockpath,
|
||||
maybe_pid=pid,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def get_sockname(
|
||||
cls,
|
||||
name: str,
|
||||
pid: int,
|
||||
bindspace: Path,
|
||||
) -> Path:
|
||||
'''
|
||||
Build a safe, deterministic UDS socket filename.
|
||||
|
||||
'''
|
||||
suffix: str = f'@{pid}.sock'
|
||||
filename: str = f'{name}{suffix}'
|
||||
unsafe: bool = (
|
||||
'\0' in name
|
||||
or
|
||||
'/' in name
|
||||
or
|
||||
bool(os.altsep and os.altsep in name)
|
||||
or
|
||||
Path(filename).is_absolute()
|
||||
)
|
||||
too_long: bool = (
|
||||
len(os.fsencode(bindspace / filename))
|
||||
>= _SUN_PATH_LIMIT
|
||||
)
|
||||
if (
|
||||
unsafe
|
||||
or
|
||||
too_long
|
||||
):
|
||||
digest: str = hashlib.blake2s(
|
||||
os.fsencode(name),
|
||||
digest_size=16,
|
||||
).hexdigest()
|
||||
filename = f'actor.{digest}{suffix}'
|
||||
|
||||
sockpath: Path = bindspace / filename
|
||||
path_nbytes: int = len(os.fsencode(sockpath))
|
||||
if path_nbytes >= _SUN_PATH_LIMIT:
|
||||
raise ValueError(
|
||||
f'UDS bindspace leaves no room for an AF_UNIX '
|
||||
f'socket filename!\n'
|
||||
f'bindspace: {bindspace}\n'
|
||||
f'name was unsafe: {unsafe}\n'
|
||||
f'name was over budget: {too_long}\n'
|
||||
f'compacted filename: {filename}\n'
|
||||
f'encoded path bytes: {path_nbytes}\n'
|
||||
f'AF_UNIX path limit: {_SUN_PATH_LIMIT}\n'
|
||||
)
|
||||
|
||||
return Path(filename)
|
||||
|
||||
@classmethod
|
||||
def get_root(cls) -> UDSAddress:
|
||||
def_uds_filename: Path = 'registry@1616.sock'
|
||||
|
|
@ -301,7 +391,16 @@ async def start_listener(
|
|||
f'>{{\n'
|
||||
f'|_{bs!r}\n'
|
||||
)
|
||||
bs.mkdir()
|
||||
bs.mkdir(
|
||||
# ensure the full ancestor tree for any nested
|
||||
# (custom `filedir`) bindspace; the default
|
||||
# `get_rt_dir()` space is pre-created but a custom
|
||||
# one may have missing parents.
|
||||
parents=True,
|
||||
# avoid `FileExistsError` from racing actors, same
|
||||
# guard as in `get_rt_dir()`.
|
||||
exist_ok=True,
|
||||
)
|
||||
|
||||
with _reraise_as_connerr(
|
||||
src_excs=(
|
||||
|
|
@ -554,11 +653,18 @@ class MsgpackUDSStream(MsgpackTransport):
|
|||
]:
|
||||
sock: trio.socket.socket = stream.socket
|
||||
|
||||
# NOTE XXX, it's unclear why one or the other ends up being
|
||||
# `bytes` versus the socket-file-path, i presume it's
|
||||
# something to do with who is the server (called `.listen()`)?
|
||||
# maybe could be better implemented using another info-query
|
||||
# on the socket like,
|
||||
# NOTE, the `bytes` case is a linux-only artifact: setting
|
||||
# `SO_PASSCRED` (see `open_unix_socket_w_passcred()`)
|
||||
# causes the kernel to *autobind* the un-named client
|
||||
# sock to an abstract-namespace addr which python
|
||||
# delivers as `bytes`; the listener-bound end is always
|
||||
# the fs-path `str`. On platforms WITHOUT autobind
|
||||
# (macOS et al) the un-bound end instead reports as an
|
||||
# empty `str` so BOTH names arrive as `str`s and the
|
||||
# real fs-path is whichever is non-empty: `peername` on
|
||||
# the connect side, `sockname` on the accept side.
|
||||
#
|
||||
# for socket-api deats see,
|
||||
# https://beej.us/guide/bgnet/html/split-wide/system-calls-or-bust.html#gethostnamewho-am-i
|
||||
sockname: str|bytes = sock.getsockname()
|
||||
# https://beej.us/guide/bgnet/html/split-wide/system-calls-or-bust.html#getpeernamewho-are-you
|
||||
|
|
@ -570,8 +676,24 @@ class MsgpackUDSStream(MsgpackTransport):
|
|||
case (bytes(), str()):
|
||||
sock_path: Path = Path(sockname)
|
||||
|
||||
case (str(), str()): # XXX, likely macOS
|
||||
sock_path: Path = Path(peername)
|
||||
# NOTE, no-autobind case (macOS): the un-bound end
|
||||
# is `''`, NOT a `bytes` abstract-ns addr; taking
|
||||
# `peername` unconditionally (as prior impl did)
|
||||
# delivers garbage `Path('')` addrs on the accept
|
||||
# side!
|
||||
case (str(), str()):
|
||||
bound_name: str = (
|
||||
peername
|
||||
or
|
||||
sockname
|
||||
)
|
||||
if not bound_name:
|
||||
raise ValueError(
|
||||
f'Empty UDS (peername, sockname) pair ??\n'
|
||||
f'peername: {peername!r}\n'
|
||||
f'sockname: {sockname!r}\n'
|
||||
)
|
||||
sock_path: Path = Path(bound_name)
|
||||
|
||||
case _:
|
||||
raise TypeError(
|
||||
|
|
|
|||
|
|
@ -26,11 +26,6 @@ built on `tractor`.
|
|||
'''
|
||||
from collections.abc import Mapping
|
||||
from functools import partial
|
||||
from inspect import (
|
||||
FrameInfo,
|
||||
getmodule,
|
||||
stack,
|
||||
)
|
||||
import sys
|
||||
import logging
|
||||
from logging import (
|
||||
|
|
@ -38,10 +33,16 @@ from logging import (
|
|||
Logger,
|
||||
StreamHandler,
|
||||
)
|
||||
from types import ModuleType
|
||||
from types import (
|
||||
FrameType,
|
||||
ModuleType,
|
||||
)
|
||||
import warnings
|
||||
|
||||
import colorlog # type: ignore
|
||||
# NOTE, `colorlog` is lazy-imported in
|
||||
# `get_console_log()` to keep it off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
#
|
||||
# ?TODO, some other (modern) alt libs?
|
||||
# import coloredlogs
|
||||
# import colored_traceback.auto # ?TODO, need better config?
|
||||
|
|
@ -436,17 +437,41 @@ def get_logger(
|
|||
pkg_name: str = _root_name
|
||||
|
||||
def get_caller_mod(
|
||||
frames_up:int = 2
|
||||
):
|
||||
frames_up: int = 2,
|
||||
) -> ModuleType|None:
|
||||
'''
|
||||
Attempt to get the module which called `tractor.get_logger()`.
|
||||
Attempt to get the module which called
|
||||
`tractor.get_logger()`.
|
||||
|
||||
Resolve the caller's frame with `sys._getframe()` and
|
||||
map its `__name__` through `sys.modules`; `inspect.stack()`
|
||||
(the previous impl) builds src-file info for EVERY frame
|
||||
on the stack, scanning all of `sys.modules` per frame via
|
||||
`inspect.getmodule()`, which made module-level
|
||||
`get_logger()` calls dominate `import tractor` time
|
||||
(see gh #470).
|
||||
|
||||
'''
|
||||
callstack: list[FrameInfo] = stack()
|
||||
caller_fi: FrameInfo = callstack[frames_up]
|
||||
caller_mod: ModuleType = getmodule(caller_fi.frame)
|
||||
try:
|
||||
caller_frame: FrameType = sys._getframe(frames_up)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
mod_name: str|None = caller_frame.f_globals.get(
|
||||
'__name__',
|
||||
)
|
||||
if mod_name is None:
|
||||
return None
|
||||
|
||||
if caller_mod := sys.modules.get(mod_name):
|
||||
return caller_mod
|
||||
|
||||
# Preserve caller discovery for `runpy`, plugin loaders,
|
||||
# and `exec()` namespaces not registered in `sys.modules`.
|
||||
# Import `inspect` only on this rare fallback path.
|
||||
from inspect import getmodule
|
||||
return getmodule(caller_frame)
|
||||
|
||||
# --- Auto--naming-CASE ---
|
||||
# -------------------------
|
||||
# Implicitly introspect the caller's module-name whenever `name`
|
||||
|
|
@ -780,6 +805,10 @@ def get_console_log(
|
|||
None,
|
||||
)
|
||||
):
|
||||
# lazy-imported to keep it off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
import colorlog # type: ignore
|
||||
|
||||
fmt: str = LOG_FORMAT # always apply our format?
|
||||
handler = StreamHandler()
|
||||
formatter = colorlog.ColoredFormatter(
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue