Compare commits
No commits in common. "fbaf933ccbcbdbfe001a6c031a3c595c9b453a40" and "3818fe863853d85345d8c3a302d24e3d7641f6e1" have entirely different histories.
fbaf933ccb
...
3818fe8638
|
|
@ -88,27 +88,10 @@ jobs:
|
|||
|
||||
|
||||
testing:
|
||||
name: '${{ matrix.os }} Python${{ matrix.python-version }} spawn_backend=${{ matrix.spawn_backend }} tpt_proto=${{ matrix.tpt_proto }} capture=${{ matrix.capture }}'
|
||||
timeout-minutes: 20
|
||||
name: '${{ matrix.os }} Python${{ matrix.python-version }} spawn_backend=${{ matrix.spawn_backend }} tpt_proto=${{ matrix.tpt_proto }}'
|
||||
timeout-minutes: 16
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
# NOTE on the matrix shape — the `capture=` mode follows
|
||||
# `spawn_backend`:
|
||||
#
|
||||
# - `trio` / `mp_*` backends use `--capture=fd` (default)
|
||||
# for per-test attribution of subactor *raw-fd* output
|
||||
# in failure reports.
|
||||
# - Fork-based backends (`main_thread_forkserver`,
|
||||
# `subint_forkserver`) REQUIRE `--capture=sys` because
|
||||
# fork-child × `--capture=fd` is a known deadlock
|
||||
# pattern. See the long NOTE in `tractor._testing.pytest`'s
|
||||
# `pytest_load_initial_conftests` for the mechanism +
|
||||
# tradeoff write-up.
|
||||
#
|
||||
# If a future matrix row adds a fork-spawn backend
|
||||
# WITHOUT setting `capture: 'sys'`, the
|
||||
# `pytest_load_initial_conftests` hook fail-fasts on `CI=1`
|
||||
# with a clear error msg. So the matrix is self-policing.
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
|
|
@ -135,26 +118,6 @@ jobs:
|
|||
'tcp',
|
||||
'uds',
|
||||
]
|
||||
capture: [
|
||||
'fd', # default for non-fork backends
|
||||
]
|
||||
|
||||
# Fork-based backends — added via `include:` so each
|
||||
# cell carries its REQUIRED `capture: 'sys'` mode.
|
||||
# Linux-only for now; macOS coverage TBD pending
|
||||
# local validation.
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
python-version: '3.13'
|
||||
spawn_backend: 'main_thread_forkserver'
|
||||
tpt_proto: 'tcp'
|
||||
capture: 'sys'
|
||||
- os: ubuntu-latest
|
||||
python-version: '3.13'
|
||||
spawn_backend: 'main_thread_forkserver'
|
||||
tpt_proto: 'uds'
|
||||
capture: 'sys'
|
||||
|
||||
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
|
||||
exclude:
|
||||
# don't do UDS run on macOS (for now)
|
||||
|
|
@ -195,11 +158,7 @@ jobs:
|
|||
-rsx
|
||||
--spawn-backend=${{ matrix.spawn_backend }}
|
||||
--tpt-proto=${{ matrix.tpt_proto }}
|
||||
--capture=${{ matrix.capture }}
|
||||
# NOTE: capture mode is matrix-driven — `fd` for
|
||||
# non-fork backends (per-test fd attribution),
|
||||
# `sys` for fork-based (avoids fork-child x
|
||||
# capture-fd deadlock). See matrix-NOTE above.
|
||||
--capture=fd
|
||||
|
||||
# XXX legacy NOTE XXX
|
||||
#
|
||||
|
|
|
|||
|
|
@ -1,314 +0,0 @@
|
|||
# Cancel-cascade `trio.TooSlowError` flakes under `main_thread_forkserver`
|
||||
|
||||
## Symptom
|
||||
|
||||
Running the full test suite under
|
||||
|
||||
```bash
|
||||
./py313/bin/python -m pytest tests/ \
|
||||
--tpt-proto=tcp \
|
||||
--spawn-backend=main_thread_forkserver
|
||||
```
|
||||
|
||||
surfaces a single, **rotating** `trio.TooSlowError`
|
||||
failure each run. The failure isn't deterministic on
|
||||
test identity — different test each run — but it
|
||||
ALWAYS looks like:
|
||||
|
||||
```
|
||||
FAILED tests/<file>::test_<name> - trio.TooSlowError
|
||||
==== 1 failed, 373 passed, 17 skipped, 11–12 xfailed,
|
||||
0–1 xpassed, ~550 warnings in ~6min ====
|
||||
```
|
||||
|
||||
Pass rate: **~99.7%** (373 of 374 non-skip tests).
|
||||
Wall-clock per full run: 5–6 min.
|
||||
|
||||
## Tests observed flaking so far
|
||||
|
||||
Each row was the SOLE failure in a separate run:
|
||||
|
||||
| run # | test |
|
||||
|---|---|
|
||||
| 1 | `tests/test_advanced_streaming.py::test_dynamic_pub_sub[KeyboardInterrupt]` |
|
||||
| 2 | `tests/test_infected_asyncio.py::test_context_spawns_aio_task_that_errors[parent_actor_cancels_child=False]` |
|
||||
|
||||
Both share the same shape:
|
||||
|
||||
- **Cancel cascade** of N subactors back to a parent root actor.
|
||||
- N ≥ `multiprocessing.cpu_count()` for `test_dynamic_pub_sub`
|
||||
(it spawns `cpus - 1` consumers + publisher + dynamic-consumer).
|
||||
- N ≈ 2 for `test_context_spawns_aio_task_that_errors` —
|
||||
but each subactor is `infect_asyncio=True`, so each
|
||||
cancel involves the trio↔asyncio guest-run unwind
|
||||
which is structurally heavier than pure-trio.
|
||||
- Test wraps the cascade in `trio.fail_after(N seconds)`
|
||||
and the cap fires before the cascade completes.
|
||||
|
||||
The exact failing test rotates because each test is
|
||||
independently close to the cap; whichever happens to
|
||||
be unlucky in scheduling/CPU-contention on a given run
|
||||
is the one that times out.
|
||||
|
||||
## Root-cause family
|
||||
|
||||
`hard_kill` (`tractor/spawn/_spawn.py:hard_kill`) runs
|
||||
the SC-graceful teardown ladder per subactor:
|
||||
|
||||
1. `Portal.cancel_actor()` — graceful IPC cancel-req.
|
||||
2. Wait `terminate_after=1.6s` for sub to exit.
|
||||
3. If still alive: `proc.kill()` (SIGKILL).
|
||||
4. (NEW) `_unlink_uds_bind_addrs()` — post-mortem
|
||||
sock-file cleanup for UDS leaks (issue #452 fix).
|
||||
|
||||
For a cascade of N subactors, each pays steps 1–4. If
|
||||
graceful-cancel doesn't complete within 1.6s for ANY
|
||||
sub, that sub eats a full 1.6s of `move_on_after` plus
|
||||
the `proc.wait()` post-SIGKILL.
|
||||
|
||||
Worst case under fork backend with N=cpus subs:
|
||||
- N × 1.6s = 16s+ on a 10-core box just for the
|
||||
graceful timeout phase
|
||||
- Plus per-spawn fork-IPC handshake cost compounds
|
||||
during teardown (each sub's IPC cleanup goes through
|
||||
the same forkserver coordinator)
|
||||
- Plus the new autouse fixtures
|
||||
(`_track_orphaned_uds_per_test`,
|
||||
`_detect_runaway_subactors_per_test`,
|
||||
`_reap_orphaned_subactors`) all run at test
|
||||
teardown, adding small (10s of ms) but cumulative
|
||||
overhead
|
||||
|
||||
Current cap: 30s (`fail_after_s = 30 if
|
||||
is_forking_spawner else 12`). Empirically fits the
|
||||
median run but the tail breaks ~0.3% of the time.
|
||||
|
||||
## NOT regressing
|
||||
|
||||
To confirm this is a flake and not a regression:
|
||||
|
||||
- Pre-`WakeupSocketpair`-patch baseline: tests
|
||||
HUNG INDEFINITELY (busy-loop never released).
|
||||
- Post-patch: pass-or-fail-fast, ~99.7% pass, the
|
||||
occasional cap-hit fails in bounded time (<60s for
|
||||
the offending test).
|
||||
- Same test PASSES under `--spawn-backend=trio`
|
||||
(no fork, no hard-kill compounding).
|
||||
|
||||
So the suite is dramatically better than before; the
|
||||
remaining flake is a known-tolerable steady-state.
|
||||
|
||||
## Possible mitigations (ranked)
|
||||
|
||||
### A. Bump the cap further
|
||||
|
||||
Cheapest. Change the per-test `fail_after_s` from 30
|
||||
to e.g. 60 for fork backends. Pros: trivial. Cons:
|
||||
masks any genuine slowness regression we'd want to
|
||||
catch.
|
||||
|
||||
### B. CPU-count-aware cap
|
||||
|
||||
For tests whose N scales with `cpu_count()`, scale
|
||||
the cap too:
|
||||
|
||||
```python
|
||||
fail_after_s = (
|
||||
max(30, cpu_count() * 3) # 3s/actor floor
|
||||
if is_forking_spawner
|
||||
else 12
|
||||
)
|
||||
```
|
||||
|
||||
Pros: scales with the actual cancel-cascade work.
|
||||
Cons: still arbitrary multiplier.
|
||||
|
||||
### C. `pytest-rerunfailures` for these tests only
|
||||
|
||||
Mark the known-flaky tests with
|
||||
`@pytest.mark.flaky(reruns=1)` (needs
|
||||
`pytest-rerunfailures` dep). Single retry hides
|
||||
genuine ~0.3% transient flakes.
|
||||
|
||||
Pros: no cap change, surfaces persistent failures
|
||||
loudly. Cons: adds a dep, retries can mask real bugs
|
||||
if used widely.
|
||||
|
||||
### D. Reduce `hard_kill`'s `terminate_after`
|
||||
|
||||
Drop from 1.6s → 0.8s. Cuts the worst-case cascade
|
||||
time roughly in half. Risks: fewer subs get a chance
|
||||
to run their cleanup before SIGKILL → more orphaned
|
||||
state for the autouse reapers to handle (ironically,
|
||||
adds back overhead elsewhere).
|
||||
|
||||
### E. Profile + targeted fix
|
||||
|
||||
Add `log.devx()` markers in `hard_kill` to time each
|
||||
phase. Identify if any subactor is consistently
|
||||
hitting the 1.6s cap (vs. exiting in <0.1s). If so,
|
||||
that sub has a teardown bug worth fixing at source.
|
||||
Pros: actually fixes the underlying slowness. Cons:
|
||||
real investigation work, deferred from this round.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Land this issue-doc as the tracker. Apply **(B)** as
|
||||
a small follow-up — cheap and proportional. If it
|
||||
still flakes, escalate to **(E)** with a `log.devx()`
|
||||
profile-pass.
|
||||
|
||||
`(C)` is a backstop if `(B)` doesn't quite get there
|
||||
and we need green CI faster than (E) can deliver.
|
||||
|
||||
## Verification protocol
|
||||
|
||||
After applying any mitigation:
|
||||
|
||||
```bash
|
||||
# Run the suite N times back-to-back, count failures.
|
||||
# A persistent failure on the SAME test == real bug.
|
||||
# Failures rotating across tests == still cap-related.
|
||||
|
||||
for i in $(seq 1 5); do
|
||||
./py313/bin/python -m pytest tests/ \
|
||||
--tpt-proto=tcp \
|
||||
--spawn-backend=main_thread_forkserver \
|
||||
-q 2>&1 | tail -2
|
||||
done
|
||||
```
|
||||
|
||||
Target: 0 failures across 5 runs ⇒ ship. 1–2 failures
|
||||
still rotating ⇒ apply (C). Same test failing twice
|
||||
⇒ escalate to (E).
|
||||
|
||||
## Snapshot evidence (2026-05-13)
|
||||
|
||||
After landing the `fail_after_w_trace` /
|
||||
`afk_alarm_w_trace` capture-on-timeout helpers
|
||||
(`tractor._testing.trace`), `test_nested_multierrors`
|
||||
on the `main_thread_forkserver` backend produces
|
||||
**reproducible diag snapshots** at
|
||||
`$XDG_CACHE_HOME/tractor/hung-dumps/test_nested_multierrors_start_method_main_thread_forkserver__<iso-ts>/`.
|
||||
|
||||
### Reproduction
|
||||
|
||||
```bash
|
||||
pytest \
|
||||
-v --verbose --durations=10 \
|
||||
--spawn-backend=main_thread_forkserver \
|
||||
--tpt-proto=uds \
|
||||
--capture=sys --show-capture=stderr -rxX \
|
||||
tests/test_cancellation.py::test_nested_multierrors
|
||||
```
|
||||
|
||||
The test is `xfail(strict=False)` for MTF — it RUNS
|
||||
each invocation so snapshots accumulate, but doesn't
|
||||
break `--lf` workflow.
|
||||
|
||||
### Consistent shape across runs
|
||||
|
||||
5+ snapshots taken back-to-back show the SAME pattern:
|
||||
|
||||
- **Timing:** ~10s wall-clock total. Inner
|
||||
`fail_after_w_trace(10)` fires at exactly T=10s;
|
||||
cascade's `nursery.__aexit__` takes ~0.6s more to
|
||||
gather + propagate the resulting
|
||||
`BaseExceptionGroup`. **Trio backend completes the
|
||||
SAME test in <6s** — so the MTF cascade is ~2x
|
||||
slower at minimum.
|
||||
|
||||
- **`BaseExceptionGroup` shape:** mixed
|
||||
`[RemoteActorError, Cancelled]`. The first
|
||||
subactor's natural error-propagation (`assert 0`
|
||||
raised → `RemoteActorError` portal-result)
|
||||
arrives before T=10s; the OTHER subactor's
|
||||
portal-wait is still in flight at T=10s, gets
|
||||
cancelled by `fail_after_w_trace`'s scope-cancel
|
||||
→ returns `Cancelled` instead.
|
||||
|
||||
- **Orphan-spawn skew:** snapshot's `orphans` bucket
|
||||
(after the `_is_tractor_subactor` cgroup-slice
|
||||
override fix) consistently shows 2-4 init-adopted
|
||||
procs at `depth_3` and `depth_1` levels — these
|
||||
are the leaves whose parent (`depth_2` spawner)
|
||||
was killed mid-cascade but who hadn't yet seen
|
||||
the cancel signal themselves.
|
||||
|
||||
- **UDS sock-leak:** 2-6 dead-orphan socks per run
|
||||
(varies with cascade timing). The
|
||||
`track_orphaned_uds_per_test` fixture reaps them
|
||||
post-test → contamination is isolated per-invocation.
|
||||
|
||||
### Capture mechanism
|
||||
|
||||
`fail_after_w_trace` covers two firing paths:
|
||||
|
||||
1. **`trio.TooSlowError`** raised at scope-exit
|
||||
(body returned cleanly past deadline) — direct
|
||||
`except` handler captures.
|
||||
|
||||
2. **Scope-cancel + body raises non-`Cancelled` exc**
|
||||
(e.g. `nursery.__aexit__` wraps timeout-induced
|
||||
`Cancelled` into a `BaseExceptionGroup` that
|
||||
escapes before `trio.fail_after`'s exit-check
|
||||
could fire `TooSlowError`) — body-raise `except`
|
||||
handler checks `scope.cancel_called` and
|
||||
captures if True. This path catches the
|
||||
`test_nested_multierrors` shape specifically (see
|
||||
"BaseExceptionGroup shape" above).
|
||||
|
||||
The snapshot dir contains:
|
||||
- `trace.txt` — `ptree` + `hung_state` (kernel
|
||||
`wchan`/`stack` + `py-spy dump --locals` when
|
||||
sudo cached), with `include_strays=True`
|
||||
surfacing any cross-test ghost subactor trees in
|
||||
the `orphans` bucket.
|
||||
- `bindspace.txt` — UDS bindspace classification
|
||||
(live-active / orphaned-alive / orphaned-dead).
|
||||
- `meta.json` — `{pid, label, captured_at, sudo_cached}`.
|
||||
|
||||
The end-of-session `pytest_terminal_summary` hook
|
||||
in `tractor._testing.pytest` lists every snapshot
|
||||
dir from the run so you don't have to scroll back
|
||||
through captured-stderr lines:
|
||||
|
||||
```
|
||||
========================= tractor hang-snapshot index ==========================
|
||||
N `fail_after_w_trace` / `afk_alarm_w_trace` snapshot(s) captured this session:
|
||||
<test-id>
|
||||
→ /home/.../.cache/tractor/hung-dumps/<label>__<ts>
|
||||
```
|
||||
|
||||
### Caveats
|
||||
|
||||
The snapshot fires AFTER the body-raise (not at the
|
||||
exact moment of scope-cancel), so the parent's
|
||||
py-spy frames show `_do_capture_snapshot` itself
|
||||
running, NOT the cancel-cascade hang frame. To see
|
||||
the actual hang state, manual `acli.ptree` /
|
||||
`acli.hung_dump` from a second terminal at T=10s
|
||||
would be needed — **not currently possible**
|
||||
because per-test reaper fixtures clean up ~0.6s
|
||||
post-timeout. See follow-up TODO in
|
||||
`tractor/_testing/trace.py` for a
|
||||
`TRACTOR_TRACE_HOLD=1` env-var pause mode.
|
||||
|
||||
## See also
|
||||
|
||||
- [#452](https://github.com/goodboy/tractor/issues/452) —
|
||||
UDS sock-file leak (related — `hard_kill`'s
|
||||
cleanup phase contributes to cascade time)
|
||||
- `ai/conc-anal/trio_wakeup_socketpair_busy_loop_under_fork_issue.md`
|
||||
— the upstream-trio fix that turned this from a
|
||||
100% hang into a 0.3% flake
|
||||
- `ai/conc-anal/infected_asyncio_under_main_thread_forkserver_hang_issue.md`
|
||||
— the asyncio variant which contributes to one of
|
||||
the rotating failures
|
||||
- `tractor/spawn/_spawn.py::hard_kill` — the SIGKILL
|
||||
cascade source
|
||||
- `tractor/_testing/_reap.py::_track_orphaned_uds_per_test`,
|
||||
`_detect_runaway_subactors_per_test`,
|
||||
`_reap_orphaned_subactors` — autouse cleanup
|
||||
fixtures whose cumulative teardown overhead
|
||||
contributes to the cascade time
|
||||
|
|
@ -89,12 +89,10 @@ except ImportError:
|
|||
# the "zero tractor imports" isolation guarantee; now that
|
||||
# CPython-level feasibility is confirmed, the validated
|
||||
# primitives have moved into tractor proper.)
|
||||
from tractor.spawn._main_thread_forkserver import (
|
||||
fork_from_worker_thread,
|
||||
wait_child,
|
||||
)
|
||||
from tractor.spawn._subint_forkserver import (
|
||||
fork_from_worker_thread,
|
||||
run_subint_in_worker_thread,
|
||||
wait_child,
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,14 +1,13 @@
|
|||
'''
|
||||
Integration exercises for the `tractor.spawn._main_thread_forkserver`
|
||||
Integration exercises for the `tractor.spawn._subint_forkserver`
|
||||
submodule at three tiers:
|
||||
|
||||
1. the low-level primitives
|
||||
(`fork_from_worker_thread()` from `_main_thread_forkserver`
|
||||
+ `run_subint_in_worker_thread()` from
|
||||
`_subint_forkserver`) driven from inside a real
|
||||
(`fork_from_worker_thread()` +
|
||||
`run_subint_in_worker_thread()`) driven from inside a real
|
||||
`trio.run()` in the parent process,
|
||||
|
||||
2. the full `main_thread_forkserver_proc` spawn backend wired
|
||||
2. the full `subint_forkserver_proc` spawn backend wired
|
||||
through tractor's normal actor-nursery + portal-RPC
|
||||
machinery — i.e. `open_root_actor` + `open_nursery` +
|
||||
`run_in_actor` against a subactor spawned via fork from a
|
||||
|
|
@ -28,15 +27,15 @@ Those smoke-test scenarios are standalone — no trio runtime
|
|||
in the *parent*. Tiers (1)+(2) here cover the primitives
|
||||
driven from inside `trio.run()` in the parent, and tier (3)
|
||||
(the `*_spawn_basic` test) drives the registered
|
||||
`main_thread_forkserver` spawn backend end-to-end against
|
||||
the tractor runtime.
|
||||
`subint_forkserver` spawn backend end-to-end against the
|
||||
tractor runtime.
|
||||
|
||||
Gating
|
||||
------
|
||||
- py3.14+ (via `concurrent.interpreters` presence)
|
||||
- no `--spawn-backend` restriction — the backend-level test
|
||||
flips `tractor.spawn._spawn._spawn_method` programmatically
|
||||
(via `try_set_start_method('main_thread_forkserver')`) and
|
||||
(via `try_set_start_method('subint_forkserver')`) and
|
||||
restores it on teardown, so these tests are independent of
|
||||
the session-level CLI backend choice.
|
||||
|
||||
|
|
@ -65,12 +64,10 @@ from tractor.devx import dump_on_hang
|
|||
# `tractor.spawn._subint` for why.
|
||||
pytest.importorskip('concurrent.interpreters')
|
||||
|
||||
from tractor.spawn._main_thread_forkserver import ( # noqa: E402
|
||||
fork_from_worker_thread,
|
||||
wait_child,
|
||||
)
|
||||
from tractor.spawn._subint_forkserver import ( # noqa: E402
|
||||
fork_from_worker_thread,
|
||||
run_subint_in_worker_thread,
|
||||
wait_child,
|
||||
)
|
||||
from tractor.spawn import _spawn as _spawn_mod # noqa: E402
|
||||
from tractor.spawn._spawn import try_set_start_method # noqa: E402
|
||||
|
|
@ -198,7 +195,7 @@ def test_fork_from_worker_thread_via_trio(
|
|||
deadline: float = 10.0
|
||||
with dump_on_hang(
|
||||
seconds=deadline,
|
||||
path='/tmp/main_thread_forkserver_baseline.dump',
|
||||
path='/tmp/subint_forkserver_baseline.dump',
|
||||
):
|
||||
pid: int = trio.run(
|
||||
partial(run_fork_in_non_trio_thread, deadline),
|
||||
|
|
@ -220,14 +217,14 @@ def test_fork_and_run_trio_in_child() -> None:
|
|||
`trio.run()` inside it on yet another worker thread.
|
||||
|
||||
This is the full "forkserver + trio-in-subint-in-child"
|
||||
pattern the proposed `main_thread_forkserver` spawn backend
|
||||
pattern the proposed `subint_forkserver` spawn backend
|
||||
would rest on.
|
||||
|
||||
'''
|
||||
deadline: float = 15.0
|
||||
with dump_on_hang(
|
||||
seconds=deadline,
|
||||
path='/tmp/main_thread_forkserver_trio_in_child.dump',
|
||||
path='/tmp/subint_forkserver_trio_in_child.dump',
|
||||
):
|
||||
pid: int = trio.run(
|
||||
partial(
|
||||
|
|
@ -240,7 +237,7 @@ def test_fork_and_run_trio_in_child() -> None:
|
|||
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# tier-3 backend test: drive the registered `main_thread_forkserver`
|
||||
# tier-3 backend test: drive the registered `subint_forkserver`
|
||||
# spawn backend end-to-end through tractor's actor-nursery +
|
||||
# portal-RPC machinery.
|
||||
# ----------------------------------------------------------------
|
||||
|
|
@ -263,7 +260,7 @@ async def _happy_path_forkserver(
|
|||
Parent-side harness: stand up a root actor, open an actor
|
||||
nursery, spawn one subactor via the currently-selected
|
||||
spawn backend (which this test will have flipped to
|
||||
`main_thread_forkserver`), run a trivial RPC through its
|
||||
`subint_forkserver`), run a trivial RPC through its
|
||||
portal, assert the round-trip result.
|
||||
|
||||
'''
|
||||
|
|
@ -307,19 +304,19 @@ def forkserver_spawn_method():
|
|||
|
||||
|
||||
@pytest.mark.timeout(60, method='thread')
|
||||
def test_main_thread_forkserver_spawn_basic(
|
||||
def test_subint_forkserver_spawn_basic(
|
||||
reg_addr: tuple[str, int | str],
|
||||
forkserver_spawn_method,
|
||||
) -> None:
|
||||
'''
|
||||
Happy-path: spawn ONE subactor via the
|
||||
`main_thread_forkserver` backend (parent-side fork from a
|
||||
`subint_forkserver` backend (parent-side fork from a
|
||||
main-interp worker thread), do a trivial portal-RPC
|
||||
round-trip, tear the nursery down cleanly.
|
||||
|
||||
If this passes, the "forkserver + tractor runtime" arch
|
||||
is proven end-to-end: the registered
|
||||
`main_thread_forkserver_proc` spawn target successfully
|
||||
`subint_forkserver_proc` spawn target successfully
|
||||
forks a child, the child runs `_actor_child_main()` +
|
||||
completes IPC handshake + serves an RPC, and the parent
|
||||
reaps via `_ForkedProc.wait()` without regressing any of
|
||||
|
|
@ -329,7 +326,7 @@ def test_main_thread_forkserver_spawn_basic(
|
|||
deadline: float = 20.0
|
||||
with dump_on_hang(
|
||||
seconds=deadline,
|
||||
path='/tmp/main_thread_forkserver_spawn_basic.dump',
|
||||
path='/tmp/subint_forkserver_spawn_basic.dump',
|
||||
):
|
||||
trio.run(
|
||||
partial(
|
||||
|
|
@ -343,7 +340,7 @@ def test_main_thread_forkserver_spawn_basic(
|
|||
# ----------------------------------------------------------------
|
||||
# tier-4 DRAFT: orphaned-subactor SIGINT survivability
|
||||
#
|
||||
# Motivating question: with `main_thread_forkserver`, the child's
|
||||
# Motivating question: with `subint_forkserver`, the child's
|
||||
# `trio.run()` lives on the fork-inherited worker thread which
|
||||
# is NOT `threading.main_thread()` — so trio cannot install its
|
||||
# `signal.set_wakeup_fd`-based SIGINT handler. If the parent
|
||||
|
|
@ -363,7 +360,7 @@ def test_main_thread_forkserver_spawn_basic(
|
|||
# Cross-backend generalization (decide after this passes):
|
||||
# - applicable to any backend whose subactors are separate OS
|
||||
# processes: `trio`, `mp_spawn`, `mp_forkserver`,
|
||||
# `main_thread_forkserver`.
|
||||
# `subint_forkserver`.
|
||||
# - NOT applicable to plain `subint` (subactors are in-process
|
||||
# subinterpreters, no orphan child process to SIGINT).
|
||||
# - move path: lift the harness script into
|
||||
|
|
@ -449,7 +446,7 @@ def _process_alive(pid: int) -> bool:
|
|||
return False
|
||||
|
||||
|
||||
# Known-gap test — `main_thread_forkserver` orphan-SIGINT
|
||||
# Known-gap test — `subint_forkserver` orphan-SIGINT
|
||||
# handling. See
|
||||
# `ai/conc-anal/subint_forkserver_orphan_sigint_hang_issue.md`.
|
||||
# `strict=True` so if a future fix closes the gap the
|
||||
|
|
@ -474,12 +471,12 @@ def test_orphaned_subactor_sigint_cleanup_DRAFT(
|
|||
) -> None:
|
||||
'''
|
||||
DRAFT — orphaned-subactor SIGINT survivability under the
|
||||
`main_thread_forkserver` backend.
|
||||
`subint_forkserver` backend.
|
||||
|
||||
Sequence:
|
||||
1. Spawn a harness subprocess that brings up a root
|
||||
actor + one `sleep_forever` subactor via
|
||||
`main_thread_forkserver`.
|
||||
`subint_forkserver`.
|
||||
2. Read the harness's stdout for `PARENT_READY=<pid>`
|
||||
and `CHILD_PID=<pid>` markers (confirms the
|
||||
parent→child IPC handshake completed).
|
||||
|
|
@ -527,7 +524,7 @@ def test_orphaned_subactor_sigint_cleanup_DRAFT(
|
|||
[
|
||||
sys.executable,
|
||||
str(script_path),
|
||||
'main_thread_forkserver',
|
||||
'subint_forkserver',
|
||||
host,
|
||||
str(port),
|
||||
],
|
||||
|
|
@ -580,7 +577,7 @@ def test_orphaned_subactor_sigint_cleanup_DRAFT(
|
|||
|
||||
pytest.fail(
|
||||
f'Orphan subactor (pid={child_pid}) did NOT exit '
|
||||
f'within 10s of SIGINT under `main_thread_forkserver` '
|
||||
f'within 10s of SIGINT under `subint_forkserver` '
|
||||
f'→ trio on non-main thread did not observe the '
|
||||
f'default CPython KeyboardInterrupt; backend needs '
|
||||
f'explicit SIGINT plumbing.'
|
||||
|
|
@ -603,50 +600,3 @@ def test_orphaned_subactor_sigint_cleanup_DRAFT(
|
|||
proc.wait(timeout=2.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# regression guard: variant-2 (`subint_forkserver`) placeholder
|
||||
# MUST raise `NotImplementedError` today — guards against future
|
||||
# commits accidentally re-aliasing the key to the variant-1
|
||||
# coroutine (which was a transient state during the rename).
|
||||
# ----------------------------------------------------------------
|
||||
def test_subint_forkserver_key_errors_cleanly() -> None:
|
||||
'''
|
||||
`--spawn-backend=subint_forkserver` is reserved for the
|
||||
eventual variant-2 (subint-isolated child runtime)
|
||||
backend, gated on jcrist/msgspec#1026 unblocking PEP 684
|
||||
isolated-mode subints upstream.
|
||||
|
||||
Until that lands, the dispatch entry MUST raise
|
||||
`NotImplementedError` immediately rather than silently
|
||||
aliasing to `main_thread_forkserver_proc`. Verify the
|
||||
error message also surfaces both the working-backend
|
||||
pointer and the upstream-blocker ref so an operator
|
||||
arriving at the error has somewhere to go.
|
||||
|
||||
'''
|
||||
import asyncio
|
||||
from tractor.spawn._spawn import _methods
|
||||
|
||||
proc = _methods['subint_forkserver']
|
||||
with pytest.raises(NotImplementedError) as ei:
|
||||
# signature args match `main_thread_forkserver_proc`'s
|
||||
# — the stub raises before touching them so dummy
|
||||
# values are fine.
|
||||
asyncio.run(
|
||||
proc(
|
||||
'x', None, None, {}, [],
|
||||
('127.0.0.1', 0), {},
|
||||
)
|
||||
)
|
||||
|
||||
msg: str = str(ei.value)
|
||||
assert 'main_thread_forkserver' in msg, (
|
||||
f'stub error msg should redirect to the working '
|
||||
f'variant-1 backend; got: {msg!r}'
|
||||
)
|
||||
assert 'msgspec#1026' in msg or '1026' in msg, (
|
||||
f'stub error msg should reference the upstream '
|
||||
f'blocker (jcrist/msgspec#1026); got: {msg!r}'
|
||||
)
|
||||
|
|
@ -38,7 +38,6 @@ Two empirical CPython properties drive the design:
|
|||
the forked child otherwise (`Fatal Python error: not main
|
||||
interpreter`). Full source-level walkthrough:
|
||||
`ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`.
|
||||
|
||||
2. **`os.fork()` from a regular `threading.Thread` attached to
|
||||
the *main* interpreter — i.e. a worker thread that has never
|
||||
entered a subint — works cleanly.** Empirically validated
|
||||
|
|
@ -87,11 +86,9 @@ costs:
|
|||
|
||||
- **Sidecar lifecycle**: a second long-lived process per
|
||||
parent, with its own start/stop/health-check semantics.
|
||||
|
||||
- **IPC overhead per spawn**: every actor-spawn round-trips
|
||||
an `mp` request message through a unix socket before any
|
||||
child code runs.
|
||||
|
||||
- **State isolation by process boundary**: the sidecar can't
|
||||
share parent state at all — every spawn is a "cold" child
|
||||
re-importing modules from disk.
|
||||
|
|
@ -109,7 +106,6 @@ For the full variant-2 picture see
|
|||
1) we already get costs 1 + 2 collapsed; cost 3 will land
|
||||
when msgspec#1026 unblocks isolated-mode subints.
|
||||
|
||||
|
||||
What survives the fork? — POSIX semantics
|
||||
-----------------------------------------
|
||||
|
||||
|
|
@ -117,58 +113,33 @@ A natural worry when forking from a parent that's running
|
|||
`trio.run()` on another thread: does that trio thread (and
|
||||
any other threads in the parent) keep running in the child?
|
||||
|
||||
**No** — but with a precise meaning that's worth pinning
|
||||
down, since the canonical trio framing
|
||||
([python-trio/trio#1614](https://github.com/python-trio/trio/issues/1614))
|
||||
puts it the opposite-sounding way:
|
||||
|
||||
> If you use `fork()` in a process with multiple threads,
|
||||
> all the other thread stacks are just leaked: there's
|
||||
> nothing else you can reasonably do with them.
|
||||
|
||||
Both statements describe the same POSIX reality from
|
||||
opposite sides:
|
||||
|
||||
- **Execution-side ("gone")**: POSIX `fork()` only
|
||||
preserves the *calling* thread as a runnable thread in
|
||||
the child. Every other thread in the parent — trio's
|
||||
runner thread, any `to_thread` cache threads, anything
|
||||
else — never executes another instruction post-fork.
|
||||
|
||||
- **Memory-side ("leaked")**: those non-running threads'
|
||||
*stacks* and per-thread heap structures are still
|
||||
COW-inherited into the child's address space. They
|
||||
persist as orphaned bytes with no owning thread, no
|
||||
scheduler entry, and no way for the child to clean
|
||||
them up — hence trio's word "leaked".
|
||||
**No.** POSIX `fork()` only preserves the *calling* thread
|
||||
in the child. Every other thread in the parent — trio's
|
||||
runner thread, any `to_thread` cache threads, anything else
|
||||
— is gone the instant `fork()` returns in the child.
|
||||
|
||||
Concretely, after the forkserver worker calls `os.fork()`:
|
||||
|
||||
| thread | parent | child (executing) | child (memory) |
|
||||
|---------------------|-----------|-------------------|-----------------------------|
|
||||
| forkserver worker | continues | sole survivor | live stack |
|
||||
| `trio.run()` thread | continues | not running | leaked stack (zombie bytes) |
|
||||
| any other thread | continues | not running | leaked stack (zombie bytes) |
|
||||
| thread | parent | child |
|
||||
|-----------------------|-----------|---------------|
|
||||
| forkserver worker | continues | sole survivor |
|
||||
| `trio.run()` thread | continues | gone |
|
||||
| any other thread | continues | gone |
|
||||
|
||||
The forkserver worker becomes the new "main" execution
|
||||
context in the child; `trio.run()` and every other parent
|
||||
thread never executes a single instruction post-fork.
|
||||
Their stack memory rides along as inert COW pages until
|
||||
the child's fresh `trio.run()` boots and overwrites/GCs
|
||||
it (or until the child `exec()`s and discards the entire
|
||||
image).
|
||||
thread never executes a single instruction post-fork in the
|
||||
child.
|
||||
|
||||
This is exactly *why* `os.fork()` is delegated to a
|
||||
dedicated worker thread that has provably never entered
|
||||
trio: we want that trio-free thread to be the surviving
|
||||
*executing* thread in the child, with the leaked trio
|
||||
stack reduced to inert COW pages we don't touch.
|
||||
one in the child.
|
||||
|
||||
The leaked-stack residue is one slice of the broader
|
||||
"fork in a multithreaded program is dangerous" hazard
|
||||
class (see `man pthread_atfork`). Other dead-thread
|
||||
artifacts that cross the fork boundary, and how we handle
|
||||
each:
|
||||
That said, dead-thread *artifacts* still cross the fork
|
||||
boundary (canonical "fork in a multithreaded program is
|
||||
dangerous" — see `man pthread_atfork`). What persists, and
|
||||
how we handle each:
|
||||
|
||||
- **Inherited file descriptors** — the dead trio thread's
|
||||
epoll fd, signal-wakeup-fd, eventfds, sockets, IPC
|
||||
|
|
@ -177,20 +148,16 @@ each:
|
|||
`_close_inherited_fds()` in the child prelude — walks
|
||||
`/proc/self/fd` and closes everything except stdio +
|
||||
the channel pipe to the forkserver.
|
||||
|
||||
- **Memory image** — trio's internal data structures
|
||||
(scheduler, task queues, runner state) sit in COW
|
||||
memory alongside the leaked stacks above. Nobody's
|
||||
executing them; they get GC'd / overwritten when the
|
||||
child's fresh `trio.run()` boots.
|
||||
|
||||
memory but nobody's executing them. Get GC'd /
|
||||
overwritten when the child's fresh `trio.run()` boots.
|
||||
- **Python thread state** — handled automatically by
|
||||
CPython. `PyOS_AfterFork_Child()` calls
|
||||
`_PyThreadState_DeleteExceptCurrent()`, so dead
|
||||
`PyThreadState` objects are cleaned and
|
||||
`threading.enumerate()` returns just the surviving
|
||||
thread.
|
||||
|
||||
- **User-level locks (`threading.Lock`)** —
|
||||
held-by-dead-thread state is the canonical fork hazard.
|
||||
Not an issue in practice for tractor: trio doesn't hold
|
||||
|
|
@ -199,7 +166,6 @@ each:
|
|||
either direction). CPython's GIL is auto-reset by the
|
||||
fork callback.
|
||||
|
||||
|
||||
FYI: how this dodges the `trio.run()` × `fork()` hazards
|
||||
--------------------------------------------------------
|
||||
|
||||
|
|
@ -217,16 +183,13 @@ design dodges each class explicitly:
|
|||
reader. *Dodge*: the inherited wakeup-fd is closed by
|
||||
`_close_inherited_fds()`, then the child's own
|
||||
`trio.run()` installs a fresh one.
|
||||
|
||||
- **`epoll`/`kqueue` instance**: trio's I/O backend holds
|
||||
one. Inherited as a dead fd; same fix as above.
|
||||
|
||||
- **Threadpool cache threads** (`trio.to_thread`): worker
|
||||
threads with cached tstate. Don't exist in the child
|
||||
(POSIX); cache state is meaningless garbage that gets
|
||||
reset when the child's trio.run() initializes its own
|
||||
thread cache.
|
||||
|
||||
- **Cancel scopes / nurseries / open `trio.Process` /
|
||||
open sockets**: these are trio-runtime objects, not
|
||||
kernel objects. The runtime that owns them is gone in
|
||||
|
|
@ -234,11 +197,9 @@ design dodges each class explicitly:
|
|||
in COW memory and get overwritten as the child runs.
|
||||
Inherited *kernel* fds those objects wrapped (sockets,
|
||||
proc pipes) are caught by `_close_inherited_fds()`.
|
||||
|
||||
- **`atexit` handlers**: trio doesn't register any that
|
||||
would mis-fire post-fork; trio's lifetime-stack is
|
||||
all `with`-block-scoped and dies with the runner.
|
||||
|
||||
- **Foreign-language I/O state** (libcurl, OpenSSL session
|
||||
caches, etc.): out of scope — same hazard as any
|
||||
fork-without-exec; users layering those on top of
|
||||
|
|
@ -250,7 +211,6 @@ isolation + `_close_inherited_fds()` cleanup gives the
|
|||
forked child a clean trio environment. Everything else
|
||||
falls under the standard fork-without-exec disclaimer.
|
||||
|
||||
|
||||
Implementation status
|
||||
---------------------
|
||||
|
||||
|
|
@ -271,11 +231,10 @@ follow-up) including the
|
|||
|
||||
Still-open work (tracked on tractor #379):
|
||||
|
||||
- [ ] no cancellation / hard-kill stress coverage yet
|
||||
- no cancellation / hard-kill stress coverage yet
|
||||
(counterpart to `tests/test_subint_cancellation.py` for
|
||||
the plain `subint` backend),
|
||||
|
||||
- [ ] `child_sigint='trio'` mode (flag scaffolded below; default
|
||||
- `child_sigint='trio'` mode (flag scaffolded below; default
|
||||
is `'ipc'`). Originally intended as a manual SIGINT →
|
||||
trio-cancel bridge, but investigation showed trio's
|
||||
handler IS already correctly installed in the fork-child
|
||||
|
|
@ -328,24 +287,21 @@ See also
|
|||
- `tractor.spawn._subint_forkserver` — variant-2 placeholder
|
||||
module; reserved for the future subint-isolated-child
|
||||
runtime once jcrist/msgspec#1026 unblocks.
|
||||
|
||||
- `tractor.spawn._subint_fork` — the stub for the
|
||||
fork-from-non-main-subint strategy that DIDN'T work (kept
|
||||
in-tree as documentation of the attempt + the CPython-level
|
||||
block).
|
||||
|
||||
- `ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`
|
||||
— CPython source walkthrough of why fork-from-subint is dead.
|
||||
|
||||
- `ai/conc-anal/subint_fork_from_main_thread_smoketest.py`
|
||||
— standalone feasibility check (delegates to this module
|
||||
for the primitives it exercises).
|
||||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import errno
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import threading
|
||||
from functools import partial
|
||||
from typing import (
|
||||
|
|
@ -368,8 +324,8 @@ from tractor.runtime._portal import Portal
|
|||
from ._spawn import (
|
||||
cancel_on_completion,
|
||||
soft_kill,
|
||||
wait_for_peer_or_proc_death,
|
||||
)
|
||||
from ._subint import _has_subints
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from tractor.discovery._addr import UnwrappedAddress
|
||||
|
|
@ -467,24 +423,9 @@ def _close_inherited_fds(
|
|||
try:
|
||||
os.close(fd)
|
||||
closed += 1
|
||||
except OSError as oserr:
|
||||
# `EBADF` is the benign-and-expected case: the
|
||||
# `os.listdir('/proc/self/fd')` call above itself
|
||||
# opens a transient dirfd that ends up in
|
||||
# `candidates`, then auto-closes before this loop
|
||||
# reaches it. Same for any fd whose Python wrapper
|
||||
# was GC'd between `listdir` and `os.close`.
|
||||
# Suppress at debug-level — surfacing every
|
||||
# EBADF as a full traceback (prior `log.exception`
|
||||
# behavior) drowned the post-fork log channel.
|
||||
if oserr.errno == errno.EBADF:
|
||||
log.debug(
|
||||
f'Skip already-closed inherited fd {fd!r} '
|
||||
f'(EBADF, benign race with listdir)\n'
|
||||
)
|
||||
continue
|
||||
# Other errnos (EIO / EPERM / EINTR / ...) are
|
||||
# genuinely unexpected — keep the loud surface.
|
||||
except OSError:
|
||||
# fd was already closed (race with listdir) or otherwise
|
||||
# unclosable — either is fine.
|
||||
log.exception(
|
||||
f'Failed to close inherited fd in child ??\n'
|
||||
f'{fd!r}\n'
|
||||
|
|
@ -761,26 +702,6 @@ class _ForkedProc:
|
|||
self._pidfd = -1
|
||||
return self._returncode
|
||||
|
||||
def terminate(self) -> None:
|
||||
'''
|
||||
OS-level `SIGTERM` to the child. Swallows
|
||||
`ProcessLookupError` (already dead).
|
||||
|
||||
Mirrors `trio.Process.terminate()` /
|
||||
`multiprocessing.Process.terminate()` — sends SIGTERM
|
||||
(graceful, allows the child a chance to clean up via
|
||||
signal-handlers) rather than SIGKILL. Used by
|
||||
`ActorNursery.cancel()`'s per-child escalation when
|
||||
`Portal.cancel_actor()` raises `ActorTooSlowError`,
|
||||
and by the legacy `hard_kill=True` branch on the same
|
||||
path.
|
||||
|
||||
'''
|
||||
try:
|
||||
os.kill(self.pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
|
||||
def kill(self) -> None:
|
||||
'''
|
||||
OS-level `SIGKILL` to the child. Swallows
|
||||
|
|
@ -851,6 +772,13 @@ async def main_thread_forkserver_proc(
|
|||
thread instead of `trio.lowlevel.open_process()`.
|
||||
|
||||
'''
|
||||
if not _has_subints:
|
||||
raise RuntimeError(
|
||||
f'The {"main_thread_forkserver"!r} spawn backend '
|
||||
f'requires Python 3.14+.\n'
|
||||
f'Current runtime: {sys.version}'
|
||||
)
|
||||
|
||||
# Backend-scoped config pulled from `proc_kwargs`. Using
|
||||
# `proc_kwargs` (vs a first-class kwarg on this function)
|
||||
# matches how other backends expose per-spawn tuning
|
||||
|
|
@ -969,18 +897,7 @@ async def main_thread_forkserver_proc(
|
|||
f' |_{proc}\n'
|
||||
)
|
||||
|
||||
# race the handshake-wait against proc-death so a
|
||||
# sub that dies during boot (e.g. crashed on import
|
||||
# before reaching `_actor_child_main`, leaving a
|
||||
# zombie + no cmdline) surfaces as `ActorFailure`
|
||||
# instead of parking the spawning task forever on
|
||||
# an unsignalled `_peer_connected[uid]` event.
|
||||
event, chan = await wait_for_peer_or_proc_death(
|
||||
ipc_server,
|
||||
uid,
|
||||
proc_wait=proc.wait,
|
||||
proc_repr=repr(proc),
|
||||
)
|
||||
event, chan = await ipc_server.wait_for_peer(uid)
|
||||
|
||||
except trio.Cancelled:
|
||||
cancelled_during_spawn = True
|
||||
|
|
|
|||
|
|
@ -43,7 +43,6 @@ from tractor.log import get_logger
|
|||
from tractor.discovery._addr import (
|
||||
UnwrappedAddress,
|
||||
)
|
||||
from .._exceptions import ActorFailure
|
||||
from ._reap import unlink_uds_bind_addrs
|
||||
from tractor.runtime._portal import Portal
|
||||
from tractor.runtime._runtime import Actor
|
||||
|
|
@ -83,15 +82,11 @@ SpawnMethodKey = Literal[
|
|||
# runtime, exactly like `trio_proc` but via fork instead
|
||||
# of subproc-exec. See `tractor.spawn._main_thread_forkserver`.
|
||||
'main_thread_forkserver',
|
||||
# Variant-2: same fork machinery as `main_thread_forkserver`
|
||||
# but the child enters a sub-interpreter to host its
|
||||
# `trio.run()`. Gated on jcrist/msgspec#1026 unblocking
|
||||
# PEP 684 isolated-mode subints upstream — until then
|
||||
# `subint_forkserver_proc` is a clean `NotImplementedError`
|
||||
# stub pointing at variant-1 (`main_thread_forkserver`) +
|
||||
# the upstream blocker. The key is reserved here (not just
|
||||
# aliased to variant-1) so once upstream lands the impl can
|
||||
# flip in-place without API churn. See
|
||||
# RESERVED for the future variant-2 subint-isolated-child
|
||||
# runtime — gated on jcrist/msgspec#1026 + PEP 684. Today
|
||||
# this key aliases to `main_thread_forkserver_proc`; once
|
||||
# the upstream unblocks land it'll dispatch to the
|
||||
# subint-hosted-trio impl. See
|
||||
# `tractor.spawn._subint_forkserver`.
|
||||
'subint_forkserver',
|
||||
]
|
||||
|
|
@ -111,71 +106,6 @@ else:
|
|||
await trio.lowlevel.wait_readable(proc.sentinel)
|
||||
|
||||
|
||||
async def wait_for_peer_or_proc_death(
|
||||
ipc_server,
|
||||
uid: tuple[str, str],
|
||||
# TODO? not not types?
|
||||
proc_wait: 'Callable[[], Awaitable]',
|
||||
proc_repr: str = '',
|
||||
|
||||
) -> 'tuple[trio.Event, Channel]':
|
||||
'''
|
||||
Race `IPCServer.wait_for_peer(uid)` against the sub-proc's
|
||||
own `.wait()` coroutine. Whichever completes first cancels
|
||||
the other.
|
||||
|
||||
Used by every spawn-backend to detect a sub-actor that
|
||||
*dies during boot* before completing the parent-handshake-
|
||||
callback (e.g. crashed on import, exec'd-out, kernel-killed
|
||||
pre-`_actor_child_main`). Without this race, the
|
||||
handshake-wait — backed by an unsignalled `trio.Event` —
|
||||
parks the spawning task forever and leaves the dead child
|
||||
as a zombie since nobody calls `proc.wait()` to reap.
|
||||
|
||||
On normal handshake-complete: returns `(event, chan)`
|
||||
identical to a bare `wait_for_peer`.
|
||||
|
||||
On proc-death-first: raises `ActorFailure` carrying the
|
||||
proc's exit code, allowing the supervisor to surface a
|
||||
clean error rather than hanging indefinitely.
|
||||
|
||||
`proc_wait` is a 0-arg async callable returning the proc's
|
||||
exit-status — kept generic so each backend can pass its
|
||||
own (`trio.Process.wait`, `_ForkedProc.wait`,
|
||||
`proc_waiter(mp.Process)`, etc.).
|
||||
|
||||
`proc_repr` is an optional string used in the
|
||||
`ActorFailure` message for diag.
|
||||
|
||||
'''
|
||||
result: dict = {}
|
||||
|
||||
async def _await_handshake():
|
||||
event, chan = await ipc_server.wait_for_peer(uid)
|
||||
result['handshake'] = (event, chan)
|
||||
boot_n.cancel_scope.cancel()
|
||||
|
||||
async def _await_death():
|
||||
rc = await proc_wait()
|
||||
result['died'] = rc
|
||||
boot_n.cancel_scope.cancel()
|
||||
|
||||
async with trio.open_nursery() as boot_n:
|
||||
boot_n.start_soon(_await_handshake)
|
||||
boot_n.start_soon(_await_death)
|
||||
|
||||
if 'handshake' in result:
|
||||
return result['handshake']
|
||||
|
||||
# only reached if proc-death won the race
|
||||
raise ActorFailure(
|
||||
f'Sub-actor {uid!r} died during boot '
|
||||
f'(rc={result.get("died")!r}) before completing '
|
||||
f'parent-handshake.\n'
|
||||
f' proc: {proc_repr}'
|
||||
)
|
||||
|
||||
|
||||
def try_set_start_method(
|
||||
key: SpawnMethodKey
|
||||
|
||||
|
|
@ -208,15 +138,13 @@ def try_set_start_method(
|
|||
case 'mp_spawn':
|
||||
_ctx = mp.get_context('spawn')
|
||||
|
||||
case (
|
||||
'trio'
|
||||
| 'main_thread_forkserver'
|
||||
):
|
||||
case 'trio':
|
||||
_ctx = None
|
||||
|
||||
case (
|
||||
'subint'
|
||||
| 'subint_fork'
|
||||
| 'main_thread_forkserver'
|
||||
| 'subint_forkserver'
|
||||
):
|
||||
# All subint-family backends need no `mp.context`;
|
||||
|
|
|
|||
|
|
@ -18,18 +18,15 @@
|
|||
Variant-2 (future) "subint forkserver" placeholder — reserved
|
||||
for the eventual subint-isolated-child runtime variant.
|
||||
|
||||
> **Status:** reserved key, stub impl. Today
|
||||
> `--spawn-backend=subint_forkserver` raises a clean
|
||||
> `NotImplementedError` from `subint_forkserver_proc()`
|
||||
> below, pointing at variant-1
|
||||
> (`--spawn-backend=main_thread_forkserver`, see
|
||||
> `tractor.spawn._main_thread_forkserver`) and the upstream
|
||||
> blocker
|
||||
> ([jcrist/msgspec#1026](https://github.com/jcrist/msgspec/issues/1026)).
|
||||
> The key is reserved here (not aliased to variant-1) so the
|
||||
> literal subint-hosted-child impl can flip in-place once
|
||||
> msgspec#1026 unblocks PEP 684 isolated-mode subints
|
||||
> upstream — no API churn at the call site.
|
||||
> **Status:** placeholder. Today
|
||||
> `--spawn-backend=subint_forkserver` aliases to
|
||||
> `main_thread_forkserver_proc` (variant 1, see
|
||||
> `tractor.spawn._main_thread_forkserver`). A follow-up commit
|
||||
> in this PR series flips the alias to a `NotImplementedError`
|
||||
> stub reserving the `'subint_forkserver'` key for the literal
|
||||
> subint-hosted-child variant once
|
||||
> [jcrist/msgspec#1026](https://github.com/jcrist/msgspec/issues/1026)
|
||||
> unblocks PEP 684 isolated-mode subints upstream.
|
||||
|
||||
Future arch — what subints would buy us
|
||||
---------------------------------------
|
||||
|
|
@ -157,6 +154,21 @@ from trio import TaskStatus
|
|||
from tractor.log import get_logger
|
||||
from ._subint import _has_subints
|
||||
|
||||
# Backward-compat re-exports of the fork primitives whose
|
||||
# canonical home is now `_main_thread_forkserver`. Kept here
|
||||
# transiently so existing
|
||||
# `from tractor.spawn._subint_forkserver import ...` callsites
|
||||
# in the tests + the conc-anal smoketest keep resolving;
|
||||
# dropped once a follow-up commit migrates those imports to
|
||||
# the new module.
|
||||
from ._main_thread_forkserver import (
|
||||
_close_inherited_fds as _close_inherited_fds,
|
||||
_format_child_exit as _format_child_exit,
|
||||
fork_from_worker_thread as fork_from_worker_thread,
|
||||
wait_child as wait_child,
|
||||
_ForkedProc as _ForkedProc,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from tractor.discovery._addr import UnwrappedAddress
|
||||
from tractor.runtime._portal import Portal
|
||||
|
|
|
|||
Loading…
Reference in New Issue