Compare commits
No commits in common. "57dae0e4a62a8edd21753e7ec207697c9994d159" and "4b5176e2c3040491c2b60cc9f321081a4a60e9c9" have entirely different histories.
57dae0e4a6
...
4b5176e2c3
|
|
@ -873,12 +873,9 @@ class Actor:
|
||||||
if self._spawn_method in (
|
if self._spawn_method in (
|
||||||
'trio',
|
'trio',
|
||||||
'subint',
|
'subint',
|
||||||
# `main_thread_forkserver` (and the future
|
# `subint_forkserver` parent-side sends a
|
||||||
# variant-2 `subint_forkserver`) parent-side
|
# `SpawnSpec` over IPC just like the other two
|
||||||
# sends a `SpawnSpec` over IPC just like the
|
# — fork child-side runtime is trio-native.
|
||||||
# other two — fork child-side runtime is
|
|
||||||
# trio-native.
|
|
||||||
'main_thread_forkserver',
|
|
||||||
'subint_forkserver',
|
'subint_forkserver',
|
||||||
):
|
):
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,973 +0,0 @@
|
||||||
# tractor: structured concurrent "actors".
|
|
||||||
# Copyright 2018-eternity Tyler Goodlet.
|
|
||||||
|
|
||||||
# This program is free software: you can redistribute it and/or modify
|
|
||||||
# it under the terms of the GNU Affero General Public License as published by
|
|
||||||
# the Free Software Foundation, either version 3 of the License, or
|
|
||||||
# (at your option) any later version.
|
|
||||||
|
|
||||||
# This program is distributed in the hope that it will be useful,
|
|
||||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
||||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
||||||
# GNU Affero General Public License for more details.
|
|
||||||
|
|
||||||
# You should have received a copy of the GNU Affero General Public License
|
|
||||||
# along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
||||||
|
|
||||||
'''
|
|
||||||
Variant-1 "main-thread forkserver" spawn backend (today's
|
|
||||||
working impl) + the generic fork-from-main-interp-worker-thread
|
|
||||||
primitives it's built on.
|
|
||||||
|
|
||||||
Spawn-method key: `'main_thread_forkserver'`. The legacy
|
|
||||||
`'subint_forkserver'` key currently aliases here too — see
|
|
||||||
`tractor.spawn._subint_forkserver` for the future variant-2
|
|
||||||
(subint-isolated-child runtime, gated on
|
|
||||||
[jcrist/msgspec#1026](https://github.com/jcrist/msgspec/issues/1026))
|
|
||||||
that key is reserved for.
|
|
||||||
|
|
||||||
Background
|
|
||||||
----------
|
|
||||||
|
|
||||||
Two empirical CPython properties drive the design:
|
|
||||||
|
|
||||||
1. **`os.fork()` from a non-main sub-interpreter is refused by
|
|
||||||
CPython.** `PyOS_AfterFork_Child()` →
|
|
||||||
`_PyInterpreterState_DeleteExceptMain()` gates on the calling
|
|
||||||
thread's tstate belonging to the main interpreter and aborts
|
|
||||||
the forked child otherwise (`Fatal Python error: not main
|
|
||||||
interpreter`). Full source-level walkthrough:
|
|
||||||
`ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`.
|
|
||||||
2. **`os.fork()` from a regular `threading.Thread` attached to
|
|
||||||
the *main* interpreter — i.e. a worker thread that has never
|
|
||||||
entered a subint — works cleanly.** Empirically validated
|
|
||||||
across four scenarios by
|
|
||||||
`ai/conc-anal/subint_fork_from_main_thread_smoketest.py` on
|
|
||||||
py3.14.
|
|
||||||
|
|
||||||
The fork-from-main-thread primitives below codify property (2)
|
|
||||||
into a reusable surface: spawn a worker thread, fork in it,
|
|
||||||
retrieve the child pid back to the caller trio task, and offer a
|
|
||||||
`trio.Process`-shaped shim around the raw pid so the existing
|
|
||||||
`soft_kill`/`hard_reap` patterns from `_spawn.py` keep working
|
|
||||||
unchanged.
|
|
||||||
|
|
||||||
Design rationale — why a forkserver, and why in-process
|
|
||||||
-------------------------------------------------------
|
|
||||||
|
|
||||||
Two design questions worth pinning down up front, since the
|
|
||||||
naming intentionally evokes the stdlib `multiprocessing.forkserver`
|
|
||||||
for comparison:
|
|
||||||
|
|
||||||
**(1) Why a forkserver pattern at all, vs. forking directly
|
|
||||||
from the trio task?**
|
|
||||||
|
|
||||||
`os.fork()` is fundamentally hostile to trio: trio owns
|
|
||||||
file descriptors, signal-wakeup-fds, threadpools, and an
|
|
||||||
event loop with non-trivial post-fork lifecycle invariants
|
|
||||||
(see python-trio/trio#1614 et al.). Forking a trio-running
|
|
||||||
thread duplicates all that state into the child, which then
|
|
||||||
either needs surgical reset (fragile) or has to immediately
|
|
||||||
`exec()` (defeats the point of fork-without-exec). The
|
|
||||||
*forkserver* sidesteps this by isolating the `os.fork()`
|
|
||||||
call in a worker that has provably never entered trio — so
|
|
||||||
the child inherits a clean, trio-free image.
|
|
||||||
|
|
||||||
**(2) Why an in-process forkserver, vs. stdlib
|
|
||||||
`multiprocessing.forkserver`?**
|
|
||||||
|
|
||||||
The stdlib design solves the same "fork from clean state"
|
|
||||||
problem by spinning up a **separate sidecar process** at
|
|
||||||
first use of `mp.set_start_method('forkserver')`. The parent
|
|
||||||
then IPC's each spawn request to that sidecar over a unix
|
|
||||||
socket; the sidecar is the process that actually calls
|
|
||||||
`os.fork()`. This works but pays for cleanliness with three
|
|
||||||
costs:
|
|
||||||
|
|
||||||
- **Sidecar lifecycle**: a second long-lived process per
|
|
||||||
parent, with its own start/stop/health-check semantics.
|
|
||||||
- **IPC overhead per spawn**: every actor-spawn round-trips
|
|
||||||
an `mp` request message through a unix socket before any
|
|
||||||
child code runs.
|
|
||||||
- **State isolation by process boundary**: the sidecar can't
|
|
||||||
share parent state at all — every spawn is a "cold" child
|
|
||||||
re-importing modules from disk.
|
|
||||||
|
|
||||||
Once the variant-2 (subint-isolated child runtime) lands the
|
|
||||||
in-process forkserver collapses all three costs:
|
|
||||||
|
|
||||||
- no sidecar — the forkserver is just another thread,
|
|
||||||
- spawn signal is a thread-local event/condition, not IPC,
|
|
||||||
- child inherits the warm parent state (loaded modules,
|
|
||||||
populated caches, etc.) for free.
|
|
||||||
|
|
||||||
For the full variant-2 picture see
|
|
||||||
`tractor.spawn._subint_forkserver`'s docstring. Today (variant
|
|
||||||
1) we already get costs 1 + 2 collapsed; cost 3 will land
|
|
||||||
when msgspec#1026 unblocks isolated-mode subints.
|
|
||||||
|
|
||||||
What survives the fork? — POSIX semantics
|
|
||||||
-----------------------------------------
|
|
||||||
|
|
||||||
A natural worry when forking from a parent that's running
|
|
||||||
`trio.run()` on another thread: does that trio thread (and
|
|
||||||
any other threads in the parent) keep running in the child?
|
|
||||||
|
|
||||||
**No.** POSIX `fork()` only preserves the *calling* thread
|
|
||||||
in the child. Every other thread in the parent — trio's
|
|
||||||
runner thread, any `to_thread` cache threads, anything else
|
|
||||||
— is gone the instant `fork()` returns in the child.
|
|
||||||
|
|
||||||
Concretely, after the forkserver worker calls `os.fork()`:
|
|
||||||
|
|
||||||
| thread | parent | child |
|
|
||||||
|-----------------------|-----------|---------------|
|
|
||||||
| forkserver worker | continues | sole survivor |
|
|
||||||
| `trio.run()` thread | continues | gone |
|
|
||||||
| any other thread | continues | gone |
|
|
||||||
|
|
||||||
The forkserver worker becomes the new "main" execution
|
|
||||||
context in the child; `trio.run()` and every other parent
|
|
||||||
thread never executes a single instruction post-fork in the
|
|
||||||
child.
|
|
||||||
|
|
||||||
This is exactly *why* `os.fork()` is delegated to a
|
|
||||||
dedicated worker thread that has provably never entered
|
|
||||||
trio: we want that trio-free thread to be the surviving
|
|
||||||
one in the child.
|
|
||||||
|
|
||||||
That said, dead-thread *artifacts* still cross the fork
|
|
||||||
boundary (canonical "fork in a multithreaded program is
|
|
||||||
dangerous" — see `man pthread_atfork`). What persists, and
|
|
||||||
how we handle each:
|
|
||||||
|
|
||||||
- **Inherited file descriptors** — the dead trio thread's
|
|
||||||
epoll fd, signal-wakeup-fd, eventfds, sockets, IPC
|
|
||||||
pipes, pytest's capture-fds, etc. are all still in the
|
|
||||||
child's fd table (kernel-level inheritance). Handled by
|
|
||||||
`_close_inherited_fds()` in the child prelude — walks
|
|
||||||
`/proc/self/fd` and closes everything except stdio +
|
|
||||||
the channel pipe to the forkserver.
|
|
||||||
- **Memory image** — trio's internal data structures
|
|
||||||
(scheduler, task queues, runner state) sit in COW
|
|
||||||
memory but nobody's executing them. Get GC'd /
|
|
||||||
overwritten when the child's fresh `trio.run()` boots.
|
|
||||||
- **Python thread state** — handled automatically by
|
|
||||||
CPython. `PyOS_AfterFork_Child()` calls
|
|
||||||
`_PyThreadState_DeleteExceptCurrent()`, so dead
|
|
||||||
`PyThreadState` objects are cleaned and
|
|
||||||
`threading.enumerate()` returns just the surviving
|
|
||||||
thread.
|
|
||||||
- **User-level locks (`threading.Lock`)** —
|
|
||||||
held-by-dead-thread state is the canonical fork hazard.
|
|
||||||
Not an issue in practice for tractor: trio doesn't hold
|
|
||||||
cross-thread locks across fork (its synchronization is
|
|
||||||
within the trio task system, which doesn't survive in
|
|
||||||
either direction). CPython's GIL is auto-reset by the
|
|
||||||
fork callback.
|
|
||||||
|
|
||||||
FYI: how this dodges the `trio.run()` × `fork()` hazards
|
|
||||||
--------------------------------------------------------
|
|
||||||
|
|
||||||
`os.fork()` is famously hostile to `trio` (see
|
|
||||||
python-trio/trio#1614 et al.) because trio owns several
|
|
||||||
classes of process-global state that all break across the
|
|
||||||
fork boundary in different ways. The forkserver-thread
|
|
||||||
design dodges each class explicitly:
|
|
||||||
|
|
||||||
- **Signal-wakeup-fd**: trio installs a wakeup-fd via
|
|
||||||
`signal.set_wakeup_fd()` on `trio.run()` startup so
|
|
||||||
signals can interrupt `epoll_wait`. The child inherits
|
|
||||||
this fd, but trio's runner that owns it is gone — so
|
|
||||||
any signal delivery in the child writes to a dead
|
|
||||||
reader. *Dodge*: the inherited wakeup-fd is closed by
|
|
||||||
`_close_inherited_fds()`, then the child's own
|
|
||||||
`trio.run()` installs a fresh one.
|
|
||||||
- **`epoll`/`kqueue` instance**: trio's I/O backend holds
|
|
||||||
one. Inherited as a dead fd; same fix as above.
|
|
||||||
- **Threadpool cache threads** (`trio.to_thread`): worker
|
|
||||||
threads with cached tstate. Don't exist in the child
|
|
||||||
(POSIX); cache state is meaningless garbage that gets
|
|
||||||
reset when the child's trio.run() initializes its own
|
|
||||||
thread cache.
|
|
||||||
- **Cancel scopes / nurseries / open `trio.Process` /
|
|
||||||
open sockets**: these are trio-runtime objects, not
|
|
||||||
kernel objects. The runtime that owns them is gone in
|
|
||||||
the child, so the Python objects exist as zombie data
|
|
||||||
in COW memory and get overwritten as the child runs.
|
|
||||||
Inherited *kernel* fds those objects wrapped (sockets,
|
|
||||||
proc pipes) are caught by `_close_inherited_fds()`.
|
|
||||||
- **`atexit` handlers**: trio doesn't register any that
|
|
||||||
would mis-fire post-fork; trio's lifetime-stack is
|
|
||||||
all `with`-block-scoped and dies with the runner.
|
|
||||||
- **Foreign-language I/O state** (libcurl, OpenSSL session
|
|
||||||
caches, etc.): out of scope — same hazard as any
|
|
||||||
fork-without-exec; users layering those on top of
|
|
||||||
tractor need their own pthread_atfork handlers.
|
|
||||||
|
|
||||||
Net effect: for the runtime surface tractor controls
|
|
||||||
(trio + IPC layer + msgspec), the forkserver-thread
|
|
||||||
isolation + `_close_inherited_fds()` cleanup gives the
|
|
||||||
forked child a clean trio environment. Everything else
|
|
||||||
falls under the standard fork-without-exec disclaimer.
|
|
||||||
|
|
||||||
Implementation status
|
|
||||||
---------------------
|
|
||||||
|
|
||||||
- A dedicated main-interp worker thread owns all `os.fork()`
|
|
||||||
calls (never enters a subint). ✓ landed.
|
|
||||||
- Parent actor's `trio.run()` lives **on the main interp**
|
|
||||||
for now (not a subint yet). The subint-hosted root
|
|
||||||
runtime is the variant-2 step gated on jcrist/msgspec#1026.
|
|
||||||
- Spawn-request signal: trio task `→ to_thread.run_sync` to
|
|
||||||
the forkserver-worker thread. ✓ landed.
|
|
||||||
- Forked child: runs `_actor_child_main` against a normal
|
|
||||||
trio runtime. ✓ landed.
|
|
||||||
|
|
||||||
Validated by `tests/spawn/test_subint_forkserver.py` (file
|
|
||||||
will be renamed to `test_main_thread_forkserver.py` in a
|
|
||||||
follow-up) including the
|
|
||||||
`test_subint_forkserver_spawn_basic` backend-tier check.
|
|
||||||
|
|
||||||
Still-open work (tracked on tractor #379):
|
|
||||||
|
|
||||||
- no cancellation / hard-kill stress coverage yet
|
|
||||||
(counterpart to `tests/test_subint_cancellation.py` for
|
|
||||||
the plain `subint` backend),
|
|
||||||
- `child_sigint='trio'` mode (flag scaffolded below; default
|
|
||||||
is `'ipc'`). Originally intended as a manual SIGINT →
|
|
||||||
trio-cancel bridge, but investigation showed trio's
|
|
||||||
handler IS already correctly installed in the fork-child
|
|
||||||
subactor — the orphan-SIGINT hang is actually a separate
|
|
||||||
bug where trio's event loop stays wedged in `epoll_wait`
|
|
||||||
despite delivery. See
|
|
||||||
`ai/conc-anal/subint_forkserver_orphan_sigint_hang_issue.md`
|
|
||||||
for the full trace + fix directions. Once that root cause
|
|
||||||
is fixed, this flag may end up a no-op / doc-only mode.
|
|
||||||
|
|
||||||
TODO — cleanup gated on msgspec PEP 684 support
|
|
||||||
-----------------------------------------------
|
|
||||||
|
|
||||||
Both worker-thread primitives below allocate a dedicated
|
|
||||||
`threading.Thread` rather than using
|
|
||||||
`trio.to_thread.run_sync()`. That's a cautious design
|
|
||||||
rooted in three distinct-but-entangled issues (GIL
|
|
||||||
starvation from legacy-config subints, tstate-recycling
|
|
||||||
destroy race on trio cache threads, fork-from-main-tstate
|
|
||||||
invariant). Some of those dissolve under PEP 684
|
|
||||||
isolated-mode subints; one requires empirical re-testing
|
|
||||||
to know.
|
|
||||||
|
|
||||||
Full analysis + audit plan in
|
|
||||||
`ai/conc-anal/subint_forkserver_thread_constraints_on_pep684_issue.md`,
|
|
||||||
tracked at #450; gated on jcrist/msgspec#1026.
|
|
||||||
|
|
||||||
What lives here
|
|
||||||
---------------
|
|
||||||
|
|
||||||
Truly generic primitives (tractor-spawn-backend-agnostic):
|
|
||||||
|
|
||||||
- `_close_inherited_fds()` — fd hygiene primitive
|
|
||||||
- `_format_child_exit()` — `waitpid()` status renderer
|
|
||||||
- `wait_child()` — synchronous waitpid wrapper
|
|
||||||
- `fork_from_worker_thread()` — the core fork primitive
|
|
||||||
- `_ForkedProc` — trio-cancellable child-wait shim
|
|
||||||
|
|
||||||
The variant-1 spawn-backend coroutine on top:
|
|
||||||
|
|
||||||
- `main_thread_forkserver_proc()` — SpawnSpec handshake, IPC
|
|
||||||
wiring, lifecycle. Registered as the
|
|
||||||
`'main_thread_forkserver'` (and currently the legacy
|
|
||||||
`'subint_forkserver'`-aliased) entry in
|
|
||||||
`tractor.spawn._spawn._methods`.
|
|
||||||
|
|
||||||
See also
|
|
||||||
--------
|
|
||||||
|
|
||||||
- `tractor.spawn._subint_forkserver` — variant-2 placeholder
|
|
||||||
module; reserved for the future subint-isolated-child
|
|
||||||
runtime once jcrist/msgspec#1026 unblocks.
|
|
||||||
- `tractor.spawn._subint_fork` — the stub for the
|
|
||||||
fork-from-non-main-subint strategy that DIDN'T work (kept
|
|
||||||
in-tree as documentation of the attempt + the CPython-level
|
|
||||||
block).
|
|
||||||
- `ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`
|
|
||||||
— CPython source walkthrough of why fork-from-subint is dead.
|
|
||||||
- `ai/conc-anal/subint_fork_from_main_thread_smoketest.py`
|
|
||||||
— standalone feasibility check (delegates to this module
|
|
||||||
for the primitives it exercises).
|
|
||||||
|
|
||||||
'''
|
|
||||||
from __future__ import annotations
|
|
||||||
import os
|
|
||||||
import signal
|
|
||||||
import sys
|
|
||||||
import threading
|
|
||||||
from functools import partial
|
|
||||||
from typing import (
|
|
||||||
Any,
|
|
||||||
Callable,
|
|
||||||
Literal,
|
|
||||||
TYPE_CHECKING,
|
|
||||||
)
|
|
||||||
|
|
||||||
import trio
|
|
||||||
from trio import TaskStatus
|
|
||||||
|
|
||||||
from tractor.log import get_logger
|
|
||||||
from tractor.msg import (
|
|
||||||
types as msgtypes,
|
|
||||||
pretty_struct,
|
|
||||||
)
|
|
||||||
from tractor.runtime._state import current_actor
|
|
||||||
from tractor.runtime._portal import Portal
|
|
||||||
from ._spawn import (
|
|
||||||
cancel_on_completion,
|
|
||||||
soft_kill,
|
|
||||||
)
|
|
||||||
from ._subint import _has_subints
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from tractor.discovery._addr import UnwrappedAddress
|
|
||||||
from tractor.ipc import (
|
|
||||||
_server,
|
|
||||||
)
|
|
||||||
from tractor.runtime._runtime import Actor
|
|
||||||
from tractor.runtime._supervise import ActorNursery
|
|
||||||
|
|
||||||
|
|
||||||
log = get_logger('tractor')
|
|
||||||
|
|
||||||
|
|
||||||
# Configurable child-side SIGINT handling for forkserver-spawned
|
|
||||||
# subactors. Threaded through `main_thread_forkserver_proc`'s
|
|
||||||
# `proc_kwargs` under the `'child_sigint'` key.
|
|
||||||
#
|
|
||||||
# - `'ipc'` (default, currently the only implemented mode):
|
|
||||||
# child has NO trio-level SIGINT handler — trio.run() is on
|
|
||||||
# the fork-inherited non-main thread, `signal.set_wakeup_fd()`
|
|
||||||
# is main-thread-only. Cancellation flows exclusively via
|
|
||||||
# the parent's `Portal.cancel_actor()` IPC path. Safe +
|
|
||||||
# deterministic for nursery-structured apps where the parent
|
|
||||||
# is always the cancel authority. Known gap: orphan
|
|
||||||
# (post-parent-SIGKILL) children don't respond to SIGINT
|
|
||||||
# — see `test_orphaned_subactor_sigint_cleanup_DRAFT`.
|
|
||||||
#
|
|
||||||
# - `'trio'` (**not yet implemented**): install a manual
|
|
||||||
# SIGINT → trio-cancel bridge in the child's fork prelude
|
|
||||||
# (pre-`trio.run()`) so external Ctrl-C reaches stuck
|
|
||||||
# grandchildren even with a dead parent. Adds signal-
|
|
||||||
# handling surface the `'ipc'` default cleanly avoids; only
|
|
||||||
# pay for it when externally-interruptible children actually
|
|
||||||
# matter (e.g. CLI tool grandchildren).
|
|
||||||
ChildSigintMode = Literal['ipc', 'trio']
|
|
||||||
_DEFAULT_CHILD_SIGINT: ChildSigintMode = 'ipc'
|
|
||||||
|
|
||||||
|
|
||||||
def _close_inherited_fds(
|
|
||||||
keep: frozenset[int] = frozenset({0, 1, 2}),
|
|
||||||
) -> int:
|
|
||||||
'''
|
|
||||||
Close every open file descriptor in the current process
|
|
||||||
EXCEPT those in `keep` (default: stdio only).
|
|
||||||
|
|
||||||
Intended as the first thing a post-`os.fork()` child runs
|
|
||||||
after closing any communication pipes it knows about. This
|
|
||||||
is the fork-child FD hygiene discipline that
|
|
||||||
`subprocess.Popen(close_fds=True)` applies by default for
|
|
||||||
its exec-based children, but which we have to implement
|
|
||||||
ourselves because our `fork_from_worker_thread()` primitive
|
|
||||||
deliberately does NOT exec.
|
|
||||||
|
|
||||||
Why it matters
|
|
||||||
--------------
|
|
||||||
Without this, a forkserver-spawned subactor inherits the
|
|
||||||
parent actor's IPC listener sockets, trio-epoll fd, trio
|
|
||||||
wakeup-pipe, peer-channel sockets, etc. If that subactor
|
|
||||||
then itself forkserver-spawns a grandchild, the grandchild
|
|
||||||
inherits the FDs transitively from *both* its direct
|
|
||||||
parent AND the root actor — IPC message routing becomes
|
|
||||||
ambiguous and the cancel cascade deadlocks. See
|
|
||||||
`ai/conc-anal/subint_forkserver_test_cancellation_leak_issue.md`
|
|
||||||
for the full diagnosis + the empirical repro.
|
|
||||||
|
|
||||||
Fresh children will open their own IPC sockets via
|
|
||||||
`_actor_child_main()`, so they don't need any of the
|
|
||||||
parent's FDs.
|
|
||||||
|
|
||||||
Returns the count of fds that were successfully closed —
|
|
||||||
useful for sanity-check logging at callsites.
|
|
||||||
|
|
||||||
'''
|
|
||||||
# Enumerate open fds via `/proc/self/fd` on Linux (the fast +
|
|
||||||
# precise path); fall back to `RLIMIT_NOFILE` range close on
|
|
||||||
# other platforms. Matches stdlib
|
|
||||||
# `subprocess._posixsubprocess.close_fds` strategy.
|
|
||||||
try:
|
|
||||||
fd_names: list[str] = os.listdir('/proc/self/fd')
|
|
||||||
candidates: list[int] = [
|
|
||||||
int(n) for n in fd_names if n.isdigit()
|
|
||||||
]
|
|
||||||
except (
|
|
||||||
FileNotFoundError,
|
|
||||||
PermissionError,
|
|
||||||
):
|
|
||||||
import resource
|
|
||||||
soft, _ = resource.getrlimit(resource.RLIMIT_NOFILE)
|
|
||||||
candidates = list(range(3, soft))
|
|
||||||
|
|
||||||
closed: int = 0
|
|
||||||
for fd in candidates:
|
|
||||||
if fd in keep:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
os.close(fd)
|
|
||||||
closed += 1
|
|
||||||
except OSError:
|
|
||||||
# fd was already closed (race with listdir) or otherwise
|
|
||||||
# unclosable — either is fine.
|
|
||||||
log.exception(
|
|
||||||
f'Failed to close inherited fd in child ??\n'
|
|
||||||
f'{fd!r}\n'
|
|
||||||
)
|
|
||||||
|
|
||||||
return closed
|
|
||||||
|
|
||||||
|
|
||||||
def _format_child_exit(
|
|
||||||
status: int,
|
|
||||||
) -> str:
|
|
||||||
'''
|
|
||||||
Render `os.waitpid()`-returned status as a short human
|
|
||||||
string (`'rc=0'` / `'signal=SIGABRT'` / etc.) for log
|
|
||||||
output.
|
|
||||||
|
|
||||||
'''
|
|
||||||
if os.WIFEXITED(status):
|
|
||||||
return f'rc={os.WEXITSTATUS(status)}'
|
|
||||||
elif os.WIFSIGNALED(status):
|
|
||||||
sig: int = os.WTERMSIG(status)
|
|
||||||
return f'signal={signal.Signals(sig).name}'
|
|
||||||
else:
|
|
||||||
return f'raw_status={status}'
|
|
||||||
|
|
||||||
|
|
||||||
def wait_child(
|
|
||||||
pid: int,
|
|
||||||
*,
|
|
||||||
expect_exit_ok: bool = True,
|
|
||||||
) -> tuple[bool, str]:
|
|
||||||
'''
|
|
||||||
`os.waitpid()` + classify the child's exit as
|
|
||||||
expected-or-not.
|
|
||||||
|
|
||||||
`expect_exit_ok=True` → expect clean `rc=0`. `False` →
|
|
||||||
expect abnormal death (any signal or nonzero rc). Used
|
|
||||||
by the control-case smoke-test scenario where CPython
|
|
||||||
is meant to abort the child.
|
|
||||||
|
|
||||||
Returns `(ok, status_str)` — `ok` reflects whether the
|
|
||||||
observed outcome matches `expect_exit_ok`, `status_str`
|
|
||||||
is a short render of the actual status.
|
|
||||||
|
|
||||||
'''
|
|
||||||
_, status = os.waitpid(pid, 0)
|
|
||||||
exited_normally: bool = (
|
|
||||||
os.WIFEXITED(status)
|
|
||||||
and
|
|
||||||
os.WEXITSTATUS(status) == 0
|
|
||||||
)
|
|
||||||
ok: bool = (
|
|
||||||
exited_normally
|
|
||||||
if expect_exit_ok
|
|
||||||
else not exited_normally
|
|
||||||
)
|
|
||||||
return ok, _format_child_exit(status)
|
|
||||||
|
|
||||||
|
|
||||||
def fork_from_worker_thread(
|
|
||||||
child_target: Callable[[], int] | None = None,
|
|
||||||
*,
|
|
||||||
thread_name: str = 'main-thread-fork',
|
|
||||||
join_timeout: float = 10.0,
|
|
||||||
|
|
||||||
) -> int:
|
|
||||||
'''
|
|
||||||
`os.fork()` from a main-interp worker thread; return the
|
|
||||||
forked child's pid.
|
|
||||||
|
|
||||||
The calling context **must** be the main interpreter
|
|
||||||
(not a subinterpreter) — that's the whole point of this
|
|
||||||
primitive. A regular `threading.Thread(target=...)`
|
|
||||||
spawned from main-interp code satisfies this
|
|
||||||
automatically because Python attaches the thread's
|
|
||||||
tstate to the *calling* interpreter, and our main
|
|
||||||
thread's calling interp is always main.
|
|
||||||
|
|
||||||
If `child_target` is provided, it runs IN the forked
|
|
||||||
child process before `os._exit` is called. The callable
|
|
||||||
should return an int used as the child's exit rc. If
|
|
||||||
`child_target` is None, the child `_exit(0)`s immediately
|
|
||||||
(useful for the baseline sanity case).
|
|
||||||
|
|
||||||
On the PARENT side, this function drives the worker
|
|
||||||
thread to completion (`fork()` returns near-instantly;
|
|
||||||
the thread is expected to exit promptly) and then
|
|
||||||
returns the forked child's pid. Raises `RuntimeError`
|
|
||||||
if the worker thread fails to return within
|
|
||||||
`join_timeout` seconds — that'd be an unexpected CPython
|
|
||||||
pathology.
|
|
||||||
|
|
||||||
'''
|
|
||||||
# Use a pipe to shuttle the forked child's pid from the
|
|
||||||
# worker thread back to the caller.
|
|
||||||
rfd, wfd = os.pipe()
|
|
||||||
|
|
||||||
def _worker() -> None:
|
|
||||||
'''
|
|
||||||
Runs on the forkserver worker thread. Forks; child
|
|
||||||
runs `child_target` (if any) and exits; parent side
|
|
||||||
writes the child pid to the pipe so the main-thread
|
|
||||||
caller can retrieve it.
|
|
||||||
|
|
||||||
'''
|
|
||||||
pid: int = os.fork()
|
|
||||||
if pid == 0:
|
|
||||||
# CHILD: close the pid-pipe ends (we don't use
|
|
||||||
# them here), then scrub ALL other inherited FDs
|
|
||||||
# so the child starts with a clean slate
|
|
||||||
# (stdio-only). Critical for multi-level spawn
|
|
||||||
# trees — see `_close_inherited_fds()` docstring.
|
|
||||||
os.close(rfd)
|
|
||||||
os.close(wfd)
|
|
||||||
_close_inherited_fds()
|
|
||||||
rc: int = 0
|
|
||||||
if child_target is not None:
|
|
||||||
try:
|
|
||||||
rc = child_target() or 0
|
|
||||||
except BaseException as err:
|
|
||||||
log.error(
|
|
||||||
f'main-thread-fork child_target '
|
|
||||||
f'raised:\n'
|
|
||||||
f'|_{type(err).__name__}: {err}'
|
|
||||||
)
|
|
||||||
rc = 2
|
|
||||||
os._exit(rc)
|
|
||||||
else:
|
|
||||||
# PARENT (still inside the worker thread):
|
|
||||||
# hand the child pid back to main via pipe.
|
|
||||||
os.write(wfd, pid.to_bytes(8, 'little'))
|
|
||||||
|
|
||||||
worker: threading.Thread = threading.Thread(
|
|
||||||
target=_worker,
|
|
||||||
name=thread_name,
|
|
||||||
daemon=False,
|
|
||||||
)
|
|
||||||
worker.start()
|
|
||||||
worker.join(timeout=join_timeout)
|
|
||||||
if worker.is_alive():
|
|
||||||
# Pipe cleanup best-effort before bail.
|
|
||||||
try:
|
|
||||||
os.close(rfd)
|
|
||||||
except OSError:
|
|
||||||
log.exception(
|
|
||||||
f'Failed to close PID-pipe read-fd in parent ??\n'
|
|
||||||
f'{rfd!r}\n'
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
os.close(wfd)
|
|
||||||
except OSError:
|
|
||||||
log.exception(
|
|
||||||
f'Failed to close PID-pipe write-fd in parent ??\n'
|
|
||||||
f'{wfd!r}\n'
|
|
||||||
)
|
|
||||||
raise RuntimeError(
|
|
||||||
f'main-thread-fork worker thread '
|
|
||||||
f'{thread_name!r} did not return within '
|
|
||||||
f'{join_timeout}s — this is unexpected since '
|
|
||||||
f'`os.fork()` should return near-instantly on '
|
|
||||||
f'the parent side.'
|
|
||||||
)
|
|
||||||
|
|
||||||
pid_bytes: bytes = os.read(rfd, 8)
|
|
||||||
os.close(rfd)
|
|
||||||
os.close(wfd)
|
|
||||||
pid: int = int.from_bytes(pid_bytes, 'little')
|
|
||||||
log.runtime(
|
|
||||||
f'main-thread-fork forked child\n'
|
|
||||||
f'(>\n'
|
|
||||||
f' |_pid={pid}\n'
|
|
||||||
)
|
|
||||||
return pid
|
|
||||||
|
|
||||||
|
|
||||||
class _ForkedProc:
|
|
||||||
'''
|
|
||||||
Thin `trio.Process`-compatible shim around a raw OS pid
|
|
||||||
returned by `fork_from_worker_thread()`, exposing just
|
|
||||||
enough surface for the `soft_kill()` / hard-reap pattern
|
|
||||||
borrowed from `trio_proc()`.
|
|
||||||
|
|
||||||
Unlike `trio.Process`, we have no direct handles on the
|
|
||||||
child's std-streams (fork-without-exec inherits the
|
|
||||||
parent's FDs, but we don't marshal them into this
|
|
||||||
wrapper) — `.stdin`/`.stdout`/`.stderr` are all `None`,
|
|
||||||
which matches what `soft_kill()` handles via its
|
|
||||||
`is not None` guards.
|
|
||||||
|
|
||||||
'''
|
|
||||||
def __init__(self, pid: int):
|
|
||||||
self.pid: int = pid
|
|
||||||
self._returncode: int | None = None
|
|
||||||
# `soft_kill`/`hard_kill` check these for pipe
|
|
||||||
# teardown — all None since we didn't wire up pipes
|
|
||||||
# on the fork-without-exec path.
|
|
||||||
self.stdin = None
|
|
||||||
self.stdout = None
|
|
||||||
self.stderr = None
|
|
||||||
# pidfd (Linux 5.3+, Python 3.9+) — a file descriptor
|
|
||||||
# referencing this child process which becomes readable
|
|
||||||
# once the child exits. Enables a fully trio-cancellable
|
|
||||||
# wait via `trio.lowlevel.wait_readable()` — same
|
|
||||||
# pattern `trio.Process.wait()` uses under the hood, and
|
|
||||||
# the same pattern `multiprocessing.Process.sentinel`
|
|
||||||
# uses for `tractor.spawn._spawn.proc_waiter()`. Without
|
|
||||||
# this, waiting via `trio.to_thread.run_sync(os.waitpid,
|
|
||||||
# ...)` blocks a cache thread on a sync syscall that is
|
|
||||||
# NOT trio-cancellable, which prevents outer cancel
|
|
||||||
# scopes from unwedging a stuck-child cancel cascade.
|
|
||||||
self._pidfd: int = os.pidfd_open(pid)
|
|
||||||
|
|
||||||
def poll(self) -> int | None:
|
|
||||||
'''
|
|
||||||
Non-blocking liveness probe. Returns `None` if the
|
|
||||||
child is still running, else its exit code (negative
|
|
||||||
for signal-death, matching `subprocess.Popen`
|
|
||||||
convention).
|
|
||||||
|
|
||||||
'''
|
|
||||||
if self._returncode is not None:
|
|
||||||
return self._returncode
|
|
||||||
try:
|
|
||||||
waited_pid, status = os.waitpid(self.pid, os.WNOHANG)
|
|
||||||
except ChildProcessError:
|
|
||||||
# already reaped (or never existed) — treat as
|
|
||||||
# clean exit for polling purposes.
|
|
||||||
self._returncode = 0
|
|
||||||
return 0
|
|
||||||
if waited_pid == 0:
|
|
||||||
return None
|
|
||||||
self._returncode = self._parse_status(status)
|
|
||||||
return self._returncode
|
|
||||||
|
|
||||||
@property
|
|
||||||
def returncode(self) -> int | None:
|
|
||||||
return self._returncode
|
|
||||||
|
|
||||||
async def wait(self) -> int:
|
|
||||||
'''
|
|
||||||
Async, fully-trio-cancellable wait for the child's
|
|
||||||
exit. Uses `trio.lowlevel.wait_readable()` on the
|
|
||||||
`pidfd` sentinel — same pattern as `trio.Process.wait`
|
|
||||||
and `tractor.spawn._spawn.proc_waiter` (mp backend).
|
|
||||||
|
|
||||||
Safe to call multiple times; subsequent calls return
|
|
||||||
the cached rc without re-issuing the syscall.
|
|
||||||
|
|
||||||
'''
|
|
||||||
if self._returncode is not None:
|
|
||||||
return self._returncode
|
|
||||||
# Park until the pidfd becomes readable — the OS
|
|
||||||
# signals this exactly once on child exit. Cancellable
|
|
||||||
# via any outer trio cancel scope (this was the key
|
|
||||||
# fix vs. the prior `to_thread.run_sync(os.waitpid,
|
|
||||||
# abandon_on_cancel=False)` which blocked a thread on
|
|
||||||
# a sync syscall and swallowed cancels).
|
|
||||||
await trio.lowlevel.wait_readable(self._pidfd)
|
|
||||||
# pidfd signaled → reap non-blocking to collect the
|
|
||||||
# exit status. `WNOHANG` here is correct: by the time
|
|
||||||
# the pidfd is readable, `waitpid()` won't block.
|
|
||||||
try:
|
|
||||||
_, status = os.waitpid(self.pid, os.WNOHANG)
|
|
||||||
except ChildProcessError:
|
|
||||||
# already reaped by something else
|
|
||||||
status = 0
|
|
||||||
self._returncode = self._parse_status(status)
|
|
||||||
# pidfd is one-shot; close it so we don't leak fds
|
|
||||||
# across many spawns.
|
|
||||||
try:
|
|
||||||
os.close(self._pidfd)
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
self._pidfd = -1
|
|
||||||
return self._returncode
|
|
||||||
|
|
||||||
def kill(self) -> None:
|
|
||||||
'''
|
|
||||||
OS-level `SIGKILL` to the child. Swallows
|
|
||||||
`ProcessLookupError` (already dead).
|
|
||||||
|
|
||||||
'''
|
|
||||||
try:
|
|
||||||
os.kill(self.pid, signal.SIGKILL)
|
|
||||||
except ProcessLookupError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def __del__(self) -> None:
|
|
||||||
# belt-and-braces: close the pidfd if `wait()` wasn't
|
|
||||||
# called (e.g. unexpected teardown path).
|
|
||||||
fd: int = getattr(self, '_pidfd', -1)
|
|
||||||
if fd >= 0:
|
|
||||||
try:
|
|
||||||
os.close(fd)
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _parse_status(self, status: int) -> int:
|
|
||||||
if os.WIFEXITED(status):
|
|
||||||
return os.WEXITSTATUS(status)
|
|
||||||
elif os.WIFSIGNALED(status):
|
|
||||||
# negative rc by `subprocess.Popen` convention
|
|
||||||
return -os.WTERMSIG(status)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
def __repr__(self) -> str:
|
|
||||||
return (
|
|
||||||
f'<_ForkedProc pid={self.pid} '
|
|
||||||
f'returncode={self._returncode}>'
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def main_thread_forkserver_proc(
|
|
||||||
name: str,
|
|
||||||
actor_nursery: ActorNursery,
|
|
||||||
subactor: Actor,
|
|
||||||
errors: dict[tuple[str, str], Exception],
|
|
||||||
|
|
||||||
# passed through to actor main
|
|
||||||
bind_addrs: list[UnwrappedAddress],
|
|
||||||
parent_addr: UnwrappedAddress,
|
|
||||||
_runtime_vars: dict[str, Any],
|
|
||||||
*,
|
|
||||||
infect_asyncio: bool = False,
|
|
||||||
task_status: TaskStatus[Portal] = trio.TASK_STATUS_IGNORED,
|
|
||||||
proc_kwargs: dict[str, any] = {},
|
|
||||||
|
|
||||||
) -> None:
|
|
||||||
'''
|
|
||||||
Spawn a subactor via `os.fork()` from a non-trio worker
|
|
||||||
thread (see `fork_from_worker_thread()`), with the forked
|
|
||||||
child running `tractor._child._actor_child_main()` and
|
|
||||||
connecting back via tractor's normal IPC handshake.
|
|
||||||
|
|
||||||
Supervision model mirrors `trio_proc()` — we manage a
|
|
||||||
real OS subprocess, so `Portal.cancel_actor()` +
|
|
||||||
`soft_kill()` on graceful teardown and `os.kill(SIGKILL)`
|
|
||||||
on hard-reap both apply directly (no
|
|
||||||
`_interpreters.destroy()` voodoo needed since the child
|
|
||||||
is in its own process).
|
|
||||||
|
|
||||||
The only real difference from `trio_proc` is the spawn
|
|
||||||
mechanism: fork from a known-clean main-interp worker
|
|
||||||
thread instead of `trio.lowlevel.open_process()`.
|
|
||||||
|
|
||||||
'''
|
|
||||||
if not _has_subints:
|
|
||||||
raise RuntimeError(
|
|
||||||
f'The {"main_thread_forkserver"!r} spawn backend '
|
|
||||||
f'requires Python 3.14+.\n'
|
|
||||||
f'Current runtime: {sys.version}'
|
|
||||||
)
|
|
||||||
|
|
||||||
# Backend-scoped config pulled from `proc_kwargs`. Using
|
|
||||||
# `proc_kwargs` (vs a first-class kwarg on this function)
|
|
||||||
# matches how other backends expose per-spawn tuning
|
|
||||||
# (`trio_proc` threads it to `trio.lowlevel.open_process`,
|
|
||||||
# etc.) and keeps `ActorNursery.start_actor(proc_kwargs=...)`
|
|
||||||
# as the single ergonomic entry point.
|
|
||||||
child_sigint: ChildSigintMode = proc_kwargs.get(
|
|
||||||
'child_sigint',
|
|
||||||
_DEFAULT_CHILD_SIGINT,
|
|
||||||
)
|
|
||||||
if child_sigint not in ('ipc', 'trio'):
|
|
||||||
raise ValueError(
|
|
||||||
f'Invalid `child_sigint={child_sigint!r}` for '
|
|
||||||
f'`main_thread_forkserver` backend.\n'
|
|
||||||
f'Expected one of: {ChildSigintMode}.'
|
|
||||||
)
|
|
||||||
if child_sigint == 'trio':
|
|
||||||
raise NotImplementedError(
|
|
||||||
"`child_sigint='trio'` mode — trio-native SIGINT "
|
|
||||||
"plumbing in the fork-child — is scaffolded but "
|
|
||||||
"not yet implemented. See the xfail'd "
|
|
||||||
"`test_orphaned_subactor_sigint_cleanup_DRAFT` "
|
|
||||||
"and the TODO in this module's docstring."
|
|
||||||
)
|
|
||||||
|
|
||||||
uid: tuple[str, str] = subactor.aid.uid
|
|
||||||
loglevel: str | None = subactor.loglevel
|
|
||||||
|
|
||||||
# Closure captured into the fork-child's memory image.
|
|
||||||
# In the child this is the first post-fork Python code to
|
|
||||||
# run, on what was the fork-worker thread in the parent.
|
|
||||||
# `child_sigint` is captured here so the impl lands inside
|
|
||||||
# this function once the `'trio'` mode is wired up —
|
|
||||||
# nothing above this comment needs to change.
|
|
||||||
def _child_target() -> int:
|
|
||||||
# Dispatch on the captured SIGINT-mode closure var.
|
|
||||||
# Today only `'ipc'` is reachable (the `'trio'` branch
|
|
||||||
# is fenced off at the backend-entry guard above); the
|
|
||||||
# match is in place so the future `'trio'` impl slots
|
|
||||||
# in as a plain case arm without restructuring.
|
|
||||||
match child_sigint:
|
|
||||||
case 'ipc':
|
|
||||||
pass # <- current behavior: no child-side
|
|
||||||
# SIGINT plumbing; rely on parent
|
|
||||||
# `Portal.cancel_actor()` IPC path.
|
|
||||||
case 'trio':
|
|
||||||
# Unreachable today (see entry-guard above);
|
|
||||||
# this stub exists so that lifting the guard
|
|
||||||
# is the only change required to enable
|
|
||||||
# `'trio'` mode once the SIGINT wakeup-fd
|
|
||||||
# bridge is implemented.
|
|
||||||
raise NotImplementedError(
|
|
||||||
"`child_sigint='trio'` fork-prelude "
|
|
||||||
"plumbing not yet wired."
|
|
||||||
)
|
|
||||||
# Lazy import so the parent doesn't pay for it on
|
|
||||||
# every spawn — it's module-level in `_child` but
|
|
||||||
# cheap enough to re-resolve here.
|
|
||||||
from tractor._child import _actor_child_main
|
|
||||||
# XXX, `os.fork()` inherits the parent's entire memory
|
|
||||||
# image, including `tractor.runtime._state._runtime_vars`
|
|
||||||
# (which in the parent encodes "this process IS the root
|
|
||||||
# actor"). A fresh `exec`-based child starts cold; we
|
|
||||||
# replicate that here by explicitly resetting runtime
|
|
||||||
# vars to their fresh-process defaults — otherwise
|
|
||||||
# `Actor.__init__` takes the `is_root_process() == True`
|
|
||||||
# branch, pre-populates `self.enable_modules`, and trips
|
|
||||||
# the `assert not self.enable_modules` gate at the top
|
|
||||||
# of `Actor._from_parent()` on the subsequent parent→
|
|
||||||
# child `SpawnSpec` handshake. (`_state._current_actor`
|
|
||||||
# is unconditionally overwritten by `_trio_main` → no
|
|
||||||
# reset needed for it.)
|
|
||||||
from tractor.runtime._state import (
|
|
||||||
get_runtime_vars,
|
|
||||||
set_runtime_vars,
|
|
||||||
)
|
|
||||||
set_runtime_vars(get_runtime_vars(clear_values=True))
|
|
||||||
_actor_child_main(
|
|
||||||
uid=uid,
|
|
||||||
loglevel=loglevel,
|
|
||||||
parent_addr=parent_addr,
|
|
||||||
infect_asyncio=infect_asyncio,
|
|
||||||
# The child's runtime is trio-native (uses
|
|
||||||
# `_trio_main` + receives `SpawnSpec` over IPC),
|
|
||||||
# but label it with the actual parent-side spawn
|
|
||||||
# mechanism so `Actor.pformat()` / log lines
|
|
||||||
# reflect reality. Downstream runtime gates that
|
|
||||||
# key on `_spawn_method` group `main_thread_forkserver`
|
|
||||||
# alongside `trio`/`subint` where the SpawnSpec
|
|
||||||
# IPC handshake is concerned — see
|
|
||||||
# `runtime._runtime.Actor._from_parent()`.
|
|
||||||
spawn_method='main_thread_forkserver',
|
|
||||||
)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
cancelled_during_spawn: bool = False
|
|
||||||
proc: _ForkedProc | None = None
|
|
||||||
ipc_server: _server.Server = actor_nursery._actor.ipc_server
|
|
||||||
|
|
||||||
try:
|
|
||||||
try:
|
|
||||||
pid: int = await trio.to_thread.run_sync(
|
|
||||||
partial(
|
|
||||||
fork_from_worker_thread,
|
|
||||||
_child_target,
|
|
||||||
thread_name=(
|
|
||||||
f'main-thread-forkserver[{name}]'
|
|
||||||
),
|
|
||||||
),
|
|
||||||
abandon_on_cancel=False,
|
|
||||||
)
|
|
||||||
proc = _ForkedProc(pid)
|
|
||||||
log.runtime(
|
|
||||||
f'Forked subactor via main-thread-forkserver\n'
|
|
||||||
f'(>\n'
|
|
||||||
f' |_{proc}\n'
|
|
||||||
)
|
|
||||||
|
|
||||||
event, chan = await ipc_server.wait_for_peer(uid)
|
|
||||||
|
|
||||||
except trio.Cancelled:
|
|
||||||
cancelled_during_spawn = True
|
|
||||||
raise
|
|
||||||
|
|
||||||
assert proc is not None
|
|
||||||
|
|
||||||
portal = Portal(chan)
|
|
||||||
actor_nursery._children[uid] = (
|
|
||||||
subactor,
|
|
||||||
proc,
|
|
||||||
portal,
|
|
||||||
)
|
|
||||||
|
|
||||||
sspec = msgtypes.SpawnSpec(
|
|
||||||
_parent_main_data=subactor._parent_main_data,
|
|
||||||
enable_modules=subactor.enable_modules,
|
|
||||||
reg_addrs=subactor.reg_addrs,
|
|
||||||
bind_addrs=bind_addrs,
|
|
||||||
_runtime_vars=_runtime_vars,
|
|
||||||
)
|
|
||||||
log.runtime(
|
|
||||||
f'Sending spawn spec to forkserver child\n'
|
|
||||||
f'{{}}=> {chan.aid.reprol()!r}\n'
|
|
||||||
f'\n'
|
|
||||||
f'{pretty_struct.pformat(sspec)}\n'
|
|
||||||
)
|
|
||||||
await chan.send(sspec)
|
|
||||||
|
|
||||||
curr_actor: Actor = current_actor()
|
|
||||||
curr_actor._actoruid2nursery[uid] = actor_nursery
|
|
||||||
|
|
||||||
task_status.started(portal)
|
|
||||||
|
|
||||||
with trio.CancelScope(shield=True):
|
|
||||||
await actor_nursery._join_procs.wait()
|
|
||||||
|
|
||||||
async with trio.open_nursery() as nursery:
|
|
||||||
if portal in actor_nursery._cancel_after_result_on_exit:
|
|
||||||
nursery.start_soon(
|
|
||||||
cancel_on_completion,
|
|
||||||
portal,
|
|
||||||
subactor,
|
|
||||||
errors,
|
|
||||||
)
|
|
||||||
|
|
||||||
# reuse `trio_proc`'s soft-kill dance — `proc`
|
|
||||||
# is our `_ForkedProc` shim which implements the
|
|
||||||
# same `.poll()` / `.wait()` / `.kill()` surface
|
|
||||||
# `soft_kill` expects.
|
|
||||||
await soft_kill(
|
|
||||||
proc,
|
|
||||||
_ForkedProc.wait,
|
|
||||||
portal,
|
|
||||||
)
|
|
||||||
nursery.cancel_scope.cancel()
|
|
||||||
|
|
||||||
finally:
|
|
||||||
# Hard reap: SIGKILL + waitpid. Cheap since we have
|
|
||||||
# the real OS pid, unlike `subint_proc` which has to
|
|
||||||
# fuss with `_interpreters.destroy()` races.
|
|
||||||
if proc is not None and proc.poll() is None:
|
|
||||||
log.cancel(
|
|
||||||
f'Hard killing main-thread-forkserver subactor\n'
|
|
||||||
f'>x)\n'
|
|
||||||
f' |_{proc}\n'
|
|
||||||
)
|
|
||||||
with trio.CancelScope(shield=True):
|
|
||||||
proc.kill()
|
|
||||||
await proc.wait()
|
|
||||||
|
|
||||||
if not cancelled_during_spawn:
|
|
||||||
actor_nursery._children.pop(uid, None)
|
|
||||||
|
|
@ -77,14 +77,7 @@ SpawnMethodKey = Literal[
|
||||||
# is CPython-legal and works cleanly; forked child runs
|
# is CPython-legal and works cleanly; forked child runs
|
||||||
# `tractor._child._actor_child_main()` against a trio
|
# `tractor._child._actor_child_main()` against a trio
|
||||||
# runtime, exactly like `trio_proc` but via fork instead
|
# runtime, exactly like `trio_proc` but via fork instead
|
||||||
# of subproc-exec. See `tractor.spawn._main_thread_forkserver`.
|
# of subproc-exec. See `tractor.spawn._subint_forkserver`.
|
||||||
'main_thread_forkserver',
|
|
||||||
# RESERVED for the future variant-2 subint-isolated-child
|
|
||||||
# runtime — gated on jcrist/msgspec#1026 + PEP 684. Today
|
|
||||||
# this key aliases to `main_thread_forkserver_proc`; once
|
|
||||||
# the upstream unblocks land it'll dispatch to the
|
|
||||||
# subint-hosted-trio impl. See
|
|
||||||
# `tractor.spawn._subint_forkserver`.
|
|
||||||
'subint_forkserver',
|
'subint_forkserver',
|
||||||
]
|
]
|
||||||
_spawn_method: SpawnMethodKey = 'trio'
|
_spawn_method: SpawnMethodKey = 'trio'
|
||||||
|
|
@ -138,23 +131,14 @@ def try_set_start_method(
|
||||||
case 'trio':
|
case 'trio':
|
||||||
_ctx = None
|
_ctx = None
|
||||||
|
|
||||||
case (
|
case 'subint' | 'subint_fork' | 'subint_forkserver':
|
||||||
'subint'
|
|
||||||
| 'subint_fork'
|
|
||||||
| 'main_thread_forkserver'
|
|
||||||
| 'subint_forkserver'
|
|
||||||
):
|
|
||||||
# All subint-family backends need no `mp.context`;
|
# All subint-family backends need no `mp.context`;
|
||||||
# all four feature-gate on the py3.14 public
|
# all three feature-gate on the py3.14 public
|
||||||
# `concurrent.interpreters` wrapper (PEP 734). See
|
# `concurrent.interpreters` wrapper (PEP 734). See
|
||||||
# `tractor.spawn._subint` for the detailed
|
# `tractor.spawn._subint` for the detailed
|
||||||
# reasoning. `subint_fork` is blocked at the
|
# reasoning. `subint_fork` is blocked at the
|
||||||
# CPython level (raises `NotImplementedError`);
|
# CPython level (raises `NotImplementedError`);
|
||||||
# `main_thread_forkserver` is the working
|
# `subint_forkserver` is the working workaround.
|
||||||
# variant-1 backend; `subint_forkserver` aliases
|
|
||||||
# to it today, reserved for the future variant-2
|
|
||||||
# subint-isolated-child runtime once upstream
|
|
||||||
# msgspec#1026 unblocks.
|
|
||||||
from ._subint import _has_subints
|
from ._subint import _has_subints
|
||||||
if not _has_subints:
|
if not _has_subints:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
|
|
@ -493,7 +477,7 @@ from ._trio import trio_proc
|
||||||
from ._mp import mp_proc
|
from ._mp import mp_proc
|
||||||
from ._subint import subint_proc
|
from ._subint import subint_proc
|
||||||
from ._subint_fork import subint_fork_proc
|
from ._subint_fork import subint_fork_proc
|
||||||
from ._main_thread_forkserver import main_thread_forkserver_proc
|
from ._subint_forkserver import subint_forkserver_proc
|
||||||
|
|
||||||
|
|
||||||
# proc spawning backend target map
|
# proc spawning backend target map
|
||||||
|
|
@ -508,19 +492,8 @@ _methods: dict[SpawnMethodKey, Callable] = {
|
||||||
# clean `NotImplementedError` with pointer to the analysis,
|
# clean `NotImplementedError` with pointer to the analysis,
|
||||||
# rather than an "invalid backend" error.
|
# rather than an "invalid backend" error.
|
||||||
'subint_fork': subint_fork_proc,
|
'subint_fork': subint_fork_proc,
|
||||||
# Variant-1 (working today): fork from a regular main-interp
|
# WIP — fork-from-non-trio-worker-thread, works on py3.14+
|
||||||
# worker thread, child runs trio on its own main interp.
|
# (validated via `ai/conc-anal/subint_fork_from_main_thread_smoketest.py`).
|
||||||
# Validated by
|
# See `tractor.spawn._subint_forkserver`.
|
||||||
# `ai/conc-anal/subint_fork_from_main_thread_smoketest.py`.
|
'subint_forkserver': subint_forkserver_proc,
|
||||||
# See `tractor.spawn._main_thread_forkserver`.
|
|
||||||
'main_thread_forkserver': main_thread_forkserver_proc,
|
|
||||||
# Variant-2 (future, reserved): same fork machinery but
|
|
||||||
# child enters a sub-interpreter to host its `trio.run()`
|
|
||||||
# — gated on jcrist/msgspec#1026 unblocking PEP 684
|
|
||||||
# isolated-mode subints. Today aliases to the variant-1
|
|
||||||
# impl so `--spawn-backend=subint_forkserver` keeps
|
|
||||||
# working; flipped to a `NotImplementedError` stub in a
|
|
||||||
# follow-up commit. See
|
|
||||||
# `tractor.spawn._subint_forkserver`.
|
|
||||||
'subint_forkserver': main_thread_forkserver_proc,
|
|
||||||
}
|
}
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue