Reduce `_subint_forkserver.py` to its variant-2 placeholder shape:
- Add `subint_forkserver_proc` async stub raising `NotImplementedError`
with a redirect msg pointing at the working variant-1 backend
(`main_thread_forkserver`), jcrist/msgspec#1026 (upstream PEP 684
blocker), and #379 (subint umbrella).
- `tractor.spawn._spawn._methods['subint_forkserver']` now dispatches to
the stub instead of aliasing the variant-1 coroutine
— `--spawn-backend=subint_forkserver` errors cleanly.
- Drop now-dead module-scope: `ChildSigintMode`
/ `_DEFAULT_CHILD_SIGINT` defs, `_has_subints` try/except (replaced
with import from `._subint`), unused imports (`partial`, `Literal`,
`sys`, msgtypes/pretty_struct, `current_actor`,
`cancel_on_completion`/`soft_kill`, `_server` TYPE_CHECKING).
- Backward-compat re-exports of fork primitives kept until the follow-up
commit migrates external test imports.
- `tests/spawn/test_subint_forkserver.py::forkserver_spawn_method`
fixture: flip hardcoded `'subint_forkserver'`
→ `'main_thread_forkserver'` so the test still exercises the working
backend (full file rename comes in the test-import migration commit).
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
(cherry picked from commit 5e83881f10)
The `subint_forkserver` name was always aspirational —
today's impl forks from a regular main-interp worker
thread and the child runs trio on its own main interp;
NO subinterp anywhere in parent or child. Splitting the
backend into two clearly-named variants drops the lie:
- **variant 1** — `main_thread_forkserver` (the working
impl). New `SpawnMethodKey` literal + `_methods`
dispatch entry + `_runtime.Actor._from_parent()`
match-arm. The spawn-coro `subint_forkserver_proc`
moves to `_main_thread_forkserver` and is renamed
`main_thread_forkserver_proc()`.
- **variant 2** — `subint_forkserver` (future, reserved).
Module shrinks to a placeholder describing the
variant-2 design (subint-isolated child runtime, gated
on jcrist/msgspec#1026 + PEP 684). Today the legacy
`'subint_forkserver'` key aliases to
`main_thread_forkserver_proc` so existing
`--spawn-backend=subint_forkserver` invocations keep
working; flipped to a `NotImplementedError` stub in a
follow-up.
Deats,
- `Actor._from_parent()` spawn-method gate now accepts
both `'main_thread_forkserver'` and
`'subint_forkserver'` (both go through the
IPC-`SpawnSpec` path).
- the variant-1 spawn-coro stamps its own `SpawnSpec` /
log lines with `spawn_method='main_thread_forkserver'`
so subactor renders reflect the actual mechanism.
- docstring reorg: trio×fork hazard breakdown, POSIX
fork-survival semantics, in-process-vs-stdlib
forkserver design notes, and the TODO/cleanup section
all move from `_subint_forkserver` to
`_main_thread_forkserver` (lives with the working
code). `_subint_forkserver` keeps a tight forward-
looking doc that motivates the reserved key.
- `run_subint_in_worker_thread()` stays in
`_subint_forkserver` as the companion primitive — it's
the subint counterpart to `fork_from_worker_thread()`
and will plug into the future variant-2 spawn-coro.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
(cherry picked from commit 57dae0e4a6)
Promote `_subint_forkserver` from primitives-only into a
registered spawn backend: `'subint_forkserver'` is now a
`SpawnMethodKey` literal, dispatched via `_methods` to
the new `subint_forkserver_proc()` target, feature-gated
under the existing `subint`-family py3.14+ case, and
selectable via `--spawn-backend=subint_forkserver`.
Deats,
- new `subint_forkserver_proc()` spawn target in
`_subint_forkserver`:
- mirrors `trio_proc()`'s supervision model — real OS
subprocess so `Portal.cancel_actor()` + `soft_kill()`
on graceful teardown, `os.kill(SIGKILL)` on hard-reap
(no `_interpreters.destroy()` race to fuss over bc the
child lives in its own process)
- only real diff from `trio_proc` is the spawn mechanism:
fork from a main-interp worker thread via
`fork_from_worker_thread()` (off-loaded to trio's
thread pool) instead of `trio.lowlevel.open_process()`
- child-side `_child_target` closure runs
`tractor._child._actor_child_main()` with
`spawn_method='trio'` — the child is a regular trio
actor, "subint_forkserver" names how the parent
spawned, not what the child runs
- new `_ForkedProc` class — thin `trio.Process`-compatible
shim around a raw OS pid: `.poll()` via
`waitpid(WNOHANG)`, async `.wait()` off-loaded to a trio
cache thread, `.kill()` via `SIGKILL`, `.returncode`
cached for repeat calls. `.stdin`/`.stdout`/`.stderr`
are `None` (fork-w/o-exec inherits parent FDs; we don't
marshal them) which matches `soft_kill()`'s `is not None`
guards
Also, new backend-tier test
`test_subint_forkserver_spawn_basic` drives the registered
backend end-to-end via `open_root_actor` + `open_nursery` +
`run_in_actor` w/ a trivial portal-RPC round-trip. Uses a
`forkserver_spawn_method` fixture to flip
`_spawn_method`/`_ctx` for the test's duration + restore on
teardown (so other session-level tests don't observe the
global flip). Test module docstring reworked to describe
the three tiers now covered: (1) primitive-level, (2)
parent-trio-driven primitives, (3) full registered backend.
Status: still-open work (tracked on `tractor#379`) doc'd
inline in the module docstring — no cancel/hard-kill stress
coverage yet, child-side subint-hosted root runtime still
future (gated on `msgspec#563`), thread-hygiene audit
pending the same unblock.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
(cherry picked from commit 26914fde75)
Empirical finding: the WIP `subint_fork_proc` scaffold
landed in `cf0e3e6f` does *not* work on current CPython.
The `fork()` syscall succeeds in the parent, but the
CHILD aborts immediately during
`PyOS_AfterFork_Child()` →
`_PyInterpreterState_DeleteExceptMain()`, which gates
on the current tstate belonging to the main interp —
the child dies with `Fatal Python error: not main
interpreter`.
CPython devs acknowledge the fragility with an in-source
comment (`// Ideally we could guarantee tstate is running
main.`) but expose no user-facing hook to satisfy the
precondition — so the strategy is structurally dead until
upstream changes.
Rather than delete the scaffold, reshape it into a
documented dead-end so the next person with this idea
lands on the reason rather than rediscovering the same
CPython-level refusal.
Deats,
- Move `subint_fork_proc` out of `tractor.spawn._subint`
into a new `tractor.spawn._subint_fork` dedicated
module (153 LOC). Module + fn docstrings now describe
the blockage directly; the fn body is trimmed to a
`NotImplementedError` pointing at the analysis doc —
no more dead-code `bootstrap` sketch bloating
`_subint.py`.
- `_spawn.py`: keep `'subint_fork'` in `SpawnMethodKey`
+ the `_methods` dispatch so
`--spawn-backend=subint_fork` routes to a clean
`NotImplementedError` rather than "invalid backend";
comment calls out the blockage. Collapse the duplicate
py3.14 feature-gate in `try_set_start_method()` into a
combined `case 'subint' | 'subint_fork':` arm.
- New 337-line analysis:
`ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`.
Annotated walkthrough from the user-visible fatal
error down to the specific `Modules/posixmodule.c` +
`Python/pystate.c` source lines enforcing the refusal,
plus an upstream-report draft.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
(cherry picked from commit 0f48ed2eb9)
The private `_interpreters` C module ships since 3.13, but that vintage
wedges under our `threading.Thread` + multi-trio usage pattern
—> `_interpreters.exec()` silently never makes progress. 3.14 fixes it.
So gate on the presence of the public `concurrent.interpreters` wrapper
(3.14+ only) even tho we still call into the private module at runtime.
Deats,
- `try_set_start_method('subint')` error msg + `_subint` module
docstring/comments rewritten to document the 3.14 floor and why 3.13
can't work.
- `_subint._has_subints` gate now imports `concurrent.interpreters` (not
`_interpreters`) as the version sentinel.
Also, reshuffle `pyproject.toml` deps into
per-python-version `[tool.uv.dependency-groups]`:
- `subints` group: `msgspec>=0.21.0`, py>=3.14
- `eventfd` group: `cffi>=1.17.1`, py>=3.13,<3.14
- `sync_pause` group: `greenback`, py>=3.13,<3.14
(was in `devx`; moved out bc no 3.14 yet)
Bump top-level `msgspec>=0.20.0` too.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
(cherry picked from commit 34d9d482e4)
(MTF-only portion: kept tractor/spawn/_spawn.py tractor/spawn/_subint.py)
Expand the comment block above the `_interpreters`
import explaining *why* we use the private C mod
over `concurrent.interpreters`: the public API only
exposes PEP 734's `'isolated'` config which breaks
`msgspec` (missing PEP 684 slot). Add reference
links to PEP 734, PEP 684, cpython sources, and
the msgspec upstream tracker (jcrist/msgspec#563).
Also,
- update error msgs in both `_spawn.py` and
`_subint.py` to say "3.13+" (matching the actual
`_interpreters` availability) instead of "3.14+".
- tweak the mod docstring to reflect py3.13+
availability via the private C module.
Review: PR #444 (copilot-pull-request-reviewer)
https://github.com/goodboy/tractor/pull/444
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
(cherry picked from commit 8a8d01e076)
Land the scaffolding for a future sub-interpreter (PEP 734
`concurrent.interpreters`) actor spawn backend per issue #379. The
spawn flow itself is not yet implemented; `subint_proc()` raises a
placeholder `NotImplementedError` pointing at the tracking issue —
this commit only wires up the registry, the py-version gate, and
the harness.
Deats,
- bump `pyproject.toml` `requires-python` to `>=3.12, <3.15` and
list the `3.14` classifier — the new stdlib
`concurrent.interpreters` module only ships on 3.14
- extend `SpawnMethodKey = Literal[..., 'subint']`
- `try_set_start_method('subint')` grows a new `match` arm that
feature-detects the stdlib module and raises `RuntimeError` with
a clear banner on py<3.14
- `_methods` registers the new `subint_proc()` via the same
bottom-of-module late-import pattern used for `._trio` / `._mp`
Also,
- new `tractor/spawn/_subint.py` — top-level `try: from concurrent
import interpreters` guards `_has_subints: bool`; `subint_proc()`
signature mirrors `trio_proc`/`mp_proc` so the Phase B.2 impl can
drop in without touching the registry
- re-add `import sys` to `_spawn.py` (needed for the py-version msg
in the gate-error)
- `_testing.pytest.pytest_configure` wraps `try_set_start_method()`
in a `pytest.UsageError` handler so `--spawn-backend=subint` on
py<3.14 prints a clean banner instead of a traceback
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
(cherry picked from commit d318f1f8f4)
(MTF-only portion: kept tractor/spawn/_spawn.py tractor/spawn/_subint.py)
Inside a new new `tractor.spawn._reap` submod which kicks off providing
post-mortem subactor cleanup primitives, parent-side; consider it the
"sibling" of `tractor._testing._reap` which is the test-harness-oriented
brother mod.
Today: `unlink_uds_bind_addrs()` provides a starter bug-fix for #454
where `hard_kill()`'s `SIGKILL` bypasses the subactor's
`_serve_ipc_eps`-`finally:` `os.unlink(addr.sockpath)`, leaking
`${XDG_RUNTIME_DIR}/tractor/<name>@<pid>.sock` files..
This adds 2 cleanup paths:
- explicit `bind_addrs` (when set at spawn time),
OR
- convention-based reconstruction from `subactor.aid.name + proc.pid`
for the random-self-assign case.
`.spawn.hard_kill()` now invokes the cleanup unconditionally
post-`SIGKILL`; graceful-exit case is a no-op via `FileNotFoundError`
skip.
Future work — authoritative tracking via a per-process
UDS bind-addr registry — documented in module docstring,
deferred to a follow-up PR.
Co-fix: `tractor/spawn/_trio.py::new_proc` already passes
`bind_addrs` + `subactor` to `hard_kill` via prior work
on this branch.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
(cherry picked from commit e9712dcaeb)
Drop unused `TYPE_CHECKING` imports (`Channel`,
`_server`), remove commented-out `import os` in
`_entry.py`, and use `get_runtime_vars()` accessor
instead of bare `_runtime_vars` in `_trio.py`.
Also,
- freshen `__init__.py` layout docstring for the
new per-backend submod structure
- update `_spawn.py` + `_trio.py` module docstrings
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Split the monolithic `spawn._spawn` into a slim
"core" + per-backend submodules so a future
`._subint` backend (per issue #379) can drop in
without piling more onto `_spawn.py`.
`._spawn` retains the cross-backend supervisor
machinery: `SpawnMethodKey`, `_methods` registry,
`_spawn_method`/`_ctx` state, `try_set_start_method()`,
the `new_proc()` dispatcher, and the shared helpers
`exhaust_portal()`, `cancel_on_completion()`,
`hard_kill()`, `soft_kill()`, `proc_waiter()`.
Deats,
- mv `trio_proc()` → new `spawn._trio`
- mv `mp_proc()` → new `spawn._mp`, reads `_ctx` and
`_spawn_method` via `from . import _spawn` for
late binding bc both get mutated by
`try_set_start_method()`
- `_methods` wires up the new submods via late
bottom-of-module imports to side-step circular
dep (both backend mods pull shared helpers from
`._spawn`)
- prune now-unused imports from `_spawn.py` — `sys`,
`is_root_process`, `current_actor`,
`is_main_process`, `_mp_main`, `ActorFailure`,
`pretty_struct`, `_pformat`
Also,
- `_testing.pytest.pytest_generate_tests()` now
drives the valid-backend set from
`typing.get_args(SpawnMethodKey)` so adding a
new backend (e.g. `'subint'`) doesn't require
touching the harness
- refresh `spawn/__init__.py` docstring for the
new layout
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Restructure the flat `tractor/` top-level private mods
into (more nested) subpackages:
- `runtime/`: `_runtime`, `_portal`, `_rpc`, `_state`,
`_supervise`
- `spawn/`: `_spawn`, `_entry`, `_forkserver_override`,
`_mp_fixup_main`
- `discovery/`: `_addr`, `_discovery`, `_multiaddr`
Each subpkg `__init__.py` is kept lazy (no eager
imports) to avoid circular import issues.
Also,
- update all intra-pkg imports across ~35 mods to use
the new subpkg paths (e.g. `from .runtime._state`
instead of `from ._state`)
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Gud Boi