Compare commits

..

No commits in common. "645e25c0411287bfa0a6c9fcf3f804a377b84ec5" and "c43df5064793a7d0ca555eb2a1aec4c7f0975518" have entirely different histories.

65 changed files with 454 additions and 4734 deletions

View File

@ -1,41 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-27T23:25:00Z
git_ref: d067505a
scope: code
substantive: true
raw_file: 20260827T232500Z_d067505a_prompt_io.raw.md
---
## Prompt
After committing the reviewed bindspace-reference changes, the human
asked to continue with the next Layer C increment.
## Response summary
Added the private Linux network-namespace entry primitive required by
future child bootstrap wiring. It verifies the inherited capability
before `setns()`, constrains entry to network namespaces, and verifies
the resulting process namespace before runtime initialization proceeds.
## Files changed
- `tractor/spawn/_netns.py` - synchronous verified netns entry.
- `tests/test_netns_spawn.py` - stale-FD, syscall and post-entry guards.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - primitive ownership and
remaining integration boundary.
## Human edits
The human requested the next incremental implementation after accepting
the prior split commits. Their established preference for end-to-end
tests informed use of real FDs where possible; only privileged `setns()`
and procfs observation are replaced to keep the suite unprivileged. In
staged review, the human requested the full project license header,
explicit comments on both intentional inode mismatches, and independent
call traces proving both fake syscall and procfs hooks execute. The
agent applied those human-directed revisions; no direct manual source
edits were observed.

View File

@ -1,25 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-27T23:25:00Z
git_ref: d067505a
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - verify pre-runtime netns entry
After committing realized bindspace references and their deferred codec
plan, the human requested the next Layer C increment.
> `git diff HEAD~1..HEAD -- tractor/spawn/_netns.py tests/test_netns_spawn.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added a private synchronous `enter_netns()` bootstrap primitive. It
validates the inherited FD and expected inode, constrains `setns()` to
`CLONE_NEWNET`, verifies `/proc/self/ns/net` afterward, and leaves FD
closure to the future spawn-bootstrap caller.
Tests use real stand-in FDs while replacing only the privileged syscall
and post-entry procfs observation. They prove stale FDs fail before
entry, the syscall receives the exact namespace type, and bootstrap
rejects an unexpected post-entry namespace. Ruff and all three focused
tests passed.

View File

@ -1,38 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
session: wg-netns-bootstrap-entry
timestamp: 2026-08-28T01:22:05Z
git_ref: 9ae7cd86
scope: code
substantive: true
raw_file: 20260828T012205Z_9ae7cd86_prompt_io.raw.md
---
## Prompt
After completing and auditing the stacked rebase, the human approved
the next WireGuard Layer C slice: integrate the verified network
namespace entry primitive into child spawn bootstrap.
## Response summary
Wire an optional child-owned namespace FD and expected inode into both
child entrypoints. Keep namespace entry and FD closure synchronous and
ordered before actor runtime initialization, while deliberately
deferring backend FD transport and public API plumbing.
## Files changed
- `tractor/spawn/_entry.py` - enter and release an optional inherited
namespace capability before multiprocessing child runtime startup.
- `tractor/_child.py` - consume the capability at the start of the
Trio child bootloader, before patching or actor construction.
- `tests/test_netns_spawn.py` - prove entry ordering, failure isolation,
and exact FD ownership for both child entrypoints.
## Human edits
The human selected spawn-time namespace integration as the next work
slice and authorized implementation. No direct manual source edits
were observed during generation.

View File

@ -1,36 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T01:22:05Z
git_ref: 9ae7cd86
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - enter inherited netns during child bootstrap
The human asked to continue the WireGuard Layer C implementation after
the rebased branch landed the verified `spawn._netns.enter_netns()`
primitive.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/_child.py tractor/spawn/_entry.py tests/test_netns_spawn.py`
Add one optional atomic `(namespace_fd, expected_inode)` bootstrap
capability to the multiprocessing entrypoint and Trio child bootloader.
Enter and verify the namespace before Trio patching, actor construction,
runtime state, logging, Trio startup, parent connection, sockets, or
worker threads, then close the child-owned inherited FD before
continuing.
Add privilege-free direct-entrypoint tests using real stand-in FDs and
a fake namespace syscall boundary. Cover helper-level no-op behavior,
successful entry ordering, exact FD closure, malformed-FD isolation,
primary-error preservation, and failure cleanup for both child
bootstrap paths.
## Scope boundary
This increment does not transfer FDs through spawn backends or expose a
public actor API. Backend-specific FD duplication, bootstrap failure
reporting, and root-process namespace entry remain follow-up work.

View File

@ -1,47 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
session: trio-pre-handshake-child-death
timestamp: 2026-08-28T04:51:19Z
git_ref: fb6d81d3
scope: code
substantive: true
raw_file: 20260828T045119Z_fb6d81d3_prompt_io.raw.md
---
## Prompt
The human approved the next WireGuard Layer C implementation step after
committing child-side network namespace bootstrap.
## Response summary
Restore and harden the boot-race design originally implemented by
`3b0724eba85b4014170ed95773e1e41a60d5c513` before transferring
namespace FDs. Race the initial peer handshake against child-process
death so a pre-connection bootstrap failure surfaces instead of
parking `ActorNursery.start_actor()` forever.
## Files changed
- `tractor/spawn/_spawn.py` - provide the generic handshake/process
death race.
- `tractor/spawn/_trio.py` - use the race during Trio child startup.
- `tests/test_spawning.py` - cover handshake-first and death-first
schedules, simultaneous outcomes, waiter errors, and backend cleanup.
## Human edits
The human approved continued Layer C implementation. The agent narrowed
the next slice to failure detection after analysis showed FD transfer
would otherwise introduce a parent hang. The human then identified the
earlier `wait_for_peer_or_proc_death()` implementation and required
explicit credit; the source docstring and commit message now cite its
original commit rather than a later cherry-pick. The human directly
reflowed `_trio.py` to the project line limit and normalized nearby
quotes. The human also required naming the race helper in its lifecycle
comment, explicit `trio.Cancelled` handling in waiter tests, expected
cancellation diagnostics, and a `trio.Process`-constrained test double.
After mainline portability verification, the human chose to commit the
fix on the current WireGuard branch and require its cherry-pick onto
`main` as a blocker in the eventual WireGuard pull request.

View File

@ -1,34 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T04:51:19Z
git_ref: fb6d81d3
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - detect child death before parent handshake
The human asked to continue WireGuard Layer C after child entrypoints
learned to consume inherited network namespace capabilities.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/spawn/_spawn.py tractor/spawn/_trio.py tests/test_spawning.py`
Race a spawned child's initial parent handshake against its process
wait operation. Return the established channel when the handshake wins;
raise `ActorFailure` with the exit status when the process dies first.
Use the race in the Trio exec backend instead of waiting indefinitely
on `IPCServer.wait_for_peer()`.
Add deterministic Trio tests for handshake-first, death-first,
simultaneous, and waiter-error schedules, controlling readiness with
events rather than sleeps. Exercise the full Trio backend with a fake
dead process to prove pre-publication event registration and exact
failed-startup cleanup.
## Scope boundary
This prerequisite prevents pre-connection namespace-entry failures
from hanging their parent. Namespace FD transfer itself remains the
next commit.

View File

@ -1,53 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
session: trio-bindspace-fd-transport
timestamp: 2026-08-28T17:29:43Z
git_ref: 2ca8c570
scope: code
substantive: true
raw_file: 20260828T172943Z_2ca8c570_prompt_io.raw.md
---
## Prompt
After committing generic pre-handshake child-death detection, the human
approved the proposed next Layer C slice: transport a live bindspace
capability through the Trio spawn backend.
## Response summary
Plumb an optional process-local `Bindspace` through actor spawn APIs.
Give a Trio exec child its own inherited namespace descriptor, preserve
existing caller `pass_fds`, transport the FD/inode pair through the CLI,
and close the temporary parent descriptor on every open-process outcome.
Reject unsupported multiprocessing transport explicitly. Exercise the
complete Trio path with a real actor that enters a distinct network
namespace and reports its namespace and inherited-FD inodes over RPC.
## Files changed
- `tractor/runtime/_supervise.py` - accept and relay a child bindspace.
- `tractor/to_actor/_api.py` - expose bindspace spawn configuration.
- `tractor/spawn/_spawn.py` - relay the bindspace to spawn backends.
- `tractor/spawn/_trio.py` - duplicate, transfer, and close the FD.
- `tractor/spawn/_mp.py` - reject unsupported MP transport.
- `tractor/_child.py` - parse and forward the atomic CLI capability.
- `tests/test_netns_spawn.py` - cover transport and cleanup schedules.
## Human edits
The human approved the Trio-first transport boundary after reviewing
the proposed dependency order, then supplied staged-diff comments across
multiple review passes. Human-directed revisions replaced abstract
ownership wording with concrete FD and process terminology, clarified
descriptor cleanup, preserved existing `pass_fds` explicitly, and
replaced the mocked success-path test with a real child actor, real
`setns()`, handshake, and RPC round trip inside unprivileged user/network
namespaces. Follow-up review added direct parent FD-table verification
that the temporary child-bound duplicate is closed after successful
spawn.
Multiprocessing FD reduction and root-process namespace entry remain
separate follow-up slices. No direct manual source edits were observed;
the review-driven source revisions were applied by the agent.

View File

@ -1,34 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T17:29:43Z
git_ref: 2ca8c570
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - transport bindspace FD through Trio spawn
The human approved the next WireGuard Layer C increment after landing
pre-handshake child-death detection.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/_child.py tractor/runtime/_supervise.py tractor/spawn/_spawn.py tractor/spawn/_trio.py tractor/spawn/_mp.py tractor/to_actor/_api.py tests/test_netns_spawn.py`
Thread an optional live `Bindspace` through actor spawn APIs. For the
Trio backend, duplicate its namespace FD, append that temporary
descriptor to existing `pass_fds`, and send the atomic descriptor/inode
pair through the child CLI. Close the parent temporary duplicate as
soon as `open_process()` returns, raises, or is cancelled, while the
child consumes its inherited descriptor before runtime startup.
Reject bindspace transport explicitly in multiprocessing backends until
their descriptor-reduction slice is implemented. Add privilege-free
tests for CLI forwarding, merged pass-FD state, caller immutability,
source/duplicate ownership, exec failure, cancellation, absent live
authority, death-before-handshake cleanup, and MP rejection.
## Scope boundary
Multiprocessing FD reduction and root-process namespace entry remain
follow-up work.

View File

@ -1,47 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-net-api-extraction
timestamp: 2026-08-30T02:52:01Z
git_ref: b1f6ade8
scope: code
substantive: true
raw_file: 20260830T025201Z_b1f6ade8_prompt_io.raw.md
---
## Prompt
After rebasing the WireGuard Layer C stack, the human questioned the
placement of bindspace APIs under `tractor.discovery`, considered new
public package names, selected `tractor.net`, and authorized the
implementation.
## Response summary
Extracted bindspace and tunnel implementations into a lazy
`tractor.net` package. Moved public multiaddr, bindspace, tunnel, and
WireGuard symbols out of `tractor.discovery`, updated internal imports,
tests, examples, and documentation, and added cold-import regressions
that keep optional networking dependencies off the root import path.
Current-netns attachment pins `/proc/thread-self/ns/net` so calls from
non-leader threads retain the caller's namespace rather than the process
leader's.
## Files changed
- `tractor/net/` - lazy public network API and implementation modules.
- `tractor/discovery/` - retain actor discovery and internal address parsing.
- `tractor/__init__.py` - expose `tractor.net` lazily.
- `tests/net/` - network declaration and lifecycle coverage.
- `tests/test_lazy_imports.py` - enforce the lazy package boundary.
- `examples/multihost/wg_lan/` - use the public network package.
- `docs/` - document network APIs separately from actor discovery.
## Human edits
The human rejected `tractor.discovery` as the long-term public home,
considered tunnel- and namespace-specific alternatives, and selected
the broader `tractor.net` boundary because bindspaces may include plain
netns, WireGuard, VRF, veth, and later network resources. The agent
applied the resulting source changes; no direct manual edits were
observed.

View File

@ -1,29 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-30T02:52:01Z
git_ref: b1f6ade8
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - extract public network APIs
The human questioned whether bindspace and WireGuard lifecycle APIs
belonged under actor discovery, selected the proposed `tractor.net`
boundary, and authorized implementation.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/net tractor/discovery tractor/__init__.py`
Move bindspace and tunnel implementations into a lazy public network
package. Keep actor discovery focused on registry and lookup behavior,
while exposing bindspace, tunnel, WireGuard, and multiaddr declarations
through `tractor.net` without loading optional networking dependencies
during `import tractor`.
> `git diff HEAD~1..HEAD -- tests/net tests/test_lazy_imports.py tests/ipc examples/multihost/wg_lan docs`
Move network-focused tests to `tests/net`, update internal and public
imports, verify lazy symbol resolution and removed discovery exports,
and document the new package boundary.

View File

@ -1,52 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
session: root-bindspace-bootstrap
timestamp: 2026-08-30T02:52:02Z
git_ref: b1f6ade8
scope: code
substantive: true
raw_file: 20260830T025202Z_b1f6ade8_prompt_io.raw.md
---
## Prompt
The human chose to defer multiprocessing bindspace FD transport because
that backend may be removed, selected root/single-actor namespace
bootstrap as the next stage, approved caller-thread restoration and
public `tractor.net.open_wg_bindspace()` composition, and authorized
implementation after the stack rebase.
## Response summary
Added `bindspace=` to `tractor.open_root_actor()`. Root startup now
duplicates and validates the live namespace FD, enters before any
debugger, registry, IPC, or actor-runtime work, and synchronously
restores the caller thread's original namespace after complete root
teardown. Added deterministic failure/cancellation coverage, a real UDS
root actor E2E across two network namespaces, and an executable public
WireGuard-bindspace/root composition regression. Bound roots reject the
persistent `mp_forkserver` backend because a helper from an earlier
runtime may retain a stale namespace; the public annotation also remains
runtime-resolvable without importing `tractor.net` eagerly.
## Files changed
- `tractor/_root.py` - public root bindspace lifecycle integration.
- `tractor/spawn/_netns.py` - temporary thread-local netns enter/restore.
- `tests/test_netns_spawn.py` - deterministic and real root regressions.
- `tests/net/test_wg_iface_lifecycle.py` - public root composition test.
- `docs/api/net.rst` - root ownership and restoration contract.
## Human edits
The human explicitly deferred multiprocessing support, selected root
bootstrap as the next implementation stage, approved the requirement to
restore the caller's original namespace so owned teardown can complete,
and approved composition through public `tractor.net` contexts rather
than a new one-shot convenience API. The human later rewrote
`_enter_netns_temporarily()` to separate restoration from FD cleanup and
make the transition flow easier to inspect. The agent retained that
intent with nested lexical FD ownership while fixing setup-error masking,
conditional restoration, and duplicate-FD cleanup found during the
requested audit.

View File

@ -1,31 +0,0 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-30T02:52:02Z
git_ref: b1f6ade8
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - bootstrap the root actor in a bindspace
The human deferred multiprocessing descriptor transport, selected root
namespace bootstrap as the next Layer C stage, approved enter/restore
semantics and public `tractor.net` composition, and authorized
implementation.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/_root.py tractor/spawn/_netns.py`
Add an optional realized `Bindspace` to `open_root_actor()`. Duplicate
its namespace FD, enter before debugger, registry, IPC, or runtime
startup, and restore the caller thread's original namespace after root
teardown. Preserve caller FD ownership and primary errors across
restoration and close failures.
> `git diff HEAD~1..HEAD -- tests/test_netns_spawn.py tests/net/test_wg_iface_lifecycle.py docs/api/net.rst`
Exercise same-netns behavior, body errors, cancellation, restoration
failures, missing and closed FDs, real root entry/restoration under an
unprivileged user namespace, and public WireGuard-bindspace/root
composition.

View File

@ -1,70 +0,0 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: dependency-neutral-discovery-types
timestamp: 2026-09-03T20:15:30Z
git_ref: d097090f
scope: code
substantive: true
raw_file: 20260903T201530Z_d097090f_discovery_types.raw.md
---
## Prompt
The maintainer rejected annotation-only model names rebound to `Any`,
questioned the self-referential `get_type_hints()` regression, selected a
dependency-neutral `tractor.discovery.types` module as the preferred
direction, and then requested implementation of that first design layer.
The maintainer separately asked how to handle the disliked staged
`tractor/net/_bootstrap.py` implementation. That file was excluded from
this implementation boundary and retained as review evidence.
## Response summary
Added a dependency-neutral module for canonical primitive address
descriptors and separate structural contracts for concrete transport
addresses and tunnel-capable address declarations. Migrated discovery,
IPC, root, runtime, and tunnel annotations to import those real types,
removing every runtime model-name assignment to `Any`.
Kept parser table aliases beside `parse_endpoints()` instead of turning
the new module into a general alias registry. Kept transport-specific
factories on TCP and UDS classes because their accepted descriptor and
bindspace types differ. Replaced the old `get_type_hints()` assertions
with cold-import and canonical-identity coverage.
## Files changed
- `tractor/discovery/types.py` - dependency-neutral descriptors and
address protocols.
- `tractor/discovery/_addr.py` - imports and re-exports canonical types;
retains concrete factory registry.
- `tractor/discovery/_multiaddr.py` - uses the structural declaration
type and runtime-safe parsed table alias.
- `tractor/ipc/_tcp.py` - imports the real TCP descriptor alias.
- `tractor/ipc/_uds.py` - imports the real Unix descriptor alias.
- `tractor/ipc/_transport.py` - imports the concrete-address protocol
without cycling through `_addr`.
- `tractor/ipc/_chan.py` - accepts structural address declarations
without a `TunnelledAddress = Any` fallback.
- `tractor/ipc/_server.py` - retains declaration metadata structurally
and routes reconstruction through the concrete address registry.
- `tractor/net/_tunnel.py` - imports real dependency-neutral address
types while retaining dynamic `Any` only for external netlink data.
- `tractor/_root.py` - removes the `Bindspace = Any` fallback and uses
concrete address-factory dispatch.
- `tractor/runtime/_runtime.py` - infers concrete registry address types
for random listener allocation.
- `tests/test_lazy_imports.py` - replaces workaround introspection with
dependency and canonical-identity checks.
## Human edits
The maintainer identified the `Any` aliases and their introspection test
as an unjustified circular contract, directed investigation of the real
cycle/import-performance cause, selected `tractor.discovery.types`, and
approved beginning implementation. The maintainer also required that the
current bootstrap implementation not be treated as acceptable design and
provided direct review annotations which remain preserved in the
worktree.

View File

@ -1,34 +0,0 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-09-03T20:15:30Z
git_ref: d097090f
diff_cmd: git diff HEAD~1..HEAD
---
Implemented a dependency-neutral discovery typing layer after reviewing
the address, transport, tunnel, and lazy-import dependency graph.
> `git diff HEAD~1..HEAD -- tractor/discovery/types.py tractor/discovery/_addr.py tractor/discovery/_multiaddr.py tractor/ipc/_tcp.py tractor/ipc/_uds.py tractor/ipc/_transport.py tractor/ipc/_chan.py tractor/ipc/_server.py tractor/net/_tunnel.py tractor/_root.py tractor/runtime/_runtime.py tests/test_lazy_imports.py`
The generated changes move primitive address descriptors and structural
address protocols into `tractor.discovery.types`. Concrete transport
factories remain outside the shared protocol because TCP and UDS accept
different descriptor and bindspace inputs. Runtime names are no longer
rebound to `Any` merely to satisfy `typing.get_type_hints()`.
The old annotation-introspection regression was replaced with coverage
that checks dependency-neutral cold imports and canonical type identity
across discovery, IPC, and tunnel modules.
Verification completed with the worktree-local Python 3.13 environment:
- focused address/lazy/tunnel coverage: 69 passed;
- discovery, IPC, and local coverage: 77 passed, 2 xpassed;
- root/runtime coverage: 6 passed with 24 multiprocessing fork
deprecation warnings.
Ruff was unavailable in the existing `py313` environment. Scoped
`git diff --check` passed. The global whitespace check reports only a
pre-existing maintainer annotation in `tractor/net/_bootstrap.py`, which
was deliberately left untouched.

View File

@ -1,39 +0,0 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_fa04e8396ffeEUnWUrYF734CSM
timestamp: 2026-09-07T01:15:33Z
git_ref: 5966e8aa
scope: code
substantive: true
raw_file: 20260907T011533Z_5966e8aa_frozen_process_local.raw.md
---
## Prompt
After landing the dependency-neutral discovery typing layer, the
maintainer approved isolating `FrozenProcessLocal` as the next clean
prerequisite before revisiting bindspace and bootstrap design.
## Response summary
Added and publicly exported the immutable process-local marker. Moved
its behavioral coverage out of dependence on bootstrap models by adding
focused test structs and regressions to the existing process-local test
module. The tests demonstrate default wire rejection, frozen struct
fields, shallow nested mutability, rejection of sentinel replacement and
`omit_defaults=True`, and guarded subclass validation hooks which cannot
replace the sentinel.
## Files changed
- `tractor/msg/_local.py` - immutable process-local marker.
- `tractor/msg/__init__.py` - public marker export.
- `tests/msg/test_process_local.py` - independent marker contract.
## Human edits
The maintainer rejected incremental salvage of the current bootstrap
planner, approved `FrozenProcessLocal` as an independent prerequisite,
and directed implementation before the bindspace re-evaluation. Existing
bootstrap review annotations and artifacts remain untouched.

View File

@ -1,25 +0,0 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-09-07T01:15:33Z
git_ref: 5966e8aa
diff_cmd: git diff HEAD~1..HEAD
---
Implemented `FrozenProcessLocal` as an independent prerequisite rather
than coupling its contract to the rejected bootstrap planner.
> `git diff HEAD~1..HEAD -- tractor/msg/_local.py tractor/msg/__init__.py tests/msg/test_process_local.py`
The marker inherits `ProcessLocal` wire rejection and msgspec's frozen
field behavior. It is exported from `tractor.msg`, while focused tests
prove direct and nested encoding rejection, field immutability, and the
intentional shallow treatment of referenced mutable values.
Construction now rejects sentinel replacement and `omit_defaults=True`,
and wraps subclass validation hooks with checks before and after their
execution so they cannot skip or replace the base marker.
Verification completed with 44 process-local, WG-config, bindspace, and
tunnel-address tests passing. Ruff and scoped whitespace checks also
passed.

View File

@ -47,7 +47,7 @@ Here is a small example from piker,
We should take whatever common API is needed to support this and
distill it into a
```python
tractor.net.parse_endpoints(
tractor.discovery.parse_endpoints(
) -> dict[
str,
list[Address]

View File

@ -135,7 +135,7 @@ examples in gh #482) used a *suffix* form
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. That parses, but it is
semantically inverted: it puts the overlay addr where the bearer
belongs, `tcp` where wg's `udp` `ListenPort` goes, and declares
no overlay endpoint at all. `tractor.net.parse_wg_maddr()`
no overlay endpoint at all. `tractor.discovery.parse_wg_maddr()`
now rejects it with an actionable error.
Observed protocol-name lists, for writing the `match`:
@ -203,7 +203,7 @@ Observed protocol-name lists, for writing the `match`:
### 3.3 pure parser helpers + explicit verification
The parser/key-codec helpers and async production verifier live in
`tractor/net/_tunnel.py`; parsing remains pure while verification
`tractor/discovery/_tunnel.py`; parsing remains pure while verification
is an explicit, impure caller step:
```python
@ -620,12 +620,6 @@ server bound in the old namespace.
- the child spawn/bootstrap trampoline calls `setns()` **before**
`_runtime.async_main()`, `IPCServer.listen_on()`, parent-channel
connection, or creation of any worker thread/socket.
- `spawn._netns.enter_netns()` is the first private bootstrap
primitive: it checks the inherited FD against the expected inode,
calls `setns(fd, CLONE_NEWNET)`, and verifies
`/proc/self/ns/net` before returning. It deliberately does not own
or close the FD; spawn propagation and status reporting remain the
caller's next integration boundary.
- only after successful entry does the child drop namespace-entry
privileges and initialize the actor runtime.
- a root/single-actor process follows the same ordering: enter during

View File

@ -53,7 +53,6 @@ Most-used names at a glance:
core
context
discovery
net
errors
msg
trionics

View File

@ -85,7 +85,6 @@ already distributed-system aware.
.. seealso::
:doc:`/explain/architecture` for the transport/server
internals, :doc:`/api/net` for network declarations,
:doc:`/api/discovery` for how channel addresses get registered
and found, and :doc:`/api/msg` for the codec layer every channel
speaks.
internals, :doc:`/api/discovery` for how channel addresses
get registered and found, and :doc:`/api/msg` for the codec
layer every channel speaks.

View File

@ -1,95 +0,0 @@
Network declarations and lifecycles
===================================
``tractor.net`` provides address composition, bindspace declarations,
and tunnel configuration. The package is lazy: importing
``tractor`` or ``tractor.net`` does not load multiaddr, WireGuard, or
pyroute2 implementation modules until a public symbol is used.
Multiaddr helpers
-----------------
.. currentmodule:: tractor.net
.. autofunction:: mk_maddr
.. autofunction:: parse_maddr
.. autofunction:: parse_endpoints
Bindspaces
----------
.. autoclass:: BindspaceSpec
.. autoclass:: BindspaceRef
.. autoclass:: Bindspace
.. autofunction:: attach_netns
.. autofunction:: open_netns
.. autofunction:: open_bindspace
Root actor composition
----------------------
A live :class:`Bindspace` can scope the root actor itself. Compose the
bindspace manager outside :func:`tractor.open_root_actor` so its network
namespace remains pinned for the complete actor runtime::
async with tractor.net.open_wg_bindspace(
bindspace_spec=bindspace_spec,
layers=layers,
role='listen',
) as bindspace:
async with tractor.open_root_actor(
bindspace=bindspace,
enable_transports=['uds'],
) as root_actor:
...
Root entry happens before registry probes, IPC listeners, runtime sockets,
or actor startup. On every exit, including cancellation or a body error,
the calling thread is restored to its original network namespace before
``open_root_actor()`` returns. The root context duplicates the live
``Bindspace.namespace_fd`` and never consumes or closes the descriptor
owned by ``open_wg_bindspace()``. Default child processes inherit the root
namespace naturally; passing an explicit alternate child ``bindspace``
continues to use that spawn backend's existing behavior. Bound roots reject
the persistent ``mp_forkserver`` backend because a server started by an
earlier runtime may retain that runtime's network namespace.
This is the current low-level composition API. A future convenience API
may accept a tunnel-bearing multiaddr, realize its WireGuard bindspace
internally, and supply that live capability to root startup.
Tunnels and WireGuard
---------------------
.. autoclass:: TunnelledAddress
.. autoclass:: WGTunnelSpec
.. autoclass:: WGInterfaceConfig
.. autoclass:: WGPeerConfig
.. autofunction:: parse_wg_maddr
.. autofunction:: mk_wg_maddr
.. autofunction:: strip_tunnels
.. autofunction:: tunnels_of
.. autofunction:: open_wg_iface
.. autofunction:: open_wg_bindspace
.. autofunction:: read_wg_pubkey
.. autofunction:: read_wg_peers
.. autofunction:: verify_wg_peer

View File

@ -264,7 +264,7 @@ terminology is retired: it's *registrar*/*registry* everywhere now
substitute "registrar" and you're up to date.
.. note::
Multihoming nerds: ``tractor.net`` ships
Multihoming nerds: ``tractor.discovery`` also ships
libp2p-style *multiaddr* helpers — ``mk_maddr()`` and
``parse_maddr()`` — for describing transport endpoints as
structured strings.

View File

@ -120,7 +120,7 @@ ping -c1 10.0.11.1 # from B
```bash
python -c "
from tractor.net import mb_pubkey
from tractor.discovery import mb_pubkey
key = open('wg_pub.key').read().strip()
print(mb_pubkey(key))
"
@ -198,23 +198,9 @@ Four corrections, all from
`_addr_to_transport` wants a `MsgTransport` per addr-type,
which `wg` doesn't have.
## root composition
## next
The `TunnelledAddress`, native maddr parser, bindspace lifecycle, and
explicit pyroute2 verification APIs live in `tractor.net`. Keep the
owning bindspace context outside the root actor so its namespace FD
remains live through complete actor teardown:
```python
async with tractor.net.open_wg_bindspace(
bindspace_spec,
layers,
role='listen',
) as bindspace:
async with tractor.open_root_actor(bindspace=bindspace):
...
```
The root actor enters before registry or IPC setup and restores the
calling thread's original namespace before the outer bindspace context
removes owned WireGuard and netns resources.
Layer A's `TunnelledAddress` and native maddr parser plus Layer B's
explicit pyroute2 verification now live in `tractor.discovery`. Next,
add `open_bindspace()` `@acm`s which create/tear down the iface and
netns.

View File

@ -10,7 +10,7 @@ from __future__ import annotations
import tractor
import trio
from tractor.net import (
from tractor.discovery import (
TunnelledAddress,
mk_maddr,
parse_wg_maddr,

View File

@ -8,7 +8,7 @@ from __future__ import annotations
import tractor
import trio
from tractor.net import (
from tractor.discovery import (
TunnelledAddress,
parse_wg_maddr,
verify_wg_peer,

View File

@ -13,7 +13,7 @@ import msgspec
import pytest
import trio
from tractor.net import (
from tractor.discovery import (
Bindspace,
BindspaceOwnership,
BindspaceRef,
@ -23,7 +23,7 @@ from tractor.net import (
open_bindspace,
open_netns,
)
from tractor.net import _bindspace
from tractor.discovery import _bindspace
from tractor.msg import ProcessLocal
@ -261,11 +261,10 @@ def test_open_bindspace_attaches_current_netns() -> None:
'''
The unnamed spec must borrow and pin the caller's current netns.
Opening `/proc/self/ns/net` could pin the thread-group leader's
namespace when this context runs from another thread. Prove the
implementation selects `/proc/thread-self/ns/net`, records the
calling thread's stable inode and borrowed ownership, then closes
the exact descriptor without altering the namespace itself.
Open `/proc/self/ns/net`, prove the yielded bindspace records its
stable inode and borrowed ownership, then exit the context and
prove the exact descriptor was closed without altering the
namespace itself.
'''
async def main() -> int:
@ -277,9 +276,6 @@ def test_open_bindspace_attaches_current_netns() -> None:
kind='netns',
)
assert spec.key is CURRENT_NETNS
assert _bindspace._THREAD_NETNS == Path(
'/proc/thread-self/ns/net'
)
async with open_bindspace(spec) as bindspace:
namespace_fd: int|None = bindspace.namespace_fd
assert namespace_fd is not None

View File

@ -1,6 +1,7 @@
'''
Multiaddr construction, parsing, and round-trip tests for
`tractor.net.mk_maddr()` and `tractor.net.parse_maddr()`.
`tractor.discovery._multiaddr.mk_maddr()` and
`tractor.discovery._multiaddr.parse_maddr()`.
'''
from pathlib import Path
@ -9,20 +10,20 @@ from types import SimpleNamespace
import pytest
from multiaddr import Multiaddr
from tractor.net import (
from tractor.discovery import (
TunnelledAddress,
WGTunnelSpec,
mb_pubkey,
mk_wg_maddr,
mk_maddr,
parse_endpoints,
parse_maddr,
parse_wg_maddr,
tunnels_of,
)
from tractor.ipc._tcp import TCPAddress
from tractor.ipc._uds import UDSAddress
from tractor.discovery._multiaddr import (
mk_maddr,
parse_maddr,
parse_endpoints,
_tpt_proto_to_maddr,
_maddr_to_tpt_proto,
)

View File

@ -18,7 +18,7 @@ from tractor.devx import dump_on_hang
from tractor.trionics import collapse_eg
from tractor._testing import tractor_test
from tractor.discovery._addr import wrap_address
from tractor.net import mk_maddr
from tractor.discovery._multiaddr import mk_maddr
import trio

View File

@ -24,7 +24,7 @@ from tractor.discovery import _api
from tractor.discovery._addr import (
wrap_address,
)
from tractor.net import mk_maddr
from tractor.discovery._multiaddr import mk_maddr
from tractor.ipc import _connect_chan
from tractor._testing.addr import get_rando_addr

View File

@ -14,7 +14,7 @@ from __future__ import annotations
import msgspec
import pytest
from tractor.net import (
from tractor.discovery import (
BindspaceRef,
TunnelledAddress,
WGTunnelSpec,

View File

@ -7,7 +7,7 @@ from __future__ import annotations
import msgspec
import pytest
from tractor.net import (
from tractor.discovery import (
WGInterfaceConfig,
WGPeerConfig,
)

View File

@ -8,14 +8,12 @@ from collections.abc import AsyncIterator
from contextlib import asynccontextmanager as acm
import os
from pathlib import Path
import sys
from typing import BinaryIO
import pytest
import trio
import tractor
from tractor.net import (
from tractor.discovery import (
Bindspace,
BindspaceRef,
BindspaceSpec,
@ -25,7 +23,7 @@ from tractor.net import (
open_wg_bindspace,
open_wg_iface,
)
from tractor.net import _tunnel
from tractor.discovery import _tunnel
_LOCAL_KEY: str = 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
@ -245,8 +243,6 @@ def test_open_wg_iface_shields_cancelled_cleanup(
def test_open_wg_bindspace_nests_resource_lifetimes(
tmp_path: Path,
request: pytest.FixtureRequest,
monkeypatch: pytest.MonkeyPatch,
) -> None:
'''
@ -272,16 +268,14 @@ def test_open_wg_bindspace_nests_resource_lifetimes(
bindspace_spec: BindspaceSpec = BindspaceSpec(
kind='netns',
)
token = (tmp_path / 'netns').open('w+b')
request.addfinalizer(token.close)
bindspace: Bindspace = Bindspace(
spec=bindspace_spec,
ref=BindspaceRef(
kind='netns',
key=None,
inode=os.fstat(token.fileno()).st_ino,
inode=1,
),
namespace_fd=token.fileno(),
namespace_fd=None,
ownership='borrowed',
)
outer_spec: WGTunnelSpec = WGTunnelSpec(
@ -382,105 +376,3 @@ def test_open_wg_bindspace_nests_resource_lifetimes(
'wg-outer-exit',
'bindspace-exit',
]
@pytest.mark.skipif(
sys.platform != 'linux',
reason='network namespaces are Linux-only',
)
def test_public_wg_bindspace_scopes_child_actor(
monkeypatch: pytest.MonkeyPatch,
tpt_proto: str,
) -> None:
'''
Public network contexts must fully enclose supervised children.
Attach the real current netns through `tractor.net`, fake only WG
interface provisioning, and supervise a child in the yielded
`Bindspace`. The nursery reaps it before network-resource teardown.
'''
events: list[str] = []
bindspace_spec: BindspaceSpec = BindspaceSpec(
kind='netns',
lifecycle='attach',
)
tunnel_spec: WGTunnelSpec = WGTunnelSpec(
peer_pubkey=_PEER_KEY,
iface='wg-root',
)
config: WGInterfaceConfig = WGInterfaceConfig(
private_key=_LOCAL_KEY,
)
@acm
async def fake_open_wg_iface(
spec: WGTunnelSpec,
iface_config: WGInterfaceConfig,
bindspace: Bindspace,
role: _tunnel.WGRole,
) -> AsyncIterator[WGTunnelSpec]:
'''
Trace one WG layer around the real root actor lifetime.
'''
assert spec is tunnel_spec
assert iface_config is config
assert role == 'listen'
assert bindspace.namespace_fd is not None
events.append('wg-enter')
try:
yield spec
finally:
events.append('wg-exit')
monkeypatch.setattr(
_tunnel,
'open_wg_iface',
fake_open_wg_iface,
)
async def main() -> None:
'''
Compose the public network and root actor context managers.
'''
async with tractor.net.open_wg_bindspace(
bindspace_spec=bindspace_spec,
layers=((tunnel_spec, config),),
role='listen',
) as bindspace:
events.append('bindspace-open')
async with tractor.open_root_actor(
start_method='trio',
enable_transports=[tpt_proto],
):
events.append('root-open')
async with tractor.open_nursery() as nursery:
portal = await nursery.start_actor(
'scoped-worker',
bindspace=bindspace,
parent_addr=tractor.current_actor().accept_addr,
registry_addrs=tractor.current_actor().reg_addrs,
)
await portal.cancel_actor()
assert bindspace.namespace_fd is not None
assert os.fstat(
bindspace.namespace_fd,
).st_ino == bindspace.ref.inode
assert Path(
'/proc/thread-self/ns/net'
).stat().st_ino == bindspace.ref.inode
events.append('root-closed')
events.append('bindspace-closed')
trio.run(main)
assert events == [
'wg-enter',
'bindspace-open',
'root-open',
'root-closed',
'wg-exit',
'bindspace-closed',
]

View File

@ -13,13 +13,13 @@ from typing import (
import pytest
import trio
from tractor.net import (
from tractor.discovery import (
read_wg_peers,
read_wg_pubkey,
verify_wg_peer,
WGTunnelSpec,
)
from tractor.net import _tunnel
from tractor.discovery import _tunnel
pyroute2: Any = pytest.importorskip('pyroute2')

View File

@ -7,7 +7,7 @@ from __future__ import annotations
import pytest
import trio
from tractor.net import (
from tractor.discovery import (
TunnelledAddress,
WGTunnelSpec,
tunnels_of,

View File

@ -6,7 +6,7 @@ from __future__ import annotations
import trio
from tractor.net import (
from tractor.discovery import (
BindspaceRef,
TunnelledAddress,
WGTunnelSpec,

View File

@ -7,100 +7,17 @@ from __future__ import annotations
import msgspec
import pytest
from tractor.msg import (
FrozenProcessLocal,
ProcessLocal,
)
from tractor.msg import ProcessLocal
class LocalHandle(ProcessLocal):
'''
Default marker whose sentinel must fail direct and nested
encoding.
Minimal process-local struct used to exercise the global marker.
'''
resource_id: int
class FrozenLocalHandle(FrozenProcessLocal):
'''
Frozen outer fields whose referenced `labels` list remains
mutable.
Supplying `_process_local=None` would also replace the
unsupported sentinel with an encodable value if construction did
not reject it.
'''
resource_id: int
# Mutating this list distinguishes shallow struct freezing from
# recursively freezing every referenced object.
labels: list[str]
# This class option would omit the default-valued sentinel from
# msgpack.
class OmittingLocalHandle(
ProcessLocal,
omit_defaults=True,
):
'''
Encoder config which would drop the default-valued sentinel.
With `omit_defaults=True`, both `_process_local` and
`resource_id`
equal their defaults, so msgspec could encode this struct as an
empty map without ever traversing `_ProcessLocalToken`.
'''
resource_id: int = 1
class ValidatedLocalHandle(ProcessLocal):
'''
Subclass validator which would shadow the marker's post-init
hook.
`ProcessLocal.__init_subclass__()` must wrap this method so
sentinel validation still runs without requiring a cooperative
`super()` call, while retaining this resource-id validation.
'''
resource_id: int
# Defining this hook normally shadows an inherited
# `__post_init__`; `ProcessLocal` must wrap it rather than rely
# on a `super()` call.
def __post_init__(self) -> None:
if self.resource_id < 0:
raise ValueError('resource_id must be non-negative')
class MutatingFrozenLocalHandle(FrozenProcessLocal):
'''
Hook which uses msgspec's frozen-field escape hatch on the
sentinel.
A check only before this hook would miss the replacement and
leave the completed struct encodable, so the wrapper must check
again after subclass validation.
'''
resource_id: int
# Replacing the sentinel inside this hook defeats a pre-hook-only
# check even though the struct is frozen.
def __post_init__(self) -> None:
# This is msgspec's supported internal mutation path for
# frozen structs, and therefore the strongest sentinel-
# replacement case.
msgspec.structs.force_setattr(
self,
'_process_local',
None,
)
@pytest.mark.parametrize(
'nested',
(
@ -138,74 +55,3 @@ def test_process_local_rejects_default_encoding(
match='_ProcessLocalToken.*unsupported',
):
msgspec.msgpack.encode(value)
def test_frozen_process_local_contract() -> None:
'''
Preserve wire rejection while freezing process-local struct
fields.
A plain frozen msgspec struct could otherwise cross actor IPC.
Build one `FrozenProcessLocal` with a mutable referenced list,
then prove direct and nested default encoding still reach
`_ProcessLocalToken`. Reject field replacement while allowing
mutation owned by the nested list, demonstrating that the marker
provides shallow immutability.
'''
handle = FrozenLocalHandle(
resource_id=1,
labels=['initial'],
)
for value in (
handle,
{'nested': [handle]},
):
with pytest.raises(
TypeError,
match='_ProcessLocalToken.*unsupported',
):
msgspec.msgpack.encode(value)
with pytest.raises(AttributeError):
handle.resource_id = 2 # type: ignore[misc]
handle.labels.append('changed')
assert handle.labels == ['initial', 'changed']
def test_process_local_rejects_sentinel_bypasses() -> None:
'''
Prevent constructor and encoder options from removing the
sentinel.
`FrozenLocalHandle` supplies an encodable sentinel replacement;
`OmittingLocalHandle` asks msgspec to omit the default sentinel;
`ValidatedLocalHandle` shadows the inherited post-init hook; and
`MutatingFrozenLocalHandle` replaces the sentinel from inside
that hook. Prove construction rejects each wire-safety bypass
while the ordinary subclass value validator still executes.
'''
with pytest.raises(TypeError, match='internal sentinel'):
FrozenLocalHandle(
resource_id=1,
labels=[],
_process_local=None, # type: ignore[arg-type]
)
with pytest.raises(TypeError, match='omit_defaults'):
OmittingLocalHandle()
with pytest.raises(TypeError, match='internal sentinel'):
ValidatedLocalHandle(
resource_id=1,
_process_local=None, # type: ignore[arg-type]
)
with pytest.raises(ValueError, match='must be non-negative'):
ValidatedLocalHandle(resource_id=-1)
with pytest.raises(TypeError, match='internal sentinel'):
MutatingFrozenLocalHandle(resource_id=1)

View File

@ -1 +0,0 @@
'''Network declaration and lifecycle tests.'''

View File

@ -7,6 +7,20 @@ import os
from statistics import median
import subprocess
import sys
from typing import (
Any,
get_type_hints,
)
from tractor.discovery import (
_addr,
_multiaddr,
)
from tractor.ipc import (
_tcp,
_uds,
)
def run_cold_import(code: str) -> dict[str, object]:
result = subprocess.run(
@ -30,8 +44,8 @@ def test_lazy_to_asyncio_package_api():
Before the lazy conversion, package import side effects exposed
`to_asyncio` to `dir()` and wildcard imports. Exercise those APIs
in cold interpreters so this test proves normal `import tractor`
leaves `asyncio` unloaded, while introspection and wildcard
access still advertise and resolve the public submodule.
leaves `asyncio` unloaded, while discovery and wildcard access
still advertise and resolve the public submodule.
'''
cold = run_cold_import(
@ -88,12 +102,6 @@ def test_cold_import_budget():
'bidict',
'colorlog',
'multiaddr',
'multibase',
'pyroute2',
'tractor.discovery._multiaddr',
'tractor.net',
'tractor.net._bindspace',
'tractor.net._tunnel',
'wrapt',
)
code = (
@ -128,173 +136,25 @@ def test_cold_import_budget():
)
def test_lazy_net_package_api():
def test_lazy_annotation_names_resolve():
'''
Keep the public network package cold until symbol access.
Resolve annotations without importing optional dependencies.
The old discovery re-exports imported bindspace, tunnel,
multiaddr and optional dependencies while initializing a package.
Import `tractor.net` in a clean interpreter, inspect its public
surface, and prove no implementation or optional dependency was
loaded. Then resolve one symbol from each backing module and
prove the facade caches each value while preserving boundaries.
Moving annotation-only third-party names under `TYPE_CHECKING`
left their runtime globals undefined, causing
`typing.get_type_hints()` to raise `NameError`. Resolve every
affected API and prove the lazy aliases retain import-free runtime
introspection.
'''
modules: tuple[str, ...] = (
'tractor.net._bindspace',
'tractor.net._tunnel',
'tractor.discovery._multiaddr',
'multiaddr',
'multibase',
'pyroute2',
)
cold: dict[str, object] = run_cold_import(
'import json, sys; import tractor.net as net; '
f'names = {modules!r}; '
'print(json.dumps({'
'"public": all(name in dir(net) for name in net.__all__), '
'"loaded": [name for name in names if name in sys.modules]'
'}))'
)
assert cold == {
'public': True,
'loaded': [],
}
resolved: dict[str, object] = run_cold_import(
'import json, sys; import tractor.net as net; '
'bindspace = net.BindspaceSpec; '
'bindspace_cached = net.BindspaceSpec is bindspace; '
'maddr = net.mk_maddr; '
'maddr_cached = net.mk_maddr is maddr; '
'tunnel = net.WGTunnelSpec; '
'tunnel_cached = net.WGTunnelSpec is tunnel; '
'print(json.dumps({'
'"bindspace_cached": bindspace_cached, '
'"maddr_cached": maddr_cached, '
'"tunnel_cached": tunnel_cached, '
'"bindspace_module": bindspace.__module__, '
'"maddr_module": maddr.__module__, '
'"tunnel_module": tunnel.__module__, '
'"multiaddr_loaded": "multiaddr" in sys.modules, '
'"pyroute2_loaded": "pyroute2" in sys.modules'
'}))'
)
assert resolved == {
'bindspace_cached': True,
'maddr_cached': True,
'tunnel_cached': True,
'bindspace_module': 'tractor.net._bindspace',
'maddr_module': 'tractor.discovery._multiaddr',
'tunnel_module': 'tractor.net._tunnel',
'multiaddr_loaded': False,
'pyroute2_loaded': False,
}
def test_net_root_export_and_old_discovery_surface():
'''
Publish networking only from its approved namespace.
Before extraction, unshipped network names and implementation
modules lived under `tractor.discovery`. Exercise root attribute
and wildcard access in clean interpreters, proving `tractor.net`
is discoverable and cached without loading implementations. Also
prove the old exports are absent and their modules no longer
resolve, preventing accidental compatibility aliases.
'''
root: dict[str, object] = run_cold_import(
'import json, sys, tractor; '
'advertised = "net" in dir(tractor); '
'net = tractor.net; '
'print(json.dumps({'
'"advertised": advertised, '
'"cached": tractor.net is net, '
'"module": net.__name__, '
'"bindspace_loaded": '
'"tractor.net._bindspace" in sys.modules, '
'"tunnel_loaded": "tractor.net._tunnel" in sys.modules'
'}))'
)
assert root == {
'advertised': True,
'cached': True,
'module': 'tractor.net',
'bindspace_loaded': False,
'tunnel_loaded': False,
}
old: dict[str, object] = run_cold_import(
'import importlib.util, json; '
'import tractor.discovery as discovery; '
'old_names = ("Bindspace", "TunnelledAddress", '
'"mk_maddr", "parse_maddr", "parse_endpoints"); '
'old_modules = ("tractor.discovery._bindspace", '
'"tractor.discovery._tunnel"); '
'print(json.dumps({'
'"exports": [name for name in old_names '
'if hasattr(discovery, name)], '
'"modules": [name for name in old_modules '
'if importlib.util.find_spec(name) is not None]'
'}))'
)
assert old == {
'exports': [],
'modules': [],
}
def test_discovery_types_are_dependency_neutral():
'''
Keep canonical address types independent of runtime implementations.
Annotation-only imports previously rebound unavailable model names
to `Any` so `get_type_hints()` could resolve them. That test only
verified its own workaround and hid the `_addr`/transport import
cycle. Import the canonical declarations in a cold interpreter and
prove they do not pull in optional networking modules. Then prove
each implementation imports the same real declarations instead of
substituting `Any` under the model names.
'''
cold = run_cold_import(
'import json, sys; '
'from tractor.discovery import types; '
'blocked = ('
'"multiaddr", "multibase", "pyroute2", '
'"tractor.discovery._multiaddr", "tractor.net", '
'"tractor.net._bindspace", "tractor.net._tunnel"); '
'print(json.dumps({'
'"address_module": types.Address.__module__, '
'"blocked": [name for name in blocked '
'if name in sys.modules]}))'
)
assert cold == {
'address_module': 'tractor.discovery.types',
'blocked': [],
}
from tractor.discovery import (
_addr,
_multiaddr,
types,
)
from tractor.ipc import (
_chan,
_server,
_tcp,
_transport,
_uds,
)
from tractor.net import _tunnel
assert _addr.Address is types.Address
assert _addr.UnwrappedAddress is types.UnwrappedAddress
assert _multiaddr.AddressDeclaration is types.AddressDeclaration
assert _tcp.TaggedTCPAddress is types.TaggedTCPAddress
assert _uds.TaggedUnixAddress is types.TaggedUnixAddress
assert _transport.Address is types.Address
assert _chan.AddressDeclaration is types.AddressDeclaration
assert _server.AddressDeclaration is types.AddressDeclaration
assert _tunnel.AddressDeclaration is types.AddressDeclaration
assert get_type_hints(_multiaddr.mk_maddr)['return'] is Any
assert get_type_hints(_tcp.MsgpackTCPStream.maddr.fget)[
'return'
] is Any
assert get_type_hints(_uds.MsgpackUDSStream.maddr.fget)[
'return'
] == Any|str
assert get_type_hints(_addr.Address.get_random)[
'current_actor'
] is Any
assert _addr.__annotations__['_address_types'].startswith('dict')

File diff suppressed because it is too large Load Diff

View File

@ -10,25 +10,15 @@ API design.
"""
from functools import partial
from types import SimpleNamespace
from typing import (
Any,
)
from unittest.mock import (
AsyncMock,
MagicMock,
)
import pytest
import trio
import tractor
from tractor._exceptions import ActorFailure
from tractor._testing import tractor_test
from tractor.spawn import (
_spawn,
_trio,
)
data_to_pass_down = {
'doggy': 10,
@ -36,396 +26,6 @@ data_to_pass_down = {
}
def test_peer_handshake_wins_child_boot_race() -> None:
'''
A connected child must cancel process-death monitoring cleanly.
Start the death waiter first and hold it at a checkpoint. Let the
fake server then return one peer event and channel. The helper must
preserve the normal handshake result and cancel the losing process
waiter before its nursery exits. The fake explicitly catches and
re-raises `trio.Cancelled` to prove cancellation caused its exit.
'''
async def main() -> None:
'''
Control the handshake-first schedule with Trio events.
'''
uid: tuple[str, str] = ('handshake-child', 'test')
death_started = trio.Event()
death_cancelled = trio.Event()
peer_event = trio.Event()
channel = object()
async def wait_for_peer(
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Return the peer only after death monitoring is active.
'''
assert child_uid == uid
await death_started.wait()
return (peer_event, channel)
async def wait_for_death() -> int:
'''
Block until the winning handshake cancels this waiter.
'''
death_started.set()
try:
await trio.sleep_forever()
except trio.Cancelled:
death_cancelled.set()
raise
result = await _spawn.wait_for_peer_or_proc_death(
ipc_server=SimpleNamespace(
wait_for_peer=wait_for_peer,
),
uid=uid,
proc_wait=wait_for_death,
proc_repr='handshake-proc',
)
assert result == (peer_event, channel)
assert death_cancelled.is_set()
trio.run(main)
def test_child_death_wins_peer_handshake_race() -> None:
'''
Pre-handshake child death must fail startup instead of hanging.
Start the peer waiter first and leave it parked like
`IPCServer.wait_for_peer()` on an unset event. Return a distinctive
process status from the competing waiter, then prove the helper
cancels the handshake and raises `ActorFailure` with child identity,
status, and process diagnostics.
'''
async def main() -> None:
'''
Control the death-first schedule with Trio events.
'''
uid: tuple[str, str] = ('dead-child', 'test')
handshake_started = trio.Event()
handshake_cancelled = trio.Event()
async def wait_for_peer(
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Park until process death cancels this handshake waiter.
'''
assert child_uid == uid
handshake_started.set()
try:
await trio.sleep_forever()
except trio.Cancelled:
handshake_cancelled.set()
raise
async def wait_for_death() -> int:
'''
Report child death after handshake monitoring is active.
'''
await handshake_started.wait()
return 23
with pytest.raises(ActorFailure) as exc_info:
await _spawn.wait_for_peer_or_proc_death(
ipc_server=SimpleNamespace(
wait_for_peer=wait_for_peer,
),
uid=uid,
proc_wait=wait_for_death,
proc_repr='dead-proc',
)
message: str = str(exc_info.value)
assert repr(uid) in message
assert 'died during boot' in message
assert '(rc=23)' in message
assert 'parent-handshake' in message
assert 'dead-proc' in message
assert handshake_cancelled.is_set()
trio.run(main)
def test_child_death_wins_simultaneous_boot_results() -> None:
'''
Observed process death must outrank a simultaneous handshake.
Hold both fake waits behind one barrier with cancellation shielding,
then release them together so both publish a committed result before
sibling cancellation takes effect. Because the child has exited
before receiving its `SpawnSpec`, bootstrap must raise `ActorFailure`
rather than return its briefly established channel.
'''
async def main() -> None:
'''
Release both boot outcomes from one controlled barrier.
'''
uid: tuple[str, str] = ('simultaneous-child', 'test')
handshake_ready = trio.Event()
death_ready = trio.Event()
release = trio.Event()
peer_event = trio.Event()
async def wait_for_peer(
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Publish a handshake despite sibling cancellation.
'''
assert child_uid == uid
handshake_ready.set()
with trio.CancelScope(shield=True):
await release.wait()
return (peer_event, object())
async def wait_for_death() -> int:
'''
Publish process death despite sibling cancellation.
'''
death_ready.set()
with trio.CancelScope(shield=True):
await release.wait()
return 0
async def release_both() -> None:
'''
Open the barrier only after both waiters are parked.
'''
await handshake_ready.wait()
await death_ready.wait()
release.set()
async with trio.open_nursery() as nursery:
nursery.start_soon(release_both)
with pytest.raises(
ActorFailure,
match=r'simultaneous-child.*rc=0',
):
await _spawn.wait_for_peer_or_proc_death(
ipc_server=SimpleNamespace(
wait_for_peer=wait_for_peer,
),
uid=uid,
proc_wait=wait_for_death,
)
trio.run(main)
@pytest.mark.parametrize('failing_waiter', ('handshake', 'death'))
def test_child_boot_race_preserves_waiter_error(
failing_waiter: str,
) -> None:
'''
Waiter failures must retain their original exception identity.
Park the non-failing sibling and raise one unique error from either
the peer or process waiter. The helper's internal nursery must
cancel the sibling and re-raise that exact exception instead of
wrapping it in an `ExceptionGroup`.
'''
async def main() -> None:
'''
Trigger one selected waiter after its sibling starts.
'''
uid: tuple[str, str] = ('errored-child', 'test')
sibling_started = trio.Event()
wait_error = RuntimeError(f'{failing_waiter} failed')
async def wait_for_peer(
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Raise or park according to the selected peer schedule.
'''
assert child_uid == uid
if failing_waiter == 'handshake':
await sibling_started.wait()
raise wait_error
sibling_started.set()
await trio.sleep_forever()
async def wait_for_death() -> int:
'''
Raise or park according to the selected process schedule.
'''
if failing_waiter == 'death':
await sibling_started.wait()
raise wait_error
sibling_started.set()
await trio.sleep_forever()
with pytest.raises(RuntimeError) as exc_info:
await _spawn.wait_for_peer_or_proc_death(
ipc_server=SimpleNamespace(
wait_for_peer=wait_for_peer,
),
uid=uid,
proc_wait=wait_for_death,
)
assert exc_info.value is wait_error
trio.run(main)
def test_trio_proc_cleans_failed_child_peer_event(
monkeypatch: pytest.MonkeyPatch,
) -> None:
'''
Death-first Trio startup must release provisional peer state.
Return one already-dead fake process while its server handshake
parks forever. The fake nursery proves the peer event exists before
provisional child publication. After `ActorFailure`, both that
exact event and the provisional child record must be gone so repeated
failed spawns cannot leak server state.
'''
uid: tuple[str, str] = ('dead-trio-child', 'test')
proc: trio.Process = MagicMock(spec=trio.Process)
proc.pid = 1234
proc.wait = AsyncMock(return_value=23)
proc.poll.return_value = 23
proc.__str__.return_value = 'dead-trio-proc'
class FakeServer:
'''
Hold the peer registry used during Trio child startup.
'''
def __init__(self) -> None:
self._peer_connected: dict[
tuple[str, str],
trio.Event,
] = {}
async def wait_for_peer(
self,
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Park like a child that never reaches its handshake.
'''
assert child_uid == uid
await trio.sleep_forever()
server = FakeServer()
class FakeNursery:
'''
Track provisional child publication and cleanup.
'''
def __init__(self) -> None:
self._actor = SimpleNamespace(ipc_server=server)
self._children: dict[tuple[str, str], tuple] = {}
def _register_child(
self,
subactor: object,
proc: object,
portal: object|None,
) -> tuple[trio.Event, trio.Event, bool]:
'''
Require peer-event registration before child publication.
'''
assert uid in server._peer_connected
assert portal is None
self._children[uid] = (subactor, proc, portal)
return (trio.Event(), trio.Event(), False)
async def fake_open_process(
command: list[str],
**kwargs: object,
) -> trio.Process:
'''
Return a process whose death wins the bootstrap race.
'''
assert command
return proc
async def fake_wait_for_debugger(**kwargs: object) -> None:
'''
Keep hard-reap cleanup deterministic and non-interactive.
'''
return None
monkeypatch.setattr(
_trio.trio.lowlevel,
'open_process',
fake_open_process,
)
monkeypatch.setattr(
_trio.debug,
'maybe_wait_for_debugger',
fake_wait_for_debugger,
)
nursery = FakeNursery()
subactor = SimpleNamespace(
aid=tractor.msg.Aid(
name=uid[0],
uuid=uid[1],
),
loglevel=None,
pformat=lambda: 'dead-trio-child',
)
async def main() -> None:
'''
Run the full Trio backend through death-first cleanup.
'''
with pytest.raises(
ActorFailure,
match=r'dead-trio-child.*rc=23',
):
await _trio.trio_proc(
name=uid[0],
actor_nursery=nursery,
subactor=subactor,
errors={},
bind_addrs=[],
parent_addr=('127.0.0.1', 1616),
_runtime_vars={},
)
trio.run(main)
assert server._peer_connected == {}
assert nursery._children == {}
async def run_same_func_in_child(
should_be_root: bool,
data: dict,

View File

@ -18,7 +18,6 @@
tractor: structured concurrent ``trio``-"actors".
"""
from types import ModuleType as _ModuleType
from ._clustering import (
open_actor_cluster as open_actor_cluster,
@ -83,7 +82,6 @@ __all__: tuple[str, ...] = tuple(
for name in globals()
if not name.startswith('_')
) + (
'net',
'to_asyncio',
)
@ -94,18 +92,21 @@ def __dir__() -> list[str]:
def __getattr__(name: str):
'''
PEP 562 lazy public sub-package loading.
PEP 562 lazy sub-module loading, presently only for
`.to_asyncio` which (transitively) imports `asyncio`
itself: a non-trivial multi-ms chunk of the eager
`import tractor` cost (gh #470) unneeded by
`trio`-only apps.
`tractor.to_asyncio` transitively imports `asyncio`, while
`tractor.net` owns optional network dependencies. Neither is
needed by most applications merely importing the root package.
Any `tractor.to_asyncio.<attr>` access (or a
`from tractor import to_asyncio`) still works, the
sub-mod is simply imported on first-access instead
of at pkg-import time.
'''
if name in ('net', 'to_asyncio'):
if name == 'to_asyncio':
from importlib import import_module
module: _ModuleType = import_module(f'.{name}', __name__)
globals()[name] = module
return module
return import_module('.to_asyncio', __name__)
raise AttributeError(
f'module {__name__!r} has no attribute {name!r}'

View File

@ -26,10 +26,7 @@ from ast import literal_eval
from typing import TYPE_CHECKING
from .runtime._runtime import Actor
from .spawn._entry import (
_consume_netns_bootstrap,
_trio_main,
)
from .spawn._entry import _trio_main
if TYPE_CHECKING:
from .discovery._addr import UnwrappedAddress
@ -49,41 +46,12 @@ def parse_ipaddr(arg):
return arg
def parse_netns_bootstrap(arg: str) -> tuple[int, int, int]:
'''
Parse one atomic inherited namespace capability.
Descriptor and device/inode validation remains in
`_consume_netns_bootstrap()` so every valid descriptor-shaped
input reaches its exact child-owned cleanup boundary.
'''
try:
value: object = literal_eval(arg)
except (ValueError, SyntaxError) as exc:
raise argparse.ArgumentTypeError(
'netns bootstrap must be an `(fd, device, inode)` tuple'
) from exc
if (
not isinstance(value, tuple)
or
len(value) != 3
):
raise argparse.ArgumentTypeError(
'netns bootstrap must be an `(fd, device, inode)` tuple'
)
return value
def _actor_child_main(
uid: tuple[str, str],
loglevel: str | None,
parent_addr: UnwrappedAddress | None,
infect_asyncio: bool,
spawn_method: SpawnMethodKey = 'trio',
netns_bootstrap: tuple[int, int, int]|None = None,
) -> None:
'''
@ -94,13 +62,7 @@ def _actor_child_main(
invokes this from inside a fresh `concurrent.interpreters`
sub-interpreter via `Interpreter.call()`.
Consume `netns_bootstrap` before Trio patching, actor construction,
process-title setup, or actor-runtime entry. The spawn backend must
supply an exclusively child-owned FD duplicate.
'''
_consume_netns_bootstrap(netns_bootstrap)
# Apply defensive monkey-patches for upstream `trio`
# bugs we've encountered while running tractor — see
# `tractor.trionics.patches` for the catalog +
@ -151,11 +113,7 @@ def _actor_child_main(
)
def main(argv: list[str]|None = None) -> None:
'''
Parse Trio child-bootstrap arguments and enter actor runtime.
'''
if __name__ == "__main__":
__tracebackhide__: bool = True
parser = argparse.ArgumentParser()
@ -163,11 +121,7 @@ def main(argv: list[str]|None = None) -> None:
parser.add_argument("--loglevel", type=str)
parser.add_argument("--parent_addr", type=parse_ipaddr)
parser.add_argument("--asyncio", action='store_true')
parser.add_argument(
'--netns_bootstrap',
type=parse_netns_bootstrap,
)
args = parser.parse_args(argv)
args = parser.parse_args()
_actor_child_main(
uid=args.uid,
@ -175,9 +129,4 @@ def main(argv: list[str]|None = None) -> None:
parent_addr=args.parent_addr,
infect_asyncio=args.asyncio,
spawn_method='trio',
netns_bootstrap=args.netns_bootstrap,
)
if __name__ == "__main__":
main()

View File

@ -18,8 +18,6 @@
Root actor runtime ignition(s).
'''
from __future__ import annotations
from contextlib import (
asynccontextmanager as acm,
)
@ -51,15 +49,13 @@ from .runtime import _state
from . import log
from .discovery._api import _probe_registry_addrs
from .discovery._addr import (
Address,
UnwrappedAddress,
default_lo_addrs,
get_address_cls,
mk_uuid,
wrap_address,
)
from .discovery.types import (
Address,
UnwrappedAddress,
)
from .discovery._tunnel import strip_tunnels
from .trionics import (
is_multi_cancelled,
collapse_eg,
@ -161,13 +157,8 @@ async def maybe_block_bp(
@acm
async def open_root_actor(
*,
# Low-level realized scope. A future tunnelled-address bootstrap
# may open and supply this capability internally.
# Root netns entry is deferred to a pre-Trio launcher; the async
# root API cannot safely switch the process networking scope.
tpt_bind_addrs: list[
Address # concrete transport address case
Address # `Address.get_random()` case
|UnwrappedAddress # registrar case `= uw_reg_addrs`
]|None = None,
@ -229,10 +220,6 @@ async def open_root_actor(
All (disjoint) actor-process-trees-as-programs are created via
this entrypoint.
To supervise a network-scoped child, open its bindspace outside
an ActorNursery and pass it to `start_actor()`. Root namespace
entry must happen before `trio.run()` and worker-thread creation.
'''
# XXX NEVER allow nested actor-trees!
if already_actor := _state.current_actor(
@ -254,11 +241,9 @@ async def open_root_actor(
)
# debug.mk_pdb().set_trace()
async with (
maybe_block_bp(
async with maybe_block_bp(
debug_mode=debug_mode,
maybe_enable_greenback=maybe_enable_greenback,
),
):
if enable_transports is None:
enable_transports: list[str] = _state.current_ipc_protos()
@ -520,13 +505,8 @@ async def open_root_actor(
# XXX INSTEAD, bind random addrs using the same tpt
# proto if not already provided.
if not tpt_bind_addrs:
from .net._tunnel import strip_tunnels
for addr in ponged_addrs:
bindable_addr: Address = strip_tunnels(addr)
address_type = get_address_cls(
bindable_addr.proto_key
)
tpt_bind_addrs.append(
# XXX, these are `Address` NOT `UnwrappedAddress`.
#
@ -534,7 +514,7 @@ async def open_root_actor(
# protos we allocate port=0 such that the system
# allocates a random value at bind time; this
# happens in the `.ipc.*` stack's backend.
address_type.get_random(
bindable_addr.get_random(
bindspace=bindable_addr.bindspace,
)
)

View File

@ -15,8 +15,48 @@
# along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
Actor discovery and registrar implementation package.
Discovery (protocols) API for automatic addressing
and location management of (service) actors.
Network declarations and helpers are public from `tractor.net`.
NOTE: this ``__init__`` only eagerly imports the lightweight
``._multiaddr`` and ``._tunnel`` submodules for public re-exports.
Heavier submodules like ``._addr`` and ``._api`` are NOT imported
here to avoid circular imports; use direct module paths for those.
'''
from ._bindspace import (
Bindspace as Bindspace,
BindspaceKind as BindspaceKind,
BindspaceLifecycle as BindspaceLifecycle,
BindspaceOwnership as BindspaceOwnership,
BindspaceRef as BindspaceRef,
BindspaceSpec as BindspaceSpec,
CURRENT_NETNS as CURRENT_NETNS,
attach_netns as attach_netns,
open_bindspace as open_bindspace,
open_netns as open_netns,
)
from ._multiaddr import (
parse_endpoints as parse_endpoints,
parse_maddr as parse_maddr,
mk_maddr as mk_maddr,
)
from ._tunnel import (
TunnelledAddress as TunnelledAddress,
TunnelSpec as TunnelSpec,
WGTunnelSpec as WGTunnelSpec,
WGInterfaceConfig as WGInterfaceConfig,
WGPeerConfig as WGPeerConfig,
WGRole as WGRole,
mb_pubkey as mb_pubkey,
mk_wg_maddr as mk_wg_maddr,
open_wg_bindspace as open_wg_bindspace,
open_wg_iface as open_wg_iface,
parse_wg_maddr as parse_wg_maddr,
read_wg_peers as read_wg_peers,
read_wg_pubkey as read_wg_pubkey,
strip_tunnels as strip_tunnels,
tunnels_of as tunnels_of,
verify_wg_peer as verify_wg_peer,
wg8_pubkey as wg8_pubkey,
)

View File

@ -15,7 +15,19 @@
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from __future__ import annotations
from uuid import uuid4
from typing import TypeAlias
from typing import (
Any,
Protocol,
ClassVar,
Literal,
Type,
TypeAlias,
TYPE_CHECKING,
)
from trio import (
SocketListener,
)
from ..log import get_logger
from ..runtime._state import (
@ -26,19 +38,70 @@ from ..ipc._uds import (
UDSAddress,
HAS_UDS,
)
from .types import (
Address as Address,
AddressDeclaration,
LegacyUnwrappedAddress,
TaggedAddress,
TaggedUDSAlias,
UnwrappedAddress,
)
if TYPE_CHECKING:
# ONLY type-annots, the eager import costs ~4.5ms
# of `import tractor` wall-time (gh #470).
from ._tunnel import (
TunnelledAddress,
)
from ..runtime._runtime import Actor
else:
Actor = Any
TunnelledAddress = Any
log = get_logger()
_AddressType: TypeAlias = type[TCPAddress]|type[UDSAddress]
# TODO, maybe breakout the netns key to a struct?
# class NetNs(Struct)[str, int]:
# ...
# TODO, can't we just use a type alias
# for this? namely just some `tuple[str, int, str, str]`?
#
# -[ ] would also just be simpler to keep this as SockAddr[tuple]
# or something, implying it's just a simple pair of values which can
# presumably be mapped to all transports?
# -[ ] `pydoc socket.socket.getsockname()` delivers a 4-tuple for
# ipv6 `(hostaddr, port, flowinfo, scope_id)`.. so how should we
# handle that?
# -[ ] as a further alternative to this wrap()/unwrap() approach we
# could just implement `enc/dec_hook()`s for the `Address`-types
# and just deal with our internal objs directly and always and
# leave it to the codec layer to figure out marshalling?
# |_ would mean only one spot to do the `.unwrap()` (which we may
# end up needing to call from the hook()s anyway?)
# -[x] rename to `UnwrappedAddress[Descriptor]` ??
# seems like the right name as per,
# https://www.geeksforgeeks.org/introduction-to-address-descriptor/
#
TaggedTCPAddress: TypeAlias = tuple[
Literal['tcp'],
str,
int,
]
TaggedUnixAddress: TypeAlias = tuple[
Literal['unix'],
str,
]
TaggedUDSAlias: TypeAlias = tuple[
Literal['uds'],
str,
]
TaggedAddress: TypeAlias = (
TaggedTCPAddress
|TaggedUnixAddress
)
# Input-only compatibility forms retained for older callers and
# serialized payloads.
LegacyTCPAddress: TypeAlias = tuple[str, int]
LegacyUDSAddress: TypeAlias = tuple[str, str]
LegacyUnwrappedAddress: TypeAlias = (
LegacyTCPAddress
|LegacyUDSAddress
)
UnwrappedAddress = TaggedAddress
# ?TODO? should we also include another 2 fields from our `Aid` msg
# such that we include the runtime `Actor.uid` of `.name` and `.uuid`?
# - would ensure uniqueness across entire net?
@ -46,14 +109,106 @@ _AddressType: TypeAlias = type[TCPAddress]|type[UDSAddress]
# name"
# TODO, maybe rename to `SocketAddress`?
class Address(Protocol):
proto_key: ClassVar[str]
unwrapped_type: ClassVar[type]
# TODO, i feel like an `.is_bound()` is a better thing to
# support?
# Lke, what use does this have besides a noop and if it's not
# valid why aren't we erroring on creation/use?
@property
def is_valid(self) -> bool:
...
# TODO, maybe `.netns` is a better name?
@property
def namespace(self) -> tuple[str, str|int]|None:
'''
The if-available, OS-specific "network namespace" key.
'''
...
@property
def bindspace(self) -> str:
'''
Deliver the socket address' "bindable space" from
a `socket.socket.bind()` and thus from the perspective of
specific transport protocol domain.
I.e. for most (layer-4) network-socket protocols this is
normally the ipv4/6 address, for UDS this is normally
a filesystem (sub-directory).
For (distributed) network protocols this is normally the routing
layer's domain/(ip-)address, though it might also include a "network namespace"
key different then the default.
For local-host-only transports this is either an explicit
namespace (with types defined by the OS: netns, Cgroup, IPC,
pid, etc. on linux) or failing that the sub-directory in the
filesys in which socket/shm files are located *under*.
'''
...
@classmethod
def from_addr(cls, addr: UnwrappedAddress) -> Address:
...
def unwrap(self) -> UnwrappedAddress:
'''
Deliver the underying minimum field set in
a primitive python data type-structure.
'''
...
@classmethod
def get_random(
cls,
current_actor: Actor,
bindspace: str|None = None,
) -> Address:
...
# TODO, this should be something like a `.get_def_registar_addr()`
# or similar since,
# - it should be a **host singleton** (not root/tree singleton)
# - we **only need this value** when one isn't provided to the
# runtime at boot and we want to implicitly provide a host-wide
# registrar.
# - each rooted-actor-tree should likely have its own
# micro-registry (likely the root being it), also see
@classmethod
def get_root(cls) -> Address:
...
def __repr__(self) -> str:
...
def __eq__(self, other) -> bool:
...
async def open_listener(
self,
**kwargs,
) -> SocketListener:
...
async def close_listener(self):
...
# the address types available on this host: TCP always, UDS only
# where usable (`HAS_UDS`). Both registries derive from this single
# list via each type's `proto_key`.
_address_protos: list[_AddressType] = [TCPAddress]
_address_protos: list[Type[Address]] = [TCPAddress]
if HAS_UDS:
_address_protos.append(UDSAddress)
_address_types: dict[str, _AddressType] = {
_address_types: dict[str, Type[Address]] = {
cls.proto_key: cls
for cls in _address_protos
}
@ -61,22 +216,13 @@ _address_types: dict[str, _AddressType] = {
# TODO! really these are discovery sys default addrs ONLY useful for
# when none is provided to a root actor on first boot.
#
# TODO, this should be something like a `.get_def_registar_addr()`
# or similar since,
# - it should be a **host singleton** (not root/tree singleton)
# - we **only need this value** when one isn't provided to the
# runtime at boot and we want to implicitly provide a host-wide
# registrar.
# - each rooted-actor-tree should likely have its own
# micro-registry (likely the root being it), also see
_default_lo_addrs: dict[str, UnwrappedAddress] = {
cls.proto_key: cls.get_root().unwrap()
for cls in _address_protos
}
def get_address_cls(name: str) -> _AddressType:
def get_address_cls(name: str) -> Type[Address]:
try:
return _address_types[name]
except KeyError:
@ -87,13 +233,12 @@ def get_address_cls(name: str) -> _AddressType:
)
def is_wrapped_addr(addr: object) -> bool:
def is_wrapped_addr(addr: any) -> bool:
# XXX NOTE, a `TunnelledAddress` is genuinely "wrapped" but is
# deliberately NOT in `_address_types`: it has no
# `MsgTransport` of its own (a tunnel is transparent to
# `socket(2)`), so it gets no proto-key entry. See
# `tractor.net._tunnel`.
from tractor.net._tunnel import TunnelledAddress
# `socket(2)`), so it gets no proto-key entry. See `._tunnel`.
from ._tunnel import TunnelledAddress
return (
type(addr) in _address_types.values()
or
@ -118,9 +263,10 @@ def wrap_address(
|LegacyUnwrappedAddress
|list[str|int]
|str
|AddressDeclaration
|Address
|TunnelledAddress
),
) -> AddressDeclaration:
) -> Address|TunnelledAddress:
'''
Wrap an `UnwrappedAddress` as an `Address`-type based
on matching builtin python data-structures which we adhoc
@ -137,7 +283,7 @@ def wrap_address(
if is_wrapped_addr(addr):
return addr
cls: _AddressType|None = None
cls: Type|None = None
# if 'sock' in addr[0]:
# import pdbp; pdbp.set_trace()
match addr:
@ -181,13 +327,13 @@ def wrap_address(
|
[None, None]
):
cls = get_address_cls(_def_tpt_proto)
cls: Type[Address] = get_address_cls(_def_tpt_proto)
addr: UnwrappedAddress = cls.get_root().unwrap()
# multiaddr-format string, e.g.
# '/ip4/127.0.0.1/tcp/1616'
case str() if addr.startswith('/'):
from tractor.net import (
from tractor.discovery._multiaddr import (
parse_maddr,
)
return parse_maddr(addr)
@ -207,7 +353,7 @@ def wrap_address(
def default_lo_addrs(
transports: list[str],
) -> list[UnwrappedAddress]:
) -> list[Type[Address]]:
'''
Return the default, host-singleton, registry address
for an input transport key set.

View File

@ -51,7 +51,7 @@ BindspaceOwnership: TypeAlias = Literal[
]
_NETNS_RUN_DIR: Path = Path('/var/run/netns')
_THREAD_NETNS: Path = Path('/proc/thread-self/ns/net')
_SELF_NETNS: Path = Path('/proc/self/ns/net')
CURRENT_NETNS: Final[None] = None
@ -124,7 +124,7 @@ class BindspaceSpec(
Serializable declaration of one requested bindspace.
For a netns spec, `.key = CURRENT_NETNS` selects the calling
thread's current namespace without a named-path lookup.
process's current namespace without a named-path lookup.
'''
kind: BindspaceKind
@ -196,7 +196,7 @@ class Bindspace(
'''
spec: BindspaceSpec
ref: BindspaceRef
namespace_fd: int
namespace_fd: int|None
ownership: BindspaceOwnership
def __post_init__(self) -> None:
@ -206,7 +206,7 @@ class Bindspace(
'''
spec: BindspaceSpec = self.spec
ref: BindspaceRef = self.ref
namespace_fd: int = self.namespace_fd
namespace_fd: int|None = self.namespace_fd
ownership: BindspaceOwnership = self.ownership
if spec.kind != ref.kind:
@ -237,13 +237,14 @@ class Bindspace(
f'`BindspaceSpec.lifecycle={spec.lifecycle!r}` '
f'requires ownership={expected_ownership!r}!'
)
if namespace_fd is not None:
if (
type(namespace_fd) is not int
or
namespace_fd < 0
):
raise ValueError(
'`namespace_fd` must be a non-negative `int`!'
'`namespace_fd` must be non-negative or `None`!'
)
fd_inode: int = os.fstat(namespace_fd).st_ino
if ref.inode != fd_inode:
@ -276,7 +277,7 @@ async def _pin_netns(
'''
key: str|None = spec.key
namespace_path: Path = (
_THREAD_NETNS
_SELF_NETNS
if key is CURRENT_NETNS
else _NETNS_RUN_DIR / key
)

View File

@ -27,15 +27,24 @@ Multiaddress support using the upstream `py-multiaddr` lib
from __future__ import annotations
import ipaddress
from pathlib import Path
from typing import TYPE_CHECKING
from .types import AddressDeclaration
from typing import (
Any,
TYPE_CHECKING,
)
if TYPE_CHECKING:
# NOTE, `multiaddr` is lazy-imported at first use
# (in the fns below) to keep it off the eager
# `import tractor` path (gh #470).
from multiaddr import Multiaddr
from tractor.discovery._addr import Address
from tractor.discovery._tunnel import (
TunnelledAddress,
)
else:
Multiaddr = Any
Address = Any
TunnelledAddress = Any
# map from tractor-internal `proto_key` identifiers
# to the standard multiaddr protocol name strings.
@ -52,7 +61,7 @@ _maddr_to_tpt_proto: dict[str, str] = {
def mk_maddr(
addr: AddressDeclaration,
addr: 'Address|TunnelledAddress',
) -> Multiaddr:
'''
Construct a `Multiaddr` from a tractor `Address` instance,
@ -62,7 +71,7 @@ def mk_maddr(
'''
from multiaddr import Multiaddr
from tractor.net._tunnel import (
from ._tunnel import (
TunnelledAddress,
mk_wg_maddr,
)
@ -102,7 +111,7 @@ def mk_maddr(
def parse_maddr(
maddr_str: str,
) -> AddressDeclaration:
) -> 'Address|TunnelledAddress':
'''
Parse a multiaddr string into a tractor `Address`.
@ -121,7 +130,7 @@ def parse_maddr(
# fails. Pre-checking the raw string would misclassify valid
# values such as `/unix/tmp/wg/service.sock`.
if '/wg/' in maddr_str:
from tractor.net._tunnel import _wg_proto_code
from ._tunnel import _wg_proto_code
_wg_proto_code()
raise
proto_names: list[str] = [
@ -147,7 +156,7 @@ def parse_maddr(
)
case _ if 'wg' in proto_names:
from tractor.net._tunnel import parse_wg_maddr
from ._tunnel import parse_wg_maddr
return parse_wg_maddr(maddr)
case _:
@ -165,14 +174,14 @@ def parse_maddr(
# `wrap_address()`).
EndpointsTable = dict[
str, # actor/service name
list[str|tuple|AddressDeclaration],
list[str|tuple], # maddr strs or UnwrappedAddress
]
# output table: actor/service name -> list of wrapped address
# declarations ready for bindspace handling.
ParsedEndpoints = dict[
str, # actor/service name
list[AddressDeclaration],
list['Address|TunnelledAddress'],
]

View File

@ -19,8 +19,7 @@ Tunnelled addresses: an `Address` that rides *inside* a tunnel.
A tunnel (`wg`, and later plain ip-in-udp, `veth`-in-netns, ..) is
**not** a `MsgTransport`. Its data plane is transparent to the
application's `socket(2)`, so it never gets its own entry in
`tractor.discovery._addr._address_types` nor a `MsgpackTransport`
impl. Instead it
`._addr._address_types` nor a `MsgpackTransport` impl. Instead it
*annotates* an existing L4 addr, and this module carries that
annotation beside it.
@ -93,11 +92,6 @@ import multibase
import trio
from ..msg._local import ProcessLocal
from ..discovery.types import (
Address,
AddressDeclaration,
UnwrappedAddress,
)
from ._bindspace import (
Bindspace,
BindspaceRef,
@ -108,6 +102,16 @@ from ._bindspace import (
if TYPE_CHECKING:
from multiaddr import Multiaddr
from ._addr import (
Address,
UnwrappedAddress,
)
else:
Address = Any
Multiaddr = Any
UnwrappedAddress = Any
class WGTunnelSpec(
msgspec.Struct,
frozen=True,
@ -479,7 +483,7 @@ def _sync_create_wg_iface(
'`tractor[wg]` extra.'
) from exc
namespace_fd: int = bindspace.namespace_fd
namespace_fd: int|None = bindspace.namespace_fd
ipr: Any = IPRoute(
netns=namespace_fd,
flags=0,
@ -874,7 +878,7 @@ class TunnelledAddress(
`strip_tunnels()`.
'''
overlay: AddressDeclaration
overlay: Address|TunnelledAddress
tunnel: TunnelSpec
bindspace_ref: BindspaceRef|None = None
@ -1051,8 +1055,8 @@ def parse_wg_maddr(
]
match overlay_names:
case [('ip4' | 'ip6'), 'tcp']:
from ..discovery._multiaddr import parse_maddr
overlay: AddressDeclaration = parse_maddr(
from ._multiaddr import parse_maddr
overlay: Address|TunnelledAddress = parse_maddr(
str(overlay_ma)
)
@ -1157,7 +1161,7 @@ def mk_wg_maddr(
f'/wg/{mb_pubkey(addr.tunnel.peer_pubkey)}'
)
from ..discovery._multiaddr import mk_maddr
from ._multiaddr import mk_maddr
overlay_ma: Multiaddr = mk_maddr(addr.overlay)
return (
bearer_ma
@ -1167,7 +1171,7 @@ def mk_wg_maddr(
def strip_tunnels(
addr: AddressDeclaration,
addr: Address|TunnelledAddress,
) -> Address:
'''
Deliver the bindable `Address`, peeling any tunnel
@ -1186,7 +1190,7 @@ def strip_tunnels(
def tunnels_of(
addr: AddressDeclaration,
addr: Address|TunnelledAddress,
) -> tuple[TunnelSpec, ...]:
'''
Deliver every tunnel spec wrapping `addr`, outermost first.

View File

@ -1,176 +0,0 @@
# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
# This program is free software: you can redistribute it and/or
# modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public
# License along with this program. If not, see
# <https://www.gnu.org/licenses/>.
'''
Dependency-neutral address typing declarations.
Keep this module independent of concrete IPC transports, actor
runtime models, network resources, tunnels, and optional
dependencies.
'''
from __future__ import annotations
from pathlib import Path
from typing import (
ClassVar,
Literal,
Protocol,
TypeAlias,
)
# TODO, maybe breakout the netns key to a struct?
# class NetNs(Struct)[str, int]:
# ...
# TODO, can't we just use a type alias
# for this? namely just some `tuple[str, int, str, str]`?
#
# -[ ] would also just be simpler to keep this as
# SockAddr[tuple] or something, implying it's just a simple pair
# of values which can presumably be mapped to all transports?
# -[ ] `pydoc socket.socket.getsockname()` delivers a 4-tuple for
# ipv6 `(hostaddr, port, flowinfo, scope_id)`.. so how should we
# handle that?
# -[ ] as a further alternative to this wrap()/unwrap() approach we
# could just implement `enc/dec_hook()`s for the `Address`-types
# and just deal with our internal objs directly and always and
# leave it to the codec layer to figure out marshalling?
# |_ would mean only one spot to do the `.unwrap()` (which we may
# end up needing to call from the hook()s anyway?)
# -[x] rename to `UnwrappedAddress[Descriptor]` ??
# seems like the right name as per the GeeksForGeeks article,
# "Introduction to Address Descriptor".
#
TaggedTCPAddress: TypeAlias = tuple[
Literal['tcp'],
str,
int,
]
TaggedUnixAddress: TypeAlias = tuple[
Literal['unix'],
str,
]
TaggedUDSAlias: TypeAlias = tuple[
Literal['uds'],
str,
]
TaggedAddress: TypeAlias = (
TaggedTCPAddress
|TaggedUnixAddress
)
# Input-only compatibility forms retained for older callers and
# serialized payloads.
LegacyTCPAddress: TypeAlias = tuple[str, int]
LegacyUDSAddress: TypeAlias = tuple[str, str]
LegacyUnwrappedAddress: TypeAlias = (
LegacyTCPAddress
|LegacyUDSAddress
)
UnwrappedAddress = TaggedAddress
class AddressDeclaration(Protocol):
'''
Common instance shape of plain and tunnelled addresses.
This protocol deliberately excludes transport registration and
listener operations. A tunnel declaration delegates these address
properties to its innermost concrete transport address.
'''
@property
def proto_key(self) -> str:
...
@property
def is_valid(self) -> bool:
...
@property
def namespace(self) -> tuple[str, str|int]|None:
...
@property
def bindspace(self) -> str|Path:
...
def unwrap(self) -> UnwrappedAddress:
...
# TODO, maybe rename to `SocketAddress`?
class Address(Protocol):
'''
Concrete address contract used by IPC transport backends.
Unlike `AddressDeclaration`, transport registries require
class-level protocol metadata. Transport-specific factories stay
on their concrete address classes because their descriptor and
bindspace inputs differ.
'''
proto_key: ClassVar[str]
unwrapped_type: ClassVar[type]
# TODO, i feel like an `.is_bound()` is a better thing to
# support?
# Lke, what use does this have besides a noop and if it's not
# valid why aren't we erroring on creation/use?
@property
def is_valid(self) -> bool:
...
# TODO, maybe `.netns` is a better name?
@property
def namespace(self) -> tuple[str, str|int]|None:
'''
The if-available, OS-specific "network namespace" key.
'''
...
@property
def bindspace(self) -> str|Path:
'''
Deliver the address' transport-specific bindable space.
'''
...
def unwrap(self) -> UnwrappedAddress:
'''
Deliver the underlying primitive address descriptor.
'''
...
__all__ = (
'Address',
'AddressDeclaration',
'LegacyTCPAddress',
'LegacyUDSAddress',
'LegacyUnwrappedAddress',
'TaggedAddress',
'TaggedTCPAddress',
'TaggedUDSAlias',
'TaggedUnixAddress',
'UnwrappedAddress',
)

View File

@ -43,12 +43,13 @@ from ._types import (
from tractor.discovery._addr import (
is_wrapped_addr,
wrap_address,
)
from tractor.discovery.types import (
Address,
AddressDeclaration,
UnwrappedAddress,
)
from tractor.discovery._tunnel import (
TunnelledAddress,
strip_tunnels,
)
from tractor.log import get_logger
from tractor._exceptions import (
MsgTypeError,
@ -185,12 +186,10 @@ class Channel:
@classmethod
async def from_addr(
cls,
addr: UnwrappedAddress|AddressDeclaration,
addr: UnwrappedAddress|Address|TunnelledAddress,
**kwargs
) -> Channel:
from tractor.net._tunnel import strip_tunnels
if not is_wrapped_addr(addr):
addr = wrap_address(addr)
@ -557,7 +556,7 @@ class Channel:
@acm
async def _connect_chan(
addr: UnwrappedAddress|AddressDeclaration,
addr: UnwrappedAddress|Address|TunnelledAddress,
close_timeout: float|None = None,
) -> typing.AsyncGenerator[Channel, None]:
'''

View File

@ -59,10 +59,8 @@ from ..msg import (
from ..trionics import maybe_open_nursery
from ..runtime import _state
from .. import log
from ..discovery._addr import get_address_cls
from ..discovery.types import (
from ..discovery._addr import (
Address,
AddressDeclaration,
UnwrappedAddress,
)
from ._chan import Channel
@ -70,6 +68,7 @@ from ._transport import MsgTransport
if TYPE_CHECKING:
from ..discovery._tunnel import TunnelledAddress
from ..runtime._runtime import Actor
from ..runtime._supervise import ActorNursery
@ -632,7 +631,7 @@ class Endpoint(Struct):
'''
addr: Address
declared_addr: AddressDeclaration
declared_addr: Address|TunnelledAddress
listen_tn: Nursery
stream_handler_tn: Nursery|None = None
@ -690,8 +689,7 @@ class Endpoint(Struct):
!=
self.addr.unwrap()
):
address_type = get_address_cls(self.addr.proto_key)
self.addr = address_type.from_addr(unwrapped)
self.addr=self.addr.from_addr(unwrapped)
self._listener = lstnr
return lstnr
@ -993,7 +991,7 @@ class Server(Struct):
self,
*,
accept_addrs: list[
UnwrappedAddress|AddressDeclaration
UnwrappedAddress|Address|TunnelledAddress
]|None = None,
stream_handler_nursery: Nursery|None = None,
) -> list[Endpoint]:
@ -1077,7 +1075,7 @@ async def _serve_ipc_eps(
*,
server: IPCServer,
stream_handler_tn: Nursery,
listen_addrs: list[AddressDeclaration],
listen_addrs: list[Address|TunnelledAddress],
task_status: TaskStatus[
Nursery,
@ -1093,7 +1091,7 @@ async def _serve_ipc_eps(
`.cancel_server()` is called.
'''
from ..net._tunnel import strip_tunnels
from ..discovery._tunnel import strip_tunnels
try:
listen_tn: Nursery

View File

@ -20,6 +20,7 @@ TCP implementation of tractor.ipc._transport.MsgTransport protocol
from __future__ import annotations
import ipaddress
from typing import (
Any,
ClassVar,
TYPE_CHECKING,
)
@ -36,16 +37,20 @@ from trio import (
from tractor.msg import MsgCodec
from tractor.log import get_logger
from tractor.discovery._multiaddr import mk_maddr
from tractor.ipc._transport import (
MsgTransport,
MsgpackTransport,
)
from tractor.discovery.types import TaggedTCPAddress
if TYPE_CHECKING:
# ONLY type-annots, the eager import costs
# `import tractor` wall-time (gh #470).
from multiaddr import Multiaddr
from tractor.discovery._addr import TaggedTCPAddress
else:
Multiaddr = Any
TaggedTCPAddress = Any
log = get_logger()
@ -230,8 +235,6 @@ class MsgpackTCPStream(MsgpackTransport):
@property
def maddr(self) -> Multiaddr:
from tractor.net import mk_maddr
return mk_maddr(self.raddr)
def connected(self) -> bool:

View File

@ -55,7 +55,9 @@ from tractor.msg import (
types as msgtypes,
pretty_struct,
)
from tractor.discovery.types import Address
if TYPE_CHECKING:
from tractor.discovery._addr import Address
log = get_logger()

View File

@ -43,6 +43,7 @@ except ImportError:
AF_UNIX = None
import struct
from typing import (
Any,
Type,
TYPE_CHECKING,
ClassVar,
@ -62,6 +63,7 @@ from trio._highlevel_open_unix_stream import (
from tractor.msg import MsgCodec
from tractor.log import get_logger
from tractor.discovery._multiaddr import mk_maddr
from tractor.ipc._transport import (
MsgpackTransport,
)
@ -70,13 +72,17 @@ from tractor.runtime._state import (
current_actor,
is_root_process,
)
from tractor.discovery.types import TaggedUnixAddress
if TYPE_CHECKING:
# ONLY type-annots, the eager import costs
# `import tractor` wall-time (gh #470).
from multiaddr import Multiaddr
from tractor.discovery._addr import TaggedUnixAddress
from tractor.runtime._runtime import Actor
else:
Multiaddr = Any
Actor = Any
TaggedUnixAddress = Any
# Platform-specific credential passing constants
@ -605,8 +611,6 @@ class MsgpackUDSStream(MsgpackTransport):
@property
def maddr(self) -> Multiaddr|str:
from tractor.net import mk_maddr
if not self.raddr:
return '<unknown-peer>'

View File

@ -28,7 +28,6 @@ from .pretty_struct import (
Struct as Struct,
)
from ._local import (
FrozenProcessLocal as FrozenProcessLocal,
ProcessLocal as ProcessLocal,
)
from ._codec import (

View File

@ -27,12 +27,6 @@ class _ProcessLocalToken:
'''
Unsupported msgspec value embedded in every `ProcessLocal`.
A sentinel is one private, unique object used as an identity
marker instead of application data. Every `ProcessLocal` holds
the same `_PROCESS_LOCAL_TOKEN` instance so construction can
verify it by identity and msgspec must encounter its unsupported
type on encode.
'''
__slots__ = ()
@ -46,71 +40,15 @@ class ProcessLocal(
repr_omit_defaults=True,
):
'''
Struct whose `_process_local` field blocks msgspec encoding.
Generic struct marker which rejects default msgspec encoding.
`_process_local` must remain the unsupported singleton
`_PROCESS_LOCAL_TOKEN`. Msgspec reaches that field during direct
or nested encoding and raises `TypeError`; replacing it with an
encodable value or omitting its default would bypass the guard.
A custom encode hook may still explicitly override the safeguard.
Construction therefore rejects a replacement sentinel and any
subclass configured with `omit_defaults=True`. A subclass
`__post_init__()` is wrapped with checks before and after its
body, preventing that hook from skipping or later replacing the
sentinel.
The hidden sentinel remains part of the encoded field set, so
msgspec encounters `_ProcessLocalToken` and raises `TypeError`
even when this value is nested inside another supported payload.
A custom encode hook may explicitly override that safeguard.
Keyword-only fields let subclasses add required fields after the
marker's default sentinel.
'''
_process_local: _ProcessLocalToken = _PROCESS_LOCAL_TOKEN
def __init_subclass__(cls, **kwargs: object) -> None:
'''
Check the sentinel around a subclass post-init hook.
'''
super().__init_subclass__(**kwargs)
subclass_post_init = cls.__dict__.get('__post_init__')
if subclass_post_init is None:
return
def checked_post_init(self: ProcessLocal) -> None:
ProcessLocal.__post_init__(self)
subclass_post_init(self)
ProcessLocal.__post_init__(self)
cls.__post_init__ = checked_post_init
def __post_init__(self) -> None:
'''
Require the singleton sentinel and forbid default omission.
'''
if self._process_local is not _PROCESS_LOCAL_TOKEN:
raise TypeError(
'`ProcessLocal._process_local` must retain its '
'internal sentinel'
)
if self.__struct_config__.omit_defaults:
raise TypeError(
'`ProcessLocal` subclasses may not enable '
'`omit_defaults`'
)
class FrozenProcessLocal(
ProcessLocal,
frozen=True,
):
'''
Frozen `ProcessLocal` whose struct fields reject reassignment.
The inherited `_process_local` sentinel still blocks direct and
nested encoding. Freezing is shallow: struct fields cannot be
replaced, while mutable objects referenced by those fields retain
their own mutation semantics.
'''

View File

@ -1,75 +0,0 @@
# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
'''
Network declarations, bindspaces and tunnels.
Public symbols are imported and cached on first access so importing
this package does not load optional network dependencies.
'''
from importlib import import_module
_SYMBOL_MODULES: dict[str, str] = {
'Bindspace': '._bindspace',
'BindspaceKind': '._bindspace',
'BindspaceLifecycle': '._bindspace',
'BindspaceOwnership': '._bindspace',
'BindspaceRef': '._bindspace',
'BindspaceSpec': '._bindspace',
'CURRENT_NETNS': '._bindspace',
'attach_netns': '._bindspace',
'open_bindspace': '._bindspace',
'open_netns': '._bindspace',
'TunnelledAddress': '._tunnel',
'TunnelSpec': '._tunnel',
'WGTunnelSpec': '._tunnel',
'WGInterfaceConfig': '._tunnel',
'WGPeerConfig': '._tunnel',
'WGRole': '._tunnel',
'mb_pubkey': '._tunnel',
'mk_wg_maddr': '._tunnel',
'open_wg_bindspace': '._tunnel',
'open_wg_iface': '._tunnel',
'parse_wg_maddr': '._tunnel',
'read_wg_peers': '._tunnel',
'read_wg_pubkey': '._tunnel',
'strip_tunnels': '._tunnel',
'tunnels_of': '._tunnel',
'verify_wg_peer': '._tunnel',
'wg8_pubkey': '._tunnel',
'mk_maddr': '..discovery._multiaddr',
'parse_maddr': '..discovery._multiaddr',
'parse_endpoints': '..discovery._multiaddr',
}
__all__: tuple[str, ...] = tuple(_SYMBOL_MODULES)
def __dir__() -> list[str]:
'''
Advertise the complete lazy public API.
'''
return sorted(set(globals()) | set(__all__))
def __getattr__(name: str) -> object:
'''
Import and cache one public network symbol on first access.
'''
try:
module_name: str = _SYMBOL_MODULES[name]
except KeyError:
raise AttributeError(
f'module {__name__!r} has no attribute {name!r}'
) from None
value: object = getattr(
import_module(module_name, __name__),
name,
)
globals()[name] = value
return value

View File

@ -60,6 +60,7 @@ import sys
from typing import (
Any,
Callable,
Type,
TYPE_CHECKING,
)
import uuid
@ -1683,10 +1684,10 @@ async def async_main(
[_state._def_tpt_proto]
)
for transport_key in enable_transports:
address_type = get_address_cls(
transport_cls: Type[Address] = get_address_cls(
transport_key
)
addr: Address = address_type.get_random()
addr: Address = transport_cls.get_random()
accept_addrs.append(addr.unwrap())
# XXX, either passed in by caller or delivered

View File

@ -18,16 +18,12 @@
``trio`` inspired apis and helpers
"""
from contextlib import (
asynccontextmanager as acm,
ExitStack,
)
from contextlib import asynccontextmanager as acm
from functools import partial
from typing import (
TYPE_CHECKING,
)
import typing
import sys
import warnings
import trio
@ -40,7 +36,6 @@ from ..devx import (
from ..discovery._addr import (
UnwrappedAddress,
mk_uuid,
wrap_address,
)
from ._state import (
current_actor,
@ -70,7 +65,6 @@ from ..spawn import _spawn
if TYPE_CHECKING:
import multiprocessing as mp
from ..net._bindspace import Bindspace
# from ..ipc._server import IPCServer
from ..ipc import IPCServer
from ..spawn._spawn import ProcessType
@ -79,22 +73,6 @@ if TYPE_CHECKING:
log = get_logger()
def _validate_spawn_address(addr: UnwrappedAddress) -> None:
'''
Require the canonical tagged address boundary for spawn routing.
'''
valid = False
if type(addr) is tuple:
match addr:
case ('tcp', str() as host, port):
valid = bool(host) and type(port) is int and 0 < port < 65536
case ('unix', str() as path):
valid = bool(path) and '\0' not in path
if not valid or wrap_address(addr).unwrap() != addr:
raise ValueError(f'Expected a canonical UnwrappedAddress, got {addr!r}')
async def _try_cancel_then_kill(
portal: Portal,
# `ProcessType` is `TYPE_CHECKING`-only (defined under that
@ -441,9 +419,6 @@ class ActorNursery:
*,
bind_addrs: list[UnwrappedAddress]|None = None,
bindspace: 'Bindspace|None' = None,
parent_addr: UnwrappedAddress|None = None,
registry_addrs: list[UnwrappedAddress]|None = None,
rpc_module_paths: list[str]|None = None,
enable_transports: list[str] = [_state._def_tpt_proto],
enable_modules: list[str]|None = None,
@ -458,12 +433,6 @@ class ActorNursery:
Start a (daemon) actor: an process that has no designated
"main task" besides the runtime.
`parent_addr` selects one canonical live supervisor listener;
`registry_addrs` replaces registrar inheritance for this child.
Both must remain reachable from the child's target netns. A
filesystem UDS is the usual cross-netns bootstrap transport.
Omitted `bindspace` on Linux/Trio pins the calling task's netns.
Pass ``inherit_parent_main=False`` to keep this child on its
own bootstrap module for the trio spawn backend instead of
applying the parent ``__main__`` re-exec fixup during startup.
@ -507,27 +476,6 @@ class ActorNursery:
)
enable_modules.extend(rpc_module_paths)
if parent_addr is None:
parent_addr = self._actor.accept_addr
else:
_validate_spawn_address(parent_addr)
if parent_addr not in self._actor.accept_addrs:
raise ValueError(
f'parent_addr {parent_addr!r} must name a live '
'supervisor listener'
)
assert parent_addr
if registry_addrs is None:
selected_reg_addrs = list(current_actor().reg_addrs)
else:
if type(registry_addrs) is not list or not registry_addrs:
raise ValueError('registry_addrs must be a non-empty list')
for addr in registry_addrs:
_validate_spawn_address(addr)
selected_reg_addrs = list(registry_addrs)
_rtv['_registry_addrs'] = list(selected_reg_addrs)
subactor = Actor(
name=name,
uuid=mk_uuid(),
@ -538,26 +486,14 @@ class ActorNursery:
inherit_parent_main=inherit_parent_main,
# verbatim relay this actor's registrar addresses
# Populate child-local state below: Actor construction with
# registry_addrs publishes into this process's runtime vars.
registry_addrs=current_actor().registry_addrs,
)
subactor.reg_addrs = selected_reg_addrs
parent_addr: UnwrappedAddress = self._actor.accept_addr
assert parent_addr
# start a task to spawn a process
# blocks until process has been started and a portal setup
# XXX: the type ignore is actually due to a `mypy` bug
with ExitStack() as scope:
if (
bindspace is None
and sys.platform == 'linux'
and _spawn._spawn_method == 'trio'
):
from ..spawn._netns import pin_current_netns
# Capture on the calling actor task before nursery.start()
# checkpoints or any subprocess worker can choose a netns.
bindspace = scope.enter_context(pin_current_netns())
return await self._da_nursery.start( # type: ignore
partial(
_spawn.new_proc,
@ -568,7 +504,6 @@ class ActorNursery:
bind_addrs,
parent_addr,
_rtv, # run time vars
bindspace=bindspace,
infect_asyncio=infect_asyncio,
proc_kwargs=proc_kwargs
)

View File

@ -21,7 +21,6 @@ Sub-process entry points.
from __future__ import annotations
from functools import partial
import multiprocessing as mp
import os
from typing import (
Any,
TYPE_CHECKING,
@ -49,7 +48,6 @@ from ..runtime._runtime import (
async_main,
Actor,
)
from ._netns import enter_netns
if TYPE_CHECKING:
from ._spawn import SpawnMethodKey
@ -58,57 +56,6 @@ if TYPE_CHECKING:
log = get_logger()
def _consume_netns_bootstrap(
netns_bootstrap: tuple[int, int, int]|None,
) -> int|None:
'''
Enter and release one child-owned network namespace capability.
The FD must be an exclusively child-owned backend duplicate. It is
closed whether namespace entry succeeds or fails, before any actor
runtime setup can continue.
'''
if netns_bootstrap is None:
return None
namespace_fd: int
expected_device: int
expected_inode: int
namespace_fd, expected_device, expected_inode = netns_bootstrap
if (
type(namespace_fd) is not int
or
namespace_fd < 0
):
# Let `enter_netns()` report its precise validation error, but
# never pass bool/non-int/negative values to `os.close()`.
return enter_netns(
namespace_fd,
expected_device,
expected_inode,
)
try:
entered_inode: int = enter_netns(
namespace_fd,
expected_device,
expected_inode,
)
except BaseException as entry_error:
try:
os.close(namespace_fd)
except Exception as close_error:
entry_error.add_note(
f'Also failed to close inherited namespace FD '
f'{namespace_fd}: {close_error!r}'
)
raise
else:
os.close(namespace_fd)
return entered_inode
def _mp_main(
actor: Actor,
@ -117,18 +64,12 @@ def _mp_main(
start_method: SpawnMethodKey,
parent_addr: UnwrappedAddress | None = None,
infect_asyncio: bool = False,
netns_bootstrap: tuple[int, int, int]|None = None,
) -> None:
'''
The routine called *after fork* which invokes a fresh `trio.run()`
Consume `netns_bootstrap` before multiprocessing or actor-runtime
setup. The spawn backend must supply a child-owned FD duplicate.
'''
_consume_netns_bootstrap(netns_bootstrap)
actor._forkserver_info = forkserver_info
from ._spawn import try_set_start_method
spawn_ctx: mp.context.BaseContext = try_set_start_method(start_method)

View File

@ -54,7 +54,6 @@ from ._spawn import (
if TYPE_CHECKING:
from tractor.net._bindspace import Bindspace
from tractor.ipc import (
_server,
)
@ -74,19 +73,12 @@ async def mp_proc(
parent_addr: UnwrappedAddress,
_runtime_vars: dict[str, Any], # serialized and sent to _child
*,
bindspace: Bindspace|None = None,
infect_asyncio: bool = False,
task_status: TaskStatus[Portal] = trio.TASK_STATUS_IGNORED,
proc_kwargs: dict[str, any] = {}
) -> None:
if bindspace is not None:
raise NotImplementedError(
'Network namespace bindspace transport is not yet '
'supported by multiprocessing spawn backends!'
)
# uggh zone
try:
from multiprocessing import semaphore_tracker # type: ignore

View File

@ -1,317 +0,0 @@
# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
Linux network-namespace actor-bootstrap primitives.
'''
from __future__ import annotations
from collections.abc import (
Callable,
Iterator,
)
from contextlib import contextmanager as cm
import errno
from pathlib import Path
import os
import sys
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from ..net._bindspace import Bindspace
# `setns(2)` mutates only the calling thread's namespace:
# https://man7.org/linux/man-pages/man2/setns.2.html
# `/proc/thread-self` addresses the caller's current task:
# https://man7.org/linux/man-pages/man5/proc_pid_task.5.html
# Do not use `/proc/self` because it resolves through the process
# leader.
_SELF_NETNS: Path = Path('/proc/thread-self/ns/net')
def enter_netns(
namespace_fd: int,
expected_device: int,
expected_inode: int,
) -> int:
'''
Enter and verify one inherited Linux network namespace.
The caller owns and closes `namespace_fd`.
'''
if sys.platform != 'linux':
raise RuntimeError(
'Network namespace entry is Linux-only!'
)
if (
type(namespace_fd) is not int
or
namespace_fd < 0
):
raise ValueError(
'`namespace_fd` must be a non-negative `int`!'
)
if (
type(expected_inode) is not int
or
expected_inode <= 0
):
raise ValueError(
'`expected_inode` must be a positive `int`!'
)
if type(expected_device) is not int or expected_device < 0:
raise ValueError('`expected_device` must be a non-negative `int`!')
target = os.fstat(namespace_fd)
expected = (expected_device, expected_inode)
if (target.st_dev, target.st_ino) != expected:
raise ValueError(
f'Inherited namespace FD identity '
f'{(target.st_dev, target.st_ino)} does not '
f'match expected identity {expected}!'
)
current = _SELF_NETNS.stat()
if (current.st_dev, current.st_ino) == expected:
return expected_inode
setns: Callable[[int, int], None]|None = getattr(
os,
'setns',
None,
)
clone_newnet: int|None = getattr(
os,
'CLONE_NEWNET',
None,
)
if (
setns is None
or
clone_newnet is None
):
raise RuntimeError(
'Python has no Linux network namespace entry support!'
)
try:
setns(namespace_fd, clone_newnet)
except OSError as exc:
raise RuntimeError(
f'Could not enter network namespace inode '
f'{expected_inode}!'
) from exc
entered = _SELF_NETNS.stat()
entered_inode: int = entered.st_ino
if (entered.st_dev, entered_inode) != expected:
raise RuntimeError(
f'Entered network namespace identity '
f'{(entered.st_dev, entered_inode)} does not '
f'match expected identity {expected}!'
)
return entered_inode
@cm
def close_fd(
owned_fd: int,
fd_name: str,
) -> Iterator[None]:
'''
Close one owned netns FD without masking a prior error.
'''
operation: str = (
f'close owned {fd_name} netns FD {owned_fd}'
)
try:
yield
except BaseException as primary_error:
try:
os.close(owned_fd)
except BaseException as close_error:
primary_error.add_note(
f'Also failed to {operation}: {close_error!r}'
)
raise primary_error
else:
try:
os.close(owned_fd)
except BaseException as close_error:
close_error.add_note(
f'Failed to {operation} during root netns cleanup.'
)
raise close_error
@cm
def dup_fd(
source_fd: int,
) -> Iterator[int]:
'''
Duplicate and own the target netns FD for this context.
'''
try:
owned_fd: int = os.dup(source_fd)
except OSError as dup_error:
if dup_error.errno != errno.EBADF:
raise dup_error
raise ValueError(
'`bindspace.namespace_fd` does not reference '
'a live FD!'
) from dup_error
with close_fd(owned_fd, 'target'):
yield owned_fd
@cm
def _enter_netns_temporarily(
bindspace: Bindspace|None,
) -> Iterator[int|None]:
'''
Enter a root bindspace and restore the caller thread's netns.
Internal synchronous primitive only. Never surround a Trio
checkpoint with this scope: entry changes only the calling thread.
The former async root adapter has been removed.
Only descriptors opened or duplicated by this context are used
for validation, entry and restoration. Since `setns()` is
thread-local, this synchronous context performs no checkpoints
around either transition.
'''
if bindspace is None:
yield None
return
if sys.platform != 'linux':
raise RuntimeError(
'Network namespace entry is Linux-only!'
)
namespace_fd: int = bindspace.namespace_fd
if (
type(namespace_fd) is not int
or
namespace_fd < 0
):
raise ValueError(
'`bindspace.namespace_fd` must be a live '
'non-negative FD!'
)
# Borrow `Bindspace.namespace_fd`; duplicate it so this context
# owns target cleanup and cannot close the caller's capability.
# Nested FD scopes aggregate later close failures as notes on the
# first body, restoration or cleanup error.
with dup_fd(namespace_fd) as tgt_fd:
tgt_stat: os.stat_result = os.fstat(tgt_fd)
tgt_inode: int = bindspace.ref.inode
if tgt_stat.st_ino != tgt_inode:
raise ValueError(
f'Target namespace FD inode '
f'{tgt_stat.st_ino} does '
f'not match bindspace inode {tgt_inode}!'
)
# Capture the calling thread's current netns before any
# transition. It need not be the process's initial netns. This
# context owns the snapshot FD even when no transition is
# needed, so keep it live through restoration and always close
# it afterward.
orig_fd = os.open(
_SELF_NETNS,
os.O_RDONLY | os.O_CLOEXEC,
)
with close_fd(orig_fd, 'original'):
orig_stat: os.stat_result = os.fstat(orig_fd)
orig_inode: int = orig_stat.st_ino
restore_needed: bool = (
tgt_stat.st_dev != orig_stat.st_dev
or
tgt_inode != orig_inode
)
try:
if restore_needed:
enter_netns(
tgt_fd,
tgt_stat.st_dev,
tgt_inode,
)
yield tgt_inode
except BaseException as primary_error:
if restore_needed:
try:
enter_netns(
orig_fd,
orig_stat.st_dev,
orig_inode,
)
except BaseException as restore_error:
primary_error.add_note(
'Also failed to restore the original '
'network namespace: '
f'{restore_error!r}'
)
raise primary_error
if restore_needed:
try:
enter_netns(
orig_fd,
orig_stat.st_dev,
orig_inode,
)
except BaseException as restore_error:
restore_error.add_note(
'Failed to restore the original network '
'namespace during root netns cleanup.'
)
raise restore_error
@cm
def pin_current_netns() -> Iterator[Bindspace]:
'''
Pin the calling task's netns synchronously, without entering it.
'''
from ..net._bindspace import Bindspace, BindspaceRef, BindspaceSpec
namespace_fd = os.open(_SELF_NETNS, os.O_RDONLY | os.O_CLOEXEC)
with close_fd(namespace_fd, 'calling task'):
yield Bindspace(
spec=BindspaceSpec(kind='netns'),
ref=BindspaceRef(
kind='netns',
key=None,
inode=os.fstat(namespace_fd).st_ino,
),
namespace_fd=namespace_fd,
ownership='borrowed',
)

View File

@ -34,7 +34,6 @@ from typing import (
import trio
from trio import TaskStatus
from .._exceptions import ActorFailure
from ..devx import debug
from tractor.runtime._state import (
_runtime_vars,
@ -50,9 +49,7 @@ from tractor.msg import types as msgtypes
if TYPE_CHECKING:
from tractor.net._bindspace import Bindspace
from tractor.ipc import (
_server,
Channel,
)
from tractor.runtime._supervise import ActorNursery
@ -84,101 +81,6 @@ else:
await trio.lowlevel.wait_readable(proc.sentinel)
async def wait_for_peer_or_proc_death(
ipc_server: _server.Server,
uid: tuple[str, str],
proc_wait: Callable[[], Awaitable[int]],
proc_repr: object = '',
) -> tuple[trio.Event, Channel]:
'''
Race a child handshake against process death during bootstrap.
A child can exit before connecting to its parent. Waiting only on
`IPCServer.wait_for_peer()` would then park its spawning task and
leave the dead process unreaped. Run both waits in one nursery and
let either completed result cancel its sibling.
Return the normal peer event and channel when the handshake wins.
Raise `ActorFailure` with the process status when death wins.
Adapted from goodboy's Claude Code-assisted implementation in
commit `3b0724eba85b4014170ed95773e1e41a60d5c513`.
'''
handshake: tuple[trio.Event, Channel]|None = None
handshake_error: BaseException|None = None
returncode: int|None = None
death_error: BaseException|None = None
async def wait_for_handshake() -> None:
'''
Publish a connected peer before cancelling the death waiter.
'''
nonlocal handshake
nonlocal handshake_error
try:
handshake = await ipc_server.wait_for_peer(uid)
except trio.Cancelled:
if (
returncode is not None
or
death_error is not None
):
log.debug(
'Peer-handshake waiter cancelled after '
f'process wait completed for {uid!r}'
)
raise
except BaseException as exc:
handshake_error = exc
nursery.cancel_scope.cancel()
async def wait_for_death() -> None:
'''
Publish child exit before cancelling the handshake waiter.
'''
nonlocal returncode
nonlocal death_error
try:
returncode = await proc_wait()
except trio.Cancelled:
if (
handshake is not None
or
handshake_error is not None
):
log.debug(
'Process-death waiter cancelled after '
f'peer wait completed for {uid!r}'
)
raise
except BaseException as exc:
death_error = exc
nursery.cancel_scope.cancel()
async with trio.open_nursery() as nursery:
nursery.start_soon(wait_for_handshake)
nursery.start_soon(wait_for_death)
if handshake_error is not None:
raise handshake_error
if death_error is not None:
raise death_error
if returncode is not None:
raise ActorFailure(
f'Sub-actor {uid!r} died during boot '
f'(rc={returncode!r}) before completing '
f'parent-handshake.\n'
f' proc: {proc_repr}'
)
assert handshake is not None
return handshake
def try_set_start_method(
key: SpawnMethodKey
@ -440,7 +342,6 @@ async def new_proc(
*,
bindspace: Bindspace|None = None,
infect_asyncio: bool = False,
task_status: TaskStatus[Portal] = trio.TASK_STATUS_IGNORED,
proc_kwargs: dict[str, any] = {}
@ -462,7 +363,6 @@ async def new_proc(
bind_addrs,
parent_addr,
_runtime_vars, # run time vars
bindspace=bindspace,
infect_asyncio=infect_asyncio,
task_status=task_status,
proc_kwargs=proc_kwargs

View File

@ -22,7 +22,6 @@ Spawns sub-actors as fresh OS processes driven by
'''
from __future__ import annotations
import os
import sys
from typing import (
Any,
@ -52,12 +51,10 @@ from tractor.msg import (
from ._spawn import (
hard_kill,
soft_kill,
wait_for_peer_or_proc_death,
)
if TYPE_CHECKING:
from tractor.net._bindspace import Bindspace
from tractor.ipc import (
_server,
)
@ -78,31 +75,29 @@ async def trio_proc(
parent_addr: UnwrappedAddress,
_runtime_vars: dict[str, Any], # serialized and sent to _child
*,
bindspace: Bindspace|None = None,
infect_asyncio: bool = False,
task_status: TaskStatus[Portal] = trio.TASK_STATUS_IGNORED,
proc_kwargs: dict[str, any] = {}
) -> None:
'''
Create a new ``Process`` using a "spawn method" as (configured
using ``try_set_start_method()``).
Create a new ``Process`` using a "spawn method" as (configured using
``try_set_start_method()``).
This routine should be started in a actor runtime task and the
logic here is to be considered the core supervision strategy.
This routine should be started in a actor runtime task and the logic
here is to be considered the core supervision strategy.
'''
spawn_cmd = [
sys.executable,
"-m",
# Hardcode this (instead of using ``_child.__name__`` to
# avoid a double import warning:
# https://stackoverflow.com/a/45070583
# Hardcode this (instead of using ``_child.__name__`` to avoid a
# double import warning: https://stackoverflow.com/a/45070583
"tractor._child",
# We provide the child's unique identifier on this exec/spawn
# line for debugging purposes when viewing the process tree
# from the OS; it otherwise can be passed via the parent
# channel if we prefer in the future (for privacy).
# line for debugging purposes when viewing the process tree from
# the OS; it otherwise can be passed via the parent channel if
# we prefer in the future (for privacy).
"--uid",
# TODO, how to pass this over "wire" encodings like
# cmdline args?
@ -120,77 +115,20 @@ async def trio_proc(
]
# Tell child to run in guest mode on top of ``asyncio`` loop
if infect_asyncio:
spawn_cmd.append('--asyncio')
child_netns_fd: int|None = None
if bindspace is not None:
namespace_fd: int = bindspace.namespace_fd
# Snapshot caller-owned process options before duplicating the
# live `Bindspace.namespace_fd`. No checkpoint separates this
# setup from `open_process()` below.
inherited_fds: tuple[int, ...] = tuple(
proc_kwargs.get('pass_fds', ())
)
proc_kwargs = dict(proc_kwargs)
child_netns_fd = os.dup(namespace_fd)
try:
target_stat = os.fstat(child_netns_fd)
netns_bootstrap: tuple[int, int, int] = (
# FD number retained in the child's descriptor table.
child_netns_fd,
# Namespace identity checked before the child enters it.
target_stat.st_dev,
bindspace.ref.inode,
)
spawn_cmd.extend((
'--netns_bootstrap',
str(netns_bootstrap),
))
# Keep every descriptor requested by the caller and append
# the namespace FD needed during child bootstrap.
proc_kwargs['pass_fds'] = (
*inherited_fds,
child_netns_fd,
)
except BaseException:
os.close(child_netns_fd)
raise
spawn_cmd.append("--asyncio")
cancelled_during_spawn: bool = False
proc: trio.Process|None = None
ipc_server: _server.Server = actor_nursery._actor.ipc_server
peer_event: trio.Event|None = None
child_registered: bool = False
try:
try:
try:
proc: trio.Process = await trio.lowlevel.open_process(
spawn_cmd,
**proc_kwargs,
)
finally:
if child_netns_fd is not None:
# The child now has its own descriptor-table entry.
# Close the temporary entry in the parent process.
os.close(child_netns_fd)
proc: trio.Process = await trio.lowlevel.open_process(spawn_cmd, **proc_kwargs)
log.runtime(
f'Started new child subproc\n'
f'(>\n'
f' |_{proc}\n'
)
# `ActorNursery.cancel()` may inspect this event as soon
# as the provisional child is published below. Register
# the event synchronously before
# `wait_for_peer_or_proc_death()` opens its nursery and
# checkpoints.
peer_event = ipc_server._peer_connected.setdefault(
subactor.aid.uid,
trio.Event(),
)
# No `Portal` exists until the IPC handshake returns
# `chan`. Replace this provisional entry with
# `Portal(chan)` below.
@ -203,7 +141,6 @@ async def trio_proc(
proc=proc,
portal=None,
)
child_registered = True
if cancel_during_registration:
cancelled_during_spawn = True
proc.kill()
@ -215,11 +152,8 @@ async def trio_proc(
# wait for actor to spawn and connect back to us
# channel should have handshake completed by the
# local actor by the time we get a ref to it
event, chan = await wait_for_peer_or_proc_death(
ipc_server=ipc_server,
uid=subactor.aid.uid,
proc_wait=proc.wait,
proc_repr=proc,
event, chan = await ipc_server.wait_for_peer(
subactor.aid.uid
)
except trio.Cancelled:
@ -335,21 +269,21 @@ async def trio_proc(
# to hold off on relaying SIGINT until that child
# is complete.
# https://github.com/goodboy/tractor/issues/320
# -[ ] we need to handle non-root parent-actors
# specially by somehow determining if a child is in
# debug and then avoiding cancel/kill of said child
# by this (intermediary) parent until such a time as
# the root says the pdb lock is released and we are
# good to tear down (our children)..
# -[ ] we need to handle non-root parent-actors specially
# by somehow determining if a child is in debug and then
# avoiding cancel/kill of said child by this
# (intermediary) parent until such a time as the root says
# the pdb lock is released and we are good to tear down
# (our children)..
#
# -[ ] so maybe something like this where we try to
# acquire the lock and get notified of who has
# it, check that uid against our known children?
# acquire the lock and get notified of who has it,
# check that uid against our known children?
# this_uid: tuple[str, str] = current_actor().uid
# await debug.acquire_debug_lock(this_uid)
if proc.poll() is None:
log.cancel(f'Attempting to hard kill {proc}')
log.cancel(f"Attempting to hard kill {proc}")
await hard_kill(
proc,
# NOTE, pass through so post-SIGKILL we
@ -368,24 +302,11 @@ async def trio_proc(
subactor=subactor,
)
log.debug(f'Joined {proc}')
log.debug(f"Joined {proc}")
else:
log.warning('Nursery cancelled before sub-proc started')
if (
peer_event is not None
and
ipc_server._peer_connected.get(
subactor.aid.uid,
) is peer_event
):
ipc_server._peer_connected.pop(subactor.aid.uid)
if (
child_registered
and
not cancelled_during_spawn
):
if not cancelled_during_spawn:
# pop child entry to indicate we no longer managing this
# subactor
actor_nursery._children.pop(subactor.aid.uid)

View File

@ -61,7 +61,6 @@ from ..runtime._supervise import (
if TYPE_CHECKING:
from ..discovery._addr import UnwrappedAddress
from ..net._bindspace import Bindspace
from ..runtime._portal import Portal
@ -255,7 +254,6 @@ async def run(
# is provided.
name: str|None = None,
bind_addrs: list[UnwrappedAddress]|None = None,
bindspace: Bindspace|None = None,
enable_modules: list[str]|None = None,
loglevel: str|None = None,
debug_mode: bool|None = None,
@ -339,7 +337,6 @@ async def run(
(enable_modules or [])
),
bind_addrs=bind_addrs,
bindspace=bindspace,
loglevel=loglevel,
debug_mode=debug_mode,
infect_asyncio=infect_asyncio,