Compare commits

...

55 Commits

Author SHA1 Message Date
Gud Boi b1f6ade840 Pass `Bindspace` FD to `trio` children
Thread an optional live `Bindspace` through actor spawn APIs and
give each Trio exec child an inherited namespace descriptor before
runtime bootstrap.

Deats,
- duplicate the namespace FD without changing parent ownership
- preserve caller `pass_fds` and process options
- send the `(fd, inode)` pair through the `_child` CLI
- close the parent duplicate on success, failure and cancellation
- reject MP backends until descriptor reduction is implemented
- avoid cleanup before child publication
- exercise real E2E namespace relay through handshake and RPC
- verify parent FD-table cleanup after successful child spawn

Prompt-IO: ai/prompt-io/opencode/20260828T172943Z_2ca8c570_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 2fb6143604 Detect child death before parent handshake
Race the initial peer handshake against process exit during Trio
child bootstrap so a dead child cannot park its spawning task.
Restore and harden the design originally implemented in `3b0724eb`.

Deats,
- register peer events before provisional nursery publication
- raise `ActorFailure` with child status when process death wins
- prioritize observed death over a simultaneous handshake
- preserve waiter exceptions without `ExceptionGroup` wrapping
- log expected losing-waiter cancellation at debug level
- remove exact failed-startup peer events during backend cleanup
- cover controlled schedules and full Trio backend cleanup

Based-on: 3b0724eba8
Prompt-IO: ai/prompt-io/opencode/20260828T045119Z_fb6d81d3_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 6d5b145787 Enter inherited netns during child bootstrap
Consume an optional child-owned `(fd, inode)` capability before
multiprocessing or Trio child bootstrap begins runtime setup.

Deats,
- enter the netns before Trio patching and actor construction
- close the inherited FD before actor runtime startup
- preserve entry errors when descriptor cleanup also fails
- reject malformed FD values without closing unrelated descriptors
- cover multiprocessing and Trio ordering with real stand-in FDs

Prompt-IO: ai/prompt-io/opencode/20260828T012205Z_9ae7cd86_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 503a3ed766 Verify inherited network namespace entry
Add private `enter_netns()` bootstrap validation before actor runtime
integration.

Deats,
- verify the inherited FD against its expected namespace inode
- constrain `setns()` to Linux network namespaces
- verify `/proc/self/ns/net` after the entry syscall
- leave inherited FD ownership to the future spawn caller
- exercise guards with real FDs and unprivileged syscall fakes

Prompt-IO: ai/prompt-io/opencode/20260827T232500Z_d067505a_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi d0261787fd Plan typed tunnel address decoding
Record native tagged encoding for complete `TunnelledAddress` graphs as
a deferred design follow-up.

Deats,
- cover concrete overlay-address and tunnel-spec unions
- preserve optional `BindspaceRef` metadata through decoding
- replace untyped payload inspection with typed roundtrip tests

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi d1ffb436d0 Retain realized `BindspaceRef` metadata
Add `TunnelledAddress.with_bindspace_ref()` to annotate frozen tunnel
declarations with stable, serializable namespace metadata.

Deats,
- omit absent refs from the existing msgspec shape
- reject declared and realized namespace-name mismatches
- report the declared key before realization and inode afterward
- keep transport peeling free of live bindspace capability state
- expose realized refs through real listener diagnostics

Prompt-IO: ai/prompt-io/opencode/20260826T030534Z_d130431c_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 244e080efe Rename bindspace resource models
Replace the unshipped `BindspaceIdentity` and `BindspaceHandle` names
with `BindspaceRef` and `Bindspace` across existing lifecycle APIs.

Deats,
- define refs as wire-safe, host-local and non-owning records
- reserve `Bindspace` for the live FD-backed capability
- rename the capability's realized-resource field to `.ref`
- update lifecycle tests and active design contracts
- omit compatibility aliases for the unshipped model names

Prompt-IO: ai/prompt-io/opencode/20260827T211115Z_d130431c_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 6dd149df3d Compose WireGuard bindspace lifecycles
Add `open_wg_bindspace()` to enter one declared bindspace and an
ordered WireGuard interface stack as one async lifetime.

Deats,
- snapshot caller layer ordering before the first checkpoint
- enter interfaces outermost-first through `AsyncExitStack`
- unwind interfaces before releasing the namespace capability
- yield the live `BindspaceHandle` for endpoint allocation
- test mutable input and cancellation ordering with lifecycle fakes

Prompt-IO: ai/prompt-io/opencode/20260826T022434Z_2245f094_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi f7bf068d09 Own WireGuard interface lifecycles
Add `open_wg_iface()` to create, configure and remove one WireGuard
interface inside a pinned bindspace through pyroute2.

Deats,
- validate listen/dial bearer policy before kernel side effects
- configure local addresses, private key, listen port and peers
- fill an omitted dial endpoint from the selected tunnel bearer
- clean partial synchronous failures before returning to Trio
- shield owned interface creation and teardown from cancellation

Prompt-IO: ai/prompt-io/opencode/20260826T003430Z_6dd39da0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 1fd857ce21 Model explicit WireGuard peers
Add process-local `WGPeerConfig` entries and make
`WGInterfaceConfig` own a unique peer tuple for listener and dial
provisioning.

Deats,
- carry peer public keys, allowed CIDRs and optional endpoints
- redact per-peer preshared keys while blocking wire encoding
- validate peer routes, endpoint ports and keepalive intervals
- reject duplicate peers before future kernel mutation
- support multi-peer listeners without overloading tunnel identity

Prompt-IO: ai/prompt-io/opencode/20260826T001442Z_dcdf4d82_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 1b0b085667 Separate local WireGuard configuration
Add process-local `WGInterfaceConfig` for key material, interface
addresses and peer-routing policy required by future provisioning.

Deats,
- redact private and preshared keys from representation
- block config from default actor-IPC encoding via `ProcessLocal`
- validate keys, interface CIDRs, allowed CIDRs and bounded integers
- keep public endpoint and peer identity in `WGTunnelSpec`
- move allowed-IP policy out of the serializable tunnel declaration

Prompt-IO: ai/prompt-io/opencode/20260825T234631Z_b973e78c_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 790a52ca7a Dispatch explicit bindspace lifecycles
Add serialized `BindspaceSpec.lifecycle` policy and dispatch it through
`open_bindspace()` without inferring ownership from transport role.

Deats,
- distinguish borrowed `attach` from owned `open` policy
- validate handle ownership against the declared lifecycle
- share policy-neutral FD pinning between both netns contexts
- reject unsupported lifecycle values before side effects
- exercise both dispatcher branches and owned cancellation cleanup

Prompt-IO: ai/prompt-io/opencode/20260825T191845Z_5b2a064a_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 7abbb2b5f0 Open owned network namespaces
Add `open_netns()` to create a named Linux netns through pyroute2,
pin its identity and yield an owned `BindspaceHandle`.

Deats,
- run synchronous creation and removal in Trio worker threads
- shield both privileged side effects from caller cancellation
- reuse `attach_netns()` to pin identity and manage the FD
- close the FD before removing the owned namespace
- fake privileged operations while testing ordering and cancellation

Prompt-IO: ai/prompt-io/opencode/20260825T190529Z_e1007547_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 40e96b2991 Attach existing network namespaces
Add `attach_netns()` to pin a current or named Linux netns in a
borrowed `BindspaceHandle` without creating or entering it.

Deats,
- name the current-namespace default `CURRENT_NETNS`
- derive stable identity from the opened FD with `fstat()`
- open descriptors with `O_CLOEXEC` and close them on context exit
- constrain named lookup beneath the standard iproute2 run directory
- report field-specific validation and missing-resource errors

Prompt-IO: ai/prompt-io/opencode/20260825T045557Z_fdccfd7e_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi b9dca728d9 Add bindspace capability models
Separate serializable `BindspaceSpec` and `BindspaceIdentity` values
from a process-local `BindspaceHandle` carrying FD and ownership
authority.

Deats,
- add global `ProcessLocal` wire guards for local handle structs
- derive valid kinds and ownership from their `Literal` aliases
- require a positive inode while keeping the mutable name optional
- pin supplied FDs to identity inodes with `fstat()`
- cover round trips, nested encoding and stale capabilities

Caught-during: review remediation
Found-via: `/run-tests` test_bindspace_handle_pins_local_capability

Prompt-IO: ai/prompt-io/opencode/20260822T042026Z_29141f0b_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi be3e78197d Expose tunnel namespaces on IPC endpoints
Make `TCPAddress` and `UDSAddress` explicitly satisfy
`Address.namespace`, then retain each original listener declaration
beside its peeled, resolved transport address.

Deats,
- remove `TunnelledAddress`'s attribute fallback
- add required `Endpoint.declared_addr` metadata
- report declaration namespaces in endpoint/server formatting
- preserve concrete `Endpoint.addr` for transport reflection
- cover plain and tunneled namespace visibility

Prompt-IO: ai/prompt-io/opencode/20260822T032520Z_d35c802b_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi ece3827bdf Add explicit `verify_wg_peer()` inspection
Validate a declared tunnel key against one `pyroute2` snapshot
containing the iface's own key and configured peers.

Deats,
- share worker offload across all WireGuard key readers
- forward `WGTunnelSpec.iface` and `.netns` to the read
- reject malformed declarations before netlink I/O
- export the async helper and cover local, peer and absent keys
- replace multihost's `wg show` subprocess probe

Prompt-IO: ai/prompt-io/opencode/20260822T023226Z_59a8ecfd_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 35e496c166 Sketch first-child `wgman` supervisor
Add a candidate Layer-C architecture where a private, eagerly
spawned manager owns pyroute2 and tunnel provisioning for a simple
WG-enabled actor tree.

Deats,
- overlap manager reconciliation with sibling process startup
- contain `AsyncWireGuard` in an infected-asyncio child
- limit requests and capabilities by bindspace security domain
- define readiness, crash, restart and teardown semantics
- retain pre-provisioned and multi-manager escape hatches

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 0a0aa64305 Read WireGuard iface keys through `pyroute2`
Add async `read_wg_pubkey()` and `read_wg_peers()` helpers which
offload `WireGuard.info()` calls to a `trio` worker thread.

Deats,
- add the Linux-only `wg` extra and pin `pyroute2`
- pass `flags=0` so a read never creates a named netns
- normalize multipart replies, validate keys and de-dup peers
- always close the netlink client, including error paths
- test thread offload, netns forwarding and client cleanup

Prompt-IO: ai/prompt-io/opencode/20260821T233204Z_5d92595f_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:11 -04:00
Gud Boi 51f4e31ce0 Emit canonical tagged addresses
- Make `TCPAddress.unwrap()` emit `('tcp', host, port)` and
  `UDSAddress.unwrap()` emit `('unix', path)` while retaining the
  compatibility readers from the preceding change.

- Pass concrete TCP fields to Trio, compose multiaddrs from tagged
  values, and let `SpawnSpec` carry protocol-specific tuple shapes
  for validation by `wrap_address()`.

- Compare runtime, registry, bind, and tunnel addresses through
  canonical serialized forms and cover both TCP and UDS operation.

Prompt-IO: ai/prompt-io/opencode/20260820T033108Z_ba07e09d_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:03 -04:00
Gud Boi 0a741282af Decode tagged transport addresses
- Define canonical `tcp` and `unix` tuple shapes while retaining
  legacy pair aliases as the emitted `UnwrappedAddress`.

- Dispatch tagged tuple/list payloads explicitly, accept `uds` as a
  Unix input alias, and preserve legacy TCP, UDS, and native IPv6
  readers.

- Cover tag aliases, msgpack-style lists, legacy payloads, and IPv6
  socket addresses before switching writers.

Prompt-IO: ai/prompt-io/opencode/20260820T033107Z_ba07e09d_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:03 -04:00
Gud Boi 419ce6a631 Model bindspaces as scoped capabilities
Separate serializable bindspace declarations from live namespace
identity, FDs, ownership and teardown resources.

Require child namespace entry during spawn bootstrap, before actor
runtime initialization, then distinguish listen/dial provisioning and
owned/borrowed cleanup without encoding operation role into maddrs.

Prompt-IO: ai/prompt-io/opencode/20260820T021516Z_dfad66a0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:03 -04:00
Gud Boi 176e0c3e61 Peel tunnels before `Endpoint` binding
Carry tunnel declarations through listener configuration, then strip
them immediately before constructing transport endpoints.

Also allocate random listener addresses from a contacted registry's
overlay, and prove a real TCP listener never stores the wrapper while
the source declaration retains its bindspace metadata.

Prompt-IO: ai/prompt-io/opencode/20260819T213145Z_f81fc5e5_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:03 -04:00
Gud Boi 6b8e9ad298 Peel tunnels before `Channel` connects
Retain tunnel annotations through address declaration, then hand only
the bindable overlay to exact-type transport lookup and dialing.

Broaden `Channel.from_addr()` and `_connect_chan()` inputs accordingly,
and cover plain plus tunnelled TCP dispatch arguments.

Prompt-IO: ai/prompt-io/opencode/20260819T213144Z_f81fc5e5_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:03 -04:00
Gud Boi bcf8f87ea5 Use discovery's `wg` parser in examples
Drop the example-local address struct and hand-rolled single-tunnel
parser now that discovery owns the production implementation.

Keep only the explicit `wg(8)` peer probe in the multihost helper,
and update the examples and plan for nested parsing, packaged codec
dependencies and tractor-owned bindspace provisioning.

Prompt-IO: ai/prompt-io/opencode/20260818T075031Z_dd02c7c0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:03 -04:00
Gud Boi c7451bb2cb Support nested `wg` maddrs
Teach discovery to preserve WireGuard bearer and identity metadata
around a bindable TCP overlay.

Deats,
- encode `wg(8)` keys as strict 32-byte multibase values
- peel nested stacks with `Multiaddr.decapsulate_code()` and compose
  them with `.encapsulate()` instead of splitting strings
- integrate wrappers with `parse_maddr()`, `mk_maddr()`,
  `wrap_address()` and `parse_endpoints()`
- pin the unreleased py-multiaddr#108 codec in package metadata
- cover exact round trips, nesting, bad grammar and missing codecs

Prompt-IO: ai/prompt-io/opencode/20260818T075031Z_dd02c7c0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:03 -04:00
Gud Boi fa8067f79f Add `TunnelledAddress` wrapper primitives
Introduce the first layer-A address type from the `wg` bindspace
plan without treating a transparent tunnel as a `MsgTransport`.

Deats,
- add frozen `WGTunnelSpec` and `TunnelledAddress` structs which
  delegate proto identity, bindspace, validity and wire
  serialization to their overlay
- add `strip_tunnels()` and `tunnels_of()` for nested wrappers
- recognize wrappers in `is_wrapped_addr()` while keeping them out
  of `_address_types`
- cover delegation, namespace fallback and nested peeling semantics

Also,
- widen `Address.namespace` ids for named netns
- export the new discovery API
- clarify that tractor's layer-C bindspace lifecycle may provision
  the kernel-owned bearer without making it a `MsgTransport`

Prompt-IO: ai/prompt-io/opencode/20260818T021729Z_d9a6e2e9_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:03:03 -04:00
Gud Boi 768b531662 Retract the hand-rolled tunnel peeler from plan-03
§3.2 specced a pure fn `_peel_tunnel_segs(proto_names) ->
(bearer_names, tunnel_specs, overlay_names)` to split a maddr at
its tunnel seg. It should never be written: `py-multiaddr` ships
that whole surface already and the plan simply missed it, even
though gh #443's 2nd bullet links the README sections in
question.

Replaced w/ a ⚠️ CORRECTION carrying the verified API table
(`.decapsulate_code(P_WG)` for the bearer, `.split()`/`.join()`
for a seg tail, `.value_for_protocol()` to read a value,
`.encapsulate()` to recompose) plus *why* it works on an infix
`/wg/` seg: the cut is by proto-code, never by matching an addr
value, and the key seg has no addr of its own.

Also,
- adopt `bearer`/`overlay` as the role names throughout, and say
  plainly why not `inner`/`outer` — the call-stack reading of
  "inner" is the exact opposite of the encapsulation one.
- warn that `value_for_protocol('ip4')` on a full tunnelled
  maddr silently yields the *bearer's* host; only call it on a
  peeled sub-maddr.
- note nesting (wg-in-wg) falls out of `.decapsulate_code()`
  cutting at the *last* occurrence, so peel repeatedly rather
  than recursing through a bespoke splitter.
- `mk_maddr()` for `TunnelledAddress` is `.encapsulate()`
  composition, not `str` building.
- README: drop the "degrades to a plain segment split" para,
  since that path is gone — no codec now means one actionable
  raise.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 4c8188652b Peel `wg` maddrs w/ `py-multiaddr`'s own tunnel API
`py-multiaddr` already ships the entire tunnel compose/peel
surface and this module was reimplementing it — a raw
`maddr.split('/')` plus index arithmetic, sitting directly under
a comment congratulating itself for not hand-rolling a parser.
Same NIH trap gh #429 existed to close, just one layer up. The
API was linked from gh #443's own 2nd bullet the whole time.

So every cut now goes through the real thing,

| need | API |
| --- | --- |
| isolate the bearer | `.decapsulate_code(P_WG)` |
| per-seg maddrs | `.split()` |
| rejoin a seg tail | `Multiaddr.join()` |
| read the key | `.value_for_protocol('wg')` |
| recompose | `.encapsulate()` |

`.decapsulate_code()` turns out to handle the infix `/wg/` seg
cleanly *because* it cuts on proto-code and never tries to match
an addr value — the key seg has no addr of its own, which was
the exact thing I'd assumed would need bespoke handling.

Deats,
- rename the role fields `inner`/`inner_proto` ->
  `overlay`/`overlay_proto`, matching `py-multiaddr`'s
  encapsulation model (earlier segs wrap later ones) and #443's
  owner table. `inner` collided head-on w/ call-stack `inner`,
  where it reads as higher-up + later-called, while here the
  encapsulated addr is bound *first* and sits deeper.
- drop `_segments()` and its degraded hand-split path entirely.
  W/o the codec there's now one actionable `RuntimeError`
  instead of a silent downgrade, superseding the swallow fix in
  7d6e7955.
- add `.as_multiaddr()` so callers can stay in `Multiaddr` land;
  `.maddr` is now just `str()` of it.
- accept `str|Multiaddr` on the way in.
- carry `bearer_ip`/`overlay_ip` so a v6 stack re-renders as v6
  — the old `.maddr` hardcoded `/ip4/` and would silently
  mangle it.
- both host scripts follow the rename to `.overlay`.

⚠️ `value_for_protocol('ip4')` on a *full* tunnelled maddr
silently returns the **first** match, i.e. the bearer's host, so
it's only ever called here on an already-peeled sub-maddr.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 8e39fd8b07 Update `wg` docs for the merged py-multiaddr#108
it lands" framing in plan-03 and the example README was stale in
both directions: the branch pin is obsolete, yet you still can't
just `pip install multiaddr`.

Deats,
- §3.2's grammar table is now re-verified against the upstream
  merge (`f86519da`) rather than only `baudco@wg_support` in a
  throwaway venv. Also notes the codec enforces a 32-byte key,
  so a truncated one is a `StringParseError` and not a silently
  mangled parse.
- §1 says merged-but-unreleased; the still-open work is spec
  registration (py-multiaddr#107 + gh #483).
- §3.4 swaps "pin the branch" for the `[tool.uv.sources]` `rev`
  pin, and fixes the `_have_wg_maddr_proto()` recipe it
  suggested — probing w/ `Multiaddr('/wg/uAAAA')` now ALWAYS
  raises bc the codec wants 32B, i.e. that feature-detect would
  report `False` even w/ the proto perfectly well known.
- risk table row goes "#108 not merged" -> "merged but
  unreleased".
- example README: `uv sync` alone now suffices bc of the pin;
  documents the 32B check and points at
  `_have_wg_maddr_proto()` as the gate.

The one surviving `baudco` mention is deliberate, it records
where the grammar was *first* verified.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 13cdc43f01 Fix silently-corrupt keys in `parse_wg_maddr()`
`_segments()` called `Multiaddr(maddr)` purely to validate, then
swallowed every failure under `except Exception: pass`. That was
harmless pre-#108 — w/o a `wg` codec there was nothing to
validate — but now that the codec is pinned in, the swallow is
load-bearing and disabled: a malformed key sails past validation
into `wg8_pubkey()`, which happily emits a corrupt b64 str, and
the returned struct then fails its own `.maddr` round-trip. No
raise, just quietly wrong output.

Deats,
- add `_have_wg_maddr_proto()`, the gate plan-03 already
  referenced but which never actually existed. Impl'd as
  `protocols.protocol_with_name('wg')` under
  `except ProtocolNotFoundError` and cached in a mod global,
  same shape as the TIPC plan's `is_tipc_available()`.
- only validate when that gate is `True`, and let
  `StringParseError` propagate — a maddr which doesn't parse
  must NOT reach `wg8_pubkey()`.
- keep the degraded split for a pre-#108 install, now w/ an
  explicit `XXX` naming the validation you give up.

So parsing stays pure but becomes total-or-raises. Our own
`ValueError`s (missing `/wg/` seg, bare tunnel w/o an overlay
ep) are unaffected, as is the `wg(8)` b64 round-trip.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi ba6b181cf5 Pin `multiaddr` to the merged `wg` codec rev
py-multiaddr#108 (the `/wg/u<key>` maddr proto) merged upstream
on 2026-07-28 as `f86519da`, but ships in no release yet — the
latest `0.2.0` predates it by ~4 months and carries no `wg`
codec at all. So `examples/multihost/wg_lan/` can't parse its
own maddrs off PyPI.

Pinned by `rev` and not `branch` so CI stays reproducible. Note
the lock now records the git source *instead of* the `>=0.2.0`
specifier, i.e. the dep floor above is fully overridden for as
long as this pin lives.

TODO, drop the pin (and bump that floor) the moment a release
carries the codec; the only consumer is the `wg_lan` example
set.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi cd1e8160fe Log prompt-io for the tpt-backend planning arc
One record covering all 9 commits on this branch, per the NLNet
generative-AI policy and the existing `ai/prompt-io/claude/`
convention.

Uses diff-ref mode for both the plan docs and the example code
(`git diff main..ng_tpts_planning -- <path>`) rather than
duplicating content already in `git log -p`. Kept verbatim in
the `.raw.md`: the four verified findings (trio's
family-agnostic `SocketStream`/`SocketListener`, the round-trip
table proving `/wg/` is infix, the proto-key `UnwrappedAddress`
rationale, and `setns(2)`'s per-thread reality), since those are
reasoning rather than diffable output.

`## Human edits` records that the steering here was substantial
and mid-session rather than post-hoc: two model claims about wg
maddr semantics were challenged and retracted (incl. in an
already-posted issue comment), and the proto-key +
netns-as-runtime-config framings were human-directed. Also notes
the one model-initiated correction — a pre-publication
self-review that downgraded the `uniffi`/asyncio thesis and the
TIPC duplicate-binder claim to explicitly-flagged assumptions.

Prompt-IO: ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 3c6bd928b7 Move the `wg_lan` examples under `examples/multihost/`
`tests/test_docs_examples.py` walks `examples/` **recursively**
and subproc-runs every collected file asserting `rc == 0`. Ran
its exact filter against the tree: all 4 of our files were being
collected — including `README.md`, since the filter never checks
the extension, so CI would have literally tried `python
README.md`. These need a real second host + a live `wg` tunnel,
so they can't ever satisfy that gate.

`'multihost' not in p[0]` is already in the test's exclusion
list w/ no dir yet using it, so this is a pure `git mv` — zero
test changes — and it's what the exclusion was plainly there
for. Collection drops 24 -> 20 files, 0 of them ours.

Also records *why* in the two places someone would look before
adding the next one: a callout at the top of the example README
and a note on plan 03's §3.4 deliverables. Anything needing a
second host or live tunnel goes under `examples/multihost/`.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 0457e8f3b1 Add a `wg`-tunnelled 2-host example set
Re-renders gh #482's examples w/ the corrected (infix) maddr
grammar, as the "layer A" slice of the wg plan: declarative
maddrs only, tunnel pre-provisioned out-of-band, zero runtime
changes.

- `wg_maddr.py`: a `frozen=True` `msgspec.Struct` addr carrying
  `bearer`/`peer_pubkey`/`inner` (+ `inner_proto`), a `.maddr`
  property that re-renders the canonical form, and pure
  `mb_pubkey()`/`wg8_pubkey()`/`parse_wg_maddr()`. The parser
  rejects #482's inverted suffix form w/ an actionable error and
  stays **side-effect free** — `verify_wg_peer()` is a separate,
  explicitly impure step the caller composes, never something a
  parse path shells out to.
- `host_a_srv.py`/`host_b_client.py`: the two-host runs, passing
  only `addr.inner` into `open_nursery()`/`open_root_actor()`,
  which is the whole point — the bearer + key layers are already
  established before any bind happens.
- `README.md`: the grammar + the 3-owners table, the `#108`
  branch install line, tunnel setup, and a "what changed vs
  #482" section enumerating the corrections.

Runnable-shaped but **not yet run against a live tunnel**; that's
next, and the reason these sit on the planning branch rather than
in `examples/` proper. `_segments()` marks its stopgap for when
the `wg` codec isn't installed.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 86417fa19c Fix the `wg` maddr grammar, `/wg/` is *infix*
The prior revision (and gh #482's examples) had it as a suffix,
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. Wrong: verified against
`baudco/py-multiaddr@wg_support` (py-multiaddr#108) installed in
a throwaway venv, the canonical form is

  /ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616

where segs *before* `/wg/` are the **bearer** — the underlay
`(ip, udp-port)` `wg(8)` itself listens on (`ListenPort`), per
the codec docstring's own example — and segs *after* are the
**overlay** ep, the only part we ever bind. The suffix form does
parse, which is why it slipped through, but it's semantically
inverted: overlay addr where the bearer belongs, `tcp` where
wg's `udp` goes, and no overlay ep declared at all.

Records the observed `[p.name for p in m.protocols()]` lists so
the `match` can be written against fact, and replaces the
"composed vs not" framing w/ what's actually the design axis:
three parts, three **owners** — bearer bound by the kernel via
`wg-quick`/`pyroute2`, `/wg/u<key>` bound by nothing (it's an
identity, verified out-of-band), overlay bound by our
`IPCServer` as `.inner`. `_peel_tunnel_segs()` correspondingly
grows a 3rd return, splitting *at* the tunnel seg so nested
tunnels fall out for free.

Also hoists the netns conclusion to the top of §5.3 where it
can't be missed: netns is a **runtime-level config API, not an
actor-app-code one**. It's a spawn/boot-time input alongside
`enable_transports`/`tpt_bind_addrs`, deliberately w/ no
`await actor.enter_netns(...)`, because `setns(2)` neither moves
already-created sockets nor applies beyond the calling thread —
so a mid-life API would silently leave the IPC server bound in
the old ns.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi a03b52619a Proto-key the unwrapped-addr form in the plans
Shape-matching in `wrap_address()` doesn't survive 4 backends and
the plans were papering over it: TIPC's natural unwrapped form is
a `(str, int)`, indistinguishable from `TCPAddress`, and iroh's
is a `(str, str)`, which the *existing* UDS case
(`case (_, filename) if type(filename) is str`) already swallows.

So the contract doc (§1.1) now carries the conclusion as a
**recommended prerequisite for all three backends**: make the
unwrapped form carry an explicit proto-key spelled with the
`multiaddr` protocol name — `('tcp', host, port)`,
`('unix', path)`, `('tipc', stype, inst, scope)`. `wrap_address()`
then collapses from an order-sensitive `match` to
`_address_types[addr[0]]` and the whole collision class stops
existing, while the on-wire form finally agrees w/
`mk_maddr()`/`parse_maddr()` instead of being an independent
invention.

Two consequences spelled out: it's a wire-format change
(`SpawnSpec`, `_root_mailbox`, `_registry_addrs`) + every fixture
+ downstream config, so it wants its own migration commit landed
*before* any new backend; and it's the moment to stop handing raw
tuples to users at all — `Address` becomes the public currency
and `UnwrappedAddress` an internal serialization detail, the same
discipline `ipaddress` uses (you pass `IPv4Address`, never a
4-tuple).

Plan 01 §2.2 is rewritten to match and to explicitly **retract**
its own earlier `('tipc:<stype>:<scope>', instance)` self-tagging
prefix hack — it keeps `wrap_address()` order-sensitive and does
nothing for the iroh/UDS collision, so the doc says don't
resurrect it. Registration checklist item 4 likewise becomes "do
the migration first, then this is a one-line `_address_types`
entry".

Also seeds a `/tipc` multiaddr-spec submission as a follow-up,
mirroring the `wg` track (multiformats/py-multiaddr#107/#108 + gh

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 9cdbe36bca Index the tpt-backend plans w/ a README
Landing page for `ai/tpt-backends/`: points at the contract spec
as required first reading, tables the 3 plans against their
issues/deps/size, and states the landing order + why.

Deats,
- TIPC first as the cheap proof the table-registration story
  generalizes to a genuinely new proto (stdlib-only, and
  `trio`'s sock wrappers are family-agnostic).
- `wg` layer-A next since it's deployable-today doc/example work.
- QUIC last, gated on its own prep PR.
- notes that plans 01 and 02 both want the same
  `Address.rebind_from_sockname` gate, so whichever lands first
  ships it.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 325369af22 Add `wg`-as-nested-bindspace plan doc
Plan doc for gh #482 + the tunnelled-maddr item of #443. Pushes
back on the framing that `wg` is a tpt: it's transparent to
`socket(2)`, so it belongs as a *bindspace* — a scoped
`@acm`-managed net ctx that an existing L4 tpt binds *inside* —
and it's what finally implements the long-spec'd (never
implemented) `Address.namespace`.

Deats, 3 independently-shippable layers,
- A) declarative: commit #482's examples, teach `parse_maddr()`
  the `/…/wg/u<key>` suffix -> a `TunnelledAddress` wrapper whose
  `.proto_key`/`.unwrap()` delegate to `.inner` so nothing new
  crosses the wire and every existing table lookup keeps working.
- B) swap the `subprocess.run(['sudo', 'wg', 'show'])` shelling
  for `pyroute2`. Default to `trio.to_thread` around the sync API
  (these are one-shot ops at bind/teardown, never hot-path), w/
  sans-io codecs + a trio `AF_NETLINK` sock as the follow-up for
  the read paths. Explicitly forbids dragging `trio-asyncio` in.
- C) `open_bindspace()`/`open_netns()`/`open_wg_iface()` `@acm`s
  folded w/ an `AsyncExitStack`, + filling in the
  `# !TODO, always be ns aware!` placeholder already sitting in
  `Endpoint.pformat()`.

Also flags the subtlest bug in the whole thing: `setns(2)` is
*per-thread*, so a `pyroute2` query issued via `trio.to_thread`
lands in the *original* netns. Test-first, per usual.

Further, designs for the generalization (`TunnelSpec` union +
`match` dispatch) while only implementing `wg`+netns, and calls
out `veth`-in-netns as the better *first* one bc it makes a
fully self-contained two-"host" integration test possible w/o
`wg` at all.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi 07ba2be07d Add `QUIC`-via-`iroh` tpt-backend plan
Plan doc for gh #353. Picks `iroh` (the `uniffi` FFI pkg) over
`aioquic`/`quiche` bc node-id addressing + hole-punching + relay
fallback is the whole point; `aioquic` stays documented as the
fallback since ~90% of the adapters here are reusable against a
sans-io core.

Deats,
- the layering: iroh `Endpoint` per actor, `Connection` per peer
  (pooled via `trionics.maybe_open_context()`, not a hand-rolled
  cache), one bi-stream per `Channel`. 4-byte prefix framing
  stays so `MsgpackTransport` is untouched.
- `_uniffi_trio.py`: uniffi only uses `asyncio` as the executor
  for its rust-future poll loop, so a ~40-line
  `TrioToken.run_sync_soon()` bridge replaces it. Spells out the
  real hazards — strong ref on the `ctypes` trampoline, poll-code
  propagation, and a *bounded* shielded cancel-drain so a wedged
  rust future can't make an actor un-cancellable.
- `IrohAddress` w/ ALPN as the `.bindspace`, the `(str, str)`
  unwrapped form's collision w/ the UDS match-case, and why
  `get_root()` needs a persisted secret key -> a lazy
  `default_lo_addrs()` + a pure-getter/explicit-setter split.
- `QuicMsgStream(trio.abc.HalfCloseableStream)` +
  `QuicListener(trio.abc.Listener)`, incl. the exact
  EOF/reset/use-after-close semantics `_transport.py` already
  match-cases on, and hanging the acceptor tasks off the
  existing `Endpoint.listen_tn`.
- a prep-PR boundary: annotation widening, the shared
  `rebind_from_sockname` gate and a `tpt_key`-based
  `transport_from_stream()` dispatch, all landable w/ tcp/uds as
  the only backends.

Further, notes this is our first tpt w/ real transport security
+ peer auth, so an inbound node-id allowlist hook belongs here —
and that it says nothing about the other backends.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi f81dd82b89 Add `TIPC` tpt-backend impl plan
Plan doc for gh #378, the cheapest new backend we can add: it's
stdlib-only (CPython ships `AF_TIPC` + 23 `TIPC_*` consts) and
per the contract doc `trio`'s stream/listener wrappers don't care
about the addr family, so `MsgpackTransport` framing and
`trio.serve_listeners()` are reused verbatim.

Deats,
- `TIPCAddress` as a *service name* `(type, instance)` w/ scope
  as the `.bindspace`; `bind()` publishes the singleton
  name-range, peers `connect()` by name and the kernel resolves
  + load-balances. I.e. registration/lookup for free, no
  registrar in the loop.
- the self-tagging `('tipc:<stype>:<scope>', instance)` unwrapped
  form + why it must be match-ordered before `TCPAddress`'s.
- `get_random()` via a blake2b digest of the actor id (there's no
  `port=0` analogue) and the silent-crosstalk risk that follows:
  TIPC *allows* dup binders and round-robins, so a collision
  doesn't `EADDRINUSE`, it cross-talks.
- an `Address.rebind_from_sockname` ClassVar to opt out of
  `Endpoint.start_listener()`'s `getsockname()` reconcile, which
  for TIPC always returns a port-id, never the bound name.
- the `TIPC_TOP_SRV` topology-service subscription as an `@acm`
  yielding a chan of typed name-table events — push-based
  register/dereg, the real "end game cluster proto" bit.
- commit sequencing, hard capability gating (`modprobe tipc`;
  bare `AF_TIPC` is `EAFNOSUPPORT` on a stock box), CI matrix
  notes, risks + follow-up seeds.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Gud Boi e25e018370 Add the `.ipc` tpt-backend contract spec
First doc of a new `ai/tpt-backends/` set: the normative
description of what a `tractor` tpt backend *is* as of `main`,
written so the 3 sibling plans (TIPC, QUIC, `wg`) can be worked
independently (by another model/provider) w/o design drift.

Deats,
- the backend duck-type as empirically derived from
  `_tcp.py`/`_uds.py`: the `Address` protocol surface, the
  mod-level `start_listener()`/`close_listener()` pair and
  `Msgpack<Proto>Stream(MsgpackTransport)`.
- the ONE reflection you can't break:
  `Endpoint.start_listener()` resolves the tpt mod via
  `inspect.getmodule(self.addr)`, so an `Address` type and its
  listener fns MUST live in the same mod.
- a 10-item registration checklist (`_address_types`,
  `_key_to_transport`, `_addr_to_transport`, `wrap_address()`
  match-cases, `TransportProtocolKey`, maddr tables, ..) incl.
  the import-time `_default_lo_addrs` trap.
- where the `trio.SocketListener` assumption is *actually*
  load-bearing (just the `getsockname()` reconcile) vs. merely
  annotated.
- the handshake/discovery invariants a new backend inherits,
  dep policy (extras + import-laziness per the #470 boot-latency
  budget), `--tpt-proto` harness plumbing and code style.

Also, records a verified finding the plans lean on hard:
`trio.SocketStream`/`SocketListener` are addr-*family* agnostic
— the only ctor checks are "is a trio sock" + `SOCK_STREAM` (+
an `OSError`-suppressed `SO_ACCEPTCONN`) — so any `SOCK_STREAM`
family CPython can make drops into the existing
`trio.serve_listeners()` path unmodified.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:02:56 -04:00
Bd f220b4641d
Merge pull request #512 from goodboy/wkt/big_boi_docs_472_follow_ups
post-#472 docs follow ups
2026-08-29 22:02:20 -04:00
Gud Boi 7796480062 Widen `accept_addr` in the UDS example
Match `Actor.accept_addr`'s declared `tuple[str, int|str]` contract
instead of narrowing its second item to the UDS-specific string path.

Review: https://github.com/goodboy/tractor/pull/512#discussion_r3888153533

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 21:46:38 -04:00
Gud Boi d799bd6f68 Show the discovered actor address
Print the discovered service's remote channel address instead of
the default `Portal` object repr.

Review: https://github.com/goodboy/tractor/pull/512#discussion_r3883250916

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 20:24:25 -04:00
Gud Boi a95ca7577f Record PR #512 prompt provenance
Capture the human direction, generated changes, review findings and
validation results for the docs-example landing pass.

Point generated-code references at the complete commit range from
the pre-remediation branch head.

Prompt-IO: ai/prompt-io/opencode/20260828T200822Z_0be872ff_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 20:10:13 -04:00
Gud Boi a40fb2ebde Correct typed-msg validation docs
Document `Started` as the eager sender-side payload check and
`Yield` plus `Return` as receiver-side decoding boundaries without
promising a symmetric error relay.

Separate the working task-scoped codec encoder from the private
per-dialog decoder and the incomplete `@context` hook params.

Link the planned typed `Start` contract and sender-side argument
validation follow-up in #514.

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 20:07:06 -04:00
Gud Boi 38883dca03 Harden the dedicated registrar example
Run the registrar in its own process and prove that a sibling client
discovers the service through registry lookup instead of an existing
peer channel.

Retry ephemeral bind collisions, publish readiness atomically and
validate bounded cross-platform shutdown. Document actual duplicate
name and multi-registrar ordering semantics alongside the example.

Move the demo under the discovery examples and wrap process ownership
in an `@acm`. Record the future public subsystem, Piker service and
pytest isolation follow-ups.

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 20:05:28 -04:00
Gud Boi 97aff52050 Polish the debugger examples
Finish the Python style, typing and docstring pass across the
debugger examples without changing their intentional breakpoints,
failures, cancellation races or timeout reproducers.

Restore full child command lines in the documented process trees
and keep the examples within the 69-column source limit.

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 19:01:34 -04:00
Gud Boi 96a1838210 Polish the non-debug docs examples
Finish the Python style, typing and docstring pass across the
ordinary, parallelism, Trio and integration examples.

Preserve each demo's runtime behavior while tightening callable,
portal, stream and nursery annotations. Use the modern `.chan`
portal attr and current actor-lifecycle terminology.

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 18:58:43 -04:00
Gud Boi 0be872ff97 Type the remaining niche examples
Finish the examples-typing sweep with the last non-docs-visible
scripts: `-> None` on the two `trio/` behavior-demo mains (plus a
`trio.TaskStatus` on `hold_lock_forever`) and nursery/portal typing
on `integration/mpi4py/inherit_parent_main.py`.

Leaves `concurrent_futures_primes` (a verbatim stdlib baseline) and
`integration/open_context_and_sleep` (its tractor nursery is
commented out) as-is, and the paren-group `trio.open_nursery()`
bindings unannotated (no clean spot for a preceding annotation).
Completes the examples-typing bullet in #472.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-28 14:53:35 -04:00
Gud Boi 319868e92d Type the docs-visible `examples/` scripts
Type the runtime objects (`ActorNursery`, `Portal`, `Context`,
`trio.Nursery`) + fn signatures across the 16 highest-visibility,
`literalinclude`-d `examples/` scripts, matching the front-page
`we_are_processes.py` style — so the rendered guides show typed
usage throughout, not just on the landing snippet.

Spans the 3 quickstart-backing scripts + `single_func`,
`remote_error_propagation`, `multiple_streams_one_portal`,
`quick_cluster`, `service_discovery`, `service_daemon_discovery`,
`asynchronous_generators`, `nested_actor_tree`,
`concurrent_actors_primes`, `streaming_broadcast_fanout`,
`rpc_bidir_streaming`, `infected_asyncio_echo_server`,
`typed_payloads`.

Annotation-only (no renames/logic changes); each runs green and the
docs build stays warning-free. Part of the examples-typing bullet
in #472.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-28 14:53:34 -04:00
Gud Boi a6c1158853 Add basic typing to the `debugging/` examples
Sweep the `examples/debugging/` set for basic typing: add `-> None`
to all 16 bare `async def main()`s and annotate the clean
single-line `open_nursery()` bindings as `tractor.ActorNursery`.

Kept to the unambiguous, runtime-safe cases (these breakpoint/crash
demos can't be run headless); the heterogeneous
multi-line/paren-group nursery bindings + `current_actor()` returns
are left for a later pass. Continues the examples-typing bullet in

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-28 14:47:18 -04:00
Gud Boi 4c507ea13c Add a dedicated-registrar example + discovery guide
Add a runnable `examples/dedicated_registrar.py` + a "A dedicated
registrar" subsection in `guide/discovery.rst` demoing the
registrar decoupled from any app tree's root: boot a bare
`tractor.run_daemon([], registry_addrs=[...])` as its own process
(a root actor that does nothing but hold the registry), point the
app tree at the same `registry_addrs`, and discover a service
*through* that external registrar.

This is the buildable-today form of the #472
"Registrar-as-subsystem (not the root actor)" bullet. Two
constraints are called out inline as follow-ups:
`enable_transports` is single-proto per runtime (no multi-backend
registrar yet), and a registrar can only be a root (no `actor_cls`
hook on `start_actor()` to spawn one as a subactor).

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-28 14:40:01 -04:00
Gud Boi a279cb39a9 Expand caps-based-msging docs w/ #365, #376 + tests
The "Toward capability-based msging" section only pointed at the
`#196`/`#36` epics. Fold in the concrete recent state,

- `#365` as the most recent step: driving the whole `pld_spec` off
  plain type-annotations (e.g. annotating a context's
  `open_stream()` with `msgspec.Struct` subtypes) rather than
  explicit `pld_spec=` kwargs.
- clarify that the decorator-level `@tractor.context(pld_spec=...)`
  is already the higher-level path (vs the lower-level
  `tractor.msg._ops.limit_plds()` escape hatch), pointing at
  `tests/msg/test_pldrx_limiting.py` + `test_ext_types_msgspec.py`
  which exercise both.
- `#376` (from @guilledk, `auto_codecs` branch) as the drafted
  public factory API for the `enc_hook`/`dec_hook` pair (today only
  reachable via `tractor.msg._ops`).

Addresses the caps-based-msging bullet in #472.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-28 14:40:01 -04:00
140 changed files with 12778 additions and 458 deletions

View File

@ -0,0 +1,136 @@
---
model: claude-opus-5
service: claude
session: 7b9c97c4-fff7-4ac4-97fb-35720453308e
timestamp: 2026-08-13T00:11:02Z
git_ref: 27c34aeb
scope: docs+code
substantive: true
raw_file: 20260813T001102Z_27c34aeb_prompt_io.raw.md
---
## Prompt
> draft hyper detailed implementation plans for [three]
> prospective new transport (tpt) backends for tractor's `.ipc`
> layer, from four GitHub issues: TIPC (gh #378) using built-in
> linux socket API w/ `trio` interfacing, leveraging TIPC's
> built-in discovery machinery; QUIC (gh #353) using the `iroh`
> lib, ideally with the py asyncio support (via ffi) rewritten
> for trio; wg (gh #482 and/or #443) with other shuttle-able
> tpts, using `pyroute2`, as much trio wrapping as possible
> where any other async support can be replaced.
With constraints: "be only slightly speculative"; realistic given
the linked info; the plans must be workable "by another
model/provider without much discrepancy on the design or lib
selections needed"; do it all in a new `/open-wkt` off main; and
be "as modular and modern-async-python-as-nearly-functional-style
as possible."
Key follow-up prompts that changed the design mid-session:
> wg requires binding a socket on both ends of the tunnel, the
> new support we added to `py-multiaddr` in their #108 enables
> declaring this in the maddr? […] so the
> `/ip4/10.0.0.1/udp/666/ip4/192.168.0.13/tcp/80` bullet example
> from #443 isn't correct for a wg tunnel maddr and would
> actually end up looking like,
> `/ip4/10.0.0.1/udp/666/wg/<pub-key>/ip4/192.168.0.13/tcp/80` no?
> right, we're likely going to need less `tuple`-formed
> `UnwrappedAddress` usage to make them all work and/or to extend
> the unwrapped form with a proto-str, likely the key from
> `multiaddrs` no?
> better encapsulating `UnwrappedAddress` for use externally is
> really way overdue; we should likely be encouraging (if not
> preventing unwrapped) wrapped addrs always much like other
> (std) libs, `ipaddress` comes to mind.
> so iirc netns mgmt becomes a lower-level runtime config-api
> rather then an actor-app-code API which can be dynamically
> changed once the actor is up ya?
## Response summary
A research-and-planning branch, `ng_tpts_planning` — **no runtime
code touched**. Five plan docs under `ai/tpt-backends/` plus a
corrected `wg` example set under `examples/multihost/wg_lan/`,
across 9 commits.
Three design conclusions came out of the planning that outlive
the branch:
1. **`wg` is a bindspace, not a `MsgTransport`.** The `/wg/`
maddr segment is *infix*: bearer (underlay ip+udp
`ListenPort`, bound by the kernel) → `/wg/u<key>` (an
identity, bound by nothing) → overlay (the only part tractor
binds). Verified empirically by installing
`baudco/py-multiaddr@wg_support` (py-multiaddr#108) in a
throwaway venv and round-tripping each candidate form; gh
#482's original suffix form parses but is semantically
inverted.
2. **Proto-key the `UnwrappedAddress` form**, spelled with the
`multiaddr` protocol names, dispatched via
`_address_types[addr[0]]`. Kills a whole collision class
(TIPC's `(str, int)``TCPAddress`; iroh's `(str, str)`
swallowed by the UDS case) and is the recommended migration
*before* any new backend lands.
3. **netns is a runtime/boot-time config API, not an app-code
one** — `setns(2)` is per-thread and won't move
already-created sockets, so there is deliberately no
`await actor.enter_netns(...)`.
Also verified that `trio.SocketStream`/`SocketListener` are
address-family agnostic (no `AF_*` check anywhere), which is what
makes TIPC the cheapest of the three backends to add.
Four related issues were annotated with the results (#378, #353,
#482, #443); #443's body was rewritten to reflect the corrected
grammar, with no existing checkbox state changed.
## Files changed
- `ai/tpt-backends/00_shared_backend_contract.md` — normative
backend duck-type contract, registration checklist, §1.1
proto-key conclusion
- `ai/tpt-backends/01_tipc_backend.md` — TIPC plan; service
addressing, `TIPC_TOP_SRV` push registry, instance-collision
hazard, step-0 probe
- `ai/tpt-backends/02_quic_iroh_backend.md``iroh` plan;
`uniffi`→`trio` bridge, listener/stream adapters, API-truth
table
- `ai/tpt-backends/03_wg_tunnel_bindspace.md``wg`-as-bindspace
plan; verified maddr grammar, 3-owner split, netns reality
- `ai/tpt-backends/README.md` — index
- `examples/multihost/wg_lan/wg_maddr.py` — frozen `msgspec`
tunnelled addr + pure parse/render helpers; impure
`verify_wg_peer()` kept separate
- `examples/multihost/wg_lan/host_a_srv.py` — host-A actor tree
- `examples/multihost/wg_lan/host_b_client.py` — host-B dialer
- `examples/multihost/wg_lan/README.md` — grammar, owner table,
setup, "what changed vs #482"
## Human edits
Substantial human steering rather than post-hoc editing; the
corrections were applied by the model in-session after being
challenged:
- rejected an initial claim that `wg` has "nothing to bind at the
tunnel layer" and supplied the correct composed maddr form,
which forced a rewrite of plan 03 §3.2 and a retraction in the
already-posted #443 comment
- rejected a supporting claim that `/ip4/../udp/443/quic-v1` was
"also composed"
- directed the proto-key/`ipaddress`-discipline conclusion and
the netns-as-runtime-config framing, both of which were then
folded back into the docs
- chose the commit boundaries and authored all commits; ran every
`git` mutation (commit, rebase, push) themselves
One model-initiated correction pre-publication: a self-review
downgraded two overconfident claims (the `uniffi`/asyncio thesis
and TIPC duplicate-binder behaviour) to explicitly-flagged
assumptions before the #353/#378 comments were posted.

View File

@ -0,0 +1,165 @@
---
model: claude-opus-5
service: claude
timestamp: 2026-08-13T00:11:02Z
git_ref: 27c34aeb
diff_cmd: git diff main..ng_tpts_planning
---
# Raw output — next-gen tpt-backend implementation plans
## Generated planning docs
> `git diff main..ng_tpts_planning -- ai/tpt-backends/`
Five markdown docs. `00_shared_backend_contract.md` is normative
and the other three are written against it so they can be worked
independently:
- **`00_shared_backend_contract.md`** — the backend duck-type
(`<Proto>Address(msgspec.Struct, frozen=True)` + module-level
`start_listener()`/`close_listener()` + a
`Msgpack<Proto>Stream(MsgpackTransport)`), the
`inspect.getmodule(self.addr)` reflection in
`Endpoint.start_listener()` that forces the Address class and
its listener fns to share a module, a 10-item registration
checklist, the dep policy, the test-harness shape, and §1.1's
proto-key conclusion (below).
- **`01_tipc_backend.md`** — service addressing via
`TIPC_ADDR_NAMESEQ` (bind/publish) and `TIPC_ADDR_NAME`
(connect/lookup), `TIPC_TOP_SRV` topology subscriptions as a
push-based registry, the `get_random()` instance-collision
hazard, and a step-0 capability-probe spike.
- **`02_quic_iroh_backend.md`** — `iroh` over
`aioquic`/`quiche`/`trio-asyncio`, a `_uniffi_trio.py` bridge
built on `TrioToken.run_sync_soon()`, `trio.abc.Listener`/
`HalfCloseableStream` adapters, and an API-truth table to fill
in during step 0.
- **`03_wg_tunnel_bindspace.md`** — `wg` as a *bindspace* rather
than a `MsgTransport`, a `TunnelledAddress` wrapper delegating
`.proto_key`/`.unwrap()` to `.inner`, `pyroute2` for layer B,
and `@acm`-managed netns/iface for layer C.
- **`README.md`** — index.
## Generated example code
> `git diff main..ng_tpts_planning -- examples/multihost/wg_lan/`
- `wg_maddr.py``WGTunnelledAddr(msgspec.Struct, frozen=True)`
carrying `bearer: tuple[str, int]`, `peer_pubkey: str`,
`inner: tuple[str, int]`, `inner_proto: Literal['tcp']`, plus a
`.maddr` property that re-renders the canonical form. Pure
helpers `mb_pubkey()`, `wg8_pubkey()`, `parse_wg_maddr()`, and
`_segments()` (with a marked stopgap for when the `wg` codec
isn't installed). `verify_wg_peer()` is impure **by design** and
kept out of the parse path.
- `host_a_srv.py` / `host_b_client.py` — the two-host runs; both
pass only `addr.inner` to `open_nursery()`/`open_root_actor()`.
- `README.md` — grammar, owner table, `#108`-branch install line,
tunnel setup, "what changed vs #482".
## Verified findings (non-code, verbatim)
### `trio` is address-family agnostic
Read against the installed `trio`. `SocketStream`/`SocketListener`
ctor checks are only "is a trio sock object" + `type ==
SOCK_STREAM`, plus an `OSError`-**suppressed** `SO_ACCEPTCONN`
probe. No `AF_*` check anywhere; `TCP_NODELAY`/`TCP_NOTSENT_LOWAT`
are set under `suppress(OSError)`. A TIPC `SOCK_STREAM` sock should
therefore drop straight into `trio.serve_listeners()` with the
existing `MsgpackTransport` framing, making TIPC mostly
table-registration boilerplate w/ zero new deps.
### the `wg` maddr grammar — `/wg/` is infix, not suffix
Installed `baudco/py-multiaddr@wg_support` (PR
multiformats/py-multiaddr#108) into a throwaway venv and
round-tripped every candidate form:
| maddr | `[p.name for p in m.protocols()]` |
| --- | --- |
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
| `/ip4/10.0.11.1/tcp/1616/wg/u<k>` | `['ip4','tcp','wg']` |
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\_______ bearer __________/\__ key __/\______ overlay ______/
```
Segments *before* `/wg/` are the bearer — the underlay
`(ip, udp-port)` that `wg(8)` itself listens on (`ListenPort`).
Segments *after* are the overlay endpoint, the only part tractor
binds. The third row above is #482's original suffix form: it
parses, but is semantically inverted.
Three parts, three owners — and only one is an `Endpoint`:
| part | bound by | in the runtime? |
| --- | --- | --- |
| bearer | kernel, via `wg-quick`/`pyroute2` | no |
| `/wg/u<key>` | nothing — an identity | no, verified out-of-band |
| overlay | `tractor`'s `IPCServer` | yes, as `.inner` |
### proto-key-tagged `UnwrappedAddress`
Shape-matching in `wrap_address()` does not survive four backends.
TIPC's natural unwrapped form is a `(str, int)`, indistinguishable
from `TCPAddress`; iroh's is a `(str, str)`, already swallowed by
the existing UDS case (`case (_, filename) if type(filename) is
str`). Ordering hacks and prefix-tagging only paper over it.
Recommended prerequisite for all three backends: carry an explicit
proto-key spelled with the `multiaddr` protocol name —
`('tcp', host, port)`, `('unix', path)`,
`('tipc', stype, inst, scope)` — so `wrap_address()` collapses to
`_address_types[addr[0]]` and the collision class stops existing.
This also makes the on-wire form agree with
`mk_maddr()`/`parse_maddr()` instead of being an independent
invention. It is a wire-format change (`SpawnSpec`,
`_root_mailbox`, `_registry_addrs`) plus every fixture and
downstream config, so it wants its own migration commit landed
before any new backend — and it is the moment to stop handing raw
tuples to users at all, making `Address` the public currency and
`UnwrappedAddress` an internal serialization detail (the
discipline `ipaddress` uses).
### netns is a runtime-level config API
`setns(2)` affects the calling thread only and does not move
already-created sockets. So a netns is a spawn/boot-time input
alongside `enable_transports`/`tpt_bind_addrs`, and there is
deliberately no `await actor.enter_netns(...)` — a mid-life API
would silently leave the IPC server bound in the old namespace.
Corollary for layer B: pass `netns=` down to `pyroute2` rather
than assuming a `trio.to_thread` worker inherits it.
### `examples/` collection would have failed CI
`tests/test_docs_examples.py` walks `examples/` recursively and
subproc-runs every collected file asserting `rc == 0`. Its filter
never checks the extension, so all four `wg_lan` files were
collected — including `README.md`, which would have been run as
`python README.md`. `'multihost' not in p[0]` was already in the
exclusion list with no directory using it. Moving the set under
`examples/multihost/wg_lan/` drops collection 24 → 20 with zero
test changes; confirmed via `pytest --collect-only`.
## Corrections applied during the session
The human corrected two claims that had been asserted without
verification, both since retracted in-place in the docs and in the
posted issue comments:
1. that `wg` has "nothing to bind at the tunnel layer, exactly one
bind" — wrong; a wg stack is genuinely composed, and the real
axis is *who owns* each layer's endpoint.
2. that `/ip4/../udp/443/quic-v1` was "also composed" — wrong;
that is one endpoint with a protocol qualifier, not a tunnel.
A self-review before publication also downgraded two
overconfident claims to explicitly-flagged assumptions: the
`uniffi`-uses-asyncio-only-as-executor thesis (contradicted that
plan's own "do not guess from memory" step 0) and TIPC's
duplicate-binder round-robin behaviour (unverified).

View File

@ -0,0 +1,69 @@
---
model: gpt-5.6-sol
service: opencode
session: intercepted-claude-7b9c97c4-fff7-4ac4-97fb-35720453308e
timestamp: 2026-08-18T02:17:29Z
git_ref: d9a6e2e9
scope: code
substantive: true
raw_file: 20260818T021729Z_d9a6e2e9_prompt_io.raw.md
---
## Prompt
> Intercept Claude session
> `7b9c97c4-fff7-4ac4-97fb-35720453308e`, pick up where it
> stopped in its open worktree, finish the tunnelled-address change
> and commit plan, and prepare any outstanding context for another
> provider.
The recovered final prompt specifically called out the `uds` versus
`unix` boundary decision, absent concrete `Address.namespace`
implementations, the corrected namespace test, and the passing
focused and wider suites.
During review, the human further required that tunnelled-maddr work
delegate to `multiaddr`'s encapsulation APIs, challenged the premature
transport-shaped listener hooks and directed their removal, corrected
the long-term bearer provisioning model, and confirmed the intended
split between discovery metadata and bindspace lifecycle code.
## Response summary
Recovered the transcript and matched it to `wkts/addr_unpacking`,
audited the staged implementation, and completed the interrupted
verification and commit-plan work. The audit removed premature
transport-shaped listener hooks, widened the namespace identifier
type, updated stale import documentation, and removed an
invalid-escape warning from the maddr diagram. It also preserved the
layer-C design where tractor provisions the kernel-owned tunnel
bearer without treating it as a message transport.
## Files changed
- `tractor/discovery/_tunnel.py` - tunnel specs, address wrapper, and
peeling helpers.
- `tractor/discovery/_addr.py` - wrapped-address recognition and
namespace typing.
- `tractor/discovery/__init__.py` - public tunnel API exports.
- `tests/discovery/test_tunnelled_addr.py` - delegation and boundary
regression coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - distinguish
tractor-owned bindspace provisioning from kernel socket ownership.
## Human edits
Substantial human-directed editing occurred over several review turns:
- required use of `multiaddr`'s `.encapsulate()`/`.decapsulate()`
family rather than a hand-rolled tunnel peeler
- rejected the premature `start_listener()`/`close_listener()` hooks
and directed their removal from this foundational change
- corrected the documentation so tractor retains ownership of future
bindspace provisioning while the kernel owns the bearer socket
- reviewed and accepted the placement of declarative tunnel metadata
under `tractor.discovery`, with lifecycle code kept separate
The final source lines were applied through the coding agents, but
these design corrections and deletion decisions came from the human
review and materially shaped the patch.

View File

@ -0,0 +1,66 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-18T02:17:29Z
git_ref: d9a6e2e9
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - tunnelled-address handoff completion
Recovered Claude Code session
`7b9c97c4-fff7-4ac4-97fb-35720453308e` and continued its
interrupted `wkts/addr_unpacking` changes.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py`
Added frozen `WGTunnelSpec` and `TunnelledAddress` structs. The
wrapper delegates transport identity, validity, bindspace, and wire
serialization to its overlay while retaining tunnel metadata locally.
Added pure helpers to peel nested wrappers and enumerate their tunnel
specs. The module documents why wrappers must be peeled before
`Endpoint` selects the overlay transport backend.
> `git diff HEAD~1..HEAD -- tractor/discovery/_addr.py`
Extended `is_wrapped_addr()` to recognize `TunnelledAddress` without
registering tunnels as message transports, and widened the namespace
identifier type to cover named network namespaces.
> `git diff HEAD~1..HEAD -- tractor/discovery/__init__.py`
Exported the tunnel address API from `tractor.discovery` and updated
the eager-import documentation.
> `git diff HEAD~1..HEAD -- tests/discovery/test_tunnelled_addr.py`
Added focused coverage for delegation, serialization, rewrapping,
namespace fallback, nested peeling order, and frozen structs.
> `git diff HEAD~1..HEAD -- ai/tpt-backends/03_wg_tunnel_bindspace.md`
Clarified that tractor owns the eventual bindspace lifecycle and may
provision the WireGuard iface, routes, and kernel UDP listener through
netlink/`pyroute2`. Kernel socket ownership does not make the bearer an
application `MsgTransport` endpoint.
## Verification
Focused tests:
```text
9 passed in 0.03s
```
Discovery and IPC suites:
```text
67 passed, 2 xpassed
```
The audit removed premature module-level listener hooks. Runtime
integration must peel the wrapper explicitly at bind and dial
boundaries rather than make `._tunnel` impersonate a transport
backend.

View File

@ -0,0 +1,50 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking-followup
timestamp: 2026-08-18T07:50:31Z
git_ref: dd02c7c0
scope: code
substantive: true
raw_file: 20260818T075031Z_dd02c7c0_prompt_io.raw.md
---
## Prompt
The human requested the complete tunnelled-maddr parsing/composition
slice as an unattended batch, with every numbered requirement finished
and atomic commit plans prepared at the end. Existing human decisions
required native `multiaddr` encapsulation APIs, no hand-written peeler,
and preservation of tractor's future bindspace lifecycle ownership.
## Response summary
Implemented strict WG key codecs and native single/nested tunnel maddr
parsing and composition, integrated them into discovery APIs, migrated
the multihost example off its duplicate parser, corrected package
dependency metadata, and added focused and end-to-end parser
regressions. Verified the complete tractor suite and built both package
artifacts.
## Files changed
- `tractor/discovery/` - WG codecs, parser/composer, wrapper typing,
public exports, and discovery dispatch.
- `tests/discovery/` - key, grammar, nesting, round-trip, and public
boundary regressions.
- `examples/multihost/wg_lan/` - production parser migration and
updated usage documentation.
- `pyproject.toml`, `uv.lock` - reproducible WG codec and multibase
dependencies for checkout and package installs.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - current layer-A state
and future bindspace ownership.
## Human edits
The human selected the five-step scope and batch execution model,
required delegation to `multiaddr`'s encapsulation APIs, rejected
transport-shaped listener placeholders in the prerequisite commit, and
clarified that tractor will eventually provision the kernel-owned
bearer through its bindspace layer. The agent implemented and tested
those decisions; no direct manual source edits were observed during
this batch.

View File

@ -0,0 +1,61 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-18T07:50:31Z
git_ref: dd02c7c0
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - native WireGuard maddr integration
The human requested completion of the five-step tunnelled-maddr slice:
port the proven WireGuard parser, delegate to `py-multiaddr`'s native
tunnel APIs, integrate public parse and composition entry points, add
regressions, and return atomic commit plans after completing the batch.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/_multiaddr.py tractor/discovery/_addr.py tractor/discovery/__init__.py`
Added strict WireGuard standard-base64/multibase key codecs and native
WG maddr parsing/composition. Nested stacks peel the last `/wg/`
repeatedly with `.decapsulate_code()`, isolate segments through
`.split()`/`.join()`, and compose recursively with `.encapsulate()`.
Public discovery parsing, wrapping, endpoint-table parsing, and maddr
composition now preserve `TunnelledAddress` metadata locally.
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock`
Declared `py-multibase` directly and pinned the unreleased
py-multiaddr#108 WG codec revision in distribution metadata. Enabled
Hatch direct-reference metadata so editable, wheel, and sdist builds
use the same dependency contract.
> `git diff HEAD~1..HEAD -- tests/discovery/test_multiaddr.py tests/discovery/test_tunnelled_addr.py`
Added key-codec, single/nested WG round-trip, malformed key, invalid
bearer/overlay, missing codec, missing bearer, `wrap_address()`, and
`parse_endpoints()` coverage while retaining plain TCP/UDS regressions.
> `git diff HEAD~1..HEAD -- examples/multihost/wg_lan ai/tpt-backends/03_wg_tunnel_bindspace.md`
Removed the duplicate example parser, migrated callers to production
`TunnelledAddress`, retained only explicit `wg(8)` verification, and
updated dependency and future bindspace ownership documentation.
## Verification
- focused discovery files: `45 passed`
- discovery and IPC suites: `81 passed, 2 xpassed`
- full suite: `443 passed, 9 skipped, 7 xfailed, 4 xpassed`
- collection: `462 tests`
- Ruff: clean
- sdist and wheel builds: successful
## Review corrections
A review caught that `[tool.uv.sources]` alone would not constrain
built-package consumers to a WG-capable py-multiaddr revision. The
dependency was moved into PEP 621 metadata and the parser now converts
missing-codec failures into an actionable tractor error without
misclassifying Unix paths containing a `wg` directory.

View File

@ -0,0 +1,37 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking-followup
timestamp: 2026-08-19T21:31:44Z
git_ref: f81fc5e5
scope: code
substantive: true
raw_file: 20260819T213144Z_f81fc5e5_prompt_io.raw.md
---
## Prompt
The human requested runtime boundary integration with the established
per-step implementation and commit-plan workflow.
## Response summary
Kept `TunnelledAddress` available to callers while peeling it at the
last outbound boundary before transport lookup and dialing. Added a
regression which captures both transport arguments and confirms plain
TCP behavior is unchanged.
## Files changed
- `tractor/ipc/_chan.py` - peel tunnel annotations before outbound
transport dispatch and connection.
- `tests/ipc/test_channel_tunnel_boundary.py` - verify plain and
tunnelled channel inputs deliver only TCP overlays.
## Human edits
The human chose the runtime-boundary slice, required the existing
per-step commit-plan flow, and previously established that wrappers
must retain bindspace metadata without impersonating transports. The
agent implemented those constraints; no direct manual source edits were
observed during this step.

View File

@ -0,0 +1,25 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-19T21:31:44Z
git_ref: f81fc5e5
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - outbound tunnel boundary
The human requested the next tunnelled-address slice using the same
per-step commit-plan flow. Existing design decisions require retaining
tunnel metadata until the narrow IPC transport boundary and never
teaching exact-type transport tables about tunnel wrappers.
> `git diff HEAD~1..HEAD -- tractor/ipc/_chan.py tests/ipc/test_channel_tunnel_boundary.py`
Extended channel address inputs to accept tunnel declarations, then
called `strip_tunnels()` immediately before exact-type transport lookup
and `connect_to()`. Added plain/tunnel parameterized coverage proving
both operations receive the identical TCP overlay while the original
wrapper retains its tunnel spec.
Verification included focused IPC tests, Ruff, discovery/IPC suites,
and the full tractor suite.

View File

@ -0,0 +1,40 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking-followup
timestamp: 2026-08-19T21:31:45Z
git_ref: f81fc5e5
scope: code
substantive: true
raw_file: 20260819T213145Z_f81fc5e5_prompt_io.raw.md
---
## Prompt
The human requested completion of inbound runtime peeling using the
same per-step implementation and commit-plan workflow.
## Response summary
Preserved tunnel declarations through listener configuration, peeled
them immediately before `Endpoint` construction, and used the overlay
for backend-specific random listener allocation after registry
discovery. Added a real listener regression for the reflection and
exact-type boundary.
## Files changed
- `tractor/ipc/_server.py` - accept wrapper declarations and peel at
`Endpoint` construction.
- `tractor/_root.py` - allocate random transport addresses from the
contacted registry's overlay.
- `tests/ipc/test_server_tunnel_boundary.py` - verify a real listener
stores only TCP while preserving the source annotation.
## Human edits
The human selected the runtime-boundary work and previously corrected
the architecture so tractor retains future bindspace provisioning
ownership while `Endpoint` sees only application transports. The agent
implemented and tested that direction; no direct manual source edits
were observed during this step.

View File

@ -0,0 +1,25 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-19T21:31:45Z
git_ref: f81fc5e5
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - inbound tunnel boundary
The human requested runtime boundary integration while preserving the
future tractor-owned bindspace lifecycle.
> `git diff HEAD~1..HEAD -- tractor/ipc/_server.py tractor/_root.py tests/ipc/test_server_tunnel_boundary.py`
Broadened listener declarations to carry tunnel wrappers until
`_serve_ipc_eps()` and peeled immediately before `Endpoint`
construction. Also peeled a contacted tunnelled registry before
backend-specific random listener allocation. Added a real TCP listener
regression proving `Endpoint` stores only the resolved overlay while
the original declaration retains bindspace metadata.
Verification included `465` collected tests, `84` passing
discovery/IPC tests with two xpasses, Ruff, and the full suite with
`447` passes.

View File

@ -0,0 +1,41 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking-followup
timestamp: 2026-08-20T02:15:16Z
git_ref: dfad66a0
scope: docs
substantive: true
raw_file: 20260820T021516Z_dfad66a0_prompt_io.raw.md
---
## Prompt
The human requested that the bindspace plan preserve the agreed
capability, spawn-bootstrap, endpoint-role, namespace augmentation,
random-address, and teardown semantics, using `github/ns_aware` as
prototype input.
## Response summary
Updated plan-03 and the shared backend contract to separate serializable
bindspace declarations from scoped live capabilities, make namespace
entry a pre-runtime spawn operation, keep maddr paths role-neutral, and
define listen/dial provisioning plus ownership-sensitive teardown.
## Files changed
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - layer-C capability,
bootstrap, role, teardown, test, and risk model.
- `ai/tpt-backends/00_shared_backend_contract.md` - distinguish
transport bind selectors from process namespace lifecycle.
## Human edits
The human supplied the core architecture: structured scoped
capabilities, spawn-time namespace entry, orthogonal namespace
augmentation, source/destination-dependent provisioning, and
role-dependent teardown. They also rejected premature assumptions about
`open_bindspace()` returning an address and requested grounding in the
existing namespace prototype. The agent translated those decisions into
the plan text; no direct manual source edits were observed.

View File

@ -0,0 +1,34 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-20T02:15:16Z
git_ref: dfad66a0
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - bindspace capability design
The human corrected the layer-C design around local network-stack
realization. They established that bindspace state should be both
structured and a scoped capability; namespace entry belongs in
subactor bootstrap; maddrs can describe source or destination network
paths while namespace selection augments them orthogonally; random
address and teardown behavior depend on operation role and ownership.
They directed comparison with the prototype on `github/ns_aware` and
requested these decisions be preserved in the plan.
> `git diff HEAD~1..HEAD -- ai/tpt-backends/03_wg_tunnel_bindspace.md ai/tpt-backends/00_shared_backend_contract.md`
Reworked layer C around serializable `BindspaceSpec`, stable
`BindspaceIdentity`, and scoped non-serializable `BindspaceHandle`
concepts. Namespace FDs pin identity and lifetime; parent/supervisor
provisioning transfers entry capability through spawn; the child enters
before runtime, channels, listeners, sockets, or worker threads and then
drops authority. Listen/dial roles and owned/borrowed teardown are
explicit, while maddrs remain role-neutral network-path declarations.
The shared backend contract now separates transport-level `.bindspace`
selectors from process namespace lifecycle. Added tests/risks for FD
identity, bootstrap ordering, privilege drop, role ownership, and
shared-resource teardown.

View File

@ -0,0 +1,41 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-20T03:31:07Z
git_ref: ba07e09d
scope: code
substantive: true
raw_file: 20260820T033107Z_ba07e09d_prompt_io.raw.md
---
## Prompt
The human requested canonical tagged transport addresses with a
reader-first migration. TCP should decode `('tcp', host, port)`, Unix
should decode `('unix', path)`, `uds` should remain an accepted input
alias and internal transport key, and legacy tuple/list inputs must keep
working before writers switch formats.
## Response summary
Introduced canonical and compatibility address aliases, explicit tagged
dispatch, transport-specific tagged readers, and focused serialization
tests. Kept legacy pair inputs and native IPv6 socket values readable so
this boundary can ship before tagged emission.
## Files changed
- `tractor/discovery/_addr.py` - address aliases and tagged dispatch.
- `tractor/ipc/_tcp.py` - tagged, legacy, and IPv6 TCP decoding.
- `tractor/ipc/_uds.py` - canonical Unix and UDS-alias decoding.
- `tests/discovery/test_address_serialization.py` - reader compatibility
coverage.
## Human edits
The human supplied the canonical `tcp` and `unix` forms, chose `uds` as
an input-only serialization alias while preserving it as the runtime
transport key, and required a reader-first commit boundary. The agent
implemented those decisions; no direct manual source edits were
observed.

View File

@ -0,0 +1,25 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-20T03:31:07Z
git_ref: ba07e09d
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - tagged address readers
The human requested a migration away from ambiguous untagged transport
tuples. They established `('tcp', host, port)` and `('unix', path)` as
canonical forms, retained `('uds', path)` as an input alias, and required
a reader-first compatibility boundary before changing emitted values.
> `git diff HEAD~1..HEAD -- tractor/discovery/_addr.py tractor/ipc/_tcp.py tractor/ipc/_uds.py tests/discovery/test_address_serialization.py`
Added explicit tagged address aliases and dispatch, taught TCP and UDS
readers to decode tagged tuple/list payloads, preserved legacy pair input,
and retained native IPv6 socket-address decoding. Added focused tests for
canonical tags, the UDS alias, msgpack-style lists, legacy pairs, and IPv6
socket values.
Focused reader tests and Ruff checks passed before the writer migration
was applied.

View File

@ -0,0 +1,45 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-20T03:31:08Z
git_ref: ba07e09d
scope: code
substantive: true
raw_file: 20260820T033108Z_ba07e09d_prompt_io.raw.md
---
## Prompt
The human asked the agent to continue after adding tagged readers,
complete canonical address emission without dropping legacy input
compatibility, verify the migration, and prepare a complete multi-commit
package when the turn was done.
## Response summary
Changed `.unwrap()` to emit tagged TCP and Unix addresses, updated direct
tuple consumers and spawn payload declarations, and aligned multiaddr,
runtime, IPC, and discovery tests with canonical serialized equality.
Kept untagged tuples and the `uds` spelling readable at input boundaries.
## Files changed
- `tractor/discovery/_addr.py` - canonical output alias.
- `tractor/discovery/_multiaddr.py` - tagged address composition.
- `tractor/ipc/_tcp.py` - tagged emission and direct socket dialing.
- `tractor/ipc/_uds.py` - tagged full-path emission.
- `tractor/msg/types.py` - protocol-neutral spawn tuple containers.
- `tests/discovery/test_address_serialization.py` - writer assertions.
- `tests/discovery/test_multiaddr.py` - canonical round-trip assertions.
- `tests/discovery/test_tpt_bind_addrs.py` - tagged bind assertions.
- `tests/ipc/test_each_tpt.py` - canonical runtime address assertions.
- `tests/ipc/test_server_tunnel_boundary.py` - tagged TCP destructuring.
- `tests/test_local.py` - canonical registry comparison.
## Human edits
The human established the reader-before-writer sequencing, canonical tag
spellings, retained compatibility expectations, and requested final
multi-commit packaging. The agent implemented and tested those choices;
no direct manual source edits were observed.

View File

@ -0,0 +1,26 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-20T03:31:08Z
git_ref: ba07e09d
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - canonical tagged address writers
After the reader compatibility boundary, the human asked the agent to
continue the migration and package the completed work as dependency-
ordered commits.
> `git diff HEAD~1..HEAD -- tractor/discovery/_addr.py tractor/discovery/_multiaddr.py tractor/ipc/_tcp.py tractor/ipc/_uds.py tractor/msg/types.py tests/discovery/test_address_serialization.py tests/discovery/test_multiaddr.py tests/discovery/test_tpt_bind_addrs.py tests/ipc/test_each_tpt.py tests/ipc/test_server_tunnel_boundary.py tests/test_local.py`
Switched TCP and Unix `.unwrap()` output to canonical tagged tuples,
updated direct transport and multiaddr consumers, widened spawn message
tuple containers for protocol-specific shapes, and migrated runtime and
test comparisons to serialized address equality. Legacy inputs remain
accepted at `wrap_address()` and backend reader boundaries.
Ruff and focused tests passed. The complete non-debugger TCP suite passed
with 412 tests; the UDS suite reached 80% without failure before the
harness timeout, then all 97 remaining tests passed on resume. Debugger
PTY coverage was excluded after an unrelated timeout.

View File

@ -0,0 +1,49 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-21T23:32:04Z
git_ref: 5d92595f
scope: code
substantive: true
raw_file: 20260821T233204Z_5d92595f_prompt_io.raw.md
---
## Prompt
The human asked to proceed with WireGuard Layer B from PR #505's
head, using the established one-change-per-turn workflow and
finishing with a commit plan. This turn was limited to read-only
pyroute2 inspection; peer verification and bindspace provisioning
remain later changes.
During review, the human required the generated Python to follow the
deployed `/py-codestyle` rules and asked for the medium-term plan to
remove pyroute2's asyncio I/O runtime from the Trio read path.
## Response summary
Added Linux-only read helpers for WireGuard device and peer public
keys. Pyroute2's synchronous API is fully contained in a Trio worker
thread, supports named netns reads without creation side effects,
validates decoded keys and always closes the netlink client.
Follow-up edits added the required local annotations, boolean layout,
helper docstrings and 69-column source formatting.
## Files changed
- `pyproject.toml` - Linux-only `wg` optional dependency.
- `uv.lock` - resolved pyroute2 0.9.6 metadata.
- `tractor/discovery/_tunnel.py` - read-only WireGuard helpers.
- `tractor/discovery/__init__.py` - public helper exports.
- `tests/discovery/test_wg_inspection.py` - fake-backed netlink and
worker-thread regressions.
## Human edits
The human chose the exact #505 head as a stacked base, required
incremental changes ending in commit plans, and limited this turn to
read-only pyroute2 inspection. The human then identified that Ruff
success had not established `/py-codestyle` compliance and directed
the agent to correct the Python-specific annotation, documentation
and layout rules. No direct manual source edits were observed.

View File

@ -0,0 +1,31 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-21T23:32:04Z
git_ref: 5d92595f
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - read WireGuard state through pyroute2
The human authorized the first incremental WireGuard Layer B change
as a stacked branch from tractor PR #505, with one atomic change and
a commit plan at the end of the turn.
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock tractor/discovery/__init__.py tractor/discovery/_tunnel.py tests/discovery/test_wg_inspection.py`
Added a Linux-only `wg` extra using pyroute2 0.9.6, plus
asynchronous public helpers for reading one interface's public key
and peer keys. The complete synchronous open/read/parse/close
sequence runs in a Trio worker thread because pyroute2 owns a private
asyncio loop internally.
Named namespace reads pass `flags=0` to override pyroute2's `O_CREAT`
default, ensuring inspection cannot create a missing namespace. Fake
netlink messages cover multipart dumps, key validation, stable peer
deduplication, worker-thread execution, netns selection and cleanup
on success/error.
Ruff and lock checks passed. Focused tunnel/multiaddr coverage passed
47 tests; the complete discovery suite passed 88 tests with 2
xpasses.

View File

@ -0,0 +1,46 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-22T02:32:26Z
git_ref: 59a8ecfd
scope: code
substantive: true
raw_file: 20260822T023226Z_59a8ecfd_prompt_io.raw.md
---
## Prompt
After committing the read-only pyroute2 helpers and `wgman` design
update, the human authorized the next isolated Layer B change:
explicit `verify_wg_peer()` composition over WireGuard inspection.
## Response summary
Added and exported async `verify_wg_peer()` using one validated
WireGuard key snapshot. It recognizes local-interface and configured
peer identities without coupling kernel inspection to address
parsing. Updated the multihost examples to use the production helper
and removed their subprocess-based probe.
## Files changed
- `tractor/discovery/_tunnel.py` - shared async snapshot reader and
explicit verification helper.
- `tractor/discovery/__init__.py` - public verification export.
- `tests/discovery/test_wg_inspection.py` - local, peer, absent and
malformed-key verification coverage.
- `examples/multihost/wg_lan/host_a_srv.py` - async local-key check.
- `examples/multihost/wg_lan/host_b_client.py` - async peer-key check.
- `examples/multihost/wg_lan/wg_maddr.py` - removed obsolete
subprocess probe.
- `examples/multihost/wg_lan/README.md` - pyroute2 requirements and
verification workflow.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - async API contract.
## Human edits
The human selected this pre-agreed verification layer as the next
atomic change after reviewing and committing the preceding read and
architecture changes. The agent implemented the source changes; no
direct manual edits or follow-up corrections were observed.

View File

@ -0,0 +1,31 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-22T02:32:26Z
git_ref: 59a8ecfd
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - verify declared WireGuard identities
The human authorized the next incremental Layer B change after
committing the read-only pyroute2 helpers and first-child `wgman`
design update.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_inspection.py examples/multihost/wg_lan ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added async `verify_wg_peer()` over one WireGuard key snapshot. It
validates the declared `WGTunnelSpec.peer_pubkey` before I/O, forwards
the spec's iface/netns, and accepts either the local interface key for
a source/listen declaration or a configured peer key for a
destination/dial declaration.
Refactored worker offload behind one shared async reader so
verification cannot compare two different netlink snapshots. Exported
the helper, added local/peer/absent/malformed-key coverage, and moved
the multihost examples from their local `wg show` subprocess probe to
the production API.
Ruff and lock checks passed. Focused WireGuard/tunnel/multiaddr
coverage passed 51 tests; the complete discovery suite passed 92
tests with 2 xpasses.

View File

@ -0,0 +1,44 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-22T03:25:20Z
git_ref: d35c802b
scope: code
substantive: true
raw_file: 20260822T032520Z_d35c802b_prompt_io.raw.md
---
## Prompt
The human reported the explicit WireGuard verification commit done,
asked the agent to check it, and authorized the next incremental
Layer C change.
## Response summary
Verified commit `d35c802b`, made plain transport namespace behavior
explicit, and retained each original listener declaration beside its
peeled transport address. Endpoint and server diagnostics can now
report a tunnel's namespace without violating the transport boundary.
## Files changed
- `tractor/ipc/_tcp.py` - explicit plain-address namespace property.
- `tractor/ipc/_uds.py` - explicit plain-address namespace property.
- `tractor/discovery/_tunnel.py` - direct overlay namespace delegation.
- `tractor/ipc/_server.py` - retained declaration and namespace
diagnostics.
- `tests/discovery/test_tunnelled_addr.py` - plain and tunnel namespace
behavior.
- `tests/ipc/test_server_tunnel_boundary.py` - declaration retention
and diagnostic coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - concrete endpoint
boundary contract.
## Human edits
The human selected continued incremental implementation after
reviewing and committing the preceding verification layer. The agent
implemented this dependency-ordered namespace slice; no direct manual
edits or follow-up corrections were observed.

View File

@ -0,0 +1,29 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-22T03:25:20Z
git_ref: d35c802b
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - retain endpoint namespace declarations
The human reported the explicit WireGuard verification commit done,
asked for it to be checked, and authorized the next incremental
change.
> `git diff HEAD~1..HEAD -- tractor/ipc/_tcp.py tractor/ipc/_uds.py tractor/discovery/_tunnel.py tractor/ipc/_server.py tests/discovery/test_tunnelled_addr.py tests/ipc/test_server_tunnel_boundary.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Confirmed commit `d35c802b` and a clean worktree, then implemented the
smallest dependency-ordered Layer C slice. Plain TCP and UDS addresses
now explicitly report no namespace, allowing `TunnelledAddress` to
delegate without an attribute fallback.
Added required `Endpoint.declared_addr` metadata beside the peeled,
resolved `Endpoint.addr`. Endpoint and server diagnostics expose the
declaration's namespace without passing a tunnel wrapper into
transport reflection. Updated the Layer C plan and tests for plain,
tunnelled, endpoint and server namespace behavior.
Ruff passed. Focused namespace tests passed 13 tests; combined
discovery and IPC coverage passed 101 tests with 2 xpasses.

View File

@ -0,0 +1,64 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-22T04:20:26Z
git_ref: 29141f0b
scope: code
substantive: true
raw_file: 20260822T042026Z_29141f0b_prompt_io.raw.md
---
## Prompt
After committing endpoint namespace visibility, the human authorized
continued Layer C implementation.
During staged review, the human requested Literal-derived validation,
ownership documentation, stable-inode clarification, explicit
non-serialization rationale and consolidated invalid-model tests.
## Response summary
Added the foundational bindspace model: serializable declarations and
stable identities are separated from a process-local live capability.
The handle validates names, ownership and FD identity. A global
`ProcessLocal` sentinel blocks default encoding while retaining
msgspec struct behavior.
Review fixes require a positive inode for every realized netns,
derive runtime choices from the Literal aliases and clarify that an FD
integer is not transferable capability authority.
The human then clarified that msgspec structs are useful generic
storage independently of serialization policy, so the live handle now
uses a struct while remaining process-local by contract.
The human first selected an opaque FD wrapper, then recognized that
future process-local handles need the same guard and directed a global
marker under `tractor.msg` instead.
## Files changed
- `tractor/discovery/_bindspace.py` - declaration, identity and live
capability models.
- `tractor/discovery/__init__.py` - public bindspace exports.
- `tractor/msg/_local.py` - reusable process-local struct marker.
- `tractor/msg/__init__.py` - public `ProcessLocal` export.
- `tests/discovery/test_bindspace.py` - serialization and capability
invariant coverage.
- `tests/msg/test_process_local.py` - direct and nested wire rejection.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - concrete initial model
contract.
## Human edits
The human selected the foundational Layer C capability model, then
reviewed the staged patch and directed all eight follow-up changes:
remove distracting ownership variation, consolidate invalid-model
tests, derive containment checks from Literal aliases, require a
stable inode and explain why the live handle is not a msgspec struct.
The human subsequently corrected that last design choice: struct use
does not imply IPC authorization, so `BindspaceHandle` should retain
msgspec's generic storage benefits. The agent applied those
human-directed edits. The human first chose an opaque process-local FD
wrapper, then refined that decision to a reusable global marker under
`tractor.msg` for future local handles. No direct manual source edits
were observed.

View File

@ -0,0 +1,27 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-22T04:20:26Z
git_ref: 29141f0b
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - model bindspace capabilities
The human authorized the next incremental Layer C change after
committing endpoint namespace visibility.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added frozen, serializable `BindspaceSpec` and
`BindspaceIdentity` models for the initial Linux netns kind. Added a
plain slotted `BindspaceHandle` which carries ownership and an optional
namespace FD without being serializable by default msgspec encoding.
The live handle validates declaration/identity kind and name, ownership
values, non-negative FDs and FD/inode correspondence. Serializable
models reject empty identities and kinds without implementations.
Added round-trip, non-serialization, identity-mismatch, empty-identity
and unsupported-kind tests. Ruff passed; discovery plus
message-serialization coverage passed 125 tests with 2 xpasses.

View File

@ -0,0 +1,48 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-25T04:55:57Z
git_ref: fdccfd7e
scope: code
substantive: true
raw_file: 20260825T045557Z_fdccfd7e_prompt_io.raw.md
---
## Prompt
After committing the foundational bindspace capability model, the
human authorized the next incremental Layer C change.
## Response summary
Added a borrow-only existing-netns context manager which pins stable
identity through an open FD and always releases that local descriptor.
The API does not create, enter or remove namespaces and rejects named
path traversal.
Staged review renamed the context to `attach_netns()`, introduced the
readable `CURRENT_NETNS` alias and made key validation errors identify
their exact owning field.
Follow-up review made the empty-key error spell out that `None` is
written as `CURRENT_NETNS` for `BindspaceSpec.key` declarations.
## Files changed
- `tractor/discovery/_bindspace.py` - existing-netns lifecycle and key
validation.
- `tractor/discovery/__init__.py` - public lifecycle export.
- `tests/discovery/test_bindspace.py` - current, named, missing and
traversal coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - borrow-only lifecycle
contract.
## Human edits
The human selected the previously deferred borrow-only netns lifecycle
as the next incremental Layer C change. The agent implemented the
source changes. During staged review, the human selected
`attach_netns()` terminology, requested explicit
`BindspaceSpec.key = CURRENT_NETNS` semantics and field-specific key
validation. Follow-up review requested the validation error itself
connect `None` to `CURRENT_NETNS`. The agent applied those
human-directed edits; no direct manual source edits were observed.

View File

@ -0,0 +1,27 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-25T04:55:57Z
git_ref: fdccfd7e
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - borrow existing netns bindspaces
The human reported the bindspace capability model committed and
authorized the next incremental Layer C change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added async `open_existing_netns()` as a borrow-only context manager.
It opens the current process netns or a named entry under the standard
iproute2 run directory, derives stable identity from the opened FD,
and yields a borrowed process-local `BindspaceHandle`.
The context uses `O_CLOEXEC`, never creates, enters or removes a
namespace, and synchronously closes only its FD on exit. Netns keys
reject paths to keep named lookup beneath the run directory.
Added current, named, missing and traversal tests. Ruff and lock checks
passed; discovery plus message coverage passed 132 tests with 2
xpasses.

View File

@ -0,0 +1,37 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-25T19:05:29Z
git_ref: e1007547
scope: code
substantive: true
raw_file: 20260825T190529Z_e1007547_prompt_io.raw.md
---
## Prompt
After committing the attach-only network namespace lifecycle, the
human authorized the next incremental Layer C change.
## Response summary
Added owned `open_netns()` creation and teardown through shielded
pyroute2 worker calls. The context pins identity through
`attach_netns()`, yields an owned handle, closes its FD before removal
and cleans up despite caller cancellation.
## Files changed
- `tractor/discovery/_bindspace.py` - owned netns lifecycle.
- `tractor/discovery/__init__.py` - public lifecycle export.
- `tests/discovery/test_bindspace.py` - ownership, ordering,
cancellation and name requirements.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - owned lifecycle and
spawn-boundary contract.
## Human edits
The human selected owned network namespace creation as the next
incremental Layer C change. The agent implemented the source changes;
no direct manual edits or follow-up corrections were observed.

View File

@ -0,0 +1,26 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-25T19:05:29Z
git_ref: e1007547
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - own created netns bindspaces
The human reported the attach-only netns lifecycle committed and
authorized the next incremental Layer C change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added async `open_netns()` as the owned counterpart to
`attach_netns()`. It requires a named spec, creates through pyroute2 in
a shielded worker call, attaches the resulting namespace FD and yields
an owned process-local `BindspaceHandle`.
FD closure occurs before shielded namespace removal on normal,
exceptional and cancelled exits. The context never calls `setns()`;
namespace entry remains a spawn/bootstrap responsibility.
Privileged operations are faked in tests. Ruff and lock checks passed;
discovery plus message coverage passed 137 tests with 2 xpasses.

View File

@ -0,0 +1,40 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-25T19:18:45Z
git_ref: 5b2a064a
scope: code
substantive: true
raw_file: 20260825T191845Z_5b2a064a_prompt_io.raw.md
---
## Prompt
The human pointed out that the agent had produced no changes after the
previous commit and expected the next incremental implementation.
## Response summary
Corrected the sequencing mistake and added explicit serialized
bindspace lifecycle policy plus generic `open_bindspace()` dispatch.
Borrowed and owned netns contexts share a policy-neutral FD-pinning
primitive while validating their public lifecycle before side effects.
## Files changed
- `tractor/discovery/_bindspace.py` - lifecycle policy, ownership
invariants, shared pinning and dispatch.
- `tractor/discovery/__init__.py` - public policy and dispatcher exports.
- `tests/discovery/test_bindspace.py` - lifecycle serialization,
validation and both dispatcher branches.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - explicit lifecycle
policy independent of transport role.
## Human edits
The human identified that the agent had accidentally repeated a
summary of already committed work instead of implementing the next
slice. That correction directly caused this lifecycle/dispatcher
change to be implemented. The agent wrote the source changes; no
direct manual edits were observed.

View File

@ -0,0 +1,30 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-25T19:18:45Z
git_ref: 5b2a064a
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - dispatch explicit bindspace lifecycle
The human noticed the agent had re-reported already committed work
without changing the worktree. The agent confirmed the sequencing
error and proceeded to the actual next Layer C slice.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added serialized `BindspaceSpec.lifecycle` policy with explicit
`attach` and `open` choices. Added `open_bindspace()` dispatch without
inferring resource ownership from listen/dial role.
Refactored FD attachment into policy-neutral `_pin_netns()` so
borrowed and owned public lifecycles can share identity pinning while
enforcing their own policy before side effects. Handle construction
also verifies lifecycle and resulting ownership agree.
One composition error introduced during implementation was caught by
the focused test: owned creation initially called the newly guarded
public attach API. `_pin_netns()` fixed that layering error. Ruff and
lock checks passed; discovery plus message coverage passed 138 tests
with 2 xpasses.

View File

@ -0,0 +1,42 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-25T23:46:31Z
git_ref: b973e78c
scope: code
substantive: true
raw_file: 20260825T234631Z_b973e78c_prompt_io.raw.md
---
## Prompt
The human pointed out that the agent had again produced no worktree
changes after the latest commit and expected actual next-layer work.
## Response summary
Corrected the sequencing mistake and added a process-local WireGuard
interface configuration boundary. Secret/routing policy is separated
from serializable maddr-derived tunnel declarations, redacted from
repr and blocked from default wire encoding.
Staged review clarified that address/network parser calls validate
every declaration and do not select or consume a first address.
## Files changed
- `tractor/discovery/_tunnel.py` - local WG config and validation.
- `tractor/discovery/__init__.py` - public config export.
- `tests/discovery/test_wg_config.py` - secrecy and validation coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - secret/config boundary
for future interface provisioning.
## Human edits
The human identified the second sequencing failure where the agent
summarized an already committed slice instead of implementing the next
one. That correction directly caused this new WG configuration layer
to be implemented. During staged review, the human requested explicit
explanation of the address and allowed-route validation loops. The
agent applied that human-directed clarification; no direct manual edits
were observed.

View File

@ -0,0 +1,28 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-25T23:46:31Z
git_ref: b973e78c
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - separate local WireGuard configuration
The human again noticed the agent had re-reported already committed
work without changing the worktree. The agent confirmed the sequencing
mistake and moved to the next unimplemented dependency.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_config.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added process-local `WGInterfaceConfig` for private/preshared keys,
local interface CIDRs, peer allowed CIDRs, listen port and persistent
keepalive. Secrets are redacted from repr and the global
`ProcessLocal` marker blocks default wire encoding.
Removed the unused serialized `WGTunnelSpec.maybe_allowed_ips`
placeholder so maddr-derived declarations retain only public identity,
endpoint and interface selection. Added validation for keys, CIDRs,
ports and keepalive before future kernel mutation.
Ruff and lock checks passed. Focused WG coverage passed 25 tests;
discovery plus message coverage passed 145 tests with 2 xpasses.

View File

@ -0,0 +1,37 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-26T00:14:42Z
git_ref: dcdf4d82
scope: code
substantive: true
raw_file: 20260826T001442Z_dcdf4d82_prompt_io.raw.md
---
## Prompt
The human selected an explicit peer-list model rather than a
dial-only single-peer shortcut before `open_wg_iface()`.
## Response summary
Added process-local per-peer configuration and refactored interface
configuration to own a unique peer tuple. This supports multi-peer
listeners and dial targets without overloading role-dependent tunnel
maddr identity.
## Files changed
- `tractor/discovery/_tunnel.py` - peer model and interface peer list.
- `tractor/discovery/__init__.py` - public peer-config export.
- `tests/discovery/test_wg_config.py` - peer secrecy and validation.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - explicit multi-peer
provisioning contract.
## Human edits
The human chose explicit peer lists over a dial-only implementation so
listeners can represent multiple client public keys and routing
policies. The agent implemented that human-selected design; no direct
manual source edits were observed.

View File

@ -0,0 +1,27 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-26T00:14:42Z
git_ref: dcdf4d82
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - model explicit WireGuard peers
After committing local WG interface configuration, the human selected
an explicit peer-list model before interface provisioning.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_config.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added process-local `WGPeerConfig` with public key, allowed CIDRs,
optional endpoint, preshared key and keepalive. Refactored
`WGInterfaceConfig` to own a tuple of unique peers beside its private
key, local addresses and listen port.
Peer PSKs remain redacted and nested `ProcessLocal` sentinels prevent
default wire encoding. Validation covers keys, routes, endpoints,
ports, keepalive and duplicate peers.
Ruff and lock checks passed. Focused peer/config coverage passed 11
tests. A broad unrelated registrar cancellation case timed out after
86 passes and 2 xpasses; its isolated rerun passed.

View File

@ -0,0 +1,42 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-26T00:34:30Z
git_ref: 6dd39da0
scope: code
substantive: true
raw_file: 20260826T003430Z_6dd39da0_prompt_io.raw.md
---
## Prompt
After committing explicit WireGuard peer configuration, the human
authorized the next incremental lifecycle change.
## Response summary
Added owned WireGuard interface provisioning and teardown through
cancellation-shielded pyroute2 worker calls. Role-specific policy is
validated before mutation, local addresses and explicit peers are
configured, and owned interfaces are removed on exit.
Staged review clarified role-specific test assertions, added the
second peer's endpoint-port check, included conflicting values in
policy errors and aligned annotated assignment with `/py-codestyle`.
## Files changed
- `tractor/discovery/_tunnel.py` - role policy and WG iface lifecycle.
- `tractor/discovery/__init__.py` - public lifecycle and role exports.
- `tests/discovery/test_wg_iface_lifecycle.py` - policy and cancellation
coverage with privileged calls faked.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - initial owned WG iface
lifecycle contract.
## Human edits
The human selected continuation from explicit peer modeling into the
owned WireGuard interface lifecycle. During staged review, the human
requested clearer role assertions, complete endpoint checks, concrete
error values and inline annotated assignment. The agent applied those
human-directed edits; no direct manual source edits were observed.

View File

@ -0,0 +1,29 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-26T00:34:30Z
git_ref: 6dd39da0
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - own WireGuard interface lifecycle
After committing explicit peer configuration, the human authorized the
next incremental lifecycle change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_iface_lifecycle.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added owned async `open_wg_iface()` lifecycle with role-policy
validation before side effects. Listen bearer ports configure the local
listener; dial bearers fill omitted endpoints only for the selected
maddr peer.
Pyroute2 worker code creates the WireGuard link, assigns local
addresses, configures private key/listen port and every explicit peer,
raises the link, and removes it on exit. Partial create failures clean
up synchronously; async create/remove calls are cancellation-shielded.
Privileged operations are faked in tests. Ruff and lock checks passed;
focused WG lifecycle/config/inspection coverage passed 22 tests. A
broad unrelated registrar cancellation parameter timed out after 76
passes and 2 xpasses; its isolated rerun passed.

View File

@ -0,0 +1,47 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-26T02:24:34Z
git_ref: 2245f094
scope: code
substantive: true
raw_file: 20260826T022434Z_2245f094_prompt_io.raw.md
---
## Prompt
After committing the owned WireGuard interface lifecycle, the human
asked to continue with the next established incremental step.
## Response summary
Added a WG-specific composition context which owns one declared
bindspace and an ordered interface stack. It snapshots caller-owned
layer order before checkpointing, enters interfaces outermost-first,
and guarantees inside-out interface teardown before bindspace release.
Its inbound type shows the `(WGTunnelSpec, WGInterfaceConfig)` pairing
directly, and its docstring depicts the stack and unwind order.
## Files changed
- `tractor/discovery/_tunnel.py` - bindspace/interface stack driver.
- `tractor/discovery/__init__.py` - public layer and driver exports.
- `tests/discovery/test_wg_iface_lifecycle.py` - mutable-input,
cancellation and nested teardown-order coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - implemented composition
and ownership-order contract.
## Human edits
The human accepted and committed the preceding reviewed lifecycle
boundary, then explicitly requested the next step. That selected the
previously established bindspace/WireGuard composition boundary. In
staged review, the human requested an explicit interface-stack analogy
and a small ASCII diagram explaining entry and exit order. The human
then identified `WGInterfaceLayer` as an unnecessary single-use alias
which obscured the inbound tuple shape. The agent applied both
human-directed edits. The human accepted the fake lifecycle test as a
focused control-flow check which avoids pyroute2 overhead, while noting
their general preference for end-to-end coverage. No direct manual
source edits were observed.

View File

@ -0,0 +1,28 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-26T02:24:34Z
git_ref: 2245f094
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - compose WireGuard bindspace lifecycles
After committing the owned WireGuard interface lifecycle, the human
requested the next incremental change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_iface_lifecycle.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added `open_wg_bindspace()` to compose one declared bindspace with an
ordered sequence of WireGuard interface layers. The context enters the
bindspace first, enters interfaces outermost-first through
`AsyncExitStack`, yields the live `BindspaceHandle`, and removes
interfaces inside-out before releasing or removing the namespace.
The caller's layer sequence is snapshotted before the first checkpoint,
so concurrent mutation cannot alter resource entry or unwind order.
Fake lifecycle tests clear a mutable input list during bindspace entry,
cancel from the yielded scope, and prove the original stack unwinds in
dependency-safe order.
Ruff passed and focused bindspace/WireGuard coverage passed 39 tests.

View File

@ -0,0 +1,54 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-26T03:05:34Z
git_ref: d130431c
scope: code
substantive: true
raw_file: 20260826T030534Z_d130431c_prompt_io.raw.md
---
## Prompt
After committing the reviewed bindspace/interface stack, the human
asked to continue with the next established Layer C step.
## Response summary
Retained a serializable realized `BindspaceRef` on frozen tunnelled
address declarations without carrying the process-local `Bindspace`
capability. The namespace API remains tuple-shaped, using the declared
key before realization and stable inode afterward. Existing transport
peeling and unrealized serialization remain unchanged.
## Files changed
- `tractor/discovery/_tunnel.py` - realized bindspace-ref retention and
immutable annotation method.
- `tests/discovery/test_tunnelled_addr.py` - ref, compatibility and
mismatch coverage.
- `tests/ipc/test_server_tunnel_boundary.py` - real listener diagnostic
coverage for the realized inode.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - realized namespace
visibility contract.
## Human edits
The human accepted and committed the preceding reviewed composition
boundary, then requested the next step. Their stated preference for
end-to-end tests informed use of the existing real TCP listener path to
verify endpoint/server diagnostics, while focused unit coverage handles
metadata invariants without pyroute2 overhead. During review, the human
chose the final unshipped terminology: `BindspaceRef` for the
serializable non-owning reference, `Bindspace` for the live
process-local capability, `.ref` for that capability's reference, and
`bindspace_ref` at the tunnel declaration API.
They then requested the prerequisite model rename as a separate commit
before this feature. The human also replaced the module-level helper
with `TunnelledAddress.with_bindspace_ref()` and requested inline
msgspec encode/decode expressions in the serialization assertions. The
human chose not to add a second typed `.namespace` projection, and
requested native tagged `TunnelledAddress` decoding remain as a separate
design-plan follow-up. The agent applied those human-directed changes;
no direct manual source edits were observed.

View File

@ -0,0 +1,31 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-26T03:05:34Z
git_ref: d130431c
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - retain realized bindspace identity
After committing WireGuard bindspace composition, the human requested
the next incremental Layer C change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_tunnelled_addr.py tests/ipc/test_server_tunnel_boundary.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added optional `BindspaceIdentity` metadata to frozen
`TunnelledAddress` declarations and a pure
`with_bindspace_identity()` annotation helper. Unrealized declarations
retain their prior serialized shape. Realized declarations retain only
serializable key/inode identity, never the FD-bearing capability.
The existing `.namespace` tuple contract remains compatible:
unrealized declarations report `(kind, key)`, while realized
declarations report the stable `(kind, inode)`. Name mismatches between
the tunnel declaration and realized bindspace are rejected.
Unit coverage verifies immutability, serialization, delegation and
mismatch handling. The existing real TCP listener test proves endpoint
and server diagnostics expose the retained inode without implying that
the process entered that namespace. Ruff passed and focused tunnel,
listener and bindspace coverage passed 30 tests.

View File

@ -0,0 +1,43 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-27T21:11:15Z
git_ref: d130431c
scope: code
substantive: true
raw_file: 20260827T211115Z_d130431c_prompt_io.raw.md
---
## Prompt
During review, the human requested concrete names for the existing
bindspace abstractions and asked to isolate that rename before realized
reference retention.
## Response summary
Renamed the serializable resource record to `BindspaceRef`, the live
FD-backed capability to `Bindspace`, and the capability's record field
to `.ref`. Updated existing lifecycle APIs, tests and active design
contracts without compatibility aliases.
## Files changed
- `tractor/discovery/_bindspace.py` - renamed models and `.ref` field.
- `tractor/discovery/_tunnel.py` - existing lifecycle annotations.
- `tractor/discovery/__init__.py` - renamed public model exports.
- `tests/discovery/test_bindspace.py` - renamed model contracts.
- `tests/discovery/test_wg_iface_lifecycle.py` - live bindspace usage.
- `ai/tpt-backends/00_shared_backend_contract.md` - shared terminology.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - request, ref and live
capability terminology.
## Human edits
The human identified the earlier identity/handle terminology as too
abstract for an IPC-passed non-owning reference and live resource. They
selected `BindspaceRef`, `Bindspace`, and `.ref`, then requested this
rename as a prerequisite commit separate from the realized-ref feature.
The agent applied those human-directed names; no direct manual source
edits were observed.

View File

@ -0,0 +1,26 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-27T21:11:15Z
git_ref: d130431c
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - simplify bindspace model names
While reviewing realized bindspace metadata, the human requested more
concrete names for the existing bindspace model family and asked that
the rename remain separate from the subsequent feature.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py tests/discovery/test_wg_iface_lifecycle.py ai/tpt-backends/00_shared_backend_contract.md ai/tpt-backends/03_wg_tunnel_bindspace.md`
Renamed `BindspaceIdentity` to `BindspaceRef` for the serializable,
non-owning, host-local resource record. Renamed `BindspaceHandle` to
`Bindspace` for the live process-local FD-backed capability, and renamed
its `.identity` field to `.ref`.
No compatibility aliases were added because the models are unshipped.
Active design contracts now distinguish the serialized request, the
wire-safe non-owning ref, and the live capability. Ruff passed, 518
tests collected, and 53 focused bindspace/WireGuard/listener tests
passed across the final two-commit tree.

View File

@ -0,0 +1,41 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-27T23:25:00Z
git_ref: d067505a
scope: code
substantive: true
raw_file: 20260827T232500Z_d067505a_prompt_io.raw.md
---
## Prompt
After committing the reviewed bindspace-reference changes, the human
asked to continue with the next Layer C increment.
## Response summary
Added the private Linux network-namespace entry primitive required by
future child bootstrap wiring. It verifies the inherited capability
before `setns()`, constrains entry to network namespaces, and verifies
the resulting process namespace before runtime initialization proceeds.
## Files changed
- `tractor/spawn/_netns.py` - synchronous verified netns entry.
- `tests/test_netns_spawn.py` - stale-FD, syscall and post-entry guards.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - primitive ownership and
remaining integration boundary.
## Human edits
The human requested the next incremental implementation after accepting
the prior split commits. Their established preference for end-to-end
tests informed use of real FDs where possible; only privileged `setns()`
and procfs observation are replaced to keep the suite unprivileged. In
staged review, the human requested the full project license header,
explicit comments on both intentional inode mismatches, and independent
call traces proving both fake syscall and procfs hooks execute. The
agent applied those human-directed revisions; no direct manual source
edits were observed.

View File

@ -0,0 +1,25 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-27T23:25:00Z
git_ref: d067505a
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - verify pre-runtime netns entry
After committing realized bindspace references and their deferred codec
plan, the human requested the next Layer C increment.
> `git diff HEAD~1..HEAD -- tractor/spawn/_netns.py tests/test_netns_spawn.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added a private synchronous `enter_netns()` bootstrap primitive. It
validates the inherited FD and expected inode, constrains `setns()` to
`CLONE_NEWNET`, verifies `/proc/self/ns/net` afterward, and leaves FD
closure to the future spawn-bootstrap caller.
Tests use real stand-in FDs while replacing only the privileged syscall
and post-entry procfs observation. They prove stale FDs fail before
entry, the syscall receives the exact namespace type, and bootstrap
rejects an unexpected post-entry namespace. Ruff and all three focused
tests passed.

View File

@ -0,0 +1,45 @@
---
model: gpt-5.6-sol
service: opencode
session: addr-unpacking-rebase-conflict
timestamp: 2026-08-28T00:38:05Z
git_ref: e40c39da
scope: code
substantive: true
raw_file: 20260828T003805Z_e40c39da_prompt_io.raw.md
---
## Prompt
The human asked the agent to resolve the first conflict in the
`wkt/addr_unpacking` rebase because the replayed history was primarily
agent-generated.
## Response summary
Preserved the lazy-import and platform-aware behavior from the new
base while restoring the replayed WireGuard address parsing and
composition behavior. Reconciled the changed dependency lock with the
replayed direct dependency contract and validated the combined result.
## Files changed
- `tractor/discovery/_addr.py` - combined address-platform and tunnel
typing changes.
- `tractor/discovery/_multiaddr.py` - combined lazy imports with WG
parse and composition dispatch.
- `tractor/discovery/_tunnel.py` - retained cold-import behavior for
the newly added WG implementation.
- `tests/discovery/test_multiaddr.py` - adapted missing-codec
monkeypatching to the lazily imported upstream protocol module.
- `uv.lock` - restored the WG-capable multiaddr source and direct
multibase metadata.
## Human edits
The human chose to abort the first rebase attempt, restart with an
explicit old-base boundary, and delegated the conflict resolution to
the agent. The human also chose not to preserve the superseded
module-level protocol-lookup test seam; the agent redirected the test
to the upstream protocol module. No direct manual source edits were
observed during this resolution.

View File

@ -0,0 +1,32 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T00:38:05Z
git_ref: e40c39da
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - addr-unpacking rebase resolution
The human delegated resolution of the first conflict while rebasing
`wkt/addr_unpacking` from `d9a6e2e9` onto `85a44588`.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/discovery/_addr.py tractor/discovery/_multiaddr.py tractor/discovery/_tunnel.py`
Combined the rebased base's platform-aware address registries and lazy
optional-dependency imports with the replayed `TunnelledAddress` typing
and WireGuard multiaddr dispatch. Adapted the new tunnel module so
importing tractor does not eagerly load `multiaddr`.
> `git diff HEAD~1..HEAD -- uv.lock`
Regenerated the lockfile to retain the replayed WG-capable
`py-multiaddr` revision and direct `py-multibase` dependency after the
new base's independently changed lockfile silently omitted that hunk.
## Verification
Conflict-marker, syntax, lock consistency, lint, collection, focused
discovery, and cold-import checks were selected for the resolution.

View File

@ -0,0 +1,38 @@
---
model: gpt-5.6-sol
service: opencode
session: wg-netns-bootstrap-entry
timestamp: 2026-08-28T01:22:05Z
git_ref: 9ae7cd86
scope: code
substantive: true
raw_file: 20260828T012205Z_9ae7cd86_prompt_io.raw.md
---
## Prompt
After completing and auditing the stacked rebase, the human approved
the next WireGuard Layer C slice: integrate the verified network
namespace entry primitive into child spawn bootstrap.
## Response summary
Wire an optional child-owned namespace FD and expected inode into both
child entrypoints. Keep namespace entry and FD closure synchronous and
ordered before actor runtime initialization, while deliberately
deferring backend FD transport and public API plumbing.
## Files changed
- `tractor/spawn/_entry.py` - enter and release an optional inherited
namespace capability before multiprocessing child runtime startup.
- `tractor/_child.py` - consume the capability at the start of the
Trio child bootloader, before patching or actor construction.
- `tests/test_netns_spawn.py` - prove entry ordering, failure isolation,
and exact FD ownership for both child entrypoints.
## Human edits
The human selected spawn-time namespace integration as the next work
slice and authorized implementation. No direct manual source edits
were observed during generation.

View File

@ -0,0 +1,36 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T01:22:05Z
git_ref: 9ae7cd86
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - enter inherited netns during child bootstrap
The human asked to continue the WireGuard Layer C implementation after
the rebased branch landed the verified `spawn._netns.enter_netns()`
primitive.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/_child.py tractor/spawn/_entry.py tests/test_netns_spawn.py`
Add one optional atomic `(namespace_fd, expected_inode)` bootstrap
capability to the multiprocessing entrypoint and Trio child bootloader.
Enter and verify the namespace before Trio patching, actor construction,
runtime state, logging, Trio startup, parent connection, sockets, or
worker threads, then close the child-owned inherited FD before
continuing.
Add privilege-free direct-entrypoint tests using real stand-in FDs and
a fake namespace syscall boundary. Cover helper-level no-op behavior,
successful entry ordering, exact FD closure, malformed-FD isolation,
primary-error preservation, and failure cleanup for both child
bootstrap paths.
## Scope boundary
This increment does not transfer FDs through spawn backends or expose a
public actor API. Backend-specific FD duplication, bootstrap failure
reporting, and root-process namespace entry remain follow-up work.

View File

@ -0,0 +1,47 @@
---
model: gpt-5.6-sol
service: opencode
session: trio-pre-handshake-child-death
timestamp: 2026-08-28T04:51:19Z
git_ref: fb6d81d3
scope: code
substantive: true
raw_file: 20260828T045119Z_fb6d81d3_prompt_io.raw.md
---
## Prompt
The human approved the next WireGuard Layer C implementation step after
committing child-side network namespace bootstrap.
## Response summary
Restore and harden the boot-race design originally implemented by
`3b0724eba85b4014170ed95773e1e41a60d5c513` before transferring
namespace FDs. Race the initial peer handshake against child-process
death so a pre-connection bootstrap failure surfaces instead of
parking `ActorNursery.start_actor()` forever.
## Files changed
- `tractor/spawn/_spawn.py` - provide the generic handshake/process
death race.
- `tractor/spawn/_trio.py` - use the race during Trio child startup.
- `tests/test_spawning.py` - cover handshake-first and death-first
schedules, simultaneous outcomes, waiter errors, and backend cleanup.
## Human edits
The human approved continued Layer C implementation. The agent narrowed
the next slice to failure detection after analysis showed FD transfer
would otherwise introduce a parent hang. The human then identified the
earlier `wait_for_peer_or_proc_death()` implementation and required
explicit credit; the source docstring and commit message now cite its
original commit rather than a later cherry-pick. The human directly
reflowed `_trio.py` to the project line limit and normalized nearby
quotes. The human also required naming the race helper in its lifecycle
comment, explicit `trio.Cancelled` handling in waiter tests, expected
cancellation diagnostics, and a `trio.Process`-constrained test double.
After mainline portability verification, the human chose to commit the
fix on the current WireGuard branch and require its cherry-pick onto
`main` as a blocker in the eventual WireGuard pull request.

View File

@ -0,0 +1,34 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T04:51:19Z
git_ref: fb6d81d3
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - detect child death before parent handshake
The human asked to continue WireGuard Layer C after child entrypoints
learned to consume inherited network namespace capabilities.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/spawn/_spawn.py tractor/spawn/_trio.py tests/test_spawning.py`
Race a spawned child's initial parent handshake against its process
wait operation. Return the established channel when the handshake wins;
raise `ActorFailure` with the exit status when the process dies first.
Use the race in the Trio exec backend instead of waiting indefinitely
on `IPCServer.wait_for_peer()`.
Add deterministic Trio tests for handshake-first, death-first,
simultaneous, and waiter-error schedules, controlling readiness with
events rather than sleeps. Exercise the full Trio backend with a fake
dead process to prove pre-publication event registration and exact
failed-startup cleanup.
## Scope boundary
This prerequisite prevents pre-connection namespace-entry failures
from hanging their parent. Namespace FD transfer itself remains the
next commit.

View File

@ -0,0 +1,53 @@
---
model: gpt-5.6-sol
service: opencode
session: trio-bindspace-fd-transport
timestamp: 2026-08-28T17:29:43Z
git_ref: 2ca8c570
scope: code
substantive: true
raw_file: 20260828T172943Z_2ca8c570_prompt_io.raw.md
---
## Prompt
After committing generic pre-handshake child-death detection, the human
approved the proposed next Layer C slice: transport a live bindspace
capability through the Trio spawn backend.
## Response summary
Plumb an optional process-local `Bindspace` through actor spawn APIs.
Give a Trio exec child its own inherited namespace descriptor, preserve
existing caller `pass_fds`, transport the FD/inode pair through the CLI,
and close the temporary parent descriptor on every open-process outcome.
Reject unsupported multiprocessing transport explicitly. Exercise the
complete Trio path with a real actor that enters a distinct network
namespace and reports its namespace and inherited-FD inodes over RPC.
## Files changed
- `tractor/runtime/_supervise.py` - accept and relay a child bindspace.
- `tractor/to_actor/_api.py` - expose bindspace spawn configuration.
- `tractor/spawn/_spawn.py` - relay the bindspace to spawn backends.
- `tractor/spawn/_trio.py` - duplicate, transfer, and close the FD.
- `tractor/spawn/_mp.py` - reject unsupported MP transport.
- `tractor/_child.py` - parse and forward the atomic CLI capability.
- `tests/test_netns_spawn.py` - cover transport and cleanup schedules.
## Human edits
The human approved the Trio-first transport boundary after reviewing
the proposed dependency order, then supplied staged-diff comments across
multiple review passes. Human-directed revisions replaced abstract
ownership wording with concrete FD and process terminology, clarified
descriptor cleanup, preserved existing `pass_fds` explicitly, and
replaced the mocked success-path test with a real child actor, real
`setns()`, handshake, and RPC round trip inside unprivileged user/network
namespaces. Follow-up review added direct parent FD-table verification
that the temporary child-bound duplicate is closed after successful
spawn.
Multiprocessing FD reduction and root-process namespace entry remain
separate follow-up slices. No direct manual source edits were observed;
the review-driven source revisions were applied by the agent.

View File

@ -0,0 +1,34 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T17:29:43Z
git_ref: 2ca8c570
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - transport bindspace FD through Trio spawn
The human approved the next WireGuard Layer C increment after landing
pre-handshake child-death detection.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/_child.py tractor/runtime/_supervise.py tractor/spawn/_spawn.py tractor/spawn/_trio.py tractor/spawn/_mp.py tractor/to_actor/_api.py tests/test_netns_spawn.py`
Thread an optional live `Bindspace` through actor spawn APIs. For the
Trio backend, duplicate its namespace FD, append that temporary
descriptor to existing `pass_fds`, and send the atomic descriptor/inode
pair through the child CLI. Close the parent temporary duplicate as
soon as `open_process()` returns, raises, or is cancelled, while the
child consumes its inherited descriptor before runtime startup.
Reject bindspace transport explicitly in multiprocessing backends until
their descriptor-reduction slice is implemented. Add privilege-free
tests for CLI forwarding, merged pass-FD state, caller immutability,
source/duplicate ownership, exec failure, cancellation, absent live
authority, death-before-handshake cleanup, and MP rejection.
## Scope boundary
Multiprocessing FD reduction and root-process namespace entry remain
follow-up work.

View File

@ -0,0 +1,62 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: pr512-docs-review
timestamp: 2026-08-28T20:08:22Z
git_ref: wkt/big_boi_docs_472_follow_ups
scope: code
substantive: true
raw_file: 20260828T200822Z_0be872ff_prompt_io.raw.md
---
## Prompt
Prime PR #512 for landing by applying the repository Python style and
typing policy across changed examples, reviewing and correcting the
result with delegated agents, validating it, and auditing documentation
changes since PR #460 for broader follow-up work.
## Response summary
Completed a branch-wide style, typing, and docstring pass over the
changed examples. Reworked the dedicated registrar example so it proves
external discovery, handles bind collisions, publishes readiness
atomically, and shuts down cleanly across platforms. Staged review moved
it under the discovery examples, introduced an async lifecycle manager,
and captured public-subsystem, Piker, and pytest follow-ups. Corrected
discovery and typed-messaging documentation after cross-checking runtime
behavior.
Created issue #514 for typed `Start` contracts and linked its planned
sender-side argument validation from the messaging guide.
Independent review passes were repeated until no actionable findings
remained. Static checks, Sphinx, runnable examples, Trio coverage, and
focused registrar checks passed; Ctrl-C debugger failures remain
unclassified after one case reproduced on untouched `main`.
## Files changed
- `examples/discovery/dedicated_registrar.py` - implement robust
standalone registrar discovery and teardown.
- `examples/**/*.py` - apply style, typing, docstring, and terminology
corrections to the 38 changed example modules.
- `docs/guide/discovery.rst` - document actual registrar and
multi-registrar behavior.
- `docs/guide/msging.rst` - document actual validation and codec-hook
boundaries.
## Human edits
The human selected PR #512, required the `/py-codestyle` policy and
typing improvements throughout the examples, required delegated review
agents to find additional fixes, authorized agents to own and push their
code through the proper review process, and requested a broad audit of
documentation changes since PR #460 rendered as an evolving Taken WIP.
The human then required the earlier `ParamSpec` proposal to be captured
as a dedicated GitHub issue and linked from the current docs patch.
The human also required the registrar demo to move under discovery, use
structural matching and an async context manager, explain socket ownership,
and preserve public API, Piker, and pytest-harness follow-ups.
The human also explicitly retracted messages intended for another
session; those messages did not direct the result. No direct manual
source edits were identified; the agent applied the human-directed
changes.

View File

@ -0,0 +1,90 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T20:08:22Z
git_ref: wkt/big_boi_docs_472_follow_ups
diff_cmd: git diff 0be872ff..HEAD
---
## Prompt
Prime PR #512 for landing. Apply the Python style policy throughout
the changed examples, improve typing, use review agents to find and
fix correctness issues, run the proper review process, push the agent's
own code, and then audit documentation changes since PR #460 for stale,
missing, defunct, or improvable coverage.
## Response
> `git diff 0be872ff..HEAD -- examples/`
Restyled and typed all 38 changed example modules while preserving
intentional debugger failures and hanging reproducers. Added required
multiline docstrings, corrected async callable annotations, modernized
legacy API terminology, and preserved complete debugger process trees.
Rebuilt `examples/discovery/dedicated_registrar.py` as a real
external-discovery example. A standalone registrar process now selects
and retries bounded ephemeral addresses, publishes readiness atomically,
rejects accidental registrar reuse, serves sibling service and client
actors, proves lookup used the registrar instead of a local-peer channel,
and performs bounded, validated, cross-platform shutdown.
Staged review moved the demo into the discovery example group, converted
collision classification to structural pattern matching, documented the
selector-socket close/rebind race, and extracted process ownership into an
async context manager. A source TODO records the future public discovery
subsystem, Piker service-management lessons, and pytest registry-isolation
use case.
> `git diff 0be872ff..HEAD -- docs/guide/discovery.rst`
Corrected registrar, duplicate-name, and multi-registrar discovery
guidance, including configured-order and `None` placeholder behavior.
> `git diff 0be872ff..HEAD -- docs/guide/msging.rst`
Corrected typed-payload validation boundaries and separated working
task-scoped codec encoding from private per-dialog decoding and the
incomplete decorator hook parameters.
## Review and validation
Multiple independent reviews found and drove fixes for registrar
discovery validity, port-selection races, teardown, process diagnostics,
shutdown status, inaccurate discovery ordering, async callable typing,
missing docstrings, truncated debugger command diagrams, stale APIs, and
payload-error relay wording.
Validation completed:
- AST parsing, Ruff, 69-column checks, and required-docstring audit for
all 38 changed Python files.
- Sphinx HTML build succeeded.
- Documentation example harness: 24 passed.
- Trio coverage: 7 passed, 4 xfailed, 1 xpassed.
- Dedicated registrar direct run and focused harness test passed with a
clean child exit and no traceback.
- Debugger suite: 20 passed, 6 skipped, and 8 reproducible
`ctl-c=True` pexpect timeouts. One exact failure reproduced on
untouched `main`, confirming a baseline failure; its root cause
remains unclassified. No leaked actor processes remained.
The post-PR-#460 audit identified follow-up work around advertised but
inert runtime selectors, platform/backend support, unresolved discovery
contracts, cached-context teardown, codec recipes, examples-as-tests
coverage, public API exports, broadcast factory contracts, stale examples,
README duplication, release notes, and process-title terminology.
## Follow-up prompt
Capture the previously proposed `ParamSpec`-based `Start` argument
validation work in a focused GitHub issue, then link it from the current
typed-messaging docs patch as planned sender-side checking.
## Follow-up response
Created https://github.com/goodboy/tractor/issues/514 to track deriving
typed `Start` contracts from endpoint signatures, preserving caller-facing
signatures, and validating arguments before sending where possible. Added
the issue link beside the guide's current `Start` validation boundary.

View File

@ -0,0 +1,409 @@
# `tractor.ipc` next-gen transport backends: the shared contract
Status: design doc / implementation spec.
Audience: any model or human implementing one of the three
sibling plans in this directory.
- [`01_tipc_backend.md`](./01_tipc_backend.md) — `AF_TIPC`
(gh #378)
- [`02_quic_iroh_backend.md`](./02_quic_iroh_backend.md) — QUIC
via `iroh` FFI, uniffi-async rewritten onto `trio` (gh #353)
- [`03_wg_tunnel_bindspace.md`](./03_wg_tunnel_bindspace.md) —
WireGuard (and other shuttle-able) tunnels as a *nested
bindspace* layer via `pyroute2` (gh #482, #443)
This doc is the **normative** description of what a `tractor`
transport backend *is* as of `main@83b34884`. Each sibling plan
assumes it and only documents its own deltas. Read this first;
do not re-derive it from the code.
---
## 0. Why a shared contract doc
The three plans are meant to be implementable *independently and
concurrently* by different models/providers without design
drift. Everything they share — the backend duck-type, the
registration tables, the test harness plumbing, the naming and
code-style rules — lives here exactly once. If an implementer
finds this doc disagrees with `main`, **the code wins**; fix this
doc in the same PR.
---
## 1. The backend duck-type (empirical, from `_tcp.py`/`_uds.py`)
A transport backend is **one module** under `tractor/ipc/`
exposing exactly four things. There is no ABC to subclass and no
plugin entrypoint; wiring is by explicit table registration
(§2) plus one piece of reflection (§1.3).
### 1.1 `class <Proto>Address(msgspec.Struct, frozen=True)`
Structurally conforms to the `Address` `Protocol` in
`tractor/discovery/_addr.py:82`. Required surface:
| member | kind | notes |
| --- | --- | --- |
| `proto_key` | `ClassVar[str]` | the wire/registry key, e.g. `'tcp'`, `'uds'` |
| `unwrapped_type` | `ClassVar[type]` | the primitive tuple shape |
| `def_bindspace` | `ClassVar` | default bindspace value |
| `is_valid` | `@property -> bool` | "is this a *dialable/bindable* addr" |
| `bindspace` | `@property` | the "set of hosts"-ish scope (see below) |
| `from_addr(cls, addr)` | `@classmethod` | primitive -> wrapped, `match`-based |
| `unwrap(self)` | method | wrapped -> primitive (must be msgpack-native!) |
| `get_random(cls, bindspace=...)` | `@classmethod` | per-subactor ephemeral addr |
| `get_root(cls)` | `@classmethod` | host-singleton default registrar addr |
| `__repr__` | method | `f'{type(self).__name__}[{...}]'` house style |
Hard constraints learned from the existing two:
- **`frozen=True`.** Addresses are dict keys
(`Server.epsdict()`, `Endpoint.peer_tpts`) and are compared by
value all over the runtime.
- **`.unwrap()` output must round-trip through `msgspec` and
through `wrap_address()`.** It is what actually crosses the
wire in `SpawnSpec`/`_root_mailbox`/`_registry_addrs`, and it
is what `Actor.reg_addrs` and every test compares against. If
your unwrapped form is not *uniquely* pattern-matchable
against the other backends' forms in
`wrap_address()` (`_addr.py:230`), you have a bug that
manifests as the wrong transport being loaded — the file's own
`XXX NOTE` warns about precisely this.
⚠️ **and shape-matching does not survive 4 backends.** Adding
TIPC and iroh breaks it outright: TIPC's natural form is a
`(str, int)` — indistinguishable from `TCPAddress` — and
iroh's is a `(str, str)`, which the *existing* UDS case
(`case (_, filename) if type(filename) is str`) already
swallows. Ordering hacks and prefix-tagging (an earlier
revision of plan 01 proposed `('tipc:<stype>:<scope>', inst)`)
paper over it at best.
**The fix, and the recommended prerequisite for all three
backends: make the unwrapped form carry an explicit
proto-key, using the `multiaddr` protocol name as the
canonical spelling** — `('tcp', host, port)`,
`('unix', path)`, `('udp', ...)`, `('tipc', stype, inst,
scope)`. Then `wrap_address()` collapses from an
order-sensitive `match` to `_address_types[addr[0]]`, and the
whole collision class stops existing. Note this *also* aligns
the on-wire form with `mk_maddr()`/`parse_maddr()`, so the two
representations stop being independent inventions.
Two consequences to plan for:
- it's a **wire-format change** (`SpawnSpec`,
`_root_mailbox`, `_registry_addrs`) plus every test fixture
and downstream config (`piker`'s `[network]` table). It
wants its **own migration commit, landed before any new
backend**, not smuggled into one.
- it's the moment to **stop handing raw unwrapped tuples to
users at all.** The long-term shape is: `Address` subtypes
are the public currency and `UnwrappedAddress` becomes an
internal serialization detail — the same discipline
`ipaddress` uses (you pass `IPv4Address`, not a 4-tuple).
Public API should accept `Address|maddr-str` and treat bare
tuples as legacy-tolerated input, ideally deprecated.
- **`.get_random()` must be collision-free without a live
runtime.** See the `UDSAddress.get_random()` uuid-token
comment (`_uds.py:207-220`): with no `current_actor()` the
sockname degenerates to a pure fn of `(prefix, pid)` and two
calls in one proc alias. Mix in a `uuid4().hex[:8]` token.
- **`.bindspace` semantics**: "the address' bindable space" —
ip/host for `tcp`, the socket-file *directory* for `uds`. For
the new backends: the TIPC *scope* (§1 of plan 01), the iroh
*ALPN + relay/discovery realm* (plan 02). Do not overload this
transport-level bind selector with process namespace lifecycle.
Plan 03 augments an maddr/address declaration with a serializable
`BindspaceSpec` request and host-local `BindspaceRef`. A scoped,
non-serializable `Bindspace` carries that ref and owns the FD/lifetime
used during spawn bootstrap before a concrete address reaches
transport bind.
`Address.namespace` is already spec'd in the Protocol as
"the if-available OS-specific network namespace key" and is
currently unimplemented by both backends — plan 03 is the
first real consumer.
### 1.2 module-level listener lifecycle
```python
async def start_listener(
addr: <Proto>Address,
**kwargs,
) -> trio.SocketListener # or a trio.abc.Listener, see §3
...
def close_listener( # OPTIONAL
addr: <Proto>Address,
lstnr: trio.abc.Listener,
) -> None:
...
```
`close_listener()` is optional; `Endpoint.close_listener()`
(`_server.py:674`) `getattr`s it and treats absence as "closing
is implicit". `uds` needs it (unlinks the sock-file), `tcp`
does not.
### 1.3 the ONE piece of reflection you must not break
`Endpoint.start_listener()` (`_server.py:656`):
```python
tpt_mod: ModuleType = inspect.getmodule(self.addr)
lstnr = await tpt_mod.start_listener(addr=self.addr)
```
The transport module is found by `inspect.getmodule()` **on the
`Address` instance**. Therefore: *the `Address` class and its
`start_listener()`/`close_listener()` MUST live in the same
module.* Do not define the address type in `_types.py` or a
`_addrs.py` and the listener elsewhere.
Immediately after, the same method does:
```python
if (unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap():
self.addr = self.addr.from_addr(unwrapped)
```
i.e. it assumes `lstnr.socket.getsockname()` exists and that its
return value is a valid `from_addr()` input. This is fine for
TIPC (§3 of plan 01) and **is the main integration hazard for
iroh** (§3 of plan 02) — plans that break it must say so
explicitly and propose the upstream `_server.py` patch.
### 1.4 `class Msgpack<Proto>Stream(MsgpackTransport)`
Subclass `tractor.ipc._transport.MsgpackTransport`. You inherit
all framing (`<I` 4-byte little-endian length prefix),
`msgspec` codec ctx-var lookup, `TransportClosed` normalization,
`.drain()`, `__aiter__`. You implement only:
| member | notes |
| --- | --- |
| `address_type` | the `<Proto>Address` class |
| `layer_key: int` | OSI-ish layer, `4` for both current backends |
| `maddr` `@property` | `-> Multiaddr\|str`, via `mk_maddr(self.raddr)` |
| `connected(self) -> bool` | `tcp`/`uds` both use `self.stream.socket.fileno() != -1` |
| `connect_to(cls, addr, prefix_size=4, codec=None, **kw)` | `@classmethod`, returns an instance |
| `get_stream_addrs(cls, stream) -> (laddr, raddr)` | `@classmethod`, called from `MsgpackTransport.__init__` |
`MsgpackTransport.__init__` requires the object passed as
`stream` to satisfy:
- `await stream.send_all(bytes)`
- usable as `tricycle.BufferedReceiveStream(transport_stream=stream)`,
i.e. `await stream.receive_some(n)`
- `trio.BrokenResourceError` / `trio.ClosedResourceError` /
`ValueError('...unclean EOF...')` on the failure paths that
`_iter_packets()` and `send()` already `match` on
(`_transport.py:221-304`, `:436-499`).
That is **`trio.abc.Stream`, not `trio.SocketStream`**. The
`MsgTransport` Protocol's `stream: trio.SocketStream`
annotation (`_transport.py:83`) is a lie of convenience — the
actual `MsgpackTransport.__init__` param is typed
`trio.abc.Stream` and nothing in the msg path touches
`.socket`. Only `connected()` (which each backend defines) and
`Endpoint.start_listener()`'s `getsockname()` do.
### 1.5 verified-good news for socket-family backends
Both `trio.SocketStream` and `trio.SocketListener` are
**address-family agnostic**. Verified against the installed
`trio` (`trio/_highlevel_socket.py`): the only constructor
checks are
- `isinstance(socket, trio.socket.SocketType)`
- `socket.type == SOCK_STREAM`
- (listener) `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)` is truthy,
with `OSError` **suppressed** (the macOS carve-out, which
also covers exotic families that reject the opt)
There is no `AF_*` check and no `IPPROTO_TCP` hard dependency
(`TCP_NODELAY`/`TCP_NOTSENT_LOWAT` are set under
`suppress(OSError)`). Consequence: **any `SOCK_STREAM` family
CPython can create — including `AF_TIPC` — drops straight into
the existing `trio.SocketStream` + `trio.serve_listeners()`
path.** This is why plan 01 is small and plan 02 is not.
---
## 2. Registration tables (the full wiring checklist)
Adding a backend touches these and only these:
1. `tractor/runtime/_state.py:46`
`TransportProtocolKey = Literal['tcp', 'uds', ...]` — add the
key. This `Literal` is the canonical set; `_testing/pytest.py`
drives `--tpt-proto` validation off `_addr._address_types`,
and the spawn-backend fixture already models the
"drive-the-set-from-the-Literal" pattern
(`pytest.py:870-880`) — do the same rather than hardcoding.
2. `tractor/discovery/_addr.py:173` `_address_types: bidict`
`{'<key>': <Proto>Address}`. Note it is a **`bidict`**, so
the mapping must stay 1:1.
3. `tractor/discovery/_addr.py:181` `_default_lo_addrs`
`'<key>': <Proto>Address.get_root().unwrap()`.
⚠️ this dict is built at **import time**, so
`get_root()` must not require a live runtime, a loaded kernel
module, or network I/O. (`UDSAddress.def_bindspace =
get_rt_dir()` is the precedent for "cheap, pure, filesystem-
ish".) A backend whose root addr needs I/O must make this
entry lazy — propose that refactor explicitly.
4. `tractor/discovery/_addr.py:230` `wrap_address()` `match`
add a case iff your `unwrapped_type` isn't already uniquely
matched. **Preferably do the proto-key migration in §1.1
first**, after which this step becomes a one-line
`_address_types` entry instead of an order-sensitive `case`.
5. `tractor/ipc/_types.py``Address` union alias,
`_msg_transports` list, `_key_to_transport[('msgpack', key)]`,
`_addr_to_transport[<Proto>Address]`.
6. `tractor/ipc/_types.py:92` `transport_from_stream()` — the
`sock.family` `match`. For a non-socket stream type (iroh)
this needs a different discriminator; see plan 02 §3.3.
7. `tractor/discovery/_multiaddr.py`
`_tpt_proto_to_maddr`, and a `case` in both `mk_maddr()` and
`parse_maddr()`.
8. `tractor/ipc/__init__.py` — re-export if the backend has a
public surface.
9. `tractor/_testing/addr.py::get_rando_addr()` — per-proto
branch so the whole suite can run under `--tpt-proto <key>`.
10. `pyproject.toml` — new deps go in an **optional extra**, never
in `[project].dependencies`. See §5.
## 3. Where the `trio.SocketListener` assumption is load-bearing
`_serve_ipc_eps()` (`_server.py:1041`) annotates
`listener: trio.abc.Listener` and hands the list to
`trio.serve_listeners(handler=handle_stream_from_peer,
listeners=..., handler_nursery=stream_handler_tn)`.
`trio.serve_listeners` itself is generic over
`trio.abc.Listener`. So the *only* `SocketListener`-specific
code in the server path is the `getsockname()` reconciliation in
`Endpoint.start_listener()` (§1.3) and the type annotations.
`handle_stream_from_peer()` (`_server.py:298`) then does
`Channel.from_stream(stream)`
`transport_from_stream(stream)``sock.family` match (§2.6).
**Therefore**: a non-socket backend needs (a) a
`trio.abc.Listener` subclass, (b) a change to
`Endpoint.start_listener()` to not blindly `getsockname()`, and
(c) a change to `transport_from_stream()`'s discrimination.
All three are small, upstream-able, and *should be landed as
their own prep PR* before the backend itself — see plan 02 §3.
## 4. Handshake / discovery invariants you inherit
- Every accepted stream immediately does
`chan._do_handshake(aid=actor.aid)`; a peer that fails it is
logged at `runtime` and dropped, **not** raised
(`_server.py:334-365`). Discovery-sys "pings" rely on this,
so your `connect_to()` must raise something that normalizes
to `TransportClosed`/`ConnectionError` on a dead peer, never
a novel exception type.
- `_root.py:381-406` fail-fasts when a `registry_addrs` entry's
`proto_key` is not in `enable_transports`. Your key must be
spellable in both.
- `_root.py:256` currently enforces `len(enable_transports) == 1`.
Multi-tpt actors are a separate work item; none of these three
plans may depend on lifting it.
- Sub-actor bind addrs come from
`_runtime.py:1600-1610`: for each key in the parent-supplied
`enable_transports`, `get_address_cls(key).get_random()`.
So `get_random()` runs *in the child, post-fork, pre-listen*.
Anything it needs (kernel module, netns membership, an iroh
secret key) must already be true at that moment.
## 5. Dependency policy
`[project].dependencies` stays lean (see the boot-latency work,
gh #470: `import tractor` is budgeted at ~0.145s). Every new
backend dep is an extra:
```toml
[project.optional-dependencies]
tipc = [] # stdlib-only!
quic = ["iroh>=0.35"] # pin per plan 02 §1
wg = ["pyroute2>=0.9"] # pin per plan 03 §1
```
and every backend module must be **import-lazy**: a
`tractor/ipc/_<proto>.py` that imports its 3rd-party dep at
module scope must not be imported by `tractor/__init__.py`,
`tractor/ipc/__init__.py`, or `tractor/discovery/_addr.py`'s
import-time table construction. The `_addr._default_lo_addrs`
eager-dict (§2.3) is the trap: keep the backend's `get_root()`
dep-free, or make that table lazy.
## 6. Test-harness plumbing (identical for all three)
- `--tpt-proto <key>` (`_testing/pytest.py:409`) selects the
session-wide proto; the `tpt_proto` fixture mutates
`_state._def_tpt_proto` + `_runtime_vars['_enable_tpts']`
(`pytest.py:807-835`). Adding the key to `_address_types` is
what makes `--tpt-proto <key>` legal (`pytest.py:795-800`
asserts the lookup).
- The **acceptance bar** for every backend is: the *entire*
existing suite passes under `--tpt-proto <key>`, unmodified.
That is the whole point of the abstraction. Backend-specific
unit tests go in `tests/ipc/test_each_tpt.py` (the existing
`test_uds_bindspace_created_implicitly` /
`test_uds_double_listen_raises_connerr` are the model).
- Capability gating: each backend needs a **cheap, pure
predicate** + a `pytest.mark.skipif`, because these are all
environment-dependent. Verified example: on this dev box
`socket.socket(AF_TIPC, SOCK_STREAM)` raises
`OSError(97, 'Address family not supported by protocol')`
because the `tipc` module isn't loaded. Put the predicate in
the backend module (so apps can use it too), not in the test.
- New pytest marks must be registered in `pyproject.toml`, per
the project's fix-warnings-at-source rule (gh #469).
## 7. Code style (non-negotiable, matches the repo)
- module header tagline: `# tractor: distributed structured
concurrency.` for **new** files (not the legacy
`structured concurrent "actors".` form the existing `_tcp.py`
carries).
- AGPL header block copied verbatim from `_tcp.py`.
- `from __future__ import annotations` first.
- annotate *everything*, including locals:
`sockpath: Path = addr.sockpath`.
- `match`/`case` over `isinstance` chains for address and
error dispatch.
- multi-line call/`import` style with trailing commas.
- never emit a whitespace-only line.
- error messages are multi-line f-strings ending in `\n`, with
the `f'...\n' f'...\n'` implicit-concat layout and the
`>[`/`[>`/`<=(` nested-op sigils where a `nest_from_op()` is
in play.
- prefer pure functions + module-level helpers over methods;
keep `Address` types data-only. Where a helper needs
scoped setup/teardown, it's an `@acm` — not a class with
`.start()`/`.stop()`.
- pure getters: no `get_*(..., mutate=True)` flags; split into
a read-only getter and an explicit sibling setter.
---
## 8. Cross-plan sequencing
The three are independent *except*:
- plan 02 (iroh) needs the `_server.py` /
`transport_from_stream()` generalization (§3) — plan 01 does
**not**, and should therefore land first as the cheap proof
that the table-registration story works for a genuinely new
proto.
- plan 03 (wg) composes *under* whatever L4 tpt is in use and
its netns work is what finally implements
`Address.namespace`. It can land before or after 02, but its
`TunnelledAddress` design must be reviewed against plan 02's
address shape so the "tunnelled maddr" grammar (gh #443)
covers `/…/quic-v1/p2p/…` inner addrs too.
- All three want first-class `wg`/`quic`/`tipc` protos in
`py-multiaddr`; that upstream track is gh #483 and
multiformats/py-multiaddr#107/#108.

View File

@ -0,0 +1,707 @@
# Plan 01 — `TIPC` transport backend (`tractor/ipc/_tipc.py`)
Tracks gh [#378]. Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: TIPC is the *cheapest* new backend we can add and
simultaneously the only one that gives us cluster-wide service
discovery **for free, in the kernel**, replacing (for
TIPC-capable deployments) the whole `tractor.discovery`
registrar round-trip with a `bind()`/`connect()` on a
*service name*. It is stdlib-only: zero new dependencies.
[#378]: https://github.com/goodboy/tractor/issues/378
---
## 1. Why this is small: three verified facts
1. **CPython already speaks TIPC.** `socket.AF_TIPC` plus 23
`TIPC_*` constants are present in the stdlib on Linux
(verified on the dev box, py3.13):
`AF_TIPC, SOL_TIPC, TIPC_ADDR_ID, TIPC_ADDR_NAME,
TIPC_ADDR_NAMESEQ, TIPC_CFG_SRV, TIPC_CLUSTER_SCOPE,
TIPC_CONN_TIMEOUT, TIPC_{CRITICAL,HIGH,MEDIUM,LOW}_IMPORTANCE,
TIPC_DEST_DROPPABLE, TIPC_IMPORTANCE, TIPC_NODE_SCOPE,
TIPC_PUBLISHED, TIPC_SRC_DROPPABLE, TIPC_SUBSCR_TIMEOUT,
TIPC_SUB_CANCEL, TIPC_SUB_PORTS, TIPC_SUB_SERVICE,
TIPC_TOP_SRV, TIPC_WAIT_FOREVER, TIPC_WITHDRAWN,
TIPC_ZONE_SCOPE`.
`sock.bind()/connect()/getsockname()` take/return the
5-tuple `(addr_type, v1, v2, v3, scope)` — the last element
is optional on input and defaults to `0`.
2. **`trio` doesn't care about the address family.** Per
contract §1.5, `trio.SocketStream` and `trio.SocketListener`
only require a trio socket object of type `SOCK_STREAM`.
TIPC's `SOCK_STREAM` is a real connection-oriented reliable
byte stream. So we reuse `trio.SocketStream`,
`trio.SocketListener`, `trio.serve_listeners()`,
`MsgpackTransport`'s framing — *all of it*.
3. **It is not available by default.** On this box
`socket.socket(AF_TIPC, SOCK_STREAM)`
`OSError(97, 'Address family not supported by protocol')`
with no `tipc` in `/proc/modules`. `modprobe tipc` is
required; cross-node needs a bearer
(`tipc bearer enable media eth device <if>` or
`media udp name <n> localip <ip>`). Everything about this
plan's testability hinges on gating (§7).
Non-goals: `SOCK_RDM`/`SOCK_DGRAM`/`SOCK_SEQPACKET` message
modes, multicast fan-out, and TIPC group messaging. They are
genuinely interesting for a future `tractor` broadcast/pubsub
transport but they do **not** fit `MsgTransport`'s
stream-of-length-prefixed-msgs shape. Note them in the
follow-up issue, do not build them here.
---
## 2. `TIPCAddress`
### 2.1 the three TIPC address flavours, and which we use
| flavour | tuple | meaning |
| --- | --- | --- |
| `TIPC_ADDR_NAMESEQ` | `(type, lower, upper, scope)` | a *published range* — what a server `bind()`s |
| `TIPC_ADDR_NAME` | `(type, instance, domain, scope)` | a *lookup* — what a client `connect()`s |
| `TIPC_ADDR_ID` | `(node, ref, 0, scope)` | a concrete port id — the "physical" address |
The design decision that makes this backend coherent:
> **A `tractor` actor's TIPC address is a *service name*
> `(type, instance)`; `bind()` publishes the singleton range
> `(type, instance, instance)`; peers `connect()` by name and
> the kernel resolves + load-balances. `TIPC_ADDR_ID` is only
> ever an *observed* address (`getpeername()`), never a
> user-facing one.**
This is exactly the "leverage the built-in discovery machinery"
ask in #378: publishing a bind *is* registration, and
`connect()` on a name *is* a lookup, with no registrar actor in
the loop.
### 2.2 the struct
```python
class TIPCAddress(
msgspec.Struct,
frozen=True,
):
_stype: int # TIPC "type" == service class
_instance: int # service instance within the type
_scope: int = TIPC_CLUSTER_SCOPE
# observed-only, never part of identity/equality-by-intent
maybe_node: int|None = None # from TIPC_ADDR_ID getpeername()
maybe_ref: int|None = None
proto_key: ClassVar[str] = 'tipc'
unwrapped_type: ClassVar[type] = tuple[str, int]
def_bindspace: ClassVar[int] = TIPC_CLUSTER_SCOPE
```
**Unwrapped form** (the wire/`SpawnSpec` shape).
TIPC's natural form is `(stype, instance, scope)` — but a
2-tuple squeeze of it is a `(str, int)`, i.e. *the same coarse
shape as `TCPAddress`*, so `wrap_address()`'s
`case (str(), int())` steals it. This backend is therefore the
forcing function for the contract-doc's conclusion (§1.1):
> **make the unwrapped form carry an explicit proto-key, spelled
> with the `multiaddr` protocol name.**
```python
def unwrap(self) -> tuple[str, int, int, int]:
return ('tipc', self._stype, self._instance, self._scope)
```
`wrap_address()` then dispatches `_address_types[addr[0]]` and
the collision class disappears. **This is a prerequisite
migration commit, not part of this backend** — see contract §1.1
for its blast radius (wire format + every fixture + `piker`
config) and for the follow-on "stop handing raw tuples to users
at all, à la `ipaddress`" direction.
⚠️ an earlier revision of this plan proposed a self-tagging
`('tipc:<stype>:<scope>', instance)` string-prefix hack with an
ordered `case` guard. **Dropped** — it papers over the problem,
keeps `wrap_address()` order-sensitive, and doesn't help iroh's
`(str, str)`-vs-UDS collision at all. Do not resurrect it.
Note `TIPCAddress` is the first backend where `.unwrap()` is
**not** a lossless view of the live socket — `maybe_node`/
`maybe_ref` are observed metadata, exactly like
`UDSAddress.maybe_pid` (which is likewise excluded from
`.unwrap()`). Follow that precedent, including its `__repr__`
treatment (`_uds.py:242`).
### 2.3 how to pick `_stype` and `_instance`
- `_stype` = a `tractor`-reserved service class. TIPC reserves
0..63 for internal use (`TIPC_TOP_SRV == 1`,
`TIPC_CFG_SRV == 0`). Use a module constant
`TRACTOR_STYPE: int = 0x74_72_00_00` ("tr\0\0") as the default
and make it overridable via `TIPCAddress._stype` so an app
can partition service classes. Document that two `tractor`
trees sharing a cluster **and** a `_stype` share a namespace.
- `_instance` for `get_root()`: `1616` — mirrors the
`TCPAddress.get_root()` port and the `registry@1616.sock`
UDS filename, so the "1616 is tractor's registrar" idiom
holds across all backends.
- `_instance` for `get_random()`: TIPC gives us no
kernel-assigned-instance analogue of `port=0`, so we must
choose. Use a *pure* fn of the actor identity so it is
reproducible and collision-free:
```python
# 32-bit instance derived from the actor's uuid4 (+ pid when
# there's no live runtime, per the UDS precedent).
inst: int = int.from_bytes(
blake2b(seed.encode(), digest_size=4).digest(),
'big',
)
```
where `seed = f'{actor.aid.name}@{pid}'` if
`current_actor(err_on_no_runtime=False)` else
`f'{prefix}.{uuid4().hex[:8]}@{pid}'`. Must avoid the reserved
low range: `inst = 64 + (inst % (2**32 - 64))`.
⚠️ *unlike* `port=0`, a collision here surfaces as a
successful-but-shared publication (TIPC allows multiple
binders on the same name and round-robins!) rather than
`EADDRINUSE`. That is a silent-crosstalk failure mode; §7 has
the test that proves the 4-byte digest is enough and §9 has
the mitigation if it isn't.
- `_scope`: `TIPC_NODE_SCOPE` for a same-host-only actor (the
UDS-equivalent), `TIPC_CLUSTER_SCOPE` (default) for
cluster-visible. **This is `.bindspace`**:
```python
@property
def bindspace(self) -> int:
return self._scope
```
It is the honest analogue of "the set of hosts this bind is
reachable from", which is precisely the docstring in
`Address.bindspace`. (`TIPC_ZONE_SCOPE` is deprecated/aliased
to cluster in modern kernels — accept it on input, normalize
to cluster, log at `transport` level.)
### 2.4 `is_valid`
```python
@property
def is_valid(self) -> bool:
return (
self._instance != 0
and
self._stype not in _tipc_reserved_stypes # {0, 1, ...}
and
self._scope in (TIPC_NODE_SCOPE, TIPC_CLUSTER_SCOPE)
)
```
---
## 3. Listener + stream
### 3.1 `start_listener()`
```python
async def start_listener(
addr: TIPCAddress,
backlog: int = 128,
**kwargs,
) -> SocketListener:
sock = trio.socket.socket(
socket.AF_TIPC,
socket.SOCK_STREAM,
)
# publish the singleton name-range == "register the service"
await sock.bind((
socket.TIPC_ADDR_NAMESEQ,
addr._stype,
addr._instance,
addr._instance,
addr._scope,
))
sock.listen(backlog)
return SocketListener(sock)
```
Notes / hazards:
- `bind()` on `AF_TIPC` is **not** a filesystem or port-table
operation and can't block on DNS, but keep it `await`ed
through `trio.socket` anyway for uniformity.
- `backlog=128` matching `_uds.start_listener()`'s hard-won
value (see its comment at `_uds.py:317-331` re: concurrent
deregistration storms). Do not use `1`.
- **no `close_listener()` needed** — nothing to unlink. Omit the
function entirely (contract §1.2: absence means implicit).
Withdrawal of the published name happens on socket close.
- ⚠️ `SocketListener.__init__` will try
`getsockopt(SOL_SOCKET, SO_ACCEPTCONN)`. If TIPC rejects it,
trio's `except OSError: pass` covers us. Assert this in a
unit test rather than assuming.
- Wrap the bind in a `_reraise_as_connerr()`-style `@cm` (copy
the `_uds.py:256` pattern) so `EADDRINUSE`-ish and
`EAFNOSUPPORT` become `ConnectionError` with the addr in the
message. `EAFNOSUPPORT` here means "kernel module not
loaded" and deserves a *specifically actionable* message:
`'TIPC unavailable — try `sudo modprobe tipc`\n'`.
### 3.2 the `getsockname()` reconciliation
`Endpoint.start_listener()` does
`if lstnr.socket.getsockname() != self.addr.unwrap(): self.addr =
self.addr.from_addr(unwrapped)`.
For TIPC, `getsockname()` on a bound-but-listening socket
returns a `TIPC_ADDR_ID`-flavoured 5-tuple (the port id), *not*
the name-seq we bound. So the `!=` is **always true** and
`from_addr()` will be handed a 5-tuple.
Handle it inside `TIPCAddress.from_addr()` — do **not** patch
`_server.py`:
```python
@classmethod
def from_addr(cls, addr) -> TIPCAddress:
match addr:
# our own unwrapped form
case (str() as tag, int() as inst) if tag.startswith('tipc:'):
_, stype, scope = tag.split(':')
return TIPCAddress(int(stype), inst, int(scope))
# a kernel-observed TIPC_ADDR_ID 5-tuple: keep the
# *service* identity we already know and only annotate
# the observed port-id.
case (int() as atype, *rest) if atype == socket.TIPC_ADDR_ID:
...
```
The `TIPC_ADDR_ID` case cannot reconstruct `(stype, instance)`
— that info isn't in a port id. So `from_addr()` alone is
insufficient for the reconciliation path. **Resolution**: make
`from_addr()` raise a clear `ValueError` for the bare
`TIPC_ADDR_ID` case, and instead prevent the reconciliation
from firing by having `start_listener()` return a listener
whose `getsockname()` we never need — i.e. land this two-line
upstream fix in `_server.py:664`:
```python
if (
(unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap()
and
self.addr.rebind_from_sockname # ClassVar[bool] = True on tcp/uds
):
```
with `TIPCAddress.rebind_from_sockname: ClassVar[bool] = False`
(and `True` on `TCPAddress`/`UDSAddress`, preserving today's
behaviour exactly). Rationale: the reconciliation exists *only*
to learn the kernel-assigned port for `port=0` TCP binds (its
own comment says so, `_server.py:662`); TIPC has no such
late-binding, so opting out is semantically right rather than a
hack. **Land this as its own commit, ahead of the backend**,
with a test that `tcp`'s `port=0` behaviour is unchanged.
Keep the observed port-id available anyway: annotate
`ep.addr = ep.addr.with_port_id(*getsockname()[1:3])` (a pure
`msgspec.structs.replace()` helper) purely for logging/repr.
### 3.3 `MsgpackTIPCStream`
```python
class MsgpackTIPCStream(MsgpackTransport):
address_type = TIPCAddress
layer_key: int = 4
@property
def maddr(self) -> Multiaddr|str:
return mk_maddr(self.raddr)
def connected(self) -> bool:
return self.stream.socket.fileno() != -1
@classmethod
async def connect_to(
cls,
destaddr: TIPCAddress,
prefix_size: int = 4,
codec: MsgCodec|None = None,
**kwargs,
) -> MsgpackTIPCStream:
sock = trio.socket.socket(AF_TIPC, SOCK_STREAM)
with close_on_error(sock):
# NOTE: connect by *name* -> kernel does the lookup,
# so this is our "discovery" call.
await sock.connect((
socket.TIPC_ADDR_NAME,
destaddr._stype,
destaddr._instance,
0, # domain: 0 == "anywhere in scope"
destaddr._scope,
))
return cls(
trio.SocketStream(sock),
prefix_size=prefix_size,
codec=codec,
)
```
- reuse `trio._highlevel_open_unix_stream.close_on_error` (the
UDS backend already imports it) or inline the equivalent
`try/except: sock.close(); raise`.
- `SO_/TIPC_` opts worth setting and documenting:
- `setsockopt(SOL_TIPC, TIPC_IMPORTANCE, TIPC_HIGH_IMPORTANCE)`
for the *parent<->child* lifetime channel — this is a real
win TIPC gives us that TCP can't: the runtime's
supervision channel can outrank bulk app traffic under
congestion. Wire it as a `connect_to(..., importance=...)`
kwarg defaulted from a module constant, and have
`_runtime.py`'s parent-chan path pass the high value **in a
follow-up** (don't couple it to this PR).
- `TIPC_CONN_TIMEOUT` — the kernel-side connect timeout;
leave at default, we have `trio` cancel scopes.
- `TIPC_DEST_DROPPABLE = 0` on the connection so undeliverable
msgs come back as errors rather than being silently dropped.
- **`connect_to()` on a name with no publisher**: TIPC returns
`ECONNREFUSED`/`EHOSTUNREACH` promptly (no SYN-timeout wait),
which is *better* discovery-ping behaviour than TCP. Confirm
the errno and make sure it surfaces as `ConnectionError`
(contract §4 — the registrar ping path depends on it).
### 3.4 `get_stream_addrs()`
```python
@classmethod
def get_stream_addrs(cls, stream) -> tuple[TIPCAddress, TIPCAddress]:
sock = stream.socket
# both return TIPC_ADDR_ID 5-tuples for a connected sock
l_id = sock.getsockname()
r_id = sock.getpeername()
...
```
Problem: neither end's port-id tells us the *service name*. The
`laddr`/`raddr` are used for logging, `Channel.raddr`,
`Server._peers` keying-adjacent repr, and `maddr`. Design:
- the **connecting** side knows the destaddr it dialled →
`connect_to()` overrides `_raddr` after construction with the
known-good `TIPCAddress`, exactly as
`MsgpackUDSStream.connect_to()` does for the peer-pid case
(`_uds.py:539-543`).
- the **accepting** side does not know the peer's service name
from the socket. Two honest options:
- **(a) accept it: `raddr` carries only `(node, ref)`** via
`maybe_node`/`maybe_ref`, `_stype/_instance` set to a
sentinel `-1`, and `__repr__` renders
`TIPCAddress[<peer-node:0x...>:<ref>]`. The `Aid` from the
handshake already gives us the peer's logical identity, so
nothing in the runtime actually *needs* the peer's service
name. **Recommended.**
- (b) piggyback the peer's own bound name in the handshake.
Rejected for this PR: touches `Aid`/msg-spec.
- `laddr` on the accepting side: the `Endpoint` knows its own
`addr`; but `get_stream_addrs()` is a `@classmethod` with only
the stream. Use `TIPC_ADDR_ID` for `laddr` too and let
`Endpoint.peer_tpts` keying (which is by *peer* addr) still
work. Verify nothing asserts `laddr == ep.addr` — grep for
`.laddr` uses before committing (`_server.py`'s
`con_status` logging, `Channel.pformat()`).
---
## 4. Multiaddr representation
There is no `/tipc` in the multiaddr protocol table. Interim
grammar, mirroring how `uds` maps to the spec-legal `/unix`:
```
/tipc/<stype>/<instance> # scope implied = cluster
/tipc/<stype>/<instance>/<scope> # explicit
```
- `_tpt_proto_to_maddr['tipc'] = 'tipc'` and a `mk_maddr()`
`case 'tipc':` building the above.
- `parse_maddr()` gets `case ['tipc']:` — but note
`py-multiaddr` will reject an unregistered protocol name
outright, so this **requires an upstream registration** (same
track as the `wg` work, gh #483 /
multiformats/py-multiaddr#107). Until that lands:
- `MsgpackTIPCStream.maddr` returns the **`str`** form (the
`MsgTransport.maddr` return type is already
`Multiaddr|str`, and `MsgpackUDSStream.maddr` already
exercises the `str` branch), and
- `parse_maddr()` special-cases the `/tipc/` prefix *before*
handing the string to `Multiaddr()`.
Document this as the reason gh #443's "standardize on
returning `Multiaddr` everywhere" item stays blocked.
Propose `/tipc/` upstream as: name `tipc`, code TBD, size
variable, value `<stype>:<instance>:<scope>` — or as three
composed protos. Prefer *one* proto with a structured value so
the maddr stays 2-segment like `/unix/...`.
---
## 5. Discovery: the actually-interesting part
Two independently-shippable layers. **Layer A is in scope for
the first PR; layer B is a fast-follow.**
### 5.1 Layer A — "discovery by bind" (free)
Because `bind(TIPC_ADDR_NAMESEQ)` publishes and
`connect(TIPC_ADDR_NAME)` resolves, a `tractor` tree whose
`registry_addrs` are TIPC service names needs **no registrar
liveness at all** for the connect path: `find_actor()`'s
"connect to the registrar and ask" becomes "connect to the
service name directly". Concretely:
- `tractor.discovery._api.find_actor()` etc. keep working
unchanged (they go through the registrar), *and*
- a new, TIPC-only fast path becomes possible: derive an actor's
service name from its `(name, uuid)` and dial it without any
registrar hop.
Do **not** build the fast path in PR 1. Instead, prove the
property with a test (§7.4) and file the follow-up: it changes
`discovery` semantics (name→instance derivation must be a
documented, stable, cross-language-able hash) and deserves its
own design.
### 5.2 Layer B — the topology service (`TIPC_TOP_SRV`)
This is what makes #378's "end game cluster proto" claim real:
a *subscription* to name-table events, i.e. push-based
`register`/`deregister` for free, replacing the registrar's
polled `find_actor()`.
Mechanics (verify each field against
`linux/include/uapi/linux/tipc.h` + `net/tipc/topsrv.c` at
implementation time — the struct layout below is from the uapi
header and the byte-order caveat is real):
```python
# SOCK_SEQPACKET connected to the topology server
sock = trio.socket.socket(AF_TIPC, SOCK_SEQPACKET)
await sock.connect((
socket.TIPC_ADDR_NAME,
socket.TIPC_TOP_SRV, # == 1
socket.TIPC_TOP_SRV,
0,
))
# struct tipc_subscr {
# struct tipc_name_seq seq; /* 3 * __u32: type, lower, upper */
# __u32 timeout; /* TIPC_WAIT_FOREVER == ~0 */
# __u32 filter; /* TIPC_SUB_{PORTS,SERVICE,CANCEL} */
# char usr_handle[8];
# } /* == 28 bytes */
_SUBSCR_FMT: str = '=IIIII8s' # ⚠ 5*I is 20 -> use '=5I8s'
```
- **byte order**: the topology server historically accepts both
host and swapped order and auto-detects; modern kernels are
strict-ish. Pack native (`'='`) first, and if the server
closes the connection immediately, retry with `'>'`. Encode
that as a one-time probe helper
`_detect_topsrv_endianness()` cached at module level — and
put a `# ?TODO` pointing at `net/tipc/topsrv.c` for someone
to make it deterministic.
- **events**: `struct tipc_event` is `event: u32`,
`found_lower: u32`, `found_upper: u32`,
`port: {ref: u32, node: u32}`, then the 28-byte subscription
echo → 40 bytes. `event ∈ {TIPC_PUBLISHED, TIPC_WITHDRAWN,
TIPC_SUBSCR_TIMEOUT}`.
- **trio shape** — this is where the "nearly-functional,
modern-async" style pays off; expose it as an `@acm` yielding
a `trio` receive-channel of typed events, *not* a class:
```python
@acm
async def open_topology_events(
stype: int = TRACTOR_STYPE,
lower: int = 0,
upper: int = 0xFFFFFFFF,
filter: int = TIPC_SUB_SERVICE,
timeout: int = TIPC_WAIT_FOREVER,
buf_size: int = 64,
) -> AsyncGenerator[
trio.MemoryReceiveChannel[TIPCNameEvent],
None,
]:
...
```
with `TIPCNameEvent(msgspec.Struct, frozen=True)` fields
`kind: Literal['published','withdrawn','timeout']`,
`addr: TIPCAddress`, `node: int`, `ref: int`. One
`trio.lowlevel`-free implementation: a nursery-spawned reader
task doing `await sock.recv(40)` in a loop and
`send_nowait()`ing decoded events, with the `@acm` closing the
socket on exit → reader gets `ClosedResourceError` → cancel
scope collapses. Standard `tractor` `@acm` discipline.
- **consumer**: `tractor/discovery/_registry.py` gains an
optional "watch" mode so a registrar (or any actor) can keep
a live view of the actor set without polling. Sketch the
integration in the follow-up issue; do not wire it in PR 1.
- **`SOCK_SEQPACKET` is fine here** because this socket never
goes through `MsgpackTransport` — it's a plain trio socket
used with `recv()`. The contract's "`SOCK_STREAM` only"
constraint applies to `MsgTransport` streams, not to this.
---
## 6. Commit sequencing (each independently reviewable + green)
1. `_server.py`: add `Address.rebind_from_sockname:
ClassVar[bool]`, gate the `getsockname()` reconciliation on
it, `True` for tcp/uds. Test: tcp `port=0` unchanged.
2. `tractor/ipc/_tipc.py`: `TIPCAddress` + `is_tipc_available()`
predicate + `start_listener()`. No transport yet.
Tests: address round-trip (`unwrap`/`from_addr`/`wrap_address`),
`get_random()` uniqueness, bind/listen + `SO_ACCEPTCONN`
tolerance, `EAFNOSUPPORT` → actionable `ConnectionError`.
3. `MsgpackTIPCStream` + `connect_to()` + `get_stream_addrs()`.
Test: two `trio` tasks in one proc exchange a msg over
`Msgpack` framing (no `tractor` runtime).
4. registration tables (contract §2 items 1-6, 9) +
`pyproject.toml` mark/extra. Test: full suite under
`--tpt-proto tipc` (§7.3).
5. maddr support (`str` form + prefix special-case) + docs.
6. `open_topology_events()` @acm + its tests (layer B).
7. docs page + `docs/` example.
Per project convention, a reproducing/guard test lands in its
own commit **before** the fix it guards.
---
## 7. Testing
### 7.1 the capability predicate (in `_tipc.py`, public)
```python
def is_tipc_available() -> bool:
'''
True iff this kernel can create an `AF_TIPC` socket, i.e.
the `tipc` module is loaded.
'''
try:
socket.socket(socket.AF_TIPC, socket.SOCK_STREAM).close()
return True
except OSError:
return False
```
Cache it in a module global (it can't change without a
`modprobe`, and a cold call costs a syscall). Pure predicate, no
side effects, no logging.
### 7.2 gating
- `pytest.mark.tipc` registered in `pyproject.toml`.
- module-level
`pytestmark = pytest.mark.skipif(not is_tipc_available(),
reason='`tipc` kernel module not loaded (`modprobe tipc`)')`
in `tests/ipc/test_tipc.py`.
- `--tpt-proto tipc` with no module must fail **loudly and
early** with the actionable message, not with 400 confusing
timeouts. Add the check to the `tpt_protos` fixture's existing
per-proto validation loop (`_testing/pytest.py:795`): if the
chosen `Address` type exposes an `is_available()`-style
classmethod, call it and `pytest.fail()` with its reason.
Generalize (don't special-case tipc) — plans 02/03 need the
same hook.
### 7.3 CI
- add a job matrix entry `--tpt-proto tipc` that runs
`sudo modprobe tipc` in a `before` step. GH's
`ubuntu-latest` runners do allow `modprobe tipc` (the module
ships with the standard Ubuntu kernel package); verify in a
throwaway workflow before wiring the matrix. If it turns out
to be unavailable, fall back to a container job with
`--privileged`/`--cap-add NET_ADMIN`, and mark the job
`continue-on-error` until it's proven stable.
- cross-node TIPC (bearer) cannot be CI'd; cover it with a
documented manual smoke test in the docs page, in the style
of gh #482's LAN examples.
### 7.4 backend-specific tests worth writing
- **name-publication is discovery**: bind a listener on
`(stype, inst)`, then from a second task `connect()` by name
and assert it lands — *without* any `tractor` registrar.
- **`get_random()` collision resistance**: 10k `get_random()`
calls with no live runtime → 10k distinct `_instance`s.
(This is the silent-crosstalk risk from §2.3; if the 4-byte
digest ever collides in this test, escalate to §9.)
- **round-robin surprise**: two listeners bound to the *same*
`(stype, inst)` both succeed (TIPC allows it) and connects
distribute. Assert the observed behaviour and reference it
from the `get_random()` docstring so the next reader knows
why the hash matters.
- **scope isolation**: a `TIPC_NODE_SCOPE` bind is not visible
to a cluster-scope lookup from another node (manual/marked).
- **importance opt** round-trips via `getsockopt`.
- **graceful + abrupt close** produce `TransportClosed` with the
same `loglevel` classification as tcp/uds — i.e. re-run the
relevant `tests/ipc/test_each_tpt.py` cases parametrized over
the new proto rather than writing new ones.
---
## 8. Deployment / docs deliverable
A `docs/` page (and/or an `examples/` script) covering:
```bash
# single host, node-scope only
sudo modprobe tipc
tipc node get addr
# multi-host over ethernet (pairs beautifully with plan 03's wg)
sudo tipc bearer enable media eth device eth0
# ...or over UDP when L2 isn't available:
sudo tipc bearer enable media udp name uc localip 10.0.11.1
tipc link list
tipc nametable show # <- see tractor's published services!
```
`tipc nametable show` displaying live `tractor` actors is the
single best demo this backend has; lead with it.
---
## 9. Known risks + escalations
| risk | mitigation |
| --- | --- |
| `_instance` hash collision → silent crosstalk (two actors share a service name, TIPC round-robins connects between them) | §7.4 test; if it bites, add a post-bind verification handshake, or bump to a 6-byte digest folded into `(stype_low, instance)` |
| kernel/module unavailability everywhere (dev boxes, macOS, CI) | hard gating (§7.2); TIPC is explicitly an *opt-in cluster* transport, never a default |
| `getsockname()` returns port-id not name | the `rebind_from_sockname` opt-out (§3.2), landed first |
| unregistered `/tipc` multiaddr proto | `str` maddr fallback (§4) + upstream track gh #483 |
| stale docs (#378 notes tipc.io docs may be out of date) | treat `include/uapi/linux/tipc.h` + `net/tipc/` as the only normative source; cite file+symbol in code comments |
| `SOCK_SEQPACKET` topology framing byte-order | probe helper + `?TODO` (§5.2) |
## 10. Follow-up issue seeds
- **register `/tipc` in the multiaddr spec**, mirroring the `wg`
track (multiformats/py-multiaddr#107/#108 + gh #483). Same
shape of work: propose the proto + code, land a codec in
`py-multiaddr`, then drop our `str`-maddr fallback (§4). Worth
filing *alongside* the `wg` spec-submission issue so both
proposals go up together rather than as one-offs.
- registrar-less discovery fast path via name derivation (§5.1)
- `TIPC_TOP_SRV`-driven push registry in
`discovery/_registry.py` (§5.2)
- `TIPC_IMPORTANCE` for the parent<->child lifetime channel
(§3.3) — genuinely novel supervision QoS, no other backend
can do it
- TIPC multicast / group messaging as a *broadcast* transport
for `tractor.trionics` fan-out (explicitly not `MsgTransport`)
- dual-link resiliency / multi-homing (#378's "hybrid dual link")
once bearers are scripted in the docs

View File

@ -0,0 +1,566 @@
# Plan 02 — QUIC backend via `iroh` FFI, uniffi-async rewritten onto `trio`
Tracks gh [#353]. Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: the value of `iroh` over "just QUIC" is
`NodeId`-addressed, NAT-traversing, relay-fallback endpoints —
i.e. a `tractor` actor tree that spans hosts *without* a
reachable listening socket. The cost is that `iroh`'s python
surface is `uniffi`-generated **asyncio** and its listener is not
a socket. This plan spends its complexity budget in exactly two
places: a `trio`-native uniffi future bridge, and a
`trio.abc.Listener`/`Stream` adapter pair. Everything else is
contract boilerplate.
[#353]: https://github.com/goodboy/tractor/issues/353
---
## 1. Library selection (decided, with the rejected alternatives)
**Chosen: `iroh` (PyPI, from `n0-computer/iroh-ffi`), pinned to
a single minor.** The `iroh` python package is a `uniffi`
binding over the rust `iroh` crate (QUIC via `quinn`/`noq`).
Rejected, and why — record these so the next implementer doesn't
relitigate:
- **`aioquic`** (sans-io + asyncio): genuinely trio-portable
(`hypercorn` already pairs its sans-io core with a trio UDP
server, see the links in #353) and dependency-light. But it
gives us *only* QUIC — no NodeId identity, no hole punching,
no relay. We'd be reimplementing iroh's whole reason for
existing. **Keep as the documented fallback** if the FFI
bridge (§2) proves unmaintainable; the `MsgTransport` and
`Listener` adapters from §3 are ~90% reusable against an
`aioquic` core, which is a deliberate design property of this
plan.
- **`quiche` / `quinn` via a hand-rolled PyO3 ext**: strictly
more work than reusing `iroh-ffi`, and puts us in the
build-wheels business.
- **`trio-asyncio`**: viable *shortcut* to run the asyncio-shaped
bindings under trio, and `tractor` already ships
infected-asyncio machinery (`tractor.to_asyncio`,
`tests/test_infected_asyncio.py`). Rejected as the *primary*
design because it makes every IPC send/recv cross a
loop-boundary shim in the hot path, and because #353 asks
explicitly for the asyncio support to be "rewritten for trio".
**But**: build it first as the throwaway spike (§6 step 0) to
de-risk the iroh API surface before writing the bridge.
Version pinning: `iroh` moves fast and has had breaking
API renames across minors. Pin `iroh>=X.Y,<X.Y+1` in a `quic`
extra, and **write down the exact resolved version + the
generated `iroh/_uniffi*` module layout** in the module
docstring, because §2 depends on generated-code internals.
**Step 0 of implementation is an API-truth pass**: install the
pinned `iroh`, `python -c "import iroh; help(iroh)"`, and record
in this doc's §1.1 the real names of: endpoint builder, secret
key type, `connect`/`accept`, bi-stream open/accept, the
send/recv methods and their exact signatures/return types, and
whether they're `async def`. Everything below uses *provisional*
names and must be reconciled. Do not skip this; do not guess
from memory.
### 1.1 API-truth table (fill in during step 0)
| concept | provisional name | actual (fill in) |
| --- | --- | --- |
| secret key | `iroh.SecretKey.generate()` | |
| endpoint builder | `iroh.Endpoint.builder(...).bind()` | |
| node id | `endpoint.node_id() -> str` | |
| node addr (relay + direct) | `iroh.NodeAddr` | |
| dial | `await endpoint.connect(node_addr, alpn)` | |
| accept conn | `await endpoint.accept()` | |
| open bi-stream | `await conn.open_bi()` | |
| accept bi-stream | `await conn.accept_bi()` | |
| send | `await send_stream.write_all(b)` | |
| recv | `await recv_stream.read(n) -> bytes\|None` | |
| half-close | `await send_stream.finish()` | |
---
## 2. The `trio`-native uniffi future bridge (`tractor/ipc/_uniffi_trio.py`)
### 2.1 what uniffi actually generates
`uniffi`'s async support does not use asyncio *semantically*
it uses asyncio only as the *executor* for a poll loop. The
generated python for an `async fn` is, in shape:
1. call `_uniffi_..._<method>(...)` → returns an opaque
`RustFuture` handle (a `void*`/`u64`).
2. loop: call
`ffi_..._rust_future_poll_<T>(handle, callback, callback_data)`.
The callback is a C-ABI fn pointer invoked **from an
arbitrary rust thread** with a poll-result code
(`READY`/`MAYBE_READY`).
3. the generated glue's callback resolves an
`asyncio.Future` via `loop.call_soon_threadsafe(...)`; the
coroutine awaits it, then re-polls.
4. on ready: `ffi_..._rust_future_complete_<T>(handle,
&call_status)` → the value; then
`ffi_..._rust_future_free_<T>(handle)`.
**The asyncio dependency is confined to step 3.** That is the
whole insight: the bridge is ~40 lines.
### 2.2 the trio version
```python
async def await_rust_future(
poll: Callable, # ffi_..._rust_future_poll_<T>
complete: Callable, # ffi_..._rust_future_complete_<T>
free: Callable, # ffi_..._rust_future_free_<T>
handle: int,
lift: Callable[[Any], Any],
) -> Any:
'''
Drive a `uniffi` rust-future to completion on the current
`trio` task, bridging rust-thread wakeups via
`TrioToken.run_sync_soon()`.
'''
token = trio.lowlevel.current_trio_token()
while True:
wake = trio.Event()
# NOTE, invoked from a *rust* thread!
def _cb(_data, poll_code):
token.run_sync_soon(wake.set)
cb = _UNIFFI_FUTURE_CALLBACK(_cb) # keep a strong ref!
poll(handle, cb, 0)
await wake.wait()
if <poll_code was READY>:
break
try:
status = _UniffiRustCallStatus.default()
res = complete(handle, status)
_uniffi_check_call_status(status) # reuse generated helper
return lift(res)
finally:
free(handle)
```
Critical details, each a real bug if missed:
- **`token.run_sync_soon()` is the only trio API callable from a
foreign thread**, and it is documented as such. Use it; do
*not* use `trio.from_thread.run_sync` (requires a trio thread
context) and do not touch the `Event` directly from the
callback.
- **the poll code must reach the trio side.** Capture it in a
`nonlocal`/1-slot list written by the callback *before*
`run_sync_soon`, since the callback owns the value. Handle
`MAYBE_READY` by re-polling (the loop above does).
- **keep the `ctypes` callback object alive** across the await —
a GC'd `CFUNCTYPE` trampoline is a segfault. Bind it to a
local *and* make sure the local outlives the `poll()` call
window.
- **cancellation.** `await wake.wait()` is a trio checkpoint, so
a `Cancelled` can fire while rust still owns the future. On
cancel we must still `free(handle)` — and per uniffi, the
correct sequence is to call the generated
`ffi_..._rust_future_cancel_<T>(handle)` then continue
polling to completion before `free`. Wrap the whole thing so
the cancel path does:
`with trio.CancelScope(shield=True): cancel(handle); <drain
poll loop>; free(handle)`. **Bounded** shield (add a
`trio.move_on_after()` with a module-level constant) so a
wedged rust future can't make an actor un-cancellable —
`tractor` is SC-first and an unbounded shield here would
violate that.
- **`trio.lowlevel.current_trio_token()`** must be captured on
the trio side (not in the callback).
### 2.3 how to apply it to the generated bindings
Do **not** fork/vendor the generated `iroh` python. Instead ship
a *narrow* re-dispatch shim:
- write `tractor/ipc/_uniffi_trio.py` with `await_rust_future()`
plus a `@cm patch_uniffi_for_trio()` that monkey-patches the
generated module's single async-driver entrypoint (in current
uniffi that's `_uniffi_rust_call_async` / `_rust_call_async`,
one function) to the trio implementation.
- verify at import time that the expected symbol exists and
raise a clear, actionable error naming the pinned `iroh`
version if not. A silent fallback to asyncio would be a
nightmare to debug.
- **plan for this to break on `iroh`/`uniffi` upgrades.** Mitigate
with (a) a unit test that drives one trivial `iroh` async call
under bare `trio.run()` and asserts no event loop was ever
created (`asyncio.get_event_loop_policy()` untouched /
`asyncio._get_running_loop() is None`), and (b) a docstring
pointing at the uniffi codegen template this mirrors.
If step 0 reveals the generated code is *structurally* hostile
to this (e.g. `asyncio` imported and used at module scope for
more than the driver), fall back to option (b): run iroh under
`tractor.to_asyncio` infected mode and open the follow-up to
revisit. Say so in the PR rather than fighting it.
---
## 3. Mapping QUIC onto `MsgTransport`
### 3.1 the layering decision
QUIC natively multiplexes streams inside one connection. The
mapping that preserves *all* existing `tractor` semantics with
the least new code:
```
iroh Endpoint == one per actor (process) -> the "listener"
iroh Connection == one per peer actor -> pooled
iroh bi-stream == one `Channel`/`MsgTransport` -> 1:1
```
- keep the 4-byte `<I` length-prefix framing **unchanged**. It's
redundant-ish over a QUIC stream but it means
`MsgpackTransport` is reused verbatim, and framing is cheap.
Revisit only after it works.
- **one-task-per-stream** falls out naturally, which is exactly
the #353 note about QUIC sub-stream QoS/cancellation fitting
`trio`.
- `layer_key: int = 4` still (QUIC is L4-ish); note in a comment
that this backend is really 4+security+multiplex.
**Connection pooling** is the one place we add state the other
backends don't have: dialing the same peer twice should reuse
the `Connection` and open a second bi-stream. Implement as a
module-level `dict[NodeId, Connection]` guarded by a
`trio.Lock`... **no** — that's a per-process cache with
lifetime/teardown hazards. Instead reuse the codebase's existing
idiom: `tractor.trionics.maybe_open_context()` keyed on the
node-id, which already solves exactly this (one-cached-resource-
per-key, refcounted, teardown-on-last-exit) and whose teardown
semantics were just hardened (gh #488). Use it; do not hand-roll
a cache. Anything concurrency-subtle here should get the
`conc-anal` skill run over it.
### 3.2 `IrohAddress`
```python
class IrohAddress(
msgspec.Struct,
frozen=True,
):
_node_id: str # 32B ed25519 pubkey, hex or z32
_alpn: str = 'tractor/0' # the bindspace!
# optional dial hints; NOT part of identity
maybe_relay_url: str|None = None
maybe_direct_addrs: tuple[str, ...] = ()
proto_key: ClassVar[str] = 'iroh' # ?or 'quic'; see §3.2.1
unwrapped_type: ClassVar[type] = tuple[str, str]
def_bindspace: ClassVar[str] = 'tractor/0'
```
- **`.unwrap() -> (node_id_str, alpn_str)`** — a `(str, str)`
tuple, which is *unambiguously distinct* from
`TCPAddress`'s `(str, int)`. But careful:
`wrap_address()`'s UDS case is
`case (_, filename) if type(filename) is str` — which
**already catches `(str, str)`**. So the iroh `case` MUST be
ordered *before* the UDS case and guarded, e.g.
`case (str() as nid, str() as alpn) if _is_node_id(nid):`
with `_is_node_id()` a cheap length+alphabet check. Add a
regression test asserting a UDS `(dir, filename)` pair still
wraps to `UDSAddress` — this is the exact "wrong transport
loaded" hazard `_addr.py:214` warns about.
- `.bindspace``self._alpn`. This is the honest analogue:
the ALPN is the set of endpoints willing to talk to you, and
two `tractor` deployments sharing an iroh network are
separated by ALPN exactly as two UDS deployments are
separated by directory. Include a `tractor` version/proto
epoch in the default ALPN so incompatible runtimes can't
handshake.
- `.is_valid` → node-id parses, alpn non-empty.
- **`get_root()` is the hard one.** There is no
well-known-port analogue: an iroh node id is a *keypair*, so
"the host's default registrar addr" requires a *persisted
secret key*. Design:
- the root/registrar's secret key lives at
`get_rt_dir() / 'iroh_registrar.key'` (0600), created on
first use.
- `get_root()` must stay **pure and import-time-safe**
(contract §2.3: `_default_lo_addrs` is built at import!).
So `get_root()` *reads* the key file if present and
otherwise returns an `IrohAddress` with
`_node_id=''`/sentinel, and the **generation** happens in
an explicit sibling — `ensure_registrar_key() ->
IrohAddress` — called from the listen path. Pure getter,
explicit setter; do not smuggle key generation into
`get_root()`.
- this almost certainly means `_default_lo_addrs` must become
lazy for this backend. **Land that refactor as its own prep
commit** (a `default_lo_addrs()` that computes per-call
instead of the import-time dict) — it also unblocks plan
03's netns-scoped defaults.
- `get_random()`: generate a fresh `SecretKey` per subactor and
return its node-id. Note this runs post-fork pre-listen
(contract §4) and costs an ed25519 keygen (~µs, fine). The
*secret* can't live in a frozen `Address`, so it must be
stashed where the listen path can find it: a module-level
`dict[node_id, SecretKey]` populated by `get_random()` and
consumed+popped by `start_listener()`. Ugly but honest;
document it and note the alternative (thread the key through
`Endpoint`) as a follow-up.
#### 3.2.1 `proto_key`: `'iroh'` vs `'quic'`
Use **`'quic'`** for the `proto_key`/`--tpt-proto` name and
name the module `_quic.py`, with `iroh` as the *implementation*.
Rationale: it keeps the door open for the `aioquic` fallback
(§1) without a user-visible rename, and it matches how `uds` is
a proto name rather than a lib name. Put `iroh`-specific bits
behind an internal `_iroh` submodule if the file gets big.
### 3.3 the `trio.abc` adapters — where the real work is
Contract §3 says a non-socket backend needs three upstream
generalizations. Land them **as a prep PR, before any iroh
code**, so they can be reviewed on their own merits with
tcp/uds still the only backends:
1. **`Endpoint.start_listener()` must not assume
`.socket.getsockname()`.** Use the same
`Address.rebind_from_sockname: ClassVar[bool]` gate that
plan 01 §3.2 introduces — coordinate so it lands once. (If
plan 01 lands first, this is free.)
2. **`transport_from_stream()` (`_types.py:92`) must not assume
`trio.SocketStream`.** Replace the `sock.family` match with:
check `isinstance(stream, trio.SocketStream)` → existing
family match; else look for a
`stream.tpt_key: ClassVar[MsgTransportKey]` attribute on the
adapter and use it. Keeps the existing path byte-identical
and makes new stream types self-describing (a much better
shape than growing an `isinstance` ladder).
3. **type annotations**: `handle_stream_from_peer(stream:
trio.SocketStream)` → `trio.abc.Stream`; `Endpoint._listener:
SocketListener|None` → `trio.abc.Listener|None`;
`MsgTransport.stream: trio.SocketStream`
`trio.abc.Stream`. Annotation-only, zero behaviour change.
Then the adapters:
```python
class QuicMsgStream(trio.abc.HalfCloseableStream):
'''
A single `iroh` bi-directional QUIC stream presented as
a `trio` byte-stream so `MsgpackTransport` can frame over
it unmodified.
'''
tpt_key: ClassVar[MsgTransportKey] = ('msgpack', 'quic')
def __init__(self, conn, send, recv) -> None: ...
async def send_all(self, data: bytes) -> None: ...
async def wait_send_all_might_not_block(self) -> None: ...
async def receive_some(self, max_bytes: int|None = None) -> bytes: ...
async def send_eof(self) -> None: ...
async def aclose(self) -> None: ...
```
Non-negotiable behaviours (each maps to a `match` case that
already exists in `_transport.py` and must keep working):
- `receive_some()` returns `b''` at clean EOF →
`MsgpackTransport._iter_packets()` sees `header == b''` and
raises `TransportClosed(loglevel='transport')`. **This is the
graceful-disconnect path the whole runtime relies on**; get it
right first.
- a reset/aborted stream → raise `trio.BrokenResourceError`.
- use after local close → raise `trio.ClosedResourceError`
(ideally with `'another task closed this fd'`-equivalent text
absent, so the `raise_on_report` branch at
`_transport.py:290` stays quiet).
- `send_all()` on a closed peer → `trio.BrokenResourceError`.
- honour `trio`'s one-task-per-direction rule: guard with
`trio._util.ConflictDetector` equivalents (or just document +
assert), because `MsgpackTransport` already serializes sends
with a `StrictFIFOLock` but recvs are single-task by
construction.
- **buffering**: if iroh's `read()` doesn't support
"read up to n", `receive_some()` must maintain an internal
leftover buffer. Note `MsgpackTransport` wraps us in
`tricycle.BufferedReceiveStream` anyway, so `receive_some()`
just needs *some* nonzero-progress contract.
```python
class QuicListener(trio.abc.Listener):
'''
Accepts iroh `Connection`s and yields one `QuicMsgStream`
per accepted bi-stream, so `trio.serve_listeners()` spawns
one `handle_stream_from_peer()` per `Channel`.
'''
async def accept(self) -> QuicMsgStream: ...
async def aclose(self) -> None: ...
```
The accept-side subtlety: `trio.abc.Listener.accept()` yields
one stream per call, but iroh gives us *connections* which then
yield *streams*. So `QuicListener` needs an internal
`trio.MemoryReceiveChannel[QuicMsgStream]` fed by a background
task-pair (one task accepting connections, one per connection
accepting bi-streams). `trio.abc.Listener` has no nursery, so:
make the listener **constructed by an `@acm`** that owns the
nursery, and have `start_listener()` be that `@acm`'s driver.
⚠️ this collides with `Endpoint.start_listener()` being a plain
`async def` returning a listener. Two options:
- **(a)** hang the nursery off the `Endpoint`'s existing
`listen_tn``_serve_ipc_eps()` already creates `listen_tn`
and passes it into every `Endpoint` (`_server.py:1063-1074`),
and `Endpoint.listen_tn` is right there. So
`start_listener()` can `self.listen_tn.start_soon(...)` the
acceptor tasks. **Recommended**: no upstream signature change,
correct lifetime (dies with the ep group), and it's why
`listen_tn` is on the struct in the first place.
- (b) change `start_listener()` to a `@acm`. Bigger blast
radius; only if (a) proves insufficient.
Since `start_listener()` is called via
`inspect.getmodule(addr)` with only `addr=` (contract §1.3),
option (a) needs the `Endpoint` itself. Either add `ep=` to the
module-level `start_listener()` call signature (all backends
ignore it except quic → small upstream change, do it as part of
the prep PR and make it keyword-only with a default) or have
`QuicListener.accept()` lazily spawn via
`trio.lowlevel.current_task().parent_nursery` (**rejected** —
fragile, implicit). Do the explicit `ep=` kwarg.
### 3.4 `maddr`
Multiaddr already standardizes the pieces:
```
/ip4/<h>/udp/<p>/quic-v1 # direct
/ip4/<h>/udp/<p>/quic-v1/p2p/<node-id> # direct + identity
/dns/<relay-host>/tcp/443/tls/ws/p2p/<node> # relay-ish
```
- primary form: `/p2p/<node-id>` alone is a legal maddr and is
the *only* required component for iroh dialling — relay +
direct addrs are discovery hints. So `mk_maddr()` emits
`/p2p/<node_id>` and, when known, prefixes the direct
`/ip4/../udp/../quic-v1/`.
- `/p2p/` values are multihash-encoded peer ids; an iroh node-id
is a raw ed25519 key. Converting requires the identity
multihash + libp2p key protobuf wrapper. **Decide**: emit the
raw node-id under a *tractor-local* `/iroh/<node-id>` segment
(needs upstream registration, same track as `wg`/`tipc`,
gh #483) rather than pretending to be a libp2p peer-id we
can't round-trip. Return the `str` form until upstream lands
(`MsgTransport.maddr` is `Multiaddr|str`).
- this backend is the strongest argument for gh #443's
**tunnelled/composed maddr** item: `/ip4/../udp/../quic-v1/..`
*is* a composed stack. Cross-reference plan 03 §5 so the two
grammars land compatibly.
---
## 4. Discovery integration
- iroh's node-id addressing means the `tractor` registrar can
hold `IrohAddress`es that are **reachable from anywhere** with
no port-forwarding — that is the headline feature. The
registrar itself works unchanged.
- iroh has its own discovery (DNS/pkarr/mdns). **Out of scope**;
note in the follow-up that `tractor.discovery` could
eventually delegate to it, which would be the direct analogue
of plan 01's TIPC-topology idea.
- relay servers: default to n0's public relays for the demo,
document self-hosting (docs.iroh.computer's dedicated-infra
page is linked from #353), and make the relay set a
`start_listener()` kwarg.
## 5. Security note
QUIC is TLS-1.3-always and iroh authenticates by node-id, so
this backend is the first `tractor` transport with real
transport security and peer authentication. Two things follow:
1. an **allowlist hook** — an actor should be able to reject
inbound connections from unknown node-ids *before* the
`Aid` handshake. Natural home: a predicate kwarg on
`start_listener()`, evaluated in `QuicListener`'s connection
acceptor task. Sketch it; ship it in PR 1 if cheap (it is).
2. do **not** claim any security property for the other
backends by association. `tcp`/`uds`/`tipc` remain
unauthenticated; that's what plan 03 (wg) is for.
## 6. Commit sequencing
0. **spike (throwaway, not committed)**: drive iroh under
`trio-asyncio`/`tractor.to_asyncio`, echo bytes over a
bi-stream between two procs. Fills in §1.1. Timebox it.
1. prep PR: annotation widening + `rebind_from_sockname` gate +
`transport_from_stream()` `tpt_key` dispatch + `ep=` kwarg on
`start_listener()` + lazy `default_lo_addrs()`. **No new
backend.** Full suite green on tcp *and* uds.
2. `_uniffi_trio.py` + its tests (drive one iroh async call
under bare `trio.run()`; assert no asyncio loop; assert
cancellation frees the future).
3. `QuicMsgStream` + tests against a *loopback* iroh endpoint
pair in one process (no `tractor` runtime): send/recv, clean
EOF → `b''`, reset → `BrokenResourceError`, use-after-close
`ClosedResourceError`.
4. `QuicListener` + `start_listener()` + `IrohAddress` +
key-file mgmt.
5. `MsgpackQuicStream(MsgpackTransport)` + `connect_to()` +
`maybe_open_context()` connection pooling.
6. registration tables + `--tpt-proto quic` + full suite.
7. maddr + docs + a two-host example (pairs with #482's format).
## 7. Testing
- capability predicate `is_quic_available()``iroh` importable
*and* the uniffi driver symbol present at the pinned version.
Same `pytest.fail`-early hook as plan 01 §7.2.
- **the acceptance bar is the same**: whole suite green under
`--tpt-proto quic`. Expect this to shake out real bugs in the
adapters (esp. teardown ordering and `TransportClosed`
classification) — that's the point.
- expect to need **timeout headroom**: iroh endpoint bind +
first connect (relay discovery) is orders of magnitude slower
than a UDS bind. Before touching any test deadline, rule out
the CPU-throttle false-positive (see the project's
`env_cpu_throttle_masquerades_as_regression` note); then, if
real, add a per-proto timeout multiplier to the test harness
rather than editing individual tests.
- a no-network test mode: iroh with relays disabled +
loopback direct addrs only, so CI doesn't depend on n0's
infra. **Make this the default in CI**; mark the relay tests
`pytest.mark.net` and keep them out of the default run.
- leak checks: assert every `SecretKey`/`Endpoint` is closed on
actor teardown (an `Endpoint` left open holds UDP sockets and
relay connections; a leak here shows up as hung tests, not
errors).
## 8. Risks
| risk | mitigation |
| --- | --- |
| uniffi codegen internals shift on upgrade | pinned minor, symbol assertion at import, the "no asyncio loop" test, documented fallback to `to_asyncio` |
| rust-thread callback → trio wakeup mishandled (segfault / lost wakeup / un-cancellable task) | strong ref on the ctypes trampoline; `run_sync_soon` only; **bounded** shielded cancel-drain; run the `conc-anal` skill over the bridge |
| `iroh` wheel availability for 3.13/3.14 on linux+macos | verify in step 0; if missing, that alone may force the `aioquic` fallback |
| QUIC latency/jitter destabilizes the existing suite's timing assumptions | per-proto timeout multiplier, relay-less CI mode |
| `(str, str)` unwrapped form collides with UDS in `wrap_address()` | guarded case ordered first + explicit regression test (§3.2) |
| scope creep into iroh's docs/blobs/gossip crates | this backend is `Endpoint`+`Connection`+bi-streams only; anything else is a separate issue |
## 9. Follow-up issue seeds
- `tractor.discovery` delegating to iroh discovery (DNS/pkarr/mdns)
- per-`Context` QUIC sub-streams: today one `Channel` == one
stream; QUIC would let each `tractor.Context` own its own
stream with independent flow-control and cancellation — this
is the genuinely novel win #353 gestures at, and it's a
runtime-layer change, not a transport one
- unreliable QUIC datagrams for a lossy-ok broadcast transport
(pairs with plan 01's TIPC-multicast seed)
- node-id allowlist → a real `tractor` authz story
- `aioquic` sans-io backend reusing §3's adapters

View File

@ -0,0 +1,757 @@
# Plan 03 — WireGuard (and other tunnels) as a *nested bindspace* via `pyroute2`
Tracks gh [#482] + the tunnelled-maddr item of [#443].
Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: WireGuard is **not** a `MsgTransport`. It is an
interface-layer tunnel that is transparent to `socket(2)`, so
the correct abstraction is a *bindspace* — a scoped,
`@acm`-managed network context that an existing L4 transport
(`tcp`, and later `quic`/`tipc`-over-UDP-bearer) binds *inside*.
This plan implements `Address.namespace` (spec'd but unused
since day one) and the composed/tunnelled maddr grammar, with
`pyroute2` as the netlink codec and as much of the I/O moved
onto `trio` as the library's sans-io layer allows.
[#482]: https://github.com/goodboy/tractor/issues/482
[#443]: https://github.com/goodboy/tractor/issues/443
---
## 1. What exists today (verified, per #482)
- `wrap_address()` accepts maddr `str`s (leading-`/` dispatch,
`_addr.py:262`). `parse_maddr()` and `mk_maddr()` support plain
TCP/UDS addresses plus nested, canonical bearer-first `/wg/`
stacks represented locally as `TunnelledAddress` wrappers.
- there is no `wg` proto in the multiaddr *spec* yet, but
multiformats/py-multiaddr#108 (key form `u<base64url>`) is
**merged** as of 2026-07-28 (`f86519da`) — and unreleased, the
latest `0.2.0` predating it. Spec registration is still tracked
by multiformats/py-multiaddr#107 and gh #483.
- **today's deployable story remains declarative**: run `wg-quick`
out-of-band, parse the maddr, strip its wrapper to the overlay
`(host, port)`, verify the pubkey against the live tunnel,
hand the overlay addr to `registry_addrs=`/`tpt_bind_addrs=`.
#482 already contains working example code for exactly this.
- `Address.namespace` exists in the Protocol
(`_addr.py:94-101`, "the if-available OS-specific network
namespace key"). `TunnelledAddress` implements it from its spec;
no concrete transport backend implements it yet.
## 2. Three layers, three PRs
| layer | what | dep | ships |
| --- | --- | --- | --- |
| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u<key>` → overlay `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first |
| **B. `pyroute2` read/verify** | replace the `subprocess.run(['sudo','wg','show'])` shelling with netlink queries | `pyroute2` extra | second |
| **C. `@acm` lifecycle** | create/configure/tear down wg ifaces + netns *from the runtime*, as nested bindspaces; implement `Address.namespace` | `pyroute2` + `CAP_NET_ADMIN` | third |
Each is independently valuable and independently reviewable.
**Do not attempt C first** — the interesting design (nested
bindspace `@acm`s) is only well-posed once A has pinned the
address grammar and B has proven the netlink path under trio.
---
## 3. Layer A — declarative `wg` maddrs
### 3.1 the address shape
The decision: **a wg segment annotates an existing address, it
does not create a new address type.** Two candidate encodings;
**pick (a)**:
- **(a) `TunnelledAddress` wrapper** (recommended):
```python
class TunnelledAddress(
msgspec.Struct,
frozen=True,
):
overlay: Address # e.g. TCPAddress
tunnel: WGTunnelSpec # proto-specific, frozen
```
with `.proto_key` **delegating to `overlay.proto_key`** so every
existing table lookup (`_addr_to_transport`,
`enable_transports` guard at `_root.py:391`,
`transport_from_addr()`) keeps working untouched, and
`.unwrap()` delegating to `overlay.unwrap()` so **nothing new
crosses the wire**. `.namespace` and `.bindspace` come from
the tunnel spec. The wrapper is stripped (`→ .overlay`) at the
moment of bind/connect.
- ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests
`type(addr) in _address_types.values()` — a `bidict` of
proto_key→type. `TunnelledAddress` isn't in it and must not
be (it's not 1:1 with a proto). So either add an explicit
`isinstance(addr, TunnelledAddress)` clause there, or give
the wrapper a marker and test structurally. Do the former;
it's two lines and honest.
- the reflection in `Endpoint.start_listener()`
(`inspect.getmodule(self.addr)`) would resolve to the
*wrapper's* module, not the transport's. **So the wrapper
must be unwrapped before it reaches `Endpoint`** — i.e. by
the bindspace `@acm` (layer C) or by `parse_maddr()`
(layer A). State this loudly in the docstring; it's the #1
way to get this wrong.
- (b) add fields to each existing `Address` type. Rejected:
duplicates tunnel logic per-backend and pollutes `.unwrap()`.
```python
class WGTunnelSpec(
msgspec.Struct,
frozen=True,
):
peer_pubkey: str # std-base64 `wg(8)` form
iface: str = 'wg0'
netns: str|None = None
# layer-C-only fields, unset in layer A
maybe_endpoint: tuple[str, int]|None = None
maybe_allowed_ips: tuple[str, ...] = ()
```
### 3.2 `parse_maddr()`/`mk_maddr()`
Grammar — **verified** against py-multiaddr#108, first on the
`baudco/py-multiaddr@wg_support` branch and re-verified after it
merged upstream (`multiformats/py-multiaddr@f86519da`); all three
forms below parse *and* round-trip. Note the codec also validates
that the key decodes to exactly 32 bytes, so a truncated key is a
`StringParseError`, not a silently-mangled parse:
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\_______ bearer __________/\__ key __/\______ overlay ______/
underlay, wg `ListenPort` the `MsgTransport` bind
```
The `/wg/` segment is **infix, not suffix** — the segments
*before* it are the wg **bearer** (the underlay `(ip, udp-port)`
that `wg(8)` itself listens on, per the codec docstring's own
`/ip4/1.2.3.4/udp/51820/wg/{key}` example), and the segments
*after* are the **overlay** endpoint that `tractor` binds.
⚠️ **CORRECTION** — an earlier revision of this plan (and the
examples in gh #482) used a *suffix* form
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. That parses, but it is
semantically inverted: it puts the overlay addr where the bearer
belongs, `tcp` where wg's `udp` `ListenPort` goes, and declares
no overlay endpoint at all. `parse_wg_maddr()` in
`examples/multihost/wg_lan/` now rejects it with an actionable
error.
Observed protocol-name lists, for writing the `match`:
| maddr | `[p.name for p in m.protocols()]` |
| --- | --- |
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
- so the three parts have **three different owners**, and only the
third is an `Endpoint`:
| part | socket owner / provisioner | runtime role |
| --- | --- | --- |
| bearer | kernel-owned; externally provisioned in layer A, tractor bindspace-provisioned in layer C | control-plane metadata, never an `Endpoint` |
| `/wg/u<key>` | nothing — it's an identity | parsed and explicitly verified |
| overlay | `tractor`'s `IPCServer` | application `MsgTransport`, as `.overlay` |
This owner-split is the real axis of the design, *not* whether
the maddr stack is "composed" (it is).
- ⚠️ **CORRECTION**, an earlier draft of this section specced a
hand-rolled `_peel_tunnel_segs(proto_names) -> (bearer_names,
tunnel_specs, overlay_names)`. **Do not write it.**
`py-multiaddr` already ships the whole tunnel compose/peel API
and it was simply missed here — see its README "En/decapsulate"
and "Tunneling" sections, and gh #443's 2nd bullet which links
them. Verified against the pinned rev:
| need | API |
| --- | --- |
| isolate the bearer | `ma.decapsulate_code(P_WG)` |
| drop the overlay, keep bearer+key | `ma.decapsulate(overlay_ma)` |
| per-seg maddrs | `ma.split()` |
| rejoin a seg tail | `Multiaddr.join(*segs)` |
| read the key | `ma.value_for_protocol('wg')` |
| recompose | `bearer.encapsulate(key).encapsulate(overlay)` |
`.decapsulate_code()` handles the infix `/wg/` seg cleanly
*because* it cuts on proto-code and never tries to match an
addr value — the key seg has no addr of its own. This is the
same NIH trap gh #429 existed to close, one layer up.
- ⚠️ `value_for_protocol('ip4')` on a *full* tunnelled maddr
silently returns the **first** match, i.e. the bearer's host.
Always call it on a peeled sub-maddr, never the whole stack.
- `parse_maddr()` gains a case on
`[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), <overlay-l4>]`
peel w/ the API above, decode the multibase key to std-base64,
and return `TunnelledAddress(overlay=..., tunnel=WGTunnelSpec(
...))` w/ the bearer recorded in the spec.
- keep the existing 2-proto cases byte-identical; add the new
case *after* them.
- nesting (wg-in-wg) falls out of `.decapsulate_code()` cutting
at the *last* occurrence — peel repeatedly rather than
recursing through a bespoke splitter.
- `mk_maddr()` inverse for `TunnelledAddress` is just
`.encapsulate()` composition; don't rebuild `str`s by hand.
- **pending an upstream release**: py-multiaddr#108 is merged, so
`Multiaddr('/…/wg/u…')` parses off a PEP 621 direct-revision pin,
since no release carries the codec. Gate parser entry on
`_wg_proto_code()`, implemented as
`protocols.protocol_with_name('wg')` under
`except ProtocolNotFoundError`. Do **not** probe by parsing a
dummy like `Multiaddr('/wg/uAAAA')` — the codec enforces a
32-byte key, so that raises even when the proto *is* known. Do
**not** hand-roll a `wg` parser in `tractor` — the whole point
of #429 was dropping the NIH parser.
### 3.3 pure codecs + explicit verification
Port #482 §2's pure helpers into
`tractor/discovery/_tunnel.py`, keeping the impure probe cleanly
separated until layer B:
```python
def parse_wg_maddr(maddr: str) -> TunnelledAddress: ... # pure
def wg8_pubkey(multibase_key: str) -> str: ... # pure
async def verify_wg_peer(spec: WGTunnelSpec) -> bool: ... # layer B
```
Layer A's example-local `verify_wg_peer()` may shell out (`wg show
<if> peers`), but layer B replaces that probe with one explicit async
function backed by pyroute2. Never call it implicitly from
`wrap_address()`/`parse_maddr()` — parsing must stay pure and
side-effect-free; verification is the *caller's* explicit step
(and later, the bindspace `@acm`'s).
### 3.4 deliverables
- `examples/` scripts distilled from #482 §§3-5 (this is the
unchecked "commit examples from ^" bullet in #443). They live
under `examples/multihost/``test_docs_examples.py` walks
`examples/` recursively and runs every collected file as a
subproc asserting `rc == 0` (it doesn't even filter by
extension, so a stray `README.md` would be `python`-run too),
and `'multihost' not in p[0]` is already in its exclusion
list. Anything needing a real second host or a live tunnel
belongs there.
- a `docs/` page: tunnel setup, the maddr form, the two-host
run. Keep prose in the docs; keep the examples runnable and
minimal.
- tests: maddr round-trip, `TunnelledAddress` delegation
(`proto_key`/`unwrap` identical to overlay), `wrap_address()`
regression (a tunnelled maddr `str``TunnelledAddress`; a
plain one → unchanged), and **a real end-to-end over a
locally-created wg pair** gated on `CAP_NET_ADMIN` (see §5.4).
---
## 4. Layer B — `pyroute2` under `trio`
### 4.1 the library situation (verify at implementation time)
`pyroute2` ≥0.9 rewrote its core onto **asyncio**
(`AsyncIPRoute`; the sync `IPRoute` wraps it with its own loop).
It also ships a `WireGuard` netlink (generic-netlink) module
supporting `.set(iface, private_key=..., peer={...})` and
`.info(iface)`, plus `pyroute2.netns` / `NetNS` for namespaces,
and `IPRoute.link('add', kind='wireguard', ifname=...)`.
Three integration options, in increasing trio-nativeness:
- **(1) `trio.to_thread.run_sync()` around the sync API.**
Netlink ops here are one-shot, sub-millisecond, and happen at
bind/teardown time only — *not* in the msg hot path. This is
the **correct default**: it's ~10 lines, uses a battle-tested
API, and costs nothing where it's used.
- **(2) sans-io: `trio.socket` + pyroute2's message codecs.**
`pyroute2`'s message classes
(`pyroute2.netlink.rtnl.*`, `pyroute2.netlink.generic.wireguard.wgmsg`)
encode/decode independently of its I/O core. So a
`tractor/ipc/_netlink.py` with a small trio `NetlinkSocket`
(`trio.socket.socket(AF_NETLINK, SOCK_RAW|SOCK_DGRAM, proto)`,
`sendto`/`recv`, seq/pid matching, `NLMSG_DONE`/`NLMSG_ERROR`
handling) + pyroute2 codecs is very achievable and is the
honest reading of "as much trio wrapping as possible where any
other async support can be replaced".
**Do this for the paths we actually need** (link add/del,
addr add, wg get/set, netns bind) and *only* those — a
general netlink client is out of scope.
- (3) reimplement the codecs. Never.
**Recommended split**: ship (1) first so layer B is a small,
reviewable, behaviour-preserving swap of `verify_wg_peer()`'s
body; then land (2) as a follow-up commit for the read path
(`wg get`, `link get`) where the sans-io surface is smallest,
and keep (1) for the privileged mutating ops. Measure before
converting anything else — there is no perf argument here, only
a "no foreign event loop in a trio actor" argument, which (1)
already satisfies (a thread is not an event loop).
Explicitly **do not** pull in `trio-asyncio` for pyroute2 or infect
every wg-using actor merely to service one-shot netlink calls. A
dedicated `wgman` actor (§5.1) is the one plausible asyncio-hosted
shape: it can use tractor's own `.to_asyncio` task linkage while
keeping the foreign loop and provisioning authority out of ordinary
actor processes.
### 4.2 API shape
Pure-ish, functional, `@acm` for anything with teardown:
```python
async def read_wg_peers(
iface: str = 'wg0',
netns: str|None = None,
) -> tuple[str, ...]: ... # base64 pubkeys
async def read_wg_pubkey(iface: str = 'wg0', ...) -> str: ...
```
and `verify_wg_peer()` becomes a thin composition over one shared key
snapshot.
Note the pure-getter rule: no `read_wg_peers(..., create=True)`.
---
## 5. Layer C — nested bindspace `@acm`s + `Address.namespace`
This is the part #443 and `multiaddr_declare_eps.md` actually
ask for: *"for any tunneled maddr-`str`-entry we deliver a
data-structure which can easily be passed to nested `@acm`s
which consecutively setup nested net bindspaces for binding the
endpoint addrs"*.
Layer C is where tractor takes ownership of bindspace orchestration.
For a fully bootstrapped deployment it may create the netns and wg
iface, configure peers/routes, and ask the kernel to establish the
bearer's UDP `ListenPort` through netlink/`pyroute2`. "Kernel-owned"
describes the data-plane socket, not who provisions it: tractor owns
the lifecycle while `Endpoint`/`MsgTransport` remain responsible only
for the overlay application socket.
### 5.1 candidate default: first-child `wgman`
For a WG-enabled deployment profile, consider eagerly spawning one
private **WireGuard manager** (`wgman`) as the root actor's logical
first child. It is a narrow network-control-plane service, not a
general worker and not an application-visible transport endpoint.
The naive profile gets one manager for the actor tree; advanced
deployments may disable it for pre-provisioned networking or place
one manager in each capability/bindspace security domain.
"First child" describes supervision and teardown ordering, not a
serial startup barrier. Submit the `wgman` spawn in the same startup
wave as ordinary children, start its pyroute2 import, generic-netlink
discovery and declared-tunnel reconciliation immediately, and publish
a readiness signal separately. Sibling processes can boot in parallel;
only their first WG-dependent bind/dial waits for manager readiness.
This overlaps setup with actor-tree startup and avoids every sibling
paying its own pyroute2/loop/socket initialization latency.
The initial manager can still call the sync helpers from §4.1. A
natural follow-up is to spawn it with `infect_asyncio=True` and keep
`AsyncWireGuard` clients alive on asyncio's host loop through
`tractor.to_asyncio.run_task()`. Tractor then owns cross-loop task
linkage, cancellation and error propagation, while normal siblings
remain plain Trio actors. Keep one client per realized namespace or
other kernel control domain; do not share a pyroute2 socket across
domains merely to reduce object count.
Keep the authority surface deliberately small:
- accept structured inspect/verify/ensure/release requests derived
from `WGTunnelSpec`, `BindspaceSpec` and explicit `role`; never
expose arbitrary pyroute2 calls, shell commands or `setns()` RPC;
- let the root/supervisor mediate access initially, or hand siblings
a scoped manager capability; do not register a privileged `wgman`
endpoint for unrestricted cluster-wide discovery;
- never return private keys or namespace FDs to application actors;
pass secrets and live capabilities into the manager through the
supervisor-owned bootstrap path;
- grant only the capabilities required for the manager's assigned
domain. Prefer a manager already placed in that user/net namespace
over one process holding ambient authority across every namespace;
- make ensure/release idempotent and reference-count ownership so one
sibling cannot tear down a tunnel still borrowed by another.
The root owns the manager's lifetime. `wgman` must outlive all
siblings borrowing its tunnels and exit before the root drops the
underlying namespace capabilities. A manager crash fails closed:
dependent operations receive an explicit service error; restart, if
enabled, reconciles declared state idempotently before advertising
readiness again. Do not silently let siblings fall back to privileged
local provisioning, since that defeats both the security boundary and
the single warm control-plane benefit.
Treat eager `wgman` as a measured deployment-profile choice. Compare
root startup with no WG declarations, pre-provisioned read-only WG,
and runtime-managed tunnels before making it unconditional whenever
the `wg` extra is installed. The intended invariant is "one warm
manager per simple WG actor tree", not "every tractor program spawns
a privileged child".
### 5.2 the composition
The maddr describes the composed network path and can be used as
either a source/listen or destination/dial handle. It does **not**
select the local instance of that network stack. A netns, VRF,
interface, user namespace, or equivalent platform resource is
orthogonal augmentation carried alongside/below the maddr.
Keep three bindspace representations with deliberately different roles
and lifetimes:
```python
class BindspaceSpec(msgspec.Struct, frozen=True):
'''Serializable spawn/config declaration.'''
kind: str # `netns`, later `vrf`, ...
key: str|None # requested name/key, if any
lifecycle: Literal['attach', 'open']
class BindspaceRef(msgspec.Struct, frozen=True):
'''Wire-safe, non-owning ref to the realized resource.'''
kind: str
key: str|None # mutable name, absent after unlink
inode: int # host-local Linux nsfs fingerprint
class Bindspace(ProcessLocal):
'''Scoped, non-serializable capability for one live bindspace.'''
spec: BindspaceSpec
ref: BindspaceRef
namespace_fd: int|None
ownership: Literal['owned', 'borrowed']
@acm
async def open_bindspace(
spec: BindspaceSpec,
) -> AsyncGenerator[Bindspace, None]:
'''
Provision/borrow one bindspace and yield its live capability.
'''
```
The initial model limits `BindspaceKind` to `netns` while preserving
the required role split. `BindspaceSpec` is the requested resource and
lifecycle policy. `BindspaceRef` is a serializable, non-owning,
host-local record of the resource that was actually opened; it can be
compared or logged, but cannot reopen, pin or enter that resource.
`Bindspace` is the live capability and uses msgspec's generic struct
storage by inheriting the global `tractor.msg.ProcessLocal` marker. Its
hidden unsupported sentinel blocks direct and nested default msgspec
encoding without a recursive IPC hot-path scan. The live bindspace
validates any supplied FD against `BindspaceRef.inode`; explicit FD
transfer belongs to the supervisor bootstrap path. An FD avoids
name-resolution TOCTOU,
survives rename/unlink, and identifies the exact namespace the parent
provisioned. Extend the kind/field union only when a second platform
resource is implemented.
`BindspaceSpec.lifecycle` is explicit serialized policy:
`'attach'` borrows an existing resource and `'open'` creates/owns one.
`open_bindspace()` dispatches that policy by bindspace kind. Never
infer it from a listen/dial role: either role may use pre-provisioned
or locally owned networking.
The first lifecycle implementation is deliberately borrow-only:
`attach_netns()` opens either `/proc/self/ns/net` when
`BindspaceSpec.key = CURRENT_NETNS`, or a named entry beneath
`/var/run/netns`. It derives a `BindspaceRef` from the opened FD, yields
`ownership='borrowed'`, and closes only that FD on exit. "Attach" does
not call `setns()`; it never creates, enters or removes a namespace.
Future `open_netns()` creation and owned teardown remain a separate
privileged supervisor change.
`open_netns()` is that owned counterpart: it requires a named spec,
creates through pyroute2 in a shielded worker call, attaches the live
FD, and yields `ownership='owned'`. FD closure precedes another
shielded pyroute2 removal call on every post-creation exit, including
cancellation. It still never calls `setns()`; process entry remains a
spawn/bootstrap operation.
`open_bindspace()` is **not** an address factory and does not return a
`TunnelledAddress`. At the declaration layer, listener allocation can
use the live bindspace to replace an overlay while preserving every
tunnel:
```python
async with open_bindspace(
bindspace_spec,
) as bindspace:
listen_decl = declared_addr.get_random(
bindspace=bindspace,
)
transport_addr = strip_tunnels(listen_decl)
```
That sketch intentionally leaves the `.get_random()`/bindspace value
contract open. A concrete transport call returns a concrete overlay;
a declaration-level call may replace the overlay and return a new
`TunnelledAddress`. In either case wrappers remain until the final
transport bind/dial boundary, where `strip_tunnels()` is mandatory.
At the listener boundary, keep the split explicit:
`Endpoint.addr` is the peeled concrete address used for transport
reflection, while `Endpoint.declared_addr` retains the original
wrapper for namespace diagnostics and later bindspace orchestration.
Per-platform provisioning still composes one resource context per
tunnel/bindspace layer:
```python
@acm
async def open_netns(
spec: BindspaceSpec,
) -> AsyncGenerator[Bindspace, None]: ...
@acm
async def open_wg_iface(
spec: WGTunnelSpec,
config: WGInterfaceConfig,
bindspace: Bindspace,
role: Literal['listen', 'dial'],
) -> AsyncGenerator[WGTunnelSpec, None]: ...
```
`WGInterfaceConfig` and each `WGPeerConfig` are process-local and
rejected by the global `ProcessLocal` wire guard. The interface config
owns its private key, local addresses and listen port; each peer owns
its public key, allowed CIDRs, optional endpoint, preshared key and
keepalive. Reprs redact private/preshared keys. `WGTunnelSpec` remains
serializable public maddr-derived identity/endpoint data. This split
supports multi-peer listeners without overloading the tunnel maddr.
The initial `open_wg_iface()` lifecycle is owned and Linux-only. It
validates role-dependent bearer policy before side effects, creates the
iface and addresses through `IPRoute`, configures keys/peers through
`WireGuard`, raises the link, and removes it on every post-creation
exit. Creation/removal run in shielded Trio worker calls. A listen
bearer supplies the local listen port; a dial bearer supplies an
omitted endpoint only for the selected maddr peer.
The composition driver folds a list of specs into nested contexts with
`contextlib.AsyncExitStack` for the N-deep case. The
`parse_endpoints()` API (`_multiaddr.py:153`) is the front door:
it already returns
`dict[name, list[Address|TunnelledAddress]]` and the
`multiaddr_declare_eps.md` sketch anticipates the recursive
`dict[str, list[Address]]|dict[...]` return for tunnelled
entries. Extend it to carry the tunnel stack, not to *enter* it.
`open_wg_bindspace()` is the initial driver for one bindspace and an
ordered sequence of `(WGTunnelSpec, WGInterfaceConfig)` layers. It
opens the bindspace first, enters WG interfaces outermost-first through
`AsyncExitStack`, and yields the live `Bindspace` for endpoint
allocation. Exit is inside-out, so every interface is removed while the
namespace FD remains pinned; only then can an owned namespace be
removed. Endpoint/channel lifetimes belong inside the yielded scope.
The caller supplies `role` to tunnel-resource contexts such as
`open_wg_iface()`; do not infer it from maddr shape. Bindspace
lifecycle remains the independent explicit policy above. The same
composed maddr can name a server source or client destination (§5.4).
### 5.3 `Address.namespace`, at last
- an unrealized `TunnelledAddress.namespace` reports its declared name
as `(kind, key)`, e.g. `('netns', 'tractor-wg0')`;
- `TunnelledAddress.with_bindspace_ref()` returns a frozen declaration
annotated with `bindspace.ref`, never the FD-bearing `Bindspace`.
Its `.namespace` reports `(kind, inode)` so the
realized ref remains stable across rename or unlink;
- existing plain backends implement it explicitly as `None`, so the
Protocol does not lie and tunnel delegation needs no `getattr()`
fallback.
- `Endpoint.namespace` reads the retained declaration rather than its
peeled transport addr; both `Endpoint.pformat()` and
`Server.pformat()` expose that value as the cheapest proof the layer
is wired.
Deferred follow-ups:
- add native tagged encoding for the complete `TunnelledAddress` graph,
including its concrete overlay-address union, tunnel-spec union and
optional `BindspaceRef`. Once that codec exists, tests should perform
typed roundtrips instead of inspecting an untyped decoded payload.
Use `github/ns_aware@e4688cad` as prototype evidence, not code to
cherry-pick unchanged. Its `/proc/<pid>/ns/<type>` inode reader and
`ip netns identify` probe establish the useful `(key, inode)` reference
record. Layer C should move that shape into `BindspaceRef`, avoid a
subprocess where netlink/procfs suffices, and hold the namespace FD in
`Bindspace` to pin the referenced resource.
### 5.4 the netns/process reality — read this before designing
**The headline consequence, stated up front**: netns is a
**runtime-level config API, not an actor-app-code API.** It is
declared as part of how an actor process is *brought up* — a
spawn-time/boot-time input alongside `enable_transports` and
`tpt_bind_addrs` — and it is **not** dynamically re-enterable by
app code once the actor is live. There is deliberately no
`await actor.enter_netns(...)`. Two hard reasons, both below:
`setns(2)` doesn't retroactively move existing sockets, and it's
per-thread rather than per-process. Anything that *looks* like a
mid-life API here would be a footgun that silently leaves the IPC
server bound in the old namespace.
- `setns(2)` with `CLONE_NEWNET` affects **the calling thread
only**, and sockets already created keep their original netns.
A trio actor is effectively single-threaded for our purposes,
so "enter the netns, *then* bind" works — but any
`to_thread` worker (§4.1 option 1!) is in the **original**
netns unless it also `setns`. Concretely: a wg query issued
via `trio.to_thread` will hit the wrong namespace. Either
pass `netns=` down to `pyroute2` (which does the
fork/setns dance itself) or pin a dedicated worker. **This is
the single subtlest bug in this plan — write the test first.**
- entering a netns is *process-global-ish and irreversible-ish*
in practice. Therefore: **netns membership belongs to the
actor process, decided before the runtime binds**, not to a
mid-life actor API. Design:
- the root/parent decides the `BindspaceSpec`, provisions or
borrows it, and passes the spec plus an inherited/transferred
namespace-FD capability through the spawn backend (there's already
`enable_transports`/`accept_addrs` plumbing at
`_runtime.py:1595-1615` — the netns rides alongside).
- the child spawn/bootstrap trampoline calls `setns()` **before**
`_runtime.async_main()`, `IPCServer.listen_on()`, parent-channel
connection, or creation of any worker thread/socket.
- `spawn._netns.enter_netns()` is the first private bootstrap
primitive: it checks the inherited FD against the expected inode,
calls `setns(fd, CLONE_NEWNET)`, and verifies
`/proc/self/ns/net` before returning. It deliberately does not own
or close the FD; spawn propagation and status reporting remain the
caller's next integration boundary.
- only after successful entry does the child drop namespace-entry
privileges and initialize the actor runtime.
- a root/single-actor process follows the same ordering: enter during
root bootstrap, never after actor runtime startup.
- iface/route/WG provisioning is genuinely scoped and remains under
the parent/supervisor's `Bindspace` context.
- document the constraint rather than hiding it; a
`RuntimeError` if namespace entry is attempted after bootstrap.
- capabilities: iface/netns creation/config needs `CAP_NET_ADMIN`;
entering an existing Linux namespace normally requires
`CAP_SYS_ADMIN` in the owning user namespace. Never `sudo` from
inside the runtime. A privileged parent/helper should provision the
stack and open the namespace FD; the child receives only the scoped
capability and temporary authority needed to enter it, then drops
that authority before actor code runs. This separates create/config
authority from enter/use authority and fits user-namespace/capability
deployments without granting every actor broad ambient caps.
Two supported modes remain:
(i) pre-provisioned out-of-band (layers A/B — the default,
and what #482 documents), (ii) runtime-managed when the supervising
process/helper holds the required caps. Probe exact required caps and
*fail loudly with an actionable message* otherwise.
- role semantics are explicit:
- `listen`: may create/own the local bindspace, iface, routes, WG
peer/listener state, and random local overlay; lifetime normally
extends through all listeners and the actor process.
- `dial`: may borrow an actor-wide bindspace or ensure local routing
and tunnel state reaches the remote stack; it does not own the
remote maddr and may need no new local resource at all.
- source/destination use is an operation property, never permanently
encoded into the maddr or inferred from segment ordering.
- teardown follows capability ownership, not just address type:
- owned listener bindspaces tear down after endpoints/channels and
the actor process have exited;
- borrowed dial/actor-wide bindspaces only release their capability;
- nested resources exit inside-out, but shared resources remain until
their owning supervisor drops the final capability.
- teardown must be idempotent and tolerant: an iface/netns
already gone must not strand the rest of the teardown — the
exact lesson `_uds.close_listener()`'s `FileNotFoundError`
tolerance and `_serve_ipc_eps()`'s per-ep `try/except`
encode. Mirror both.
### 5.5 tests for layer C
- unit: fold-N-tunnel-specs-into-nested-`@acm`s, with fakes; assert
enter/exit ordering (outermost-last-out) via a trace list.
- integration, gated on `CAP_NET_ADMIN` (skip otherwise, and in
CI run it in a `--cap-add NET_ADMIN` container job): create two
netns + a wg pair entirely in-process, boot a `tractor` root in
one and a subactor in the other, `find_actor()` across the
tunnel. This is a *fantastic* test to have and is fully
self-contained — no second host, no `sudo` in the test body.
- the `to_thread`-netns-mismatch regression from §5.4, written
**first** (red), then the fix (green), per project convention.
- bootstrap ordering: assert the child reports the expected namespace
inode before parent-channel connect and listener creation.
- FD capability: rename/unlink the namespace name after opening its FD
and prove child entry still selects the pinned inode.
- privilege drop: prove actor code lacks provisioning caps after entry.
- role/ownership: fake listen/dial resources and assert owned listener
teardown versus borrowed dial-handle release.
- `wgman` bootstrap: prove sibling process startup overlaps manager
reconciliation while the first WG operation still waits for its
readiness signal.
- `wgman` authority: reject arbitrary callers/operations and prove an
unprivileged sibling cannot receive secrets, FDs or provisioning
authority through the manager API.
- `wgman` lifetime: prove it outlives tunnel borrowers, fails pending
requests explicitly on crash and reconciles before restart-ready.
---
## 6. "Other shuttle-able tpts"
The generalization the #482 follow-up gestures at: once
`TunnelledAddress` + `open_bindspace()` exist, the same
machinery covers any iface-layer tunnel `pyroute2` can drive —
`ipip`/`gre`/`sit`/`vxlan`/`geneve`/`bridge`/`veth`. Keep
`WGTunnelSpec` as *one* frozen struct among a
`TunnelSpec = WGTunnelSpec|VxlanTunnelSpec|...` union with a
`kind: ClassVar[str]`, and dispatch `open_*` by `match` on it.
Design for it now (union + `match`), implement only `wg` +
`netns`. `veth`-pairs-in-netns is the natural second one because
it makes the §5.5 integration test possible without wg at all —
consider doing it *first* for exactly that reason.
## 7. Non-goals
- no wg userspace implementation, no key exchange, no
`wg-quick` reimplementation (config-file parsing is
out of scope; take structured input).
- no persistence of private keys beyond what layer C's iface
creation needs (and that stays in `get_rt_dir()`, 0600).
- macOS/Windows: layers B/C are Linux-only. Layer A (declarative)
works anywhere `wg` does. Gate accordingly and say so in the
docs — do not silently no-op.
## 8. Risks
| risk | mitigation |
| --- | --- |
| `to_thread` worker runs in the wrong netns | §5.4; pass `netns=` to pyroute2 or pin a worker; test-first |
| namespace name is renamed/replaced between provision and spawn | pass an open namespace FD; verify `(key, inode)` after child entry |
| child starts sockets/threads before `setns()` | enter in the spawn bootstrap trampoline before `_runtime.async_main()`; assert inode ordering |
| ambient capabilities leak into actor app code | split provision/enter authority and drop caps before runtime initialization |
| dial path tears down a shared actor bindspace | encode ownership in `Bindspace`; borrowed bindspaces never remove resources |
| py-multiaddr#108 merged but unreleased | PEP 621 direct-revision pin + `_wg_proto_code()` gate; replace with a release floor once published |
| `TunnelledAddress` leaks into transport reflection/type dispatch | keep wrappers through declaration/bindspace handling, call `strip_tunnels()` at channel/endpoint boundaries, and retain the boundary regressions |
| privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default |
| pyroute2 0.9 asyncio core drags a loop into every actor | use a worker for one-shots; confine persistent asyncio to an infected `wgman` (§4.1, §5.1) |
| eager `wgman` serializes or slows root bootstrap | spawn it in parallel; gate only WG-dependent operations on readiness; measure before making the profile unconditional |
| `wgman` becomes a cluster-wide privilege oracle | keep it private/scoped, expose structured verbs only and split managers by capability domain |
| netns teardown strands actor teardown | idempotent/tolerant teardown mirroring `_uds.close_listener()` |
## 9. Follow-up issue seeds
- `veth`-in-netns bindspace (unblocks capless-ish integration
testing, and is a great local multi-"host" test rig)
- composed/tunnelled maddr grammar shared with plan 02's
`/…/quic-v1/…` stacks (gh #443)
- `wg` proto into the multiaddr **spec** (gh #483), then flip
`MsgTransport.maddr` to always return `Multiaddr` (the third
#443 bullet)
- first-child `wgman` prototype: concurrent bootstrap, scoped sibling
access, infected-asyncio pyroute2 ownership and restart reconciliation
- runtime-managed wg key rotation / peer add-remove through `wgman`
the natural "actor that owns the network" demo

View File

@ -0,0 +1,54 @@
# next-gen `tractor.ipc` transport backend plans
Implementation specs for three prospective `.ipc` transport
backends, written so each can be worked independently (by a
different model/provider) without design or lib-selection drift.
**Read [`00_shared_backend_contract.md`](./00_shared_backend_contract.md)
first** — it is the normative description of what a `tractor`
transport backend *is* as of `main@83b34884` (the backend
duck-type, the 10-item registration checklist, the test-harness
plumbing, the code-style rules). The three plans assume it and
document only their own deltas.
| plan | issue | dep | size | lands |
| --- | --- | --- | --- | --- |
| [01 — TIPC](./01_tipc_backend.md) | [#378] | **none** (stdlib) | small | first |
| [02 — QUIC/`iroh`](./02_quic_iroh_backend.md) | [#353] | `iroh` (uniffi FFI) | large | needs a prep PR |
| [03 — `wg` bindspace](./03_wg_tunnel_bindspace.md) | [#482], [#443] | `pyroute2` | medium, 3 layers | layer A now |
Headline conclusions:
- **TIPC is the cheap win.** Verified: `trio.SocketStream` and
`trio.SocketListener` are address-family agnostic (only
`SOCK_STREAM` + a trio socket), and CPython ships `AF_TIPC` +
23 `TIPC_*` constants. So the backend is ~one module of
contract boilerplate, zero new deps, and it buys
*kernel-native* service discovery: `bind()` publishes,
`connect()`-by-name resolves — no registrar in the loop.
(`modprobe tipc` is required; hard-gate everything.)
- **QUIC's cost is entirely in two adapters**, not in QUIC. The
`iroh` python bindings are `uniffi`-generated asyncio, but the
asyncio dependency is confined to *one* future-poll callback —
a ~40-line `trio` bridge (`TrioToken.run_sync_soon`) replaces
it. The second cost is that an iroh listener isn't a socket,
which needs a small, independently-reviewable prep PR to
`_server.py`/`_types.py`.
- **WireGuard is not a transport.** It's an iface-layer tunnel,
so it belongs as a *nested bindspace* (`TunnelledAddress` +
`open_bindspace()` `@acm`s) wrapping whatever L4 tpt is in
use — which is also what finally implements the long-spec'd
`Address.namespace`, and what generalizes to
`veth`/`vxlan`/`gre`.
Ordering rationale: plan 01 first as the cheap proof the
table-registration story generalizes to a genuinely new proto;
plan 03 layer A is already deployable-today doc/example work;
plan 02 last (and gated on its prep PR). Plans 01 and 02 both
want the same `Address.rebind_from_sockname` gate — whichever
lands first ships it.
[#378]: https://github.com/goodboy/tractor/issues/378
[#353]: https://github.com/goodboy/tractor/issues/353
[#482]: https://github.com/goodboy/tractor/issues/482
[#443]: https://github.com/goodboy/tractor/issues/443

View File

@ -30,9 +30,9 @@ the registry tracks the live tree as it grows and shrinks.
.. note::
Actor names are **not** enforced unique — the registry is keyed
by the full ``(name, uuid)`` pair. Name-based lookups simply
resolve to the *last* registered match, so if you boot five
actors all named ``'bob'``, you get the freshest ``'bob'`` B)
by the full ``(name, uuid)`` pair. A name lookup returns one
matching registration, but the API does not promise which match
wins. Use unique service names when selection matters.
First boot: who's the registrar?
--------------------------------
@ -65,6 +65,43 @@ the one-and-only registrar; boot then fails loudly with a
``RuntimeError`` if some other process already bound the registry
socket(s).
A dedicated registrar
---------------------
That second rule — *"if a registrar answers, boot as a plain
root"* — is all you need to run the registry as its own
**standalone process**, decoupled from any app tree's root. In the
daemon process, enter ``open_root_actor()`` with an explicit
``registry_addrs`` and ``ensure_registry=True``; the latter makes
startup fail instead of silently joining a registrar that won the
address. Point each app tree at the address that daemon actually
bound:
.. literalinclude:: ../../examples/discovery/dedicated_registrar.py
:caption: examples/discovery/dedicated_registrar.py
:language: python
The example's selector socket binds but deliberately never listens.
It owns the kernel-selected local address only long enough to read it,
then closes so Tractor's actual listener can bind the same address.
This is not a socket transfer: the close/rebind handoff is non-atomic,
so the example retries with a fresh candidate only when registrar
startup reports that another process claimed the released address.
Retries are bounded, and other startup failures remain visible. It
publishes the selected address only after the actor context enters.
It also performs the lookup inside a separate ``client`` actor. The
service is its sibling, not its child, so the client has no spawn-time
service channel to satisfy the local-peer fast path. The
``query_actor()`` assertion verifies that a registrar portal handled
the lookup before ``find_actor()`` makes the service RPC.
This is the "registrar as a subsystem, not the app root actor"
shape. Two caveats today (both tracked as #472 follow-ups):
``enable_transports`` is single-proto per runtime, so a registrar
can't yet serve multiple backends at once; and there's no way to
spawn a registrar as a *sub*-actor of a shared tree (only as its
own root), since ``start_actor()`` has no custom-``actor_cls``
hook.
Looking up actors
-----------------
@ -92,13 +129,22 @@ Knobs worth knowing:
- ``registry_addrs=[...]``: query specific (possibly multiple,
possibly remote) registrars instead of your tree's default,
- ``only_first=False``: deliver a ``list[Portal]`` of *all*
matches found across the queried registrars instead of just the
first,
- ``only_first=True``: after all configured registrars are queried
concurrently, yield the result in the first ``registry_addrs``
position. This is configured order, not first-reachable order, so
the result can be ``None`` even when a later registrar returned a
portal,
- ``raise_on_none=True``: raise a ``RuntimeError`` instead of
yielding ``None`` when no match is found — for when absence is
a hard error in your app.
- ``only_first=False``: when any query succeeds, yield an ordered
``list[Portal | None]`` with one result per ``registry_addrs``
position; misses remain ``None`` placeholders. When every query
misses, yield ``None`` instead of a list. This does not enumerate
every duplicate name in one registrar,
- ``raise_on_none=True``: raise a ``RuntimeError`` when every
registrar query returns ``None``. With ``only_first=True`` it does
not raise merely because the first ordered result is ``None`` when
a later result is a portal.
``wait_for_actor()``
********************
@ -131,10 +177,11 @@ Yields a portal straight to the registrar actor itself — or a
Fast paths and address preference
---------------------------------
Before doing any RPC to the registrar, every lookup first scans
the calling actor's *already-connected peers*: if you have a live
channel to an actor named ``name`` you get a portal over it
immediately, no registrar round-trip at all.
Before doing any RPC to the registrar, ``query_actor()``,
``wait_for_actor()``, and the default ``find_actor()`` lookup first
scan the calling actor's *already-connected peers*. If the caller
has a live channel to an actor named ``name``, it gets a portal over
that channel immediately, with no registrar round-trip.
When a registry entry holds *multiple* addresses (a multihomed
actor) the "best" one is chosen by locality:
@ -201,8 +248,8 @@ the existing registrar:
Per the bootstrap rules above, if those addrs are absent this process
becomes its own registrar root, so the same code works standalone and
as a tree-joiner. An occupied address that does not complete a Tractor
registrar handshake fails startup instead of being rebound.
as a tree-joiner. An occupied address that does not complete a
Tractor registrar handshake fails startup instead of being rebound.
"Arbiter"? A legacy naming note
-------------------------------
@ -226,10 +273,11 @@ Very naive, very honest
-----------------------
To be clear, this is a **very naive** discovery system: one
process-tree-local registrar holding a dict, no replication, no
re-election when it dies, no cross-host propagation. That's
intentional (for now); it covers the "wire up my services on this
host" case without dragging in a consensus protocol.
in-memory registrar holding a dict, no replication, no re-election
when it dies, and no automatic cross-host propagation. Separate
programs can use the same reachable registrar, as above, but must be
configured with its address. That's intentional (for now); it covers
the "wire up my services" case without a consensus protocol.
On the roadmap (issue `#216`_ tracks a chunk of it):
@ -254,6 +302,7 @@ to hear from you.
:class:`tractor.Registrar`.
.. _gossip protocol: https://en.wikipedia.org/wiki/Gossip_protocol
.. _modern protocol: https://en.wikipedia.org/wiki/Rendezvous_protocol
.. _modern protocol:
https://en.wikipedia.org/wiki/Rendezvous_protocol
.. _discovery: https://zguide.zeromq.org/docs/chapter8/#Discovery
.. _#216: https://github.com/goodboy/tractor/issues/216

View File

@ -153,14 +153,16 @@ the high-rate stream path.
never even hits the wire. (You can opt out per-call with
``ctx.started(..., validate_pld_spec=False)`` if you measure
a real cost.)
- ``Yield`` payloads are **never** checked inside
``MsgStream.send()``; they're validated receiver-side on each
``MsgStream.receive()``. A violation raises a ``MsgTypeError``
in the receiver *and* relays an ``Error`` msg back so the
offending sender gets one raised too.
- the remaining control msgs (``Start``, ``Return``) are likewise
validated such that violations raise in the **sending** actor,
pointing the traceback at the code that actually goofed.
- ``Yield`` and ``Return`` payloads are not checked before sending;
they're decoded against the dialog's spec by the receiver. A
violation raises a ``MsgTypeError`` there and terminates that
dialog. The peer then observes the resulting protocol teardown;
it is not guaranteed to receive the same ``MsgTypeError``.
- ``Start`` arguments are dispatched through the RPC endpoint's
Python signature. They are not payloads covered by the dialog's
``pld_spec``. A planned follow-up will derive a typed ``Start``
contract from endpoint annotations and validate arguments
sender-side; see `#514`_.
Anatomy of a ``MsgTypeError``
-----------------------------
@ -177,21 +179,21 @@ a msg fails to decode against the active spec. The useful bits:
``.src_uid``, ``.ipc_msg`` and the fancy ``.pformat()`` tb-box
rendering.
Practical reading guide: a *sender-side* MTE (``Started``,
``Return``) points straight at your offending ``await
ctx.started()`` or ``return`` statement, while a *receiver-side*
MTE (``Yield``) surfaces from the consumer's ``receive()`` call
with the relay copy delivered back to the producer. Either way
the failure is scoped to that one dialog; sibling contexts on the
same channel keep right on trucking.
Practical reading guide: a *sender-side* MTE for ``Started`` points
straight at the offending ``await ctx.started()`` call. A
*receiver-side* MTE for ``Yield`` or ``Return`` surfaces while the
peer decodes the payload. Either way the failure is scoped to that
one dialog; sibling contexts on the same channel keep right on
trucking.
Custom wire types: ``mk_codec()`` and friends
---------------------------------------------
msgspec covers a wide set of `builtin types`__ natively; for
anything else you teach the codec via extension hooks. The
easiest path is per-endpoint: ``@tractor.context()`` accepts
``enc_hook``/``dec_hook`` params right alongside ``pld_spec``.
For full control build and apply a codec yourself; encode-side:
anything else you teach the codec via extension hooks. The complete
public path currently available is task-scoped encoding:
``tractor.msg.mk_codec()`` builds a codec with an ``enc_hook``, and
``tractor.msg.apply_codec()`` installs it for the current task. To
build and apply that transport codec:
__ https://jcristharif.com/msgspec/supported-types.html
@ -206,8 +208,9 @@ __ https://jcristharif.com/msgspec/supported-types.html
with apply_codec(codec): # ContextVar-scoped override
... # msgs sent by this task now encode NSPs
and decode-side, scoped to an open context (note the import from
``tractor.msg._ops``, not yet re-exported):
The context manager which temporarily installs payload-decoder
settings on an open context is separate and still private (note
the ``tractor.msg._ops`` import):
.. code:: python
@ -220,11 +223,15 @@ and decode-side, scoped to an open context (note the import from
):
... # this dialog's payloads decode as NSPs
``apply_codec()`` is ``ContextVar``-scoped: it overrides the
codec for the current task (and only that task), not the whole
process. For complete working flows, including hook pairing rules
and roundtrip cases, see ``tests/msg/test_ext_types_msgspec.py``
and ``tests/msg/test_pldrx_limiting.py``.
``apply_codec()`` is ``ContextVar``-scoped: it overrides the codec
for the current task (and only that task), not the whole process.
``@tractor.context()`` accepts ``enc_hook`` and ``dec_hook``
parameters, but their runtime wiring is not yet a symmetric,
end-to-end public hook pair: the encode hook is not consumed and
the decode hook is not applied on both peers. For the working flows
and their current boundaries, see
``tests/msg/test_ext_types_msgspec.py`` and
``tests/msg/test_pldrx_limiting.py``.
The runtime dogfoods this pattern with
:class:`tractor.msg.NamespacePath`: a ``str``-subtype shaped like
@ -238,9 +245,33 @@ Toward capability-based msging
The ``pld_spec`` + codec-hook layer is the foundation for the
long-game: **capability-based msging** where each dialog's
type contract doubles as a capability grant, negotiated as part
of the protocol itself. That work is tracked in `#196`_ (with the
original typed-proto epic in `#36`_); if strongly-typed
distributed systems get you going, we'd love your input.
of the protocol itself. The epic is tracked in `#196`_ (evolving
the original typed-proto work in `#36`_), and the most recent
concrete step is `#365`_ — driving the whole ``pld_spec`` off
plain type-annotations (e.g. annotating a context's
``open_stream()`` with ``msgspec.Struct`` subtypes) instead of
explicit ``pld_spec=`` kwargs.
You don't have to wait for that, though: the decorator-level
``@tractor.context(pld_spec=...)`` shown above is already the
*higher-level* way to pin a dialog's payload contract, while
``tractor.msg._ops.limit_plds()`` is the lower-level, per-block
escape hatch. Both are exercised end-to-end in
``tests/msg/test_pldrx_limiting.py`` and
``tests/msg/test_ext_types_msgspec.py``.
The codec constructor and task-scoped override are public; the
per-dialog decoder override remains private, and the decorator hook
parameters remain incomplete. `#376`_ (from
`@guilledk <https://github.com/guilledk>`_, on the
`auto_codecs <https://github.com/goodboy/tractor/tree/auto_codecs>`_
branch) instead drafts pair-building factories which derive
matching ``enc_hook``/``dec_hook`` functions and encoder/decoder
pairs from a type spec. That automation, not hook availability,
is the proposed long-term home for custom-type (de)serialization.
If strongly-typed distributed systems get you going, we'd love
your input on any of the above.
Where to next?
--------------
@ -258,3 +289,6 @@ Where to next?
.. _(un)protocol: https://zguide.zeromq.org/docs/chapter7/#Unprotocols
.. _#196: https://github.com/goodboy/tractor/issues/196
.. _#36: https://github.com/goodboy/tractor/issues/36
.. _#365: https://github.com/goodboy/tractor/issues/365
.. _#376: https://github.com/goodboy/tractor/pull/376
.. _#514: https://github.com/goodboy/tractor/issues/514

View File

@ -1,28 +1,40 @@
import trio
import tractor
_this_module = __name__
the_line = 'Hi my name is {}'
_this_module: str = __name__
the_line: str = 'Hi my name is {}'
tractor.log.get_console_log("INFO")
tractor.log.get_console_log('INFO')
async def hi():
async def hi() -> str:
'''
Return a greeting naming the current actor.
'''
return the_line.format(tractor.current_actor().name)
async def say_hello(other_actor):
async def say_hello(other_actor: str) -> str:
'''
Ask another actor to return its greeting.
'''
portal: tractor.Portal
async with tractor.wait_for_actor(other_actor) as portal:
return await portal.run(hi)
async def main():
"""Main tractor entry point, the "master" process (for now
async def main() -> None:
'''
Main tractor entry point, the "master" process (for now
acts as the "director").
"""
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
print("Alright... Action!")
print('Alright... Action!')
# both actors wait on (then dial!) the *other*, so each
# must outlive both hellos: spawn as daemons, run the
@ -39,6 +51,10 @@ async def main():
name: str,
other_actor: str,
) -> None:
'''
Print a greeting fetched through a named actor.
'''
print(
# RPC through an existing actor's `Portal`.
await portals[name].run(
@ -47,13 +63,14 @@ async def main():
)
)
tn: trio.Nursery
async with trio.open_nursery() as tn:
tn.start_soon(run_and_print, 'donny', 'gretchen')
tn.start_soon(run_and_print, 'gretchen', 'donny')
await an.cancel()
print("CUTTTT CUUTT CUT!!! Donny!! You're supposed to say...")
print('CUTTTT CUUTT CUT!!! Donny!! You\'re supposed to say...')
if __name__ == '__main__':

View File

@ -2,14 +2,20 @@ import trio
import tractor
async def cellar_door():
async def cellar_door() -> str:
'''
Return a phrase from a spawned actor.
'''
assert not tractor.is_root_process()
return "Dang that's beautiful"
return 'Dang that\'s beautiful'
async def main():
"""The main ``tractor`` routine.
"""
async def main() -> None:
'''
The main ``tractor`` routine.
'''
# spawn a subactor, run ``cellar_door()`` as its lone task,
# block until its result arrives and the subactor is reaped.
print(

View File

@ -2,19 +2,24 @@ import trio
import tractor
async def movie_theatre_question():
"""A question asked in a dark theatre, in a tangent
async def movie_theatre_question() -> str:
'''
A question asked in a dark theatre, in a tangent
(errr, I mean different) process.
"""
'''
return 'have you ever seen a portal?'
async def main():
"""The main ``tractor`` routine.
"""
async def main() -> None:
'''
The main ``tractor`` routine.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'frank',
# enable the actor to run funcs from this current module
enable_modules=[__name__],
@ -24,9 +29,8 @@ async def main():
# call the subactor a 2nd time
print(await portal.run(movie_theatre_question))
# the async with will block here indefinitely waiting
# for our actor "frank" to complete, but since it's an
# "outlive_main" actor it will never end until cancelled
# the async with will wait indefinitely for "frank" because
# its runtime remains active until explicitly cancelled
await portal.cancel_actor()

View File

@ -1,33 +1,46 @@
from typing import AsyncIterator
from itertools import repeat
from typing import AsyncIterator
import trio
import tractor
async def stream_forever() -> AsyncIterator[int]:
async def stream_forever() -> AsyncIterator[str]:
'''
Stream the same message indefinitely.
for i in repeat("I can see these little future bubble things"):
# each yielded value is sent over the ``Channel`` to the parent actor
yield i
'''
message: str
for message in repeat(
'I can see these little future bubble things',
):
# each yielded value is sent over the ``Channel`` to the
# parent actor
yield message
await trio.sleep(0.01)
async def main():
async def main() -> None:
'''
Print messages streamed from a subactor.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'donny',
enable_modules=[__name__],
)
# this async for loop streams values from the above
# async generator running in a separate process
stream: tractor.MsgStream
async with portal.open_stream_from(stream_forever) as stream:
count = 0
async for letter in stream:
print(letter)
count: int = 0
message: str
async for message in stream:
print(message)
count += 1
if count > 50:

View File

@ -13,7 +13,11 @@ import tractor
@tractor.context
async def sleep(
ctx: tractor.Context,
):
) -> None:
'''
Start a context after a brief initialization delay.
'''
await trio.sleep(0.5)
await ctx.started()
await trio.sleep_forever()
@ -21,10 +25,13 @@ async def sleep(
async def open_ctx(
n: tractor.runtime._supervise.ActorNursery
):
) -> None:
'''
Spawn a sleeper and open a context with it.
'''
# spawn both actors
portal = await n.start_actor(
portal: tractor.Portal = await n.start_actor(
name='sleeper',
enable_modules=[__name__],
)
@ -35,8 +42,11 @@ async def open_ctx(
assert first is None
async def main():
async def main() -> None:
'''
Fail the root while a subactor context is still starting.
'''
async with tractor.open_nursery(
debug_mode=True,
loglevel='runtime',

View File

@ -1,9 +1,14 @@
from collections.abc import AsyncIterator
import tractor
import trio
async def breakpoint_forever():
"Indefinitely re-enter debugger in child actor."
async def breakpoint_forever() -> AsyncIterator[str]:
'''
Indefinitely re-enter debugger in child actor.
'''
try:
while True:
yield 'yo'
@ -15,12 +20,15 @@ async def breakpoint_forever():
raise
async def name_error():
"Raise a ``NameError``"
async def name_error() -> None:
'''
Raise a ``NameError``.
'''
getattr(doggypants) # noqa
async def main():
async def main() -> None:
'''
Test breakpoint in a streaming actor.
@ -28,8 +36,14 @@ async def main():
async with tractor.open_nursery(
debug_mode=True,
) as an:
p0 = await an.start_actor('bp_forever', enable_modules=[__name__])
p1 = await an.start_actor('name_error', enable_modules=[__name__])
p0: tractor.Portal = await an.start_actor(
'bp_forever',
enable_modules=[__name__],
)
p1: tractor.Portal = await an.start_actor(
'name_error',
enable_modules=[__name__],
)
# retreive results
async with p0.open_stream_from(breakpoint_forever) as stream:
@ -40,6 +54,7 @@ async def main():
except tractor.RemoteActorError as rae:
assert rae.boxed_type is NameError
i: str
async for i in stream:
# a second time try the failing subactor and this tie

View File

@ -4,13 +4,19 @@ import trio
import tractor
async def name_error():
"Raise a ``NameError``"
async def name_error() -> None:
'''
Raise a ``NameError``.
'''
getattr(doggypants) # noqa
async def breakpoint_forever():
"Indefinitely re-enter debugger in child actor."
async def breakpoint_forever() -> None:
'''
Indefinitely re-enter debugger in child actor.
'''
while True:
await tractor.pause()
@ -20,9 +26,13 @@ async def breakpoint_forever():
# await trio.sleep(0)
async def spawn_until(depth=0):
""""A nested nursery that triggers another ``NameError``.
"""
async def spawn_until(
depth: int = 0,
) -> None:
'''
A nested nursery that triggers another ``NameError``.
'''
async with (
tractor.open_nursery() as an,
trio.open_nursery() as tn,
@ -37,7 +47,8 @@ async def spawn_until(depth=0):
)
)
# Let the background one-shot enter `breakpoint_forever()`
# Let the background one-shot enter
# `breakpoint_forever()`
# before its sibling raises and cancellation propagates.
await trio.sleep(0.5)
# rx and propagate error from child
@ -48,9 +59,9 @@ async def spawn_until(depth=0):
)
else:
# recusrive call to spawn another process branching layer of
# the tree; blocks (up) each level until the leaf's
# `name_error` relays through.
# recusrive call to spawn another process branching
# layer of the tree; blocks (up) each level until the
# leaf's `name_error` relays through.
depth -= 1
await tractor.to_actor.run(
partial(
@ -63,25 +74,35 @@ async def spawn_until(depth=0):
# TODO: notes on the new boxed-relayed errors through proxy actors
async def main():
"""The main ``tractor`` routine.
async def main() -> None:
'''
The main ``tractor`` routine.
The process tree should look as approximately as follows when the debugger
first engages:
The process tree should look approximately as follows when the
debugger first engages:
python examples/debugging/multi_nested_subactors_bp_forever.py
python -m tractor._child --uid ('spawner1', '7eab8462 ...)
python -m tractor._child --uid ('spawn_until_3', 'afcba7a8 ...)
python -m tractor._child --uid ('spawn_until_2', 'd2433d13 ...)
python -m tractor._child --uid ('spawn_until_1', '1df589de ...)
python -m tractor._child --uid ('spawn_until_0', '3720602b ...)
python -m tractor._child --uid
('spawner1', '7eab8462 ...')
python -m tractor._child --uid
('spawn_until_3', 'afcba7a8 ...')
python -m tractor._child --uid
('spawn_until_2', 'd2433d13 ...')
python -m tractor._child --uid
('spawn_until_1', '1df589de ...')
python -m tractor._child --uid
('spawn_until_0', '3720602b ...')
python -m tractor._child --uid ('spawner0', '1d42012b ...)
python -m tractor._child --uid ('spawn_until_2', '2877e155 ...)
python -m tractor._child --uid ('spawn_until_1', '0502d786 ...)
python -m tractor._child --uid ('spawn_until_0', 'de918e6d ...)
python -m tractor._child --uid
('spawner0', '1d42012b ...')
python -m tractor._child --uid
('spawn_until_2', '2877e155 ...')
python -m tractor._child --uid
('spawn_until_1', '0502d786 ...')
python -m tractor._child --uid
('spawn_until_0', 'de918e6d ...')
"""
'''
async with (
tractor.open_nursery(
debug_mode=True,

View File

@ -7,14 +7,19 @@ import trio
import tractor
async def name_error():
"Raise a ``NameError``"
async def name_error() -> None:
'''
Raise a ``NameError``.
'''
getattr(doggypants) # noqa
async def spawn_error():
""""A nested nursery that triggers another ``NameError``.
"""
async def spawn_error() -> None:
'''
A nested nursery that triggers another ``NameError``.
'''
async with tractor.open_nursery() as an:
return await tractor.to_actor.run(
name_error,
@ -23,8 +28,9 @@ async def spawn_error():
)
async def main():
"""The main ``tractor`` routine.
async def main() -> None:
'''
The main ``tractor`` routine.
The process tree should look as approximately as follows:
@ -37,7 +43,8 @@ async def main():
- nested name_error sub-sub-actor
- root actor should then fail on assert
- program termination
"""
'''
async with (
tractor.open_nursery(
debug_mode=True,
@ -46,11 +53,11 @@ async def main():
trio.open_nursery() as tn,
):
# spawn both actors..
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'name_error',
enable_modules=[__name__],
)
portal1 = await an.start_actor(
portal1: tractor.Portal = await an.start_actor(
'spawn_error',
enable_modules=[__name__],
)

View File

@ -1,22 +1,32 @@
from collections.abc import Awaitable, Callable
import tractor
import trio
async def breakpoint_forever():
"Indefinitely re-enter debugger in child actor."
async def breakpoint_forever() -> None:
'''
Indefinitely re-enter debugger in child actor.
'''
while True:
await trio.sleep(0.1)
await tractor.pause()
async def name_error():
"Raise a ``NameError``"
async def name_error() -> None:
'''
Raise a ``NameError``.
'''
getattr(doggypants) # noqa
async def spawn_error():
""""A nested nursery that triggers another ``NameError``.
"""
async def spawn_error() -> None:
'''
A nested nursery that triggers another ``NameError``.
'''
async with tractor.open_nursery() as an:
return await tractor.to_actor.run(
name_error,
@ -25,8 +35,9 @@ async def spawn_error():
)
async def main():
"""The main ``tractor`` routine.
async def main() -> None:
'''
The main ``tractor`` routine.
The process tree should look as approximately as follows:
@ -35,7 +46,8 @@ async def main():
|-python -m tractor._child --uid ('bp_forever', '1f787a7e ...)
`-python -m tractor._child --uid ('spawn_error', '52ee14a5 ...)
`-python -m tractor._child --uid ('name_error', '3391222c ...)
"""
'''
errors: list[BaseException] = []
async with tractor.open_nursery(
@ -43,12 +55,14 @@ async def main():
# loglevel='runtime',
) as an:
async def run_and_collect(fn):
async def run_and_collect(
fn: Callable[[], Awaitable[object]],
) -> None:
'''
One-shot whose (boxed) error is stashed instead of
raised so a sibling's crash never cancels the others
before they've had their own debugger sessions (the
"collect all errors" the legacy `run_in_actor()` API
'collect all errors' the legacy `run_in_actor()` API
did implicitly at nursery teardown).
'''

View File

@ -1,19 +1,28 @@
import trio
import tractor
async def die():
async def die() -> None:
'''
Deliberately crash the calling actor.
'''
raise RuntimeError
async def main():
async def main() -> None:
'''
Crash actors with different debugger settings concurrently.
'''
async with tractor.open_nursery() as an:
debug_actor = await an.start_actor(
debug_actor: tractor.Portal = await an.start_actor(
'debugged_boi',
enable_modules=[__name__],
debug_mode=True,
)
crash_boi = await an.start_actor(
crash_boi: tractor.Portal = await an.start_actor(
'crash_boi',
enable_modules=[__name__],
# debug_mode=True,

View File

@ -5,7 +5,7 @@ import tractor
@tractor.context
async def name_error(
ctx: tractor.Context,
):
) -> None:
'''
Raise a `NameError`, catch it and enter `.post_mortem()`, then
expect the `._rpc._invoke()` crash handler to also engage.
@ -18,7 +18,7 @@ async def name_error(
raise
async def main():
async def main() -> None:
'''
Test 3 `PdbREPL` entries:
- one in the child due to manual `.post_mortem()`,
@ -49,7 +49,9 @@ async def main():
await tractor.post_mortem()
raise
else:
raise RuntimeError('IPC ctx should have remote errored!?')
raise RuntimeError(
'IPC ctx should have remote errored!?'
)
if __name__ == '__main__':

View File

@ -2,8 +2,11 @@ import trio
import tractor
async def main():
async def main() -> None:
'''
Pause in the root actor to exercise its debugger REPL.
'''
async with tractor.open_root_actor(
debug_mode=True,
):

View File

@ -2,7 +2,11 @@ import trio
import tractor
async def main():
async def main() -> None:
'''
Raise an assertion error from the debug-enabled root actor.
'''
async with tractor.open_root_actor(
debug_mode=True,
):

View File

@ -4,14 +4,21 @@ import trio
import tractor
async def name_error():
"Raise a ``NameError``"
async def name_error() -> None:
'''
Raise a ``NameError``.
'''
getattr(doggypants) # noqa
async def spawn_until(depth=0):
""""A nested nursery that triggers another ``NameError``.
"""
async def spawn_until(
depth: int = 0,
) -> None:
'''
A nested nursery that triggers another ``NameError``.
'''
async with tractor.open_nursery() as an:
if depth < 1:
await tractor.to_actor.run(name_error, an=an)
@ -27,18 +34,23 @@ async def spawn_until(depth=0):
)
async def main():
async def main() -> None:
'''
The process tree should look as approximately as follows when the
debugger first engages:
python examples/debugging/multi_nested_subactors_bp_forever.py
python -m tractor._child --uid ('spawner1', '7eab8462 ...)
python -m tractor._child --uid ('spawn_until_0', '3720602b ...)
python -m tractor._child --uid ('name_error', '505bf71d ...)
python -m tractor._child --uid
('spawner1', '7eab8462 ...')
python -m tractor._child --uid
('spawn_until_0', '3720602b ...')
python -m tractor._child --uid
('name_error', '505bf71d ...')
python -m tractor._child --uid ('spawner0', '1d42012b ...)
python -m tractor._child --uid ('name_error', '6c2733b8 ...)
python -m tractor._child --uid
('spawner0', '1d42012b ...')
python -m tractor._child --uid
('name_error', '6c2733b8 ...')
'''
async with (

View File

@ -2,7 +2,11 @@ import trio
import tractor
async def main():
async def main() -> None:
'''
Enter shielded debugging after root cancellation, then fail.
'''
async with tractor.open_root_actor(
debug_mode=True,
loglevel='cancel',
@ -18,16 +22,19 @@ async def main():
try:
await tractor.pause()
except trio.Cancelled as _taskc:
assert (root_cs := _root._root_tn.cancel_scope).cancel_called
root_cs: trio.CancelScope
assert (
root_cs := _root._root_tn.cancel_scope
).cancel_called
# NOTE^^ above logic but inside `open_root_actor()` and
# passed to the `shield=` expression is effectively what
# we're testing here!
await tractor.pause(shield=root_cs.cancel_called)
# XXX, if shield logic *is wrong* inside `open_root_actor()`'s
# crash-handler block this should never be interacted,
# instead `trio.Cancelled` would be bubbled up: the original
# BUG.
# XXX, if shield logic *is wrong* inside
# `open_root_actor()`'s crash-handler block this should never
# be interacted, instead `trio.Cancelled` would be bubbled
# up: the original BUG.
assert 0

View File

@ -2,12 +2,15 @@ import trio
import tractor
async def key_error():
"Raise a ``NameError``"
async def key_error() -> None:
'''
Raise a ``KeyError``.
'''
return {}['doggy']
async def main():
async def main() -> None:
'''
Root is fail-after-cancelled while blocking and child RPC fails
simultaneously.
@ -21,7 +24,7 @@ async def main():
trio.open_nursery() as tn,
):
# spawn the actor..
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'key_error',
enable_modules=[__name__],
)
@ -32,9 +35,9 @@ async def main():
# root blocks below.
tn.start_soon(portal.run, key_error)
# XXX: originally a bug caused by this is where root would enter
# the debugger and clobber the tty used by the repl even though
# child should have it locked.
# XXX: originally a bug caused by this is where root would
# enter the debugger and clobber the tty used by the repl
# even though child should have it locked.
with trio.fail_after(1):
await trio.Event().wait()

View File

@ -3,8 +3,15 @@ import tractor
async def cancellable_pause_loop(
task_status: trio.TaskStatus[trio.CancelScope] = trio.TASK_STATUS_IGNORED
):
task_status: trio.TaskStatus[
trio.CancelScope
] = trio.TASK_STATUS_IGNORED,
) -> None:
'''
Exercise shielded debugger pauses under cancellation.
'''
cs: trio.CancelScope
with trio.CancelScope() as cs:
task_status.started(cs)
for _ in range(3):
@ -30,7 +37,11 @@ async def cancellable_pause_loop(
await trio.lowlevel.checkpoint()
async def pm_on_cancelled():
async def pm_on_cancelled() -> None:
'''
Compare shielded and unshielded post-mortem entry.
'''
async with trio.open_nursery() as tn:
tn.cancel_scope.cancel()
try:
@ -56,7 +67,7 @@ async def pm_on_cancelled():
async def cancelled_before_pause(
):
) -> None:
'''
Verify that using a shielded pause works despite surrounding
cancellation called state in the calling task.
@ -71,7 +82,11 @@ async def cancelled_before_pause(
await pm_on_cancelled()
async def main():
async def main() -> None:
'''
Exercise shielded debugger entry in subactor and root tasks.
'''
async with tractor.open_nursery(
debug_mode=True,
) as an:

View File

@ -1,10 +1,15 @@
import platform
from collections.abc import AsyncIterator
import tractor
import trio
async def gen():
async def gen() -> AsyncIterator[str]:
'''
Yield values around debugger pauses.
'''
yield 'yo'
await tractor.pause()
yield 'yo'
@ -15,11 +20,15 @@ async def gen():
async def just_bp(
ctx: tractor.Context,
) -> None:
'''
Pause repeatedly before deliberately breaking the context.
'''
await ctx.started()
await tractor.pause()
# TODO: bps and errors in this call..
val: str
async for val in gen():
print(val)
@ -34,15 +43,18 @@ async def just_bp(
async def main():
async def main() -> None:
'''
Run the breakpoint context over a supported transport.
'''
# !TODO, parametrize the --tpt-proto={key} with osenv vars just
# like we do for loglevel/spawn-backend!
# - [ ] run on both tpts for all such debugger tests?
# - [ ] special skip for macos!
#
if platform.system() != 'Darwin':
tpt = 'uds'
tpt: str = 'uds'
else:
# XXX, precisely we can't use pytest's tmp-path generation
# for tests.. apparently because:
@ -59,7 +71,7 @@ async def main():
enable_transports=[tpt],
loglevel='devx',
) as an:
p = await an.start_actor(
p: tractor.Portal = await an.start_actor(
'bp_boi',
enable_modules=[__name__],
)

View File

@ -2,7 +2,7 @@ import trio
import tractor
async def breakpoint_forever():
async def breakpoint_forever() -> None:
'''
Indefinitely re-enter debugger in child actor.
@ -12,8 +12,11 @@ async def breakpoint_forever():
await tractor.pause()
async def main():
async def main() -> None:
'''
Run a subactor that repeatedly pauses in the debugger.
'''
async with tractor.open_nursery(
debug_mode=True,
loglevel='cancel',

View File

@ -2,11 +2,19 @@ import trio
import tractor
async def name_error():
async def name_error() -> None:
'''
Deliberately raise a ``NameError`` in a subactor.
'''
getattr(doggypants) # noqa (on purpose)
async def main():
async def main() -> None:
'''
Surface a subactor `NameError` at the waiting root task.
'''
async with tractor.open_nursery(
debug_mode=True,
) as an:

View File

@ -0,0 +1,394 @@
'''
Run a dedicated registrar in a standalone process.
The service and discovery client are sibling actors. The client has
no pre-existing channel to the service, so its lookup must use the
external registrar instead of the local-peer fast path.
'''
from __future__ import annotations
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager as acm
import errno
from pathlib import Path
import signal
import socket
import subprocess
import sys
import tempfile
import time
import trio
import tractor
MAX_BIND_ATTEMPTS: int = 5
def _is_addr_collision(exc: BaseException) -> bool:
'''
Return whether registrar startup lost the selected TCP address.
Tractor can notice the collision while probing the address or
later when its listener binds. Exception groups are retryable
only when every contained failure reports the same collision.
'''
match exc:
case BaseExceptionGroup(exceptions=exceptions):
return bool(exceptions) and all(
_is_addr_collision(child)
for child in exceptions
)
case OSError() as os_error:
return (
os_error.errno in {errno.EADDRINUSE, 10048}
or getattr(os_error, 'winerror', None) == 10048
)
case RuntimeError() as runtime_error:
message: str = str(runtime_error)
return (
'Registry address(es) are occupied' in message
or 'registry socket(s) already bound' in message
)
case _:
return False
def run_registrar(ready_path: str) -> None:
'''
Serve as the required registrar and report its selected address.
The kernel selects ephemeral loopback candidates in this process.
If another process claims a released candidate first, retry with
a fresh candidate up to `MAX_BIND_ATTEMPTS`. Other startup errors
and the final collision remain visible. `ensure_registry=True`
prevents silently joining a registrar that won the address.
'''
ready_file: Path = Path(ready_path)
async def serve() -> None:
'''
Open the registrar, publish readiness, and serve forever.
'''
for attempt in range(1, MAX_BIND_ATTEMPTS + 1):
# This selector socket reserves and reports a
# kernel-selected candidate; it never listens and is
# not transferred to Tractor. Closing it lets
# `open_root_actor()` create its own listener on the
# same addr. The close/rebind handoff is non-atomic,
# hence the bounded collision retries.
sock: socket.socket
with socket.socket(
socket.AF_INET,
socket.SOCK_STREAM,
) as sock:
sock.bind(('127.0.0.1', 0))
selected: tuple[str, int] = sock.getsockname()
registry_addr: tuple[str, int] = (
selected[0],
selected[1],
)
try:
actor: tractor.Actor
async with tractor.open_root_actor(
name='dedicated_registrar',
registry_addrs=[registry_addr],
enable_transports=['tcp'],
enable_modules=[],
ensure_registry=True,
loglevel='error',
) as actor:
if not actor.is_registrar:
raise RuntimeError(
'daemon did not become registrar'
)
tmp_file: Path = ready_file.with_suffix('.tmp')
tmp_file.write_text(
str(registry_addr[1]),
encoding='ascii',
)
tmp_file.replace(ready_file)
await trio.sleep_forever()
except BaseException as exc:
if (
not _is_addr_collision(exc)
or attempt == MAX_BIND_ATTEMPTS
):
raise
await trio.sleep(.05 * attempt)
try:
trio.run(serve)
except KeyboardInterrupt:
pass
def _registrar_command(ready_path: Path) -> list[str]:
'''
Build a child command that loads without running `main()`.
`runpy.run_path()` also works when the docs test copies and
renames this example before executing it.
'''
module_path: str = repr(str(Path(__file__).resolve()))
function_name: str = repr('run_registrar')
ready_arg: str = repr(str(ready_path))
code: str = (
f'import runpy; module = runpy.run_path({module_path}); '
f'module[{function_name}]({ready_arg})'
)
return [sys.executable, '-c', code]
def _wait_registrar_ready(
ready_path: Path,
proc: subprocess.Popen,
deadline: float = 10.0,
) -> tuple[str, int]:
'''
Wait until the child has entered its registrar actor context.
The child atomically publishes its selected port only after
`open_root_actor()` completes. Fail early if startup crashes.
'''
end: float = time.monotonic() + deadline
while time.monotonic() < end:
if proc.poll() is not None:
returncode: int|None = proc.returncode
raise RuntimeError(
f'registrar exited during startup: {returncode=}'
)
try:
port: int = int(
ready_path.read_text(encoding='ascii')
)
except (
OSError,
ValueError,
):
time.sleep(.05)
continue
if not 0 < port < 2**16:
raise RuntimeError(f'invalid registrar port: {port!r}')
if proc.poll() is not None:
raise RuntimeError(
'registrar exited after reporting ready'
)
return ('127.0.0.1', port)
raise TimeoutError('registrar did not report ready')
def _stop_registrar(
proc: subprocess.Popen,
graceful_timeout: float = 5.0,
) -> None:
'''
Stop and reap the registrar, escalating after a bounded wait.
Windows children receive `CTRL_C_EVENT` in their new process
group; POSIX children receive `SIGINT`. A child that ignores
graceful shutdown is killed, and every path finishes with
`wait()`. A non-zero child exit remains visible to the caller.
'''
if proc.poll() is None:
graceful_signal: int = (
signal.CTRL_C_EVENT
if sys.platform == 'win32'
else signal.SIGINT
)
try:
proc.send_signal(graceful_signal)
except OSError:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=graceful_timeout)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
if proc.returncode:
raise RuntimeError(
'registrar shutdown failed: '
f'returncode={proc.returncode}'
)
async def greet() -> str:
'''
Return a greeting identifying the actor serving the RPC.
'''
actor_name: str = tractor.current_actor().name
return f'hello from {actor_name}!'
async def discover_and_greet(
registry_addr: tuple[str, int],
) -> tuple[str, str, str]:
'''
Prove registrar lookup from a client without a service channel.
The parent spawns this actor as `greeter`'s sibling. A non-`None`
registry portal from `query_actor()` proves that discovery did
not take the existing-peer fast path, which returns no registry
portal.
'''
service_addr: tuple[str, int]|None
registry_portal: tractor.Portal|None
async with tractor.query_actor(
'greeter',
regaddr=registry_addr,
) as (service_addr, registry_portal):
if registry_portal is None:
raise RuntimeError('lookup used a local service channel')
if service_addr is None:
raise RuntimeError('greeter was not registered')
service_portal: tractor.Portal|None
async with tractor.find_actor(
'greeter',
registry_addrs=[registry_addr],
) as service_portal:
if service_portal is None:
raise RuntimeError('greeter disappeared before RPC')
greeting: str = await service_portal.run(greet)
client_name: str = tractor.current_actor().name
return client_name, repr(service_addr), greeting
async def app(registry_addr: tuple[str, int]) -> None:
'''
Use sibling service and client actors with an external registrar.
Only the parent receives both spawn-time portals. The `client`
actor performs discovery in its own process and has no direct
`greeter` channel before the lookup.
'''
actor_nursery: tractor.ActorNursery
async with tractor.open_nursery(
registry_addrs=[registry_addr],
enable_transports=['tcp'],
) as actor_nursery:
await actor_nursery.start_actor(
'greeter',
enable_modules=[__name__],
)
client_portal: tractor.Portal = (
await actor_nursery.start_actor(
'client',
enable_modules=[__name__],
)
)
result: tuple[str, str, str] = await client_portal.run(
discover_and_greet,
registry_addr=registry_addr,
)
client_name: str
service_addr: str
greeting: str
(
client_name,
service_addr,
greeting,
) = result
print(
f'{client_name!r} found `greeter` through registrar '
f'{registry_addr!r}; service address: {service_addr}\n'
f'{greeting}'
)
await actor_nursery.cancel()
# TODO: Promote this lifecycle into an OTB `tractor.discovery`
# registrar subsystem. Reuse attach-or-create ownership from
# `piker.service.maybe_open_pikerd()` and named service supervision
# from `piker.service.Services`; replace the file readiness
# handshake, then use the API from `tractor._testing.pytest` to
# isolate remaining hard-coded `reg_addr` cases.
@acm
async def _open_registrar(
) -> AsyncIterator[tuple[str, int]]:
'''
Start, publish, and reap one dedicated registrar process.
The Windows child gets a distinct console process group so the
graceful control event targets it without interrupting this
process.
'''
temp_dir: str
with tempfile.TemporaryDirectory(
prefix='tractor-registrar-',
) as temp_dir:
ready_path: Path = Path(temp_dir) / 'ready'
creationflags: int = (
subprocess.CREATE_NEW_PROCESS_GROUP
if sys.platform == 'win32'
else 0
)
registrar: subprocess.Popen = subprocess.Popen(
_registrar_command(ready_path),
stdout=subprocess.DEVNULL,
creationflags=creationflags,
)
primary_error: BaseException|None = None
try:
registry_addr: tuple[str, int] = _wait_registrar_ready(
ready_path,
registrar,
)
print(
f'dedicated registrar ready at {registry_addr!r} '
f'(pid {registrar.pid})'
)
yield registry_addr
except BaseException as error:
primary_error = error
raise
finally:
try:
_stop_registrar(registrar)
except BaseException as cleanup_error:
if primary_error is None:
raise
cleanup_note: str = (
'registrar cleanup also failed: '
f'{cleanup_error!r}'
)
primary_error.add_note(cleanup_note)
print('dedicated registrar shut down')
async def main() -> None:
'''
Run the external registrar and sibling discovery actors.
'''
registry_addr: tuple[str, int]
async with _open_registrar() as registry_addr:
await app(registry_addr)
if __name__ == '__main__':
trio.run(main)

View File

@ -1,4 +1,6 @@
import time
from typing import AsyncIterator
import trio
import tractor
from tractor import (
@ -9,14 +11,19 @@ from tractor import (
# this is the first 2 actors, streamer_1 and streamer_2
async def stream_data(seed):
async def stream_data(seed: int) -> AsyncIterator[int]:
'''
Stream integers up to a seed value.
'''
i: int
for i in range(seed):
yield i
await trio.sleep(0.0001) # trigger scheduler
# this is the third actor; the aggregator
async def aggregate(seed):
async def aggregate(seed: int) -> AsyncIterator[int]:
'''
Ensure that the two streams we receive match but only stream
a single set of values to the parent.
@ -25,30 +32,47 @@ async def aggregate(seed):
an: ActorNursery
async with tractor.open_nursery() as an:
portals: list[Portal] = []
i: int
for i in range(1, 3):
# fork/spawn call
portal = await an.start_actor(
portal: Portal = await an.start_actor(
name=f'streamer_{i}',
enable_modules=[__name__],
)
portals.append(portal)
send_chan: trio.MemorySendChannel[int]
recv_chan: trio.MemoryReceiveChannel[int]
send_chan, recv_chan = trio.open_memory_channel(500)
async def push_to_chan(portal, send_chan):
async def push_to_chan(
portal: Portal,
send_chan: trio.MemorySendChannel[int],
) -> None:
'''
Forward one remote stream into a local channel.
'''
# TODO: https://github.com/goodboy/tractor/issues/207
async with send_chan:
async with portal.open_stream_from(stream_data, seed=seed) as stream:
stream: MsgStream
async with portal.open_stream_from(
stream_data,
seed=seed,
) as stream:
value: int
async for value in stream:
# leverage trio's built-in backpressure
await send_chan.send(value)
print(f"FINISHED ITERATING {portal.channel.uid}")
uid: tuple[str, str] = portal.chan.uid
print(f'FINISHED ITERATING {uid}')
# spawn 2 trio tasks to collect streams and push to a local queue
# spawn 2 trio tasks to collect streams and push to a local
# queue
n: trio.Nursery
async with trio.open_nursery() as n:
for portal in portals:
@ -61,8 +85,9 @@ async def aggregate(seed):
# close this local task's reference to send side
await send_chan.aclose()
unique_vals = set()
unique_vals: set[int] = set()
async with recv_chan:
value: int
async for value in recv_chan:
if value not in unique_vals:
unique_vals.add(value)
@ -71,11 +96,11 @@ async def aggregate(seed):
assert value in unique_vals
print("FINISHED ITERATING in aggregator")
print('FINISHED ITERATING in aggregator')
await an.cancel()
print("WAITING on `ActorNursery` to finish")
print("AGGREGATOR COMPLETE!")
print('WAITING on `ActorNursery` to finish')
print('AGGREGATOR COMPLETE!')
async def main() -> list[int]:
@ -94,8 +119,8 @@ async def main() -> list[int]:
# debug_mode=True,
) as an:
seed = int(1e3)
pre_start = time.time()
seed: int = int(1e3)
pre_start: float = time.time()
portal: Portal = await an.start_actor(
name='aggregator',
@ -108,23 +133,27 @@ async def main() -> list[int]:
seed=seed,
) as stream:
start = time.time()
start: float = time.time()
# the portal call returns exactly what you'd expect
# as if the remote "aggregate" function was called locally
# as if the remote "aggregate" function was called
# locally
result_stream: list[int] = []
value: int
async for value in stream:
result_stream.append(value)
cancelled: bool = await portal.cancel_actor()
assert cancelled
stream_time: float = time.time() - start
total_time: float = time.time() - pre_start
print(
f"STREAM TIME = {time.time() - start}\n"
f"STREAM + SPAWN TIME = {time.time() - pre_start}\n"
f'STREAM TIME = {stream_time}\n'
f'STREAM + SPAWN TIME = {total_time}\n'
)
assert result_stream == list(range(seed))
return result_stream
if __name__ == '__main__':
final_stream = trio.run(main)
final_stream: list[int] = trio.run(main)

View File

@ -13,7 +13,10 @@ import tractor
async def aio_echo_server(
chan: tractor.to_asyncio.LinkedTaskChannel,
) -> None:
'''
Echo messages received through an asyncio task channel.
'''
# a first message must be sent **from** this ``asyncio``
# task or the ``trio`` side will never unblock from
# ``tractor.to_asyncio.open_channel_from():``
@ -28,9 +31,15 @@ async def aio_echo_server(
@tractor.context
async def trio_to_aio_echo_server(
ctx: tractor.Context,
):
) -> None:
'''
Bridge an actor stream to the asyncio echo server.
'''
# this will block until the ``asyncio`` task sends a "first"
# message.
chan: tractor.to_asyncio.LinkedTaskChannel
first: str
async with tractor.to_asyncio.open_channel_from(
aio_echo_server,
) as (chan, first):
@ -38,39 +47,49 @@ async def trio_to_aio_echo_server(
assert first == 'start'
await ctx.started(first)
stream: tractor.MsgStream
async with ctx.open_stream() as stream:
msg: int
async for msg in stream:
await chan.send(msg)
out = await chan.receive()
out: int = await chan.receive()
# echo back to parent actor-task
await stream.send(out)
async def main():
async def main() -> None:
'''
Run the infected asyncio echo-server example.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
p = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'aio_server',
enable_modules=[__name__],
infect_asyncio=True,
)
async with p.open_context(
ctx: tractor.Context
first: str
async with portal.open_context(
trio_to_aio_echo_server,
) as (ctx, first):
assert first == 'start'
count = 0
count: int = 0
stream: tractor.MsgStream
async with ctx.open_stream() as stream:
delays = []
send = time.time()
delays: list[float] = []
send: float = time.time()
await stream.send(count)
msg: int
async for msg in stream:
recv = time.time()
recv: float = time.time()
delays.append(recv - send)
assert msg == count
count += 1
@ -81,7 +100,7 @@ async def main():
break
print(f'mean round trip rate (Hz): {1/mean(delays)}')
await p.cancel_actor()
await portal.cancel_actor()
if __name__ == '__main__':

View File

@ -1,9 +1,10 @@
"""
'''
Integration test: spawning tractor actors from an MPI process.
When a parent is launched via ``mpirun``, Open MPI sets ``OMPI_*`` env
vars that bind ``MPI_Init`` to the ``orted`` daemon. Tractor children
inherit those env vars, so if ``inherit_parent_main=True`` (the default)
When a parent is launched via ``mpirun``, Open MPI sets
``OMPI_*`` env vars that bind ``MPI_Init`` to the ``orted``
daemon. Tractor children inherit those env vars, so if
``inherit_parent_main=True`` (the default)
the child re-executes ``__main__``, re-imports ``mpi4py``, and
``MPI_Init_thread`` fails because the child was never spawned by
``orted``::
@ -12,13 +13,15 @@ the child re-executes ``__main__``, re-imports ``mpi4py``, and
--> Returned value No permission (-17) instead of ORTE_SUCCESS
Passing ``inherit_parent_main=False`` and placing RPC functions in a
separate importable module (``_child``) avoids the re-import entirely.
separate importable module (``_child``) avoids the re-import
entirely.
Usage::
mpirun --allow-run-as-root -np 1 python -m \
examples.integration.mpi4py.inherit_parent_main
"""
'''
from mpi4py import MPI
@ -30,21 +33,27 @@ from ._child import child_fn
async def main() -> None:
rank = MPI.COMM_WORLD.Get_rank()
print(f"[parent] rank={rank} pid={os.getpid()}", flush=True)
'''
Spawn an MPI-safe child without replaying the parent main.
'''
rank: int = MPI.COMM_WORLD.Get_rank()
pid: int = os.getpid()
print(f'[parent] rank={rank} pid={pid}', flush=True)
an: tractor.ActorNursery
async with tractor.open_nursery(start_method='trio') as an:
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'mpi-child',
enable_modules=[child_fn.__module__],
# Without this the child replays __main__, which
# re-imports mpi4py and crashes on MPI_Init.
inherit_parent_main=False,
)
result = await portal.run(child_fn)
print(f"[parent] got: {result}", flush=True)
result: str = await portal.run(child_fn)
print(f'[parent] got: {result}', flush=True)
await portal.cancel_actor()
if __name__ == "__main__":
if __name__ == '__main__':
trio.run(main)

View File

@ -0,0 +1,172 @@
# `tractor` over a WireGuard tunnel, declared as one maddr
A two-host LAN setup: a `tractor` actor tree on host A, dialed
from host B, with the endpoint declared as a single `wg`
multiaddr.
Supersedes the example set in gh
[#482](https://github.com/goodboy/tractor/issues/482) — see
[what changed](#what-changed-vs-482).
> **Why `examples/multihost/`?** `tests/test_docs_examples.py`
> walks `examples/` recursively and runs everything it collects
> as a subproc, asserting `rc == 0`. These need a real second
> host and a live `wg` tunnel, so they can't satisfy that;
> `'multihost' not in p[0]` is already in the test's exclusion
> list, which is what keeps them out of CI.
## the maddr form
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\____ wg bearer ___________/\__ key __/\____ tractor ep _____/
underlay, wg `ListenPort` overlay, on the wg iface
(kernel owns the socket) (`MsgTransport` binds this)
```
Three parts, three different owners:
| part | socket owner / provisioner | runtime role |
| --- | --- | --- |
| `/ip4/../udp/51820` bearer | kernel-owned; `wg-quick` now, tractor bindspace later | control-plane metadata |
| `/wg/u<key>` | nothing — it's an identity | parsed, verified explicitly |
| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | application `MsgTransport` |
Verified against py-multiaddr
[#108](https://github.com/multiformats/py-multiaddr/pull/108):
this composed form parses and round-trips
(`['ip4','udp','wg','ip4','tcp']`).
## requirements
py-multiaddr #108 is **merged** (2026-07-28) but ships in no
release yet — the latest `0.2.0` (2026-03-17) predates it and has
no `wg` codec. So `pyproject.toml` temporarily pins the merge commit
in its PEP 621 dependency metadata, and a plain
```bash
uv sync --extra wg
```
gets you a `wg`-aware `multiaddr` plus pyroute2's Linux netlink API.
The multiaddr pin goes away once a release carries the codec.
`py-multibase` is a direct dependency.
Without the codec `parse_wg_maddr()` raises immediately with an
actionable message — there is deliberately **no** degraded
hand-split fallback. `_wg_proto_code()` performs the capability
check before parsing.
Every peel and re-compose here goes through `py-multiaddr`'s own
tunnel API (`.decapsulate_code()`, `.split()`, `.join()`,
`.encapsulate()`, `.value_for_protocol()`) rather than any
bespoke segment slicing — see its README "En/decapsulate" and
"Tunneling" sections. gh #429 was about *dropping* our NIH
parser, and that applies to peeling a tunnel stack just as much
as to decoding one proto.
## 0. tunnel setup (out-of-band, both hosts)
Host A is the service host (underlay e.g. `192.168.1.50`), host B
your workstation. Overlay net `10.0.11.0/24`.
```bash
umask 077
wg genkey | tee wg_priv.key | wg pubkey > wg_pub.key
```
`/etc/wireguard/wg0.conf` on **host A**:
```ini
[Interface]
PrivateKey = <A_priv>
Address = 10.0.11.1/24
ListenPort = 51820
```
```ini
[Peer]
PublicKey = <B_pub>
AllowedIPs = 10.0.11.2/32
```
on **host B**:
```ini
[Interface]
PrivateKey = <B_priv>
Address = 10.0.11.2/24
```
```ini
[Peer]
PublicKey = <A_pub>
Endpoint = 192.168.1.50:51820
AllowedIPs = 10.0.11.1/32
PersistentKeepalive = 25
```
Note how `ListenPort` and `Endpoint` are exactly the maddr's
bearer segment, and `[Interface] Address` is its overlay host.
```bash
sudo wg-quick up wg0 # both hosts
ping -c1 10.0.11.1 # from B
```
## 1. get your pubkey into the maddr
```bash
python -c "
from tractor.discovery import mb_pubkey
key = open('wg_pub.key').read().strip()
print(mb_pubkey(key))
"
```
Paste the `u...` output into `WG_MADDR` in both scripts (they use
the same string — A's bearer, A's key, A's overlay ep).
## 2. run
```bash
# host A
python host_a_srv.py
# host B
python host_b_client.py
```
`host_a_srv.py` must be importable on host B too, since
`portal.run()` refs the fn by module path — standard `tractor`
RPC semantics.
## what changed vs #482
Four corrections, all from
`ai/tpt-backends/03_wg_tunnel_bindspace.md`:
1. **the maddr semantics were inverted.** #482 used
`/ip4/10.0.11.1/tcp/1616/wg/u<key>` — that parses, but it puts
the *overlay* addr where the bearer belongs and `tcp` where
wg's `udp` `ListenPort` goes, and it declares no overlay ep at
all. `parse_wg_maddr()` now rejects it with an actionable
error.
2. **parsing is pure.** #482's helper had the key-check adjacent
to the parse; async `verify_wg_peer()` is now a separate,
explicitly composed step that the caller invokes. Implicit
kernel inspection from a parser is a nasty surprise.
3. **no `sudo` or subprocess.** #482 ran `sudo wg show`; tractor's
helper reads generic netlink through pyroute2 and never attempts
privilege escalation or namespace creation.
4. **no new `Address` proto-type.** The tunnel rides *beside* the
overlay addr in a frozen `TunnelledAddress`, and only `.overlay`
crosses into `open_nursery()`. #482 §6 floated a `WGAddress`
registered in `_address_types` — that table is a `bidict`
(1:1 proto-key↔type) and `_addr_to_transport` wants a
`MsgTransport` per addr-type, which `wg` doesn't have.
## next
Layer A's `TunnelledAddress` and native maddr parser plus Layer B's
explicit pyroute2 verification now live in `tractor.discovery`. Next,
add `open_bindspace()` `@acm`s which create/tear down the iface and
netns.

View File

@ -0,0 +1,61 @@
# tractor: distributed structured concurrency.
'''
Host A: the service host, reachable over a `wg` tunnel.
Binds `tractor`'s registrar + an `echo_srv` sub-actor on the
tunnel's *overlay* addr, declared as a single `wg` maddr.
'''
from __future__ import annotations
import tractor
import trio
from tractor.discovery import (
TunnelledAddress,
mk_maddr,
parse_wg_maddr,
verify_wg_peer,
)
# bearer = host A's underlay `(ip, wg ListenPort)`
# key = host A's OWN tunnel pubkey
# overlay = the ep `tractor` binds, on the wg iface's addr
WG_MADDR: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/u<A_pub_b64url>'
'/ip4/10.0.11.1/tcp/1616'
)
async def echo(msg: str) -> str:
actor = tractor.current_actor()
return f'{actor.aid.name!r} echoes: {msg}'
async def main():
addr: TunnelledAddress = parse_wg_maddr(WG_MADDR)
assert await verify_wg_peer(addr.tunnel), (
f'wg pubkey from maddr not active on wg0 !\n'
f'maddr: {WG_MADDR}\n'
f'key: {addr.tunnel.peer_pubkey}\n'
)
print(
f'wg bearer (kernel-owned): {addr.tunnel.bearer}\n'
f'tractor overlay ep: {addr.overlay}\n'
)
async with tractor.open_nursery(
# XXX only `.overlay` crosses into the runtime; the bearer
# + key are bindspace metadata, never `Endpoint` addrs.
registry_addrs=[addr.overlay],
enable_transports=[addr.overlay.proto_key],
) as an:
await an.start_actor(
'echo_srv',
enable_modules=[__name__],
)
print(f'echo_srv up on\n {mk_maddr(addr)}\n')
await trio.sleep_forever()
if __name__ == '__main__':
trio.run(main)

View File

@ -0,0 +1,51 @@
# tractor: distributed structured concurrency.
'''
Host B: workstation dialing host A's actor tree through the
`wg` tunnel.
'''
from __future__ import annotations
import tractor
import trio
from tractor.discovery import (
TunnelledAddress,
parse_wg_maddr,
verify_wg_peer,
)
from host_a_srv import echo # noqa: F401 (RPC refs it by mod path)
# same maddr as host A: A's bearer, A's key, A's overlay ep
WG_MADDR: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/u<A_pub_b64url>'
'/ip4/10.0.11.1/tcp/1616'
)
async def main():
addr: TunnelledAddress = parse_wg_maddr(WG_MADDR)
assert await verify_wg_peer(addr.tunnel), (
f'wg pubkey from maddr not a peer on wg0 !\n'
f'maddr: {WG_MADDR}\n'
)
async with (
tractor.open_root_actor(
name='wg_client',
registry_addrs=[addr.overlay],
enable_transports=[addr.overlay.proto_key],
),
tractor.find_actor(
'echo_srv',
registry_addrs=[addr.overlay],
) as portal,
):
res: str = await portal.run(
echo,
msg='hello over wg!',
)
print(res)
if __name__ == '__main__':
trio.run(main)

View File

@ -1,3 +1,5 @@
from typing import AsyncIterator
import trio
import tractor
@ -5,17 +7,30 @@ import tractor
log = tractor.log.get_logger('multiportal')
async def stream_data(seed=10):
log.info("Starting stream task")
async def stream_data(seed: int = 10) -> AsyncIterator[int]:
'''
Stream a finite sequence of integers.
'''
log.info('Starting stream task')
i: int
for i in range(seed):
yield i
await trio.sleep(0) # trigger scheduler
async def stream_from_portal(p, consumed):
async def stream_from_portal(
portal: tractor.Portal,
consumed: list[int],
) -> None:
'''
Consume one stream and toggle each value in a shared list.
async with p.open_stream_from(stream_data) as stream:
'''
stream: tractor.MsgStream
async with portal.open_stream_from(stream_data) as stream:
item: int
async for item in stream:
if item in consumed:
consumed.remove(item)
@ -23,20 +38,33 @@ async def stream_from_portal(p, consumed):
consumed.append(item)
async def main():
async def main() -> None:
'''
Consume two concurrent streams through one portal.
'''
an: tractor.ActorNursery
async with tractor.open_nursery(loglevel='info') as an:
p = await an.start_actor('stream_boi', enable_modules=[__name__])
portal: tractor.Portal = await an.start_actor(
'stream_boi',
enable_modules=[__name__],
)
consumed = []
consumed: list[int] = []
n: trio.Nursery
async with trio.open_nursery() as n:
for i in range(2):
n.start_soon(stream_from_portal, p, consumed)
for _ in range(2):
n.start_soon(
stream_from_portal,
portal,
consumed,
)
# both streaming consumer tasks have completed and so we should
# have nothing in our list thanks to single threadedness
# both streaming consumer tasks have completed and so we
# should have nothing in our list thanks to single
# threadedness
assert not consumed
await an.cancel()

View File

@ -41,6 +41,7 @@ async def fan_out_squares(
aggregated squares to our parent.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
portals: list[tractor.Portal] = []
for i in (1, 2):
@ -52,21 +53,28 @@ async def fan_out_squares(
)
# unblock the parent's `.open_context()` entry and
# report which leaves came up.
await ctx.started(
[p.chan.aid.name for p in portals]
)
leaf_names: list[str] = [
portal.chan.aid.name
for portal in portals
]
await ctx.started(leaf_names)
squares: dict[int, int] = {}
async def run_in_leaf(
portal: tractor.Portal,
x: int,
) -> None:
'''
Run one square calculation in a leaf actor.
'''
squares[x] = await portal.run(
compute_square,
x=x,
)
# fan out one sub-RPC per input val, concurrently.
tn: trio.Nursery
async with trio.open_nursery() as tn:
for i, x in enumerate(vals):
tn.start_soon(
@ -83,11 +91,18 @@ async def fan_out_squares(
async def main() -> None:
'''
Run the nested actor-tree example.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'supervisor',
enable_modules=[__name__],
)
ctx: tractor.Context
leaf_names: list[str]
async with portal.open_context(
fan_out_squares,
vals=[1, 2, 3, 4],

View File

@ -1,18 +1,23 @@
"""
'''
Demonstration of the prime number detector example from the
``concurrent.futures`` docs:
https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor-example
https://docs.python.org/3/library/concurrent.futures.html\
#processpoolexecutor-example
This uses no extra threads, fancy semaphores or futures; all we need
is ``tractor``'s channels.
"""
'''
from contextlib import (
asynccontextmanager as acm,
aclosing,
)
from typing import Callable
from typing import (
AsyncIterator,
Awaitable,
Callable,
)
import itertools
import math
import time
@ -21,7 +26,12 @@ import tractor
import trio
PRIMES = [
type ActorMap = Callable[
[Callable[[int], Awaitable[bool]], list[int]],
AsyncIterator[tuple[int, bool]],
]
PRIMES: list[int] = [
112272535095293,
112582705942171,
112272535095293,
@ -31,7 +41,11 @@ PRIMES = [
]
async def is_prime(n):
async def is_prime(n: int) -> bool:
'''
Return whether ``n`` is prime.
'''
if n < 2:
return False
if n == 2:
@ -47,23 +61,32 @@ async def is_prime(n):
@acm
async def worker_pool(workers=4):
"""Though it's a trivial special case for ``tractor``, the well
async def worker_pool(
workers: int = 4,
) -> AsyncIterator[ActorMap]:
'''
Though it's a trivial special case for ``tractor``, the well
known "worker pool" seems to be the defacto "but, I want this
process pattern!" for most parallelism pilgrims.
Yes, the workers stay alive (and ready for work) until you close
the context.
"""
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
portals = []
portals: list[tractor.Portal] = []
snd_chan: trio.MemorySendChannel[tuple[int, bool]]
recv_chan: trio.MemoryReceiveChannel[tuple[int, bool]]
snd_chan, recv_chan = trio.open_memory_channel(len(PRIMES))
i: int
for i in range(workers):
# this starts a new sub-actor (process + trio runtime) and
# stores it's "portal" for later use to "submit jobs" (ugh).
# this starts a new sub-actor (process + trio
# runtime) and stores it's "portal" for later use to
# "submit jobs" (ugh).
portals.append(
await an.start_actor(
f'worker_{i}',
@ -72,17 +95,36 @@ async def worker_pool(workers=4):
)
async def _map(
worker_func: Callable[[int], bool],
sequence: list[int]
) -> list[bool]:
worker_func: Callable[[int], Awaitable[bool]],
sequence: list[int],
) -> AsyncIterator[tuple[int, bool]]:
'''
Dispatch values across workers and yield their results.
# define an async (local) task to collect results from workers
async def send_result(func, value, portal):
await snd_chan.send((value, await portal.run(func, n=value)))
'''
# define an async (local) task to collect results from
# workers
async def send_result(
func: Callable[[int], Awaitable[bool]],
value: int,
portal: tractor.Portal,
) -> None:
'''
Run one remote worker call and send its result.
'''
result: bool = await portal.run(func, n=value)
await snd_chan.send((value, result))
tn: trio.Nursery
async with trio.open_nursery() as tn:
for value, portal in zip(sequence, itertools.cycle(portals)):
value: int
portal: tractor.Portal
for value, portal in zip(
sequence,
itertools.cycle(portals),
):
tn.start_soon(
send_result,
worker_func,
@ -101,21 +143,30 @@ async def worker_pool(workers=4):
await an.cancel()
async def main():
async def main() -> None:
'''
Report primality results from a pool of actors.
'''
actor_map: ActorMap
async with worker_pool() as actor_map:
start = time.time()
start: float = time.time()
results: AsyncIterator[tuple[int, bool]]
async with aclosing(actor_map(is_prime, PRIMES)) as results:
number: int
prime: bool
async for number, prime in results:
print(f'{number} is prime: {prime}')
print(f'processing took {time.time() - start} seconds')
elapsed: float = time.time() - start
print(f'processing took {elapsed} seconds')
if __name__ == '__main__':
start = time.time()
start: float = time.time()
trio.run(main)
print(f'script took {time.time() - start} seconds')
elapsed: float = time.time() - start
print(f'script took {elapsed} seconds')

View File

@ -1,30 +1,36 @@
"""
'''
Run with a process monitor from a terminal using::
$TERM -e watch -n 0.1 "pstree -a $$" \
& python examples/parallelism/single_func.py \
&& kill $!
"""
'''
import os
import tractor
import trio
async def burn_cpu():
async def burn_cpu() -> int:
'''
Burn CPU briefly and return the current process ID.
pid = os.getpid()
'''
pid: int = os.getpid()
# burn a core @ ~ 50kHz
for _ in range(50000):
await trio.sleep(1/50000/50)
await trio.sleep(1 / 50000 / 50)
return pid
async def main():
async def main() -> None:
'''
Run ``burn_cpu()`` in the parent and a subactor.
'''
async with trio.open_nursery() as tn:
# burn rubber in the parent too
@ -32,9 +38,9 @@ async def main():
# run the same func as the lone task in a subactor,
# block on and collect its PID as the caller-side result
pid = await tractor.to_actor.run(burn_cpu)
pid: int = await tractor.to_actor.run(burn_cpu)
print(f"Collected subproc {pid}")
print(f'Collected subproc {pid}')
if __name__ == '__main__':

View File

@ -1,20 +1,24 @@
import trio
import tractor
async def sleepy_jane() -> None:
uid: tuple = tractor.current_actor().uid
'''
Identify the current actor and sleep forever.
'''
uid: tuple[str, str] = tractor.current_actor().uid
print(f'Yo i am actor {uid}')
await trio.sleep_forever()
async def main():
async def main() -> None:
'''
Spawn a flat actor cluster, with one process per detected core.
'''
portal_map: dict[str, tractor.Portal]
tn: trio.Nursery
# look at this hip new syntax!
async with (
@ -27,7 +31,9 @@ async def main():
trio.open_nursery() as tn,
):
for (name, portal) in portal_map.items():
name: str
portal: tractor.Portal
for name, portal in portal_map.items():
tn.start_soon(
portal.run,
sleepy_jane,

View File

@ -2,18 +2,30 @@ import trio
import tractor
async def assert_err():
async def assert_err() -> None:
'''
Raise an assertion error in the current actor.
'''
assert 0
async def main():
async def main() -> None:
'''
Propagate a failing one-shot task from a subactor.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
real_actors = []
real_actors: list[tractor.Portal] = []
i: int
for i in range(3):
real_actors.append(await an.start_actor(
f'actor_{i}',
enable_modules=[__name__],
))
real_actors.append(
await an.start_actor(
f'actor_{i}',
enable_modules=[__name__],
)
)
# run one one-shot task actor that will fail immediately;
# its error raises right here in the caller's task..
@ -28,4 +40,4 @@ if __name__ == '__main__':
# also raises
trio.run(main)
except tractor.RemoteActorError:
print("Look Maa that actor failed hard, hehhh!")
print('Look Maa that actor failed hard, hehhh!')

View File

@ -9,16 +9,19 @@ async def simple_rpc(
data: int,
) -> None:
'''Test a small ping-pong 2-way streaming server.
'''
Test a small ping-pong 2-way streaming server.
'''
# signal to parent that we're up much like
# ``trio.TaskStatus.started()``
await ctx.started(data + 1)
stream: tractor.MsgStream
async with ctx.open_stream() as stream:
count = 0
count: int = 0
msg: str
async for msg in stream:
assert msg == 'ping'
@ -30,15 +33,22 @@ async def simple_rpc(
async def main() -> None:
'''
Exercise bidirectional streaming with a remote actor.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'rpc_server',
enable_modules=[__name__],
)
# XXX: syntax requires py3.9
ctx: tractor.Context
sent: int
stream: tractor.MsgStream
async with (
portal.open_context(
@ -52,10 +62,11 @@ async def main() -> None:
assert sent == 11
count = 0
count: int = 0
# receive msgs using async for style
await stream.send('ping')
msg: str
async for msg in stream:
assert msg == 'pong'
await stream.send('ping')

View File

@ -35,11 +35,13 @@ async def client_task() -> None:
'''
# a lookup miss yields `None` (not an error).
async with tractor.find_actor('no_such_svc') as portal:
assert portal is None
maybe_portal: tractor.Portal|None
async with tractor.find_actor('no_such_svc') as maybe_portal:
assert maybe_portal is None
print('client: "no_such_svc" is not registered')
# block until the service shows up in the registry,
# then call into it through the delivered portal.
portal: tractor.Portal
async with tractor.wait_for_actor('quote_svc') as portal:
quote: float = await portal.run(
get_quote,
@ -49,13 +51,19 @@ async def client_task() -> None:
async def main() -> None:
'''
Run a discoverable quote service and its client.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'quote_svc',
enable_modules=[__name__],
)
# run the client in a separate task which discovers
# the daemon purely by its registered name.
tn: trio.Nursery
async with trio.open_nursery() as tn:
tn.start_soon(client_task)
# explicit graceful teardown of the daemon.

View File

@ -1,19 +1,29 @@
import trio
import tractor
tractor.log.get_console_log("INFO")
tractor.log.get_console_log('INFO')
async def main(service_name):
async def main(service_name: str) -> None:
'''
Discover one actor and inspect its registrar connection.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
await an.start_actor(service_name)
async with tractor.get_registry() as portal:
print(f"Registrar is listening on {portal.channel}")
async with tractor.get_registry() as reg_portal:
print(
f'Registrar is listening on {reg_portal.channel}'
)
async with tractor.wait_for_actor(service_name) as sockaddr:
print(f"my_service is found at {sockaddr}")
actor_portal: tractor.Portal
async with tractor.wait_for_actor(
service_name,
) as actor_portal:
service_addr = actor_portal.chan.raddr
print(f'my_service is found at {service_addr}')
await an.cancel()

View File

@ -28,6 +28,7 @@ async def tick_stream(
# wait for the go-signal ensuring every parent-side
# subscriber is attached before any tick is sent.
assert await stream.receive() == 'go'
i: int
for i in range(count):
await stream.send(i)
# falling out gracefully closes our stream side;
@ -37,15 +38,17 @@ async def tick_stream(
async def consume(
name: str,
stream: tractor.MsgStream,
task_status: trio.TaskStatus = trio.TASK_STATUS_IGNORED,
task_status: trio.TaskStatus[None] = trio.TASK_STATUS_IGNORED,
) -> None:
'''
Consume a private broadcast-copy of the IPC stream.
'''
bcaster: tractor.trionics.BroadcastReceiver
async with stream.subscribe() as bcaster:
task_status.started()
ticks: list[int] = []
tick: int
async for tick in bcaster:
print(f'{name}: rx {tick}')
ticks.append(tick)
@ -54,11 +57,19 @@ async def consume(
async def main() -> None:
'''
Fan one remote stream out to local subscribers.
'''
an: tractor.ActorNursery
async with tractor.open_nursery() as an:
portal = await an.start_actor(
portal: tractor.Portal = await an.start_actor(
'ticker',
enable_modules=[__name__],
)
ctx: tractor.Context
first: int
stream: tractor.MsgStream
async with (
portal.open_context(
tick_stream,
@ -67,9 +78,11 @@ async def main() -> None:
ctx.open_stream() as stream,
):
assert first == 5
tn: trio.Nursery
async with trio.open_nursery() as tn:
# use `.start()` so each consumer is known
# to be subscribed before the ticks flow.
i: int
for i in range(3):
await tn.start(
consume,

View File

@ -1,3 +1,4 @@
from collections.abc import AsyncIterator
from contextlib import (
asynccontextmanager as acm,
)
@ -16,7 +17,11 @@ _lock: trio.Lock|None = None
@acm
async def acquire_singleton_lock(
) -> None:
) -> AsyncIterator[trio.Lock]:
'''
Acquire and yield the process-wide lock.
'''
global _lock
if _lock is None:
log.info('Allocating LOCK')
@ -32,8 +37,15 @@ async def acquire_singleton_lock(
async def hold_lock_forever(
task_status=trio.TASK_STATUS_IGNORED
):
task_status: trio.TaskStatus[
trio.Lock,
] = trio.TASK_STATUS_IGNORED,
) -> None:
'''
Hold the singleton lock until cancellation.
'''
lock: trio.Lock
async with (
tractor.trionics.maybe_raise_from_masking_exc(),
acquire_singleton_lock() as lock,
@ -46,7 +58,12 @@ async def main(
ignore_special_cases: bool,
loglevel: str = 'info',
debug_mode: bool = True,
):
) -> None:
'''
Exercise lock acquisition while cancellation is masked.
'''
tn: trio.Nursery
async with (
trio.open_nursery() as tn,
@ -58,7 +75,7 @@ async def main(
from tractor.trionics import _taskc
_taskc._mask_cases.clear()
_lock = await tn.start(
_held_lock: trio.Lock = await tn.start(
hold_lock_forever,
)
with trio.move_on_after(0.2):
@ -74,8 +91,8 @@ if __name__ == '__main__':
tractor.log.get_console_log(level='info')
for case in [True, False]:
log.info(
f'\n'
f'------ RUNNING SCRIPT TRIAL ------\n'
'\n'
'------ RUNNING SCRIPT TRIAL ------\n'
f'ignore_special_cases: {case!r}\n'
)
trio.run(partial(

View File

@ -1,3 +1,4 @@
from collections.abc import Iterator
from contextlib import (
contextmanager as cm,
# TODO, any diff in async case(s)??
@ -17,7 +18,7 @@ log = tractor.log.get_logger(
@cm
def teardown_on_exc(
raise_from_handler: bool = False,
):
) -> Iterator[None]:
'''
You could also have a teardown handler which catches any exc and
does some required teardown. In this case the problem is
@ -30,7 +31,7 @@ def teardown_on_exc(
except BaseException as _berr:
berr = _berr
log.exception(
f'Handling termination teardown in child due to,\n'
'Handling termination teardown in child due to,\n'
f'{berr!r}\n'
)
if raise_from_handler:
@ -54,14 +55,18 @@ def teardown_on_exc(
async def finite_stream_to_rent(
tx: trio.abc.SendChannel,
tx: trio.abc.SendChannel[int],
child_errors_mid_stream: bool,
raise_unmasked: bool,
task_status: trio.TaskStatus[
trio.CancelScope,
trio.CancelScope|None,
] = trio.TASK_STATUS_IGNORED,
):
) -> None:
'''
Stream values while reproducing exception masking on close.
'''
async with (
# XXX without this unmasker the mid-streaming RTE is never
# reported since it is masked by the `tx.aclose()`
@ -134,19 +139,26 @@ async def main(
raise_unmasked: bool = False,
loglevel: str = 'info',
):
) -> None:
'''
Reproduce cancellation masking a child-stream failure.
'''
tractor.log.get_console_log(level=loglevel)
# the `.aclose()` being checkpoints on these
# is the source of the problem..
tx: trio.MemorySendChannel[int]
rx: trio.MemoryReceiveChannel[int]
tx, rx = trio.open_memory_channel(1)
tn: trio.Nursery
async with (
tractor.trionics.collapse_eg(),
trio.open_nursery() as tn,
rx as rx,
):
_child_cs = await tn.start(
_child_cs: trio.CancelScope|None = await tn.start(
partial(
finite_stream_to_rent,
child_errors_mid_stream=child_errors_mid_stream,
@ -154,6 +166,7 @@ async def main(
tx=tx,
)
)
msg: int
async for msg in rx:
log.debug(
f'Rent rx {msg!r}\n'
@ -162,12 +175,13 @@ async def main(
# simulate some external cancellation
# request **JUST BEFORE** the child errors.
if msg == 65:
log.cancel(
f'Cancelling parent on,\n'
f'msg={msg}\n'
f'\n'
f'Simulates OOB cancel request!\n'
)
cancel_msg: str = (
'Cancelling parent on,\n'
'msg={msg}\n'
'\n'
'Simulates OOB cancel request!\n'
).format(msg=msg)
log.cancel(cancel_msg)
tn.cancel_scope.cancel()
@ -176,8 +190,8 @@ if __name__ == '__main__':
tractor.log.get_console_log(level='info')
for case in [True, False]:
log.info(
f'\n'
f'------ RUNNING SCRIPT TRIAL ------\n'
'\n'
'------ RUNNING SCRIPT TRIAL ------\n'
f'child_errors_midstream: {case!r}\n'
)
try:

Some files were not shown because too many files have changed in this diff Show More