Compare commits

...

56 Commits

Author SHA1 Message Date
Gud Boi a834487838 Pass `Bindspace` FD to `trio` children
Thread an optional live `Bindspace` through actor spawn APIs and
give each Trio exec child an inherited namespace descriptor before
runtime bootstrap.

Deats,
- duplicate the namespace FD without changing parent ownership
- preserve caller `pass_fds` and process options
- send the `(fd, inode)` pair through the `_child` CLI
- close the parent duplicate on success, failure and cancellation
- reject MP backends until descriptor reduction is implemented
- avoid cleanup before child publication
- exercise real E2E namespace relay through handshake and RPC
- verify parent FD-table cleanup after successful child spawn

Prompt-IO: ai/prompt-io/opencode/20260828T172943Z_2ca8c570_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi dc72ecdfcd Detect child death before parent handshake
Race the initial peer handshake against process exit during Trio
child bootstrap so a dead child cannot park its spawning task.
Restore and harden the design originally implemented in `3b0724eb`.

Deats,
- register peer events before provisional nursery publication
- raise `ActorFailure` with child status when process death wins
- prioritize observed death over a simultaneous handshake
- preserve waiter exceptions without `ExceptionGroup` wrapping
- log expected losing-waiter cancellation at debug level
- remove exact failed-startup peer events during backend cleanup
- cover controlled schedules and full Trio backend cleanup

Based-on: 3b0724eba8
Prompt-IO: ai/prompt-io/opencode/20260828T045119Z_fb6d81d3_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 7dfefddae3 Enter inherited netns during child bootstrap
Consume an optional child-owned `(fd, inode)` capability before
multiprocessing or Trio child bootstrap begins runtime setup.

Deats,
- enter the netns before Trio patching and actor construction
- close the inherited FD before actor runtime startup
- preserve entry errors when descriptor cleanup also fails
- reject malformed FD values without closing unrelated descriptors
- cover multiprocessing and Trio ordering with real stand-in FDs

Prompt-IO: ai/prompt-io/opencode/20260828T012205Z_9ae7cd86_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 6e338a1173 Verify inherited network namespace entry
Add private `enter_netns()` bootstrap validation before actor runtime
integration.

Deats,
- verify the inherited FD against its expected namespace inode
- constrain `setns()` to Linux network namespaces
- verify `/proc/self/ns/net` after the entry syscall
- leave inherited FD ownership to the future spawn caller
- exercise guards with real FDs and unprivileged syscall fakes

Prompt-IO: ai/prompt-io/opencode/20260827T232500Z_d067505a_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi e1b1cf66f1 Plan typed tunnel address decoding
Record native tagged encoding for complete `TunnelledAddress` graphs as
a deferred design follow-up.

Deats,
- cover concrete overlay-address and tunnel-spec unions
- preserve optional `BindspaceRef` metadata through decoding
- replace untyped payload inspection with typed roundtrip tests

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 91bbcfdb0a Retain realized `BindspaceRef` metadata
Add `TunnelledAddress.with_bindspace_ref()` to annotate frozen tunnel
declarations with stable, serializable namespace metadata.

Deats,
- omit absent refs from the existing msgspec shape
- reject declared and realized namespace-name mismatches
- report the declared key before realization and inode afterward
- keep transport peeling free of live bindspace capability state
- expose realized refs through real listener diagnostics

Prompt-IO: ai/prompt-io/opencode/20260826T030534Z_d130431c_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi e09f6beb27 Rename bindspace resource models
Replace the unshipped `BindspaceIdentity` and `BindspaceHandle` names
with `BindspaceRef` and `Bindspace` across existing lifecycle APIs.

Deats,
- define refs as wire-safe, host-local and non-owning records
- reserve `Bindspace` for the live FD-backed capability
- rename the capability's realized-resource field to `.ref`
- update lifecycle tests and active design contracts
- omit compatibility aliases for the unshipped model names

Prompt-IO: ai/prompt-io/opencode/20260827T211115Z_d130431c_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi acc594b312 Compose WireGuard bindspace lifecycles
Add `open_wg_bindspace()` to enter one declared bindspace and an
ordered WireGuard interface stack as one async lifetime.

Deats,
- snapshot caller layer ordering before the first checkpoint
- enter interfaces outermost-first through `AsyncExitStack`
- unwind interfaces before releasing the namespace capability
- yield the live `BindspaceHandle` for endpoint allocation
- test mutable input and cancellation ordering with lifecycle fakes

Prompt-IO: ai/prompt-io/opencode/20260826T022434Z_2245f094_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 9bbf0d7808 Own WireGuard interface lifecycles
Add `open_wg_iface()` to create, configure and remove one WireGuard
interface inside a pinned bindspace through pyroute2.

Deats,
- validate listen/dial bearer policy before kernel side effects
- configure local addresses, private key, listen port and peers
- fill an omitted dial endpoint from the selected tunnel bearer
- clean partial synchronous failures before returning to Trio
- shield owned interface creation and teardown from cancellation

Prompt-IO: ai/prompt-io/opencode/20260826T003430Z_6dd39da0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 4c10ce2bd0 Model explicit WireGuard peers
Add process-local `WGPeerConfig` entries and make
`WGInterfaceConfig` own a unique peer tuple for listener and dial
provisioning.

Deats,
- carry peer public keys, allowed CIDRs and optional endpoints
- redact per-peer preshared keys while blocking wire encoding
- validate peer routes, endpoint ports and keepalive intervals
- reject duplicate peers before future kernel mutation
- support multi-peer listeners without overloading tunnel identity

Prompt-IO: ai/prompt-io/opencode/20260826T001442Z_dcdf4d82_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 5ec7cbaf98 Separate local WireGuard configuration
Add process-local `WGInterfaceConfig` for key material, interface
addresses and peer-routing policy required by future provisioning.

Deats,
- redact private and preshared keys from representation
- block config from default actor-IPC encoding via `ProcessLocal`
- validate keys, interface CIDRs, allowed CIDRs and bounded integers
- keep public endpoint and peer identity in `WGTunnelSpec`
- move allowed-IP policy out of the serializable tunnel declaration

Prompt-IO: ai/prompt-io/opencode/20260825T234631Z_b973e78c_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 2db2b35a4e Dispatch explicit bindspace lifecycles
Add serialized `BindspaceSpec.lifecycle` policy and dispatch it through
`open_bindspace()` without inferring ownership from transport role.

Deats,
- distinguish borrowed `attach` from owned `open` policy
- validate handle ownership against the declared lifecycle
- share policy-neutral FD pinning between both netns contexts
- reject unsupported lifecycle values before side effects
- exercise both dispatcher branches and owned cancellation cleanup

Prompt-IO: ai/prompt-io/opencode/20260825T191845Z_5b2a064a_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 0d16e17e44 Open owned network namespaces
Add `open_netns()` to create a named Linux netns through pyroute2,
pin its identity and yield an owned `BindspaceHandle`.

Deats,
- run synchronous creation and removal in Trio worker threads
- shield both privileged side effects from caller cancellation
- reuse `attach_netns()` to pin identity and manage the FD
- close the FD before removing the owned namespace
- fake privileged operations while testing ordering and cancellation

Prompt-IO: ai/prompt-io/opencode/20260825T190529Z_e1007547_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi d5c9945a52 Attach existing network namespaces
Add `attach_netns()` to pin a current or named Linux netns in a
borrowed `BindspaceHandle` without creating or entering it.

Deats,
- name the current-namespace default `CURRENT_NETNS`
- derive stable identity from the opened FD with `fstat()`
- open descriptors with `O_CLOEXEC` and close them on context exit
- constrain named lookup beneath the standard iproute2 run directory
- report field-specific validation and missing-resource errors

Prompt-IO: ai/prompt-io/opencode/20260825T045557Z_fdccfd7e_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi fc0049da18 Add bindspace capability models
Separate serializable `BindspaceSpec` and `BindspaceIdentity` values
from a process-local `BindspaceHandle` carrying FD and ownership
authority.

Deats,
- add global `ProcessLocal` wire guards for local handle structs
- derive valid kinds and ownership from their `Literal` aliases
- require a positive inode while keeping the mutable name optional
- pin supplied FDs to identity inodes with `fstat()`
- cover round trips, nested encoding and stale capabilities

Caught-during: review remediation
Found-via: `/run-tests` test_bindspace_handle_pins_local_capability

Prompt-IO: ai/prompt-io/opencode/20260822T042026Z_29141f0b_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi abfb5e3adf Expose tunnel namespaces on IPC endpoints
Make `TCPAddress` and `UDSAddress` explicitly satisfy
`Address.namespace`, then retain each original listener declaration
beside its peeled, resolved transport address.

Deats,
- remove `TunnelledAddress`'s attribute fallback
- add required `Endpoint.declared_addr` metadata
- report declaration namespaces in endpoint/server formatting
- preserve concrete `Endpoint.addr` for transport reflection
- cover plain and tunneled namespace visibility

Prompt-IO: ai/prompt-io/opencode/20260822T032520Z_d35c802b_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi ee3cc24657 Add explicit `verify_wg_peer()` inspection
Validate a declared tunnel key against one `pyroute2` snapshot
containing the iface's own key and configured peers.

Deats,
- share worker offload across all WireGuard key readers
- forward `WGTunnelSpec.iface` and `.netns` to the read
- reject malformed declarations before netlink I/O
- export the async helper and cover local, peer and absent keys
- replace multihost's `wg show` subprocess probe

Prompt-IO: ai/prompt-io/opencode/20260822T023226Z_59a8ecfd_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi e4dc8d2406 Sketch first-child `wgman` supervisor
Add a candidate Layer-C architecture where a private, eagerly
spawned manager owns pyroute2 and tunnel provisioning for a simple
WG-enabled actor tree.

Deats,
- overlap manager reconciliation with sibling process startup
- contain `AsyncWireGuard` in an infected-asyncio child
- limit requests and capabilities by bindspace security domain
- define readiness, crash, restart and teardown semantics
- retain pre-provisioned and multi-manager escape hatches

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi f671b589b7 Read WireGuard iface keys through `pyroute2`
Add async `read_wg_pubkey()` and `read_wg_peers()` helpers which
offload `WireGuard.info()` calls to a `trio` worker thread.

Deats,
- add the Linux-only `wg` extra and pin `pyroute2`
- pass `flags=0` so a read never creates a named netns
- normalize multipart replies, validate keys and de-dup peers
- always close the netlink client, including error paths
- test thread offload, netns forwarding and client cleanup

Prompt-IO: ai/prompt-io/opencode/20260821T233204Z_5d92595f_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:29 -04:00
Gud Boi 5b606ba1e1 Emit canonical tagged addresses
- Make `TCPAddress.unwrap()` emit `('tcp', host, port)` and
  `UDSAddress.unwrap()` emit `('unix', path)` while retaining the
  compatibility readers from the preceding change.

- Pass concrete TCP fields to Trio, compose multiaddrs from tagged
  values, and let `SpawnSpec` carry protocol-specific tuple shapes
  for validation by `wrap_address()`.

- Compare runtime, registry, bind, and tunnel addresses through
  canonical serialized forms and cover both TCP and UDS operation.

Prompt-IO: ai/prompt-io/opencode/20260820T033108Z_ba07e09d_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:16 -04:00
Gud Boi 68addd86fb Decode tagged transport addresses
- Define canonical `tcp` and `unix` tuple shapes while retaining
  legacy pair aliases as the emitted `UnwrappedAddress`.

- Dispatch tagged tuple/list payloads explicitly, accept `uds` as a
  Unix input alias, and preserve legacy TCP, UDS, and native IPv6
  readers.

- Cover tag aliases, msgpack-style lists, legacy payloads, and IPv6
  socket addresses before switching writers.

Prompt-IO: ai/prompt-io/opencode/20260820T033107Z_ba07e09d_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:16 -04:00
Gud Boi a753b1e89e Model bindspaces as scoped capabilities
Separate serializable bindspace declarations from live namespace
identity, FDs, ownership and teardown resources.

Require child namespace entry during spawn bootstrap, before actor
runtime initialization, then distinguish listen/dial provisioning and
owned/borrowed cleanup without encoding operation role into maddrs.

Prompt-IO: ai/prompt-io/opencode/20260820T021516Z_dfad66a0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:16 -04:00
Gud Boi f44dad362d Peel tunnels before `Endpoint` binding
Carry tunnel declarations through listener configuration, then strip
them immediately before constructing transport endpoints.

Also allocate random listener addresses from a contacted registry's
overlay, and prove a real TCP listener never stores the wrapper while
the source declaration retains its bindspace metadata.

Prompt-IO: ai/prompt-io/opencode/20260819T213145Z_f81fc5e5_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:16 -04:00
Gud Boi 9c4b3a60ee Peel tunnels before `Channel` connects
Retain tunnel annotations through address declaration, then hand only
the bindable overlay to exact-type transport lookup and dialing.

Broaden `Channel.from_addr()` and `_connect_chan()` inputs accordingly,
and cover plain plus tunnelled TCP dispatch arguments.

Prompt-IO: ai/prompt-io/opencode/20260819T213144Z_f81fc5e5_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:16 -04:00
Gud Boi 78d6f3b2bc Use discovery's `wg` parser in examples
Drop the example-local address struct and hand-rolled single-tunnel
parser now that discovery owns the production implementation.

Keep only the explicit `wg(8)` peer probe in the multihost helper,
and update the examples and plan for nested parsing, packaged codec
dependencies and tractor-owned bindspace provisioning.

Prompt-IO: ai/prompt-io/opencode/20260818T075031Z_dd02c7c0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:16 -04:00
Gud Boi 68ca651f67 Support nested `wg` maddrs
Teach discovery to preserve WireGuard bearer and identity metadata
around a bindable TCP overlay.

Deats,
- encode `wg(8)` keys as strict 32-byte multibase values
- peel nested stacks with `Multiaddr.decapsulate_code()` and compose
  them with `.encapsulate()` instead of splitting strings
- integrate wrappers with `parse_maddr()`, `mk_maddr()`,
  `wrap_address()` and `parse_endpoints()`
- pin the unreleased py-multiaddr#108 codec in package metadata
- cover exact round trips, nesting, bad grammar and missing codecs

Prompt-IO: ai/prompt-io/opencode/20260818T075031Z_dd02c7c0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:16 -04:00
Gud Boi 41e7bb61c6 Add `TunnelledAddress` wrapper primitives
Introduce the first layer-A address type from the `wg` bindspace
plan without treating a transparent tunnel as a `MsgTransport`.

Deats,
- add frozen `WGTunnelSpec` and `TunnelledAddress` structs which
  delegate proto identity, bindspace, validity and wire
  serialization to their overlay
- add `strip_tunnels()` and `tunnels_of()` for nested wrappers
- recognize wrappers in `is_wrapped_addr()` while keeping them out
  of `_address_types`
- cover delegation, namespace fallback and nested peeling semantics

Also,
- widen `Address.namespace` ids for named netns
- export the new discovery API
- clarify that tractor's layer-C bindspace lifecycle may provision
  the kernel-owned bearer without making it a `MsgTransport`

Prompt-IO: ai/prompt-io/opencode/20260818T021729Z_d9a6e2e9_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-29 22:00:16 -04:00
Gud Boi 4730b4c9cf Retract the hand-rolled tunnel peeler from plan-03
§3.2 specced a pure fn `_peel_tunnel_segs(proto_names) ->
(bearer_names, tunnel_specs, overlay_names)` to split a maddr at
its tunnel seg. It should never be written: `py-multiaddr` ships
that whole surface already and the plan simply missed it, even
though gh #443's 2nd bullet links the README sections in
question.

Replaced w/ a ⚠️ CORRECTION carrying the verified API table
(`.decapsulate_code(P_WG)` for the bearer, `.split()`/`.join()`
for a seg tail, `.value_for_protocol()` to read a value,
`.encapsulate()` to recompose) plus *why* it works on an infix
`/wg/` seg: the cut is by proto-code, never by matching an addr
value, and the key seg has no addr of its own.

Also,
- adopt `bearer`/`overlay` as the role names throughout, and say
  plainly why not `inner`/`outer` — the call-stack reading of
  "inner" is the exact opposite of the encapsulation one.
- warn that `value_for_protocol('ip4')` on a full tunnelled
  maddr silently yields the *bearer's* host; only call it on a
  peeled sub-maddr.
- note nesting (wg-in-wg) falls out of `.decapsulate_code()`
  cutting at the *last* occurrence, so peel repeatedly rather
  than recursing through a bespoke splitter.
- `mk_maddr()` for `TunnelledAddress` is `.encapsulate()`
  composition, not `str` building.
- README: drop the "degrades to a plain segment split" para,
  since that path is gone — no codec now means one actionable
  raise.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi c3d8e28260 Peel `wg` maddrs w/ `py-multiaddr`'s own tunnel API
`py-multiaddr` already ships the entire tunnel compose/peel
surface and this module was reimplementing it — a raw
`maddr.split('/')` plus index arithmetic, sitting directly under
a comment congratulating itself for not hand-rolling a parser.
Same NIH trap gh #429 existed to close, just one layer up. The
API was linked from gh #443's own 2nd bullet the whole time.

So every cut now goes through the real thing,

| need | API |
| --- | --- |
| isolate the bearer | `.decapsulate_code(P_WG)` |
| per-seg maddrs | `.split()` |
| rejoin a seg tail | `Multiaddr.join()` |
| read the key | `.value_for_protocol('wg')` |
| recompose | `.encapsulate()` |

`.decapsulate_code()` turns out to handle the infix `/wg/` seg
cleanly *because* it cuts on proto-code and never tries to match
an addr value — the key seg has no addr of its own, which was
the exact thing I'd assumed would need bespoke handling.

Deats,
- rename the role fields `inner`/`inner_proto` ->
  `overlay`/`overlay_proto`, matching `py-multiaddr`'s
  encapsulation model (earlier segs wrap later ones) and #443's
  owner table. `inner` collided head-on w/ call-stack `inner`,
  where it reads as higher-up + later-called, while here the
  encapsulated addr is bound *first* and sits deeper.
- drop `_segments()` and its degraded hand-split path entirely.
  W/o the codec there's now one actionable `RuntimeError`
  instead of a silent downgrade, superseding the swallow fix in
  7d6e7955.
- add `.as_multiaddr()` so callers can stay in `Multiaddr` land;
  `.maddr` is now just `str()` of it.
- accept `str|Multiaddr` on the way in.
- carry `bearer_ip`/`overlay_ip` so a v6 stack re-renders as v6
  — the old `.maddr` hardcoded `/ip4/` and would silently
  mangle it.
- both host scripts follow the rename to `.overlay`.

⚠️ `value_for_protocol('ip4')` on a *full* tunnelled maddr
silently returns the **first** match, i.e. the bearer's host, so
it's only ever called here on an already-peeled sub-maddr.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi d83e9c90d7 Update `wg` docs for the merged py-multiaddr#108
it lands" framing in plan-03 and the example README was stale in
both directions: the branch pin is obsolete, yet you still can't
just `pip install multiaddr`.

Deats,
- §3.2's grammar table is now re-verified against the upstream
  merge (`f86519da`) rather than only `baudco@wg_support` in a
  throwaway venv. Also notes the codec enforces a 32-byte key,
  so a truncated one is a `StringParseError` and not a silently
  mangled parse.
- §1 says merged-but-unreleased; the still-open work is spec
  registration (py-multiaddr#107 + gh #483).
- §3.4 swaps "pin the branch" for the `[tool.uv.sources]` `rev`
  pin, and fixes the `_have_wg_maddr_proto()` recipe it
  suggested — probing w/ `Multiaddr('/wg/uAAAA')` now ALWAYS
  raises bc the codec wants 32B, i.e. that feature-detect would
  report `False` even w/ the proto perfectly well known.
- risk table row goes "#108 not merged" -> "merged but
  unreleased".
- example README: `uv sync` alone now suffices bc of the pin;
  documents the 32B check and points at
  `_have_wg_maddr_proto()` as the gate.

The one surviving `baudco` mention is deliberate, it records
where the grammar was *first* verified.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi f086664b3e Fix silently-corrupt keys in `parse_wg_maddr()`
`_segments()` called `Multiaddr(maddr)` purely to validate, then
swallowed every failure under `except Exception: pass`. That was
harmless pre-#108 — w/o a `wg` codec there was nothing to
validate — but now that the codec is pinned in, the swallow is
load-bearing and disabled: a malformed key sails past validation
into `wg8_pubkey()`, which happily emits a corrupt b64 str, and
the returned struct then fails its own `.maddr` round-trip. No
raise, just quietly wrong output.

Deats,
- add `_have_wg_maddr_proto()`, the gate plan-03 already
  referenced but which never actually existed. Impl'd as
  `protocols.protocol_with_name('wg')` under
  `except ProtocolNotFoundError` and cached in a mod global,
  same shape as the TIPC plan's `is_tipc_available()`.
- only validate when that gate is `True`, and let
  `StringParseError` propagate — a maddr which doesn't parse
  must NOT reach `wg8_pubkey()`.
- keep the degraded split for a pre-#108 install, now w/ an
  explicit `XXX` naming the validation you give up.

So parsing stays pure but becomes total-or-raises. Our own
`ValueError`s (missing `/wg/` seg, bare tunnel w/o an overlay
ep) are unaffected, as is the `wg(8)` b64 round-trip.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 3ba93468ef Pin `multiaddr` to the merged `wg` codec rev
py-multiaddr#108 (the `/wg/u<key>` maddr proto) merged upstream
on 2026-07-28 as `f86519da`, but ships in no release yet — the
latest `0.2.0` predates it by ~4 months and carries no `wg`
codec at all. So `examples/multihost/wg_lan/` can't parse its
own maddrs off PyPI.

Pinned by `rev` and not `branch` so CI stays reproducible. Note
the lock now records the git source *instead of* the `>=0.2.0`
specifier, i.e. the dep floor above is fully overridden for as
long as this pin lives.

TODO, drop the pin (and bump that floor) the moment a release
carries the codec; the only consumer is the `wg_lan` example
set.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 6e2edbd0d7 Log prompt-io for the tpt-backend planning arc
One record covering all 9 commits on this branch, per the NLNet
generative-AI policy and the existing `ai/prompt-io/claude/`
convention.

Uses diff-ref mode for both the plan docs and the example code
(`git diff main..ng_tpts_planning -- <path>`) rather than
duplicating content already in `git log -p`. Kept verbatim in
the `.raw.md`: the four verified findings (trio's
family-agnostic `SocketStream`/`SocketListener`, the round-trip
table proving `/wg/` is infix, the proto-key `UnwrappedAddress`
rationale, and `setns(2)`'s per-thread reality), since those are
reasoning rather than diffable output.

`## Human edits` records that the steering here was substantial
and mid-session rather than post-hoc: two model claims about wg
maddr semantics were challenged and retracted (incl. in an
already-posted issue comment), and the proto-key +
netns-as-runtime-config framings were human-directed. Also notes
the one model-initiated correction — a pre-publication
self-review that downgraded the `uniffi`/asyncio thesis and the
TIPC duplicate-binder claim to explicitly-flagged assumptions.

Prompt-IO: ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 247bf5be15 Move the `wg_lan` examples under `examples/multihost/`
`tests/test_docs_examples.py` walks `examples/` **recursively**
and subproc-runs every collected file asserting `rc == 0`. Ran
its exact filter against the tree: all 4 of our files were being
collected — including `README.md`, since the filter never checks
the extension, so CI would have literally tried `python
README.md`. These need a real second host + a live `wg` tunnel,
so they can't ever satisfy that gate.

`'multihost' not in p[0]` is already in the test's exclusion
list w/ no dir yet using it, so this is a pure `git mv` — zero
test changes — and it's what the exclusion was plainly there
for. Collection drops 24 -> 20 files, 0 of them ours.

Also records *why* in the two places someone would look before
adding the next one: a callout at the top of the example README
and a note on plan 03's §3.4 deliverables. Anything needing a
second host or live tunnel goes under `examples/multihost/`.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 126405b785 Add a `wg`-tunnelled 2-host example set
Re-renders gh #482's examples w/ the corrected (infix) maddr
grammar, as the "layer A" slice of the wg plan: declarative
maddrs only, tunnel pre-provisioned out-of-band, zero runtime
changes.

- `wg_maddr.py`: a `frozen=True` `msgspec.Struct` addr carrying
  `bearer`/`peer_pubkey`/`inner` (+ `inner_proto`), a `.maddr`
  property that re-renders the canonical form, and pure
  `mb_pubkey()`/`wg8_pubkey()`/`parse_wg_maddr()`. The parser
  rejects #482's inverted suffix form w/ an actionable error and
  stays **side-effect free** — `verify_wg_peer()` is a separate,
  explicitly impure step the caller composes, never something a
  parse path shells out to.
- `host_a_srv.py`/`host_b_client.py`: the two-host runs, passing
  only `addr.inner` into `open_nursery()`/`open_root_actor()`,
  which is the whole point — the bearer + key layers are already
  established before any bind happens.
- `README.md`: the grammar + the 3-owners table, the `#108`
  branch install line, tunnel setup, and a "what changed vs
  #482" section enumerating the corrections.

Runnable-shaped but **not yet run against a live tunnel**; that's
next, and the reason these sit on the planning branch rather than
in `examples/` proper. `_segments()` marks its stopgap for when
the `wg` codec isn't installed.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 8f1e69693f Fix the `wg` maddr grammar, `/wg/` is *infix*
The prior revision (and gh #482's examples) had it as a suffix,
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. Wrong: verified against
`baudco/py-multiaddr@wg_support` (py-multiaddr#108) installed in
a throwaway venv, the canonical form is

  /ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616

where segs *before* `/wg/` are the **bearer** — the underlay
`(ip, udp-port)` `wg(8)` itself listens on (`ListenPort`), per
the codec docstring's own example — and segs *after* are the
**overlay** ep, the only part we ever bind. The suffix form does
parse, which is why it slipped through, but it's semantically
inverted: overlay addr where the bearer belongs, `tcp` where
wg's `udp` goes, and no overlay ep declared at all.

Records the observed `[p.name for p in m.protocols()]` lists so
the `match` can be written against fact, and replaces the
"composed vs not" framing w/ what's actually the design axis:
three parts, three **owners** — bearer bound by the kernel via
`wg-quick`/`pyroute2`, `/wg/u<key>` bound by nothing (it's an
identity, verified out-of-band), overlay bound by our
`IPCServer` as `.inner`. `_peel_tunnel_segs()` correspondingly
grows a 3rd return, splitting *at* the tunnel seg so nested
tunnels fall out for free.

Also hoists the netns conclusion to the top of §5.3 where it
can't be missed: netns is a **runtime-level config API, not an
actor-app-code one**. It's a spawn/boot-time input alongside
`enable_transports`/`tpt_bind_addrs`, deliberately w/ no
`await actor.enter_netns(...)`, because `setns(2)` neither moves
already-created sockets nor applies beyond the calling thread —
so a mid-life API would silently leave the IPC server bound in
the old ns.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi c6267d08a2 Proto-key the unwrapped-addr form in the plans
Shape-matching in `wrap_address()` doesn't survive 4 backends and
the plans were papering over it: TIPC's natural unwrapped form is
a `(str, int)`, indistinguishable from `TCPAddress`, and iroh's
is a `(str, str)`, which the *existing* UDS case
(`case (_, filename) if type(filename) is str`) already swallows.

So the contract doc (§1.1) now carries the conclusion as a
**recommended prerequisite for all three backends**: make the
unwrapped form carry an explicit proto-key spelled with the
`multiaddr` protocol name — `('tcp', host, port)`,
`('unix', path)`, `('tipc', stype, inst, scope)`. `wrap_address()`
then collapses from an order-sensitive `match` to
`_address_types[addr[0]]` and the whole collision class stops
existing, while the on-wire form finally agrees w/
`mk_maddr()`/`parse_maddr()` instead of being an independent
invention.

Two consequences spelled out: it's a wire-format change
(`SpawnSpec`, `_root_mailbox`, `_registry_addrs`) + every fixture
+ downstream config, so it wants its own migration commit landed
*before* any new backend; and it's the moment to stop handing raw
tuples to users at all — `Address` becomes the public currency
and `UnwrappedAddress` an internal serialization detail, the same
discipline `ipaddress` uses (you pass `IPv4Address`, never a
4-tuple).

Plan 01 §2.2 is rewritten to match and to explicitly **retract**
its own earlier `('tipc:<stype>:<scope>', instance)` self-tagging
prefix hack — it keeps `wrap_address()` order-sensitive and does
nothing for the iroh/UDS collision, so the doc says don't
resurrect it. Registration checklist item 4 likewise becomes "do
the migration first, then this is a one-line `_address_types`
entry".

Also seeds a `/tipc` multiaddr-spec submission as a follow-up,
mirroring the `wg` track (multiformats/py-multiaddr#107/#108 + gh

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 048b112a4c Index the tpt-backend plans w/ a README
Landing page for `ai/tpt-backends/`: points at the contract spec
as required first reading, tables the 3 plans against their
issues/deps/size, and states the landing order + why.

Deats,
- TIPC first as the cheap proof the table-registration story
  generalizes to a genuinely new proto (stdlib-only, and
  `trio`'s sock wrappers are family-agnostic).
- `wg` layer-A next since it's deployable-today doc/example work.
- QUIC last, gated on its own prep PR.
- notes that plans 01 and 02 both want the same
  `Address.rebind_from_sockname` gate, so whichever lands first
  ships it.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 93aa1e25af Add `wg`-as-nested-bindspace plan doc
Plan doc for gh #482 + the tunnelled-maddr item of #443. Pushes
back on the framing that `wg` is a tpt: it's transparent to
`socket(2)`, so it belongs as a *bindspace* — a scoped
`@acm`-managed net ctx that an existing L4 tpt binds *inside* —
and it's what finally implements the long-spec'd (never
implemented) `Address.namespace`.

Deats, 3 independently-shippable layers,
- A) declarative: commit #482's examples, teach `parse_maddr()`
  the `/…/wg/u<key>` suffix -> a `TunnelledAddress` wrapper whose
  `.proto_key`/`.unwrap()` delegate to `.inner` so nothing new
  crosses the wire and every existing table lookup keeps working.
- B) swap the `subprocess.run(['sudo', 'wg', 'show'])` shelling
  for `pyroute2`. Default to `trio.to_thread` around the sync API
  (these are one-shot ops at bind/teardown, never hot-path), w/
  sans-io codecs + a trio `AF_NETLINK` sock as the follow-up for
  the read paths. Explicitly forbids dragging `trio-asyncio` in.
- C) `open_bindspace()`/`open_netns()`/`open_wg_iface()` `@acm`s
  folded w/ an `AsyncExitStack`, + filling in the
  `# !TODO, always be ns aware!` placeholder already sitting in
  `Endpoint.pformat()`.

Also flags the subtlest bug in the whole thing: `setns(2)` is
*per-thread*, so a `pyroute2` query issued via `trio.to_thread`
lands in the *original* netns. Test-first, per usual.

Further, designs for the generalization (`TunnelSpec` union +
`match` dispatch) while only implementing `wg`+netns, and calls
out `veth`-in-netns as the better *first* one bc it makes a
fully self-contained two-"host" integration test possible w/o
`wg` at all.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 8f525d20e9 Add `QUIC`-via-`iroh` tpt-backend plan
Plan doc for gh #353. Picks `iroh` (the `uniffi` FFI pkg) over
`aioquic`/`quiche` bc node-id addressing + hole-punching + relay
fallback is the whole point; `aioquic` stays documented as the
fallback since ~90% of the adapters here are reusable against a
sans-io core.

Deats,
- the layering: iroh `Endpoint` per actor, `Connection` per peer
  (pooled via `trionics.maybe_open_context()`, not a hand-rolled
  cache), one bi-stream per `Channel`. 4-byte prefix framing
  stays so `MsgpackTransport` is untouched.
- `_uniffi_trio.py`: uniffi only uses `asyncio` as the executor
  for its rust-future poll loop, so a ~40-line
  `TrioToken.run_sync_soon()` bridge replaces it. Spells out the
  real hazards — strong ref on the `ctypes` trampoline, poll-code
  propagation, and a *bounded* shielded cancel-drain so a wedged
  rust future can't make an actor un-cancellable.
- `IrohAddress` w/ ALPN as the `.bindspace`, the `(str, str)`
  unwrapped form's collision w/ the UDS match-case, and why
  `get_root()` needs a persisted secret key -> a lazy
  `default_lo_addrs()` + a pure-getter/explicit-setter split.
- `QuicMsgStream(trio.abc.HalfCloseableStream)` +
  `QuicListener(trio.abc.Listener)`, incl. the exact
  EOF/reset/use-after-close semantics `_transport.py` already
  match-cases on, and hanging the acceptor tasks off the
  existing `Endpoint.listen_tn`.
- a prep-PR boundary: annotation widening, the shared
  `rebind_from_sockname` gate and a `tpt_key`-based
  `transport_from_stream()` dispatch, all landable w/ tcp/uds as
  the only backends.

Further, notes this is our first tpt w/ real transport security
+ peer auth, so an inbound node-id allowlist hook belongs here —
and that it says nothing about the other backends.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 6a46f1f0f9 Add `TIPC` tpt-backend impl plan
Plan doc for gh #378, the cheapest new backend we can add: it's
stdlib-only (CPython ships `AF_TIPC` + 23 `TIPC_*` consts) and
per the contract doc `trio`'s stream/listener wrappers don't care
about the addr family, so `MsgpackTransport` framing and
`trio.serve_listeners()` are reused verbatim.

Deats,
- `TIPCAddress` as a *service name* `(type, instance)` w/ scope
  as the `.bindspace`; `bind()` publishes the singleton
  name-range, peers `connect()` by name and the kernel resolves
  + load-balances. I.e. registration/lookup for free, no
  registrar in the loop.
- the self-tagging `('tipc:<stype>:<scope>', instance)` unwrapped
  form + why it must be match-ordered before `TCPAddress`'s.
- `get_random()` via a blake2b digest of the actor id (there's no
  `port=0` analogue) and the silent-crosstalk risk that follows:
  TIPC *allows* dup binders and round-robins, so a collision
  doesn't `EADDRINUSE`, it cross-talks.
- an `Address.rebind_from_sockname` ClassVar to opt out of
  `Endpoint.start_listener()`'s `getsockname()` reconcile, which
  for TIPC always returns a port-id, never the bound name.
- the `TIPC_TOP_SRV` topology-service subscription as an `@acm`
  yielding a chan of typed name-table events — push-based
  register/dereg, the real "end game cluster proto" bit.
- commit sequencing, hard capability gating (`modprobe tipc`;
  bare `AF_TIPC` is `EAFNOSUPPORT` on a stock box), CI matrix
  notes, risks + follow-up seeds.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Gud Boi 7c8dd986da Add the `.ipc` tpt-backend contract spec
First doc of a new `ai/tpt-backends/` set: the normative
description of what a `tractor` tpt backend *is* as of `main`,
written so the 3 sibling plans (TIPC, QUIC, `wg`) can be worked
independently (by another model/provider) w/o design drift.

Deats,
- the backend duck-type as empirically derived from
  `_tcp.py`/`_uds.py`: the `Address` protocol surface, the
  mod-level `start_listener()`/`close_listener()` pair and
  `Msgpack<Proto>Stream(MsgpackTransport)`.
- the ONE reflection you can't break:
  `Endpoint.start_listener()` resolves the tpt mod via
  `inspect.getmodule(self.addr)`, so an `Address` type and its
  listener fns MUST live in the same mod.
- a 10-item registration checklist (`_address_types`,
  `_key_to_transport`, `_addr_to_transport`, `wrap_address()`
  match-cases, `TransportProtocolKey`, maddr tables, ..) incl.
  the import-time `_default_lo_addrs` trap.
- where the `trio.SocketListener` assumption is *actually*
  load-bearing (just the `getsockname()` reconcile) vs. merely
  annotated.
- the handshake/discovery invariants a new backend inherits,
  dep policy (extras + import-laziness per the #470 boot-latency
  budget), `--tpt-proto` harness plumbing and code style.

Also, records a verified finding the plans lean on hard:
`trio.SocketStream`/`SocketListener` are addr-*family* agnostic
— the only ctor checks are "is a trio sock" + `SOCK_STREAM` (+
an `OSError`-suppressed `SO_ACCEPTCONN`) — so any `SOCK_STREAM`
family CPython can make drops into the existing
`trio.serve_listeners()` path unmodified.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-29 22:00:06 -04:00
Bd f40efdd8da
Merge pull request #490 from goodboy/wkt/fix_broadcast_lag_count
Harden broadcast fan-out ownership and failures
2026-08-28 14:31:52 -04:00
Gud Boi 6429e6a515 Reject concurrent `BroadcastReceiver.receive()` calls
Two tasks could receive through one `BroadcastReceiver` and race its
single `_wait_scope`. A later call could replace the scope, leaving
the first blocked through `.aclose()`.

Track the active `trio.lowlevel.Task` per receiver and reject an
overlapping call with `trio.BusyResourceError`. Include the owner's
task name and repr in the error, then clear ownership in `finally`.

Define each receiver as one logical subscription cursor. Concurrent
consumers must use separate `.subscribe()` receivers so each retains
an independent cursor and lag policy. Strict mode reads every retained
value or raises `Lagged`; `raise_on_lag=False` explicitly permits
drops.

Cover the regression with a checkpoint loop synchronized on
`recv_ready`; explain the interleaving inline, assert the error names
the active consumer, then release the source and prove the original
receive remains usable.

Prompt-IO: ai/prompt-io/opencode/20260828T014247Z_d06ee34b_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 22:11:55 -04:00
Gud Boi d06ee34b83 Clean broadcast cancellation diagnostics
Bound `BroadcastState.cancelled` entries to receiver progress,
terminal state and resource lifetime instead of retaining completed
`Task`s indefinitely.

Deats,
- make EOC durable so awakened peers never re-enter a closed source.
- close root broadcasters during explicit `MsgStream` and
  `LinkedTaskChannel` teardown without re-entrant EOC closure or
  breaking `MsgStream.aclose()` overrides.
- reject non-positive fan-out retention capacity before constructing
  an unusable zero-length queue.
- cover child/root cancellation cleanup, terminal peer wakeups,
  wrapper teardown, subclass compatibility and zero-buffer rejection.

Prompt-IO: ai/prompt-io/opencode/20260813T181901Z_a2e0df4b_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 21:18:00 -04:00
Gud Boi ff4391669b Expose stream subscriber lag policy
`MsgStream.subscribe()` and `LinkedTaskChannel.subscribe()` omitted
`BroadcastReceiver.raise_on_lag`, forcing downstream consumers to
mutate a private receiver attribute when overruns were acceptable.

Add `raise_on_lag` to both public wrappers. The first subscription
sets the irreversible root broadcaster's policy, while every child
selects its own strict or warn/drop/resume behavior independently.

Document both fan-out APIs. Cover policy forwarding plus real IPC
and infected-asyncio paths.

Prompt-IO: ai/prompt-io/opencode/20260812T213117Z_51185487_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 21:18:00 -04:00
Gud Boi a5259a8ee4 Isolate `BroadcastReceiver.aclose()` wakeups
Closing any subscriber set the shared `recv_ready` event, even when
another receiver owned the source read. Waiting peers then looped
until an idle source produced another value.

Give each receiver private source-read and peer-wait cancellation
scopes. Closing a waiting peer interrupts only that peer. Closing
the source owner discards post-close source outcomes and then wakes
peers for a clean ownership handoff.

Keep outer task cancellation as `trio.Cancelled`; only explicit
receiver close maps either private scope's cancellation to
`ClosedResourceError`. Assert that scope cancellation implies the
receiver is closed and document the source-owner key check.

Prompt-IO: ai/prompt-io/opencode/20260812T150027Z_c2a6ccef_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 21:18:00 -04:00
Gud Boi d6da0e7a22 Wake broadcast peers on shared receive failures
Only `EndOfChannel` and direct cancellation woke tasks waiting
behind the subscriber which owned the underlying receive. Any other
failure cleared `BroadcastState.recv_ready` while peers remained
blocked on its unreachable event.

Publish ordinary receive exceptions as terminal broadcast state.
The owner keeps the original failure while peers drain retained
values and then raise `BroadcastReceiveError` from that cause. Also
wake peers on process-control exits without retaining them as state.

Document the public owner/peer contract and cover current, late and
control-flow subscribers with deterministic bounded regressions.

Prompt-IO: ai/prompt-io/opencode/20260812T030608Z_1095e7f7_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 21:18:00 -04:00
Gud Boi 5afd834732 Fix `BroadcastState.statistics()` queue counts
`BroadcastState.subs` stores each receiver's next unread deque
index, but `.statistics()` exposed that index as a queue length. A
caught-up receiver looked correct by accident while every queued
count was one short.

Convert cursors to retained, receivable counts and clamp lagged
receivers to the current queue length. Also avoid deprecated
`trio.Event` truthiness when reporting waiter counts.

Cover caught-up, queued, lagged and real-event states using actual
broadcast sends and receives.

Prompt-IO: ai/prompt-io/opencode/20260812T012324Z_06c4af17_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 21:18:00 -04:00
Gud Boi 206b0354c5 Fix `BroadcastReceiver` lag counts
`BroadcastReceiver.receive_nowait()` treated `seq` as a deque index
but subtracted `BroadcastState.maxlen` without counting the first
invalid index. A one-slot queue thus claimed it dropped zero values
after its subscriber missed one.

Include that first displaced value in the count. Preserve the
existing Tokio-style reset to the oldest retained item.

Also, cover exact loss reporting and recovery for one- and
three-slot retention windows.

Prompt-IO: ai/prompt-io/opencode/20260811T233833Z_7cbd64ee_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 21:18:00 -04:00
Bd 11dc534a5d
Merge pull request #510 from goodboy/wkt/ai_skillz_run_tests_landing
`ai.skillz` multi-harness deploy (and a proper `/run-tests` landing)
2026-08-27 20:25:14 -04:00
Gud Boi 24e55e918e Document canonical `/run-tests` deployment
Link Tractor's project reference to the immutable canonical skill
and its repository, then show the exact hybrid deployment command.

Also clarify the Prompt-IO record's commit boundaries and PR #510
review direction.

Review: PR #510 (copilot-pull-request-reviewer)
https://github.com/goodboy/tractor/pull/510#pullrequestreview-5046306236

Prompt-IO: ai/prompt-io/opencode/20260822T022526Z_5562fd9a_prompt_io.md

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-27 19:47:31 -04:00
Gud Boi c9cfce441a Harden `run-tests` signal and reaper guidance
Prevent unsafe harness guidance from sending SIGUSR1 before
`stackscope` installs its handler or using shared-memory scans on
unsupported platforms.

Deats,
- verify `stackscope` in the pytest env before signaling.
- target pytest by default and require a separately verified handler
  before signaling subactors.
- keep `tractor-reap --shm*` guidance to Linux and FreeBSD.
- pin staged Prompt-IO provenance to `eb3c99c9` and record the
  PR #510 review direction.

Prompt-IO: ai/prompt-io/opencode/20260822T022526Z_5562fd9a_prompt_io.md

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-27 18:52:16 -04:00
Gud Boi 7b89995089 Ignore deployed `ai.skillz/` state
Record generated ignore blocks for deployed skills, commands, and
runtime artifacts so direct symlink deployments remain status-clean.

(this commit msg was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-08-27 18:34:49 -04:00
Gud Boi eb3c99c9fd Correct `run-tests` harness guidance
The migration reference overstated cleanup safety and omitted
current environment, platform, debugger and timeout behavior.

- document runtime env overrides and the actual CI matrix
- distinguish interactive debugger capture from hang diagnosis
- describe Trio-aware and SIGALRM timeout guards separately
- expose reaper platform limits and dry-run descendant expansion
- restore exact change-to-test mappings and randomized-port wording
- drop unrelated Taken and broad command ignores from the migration

Prompt-IO: ai/prompt-io/opencode/20260822T022526Z_5562fd9a_prompt_io.md

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-27 18:34:48 -04:00
Gud Boi 11ae5171d1 Use canonical `/run-tests` skill
Move Tractor's project-specific test knowledge into the local harness
reference and let `ai.skillz` own shared execution and safety policy.

Deats,
- document current Python envs, pytest opts, backends, and transports.
- replace broad registry and process cleanup rules with scoped checks.
- record capture, diagnostics, reaper, and expected-outcome caveats.
- ignore canonical Claude and OpenCode links plus Taken runtime state.

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-27 18:34:48 -04:00
125 changed files with 13755 additions and 787 deletions

View File

@ -1,632 +0,0 @@
---
name: run-tests
description: >
Run tractor test suite (or subsets). Use when the user wants
to run tests, verify changes, or check for regressions.
argument-hint: "[test-path-or-pattern] [--opts]"
allowed-tools:
- Bash(python -m pytest *)
- Bash(python -c *)
- Bash(python --version *)
- Bash(UV_PROJECT_ENVIRONMENT=py* uv run python *)
- Bash(UV_PROJECT_ENVIRONMENT=py* uv run pytest *)
- Bash(UV_PROJECT_ENVIRONMENT=py* uv sync *)
- Bash(UV_PROJECT_ENVIRONMENT=py* uv pip show *)
- Bash(git rev-parse *)
- Bash(ls *)
- Bash(cat *)
- Bash(jq * .pytest_cache/*)
# process inspection + SIGINT-first cleanup ladder (see
# the zombie-actor pre-flight / teardown steps below).
- Bash(ss *)
- Bash(pgrep *)
- Bash(pkill *)
- Bash(sleep *)
- Bash(rm -f /tmp/registry@*.sock)
- Read
- Grep
- Glob
- Task
- AskUserQuestion
---
Run the `tractor` test suite using `pytest`. Follow this
process:
## 1. Parse user intent
From the user's message and any arguments, determine:
- **scope**: full suite, specific file(s), specific
test(s), or a keyword pattern (`-k`).
- **transport**: which IPC transport protocol to test
against (default: `tcp`, also: `uds`).
- **options**: any extra pytest flags the user wants
(e.g. `--ll debug`, `--tpdb`, `-x`, `-v`).
If the user provides a bare path or pattern as argument,
treat it as the test target. Examples:
- `/run-tests` → full suite
- `/run-tests test_local.py` → single file
- `/run-tests test_registrar -v` → file + verbose
- `/run-tests -k cancel` → keyword filter
- `/run-tests tests/ipc/ --tpt-proto uds` → subdir + UDS
## 2. Construct the pytest command
Base command:
```
python -m pytest
```
### Default flags (always include unless user overrides):
- `-x` (stop on first failure)
- `--tb=short` (concise tracebacks)
- `--no-header` (reduce noise)
### Path resolution:
- If the user gives a bare filename like `test_local.py`,
resolve it under `tests/`.
- If the user gives a subdirectory like `ipc/`, resolve
under `tests/ipc/`.
- Glob if needed: `tests/**/test_*<pattern>*.py`
### Key pytest options for this project:
| Flag | Purpose |
|---|---|
| `--ll <level>` | Set tractor log level (e.g. `debug`, `info`, `runtime`) |
| `--tpdb` / `--debug-mode` | Enable tractor's multi-proc debugger |
| `--tpt-proto <key>` | IPC transport: `tcp` (default) or `uds` |
| `--spawn-backend <be>` | Spawn method: `trio` (default), `mp_spawn`, `mp_forkserver` |
| `-k <expr>` | pytest keyword filter |
| `-v` / `-vv` | Verbosity |
| `-s` | No output capture (useful with `--tpdb`) |
### Common combos:
```sh
# quick smoke test of core modules
python -m pytest tests/test_local.py tests/test_rpc.py -x --tb=short --no-header
# full suite, stop on first failure
python -m pytest tests/ -x --tb=short --no-header
# specific test with debug
python -m pytest tests/discovery/test_registrar.py::test_reg_then_unreg -x -s --tpdb --ll debug
# run with UDS transport
python -m pytest tests/ -x --tb=short --no-header --tpt-proto uds
# keyword filter
python -m pytest tests/ -x --tb=short --no-header -k "cancel and not slow"
```
## 3. Pre-flight: venv detection (MANDATORY)
**Always verify a `uv` venv is active before running
`python` or `pytest`.** This project uses
`UV_PROJECT_ENVIRONMENT=py<MINOR>` naming (e.g.
`py313`) — never `.venv`.
### Step 1: detect active venv
Run this check first:
```sh
python -c "
import sys, os
venv = os.environ.get('VIRTUAL_ENV', '')
prefix = sys.prefix
print(f'VIRTUAL_ENV={venv}')
print(f'sys.prefix={prefix}')
print(f'executable={sys.executable}')
"
```
### Step 2: interpret results
**Case A — venv is active** (`VIRTUAL_ENV` is set
and points to a `py<MINOR>/` dir under the project
root or worktree):
Use bare `python` / `python -m pytest` for all
commands. This is the normal, fast path.
**Case B — no venv active** (`VIRTUAL_ENV` is empty
or `sys.prefix` points to a system Python):
Use `AskUserQuestion` to ask the user:
> "No uv venv is active. Should I activate one
> via `UV_PROJECT_ENVIRONMENT=py<MINOR> uv sync`,
> or would you prefer to activate your shell venv
> first?"
Options:
1. **"Create/sync venv"** — run
`UV_PROJECT_ENVIRONMENT=py<MINOR> uv sync` where
`<MINOR>` is detected from `python --version`
(e.g. `313` for 3.13). Then use
`py<MINOR>/bin/python` for all subsequent
commands in this session.
2. **"I'll activate it myself"** — stop and let the
user `source py<MINOR>/bin/activate` or similar.
**Case C — inside a git worktree** (`git rev-parse
--git-common-dir` differs from `--git-dir`):
Verify Python resolves from the **worktree's own
venv**, not the main repo's:
```sh
python -c "import tractor; print(tractor.__file__)"
```
If the path points outside the worktree, create a
worktree-local venv:
```sh
UV_PROJECT_ENVIRONMENT=py<MINOR> uv sync
```
Then use `py<MINOR>/bin/python` for all commands.
**Why this matters**: without the correct venv,
subprocesses spawned by tractor resolve modules
from the wrong editable install, causing spurious
`AttributeError` / `ModuleNotFoundError`.
### Fallback: `uv run`
If the user can't or won't activate a venv, all
`python` and `pytest` commands can be prefixed
with `UV_PROJECT_ENVIRONMENT=py<MINOR> uv run`:
```sh
# instead of: python -m pytest tests/ -x
UV_PROJECT_ENVIRONMENT=py313 uv run pytest tests/ -x
# instead of: python -c 'import tractor'
UV_PROJECT_ENVIRONMENT=py313 uv run python -c 'import tractor'
```
`uv run` auto-discovers the project and venv,
but is slower than a pre-activated venv due to
lock-file resolution on each invocation. Prefer
activating the venv when possible.
### Step 3: import + collection checks
After venv is confirmed, always run these
(especially after refactors or module moves):
```sh
# 1. package import smoke check
python -c 'import tractor; print(tractor)'
# 2. verify all tests collect (no import errors)
python -m pytest tests/ -x -q --co 2>&1 | tail -5
```
If either fails, fix the import error before running
any actual tests.
### Step 4: zombie-actor / stale-registry check (MANDATORY)
The tractor runtime's default registry address is
**`127.0.0.1:1616`** (TCP) / `/tmp/registry@1616.sock`
(UDS). Whenever any prior test run — especially one
using a fork-based backend like `subint_forkserver`
leaks a child actor process, that zombie keeps the
registry port bound and **every subsequent test
session fails to bind**, often presenting as 50+
unrelated failures ("all tests broken"!) across
backends.
**This has to be checked before the first run AND
after any cancelled/SIGINT'd run** — signal failures
in the middle of a test can leave orphan children.
```sh
# 1. TCP registry — any listener on :1616? (primary signal)
ss -tlnp 2>/dev/null | grep ':1616' || echo 'TCP :1616 free'
# 2. leftover actor/forkserver procs — scoped to THIS
# repo's python path, so we don't false-flag legit
# long-running tractor-using apps (e.g. `piker`,
# downstream projects that embed tractor).
pgrep -af "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv" \
| grep -v 'grep\|pgrep' \
|| echo 'no leaked actor procs from this repo'
# 3. stale UDS registry sockets
ls -la /tmp/registry@*.sock 2>/dev/null \
|| echo 'no leaked UDS registry sockets'
```
**Interpretation:**
- **TCP :1616 free AND no stale sockets** → clean,
proceed. The actor-procs probe is secondary — false
positives are common (piker, any other tractor-
embedding app); only cleanup if `:1616` is bound or
sockets linger.
- **TCP :1616 bound OR stale sockets present**
surface PIDs + cmdlines to the user, offer cleanup:
```sh
# 1. GRACEFUL FIRST (tractor is structured concurrent — it
# catches SIGINT as an OS-cancel in `_trio_main` and
# cascades Portal.cancel_actor via IPC to every descendant.
# So always try SIGINT first with a bounded timeout; only
# escalate to SIGKILL if graceful cleanup doesn't complete).
pkill -INT -f "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv"
# 2. bounded wait for graceful teardown (usually sub-second).
# Loop until the processes exit, or timeout. Keep the
# bound tight — hung/abrupt-killed descendants usually
# hang forever, so don't wait more than a few seconds.
for i in $(seq 1 10); do
pgrep -f "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv" >/dev/null || break
sleep 0.3
done
# 3. ESCALATE TO SIGKILL only if graceful didn't finish.
if pgrep -f "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv" >/dev/null; then
echo 'graceful teardown timed out — escalating to SIGKILL'
pkill -9 -f "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv"
fi
# 4. if a test zombie holds :1616 specifically and doesn't
# match the above pattern, find its PID the hard way:
ss -tlnp 2>/dev/null | grep ':1616' # prints `users:(("<name>",pid=NNNN,...))`
# then (same SIGINT-first ladder):
# kill -INT <NNNN>; sleep 1; kill -9 <NNNN> 2>/dev/null
# 5. remove stale UDS sockets
rm -f /tmp/registry@*.sock
# 6. re-verify
ss -tlnp 2>/dev/null | grep ':1616' || echo 'TCP :1616 now free'
```
**Never ignore stale registry state.** If you see the
"all tests failing" pattern — especially
`trio.TooSlowError` / connection refused / address in
use on many unrelated tests — check registry **before**
spelunking into test code. The failure signature will
be identical across backends because they're all
fighting for the same port.
**False-positive warning for step 2:** a plain
`pgrep -af '_actor_child_main'` will also match
legit long-running tractor-embedding apps (e.g.
`piker` at `~/repos/piker/py*/bin/python3 -m
tractor._child ...`). Always scope to the current
repo's python path, or only use step 1 (`:1616`) as
the authoritative signal.
## 4. Run and report
- Run the constructed command.
- Use a timeout of **600000ms** (10min) for full suite
runs, **120000ms** (2min) for single-file runs.
- If the suite is large (full `tests/`), consider running
in the background and checking output when done.
- Use `--lf` (last-failed) to re-run only previously
failing tests when iterating on a fix.
### On failure:
- Show the failing test name(s) and short traceback.
- If the failure looks related to recent changes, point
out the likely cause and suggest a fix.
- **Check the known-flaky list** (section 8) before
investigating — don't waste time on pre-existing
timeout issues.
- **NEVER auto-commit fixes.** If you apply a code fix
during test iteration, leave it unstaged. Tell the
user what changed and suggest they review the
worktree state, stage files manually, and use
`/commit-msg` (inline or in a separate session) to
generate the commit message. The human drives all
`git add` and `git commit` operations.
### On success:
- Report the pass/fail/skip counts concisely.
## 5. Test directory layout (reference)
```
tests/
├── conftest.py # root fixtures, daemon, signals
├── devx/ # debugger/tooling tests
├── ipc/ # transport protocol tests
├── msg/ # messaging layer tests
├── discovery/ # discovery subsystem tests
│ ├── test_multiaddr.py # multiaddr construction
│ └── test_registrar.py # registry/discovery protocol
├── test_local.py # registrar + local actor basics
├── test_rpc.py # RPC error handling
├── test_spawning.py # subprocess spawning
├── test_multi_program.py # multi-process tree tests
├── test_cancellation.py # cancellation semantics
├── test_context_stream_semantics.py # ctx streaming
├── test_inter_peer_cancellation.py # peer cancel
├── test_infected_asyncio.py # trio-in-asyncio
└── ...
```
## 6. Change-type → test mapping
After modifying specific modules, run the corresponding
test subset first for fast feedback:
| Changed module(s) | Run these tests first |
|---|---|
| `runtime/_runtime.py`, `runtime/_state.py` | `test_local.py test_rpc.py test_spawning.py test_root_runtime.py` |
| `discovery/` (`_registry`, `_discovery`, `_addr`) | `tests/discovery/ test_multi_program.py test_local.py` |
| `_context.py`, `_streaming.py` | `test_context_stream_semantics.py test_advanced_streaming.py` |
| `ipc/` (`_chan`, `_server`, `_transport`) | `tests/ipc/ test_2way.py` |
| `runtime/_portal.py`, `runtime/_rpc.py` | `test_rpc.py test_cancellation.py` |
| `spawn/` (`_spawn`, `_entry`) | `test_spawning.py test_multi_program.py` |
| `devx/debug/` | `tests/devx/test_debugger.py` (slow!) |
| `to_asyncio.py` | `test_infected_asyncio.py test_root_infect_asyncio.py` |
| `msg/` | `tests/msg/` |
| `_exceptions.py` | `test_remote_exc_relay.py test_inter_peer_cancellation.py` |
| `runtime/_supervise.py` | `test_cancellation.py test_spawning.py` |
## 7. Quick-check shortcuts
### After refactors (fastest first-pass):
```sh
# import + collect check
python -c 'import tractor' && python -m pytest tests/ -x -q --co 2>&1 | tail -3
# core subset (~10s)
python -m pytest tests/test_local.py tests/test_rpc.py tests/test_spawning.py tests/discovery/test_registrar.py -x --tb=short --no-header
```
### Inspect last failures (without re-running):
When the user asks "what failed?", "show failures",
or wants to check the last-failed set before
re-running — read the pytest cache directly. This
is instant and avoids test collection overhead.
```sh
python -c "
import json, pathlib, sys
p = pathlib.Path('.pytest_cache/v/cache/lastfailed')
if not p.exists():
print('No lastfailed cache found.'); sys.exit()
data = json.loads(p.read_text())
# filter to real test node IDs (ignore junk
# entries that can accumulate from system paths)
tests = sorted(k for k in data if k.startswith('tests/'))
if not tests:
print('No failures recorded.')
else:
print(f'{len(tests)} last-failed test(s):')
for t in tests:
print(f' {t}')
"
```
**Why not `--cache-show` or `--co --lf`?**
- `pytest --cache-show 'cache/lastfailed'` works
but dumps raw dict repr including junk entries
(stale system paths that leak into the cache).
- `pytest --co --lf` actually *collects* tests which
triggers import resolution and is slow (~0.5s+).
Worse, when cached node IDs don't exactly match
current parametrize IDs (e.g. param names changed
between runs), pytest falls back to collecting
the *entire file*, giving false positives.
- Reading the JSON directly is instant, filterable
to `tests/`-prefixed entries, and shows exactly
what pytest recorded — no interpretation.
**After inspecting**, re-run the failures:
```sh
python -m pytest --lf -x --tb=short --no-header
```
### Full suite in background:
When core tests pass and you want full coverage while
continuing other work, run in background:
```sh
python -m pytest tests/ -x --tb=short --no-header -q
```
(use `run_in_background=true` on the Bash tool)
## 8. Known flaky tests
These tests have **pre-existing** timing/environment
sensitivity. If they fail with `TooSlowError` or
pexpect `TIMEOUT`, they are almost certainly NOT caused
by your changes — note them and move on.
| Test | Typical error | Notes |
|---|---|---|
| `devx/test_debugger.py::test_multi_nested_subactors_error_through_nurseries` | pexpect TIMEOUT | Debugger pexpect timing |
| `test_cancellation.py::test_cancel_via_SIGINT_other_task` | TooSlowError | Signal handling race |
| `test_inter_peer_cancellation.py::test_peer_spawns_and_cancels_service_subactor` | TooSlowError | Async timing (both param variants) |
| `test_docs_examples.py::test_example[we_are_processes.py]` | `assert None == 0` | `__main__` missing `__file__` in subproc |
**Rule of thumb**: if a test fails with `TooSlowError`,
`trio.TooSlowError`, or `pexpect.TIMEOUT` and you didn't
touch the relevant code path, it's flaky — skip it.
## 9. The pytest-capture hang pattern (CHECK THIS FIRST)
**Symptom:** a tractor test hangs indefinitely under
default `pytest` but passes instantly when you add
`-s` (`--capture=no`).
**Cause:** tractor subactors (especially under fork-
based backends) inherit pytest's stdout/stderr
capture pipes via fds 1,2. Under high-volume error
logging (e.g. multi-level cancel cascade, nested
`run_in_actor` failures, anything triggering
`RemoteActorError` + `ExceptionGroup` traceback
spew), the **64KB Linux pipe buffer fills** faster
than pytest drains it. Subactor writes block → can't
finish exit → parent's `waitpid`/pidfd wait blocks →
deadlock cascades up the tree.
**Pre-existing guards in the tractor harness** that
encode this same knowledge — grep these FIRST
before spelunking:
- `tests/conftest.py:258-260` (in the `daemon`
fixture): `# XXX: too much logging will lock up
the subproc (smh)` — downgrades `trace`/`debug`
loglevel to `info` to prevent the hang.
- `tests/conftest.py:316`: `# can lock up on the
_io.BufferedReader and hang..` — noted on the
`proc.stderr.read()` post-SIGINT.
**Debug recipe (in priority order):**
1. **Try `-s` first.** If the hang disappears with
`pytest -s`, you've confirmed it's capture-pipe
fill. Skip spelunking.
2. **Lower the loglevel.** Default `--ll=error` on
this project; if you've bumped it to `debug` /
`info`, try dropping back. Each log level
multiplies pipe-pressure under fault cascades.
3. **If you MUST use default capture + high log
volume**, redirect subactor stdout/stderr in the
child prelude (e.g.
`tractor.spawn._subint_forkserver._child_target`
post-`_close_inherited_fds`) to `/dev/null` or a
file.
**Signature tells you it's THIS bug (vs. a real
code hang):**
- Multi-actor test under fork-based backend
(`subint_forkserver`, eventually `trio_proc` too
under enough log volume).
- Multiple `RemoteActorError` / `ExceptionGroup`
tracebacks in the error path.
- Test passes with `-s` in the 5-10s range, hangs
past pytest-timeout (usually 30+ s) without `-s`.
- Subactor processes visible via `pgrep -af
subint-forkserv` or similar after the hang —
they're alive but blocked on `write()` to an
inherited stdout fd.
**Historical reference:** this deadlock cost a
multi-session investigation (4 genuine cascade
fixes landed along the way) that only surfaced the
capture-pipe issue AFTER the deeper fixes let the
tree actually tear down enough to produce pipe-
filling log volume. Full post-mortem in
`ai/conc-anal/subint_forkserver_test_cancellation_leak_issue.md`.
Lesson codified here so future-me grep-finds the
workaround before digging.
## 10. Reaping zombie subactors (`tractor-reap`)
**Symptom:** after a `pytest` run crashes, times out,
or is `Ctrl+C`'d, subactor forks (esp. under
`subint_forkserver`) can be reparented to `init`
(PPid==1) and linger. They hold onto ports, inherit
pytest's capture-pipe fds, and flakify later
sessions.
**Two layers of defense:**
### a) Session-scoped auto-fixture (always on)
`tractor/_testing/pytest.py::_reap_orphaned_subactors`
runs at pytest session teardown. It walks `/proc` for
direct descendants of the pytest pid, SIGINTs them,
waits up to 3s, then SIGKILLs survivors. SC-polite:
gives the subactor runtime a chance to run its trio
cancel shield + IPC teardown before escalation.
This is *autouse* and session-scoped — you don't need
to do anything. It just runs.
### b) `scripts/tractor-reap` CLI (manual reap)
For the **pytest-died-mid-session** case (Ctrl+C, OOM
kill, hung process you had to `kill -9`), the fixture
never ran. Reach for the CLI:
```sh
# default: orphans (PPid==1, cwd==repo, cmd contains python)
scripts/tractor-reap
# descendant-mode: from a still-live supervisor
scripts/tractor-reap --parent <pytest-pid>
# see what would be reaped, don't signal
scripts/tractor-reap -n
# tune the SIGINT → SIGKILL grace window
scripts/tractor-reap --grace 5
```
Exit code: `0` if everyone exited on SIGINT, `1` if
SIGKILL had to escalate — so you can chain it in CI
health-checks (`scripts/tractor-reap || <alert>`).
**What it matches** (orphan-mode):
- `PPid == 1` (reparented to init → definitely
orphaned, not just a currently-running child)
- `cwd == <repo-root>` (keeps the sweep scoped; won't
touch unrelated init-children elsewhere)
- `python` in cmdline
**What it does not do:** kill anything whose PPid is
still a live tractor parent. If the parent is alive
it's not an orphan; use `--parent <pid>` if you need
to force-reap under a still-live supervisor.
**When NOT to run it:** while a pytest session is
active in another terminal. It's safe (won't touch
that session's live children in orphan-mode) but can
race if the target session is mid-teardown.
### c) `--shm` / `--shm-only`: orphan-segment sweep
Because `tractor.ipc._mp_bs.disable_mantracker()`
turns off `mp.resource_tracker` (see
`ai/conc-anal/subint_forkserver_mp_shared_memory_issue.md`),
a hard-crashing actor can leave `/dev/shm/<key>`
segments behind that nothing else GCs.
```sh
# process reap THEN shm sweep
scripts/tractor-reap --shm
# shm sweep only (skip process phase)
scripts/tractor-reap --shm-only
# dry-run: list candidates, don't unlink
scripts/tractor-reap --shm -n
```
**Match criteria** (very conservative — this is a
shared-system path, can't be wrong):
- segment is a regular file under `/dev/shm`,
- owned by the **current uid** (`stat.st_uid`),
- AND **no live process holds it open**
enumerated by walking every readable
`/proc/<pid>/maps` (post-mmap mappings) AND
`/proc/<pid>/fd/*` (pre-mmap shm-opened fds).
The "nobody has it open" check is the
kernel-canonical "is this leaked?" test — same
answer `lsof /dev/shm/<key>` would give. No
reliance on tractor-specific naming, so it works
for any tractor app. Critically, it WILL NOT touch
segments held by other apps you have running
(e.g. `piker`, `lttng-ust-*`, `aja-shm-*`
verified locally with 81 in-use segments correctly
preserved).

View File

@ -0,0 +1,255 @@
# Tractor Test Harness Reference
This repository-local file supplements the canonical [`/run-tests` skill][1]
from the [`ai.skillz` repository][2]. Its deployer links the shared `SKILL.md`
into both
`.claude/skills/run-tests/` and `.opencode/skills/run-tests/` while preserving
this project-owned reference:
```text
bash /path/to/ai.skillz/scripts/deploy.sh run-tests /path/to/tractor --provider all --method symlink
```
Keep shared environment permission, process-signal safety, target selection,
failure inspection, and result reporting policy in the deployed `SKILL.md`.
[1]: https://github.com/baudco/ai.skillz/blob/2d4896ca7e38fe2cb3090cdefc7245be4241a6d2/skills/run-tests/SKILL.md
[2]: https://github.com/baudco/ai.skillz
## Project And Environment
- Project/import: `tractor`
- Test root: `tests/`
- Supported Python: `>=3.13,<3.15`
- Runner: pytest `>=9.0.3`
- Test dependencies: the `dev` group includes the `testing` group
- CI uses uv's default `.venv`; the Nix flake uses `py313`.
- Run from the repository root so pytest loads `pyproject.toml`.
- Do not use `default.nix` as current test-environment authority; it still
selects unsupported Python 3.12.
Environment directory naming is not a harness invariant. Use an already
verified active project environment when available. Otherwise, use an
existing uv environment without syncing it:
```text
uv run --frozen --no-sync python -c 'import pathlib, sys, tractor; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(tractor.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
```
After module moves or collection failures, check collection with:
```text
uv run --frozen --no-sync pytest --collect-only -q tests/
```
Collection is not a mandatory precursor to every narrow run. Ask before
provisioning or changing an environment.
Before trusting CLI-selected runtime settings, inspect
`TRACTOR_SPAWN_METHOD` and `TRACTOR_LOGLEVEL`. They override the spawn method
and runtime log level passed by callers, so report active values with test
results rather than claiming the CLI flags alone selected the runtime.
## Pytest Configuration And Commands
`pyproject.toml` configures:
- `testpaths = ["tests"]` and `--rootdir=./tests`;
- importlib import mode;
- the `tractor._testing.pytest` plugin;
- xonsh plugin disablement;
- `--show-capture=no` and `--capture=fd`.
Do not silently add `-x`, `--tb=short`, or `--no-header`; those are not
project defaults. In a verified active environment, replace `uv run
--frozen --no-sync pytest` below with `python -m pytest`.
```text
# Full suite
uv run --frozen --no-sync pytest tests/
# Narrow file
uv run --frozen --no-sync pytest tests/test_local.py
# Exact node
uv run --frozen --no-sync pytest tests/discovery/test_registrar.py::test_reg_then_unreg
# Keyword selection
uv run --frozen --no-sync pytest tests/ -k 'cancel and not slow'
# Previous failures
uv run --frozen --no-sync pytest --lf
```
After verifying that the no-sync environment is current, these pytest
arguments match the Linux TCP CI row:
```text
CI=1 uv run --frozen --no-sync pytest tests/ -rsx --spawn-backend=trio --tpt-proto=tcp --capture=fd
```
## Plugin Options And Matrices
Supported spawn backends:
- `trio` (default)
- `mp_spawn`
- `mp_forkserver`
Do not advertise `subint`, `subint_forkserver`, or
`main_thread_forkserver` as runnable backends. Supported transports are
`tcp` (default) and `uds`. Run one transport per pytest session.
`mp_forkserver` and UDS are POSIX-only.
Other Tractor plugin options include:
- `--tpdb` / `--debug-mode`
- `--ll` / `--loglevel`
- `--tl` / `--tractor-loglevel`
- `--enable-stackscope`
Examples:
```text
uv run --frozen --no-sync pytest tests/ipc/ --tpt-proto=uds
uv run --frozen --no-sync pytest tests/test_spawning.py --spawn-backend=mp_spawn
uv run --frozen --no-sync pytest tests/test_spawning.py --spawn-backend=mp_forkserver --capture=sys
```
CI currently exercises Python 3.13 with the `trio` backend: TCP and UDS on
Linux and macOS, plus an informational TCP row on Windows whose pytest step
uses `continue-on-error`.
## Registry And Transport Isolation
Tests requesting the `reg_addr` fixture use addresses randomized per session:
an unreserved unprivileged loopback port for TCP or a unique socket name under
the platform runtime directory for UDS. A TCP collision remains possible.
The runtime fallback remains `127.0.0.1:1616` or `registry@1616.sock`.
Inspect that fallback only when the selected test intentionally uses runtime
defaults or a failure identifies that address. Do not perform a mandatory
`:1616` preflight or assume UDS sockets live under `/tmp`.
## Capture And Hang Diagnosis
Normal capture is `fd`. Use `--capture=sys` with `mp_forkserver`; some tests
switch to `capsys`, but the harness does not enforce that suite-wide.
For a suspected capture interaction, compare only the exact node:
```text
uv run --frozen --no-sync pytest <node> --capture=sys
uv run --frozen --no-sync pytest <node> -s
```
Treat `-s` as a diagnostic comparison, not a pass-equivalent workaround. Do
not use it to reinterpret an ordinary captured pass. Interactive `--tpdb` or
`tractor.pause()` sessions are different: they require a real TTY and disabled
capture, normally `-s`.
Do not add a global pytest timeout. `fail_after_w_trace` is Trio-cooperative;
`afk_alarm_w_trace` is a POSIX main-thread `SIGALRM` hard backstop and can
raise asynchronously. Use the latter only as a last resort, not as a generally
Trio-safe timeout replacement.
For live task-tree diagnosis:
```text
uv run --frozen --no-sync python -c 'import stackscope'
uv run --frozen --no-sync pytest <node> --enable-stackscope --capture=sys
kill -USR1 <pytest-pid>
```
The import check and pytest command must use the same environment. Do not send
SIGUSR1 if the import fails or setup warns that stackscope or SIGUSR1 is
unavailable: without the installed handler, SIGUSR1 normally terminates the
target process. Signal a subactor only after separately confirming that it
installed the same handler.
Stackscope appends dumps to `/tmp/tractor-stackscope-<pid>.log`, including when
pytest capture hides terminal output. SIGUSR1 stackscope is unavailable on
Windows and degrades to a no-op there.
When a trace guard actually fires and snapshot capture succeeds, it writes
under `$XDG_CACHE_HOME/tractor/hung-dumps/`, falling back beneath
`~/.cache/tractor/hung-dumps/`, and prints an end-of-session index. A normal
non-timeout run creates no snapshot.
## Cleanup And `tractor-reap`
On Linux, normal pytest teardown discovers surviving descendants through
`/proc`, sends SIGINT, waits three seconds, then escalates survivors to
SIGKILL. It does not sweep shared memory and cannot run if pytest never reaches
fixture teardown.
Process discovery is a no-op off Linux. UDS PID liveness also depends on
`/proc`; on macOS, recognized PID-named sockets can therefore be classified as
dead without proof. The session-scoped autouse fixture currently passes those
candidates directly to `reap_uds()` at teardown. Do not treat its non-Linux
classification as proof of orphanhood or run concurrent live Tractor sessions
against the same UDS bindspace.
Use the CLI in inspection-only mode first:
`--shm` and `--shm-only` are Linux/FreeBSD-only and raise
`NotImplementedError` elsewhere. On other platforms, use the UDS-only command.
```text
uv run --frozen --no-sync scripts/tractor-reap -n
uv run --frozen --no-sync scripts/tractor-reap --parent <pytest-pid> -n
uv run --frozen --no-sync scripts/tractor-reap --shm --uds -n
uv run --frozen --no-sync scripts/tractor-reap --uds-only -n
```
Direct `scripts/tractor-reap` execution is acceptable only after verifying its
`python3` shebang resolves the intended project environment.
Review every candidate before requesting a mutating run:
- default orphan mode is not repository-scoped;
- `--parent` trusts the supplied PID and can include non-Tractor children;
- `--shm` scans all current-user candidate files, not just Tractor-named
files;
- `--uds` treats `registry@1616.sock` as removable even if a live default UDS
registrar uses it.
Dry-run output prints only the initially matched root PIDs. A mutating run can
recursively expand those roots to additional descendants when `psutil` is
available. Inspect the descendant process tree separately; `-n` is not exact
signal-set parity and does not by itself authorize signaling unseen children.
The canonical skill owns signaling and unlinking authorization.
## Test Layout And Change Mapping
| Changed area | Run first |
|---|---|
| `tractor/runtime/_runtime.py`, `_state.py`, `tractor/_root.py` | `tests/test_local.py`, `tests/test_root_runtime.py`, `tests/test_runtime.py`, `tests/test_rpc.py` |
| `tractor/runtime/_portal.py`, `_rpc.py` | `tests/test_rpc.py`, `tests/test_cancellation.py` |
| `tractor/runtime/_supervise.py` | `tests/test_cancellation.py`, `tests/test_spawning.py` |
| `tractor/discovery/` | `tests/discovery/`, `tests/test_local.py` |
| `tractor/ipc/` | `tests/ipc/`, `tests/test_2way.py`, `tests/test_shm.py` as relevant |
| `tractor/spawn/` | `tests/test_spawning.py`, `tests/discovery/test_multi_program.py`, `tests/test_cancellation.py` |
| `tractor/_context.py`, `_streaming.py` | `tests/test_context_stream_semantics.py`, `tests/test_advanced_streaming.py`, `tests/test_legacy_one_way_streaming.py` |
| `tractor/to_asyncio.py` | `tests/test_infected_asyncio.py`, `tests/test_root_infect_asyncio.py` |
| `tractor/msg/` | `tests/msg/` |
| `tractor/devx/` | `tests/devx/`; debugger tests use pexpect and are comparatively slow |
| `tractor/_exceptions.py` | `tests/test_remote_exc_relay.py`, `tests/test_reg_err_types.py`, `tests/test_inter_peer_cancellation.py`, `tests/test_cancellation.py`, `tests/msg/` |
Current subdirectories include `discovery/`, `ipc/`, `msg/`, `devx/`, and
`trionics/`. There is no `tests/spawn/` directory.
## Expected Outcomes
Do not maintain a blanket known-flaky exemption list. Classify only current
explicit skip or xfail marks and exact expected signatures. Notable tracked
outcomes include:
- duplicate-name `n_dups=4` and `n_dups=8` variants in
`tests/discovery/test_multi_program.py` are non-strict xfails;
- `tests/test_ringbuf.py` is module-skipped;
- some documentation examples have explicit macOS-CI skips.
A generic `TooSlowError` or `pexpect.TIMEOUT` is not enough to classify a
failure as pre-existing.

264
.gitignore vendored
View File

@ -168,3 +168,267 @@ gh/
# LLM conversations that should remain private
docs/conversations/
# BEGIN ai.skillz: direct:symlink:claude:run-tests
/.claude/skills/run-tests/SKILL.md
# END ai.skillz: direct:symlink:claude:run-tests
# BEGIN ai.skillz: direct:symlink:opencode:run-tests
/.opencode/skills/run-tests/SKILL.md
# END ai.skillz: direct:symlink:opencode:run-tests
# BEGIN ai.skillz: direct:symlink:opencode:command:run-tests
/.opencode/commands/run-tests.md
# END ai.skillz: direct:symlink:opencode:command:run-tests
# BEGIN ai.skillz: direct:symlink:claude:gish
/.claude/skills/gish
# END ai.skillz: direct:symlink:claude:gish
# BEGIN ai.skillz: direct:symlink:opencode:gish
/.opencode/skills/gish
# END ai.skillz: direct:symlink:opencode:gish
# BEGIN ai.skillz: direct:symlink:claude:resolve-conflicts
/.claude/skills/resolve-conflicts
# END ai.skillz: direct:symlink:claude:resolve-conflicts
# BEGIN ai.skillz: direct:symlink:opencode:resolve-conflicts
/.opencode/skills/resolve-conflicts
# END ai.skillz: direct:symlink:opencode:resolve-conflicts
# BEGIN ai.skillz: direct:symlink:claude:git-mgmt
/.claude/skills/git-mgmt
# END ai.skillz: direct:symlink:claude:git-mgmt
# BEGIN ai.skillz: direct:symlink:opencode:git-mgmt
/.opencode/skills/git-mgmt
# END ai.skillz: direct:symlink:opencode:git-mgmt
# BEGIN ai.skillz: runtime:open-wkt
/wkts/
# END ai.skillz: runtime:open-wkt
# BEGIN ai.skillz: direct:symlink:claude:open-wkt
/.claude/skills/open-wkt
# END ai.skillz: direct:symlink:claude:open-wkt
# BEGIN ai.skillz: direct:symlink:opencode:open-wkt
/.opencode/skills/open-wkt
# END ai.skillz: direct:symlink:opencode:open-wkt
# BEGIN ai.skillz: direct:symlink:claude:close-wkt
/.claude/skills/close-wkt
# END ai.skillz: direct:symlink:claude:close-wkt
# BEGIN ai.skillz: direct:symlink:opencode:close-wkt
/.opencode/skills/close-wkt
# END ai.skillz: direct:symlink:opencode:close-wkt
# BEGIN ai.skillz: runtime:code-review
.ai/code-review/reports/
# END ai.skillz: runtime:code-review
# BEGIN ai.skillz: direct:symlink:claude:code-review
/.claude/skills/code-review
# END ai.skillz: direct:symlink:claude:code-review
# BEGIN ai.skillz: direct:symlink:opencode:code-review
/.opencode/skills/code-review
# END ai.skillz: direct:symlink:opencode:code-review
# BEGIN ai.skillz: direct:symlink:claude:code-nav-refs
/.claude/skills/code-nav-refs
# END ai.skillz: direct:symlink:claude:code-nav-refs
# BEGIN ai.skillz: direct:symlink:opencode:code-nav-refs
/.opencode/skills/code-nav-refs
# END ai.skillz: direct:symlink:opencode:code-nav-refs
# BEGIN ai.skillz: runtime:code-review-changes
.claude/review_context.md
.claude/review_regression.md
.claude/review_replies/
# END ai.skillz: runtime:code-review-changes
# BEGIN ai.skillz: direct:symlink:claude:code-review-changes
/.claude/skills/code-review-changes
# END ai.skillz: direct:symlink:claude:code-review-changes
# BEGIN ai.skillz: direct:symlink:opencode:code-review-changes
/.opencode/skills/code-review-changes
# END ai.skillz: direct:symlink:opencode:code-review-changes
# BEGIN ai.skillz: runtime:commit-msg
.claude/skills/commit-msg/msgs/
.claude/git_commit_msg_LATEST.md
# END ai.skillz: runtime:commit-msg
# BEGIN ai.skillz: direct:symlink:claude:commit-msg
/.claude/skills/commit-msg/SKILL.md
# END ai.skillz: direct:symlink:claude:commit-msg
# BEGIN ai.skillz: direct:symlink:opencode:commit-msg
/.opencode/skills/commit-msg/SKILL.md
# END ai.skillz: direct:symlink:opencode:commit-msg
# BEGIN ai.skillz: direct:symlink:claude:commit-plan
/.claude/skills/commit-plan
# END ai.skillz: direct:symlink:claude:commit-plan
# BEGIN ai.skillz: direct:symlink:opencode:commit-plan
/.opencode/skills/commit-plan
# END ai.skillz: direct:symlink:opencode:commit-plan
# BEGIN ai.skillz: direct:symlink:claude:dep-supersede-scan
/.claude/skills/dep-supersede-scan
# END ai.skillz: direct:symlink:claude:dep-supersede-scan
# BEGIN ai.skillz: direct:symlink:opencode:dep-supersede-scan
/.opencode/skills/dep-supersede-scan
# END ai.skillz: direct:symlink:opencode:dep-supersede-scan
# BEGIN ai.skillz: direct:symlink:claude:harness-perf
/.claude/skills/harness-perf
# END ai.skillz: direct:symlink:claude:harness-perf
# BEGIN ai.skillz: direct:symlink:opencode:harness-perf
/.opencode/skills/harness-perf
# END ai.skillz: direct:symlink:opencode:harness-perf
# BEGIN ai.skillz: direct:symlink:claude:inter-skill-review
/.claude/skills/inter-skill-review
# END ai.skillz: direct:symlink:claude:inter-skill-review
# BEGIN ai.skillz: direct:symlink:opencode:inter-skill-review
/.opencode/skills/inter-skill-review
# END ai.skillz: direct:symlink:opencode:inter-skill-review
# BEGIN ai.skillz: direct:symlink:claude:opencode-cleaning
/.claude/skills/opencode-cleaning
# END ai.skillz: direct:symlink:claude:opencode-cleaning
# BEGIN ai.skillz: direct:symlink:opencode:opencode-cleaning
/.opencode/skills/opencode-cleaning
# END ai.skillz: direct:symlink:opencode:opencode-cleaning
# BEGIN ai.skillz: direct:symlink:claude:plan-io
/.claude/skills/plan-io
# END ai.skillz: direct:symlink:claude:plan-io
# BEGIN ai.skillz: direct:symlink:opencode:plan-io
/.opencode/skills/plan-io
# END ai.skillz: direct:symlink:opencode:plan-io
# BEGIN ai.skillz: runtime:pr-msg
.claude/skills/pr-msg/msgs/
.claude/skills/pr-msg/pr_msg_LATEST.md
# END ai.skillz: runtime:pr-msg
# BEGIN ai.skillz: direct:symlink:claude:pr-msg
/.claude/skills/pr-msg/SKILL.md
/.claude/skills/pr-msg/references
/.claude/skills/pr-msg/scripts
# END ai.skillz: direct:symlink:claude:pr-msg
# BEGIN ai.skillz: direct:symlink:opencode:pr-msg
/.opencode/skills/pr-msg/SKILL.md
/.opencode/skills/pr-msg/references
/.opencode/skills/pr-msg/scripts
# END ai.skillz: direct:symlink:opencode:pr-msg
# BEGIN ai.skillz: direct:symlink:claude:prompt-io
/.claude/skills/prompt-io
# END ai.skillz: direct:symlink:claude:prompt-io
# BEGIN ai.skillz: direct:symlink:opencode:prompt-io
/.opencode/skills/prompt-io
# END ai.skillz: direct:symlink:opencode:prompt-io
# BEGIN ai.skillz: direct:symlink:claude:py-codestyle
/.claude/skills/py-codestyle
# END ai.skillz: direct:symlink:claude:py-codestyle
# BEGIN ai.skillz: direct:symlink:opencode:py-codestyle
/.opencode/skills/py-codestyle
# END ai.skillz: direct:symlink:opencode:py-codestyle
# BEGIN ai.skillz: runtime:taken-export
.ai/taken/exports/
# END ai.skillz: runtime:taken-export
# BEGIN ai.skillz: direct:symlink:claude:taken-export
/.claude/skills/taken-export
# END ai.skillz: direct:symlink:claude:taken-export
# BEGIN ai.skillz: direct:symlink:opencode:taken-export
/.opencode/skills/taken-export
# END ai.skillz: direct:symlink:opencode:taken-export
# BEGIN ai.skillz: direct:symlink:claude:yt-url-lookup
/.claude/skills/yt-url-lookup
# END ai.skillz: direct:symlink:claude:yt-url-lookup
# BEGIN ai.skillz: direct:symlink:opencode:yt-url-lookup
/.opencode/skills/yt-url-lookup
# END ai.skillz: direct:symlink:opencode:yt-url-lookup
# BEGIN ai.skillz: direct:symlink:opencode:command:gish
/.opencode/commands/gish.md
# END ai.skillz: direct:symlink:opencode:command:gish
# BEGIN ai.skillz: direct:symlink:opencode:command:resolve-conflicts
/.opencode/commands/resolve-conflicts.md
# END ai.skillz: direct:symlink:opencode:command:resolve-conflicts
# BEGIN ai.skillz: direct:symlink:opencode:command:git-mgmt
/.opencode/commands/git-mgmt.md
# END ai.skillz: direct:symlink:opencode:command:git-mgmt
# BEGIN ai.skillz: direct:symlink:opencode:command:open-wkt
/.opencode/commands/open-wkt.md
# END ai.skillz: direct:symlink:opencode:command:open-wkt
# BEGIN ai.skillz: direct:symlink:opencode:command:close-wkt
/.opencode/commands/close-wkt.md
# END ai.skillz: direct:symlink:opencode:command:close-wkt
# BEGIN ai.skillz: direct:symlink:opencode:command:code-review
/.opencode/commands/code-review.md
# END ai.skillz: direct:symlink:opencode:command:code-review
# BEGIN ai.skillz: direct:symlink:opencode:command:code-review-changes
/.opencode/commands/code-review-changes.md
# END ai.skillz: direct:symlink:opencode:command:code-review-changes
# BEGIN ai.skillz: direct:symlink:opencode:command:commit-msg
/.opencode/commands/commit-msg.md
# END ai.skillz: direct:symlink:opencode:command:commit-msg
# BEGIN ai.skillz: direct:symlink:opencode:command:commit-plan
/.opencode/commands/commit-plan.md
# END ai.skillz: direct:symlink:opencode:command:commit-plan
# BEGIN ai.skillz: direct:symlink:opencode:command:dep-supersede-scan
/.opencode/commands/dep-supersede-scan.md
# END ai.skillz: direct:symlink:opencode:command:dep-supersede-scan
# BEGIN ai.skillz: direct:symlink:opencode:command:harness-perf
/.opencode/commands/harness-perf.md
# END ai.skillz: direct:symlink:opencode:command:harness-perf
# BEGIN ai.skillz: direct:symlink:opencode:command:opencode-cleaning
/.opencode/commands/opencode-cleaning.md
# END ai.skillz: direct:symlink:opencode:command:opencode-cleaning
# BEGIN ai.skillz: direct:symlink:opencode:command:pr-msg
/.opencode/commands/pr-msg.md
# END ai.skillz: direct:symlink:opencode:command:pr-msg
# BEGIN ai.skillz: direct:symlink:opencode:command:taken-export
/.opencode/commands/taken-export.md
# END ai.skillz: direct:symlink:opencode:command:taken-export
# BEGIN ai.skillz: direct:symlink:opencode:command:yt-url-lookup
/.opencode/commands/yt-url-lookup.md
# END ai.skillz: direct:symlink:opencode:command:yt-url-lookup

View File

@ -0,0 +1,136 @@
---
model: claude-opus-5
service: claude
session: 7b9c97c4-fff7-4ac4-97fb-35720453308e
timestamp: 2026-08-13T00:11:02Z
git_ref: 27c34aeb
scope: docs+code
substantive: true
raw_file: 20260813T001102Z_27c34aeb_prompt_io.raw.md
---
## Prompt
> draft hyper detailed implementation plans for [three]
> prospective new transport (tpt) backends for tractor's `.ipc`
> layer, from four GitHub issues: TIPC (gh #378) using built-in
> linux socket API w/ `trio` interfacing, leveraging TIPC's
> built-in discovery machinery; QUIC (gh #353) using the `iroh`
> lib, ideally with the py asyncio support (via ffi) rewritten
> for trio; wg (gh #482 and/or #443) with other shuttle-able
> tpts, using `pyroute2`, as much trio wrapping as possible
> where any other async support can be replaced.
With constraints: "be only slightly speculative"; realistic given
the linked info; the plans must be workable "by another
model/provider without much discrepancy on the design or lib
selections needed"; do it all in a new `/open-wkt` off main; and
be "as modular and modern-async-python-as-nearly-functional-style
as possible."
Key follow-up prompts that changed the design mid-session:
> wg requires binding a socket on both ends of the tunnel, the
> new support we added to `py-multiaddr` in their #108 enables
> declaring this in the maddr? […] so the
> `/ip4/10.0.0.1/udp/666/ip4/192.168.0.13/tcp/80` bullet example
> from #443 isn't correct for a wg tunnel maddr and would
> actually end up looking like,
> `/ip4/10.0.0.1/udp/666/wg/<pub-key>/ip4/192.168.0.13/tcp/80` no?
> right, we're likely going to need less `tuple`-formed
> `UnwrappedAddress` usage to make them all work and/or to extend
> the unwrapped form with a proto-str, likely the key from
> `multiaddrs` no?
> better encapsulating `UnwrappedAddress` for use externally is
> really way overdue; we should likely be encouraging (if not
> preventing unwrapped) wrapped addrs always much like other
> (std) libs, `ipaddress` comes to mind.
> so iirc netns mgmt becomes a lower-level runtime config-api
> rather then an actor-app-code API which can be dynamically
> changed once the actor is up ya?
## Response summary
A research-and-planning branch, `ng_tpts_planning` — **no runtime
code touched**. Five plan docs under `ai/tpt-backends/` plus a
corrected `wg` example set under `examples/multihost/wg_lan/`,
across 9 commits.
Three design conclusions came out of the planning that outlive
the branch:
1. **`wg` is a bindspace, not a `MsgTransport`.** The `/wg/`
maddr segment is *infix*: bearer (underlay ip+udp
`ListenPort`, bound by the kernel) → `/wg/u<key>` (an
identity, bound by nothing) → overlay (the only part tractor
binds). Verified empirically by installing
`baudco/py-multiaddr@wg_support` (py-multiaddr#108) in a
throwaway venv and round-tripping each candidate form; gh
#482's original suffix form parses but is semantically
inverted.
2. **Proto-key the `UnwrappedAddress` form**, spelled with the
`multiaddr` protocol names, dispatched via
`_address_types[addr[0]]`. Kills a whole collision class
(TIPC's `(str, int)``TCPAddress`; iroh's `(str, str)`
swallowed by the UDS case) and is the recommended migration
*before* any new backend lands.
3. **netns is a runtime/boot-time config API, not an app-code
one** — `setns(2)` is per-thread and won't move
already-created sockets, so there is deliberately no
`await actor.enter_netns(...)`.
Also verified that `trio.SocketStream`/`SocketListener` are
address-family agnostic (no `AF_*` check anywhere), which is what
makes TIPC the cheapest of the three backends to add.
Four related issues were annotated with the results (#378, #353,
#482, #443); #443's body was rewritten to reflect the corrected
grammar, with no existing checkbox state changed.
## Files changed
- `ai/tpt-backends/00_shared_backend_contract.md` — normative
backend duck-type contract, registration checklist, §1.1
proto-key conclusion
- `ai/tpt-backends/01_tipc_backend.md` — TIPC plan; service
addressing, `TIPC_TOP_SRV` push registry, instance-collision
hazard, step-0 probe
- `ai/tpt-backends/02_quic_iroh_backend.md``iroh` plan;
`uniffi`→`trio` bridge, listener/stream adapters, API-truth
table
- `ai/tpt-backends/03_wg_tunnel_bindspace.md``wg`-as-bindspace
plan; verified maddr grammar, 3-owner split, netns reality
- `ai/tpt-backends/README.md` — index
- `examples/multihost/wg_lan/wg_maddr.py` — frozen `msgspec`
tunnelled addr + pure parse/render helpers; impure
`verify_wg_peer()` kept separate
- `examples/multihost/wg_lan/host_a_srv.py` — host-A actor tree
- `examples/multihost/wg_lan/host_b_client.py` — host-B dialer
- `examples/multihost/wg_lan/README.md` — grammar, owner table,
setup, "what changed vs #482"
## Human edits
Substantial human steering rather than post-hoc editing; the
corrections were applied by the model in-session after being
challenged:
- rejected an initial claim that `wg` has "nothing to bind at the
tunnel layer" and supplied the correct composed maddr form,
which forced a rewrite of plan 03 §3.2 and a retraction in the
already-posted #443 comment
- rejected a supporting claim that `/ip4/../udp/443/quic-v1` was
"also composed"
- directed the proto-key/`ipaddress`-discipline conclusion and
the netns-as-runtime-config framing, both of which were then
folded back into the docs
- chose the commit boundaries and authored all commits; ran every
`git` mutation (commit, rebase, push) themselves
One model-initiated correction pre-publication: a self-review
downgraded two overconfident claims (the `uniffi`/asyncio thesis
and TIPC duplicate-binder behaviour) to explicitly-flagged
assumptions before the #353/#378 comments were posted.

View File

@ -0,0 +1,165 @@
---
model: claude-opus-5
service: claude
timestamp: 2026-08-13T00:11:02Z
git_ref: 27c34aeb
diff_cmd: git diff main..ng_tpts_planning
---
# Raw output — next-gen tpt-backend implementation plans
## Generated planning docs
> `git diff main..ng_tpts_planning -- ai/tpt-backends/`
Five markdown docs. `00_shared_backend_contract.md` is normative
and the other three are written against it so they can be worked
independently:
- **`00_shared_backend_contract.md`** — the backend duck-type
(`<Proto>Address(msgspec.Struct, frozen=True)` + module-level
`start_listener()`/`close_listener()` + a
`Msgpack<Proto>Stream(MsgpackTransport)`), the
`inspect.getmodule(self.addr)` reflection in
`Endpoint.start_listener()` that forces the Address class and
its listener fns to share a module, a 10-item registration
checklist, the dep policy, the test-harness shape, and §1.1's
proto-key conclusion (below).
- **`01_tipc_backend.md`** — service addressing via
`TIPC_ADDR_NAMESEQ` (bind/publish) and `TIPC_ADDR_NAME`
(connect/lookup), `TIPC_TOP_SRV` topology subscriptions as a
push-based registry, the `get_random()` instance-collision
hazard, and a step-0 capability-probe spike.
- **`02_quic_iroh_backend.md`** — `iroh` over
`aioquic`/`quiche`/`trio-asyncio`, a `_uniffi_trio.py` bridge
built on `TrioToken.run_sync_soon()`, `trio.abc.Listener`/
`HalfCloseableStream` adapters, and an API-truth table to fill
in during step 0.
- **`03_wg_tunnel_bindspace.md`** — `wg` as a *bindspace* rather
than a `MsgTransport`, a `TunnelledAddress` wrapper delegating
`.proto_key`/`.unwrap()` to `.inner`, `pyroute2` for layer B,
and `@acm`-managed netns/iface for layer C.
- **`README.md`** — index.
## Generated example code
> `git diff main..ng_tpts_planning -- examples/multihost/wg_lan/`
- `wg_maddr.py``WGTunnelledAddr(msgspec.Struct, frozen=True)`
carrying `bearer: tuple[str, int]`, `peer_pubkey: str`,
`inner: tuple[str, int]`, `inner_proto: Literal['tcp']`, plus a
`.maddr` property that re-renders the canonical form. Pure
helpers `mb_pubkey()`, `wg8_pubkey()`, `parse_wg_maddr()`, and
`_segments()` (with a marked stopgap for when the `wg` codec
isn't installed). `verify_wg_peer()` is impure **by design** and
kept out of the parse path.
- `host_a_srv.py` / `host_b_client.py` — the two-host runs; both
pass only `addr.inner` to `open_nursery()`/`open_root_actor()`.
- `README.md` — grammar, owner table, `#108`-branch install line,
tunnel setup, "what changed vs #482".
## Verified findings (non-code, verbatim)
### `trio` is address-family agnostic
Read against the installed `trio`. `SocketStream`/`SocketListener`
ctor checks are only "is a trio sock object" + `type ==
SOCK_STREAM`, plus an `OSError`-**suppressed** `SO_ACCEPTCONN`
probe. No `AF_*` check anywhere; `TCP_NODELAY`/`TCP_NOTSENT_LOWAT`
are set under `suppress(OSError)`. A TIPC `SOCK_STREAM` sock should
therefore drop straight into `trio.serve_listeners()` with the
existing `MsgpackTransport` framing, making TIPC mostly
table-registration boilerplate w/ zero new deps.
### the `wg` maddr grammar — `/wg/` is infix, not suffix
Installed `baudco/py-multiaddr@wg_support` (PR
multiformats/py-multiaddr#108) into a throwaway venv and
round-tripped every candidate form:
| maddr | `[p.name for p in m.protocols()]` |
| --- | --- |
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
| `/ip4/10.0.11.1/tcp/1616/wg/u<k>` | `['ip4','tcp','wg']` |
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\_______ bearer __________/\__ key __/\______ overlay ______/
```
Segments *before* `/wg/` are the bearer — the underlay
`(ip, udp-port)` that `wg(8)` itself listens on (`ListenPort`).
Segments *after* are the overlay endpoint, the only part tractor
binds. The third row above is #482's original suffix form: it
parses, but is semantically inverted.
Three parts, three owners — and only one is an `Endpoint`:
| part | bound by | in the runtime? |
| --- | --- | --- |
| bearer | kernel, via `wg-quick`/`pyroute2` | no |
| `/wg/u<key>` | nothing — an identity | no, verified out-of-band |
| overlay | `tractor`'s `IPCServer` | yes, as `.inner` |
### proto-key-tagged `UnwrappedAddress`
Shape-matching in `wrap_address()` does not survive four backends.
TIPC's natural unwrapped form is a `(str, int)`, indistinguishable
from `TCPAddress`; iroh's is a `(str, str)`, already swallowed by
the existing UDS case (`case (_, filename) if type(filename) is
str`). Ordering hacks and prefix-tagging only paper over it.
Recommended prerequisite for all three backends: carry an explicit
proto-key spelled with the `multiaddr` protocol name —
`('tcp', host, port)`, `('unix', path)`,
`('tipc', stype, inst, scope)` — so `wrap_address()` collapses to
`_address_types[addr[0]]` and the collision class stops existing.
This also makes the on-wire form agree with
`mk_maddr()`/`parse_maddr()` instead of being an independent
invention. It is a wire-format change (`SpawnSpec`,
`_root_mailbox`, `_registry_addrs`) plus every fixture and
downstream config, so it wants its own migration commit landed
before any new backend — and it is the moment to stop handing raw
tuples to users at all, making `Address` the public currency and
`UnwrappedAddress` an internal serialization detail (the
discipline `ipaddress` uses).
### netns is a runtime-level config API
`setns(2)` affects the calling thread only and does not move
already-created sockets. So a netns is a spawn/boot-time input
alongside `enable_transports`/`tpt_bind_addrs`, and there is
deliberately no `await actor.enter_netns(...)` — a mid-life API
would silently leave the IPC server bound in the old namespace.
Corollary for layer B: pass `netns=` down to `pyroute2` rather
than assuming a `trio.to_thread` worker inherits it.
### `examples/` collection would have failed CI
`tests/test_docs_examples.py` walks `examples/` recursively and
subproc-runs every collected file asserting `rc == 0`. Its filter
never checks the extension, so all four `wg_lan` files were
collected — including `README.md`, which would have been run as
`python README.md`. `'multihost' not in p[0]` was already in the
exclusion list with no directory using it. Moving the set under
`examples/multihost/wg_lan/` drops collection 24 → 20 with zero
test changes; confirmed via `pytest --collect-only`.
## Corrections applied during the session
The human corrected two claims that had been asserted without
verification, both since retracted in-place in the docs and in the
posted issue comments:
1. that `wg` has "nothing to bind at the tunnel layer, exactly one
bind" — wrong; a wg stack is genuinely composed, and the real
axis is *who owns* each layer's endpoint.
2. that `/ip4/../udp/443/quic-v1` was "also composed" — wrong;
that is one endpoint with a protocol qualifier, not a tunnel.
A self-review before publication also downgraded two
overconfident claims to explicitly-flagged assumptions: the
`uniffi`-uses-asyncio-only-as-executor thesis (contradicted that
plan's own "do not guess from memory" step 0) and TIPC's
duplicate-binder round-robin behaviour (unverified).

View File

@ -0,0 +1,30 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-11T23:38:33Z
git_ref: 7cbd64ee
scope: code
substantive: true
raw_file: 20260811T233833Z_7cbd64ee_prompt_io.raw.md
---
## Prompt
Open a new isolated worktree in the local Tractor repository and draft a fix
for `BroadcastReceiver` reporting that a lagged one-slot consumer dropped
zero values when one value had actually been displaced.
## Response summary
Corrected the off-by-one lag count while preserving cursor recovery and added
deterministic narrow- and wider-window regressions for exact loss reporting.
## Files changed
- `tractor/trionics/_broadcast.py` - exact broadcast overrun count.
- `tests/test_task_broadcasting.py` - lag count and recovery regression.
## Human edits
None - generated output follows the user's diagnosed edge case.

View File

@ -0,0 +1,50 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-11T23:38:33Z
git_ref: 7cbd64ee
diff_cmd: git diff HEAD~1..HEAD
---
The user asked for a Tractor fix in a new isolated worktree after a live piker
failure reported:
```text
tractor.trionics._broadcast.Lagged:
Task `piker.brokers.ib.broker.handle_order_requests` overrun and
dropped `0` values
```
Inspection showed the lag exception was valid but its count was off by one.
`BroadcastReceiver.receive_nowait()` treats `seq` as a deque index. With a
one-entry queue, index zero is the only retained value and `seq == 1` already
means one value was displaced. The old `seq - maxlen` calculation therefore
reported zero instead of one.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Adjusted the lag count to `seq - maxlen + 1` and documented why the first
invalid deque index must be included. The existing Tokio-style cursor reset
remains unchanged.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added a deterministic parameterized regression covering a one-slot queue
with one dropped value and a three-slot queue with two dropped values. The
test keeps the root receiver idle while a child subscriber synchronously
drains each produced value, asserts the exact `Lagged` message, and proves the
next receive resumes at the oldest retained item.
Verification output:
```text
.. [100%]
2 passed in 0.04s
.......... [100%]
10 passed in 4.65s
```
The targeted import resolved to the new Tractor worktree. Python compilation
and `git diff --check` passed. Adversarial review found no actionable issues;
zero-capacity channels remain a pre-existing untested edge outside this fix.

View File

@ -0,0 +1,31 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T01:23:24Z
git_ref: 06c4af17
scope: code
substantive: true
raw_file: 20260812T012324Z_06c4af17_prompt_io.raw.md
---
## Prompt
Iteratively refine Tractor PR 490. For item one, correct broadcast statistics
queue counts and Trio event checks, verify and review the exact change, then
return a complete commit plan before proceeding.
## Response summary
Converted subscriber cursor indexes into clamped retained queue counts,
removed deprecated event truthiness, and added deterministic state and
deprecation regressions.
## Files changed
- `tractor/trionics/_broadcast.py` - accurate queue and waiter statistics.
- `tests/test_task_broadcasting.py` - retained-count and event regression.
## Human edits
None - generated output follows the requested first iterative item.

View File

@ -0,0 +1,37 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T01:23:24Z
git_ref: 06c4af17
diff_cmd: git diff HEAD~1..HEAD
---
After opening draft Tractor PR 490, the user requested an iterative pass over
additional broadcast subsystem findings. The first item was to correct
`BroadcastState.statistics()` queued counts and its deprecated Trio event
truthiness check, then stop for a complete commit plan.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Changed `queued_len_by_task` from raw deque cursor indexes to retained,
receivable counts. Caught-up `-1` reports zero, valid indexes report index plus
one, and lagged cursors clamp to the current retained queue length. Replaced
`trio.Event` truthiness with an explicit `is not None` branch.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added a deterministic statistics regression using actual sends and receives.
It verifies caught-up and one-queued states, drives a root receiver beyond a
three-slot retention window to prove clamping, and installs a real
`trio.Event` while treating deprecations as errors.
Verification output:
```text
........... [100%]
11 passed in 5.76s
```
Python compilation and `git diff --check` passed. Initial adversarial review
caught unclamped lagged cursors and an ineffective event test; both were
fixed. Final review found no actionable issues.

View File

@ -0,0 +1,33 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T03:06:08Z
git_ref: 1095e7f7
scope: code
substantive: true
raw_file: 20260812T030608Z_1095e7f7_prompt_io.raw.md
---
## Prompt
For Tractor PR 490 item two, make shared underlying receive failures wake and
terminate every broadcast subscriber without losing retained values. Review,
verify and return a complete commit plan before continuing.
## Response summary
Published ordinary receive failures as terminal broadcast state, introduced
a public chained peer exception, kept control-flow exits transient while
waking peers, documented the contract, and added deterministic regressions.
## Files changed
- `tractor/trionics/_broadcast.py` - terminal failure and peer wake protocol.
- `tractor/trionics/__init__.py` - public peer exception export.
- `docs/api/trionics.rst` - failure-delivery API contract.
- `tests/test_task_broadcasting.py` - terminal and transient failure tests.
## Human edits
None - generated output follows the requested second iterative item.

View File

@ -0,0 +1,49 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T03:06:08Z
git_ref: 1095e7f7
diff_cmd: git diff HEAD~1..HEAD
---
The user requested the second iterative refinement for Tractor PR 490: ensure
non-EOC failures from a shared underlying broadcast receiver do not leave peer
subscribers blocked forever, then stop for a complete commit plan.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Added shared terminal failure publication for ordinary `Exception` values.
The receive owner gets the original exception; peers may drain retained
values and then get a fresh `BroadcastReceiveError` chained from the original.
Late subscribers observe the same terminal state without retrying the failed
underlying receiver. Process-control and cancellation-like `BaseException`
values wake peers but are re-raised without becoming durable channel state.
> `git diff HEAD~1..HEAD -- tractor/trionics/__init__.py`
Exported `BroadcastReceiveError` as the public peer-delivery exception.
> `git diff HEAD~1..HEAD -- docs/api/trionics.rst`
Documented `BroadcastReceiveError` and the owner-versus-peer delivery
contract, including retained-value draining and late subscribers.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added deterministic bounded regressions. One scripts a successful receive
followed by `RuntimeError`, proving the root drains retained data, all current
and late receivers observe terminal failure, and the source is not retried.
The second scripts a custom `BaseException`, proving peers wake and take over
the next source receive without retaining control-flow state.
Verification output:
```text
............. [100%]
13 passed in 5.62s
```
Python compilation and `git diff --check` passed. Iterative adversarial review
drove independent peer exception wrappers, ordinary-versus-control-flow
classification, bounded test completion, public docs, and the final catch-all
peer wake. Final review found no issues.

View File

@ -0,0 +1,33 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T15:00:27Z
git_ref: c2a6ccef
scope: code
substantive: true
raw_file: 20260812T150027Z_c2a6ccef_prompt_io.raw.md
---
## Prompt
For Tractor PR 490 item three, prevent subscriber closure from waking another
receiver's shared event or stranding peers. Preserve close-safe source-read
ownership handoff, then review, verify and return a complete commit plan.
## Response summary
Introduced receiver-local wait/source cancellation, owner-specific handoff,
and close precedence over shielded source values/errors/EOC. Clarified that
private scope cancellation means explicit close while outer task cancellation
remains `trio.Cancelled` for both source-read and peer-wait scopes, with
deterministic receiver-close regressions.
## Files changed
- `tractor/trionics/_broadcast.py` - receiver-local close and ownership scopes.
- `tests/test_task_broadcasting.py` - peer close and owner handoff regressions.
## Human edits
None - generated output follows the requested third iterative item.

View File

@ -0,0 +1,51 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T15:00:27Z
git_ref: c2a6ccef
diff_cmd: git diff HEAD~1..HEAD
---
The user requested the third iterative refinement for Tractor PR 490: closing
one broadcast subscriber must not set another receiver owner's shared event
and create a runnable hot loop, then stop for a complete commit plan.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Added receiver-local wait cancellation and source-read ownership scopes.
Closing a non-owner waiting behind another source reader cancels only that
receiver's private wait and maps it to `ClosedResourceError`; the shared event
remains untouched. Closing the active source owner cancels only its private
source-read scope, wakes peers after cleanup, and lets one peer take ownership.
Source outcomes are captured inside the owner scope and classified only after
checking close/cancel state. A cancellation-shielding source therefore cannot
publish a returned value, ordinary error, or EOC after its owner was closed.
The private scope's `cancel_called` bit is asserted to imply receiver closure;
outer task cancellation remains `trio.Cancelled` and is not translated into
`ClosedResourceError`. Owner-key comments document that only the receiver
identified by `recv_ready[0]` may cancel the shared source-read scope. The
same explicit-close invariant is enforced symmetrically for private peer-wait
scope cancellation.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added deterministic bounded regressions for both close positions. The
non-owner test places two peers behind an active source read, closes one and
proves only that peer exits while the shared event stays unset. The owner test
closes a source owner whose receive shields cancellation and parameterizes a
returned value, `RuntimeError`, and `EndOfChannel`; each discarded outcome
hands the next source receive to the waiting root without terminal-state or
EOC publication.
Verification output:
```text
................. [100%]
17 passed in 5.84s
```
Python compilation and `git diff --check` passed. Iterative adversarial review
caught a waiting non-owner hang, cancellation-shielded source returns, and
shielded source exceptions. All were fixed. Final review found no actionable
issues.

View File

@ -0,0 +1,34 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T21:31:17Z
git_ref: 51185487
scope: code
substantive: true
raw_file: 20260812T213117Z_51185487_prompt_io.raw.md
---
## Prompt
For Tractor PR 490 item four, expose `raise_on_lag` through the public IPC and
asyncio linked-channel subscription wrappers. Review, verify and return a
complete commit plan before applying the API downstream in piker.
## Response summary
Added public per-subscription lag policy to both wrappers, preserved first-call
root policy, documented the semantics, and covered forwarding plus real fan-out
paths.
## Files changed
- `tractor/_streaming.py` - `MsgStream` lag policy forwarding.
- `tractor/to_asyncio.py` - linked-channel lag policy forwarding.
- `docs/guide/streaming.rst` - IPC fan-out policy docs.
- `docs/guide/asyncio.rst` - linked-channel fan-out policy docs.
- `tests/test_task_broadcasting.py` - wrapper policy regression.
## Human edits
None - generated output follows the requested fourth iterative item.

View File

@ -0,0 +1,53 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T21:31:17Z
git_ref: 51185487
diff_cmd: git diff HEAD~1..HEAD
---
The user requested the fourth iterative refinement for Tractor PR 490: expose
subscriber lag policy through the public `MsgStream.subscribe()` and
`LinkedTaskChannel.subscribe()` wrappers, then stop for a complete commit
plan. This enables piker to replace private receiver mutation.
> `git diff HEAD~1..HEAD -- tractor/_streaming.py`
Added `raise_on_lag: bool = True` to `MsgStream.subscribe()`. The first call
passes the policy to the irreversibly allocated root broadcaster and its
child; later calls configure each child independently while retaining the
root's first-call policy.
> `git diff HEAD~1..HEAD -- tractor/to_asyncio.py`
Added equivalent lag-policy forwarding to `LinkedTaskChannel.subscribe()`.
> `git diff HEAD~1..HEAD -- docs/guide/streaming.rst`
> `git diff HEAD~1..HEAD -- docs/guide/asyncio.rst`
Documented strict versus warn/drop/resume behavior, independent child policy,
and first-call root policy for both wrapper types.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added a parameterized wrapper-level regression using minimal receive-compatible
handles. It verifies a first non-raising subscription configures root and
child, then a later strict child does not mutate the sticky root policy.
Verification output:
```text
................... [100%]
19 passed in 5.71s
.................... [100%]
20 passed in 6.88s
. [100%]
1 passed in 0.86s
```
The second and third runs cover actual `MsgStream` and infected-asyncio
`LinkedTaskChannel` fan-out respectively. Python compilation and
`git diff --check` passed. Adversarial review found no actionable issues.

View File

@ -0,0 +1,36 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: unavailable
timestamp: 2026-08-13T18:19:01Z
git_ref: a2e0df4b
scope: code
substantive: true
raw_file: 20260813T181901Z_a2e0df4b_prompt_io.raw.md
---
## Prompt
Continue Tractor PR 490 after the paired piker EMS consumer commit. Clean
cancelled-task diagnostics, define or reject zero-buffer broadcast behavior,
review and verify the change, then stop at a complete commit plan.
## Response summary
Bound cancellation diagnostics to receiver progress, terminal state and
resource lifetime; made EOC durable across peers; released wrapper-owned root
broadcasters without breaking graceful EOC or subclass overrides; and rejected
non-positive fan-out retention capacity.
## Files changed
- `tractor/trionics/_broadcast.py` - diagnostic lifecycle, durable EOC and
buffer validation.
- `tractor/_streaming.py` - safe `MsgStream` root broadcaster cleanup.
- `tractor/to_asyncio.py` - linked-channel root broadcaster cleanup.
- `tests/test_task_broadcasting.py` - cancellation, EOC, wrapper and capacity
regressions.
## Human edits
None - generated output follows the requested fifth iterative item.

View File

@ -0,0 +1,58 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-13T18:19:01Z
git_ref: a2e0df4b
diff_cmd: git diff HEAD~1..HEAD
---
The user asked to continue after committing the paired piker EMS consumer
fix. The next isolated Tractor PR 490 item was to clean cancelled-task
diagnostics and define zero-buffer broadcast behavior, then review, test and
stop at a complete commit plan.
> `git diff HEAD~1..HEAD -- tractor/trionics/_broadcast.py`
Made `BroadcastState.cancelled` transient: receiver progress and close clear
that receiver's diagnostic, terminal EOC and shared receive failure clear all
stale cancelled tasks, and durable EOC prevents peers from re-entering the
closed source. `broadcast_receiver()` now rejects non-positive retention
capacity before creating an unusable zero-length deque.
> `git diff HEAD~1..HEAD -- tractor/_streaming.py`
Made explicit `MsgStream.aclose()` release its internally allocated root
broadcaster while preserving graceful receive-internal EOC teardown. Used a
task-local marker so the public zero-argument `aclose()` signature and valid
subclass overrides remain compatible.
> `git diff HEAD~1..HEAD -- tractor/to_asyncio.py`
Made `LinkedTaskChannel.aclose()` release its internally allocated root
broadcaster before closing the underlying Trio receive channel.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added synchronized regressions for transient child cancellation diagnostics,
cross-receiver terminal cleanup, durable EOC peer wakeups, root broadcaster
cleanup through both public wrappers, `MsgStream.aclose()` subclass
compatibility, and zero-buffer rejection.
Verification output:
```text
........................... [100%]
27 passed in 5.89s
. [100%]
1 passed in 1.22s
. [100%]
1 passed in 0.88s
```
The integration runs cover real `MsgStream` actor fan-out and infected-asyncio
`LinkedTaskChannel` fan-out. Ruff, Python compilation and `git diff --check`
passed. Repeated adversarial review found and resolved root close re-entrancy,
cross-receiver terminal retention, durable-EOC and subclass-compatibility
issues; final review reported no findings.

View File

@ -0,0 +1,69 @@
---
model: gpt-5.6-sol
service: opencode
session: intercepted-claude-7b9c97c4-fff7-4ac4-97fb-35720453308e
timestamp: 2026-08-18T02:17:29Z
git_ref: d9a6e2e9
scope: code
substantive: true
raw_file: 20260818T021729Z_d9a6e2e9_prompt_io.raw.md
---
## Prompt
> Intercept Claude session
> `7b9c97c4-fff7-4ac4-97fb-35720453308e`, pick up where it
> stopped in its open worktree, finish the tunnelled-address change
> and commit plan, and prepare any outstanding context for another
> provider.
The recovered final prompt specifically called out the `uds` versus
`unix` boundary decision, absent concrete `Address.namespace`
implementations, the corrected namespace test, and the passing
focused and wider suites.
During review, the human further required that tunnelled-maddr work
delegate to `multiaddr`'s encapsulation APIs, challenged the premature
transport-shaped listener hooks and directed their removal, corrected
the long-term bearer provisioning model, and confirmed the intended
split between discovery metadata and bindspace lifecycle code.
## Response summary
Recovered the transcript and matched it to `wkts/addr_unpacking`,
audited the staged implementation, and completed the interrupted
verification and commit-plan work. The audit removed premature
transport-shaped listener hooks, widened the namespace identifier
type, updated stale import documentation, and removed an
invalid-escape warning from the maddr diagram. It also preserved the
layer-C design where tractor provisions the kernel-owned tunnel
bearer without treating it as a message transport.
## Files changed
- `tractor/discovery/_tunnel.py` - tunnel specs, address wrapper, and
peeling helpers.
- `tractor/discovery/_addr.py` - wrapped-address recognition and
namespace typing.
- `tractor/discovery/__init__.py` - public tunnel API exports.
- `tests/discovery/test_tunnelled_addr.py` - delegation and boundary
regression coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - distinguish
tractor-owned bindspace provisioning from kernel socket ownership.
## Human edits
Substantial human-directed editing occurred over several review turns:
- required use of `multiaddr`'s `.encapsulate()`/`.decapsulate()`
family rather than a hand-rolled tunnel peeler
- rejected the premature `start_listener()`/`close_listener()` hooks
and directed their removal from this foundational change
- corrected the documentation so tractor retains ownership of future
bindspace provisioning while the kernel owns the bearer socket
- reviewed and accepted the placement of declarative tunnel metadata
under `tractor.discovery`, with lifecycle code kept separate
The final source lines were applied through the coding agents, but
these design corrections and deletion decisions came from the human
review and materially shaped the patch.

View File

@ -0,0 +1,66 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-18T02:17:29Z
git_ref: d9a6e2e9
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - tunnelled-address handoff completion
Recovered Claude Code session
`7b9c97c4-fff7-4ac4-97fb-35720453308e` and continued its
interrupted `wkts/addr_unpacking` changes.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py`
Added frozen `WGTunnelSpec` and `TunnelledAddress` structs. The
wrapper delegates transport identity, validity, bindspace, and wire
serialization to its overlay while retaining tunnel metadata locally.
Added pure helpers to peel nested wrappers and enumerate their tunnel
specs. The module documents why wrappers must be peeled before
`Endpoint` selects the overlay transport backend.
> `git diff HEAD~1..HEAD -- tractor/discovery/_addr.py`
Extended `is_wrapped_addr()` to recognize `TunnelledAddress` without
registering tunnels as message transports, and widened the namespace
identifier type to cover named network namespaces.
> `git diff HEAD~1..HEAD -- tractor/discovery/__init__.py`
Exported the tunnel address API from `tractor.discovery` and updated
the eager-import documentation.
> `git diff HEAD~1..HEAD -- tests/discovery/test_tunnelled_addr.py`
Added focused coverage for delegation, serialization, rewrapping,
namespace fallback, nested peeling order, and frozen structs.
> `git diff HEAD~1..HEAD -- ai/tpt-backends/03_wg_tunnel_bindspace.md`
Clarified that tractor owns the eventual bindspace lifecycle and may
provision the WireGuard iface, routes, and kernel UDP listener through
netlink/`pyroute2`. Kernel socket ownership does not make the bearer an
application `MsgTransport` endpoint.
## Verification
Focused tests:
```text
9 passed in 0.03s
```
Discovery and IPC suites:
```text
67 passed, 2 xpassed
```
The audit removed premature module-level listener hooks. Runtime
integration must peel the wrapper explicitly at bind and dial
boundaries rather than make `._tunnel` impersonate a transport
backend.

View File

@ -0,0 +1,50 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking-followup
timestamp: 2026-08-18T07:50:31Z
git_ref: dd02c7c0
scope: code
substantive: true
raw_file: 20260818T075031Z_dd02c7c0_prompt_io.raw.md
---
## Prompt
The human requested the complete tunnelled-maddr parsing/composition
slice as an unattended batch, with every numbered requirement finished
and atomic commit plans prepared at the end. Existing human decisions
required native `multiaddr` encapsulation APIs, no hand-written peeler,
and preservation of tractor's future bindspace lifecycle ownership.
## Response summary
Implemented strict WG key codecs and native single/nested tunnel maddr
parsing and composition, integrated them into discovery APIs, migrated
the multihost example off its duplicate parser, corrected package
dependency metadata, and added focused and end-to-end parser
regressions. Verified the complete tractor suite and built both package
artifacts.
## Files changed
- `tractor/discovery/` - WG codecs, parser/composer, wrapper typing,
public exports, and discovery dispatch.
- `tests/discovery/` - key, grammar, nesting, round-trip, and public
boundary regressions.
- `examples/multihost/wg_lan/` - production parser migration and
updated usage documentation.
- `pyproject.toml`, `uv.lock` - reproducible WG codec and multibase
dependencies for checkout and package installs.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - current layer-A state
and future bindspace ownership.
## Human edits
The human selected the five-step scope and batch execution model,
required delegation to `multiaddr`'s encapsulation APIs, rejected
transport-shaped listener placeholders in the prerequisite commit, and
clarified that tractor will eventually provision the kernel-owned
bearer through its bindspace layer. The agent implemented and tested
those decisions; no direct manual source edits were observed during
this batch.

View File

@ -0,0 +1,61 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-18T07:50:31Z
git_ref: dd02c7c0
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - native WireGuard maddr integration
The human requested completion of the five-step tunnelled-maddr slice:
port the proven WireGuard parser, delegate to `py-multiaddr`'s native
tunnel APIs, integrate public parse and composition entry points, add
regressions, and return atomic commit plans after completing the batch.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/_multiaddr.py tractor/discovery/_addr.py tractor/discovery/__init__.py`
Added strict WireGuard standard-base64/multibase key codecs and native
WG maddr parsing/composition. Nested stacks peel the last `/wg/`
repeatedly with `.decapsulate_code()`, isolate segments through
`.split()`/`.join()`, and compose recursively with `.encapsulate()`.
Public discovery parsing, wrapping, endpoint-table parsing, and maddr
composition now preserve `TunnelledAddress` metadata locally.
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock`
Declared `py-multibase` directly and pinned the unreleased
py-multiaddr#108 WG codec revision in distribution metadata. Enabled
Hatch direct-reference metadata so editable, wheel, and sdist builds
use the same dependency contract.
> `git diff HEAD~1..HEAD -- tests/discovery/test_multiaddr.py tests/discovery/test_tunnelled_addr.py`
Added key-codec, single/nested WG round-trip, malformed key, invalid
bearer/overlay, missing codec, missing bearer, `wrap_address()`, and
`parse_endpoints()` coverage while retaining plain TCP/UDS regressions.
> `git diff HEAD~1..HEAD -- examples/multihost/wg_lan ai/tpt-backends/03_wg_tunnel_bindspace.md`
Removed the duplicate example parser, migrated callers to production
`TunnelledAddress`, retained only explicit `wg(8)` verification, and
updated dependency and future bindspace ownership documentation.
## Verification
- focused discovery files: `45 passed`
- discovery and IPC suites: `81 passed, 2 xpassed`
- full suite: `443 passed, 9 skipped, 7 xfailed, 4 xpassed`
- collection: `462 tests`
- Ruff: clean
- sdist and wheel builds: successful
## Review corrections
A review caught that `[tool.uv.sources]` alone would not constrain
built-package consumers to a WG-capable py-multiaddr revision. The
dependency was moved into PEP 621 metadata and the parser now converts
missing-codec failures into an actionable tractor error without
misclassifying Unix paths containing a `wg` directory.

View File

@ -0,0 +1,37 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking-followup
timestamp: 2026-08-19T21:31:44Z
git_ref: f81fc5e5
scope: code
substantive: true
raw_file: 20260819T213144Z_f81fc5e5_prompt_io.raw.md
---
## Prompt
The human requested runtime boundary integration with the established
per-step implementation and commit-plan workflow.
## Response summary
Kept `TunnelledAddress` available to callers while peeling it at the
last outbound boundary before transport lookup and dialing. Added a
regression which captures both transport arguments and confirms plain
TCP behavior is unchanged.
## Files changed
- `tractor/ipc/_chan.py` - peel tunnel annotations before outbound
transport dispatch and connection.
- `tests/ipc/test_channel_tunnel_boundary.py` - verify plain and
tunnelled channel inputs deliver only TCP overlays.
## Human edits
The human chose the runtime-boundary slice, required the existing
per-step commit-plan flow, and previously established that wrappers
must retain bindspace metadata without impersonating transports. The
agent implemented those constraints; no direct manual source edits were
observed during this step.

View File

@ -0,0 +1,25 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-19T21:31:44Z
git_ref: f81fc5e5
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - outbound tunnel boundary
The human requested the next tunnelled-address slice using the same
per-step commit-plan flow. Existing design decisions require retaining
tunnel metadata until the narrow IPC transport boundary and never
teaching exact-type transport tables about tunnel wrappers.
> `git diff HEAD~1..HEAD -- tractor/ipc/_chan.py tests/ipc/test_channel_tunnel_boundary.py`
Extended channel address inputs to accept tunnel declarations, then
called `strip_tunnels()` immediately before exact-type transport lookup
and `connect_to()`. Added plain/tunnel parameterized coverage proving
both operations receive the identical TCP overlay while the original
wrapper retains its tunnel spec.
Verification included focused IPC tests, Ruff, discovery/IPC suites,
and the full tractor suite.

View File

@ -0,0 +1,40 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking-followup
timestamp: 2026-08-19T21:31:45Z
git_ref: f81fc5e5
scope: code
substantive: true
raw_file: 20260819T213145Z_f81fc5e5_prompt_io.raw.md
---
## Prompt
The human requested completion of inbound runtime peeling using the
same per-step implementation and commit-plan workflow.
## Response summary
Preserved tunnel declarations through listener configuration, peeled
them immediately before `Endpoint` construction, and used the overlay
for backend-specific random listener allocation after registry
discovery. Added a real listener regression for the reflection and
exact-type boundary.
## Files changed
- `tractor/ipc/_server.py` - accept wrapper declarations and peel at
`Endpoint` construction.
- `tractor/_root.py` - allocate random transport addresses from the
contacted registry's overlay.
- `tests/ipc/test_server_tunnel_boundary.py` - verify a real listener
stores only TCP while preserving the source annotation.
## Human edits
The human selected the runtime-boundary work and previously corrected
the architecture so tractor retains future bindspace provisioning
ownership while `Endpoint` sees only application transports. The agent
implemented and tested that direction; no direct manual source edits
were observed during this step.

View File

@ -0,0 +1,25 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-19T21:31:45Z
git_ref: f81fc5e5
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - inbound tunnel boundary
The human requested runtime boundary integration while preserving the
future tractor-owned bindspace lifecycle.
> `git diff HEAD~1..HEAD -- tractor/ipc/_server.py tractor/_root.py tests/ipc/test_server_tunnel_boundary.py`
Broadened listener declarations to carry tunnel wrappers until
`_serve_ipc_eps()` and peeled immediately before `Endpoint`
construction. Also peeled a contacted tunnelled registry before
backend-specific random listener allocation. Added a real TCP listener
regression proving `Endpoint` stores only the resolved overlay while
the original declaration retains bindspace metadata.
Verification included `465` collected tests, `84` passing
discovery/IPC tests with two xpasses, Ruff, and the full suite with
`447` passes.

View File

@ -0,0 +1,41 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking-followup
timestamp: 2026-08-20T02:15:16Z
git_ref: dfad66a0
scope: docs
substantive: true
raw_file: 20260820T021516Z_dfad66a0_prompt_io.raw.md
---
## Prompt
The human requested that the bindspace plan preserve the agreed
capability, spawn-bootstrap, endpoint-role, namespace augmentation,
random-address, and teardown semantics, using `github/ns_aware` as
prototype input.
## Response summary
Updated plan-03 and the shared backend contract to separate serializable
bindspace declarations from scoped live capabilities, make namespace
entry a pre-runtime spawn operation, keep maddr paths role-neutral, and
define listen/dial provisioning plus ownership-sensitive teardown.
## Files changed
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - layer-C capability,
bootstrap, role, teardown, test, and risk model.
- `ai/tpt-backends/00_shared_backend_contract.md` - distinguish
transport bind selectors from process namespace lifecycle.
## Human edits
The human supplied the core architecture: structured scoped
capabilities, spawn-time namespace entry, orthogonal namespace
augmentation, source/destination-dependent provisioning, and
role-dependent teardown. They also rejected premature assumptions about
`open_bindspace()` returning an address and requested grounding in the
existing namespace prototype. The agent translated those decisions into
the plan text; no direct manual source edits were observed.

View File

@ -0,0 +1,34 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-20T02:15:16Z
git_ref: dfad66a0
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - bindspace capability design
The human corrected the layer-C design around local network-stack
realization. They established that bindspace state should be both
structured and a scoped capability; namespace entry belongs in
subactor bootstrap; maddrs can describe source or destination network
paths while namespace selection augments them orthogonally; random
address and teardown behavior depend on operation role and ownership.
They directed comparison with the prototype on `github/ns_aware` and
requested these decisions be preserved in the plan.
> `git diff HEAD~1..HEAD -- ai/tpt-backends/03_wg_tunnel_bindspace.md ai/tpt-backends/00_shared_backend_contract.md`
Reworked layer C around serializable `BindspaceSpec`, stable
`BindspaceIdentity`, and scoped non-serializable `BindspaceHandle`
concepts. Namespace FDs pin identity and lifetime; parent/supervisor
provisioning transfers entry capability through spawn; the child enters
before runtime, channels, listeners, sockets, or worker threads and then
drops authority. Listen/dial roles and owned/borrowed teardown are
explicit, while maddrs remain role-neutral network-path declarations.
The shared backend contract now separates transport-level `.bindspace`
selectors from process namespace lifecycle. Added tests/risks for FD
identity, bootstrap ordering, privilege drop, role ownership, and
shared-resource teardown.

View File

@ -0,0 +1,41 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-20T03:31:07Z
git_ref: ba07e09d
scope: code
substantive: true
raw_file: 20260820T033107Z_ba07e09d_prompt_io.raw.md
---
## Prompt
The human requested canonical tagged transport addresses with a
reader-first migration. TCP should decode `('tcp', host, port)`, Unix
should decode `('unix', path)`, `uds` should remain an accepted input
alias and internal transport key, and legacy tuple/list inputs must keep
working before writers switch formats.
## Response summary
Introduced canonical and compatibility address aliases, explicit tagged
dispatch, transport-specific tagged readers, and focused serialization
tests. Kept legacy pair inputs and native IPv6 socket values readable so
this boundary can ship before tagged emission.
## Files changed
- `tractor/discovery/_addr.py` - address aliases and tagged dispatch.
- `tractor/ipc/_tcp.py` - tagged, legacy, and IPv6 TCP decoding.
- `tractor/ipc/_uds.py` - canonical Unix and UDS-alias decoding.
- `tests/discovery/test_address_serialization.py` - reader compatibility
coverage.
## Human edits
The human supplied the canonical `tcp` and `unix` forms, chose `uds` as
an input-only serialization alias while preserving it as the runtime
transport key, and required a reader-first commit boundary. The agent
implemented those decisions; no direct manual source edits were
observed.

View File

@ -0,0 +1,25 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-20T03:31:07Z
git_ref: ba07e09d
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - tagged address readers
The human requested a migration away from ambiguous untagged transport
tuples. They established `('tcp', host, port)` and `('unix', path)` as
canonical forms, retained `('uds', path)` as an input alias, and required
a reader-first compatibility boundary before changing emitted values.
> `git diff HEAD~1..HEAD -- tractor/discovery/_addr.py tractor/ipc/_tcp.py tractor/ipc/_uds.py tests/discovery/test_address_serialization.py`
Added explicit tagged address aliases and dispatch, taught TCP and UDS
readers to decode tagged tuple/list payloads, preserved legacy pair input,
and retained native IPv6 socket-address decoding. Added focused tests for
canonical tags, the UDS alias, msgpack-style lists, legacy pairs, and IPv6
socket values.
Focused reader tests and Ruff checks passed before the writer migration
was applied.

View File

@ -0,0 +1,45 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-20T03:31:08Z
git_ref: ba07e09d
scope: code
substantive: true
raw_file: 20260820T033108Z_ba07e09d_prompt_io.raw.md
---
## Prompt
The human asked the agent to continue after adding tagged readers,
complete canonical address emission without dropping legacy input
compatibility, verify the migration, and prepare a complete multi-commit
package when the turn was done.
## Response summary
Changed `.unwrap()` to emit tagged TCP and Unix addresses, updated direct
tuple consumers and spawn payload declarations, and aligned multiaddr,
runtime, IPC, and discovery tests with canonical serialized equality.
Kept untagged tuples and the `uds` spelling readable at input boundaries.
## Files changed
- `tractor/discovery/_addr.py` - canonical output alias.
- `tractor/discovery/_multiaddr.py` - tagged address composition.
- `tractor/ipc/_tcp.py` - tagged emission and direct socket dialing.
- `tractor/ipc/_uds.py` - tagged full-path emission.
- `tractor/msg/types.py` - protocol-neutral spawn tuple containers.
- `tests/discovery/test_address_serialization.py` - writer assertions.
- `tests/discovery/test_multiaddr.py` - canonical round-trip assertions.
- `tests/discovery/test_tpt_bind_addrs.py` - tagged bind assertions.
- `tests/ipc/test_each_tpt.py` - canonical runtime address assertions.
- `tests/ipc/test_server_tunnel_boundary.py` - tagged TCP destructuring.
- `tests/test_local.py` - canonical registry comparison.
## Human edits
The human established the reader-before-writer sequencing, canonical tag
spellings, retained compatibility expectations, and requested final
multi-commit packaging. The agent implemented and tested those choices;
no direct manual source edits were observed.

View File

@ -0,0 +1,26 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-20T03:31:08Z
git_ref: ba07e09d
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - canonical tagged address writers
After the reader compatibility boundary, the human asked the agent to
continue the migration and package the completed work as dependency-
ordered commits.
> `git diff HEAD~1..HEAD -- tractor/discovery/_addr.py tractor/discovery/_multiaddr.py tractor/ipc/_tcp.py tractor/ipc/_uds.py tractor/msg/types.py tests/discovery/test_address_serialization.py tests/discovery/test_multiaddr.py tests/discovery/test_tpt_bind_addrs.py tests/ipc/test_each_tpt.py tests/ipc/test_server_tunnel_boundary.py tests/test_local.py`
Switched TCP and Unix `.unwrap()` output to canonical tagged tuples,
updated direct transport and multiaddr consumers, widened spawn message
tuple containers for protocol-specific shapes, and migrated runtime and
test comparisons to serialized address equality. Legacy inputs remain
accepted at `wrap_address()` and backend reader boundaries.
Ruff and focused tests passed. The complete non-debugger TCP suite passed
with 412 tests; the UDS suite reached 80% without failure before the
harness timeout, then all 97 remaining tests passed on resume. Debugger
PTY coverage was excluded after an unrelated timeout.

View File

@ -0,0 +1,49 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-21T23:32:04Z
git_ref: 5d92595f
scope: code
substantive: true
raw_file: 20260821T233204Z_5d92595f_prompt_io.raw.md
---
## Prompt
The human asked to proceed with WireGuard Layer B from PR #505's
head, using the established one-change-per-turn workflow and
finishing with a commit plan. This turn was limited to read-only
pyroute2 inspection; peer verification and bindspace provisioning
remain later changes.
During review, the human required the generated Python to follow the
deployed `/py-codestyle` rules and asked for the medium-term plan to
remove pyroute2's asyncio I/O runtime from the Trio read path.
## Response summary
Added Linux-only read helpers for WireGuard device and peer public
keys. Pyroute2's synchronous API is fully contained in a Trio worker
thread, supports named netns reads without creation side effects,
validates decoded keys and always closes the netlink client.
Follow-up edits added the required local annotations, boolean layout,
helper docstrings and 69-column source formatting.
## Files changed
- `pyproject.toml` - Linux-only `wg` optional dependency.
- `uv.lock` - resolved pyroute2 0.9.6 metadata.
- `tractor/discovery/_tunnel.py` - read-only WireGuard helpers.
- `tractor/discovery/__init__.py` - public helper exports.
- `tests/discovery/test_wg_inspection.py` - fake-backed netlink and
worker-thread regressions.
## Human edits
The human chose the exact #505 head as a stacked base, required
incremental changes ending in commit plans, and limited this turn to
read-only pyroute2 inspection. The human then identified that Ruff
success had not established `/py-codestyle` compliance and directed
the agent to correct the Python-specific annotation, documentation
and layout rules. No direct manual source edits were observed.

View File

@ -0,0 +1,31 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-21T23:32:04Z
git_ref: 5d92595f
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - read WireGuard state through pyroute2
The human authorized the first incremental WireGuard Layer B change
as a stacked branch from tractor PR #505, with one atomic change and
a commit plan at the end of the turn.
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock tractor/discovery/__init__.py tractor/discovery/_tunnel.py tests/discovery/test_wg_inspection.py`
Added a Linux-only `wg` extra using pyroute2 0.9.6, plus
asynchronous public helpers for reading one interface's public key
and peer keys. The complete synchronous open/read/parse/close
sequence runs in a Trio worker thread because pyroute2 owns a private
asyncio loop internally.
Named namespace reads pass `flags=0` to override pyroute2's `O_CREAT`
default, ensuring inspection cannot create a missing namespace. Fake
netlink messages cover multipart dumps, key validation, stable peer
deduplication, worker-thread execution, netns selection and cleanup
on success/error.
Ruff and lock checks passed. Focused tunnel/multiaddr coverage passed
47 tests; the complete discovery suite passed 88 tests with 2
xpasses.

View File

@ -0,0 +1,49 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: 76c5d31c-5a2f-4503-9b16-410ee7f4fab3
timestamp: 2026-08-22T02:25:26Z
git_ref: eb3c99c9
scope: config
substantive: true
raw_file: 20260822T022526Z_5562fd9a_prompt_io.raw.md
---
## Prompt
Perform a full Tractor repository scan for related `ai.skillz` work,
then correct the run-tests landing branch, prune unrelated `.gitignore`
additions, preserve only the focused migration, and provide canonical
deployment commands.
## Response summary
Audited all local branches, worktrees, affected-path history, deployment
state, canonical skill dependencies, and current Tractor harness behavior.
Corrected the local test reference where it overstated cleanup safety or
omitted current environment, platform, debugger, timeout, and CI details.
Narrowed the correction commit to three managed `run-tests` deployment
blocks. A later dedicated commit records the complete generated `ai.skillz`
deployment state.
## Files changed
- `.claude/skills/run-tests/test-harness-reference.md` - correct the
project-specific test and cleanup contract.
- `.gitignore` - narrow the correction commit before the later dedicated
deployment-state expansion.
- `ai/prompt-io/opencode/20260822T022526Z_5562fd9a_prompt_io.md` - record
the migration review provenance.
- `ai/prompt-io/opencode/20260822T022526Z_5562fd9a_prompt_io.raw.md` -
preserve the unedited response record.
## Human edits
The human required an existing-work scan after duplicate implementation
was discovered, approved correcting the landing branch during PR #481
review, and directed removal or reconciliation of unrelated ignore rules.
During PR #510 review, the human required the stackscope and shared-memory
safety clarifications and immutable provenance pointers before landing. No
direct source-line edits were made by the human. Copilot review then prompted
the human to require explicit canonical deployment instructions and clarify
the later `.gitignore` expansion.

View File

@ -0,0 +1,24 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-22T02:25:26Z
git_ref: eb3c99c9
diff_cmd: git diff eb3c99c9^..eb3c99c9
---
Audit Tractor's repository-wide `ai.skillz` state and correct the
`wkt/ai_skillz_run_tests_landing` migration before landing it.
> `git diff eb3c99c9^..eb3c99c9 -- .claude/skills/run-tests/test-harness-reference.md`
The harness reference is corrected against current Tractor behavior:
environment overrides, CI/platform matrices, randomized registry
addresses, interactive debugger capture, timeout mechanisms, stackscope
artifacts, reaper platform limits, dry-run expansion, and exact
change-to-test mappings.
> `git diff eb3c99c9^..eb3c99c9 -- .gitignore`
Unrelated Taken and broad OpenCode command ignore additions are removed;
only the managed canonical `run-tests` link and command blocks remain in
this migration.

View File

@ -0,0 +1,46 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-22T02:32:26Z
git_ref: 59a8ecfd
scope: code
substantive: true
raw_file: 20260822T023226Z_59a8ecfd_prompt_io.raw.md
---
## Prompt
After committing the read-only pyroute2 helpers and `wgman` design
update, the human authorized the next isolated Layer B change:
explicit `verify_wg_peer()` composition over WireGuard inspection.
## Response summary
Added and exported async `verify_wg_peer()` using one validated
WireGuard key snapshot. It recognizes local-interface and configured
peer identities without coupling kernel inspection to address
parsing. Updated the multihost examples to use the production helper
and removed their subprocess-based probe.
## Files changed
- `tractor/discovery/_tunnel.py` - shared async snapshot reader and
explicit verification helper.
- `tractor/discovery/__init__.py` - public verification export.
- `tests/discovery/test_wg_inspection.py` - local, peer, absent and
malformed-key verification coverage.
- `examples/multihost/wg_lan/host_a_srv.py` - async local-key check.
- `examples/multihost/wg_lan/host_b_client.py` - async peer-key check.
- `examples/multihost/wg_lan/wg_maddr.py` - removed obsolete
subprocess probe.
- `examples/multihost/wg_lan/README.md` - pyroute2 requirements and
verification workflow.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - async API contract.
## Human edits
The human selected this pre-agreed verification layer as the next
atomic change after reviewing and committing the preceding read and
architecture changes. The agent implemented the source changes; no
direct manual edits or follow-up corrections were observed.

View File

@ -0,0 +1,31 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-22T02:32:26Z
git_ref: 59a8ecfd
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - verify declared WireGuard identities
The human authorized the next incremental Layer B change after
committing the read-only pyroute2 helpers and first-child `wgman`
design update.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_inspection.py examples/multihost/wg_lan ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added async `verify_wg_peer()` over one WireGuard key snapshot. It
validates the declared `WGTunnelSpec.peer_pubkey` before I/O, forwards
the spec's iface/netns, and accepts either the local interface key for
a source/listen declaration or a configured peer key for a
destination/dial declaration.
Refactored worker offload behind one shared async reader so
verification cannot compare two different netlink snapshots. Exported
the helper, added local/peer/absent/malformed-key coverage, and moved
the multihost examples from their local `wg show` subprocess probe to
the production API.
Ruff and lock checks passed. Focused WireGuard/tunnel/multiaddr
coverage passed 51 tests; the complete discovery suite passed 92
tests with 2 xpasses.

View File

@ -0,0 +1,44 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-22T03:25:20Z
git_ref: d35c802b
scope: code
substantive: true
raw_file: 20260822T032520Z_d35c802b_prompt_io.raw.md
---
## Prompt
The human reported the explicit WireGuard verification commit done,
asked the agent to check it, and authorized the next incremental
Layer C change.
## Response summary
Verified commit `d35c802b`, made plain transport namespace behavior
explicit, and retained each original listener declaration beside its
peeled transport address. Endpoint and server diagnostics can now
report a tunnel's namespace without violating the transport boundary.
## Files changed
- `tractor/ipc/_tcp.py` - explicit plain-address namespace property.
- `tractor/ipc/_uds.py` - explicit plain-address namespace property.
- `tractor/discovery/_tunnel.py` - direct overlay namespace delegation.
- `tractor/ipc/_server.py` - retained declaration and namespace
diagnostics.
- `tests/discovery/test_tunnelled_addr.py` - plain and tunnel namespace
behavior.
- `tests/ipc/test_server_tunnel_boundary.py` - declaration retention
and diagnostic coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - concrete endpoint
boundary contract.
## Human edits
The human selected continued incremental implementation after
reviewing and committing the preceding verification layer. The agent
implemented this dependency-ordered namespace slice; no direct manual
edits or follow-up corrections were observed.

View File

@ -0,0 +1,29 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-22T03:25:20Z
git_ref: d35c802b
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - retain endpoint namespace declarations
The human reported the explicit WireGuard verification commit done,
asked for it to be checked, and authorized the next incremental
change.
> `git diff HEAD~1..HEAD -- tractor/ipc/_tcp.py tractor/ipc/_uds.py tractor/discovery/_tunnel.py tractor/ipc/_server.py tests/discovery/test_tunnelled_addr.py tests/ipc/test_server_tunnel_boundary.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Confirmed commit `d35c802b` and a clean worktree, then implemented the
smallest dependency-ordered Layer C slice. Plain TCP and UDS addresses
now explicitly report no namespace, allowing `TunnelledAddress` to
delegate without an attribute fallback.
Added required `Endpoint.declared_addr` metadata beside the peeled,
resolved `Endpoint.addr`. Endpoint and server diagnostics expose the
declaration's namespace without passing a tunnel wrapper into
transport reflection. Updated the Layer C plan and tests for plain,
tunnelled, endpoint and server namespace behavior.
Ruff passed. Focused namespace tests passed 13 tests; combined
discovery and IPC coverage passed 101 tests with 2 xpasses.

View File

@ -0,0 +1,64 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-22T04:20:26Z
git_ref: 29141f0b
scope: code
substantive: true
raw_file: 20260822T042026Z_29141f0b_prompt_io.raw.md
---
## Prompt
After committing endpoint namespace visibility, the human authorized
continued Layer C implementation.
During staged review, the human requested Literal-derived validation,
ownership documentation, stable-inode clarification, explicit
non-serialization rationale and consolidated invalid-model tests.
## Response summary
Added the foundational bindspace model: serializable declarations and
stable identities are separated from a process-local live capability.
The handle validates names, ownership and FD identity. A global
`ProcessLocal` sentinel blocks default encoding while retaining
msgspec struct behavior.
Review fixes require a positive inode for every realized netns,
derive runtime choices from the Literal aliases and clarify that an FD
integer is not transferable capability authority.
The human then clarified that msgspec structs are useful generic
storage independently of serialization policy, so the live handle now
uses a struct while remaining process-local by contract.
The human first selected an opaque FD wrapper, then recognized that
future process-local handles need the same guard and directed a global
marker under `tractor.msg` instead.
## Files changed
- `tractor/discovery/_bindspace.py` - declaration, identity and live
capability models.
- `tractor/discovery/__init__.py` - public bindspace exports.
- `tractor/msg/_local.py` - reusable process-local struct marker.
- `tractor/msg/__init__.py` - public `ProcessLocal` export.
- `tests/discovery/test_bindspace.py` - serialization and capability
invariant coverage.
- `tests/msg/test_process_local.py` - direct and nested wire rejection.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - concrete initial model
contract.
## Human edits
The human selected the foundational Layer C capability model, then
reviewed the staged patch and directed all eight follow-up changes:
remove distracting ownership variation, consolidate invalid-model
tests, derive containment checks from Literal aliases, require a
stable inode and explain why the live handle is not a msgspec struct.
The human subsequently corrected that last design choice: struct use
does not imply IPC authorization, so `BindspaceHandle` should retain
msgspec's generic storage benefits. The agent applied those
human-directed edits. The human first chose an opaque process-local FD
wrapper, then refined that decision to a reusable global marker under
`tractor.msg` for future local handles. No direct manual source edits
were observed.

View File

@ -0,0 +1,27 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-22T04:20:26Z
git_ref: 29141f0b
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - model bindspace capabilities
The human authorized the next incremental Layer C change after
committing endpoint namespace visibility.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added frozen, serializable `BindspaceSpec` and
`BindspaceIdentity` models for the initial Linux netns kind. Added a
plain slotted `BindspaceHandle` which carries ownership and an optional
namespace FD without being serializable by default msgspec encoding.
The live handle validates declaration/identity kind and name, ownership
values, non-negative FDs and FD/inode correspondence. Serializable
models reject empty identities and kinds without implementations.
Added round-trip, non-serialization, identity-mismatch, empty-identity
and unsupported-kind tests. Ruff passed; discovery plus
message-serialization coverage passed 125 tests with 2 xpasses.

View File

@ -0,0 +1,48 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-25T04:55:57Z
git_ref: fdccfd7e
scope: code
substantive: true
raw_file: 20260825T045557Z_fdccfd7e_prompt_io.raw.md
---
## Prompt
After committing the foundational bindspace capability model, the
human authorized the next incremental Layer C change.
## Response summary
Added a borrow-only existing-netns context manager which pins stable
identity through an open FD and always releases that local descriptor.
The API does not create, enter or remove namespaces and rejects named
path traversal.
Staged review renamed the context to `attach_netns()`, introduced the
readable `CURRENT_NETNS` alias and made key validation errors identify
their exact owning field.
Follow-up review made the empty-key error spell out that `None` is
written as `CURRENT_NETNS` for `BindspaceSpec.key` declarations.
## Files changed
- `tractor/discovery/_bindspace.py` - existing-netns lifecycle and key
validation.
- `tractor/discovery/__init__.py` - public lifecycle export.
- `tests/discovery/test_bindspace.py` - current, named, missing and
traversal coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - borrow-only lifecycle
contract.
## Human edits
The human selected the previously deferred borrow-only netns lifecycle
as the next incremental Layer C change. The agent implemented the
source changes. During staged review, the human selected
`attach_netns()` terminology, requested explicit
`BindspaceSpec.key = CURRENT_NETNS` semantics and field-specific key
validation. Follow-up review requested the validation error itself
connect `None` to `CURRENT_NETNS`. The agent applied those
human-directed edits; no direct manual source edits were observed.

View File

@ -0,0 +1,27 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-25T04:55:57Z
git_ref: fdccfd7e
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - borrow existing netns bindspaces
The human reported the bindspace capability model committed and
authorized the next incremental Layer C change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added async `open_existing_netns()` as a borrow-only context manager.
It opens the current process netns or a named entry under the standard
iproute2 run directory, derives stable identity from the opened FD,
and yields a borrowed process-local `BindspaceHandle`.
The context uses `O_CLOEXEC`, never creates, enters or removes a
namespace, and synchronously closes only its FD on exit. Netns keys
reject paths to keep named lookup beneath the run directory.
Added current, named, missing and traversal tests. Ruff and lock checks
passed; discovery plus message coverage passed 132 tests with 2
xpasses.

View File

@ -0,0 +1,37 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-25T19:05:29Z
git_ref: e1007547
scope: code
substantive: true
raw_file: 20260825T190529Z_e1007547_prompt_io.raw.md
---
## Prompt
After committing the attach-only network namespace lifecycle, the
human authorized the next incremental Layer C change.
## Response summary
Added owned `open_netns()` creation and teardown through shielded
pyroute2 worker calls. The context pins identity through
`attach_netns()`, yields an owned handle, closes its FD before removal
and cleans up despite caller cancellation.
## Files changed
- `tractor/discovery/_bindspace.py` - owned netns lifecycle.
- `tractor/discovery/__init__.py` - public lifecycle export.
- `tests/discovery/test_bindspace.py` - ownership, ordering,
cancellation and name requirements.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - owned lifecycle and
spawn-boundary contract.
## Human edits
The human selected owned network namespace creation as the next
incremental Layer C change. The agent implemented the source changes;
no direct manual edits or follow-up corrections were observed.

View File

@ -0,0 +1,26 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-25T19:05:29Z
git_ref: e1007547
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - own created netns bindspaces
The human reported the attach-only netns lifecycle committed and
authorized the next incremental Layer C change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added async `open_netns()` as the owned counterpart to
`attach_netns()`. It requires a named spec, creates through pyroute2 in
a shielded worker call, attaches the resulting namespace FD and yields
an owned process-local `BindspaceHandle`.
FD closure occurs before shielded namespace removal on normal,
exceptional and cancelled exits. The context never calls `setns()`;
namespace entry remains a spawn/bootstrap responsibility.
Privileged operations are faked in tests. Ruff and lock checks passed;
discovery plus message coverage passed 137 tests with 2 xpasses.

View File

@ -0,0 +1,40 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-25T19:18:45Z
git_ref: 5b2a064a
scope: code
substantive: true
raw_file: 20260825T191845Z_5b2a064a_prompt_io.raw.md
---
## Prompt
The human pointed out that the agent had produced no changes after the
previous commit and expected the next incremental implementation.
## Response summary
Corrected the sequencing mistake and added explicit serialized
bindspace lifecycle policy plus generic `open_bindspace()` dispatch.
Borrowed and owned netns contexts share a policy-neutral FD-pinning
primitive while validating their public lifecycle before side effects.
## Files changed
- `tractor/discovery/_bindspace.py` - lifecycle policy, ownership
invariants, shared pinning and dispatch.
- `tractor/discovery/__init__.py` - public policy and dispatcher exports.
- `tests/discovery/test_bindspace.py` - lifecycle serialization,
validation and both dispatcher branches.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - explicit lifecycle
policy independent of transport role.
## Human edits
The human identified that the agent had accidentally repeated a
summary of already committed work instead of implementing the next
slice. That correction directly caused this lifecycle/dispatcher
change to be implemented. The agent wrote the source changes; no
direct manual edits were observed.

View File

@ -0,0 +1,30 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-25T19:18:45Z
git_ref: 5b2a064a
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - dispatch explicit bindspace lifecycle
The human noticed the agent had re-reported already committed work
without changing the worktree. The agent confirmed the sequencing
error and proceeded to the actual next Layer C slice.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added serialized `BindspaceSpec.lifecycle` policy with explicit
`attach` and `open` choices. Added `open_bindspace()` dispatch without
inferring resource ownership from listen/dial role.
Refactored FD attachment into policy-neutral `_pin_netns()` so
borrowed and owned public lifecycles can share identity pinning while
enforcing their own policy before side effects. Handle construction
also verifies lifecycle and resulting ownership agree.
One composition error introduced during implementation was caught by
the focused test: owned creation initially called the newly guarded
public attach API. `_pin_netns()` fixed that layering error. Ruff and
lock checks passed; discovery plus message coverage passed 138 tests
with 2 xpasses.

View File

@ -0,0 +1,42 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-25T23:46:31Z
git_ref: b973e78c
scope: code
substantive: true
raw_file: 20260825T234631Z_b973e78c_prompt_io.raw.md
---
## Prompt
The human pointed out that the agent had again produced no worktree
changes after the latest commit and expected actual next-layer work.
## Response summary
Corrected the sequencing mistake and added a process-local WireGuard
interface configuration boundary. Secret/routing policy is separated
from serializable maddr-derived tunnel declarations, redacted from
repr and blocked from default wire encoding.
Staged review clarified that address/network parser calls validate
every declaration and do not select or consume a first address.
## Files changed
- `tractor/discovery/_tunnel.py` - local WG config and validation.
- `tractor/discovery/__init__.py` - public config export.
- `tests/discovery/test_wg_config.py` - secrecy and validation coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - secret/config boundary
for future interface provisioning.
## Human edits
The human identified the second sequencing failure where the agent
summarized an already committed slice instead of implementing the next
one. That correction directly caused this new WG configuration layer
to be implemented. During staged review, the human requested explicit
explanation of the address and allowed-route validation loops. The
agent applied that human-directed clarification; no direct manual edits
were observed.

View File

@ -0,0 +1,28 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-25T23:46:31Z
git_ref: b973e78c
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - separate local WireGuard configuration
The human again noticed the agent had re-reported already committed
work without changing the worktree. The agent confirmed the sequencing
mistake and moved to the next unimplemented dependency.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_config.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added process-local `WGInterfaceConfig` for private/preshared keys,
local interface CIDRs, peer allowed CIDRs, listen port and persistent
keepalive. Secrets are redacted from repr and the global
`ProcessLocal` marker blocks default wire encoding.
Removed the unused serialized `WGTunnelSpec.maybe_allowed_ips`
placeholder so maddr-derived declarations retain only public identity,
endpoint and interface selection. Added validation for keys, CIDRs,
ports and keepalive before future kernel mutation.
Ruff and lock checks passed. Focused WG coverage passed 25 tests;
discovery plus message coverage passed 145 tests with 2 xpasses.

View File

@ -0,0 +1,37 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-26T00:14:42Z
git_ref: dcdf4d82
scope: code
substantive: true
raw_file: 20260826T001442Z_dcdf4d82_prompt_io.raw.md
---
## Prompt
The human selected an explicit peer-list model rather than a
dial-only single-peer shortcut before `open_wg_iface()`.
## Response summary
Added process-local per-peer configuration and refactored interface
configuration to own a unique peer tuple. This supports multi-peer
listeners and dial targets without overloading role-dependent tunnel
maddr identity.
## Files changed
- `tractor/discovery/_tunnel.py` - peer model and interface peer list.
- `tractor/discovery/__init__.py` - public peer-config export.
- `tests/discovery/test_wg_config.py` - peer secrecy and validation.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - explicit multi-peer
provisioning contract.
## Human edits
The human chose explicit peer lists over a dial-only implementation so
listeners can represent multiple client public keys and routing
policies. The agent implemented that human-selected design; no direct
manual source edits were observed.

View File

@ -0,0 +1,27 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-26T00:14:42Z
git_ref: dcdf4d82
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - model explicit WireGuard peers
After committing local WG interface configuration, the human selected
an explicit peer-list model before interface provisioning.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_config.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added process-local `WGPeerConfig` with public key, allowed CIDRs,
optional endpoint, preshared key and keepalive. Refactored
`WGInterfaceConfig` to own a tuple of unique peers beside its private
key, local addresses and listen port.
Peer PSKs remain redacted and nested `ProcessLocal` sentinels prevent
default wire encoding. Validation covers keys, routes, endpoints,
ports, keepalive and duplicate peers.
Ruff and lock checks passed. Focused peer/config coverage passed 11
tests. A broad unrelated registrar cancellation case timed out after
86 passes and 2 xpasses; its isolated rerun passed.

View File

@ -0,0 +1,42 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-26T00:34:30Z
git_ref: 6dd39da0
scope: code
substantive: true
raw_file: 20260826T003430Z_6dd39da0_prompt_io.raw.md
---
## Prompt
After committing explicit WireGuard peer configuration, the human
authorized the next incremental lifecycle change.
## Response summary
Added owned WireGuard interface provisioning and teardown through
cancellation-shielded pyroute2 worker calls. Role-specific policy is
validated before mutation, local addresses and explicit peers are
configured, and owned interfaces are removed on exit.
Staged review clarified role-specific test assertions, added the
second peer's endpoint-port check, included conflicting values in
policy errors and aligned annotated assignment with `/py-codestyle`.
## Files changed
- `tractor/discovery/_tunnel.py` - role policy and WG iface lifecycle.
- `tractor/discovery/__init__.py` - public lifecycle and role exports.
- `tests/discovery/test_wg_iface_lifecycle.py` - policy and cancellation
coverage with privileged calls faked.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - initial owned WG iface
lifecycle contract.
## Human edits
The human selected continuation from explicit peer modeling into the
owned WireGuard interface lifecycle. During staged review, the human
requested clearer role assertions, complete endpoint checks, concrete
error values and inline annotated assignment. The agent applied those
human-directed edits; no direct manual source edits were observed.

View File

@ -0,0 +1,29 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-26T00:34:30Z
git_ref: 6dd39da0
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - own WireGuard interface lifecycle
After committing explicit peer configuration, the human authorized the
next incremental lifecycle change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_iface_lifecycle.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added owned async `open_wg_iface()` lifecycle with role-policy
validation before side effects. Listen bearer ports configure the local
listener; dial bearers fill omitted endpoints only for the selected
maddr peer.
Pyroute2 worker code creates the WireGuard link, assigns local
addresses, configures private key/listen port and every explicit peer,
raises the link, and removes it on exit. Partial create failures clean
up synchronously; async create/remove calls are cancellation-shielded.
Privileged operations are faked in tests. Ruff and lock checks passed;
focused WG lifecycle/config/inspection coverage passed 22 tests. A
broad unrelated registrar cancellation parameter timed out after 76
passes and 2 xpasses; its isolated rerun passed.

View File

@ -0,0 +1,47 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-26T02:24:34Z
git_ref: 2245f094
scope: code
substantive: true
raw_file: 20260826T022434Z_2245f094_prompt_io.raw.md
---
## Prompt
After committing the owned WireGuard interface lifecycle, the human
asked to continue with the next established incremental step.
## Response summary
Added a WG-specific composition context which owns one declared
bindspace and an ordered interface stack. It snapshots caller-owned
layer order before checkpointing, enters interfaces outermost-first,
and guarantees inside-out interface teardown before bindspace release.
Its inbound type shows the `(WGTunnelSpec, WGInterfaceConfig)` pairing
directly, and its docstring depicts the stack and unwind order.
## Files changed
- `tractor/discovery/_tunnel.py` - bindspace/interface stack driver.
- `tractor/discovery/__init__.py` - public layer and driver exports.
- `tests/discovery/test_wg_iface_lifecycle.py` - mutable-input,
cancellation and nested teardown-order coverage.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - implemented composition
and ownership-order contract.
## Human edits
The human accepted and committed the preceding reviewed lifecycle
boundary, then explicitly requested the next step. That selected the
previously established bindspace/WireGuard composition boundary. In
staged review, the human requested an explicit interface-stack analogy
and a small ASCII diagram explaining entry and exit order. The human
then identified `WGInterfaceLayer` as an unnecessary single-use alias
which obscured the inbound tuple shape. The agent applied both
human-directed edits. The human accepted the fake lifecycle test as a
focused control-flow check which avoids pyroute2 overhead, while noting
their general preference for end-to-end coverage. No direct manual
source edits were observed.

View File

@ -0,0 +1,28 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-26T02:24:34Z
git_ref: 2245f094
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - compose WireGuard bindspace lifecycles
After committing the owned WireGuard interface lifecycle, the human
requested the next incremental change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_wg_iface_lifecycle.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added `open_wg_bindspace()` to compose one declared bindspace with an
ordered sequence of WireGuard interface layers. The context enters the
bindspace first, enters interfaces outermost-first through
`AsyncExitStack`, yields the live `BindspaceHandle`, and removes
interfaces inside-out before releasing or removing the namespace.
The caller's layer sequence is snapshotted before the first checkpoint,
so concurrent mutation cannot alter resource entry or unwind order.
Fake lifecycle tests clear a mutable input list during bindspace entry,
cancel from the yielded scope, and prove the original stack unwinds in
dependency-safe order.
Ruff passed and focused bindspace/WireGuard coverage passed 39 tests.

View File

@ -0,0 +1,54 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-26T03:05:34Z
git_ref: d130431c
scope: code
substantive: true
raw_file: 20260826T030534Z_d130431c_prompt_io.raw.md
---
## Prompt
After committing the reviewed bindspace/interface stack, the human
asked to continue with the next established Layer C step.
## Response summary
Retained a serializable realized `BindspaceRef` on frozen tunnelled
address declarations without carrying the process-local `Bindspace`
capability. The namespace API remains tuple-shaped, using the declared
key before realization and stable inode afterward. Existing transport
peeling and unrealized serialization remain unchanged.
## Files changed
- `tractor/discovery/_tunnel.py` - realized bindspace-ref retention and
immutable annotation method.
- `tests/discovery/test_tunnelled_addr.py` - ref, compatibility and
mismatch coverage.
- `tests/ipc/test_server_tunnel_boundary.py` - real listener diagnostic
coverage for the realized inode.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - realized namespace
visibility contract.
## Human edits
The human accepted and committed the preceding reviewed composition
boundary, then requested the next step. Their stated preference for
end-to-end tests informed use of the existing real TCP listener path to
verify endpoint/server diagnostics, while focused unit coverage handles
metadata invariants without pyroute2 overhead. During review, the human
chose the final unshipped terminology: `BindspaceRef` for the
serializable non-owning reference, `Bindspace` for the live
process-local capability, `.ref` for that capability's reference, and
`bindspace_ref` at the tunnel declaration API.
They then requested the prerequisite model rename as a separate commit
before this feature. The human also replaced the module-level helper
with `TunnelledAddress.with_bindspace_ref()` and requested inline
msgspec encode/decode expressions in the serialization assertions. The
human chose not to add a second typed `.namespace` projection, and
requested native tagged `TunnelledAddress` decoding remain as a separate
design-plan follow-up. The agent applied those human-directed changes;
no direct manual source edits were observed.

View File

@ -0,0 +1,31 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-26T03:05:34Z
git_ref: d130431c
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - retain realized bindspace identity
After committing WireGuard bindspace composition, the human requested
the next incremental Layer C change.
> `git diff HEAD~1..HEAD -- tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_tunnelled_addr.py tests/ipc/test_server_tunnel_boundary.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added optional `BindspaceIdentity` metadata to frozen
`TunnelledAddress` declarations and a pure
`with_bindspace_identity()` annotation helper. Unrealized declarations
retain their prior serialized shape. Realized declarations retain only
serializable key/inode identity, never the FD-bearing capability.
The existing `.namespace` tuple contract remains compatible:
unrealized declarations report `(kind, key)`, while realized
declarations report the stable `(kind, inode)`. Name mismatches between
the tunnel declaration and realized bindspace are rejected.
Unit coverage verifies immutability, serialization, delegation and
mismatch handling. The existing real TCP listener test proves endpoint
and server diagnostics expose the retained inode without implying that
the process entered that namespace. Ruff passed and focused tunnel,
listener and bindspace coverage passed 30 tests.

View File

@ -0,0 +1,43 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-27T21:11:15Z
git_ref: d130431c
scope: code
substantive: true
raw_file: 20260827T211115Z_d130431c_prompt_io.raw.md
---
## Prompt
During review, the human requested concrete names for the existing
bindspace abstractions and asked to isolate that rename before realized
reference retention.
## Response summary
Renamed the serializable resource record to `BindspaceRef`, the live
FD-backed capability to `Bindspace`, and the capability's record field
to `.ref`. Updated existing lifecycle APIs, tests and active design
contracts without compatibility aliases.
## Files changed
- `tractor/discovery/_bindspace.py` - renamed models and `.ref` field.
- `tractor/discovery/_tunnel.py` - existing lifecycle annotations.
- `tractor/discovery/__init__.py` - renamed public model exports.
- `tests/discovery/test_bindspace.py` - renamed model contracts.
- `tests/discovery/test_wg_iface_lifecycle.py` - live bindspace usage.
- `ai/tpt-backends/00_shared_backend_contract.md` - shared terminology.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - request, ref and live
capability terminology.
## Human edits
The human identified the earlier identity/handle terminology as too
abstract for an IPC-passed non-owning reference and live resource. They
selected `BindspaceRef`, `Bindspace`, and `.ref`, then requested this
rename as a prerequisite commit separate from the realized-ref feature.
The agent applied those human-directed names; no direct manual source
edits were observed.

View File

@ -0,0 +1,26 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-27T21:11:15Z
git_ref: d130431c
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - simplify bindspace model names
While reviewing realized bindspace metadata, the human requested more
concrete names for the existing bindspace model family and asked that
the rename remain separate from the subsequent feature.
> `git diff HEAD~1..HEAD -- tractor/discovery/_bindspace.py tractor/discovery/_tunnel.py tractor/discovery/__init__.py tests/discovery/test_bindspace.py tests/discovery/test_wg_iface_lifecycle.py ai/tpt-backends/00_shared_backend_contract.md ai/tpt-backends/03_wg_tunnel_bindspace.md`
Renamed `BindspaceIdentity` to `BindspaceRef` for the serializable,
non-owning, host-local resource record. Renamed `BindspaceHandle` to
`Bindspace` for the live process-local FD-backed capability, and renamed
its `.identity` field to `.ref`.
No compatibility aliases were added because the models are unshipped.
Active design contracts now distinguish the serialized request, the
wire-safe non-owning ref, and the live capability. Ruff passed, 518
tests collected, and 53 focused bindspace/WireGuard/listener tests
passed across the final two-commit tree.

View File

@ -0,0 +1,41 @@
---
model: gpt-5.6-sol
service: opencode
session: tractor-addr-unpacking
timestamp: 2026-08-27T23:25:00Z
git_ref: d067505a
scope: code
substantive: true
raw_file: 20260827T232500Z_d067505a_prompt_io.raw.md
---
## Prompt
After committing the reviewed bindspace-reference changes, the human
asked to continue with the next Layer C increment.
## Response summary
Added the private Linux network-namespace entry primitive required by
future child bootstrap wiring. It verifies the inherited capability
before `setns()`, constrains entry to network namespaces, and verifies
the resulting process namespace before runtime initialization proceeds.
## Files changed
- `tractor/spawn/_netns.py` - synchronous verified netns entry.
- `tests/test_netns_spawn.py` - stale-FD, syscall and post-entry guards.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - primitive ownership and
remaining integration boundary.
## Human edits
The human requested the next incremental implementation after accepting
the prior split commits. Their established preference for end-to-end
tests informed use of real FDs where possible; only privileged `setns()`
and procfs observation are replaced to keep the suite unprivileged. In
staged review, the human requested the full project license header,
explicit comments on both intentional inode mismatches, and independent
call traces proving both fake syscall and procfs hooks execute. The
agent applied those human-directed revisions; no direct manual source
edits were observed.

View File

@ -0,0 +1,25 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-27T23:25:00Z
git_ref: d067505a
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - verify pre-runtime netns entry
After committing realized bindspace references and their deferred codec
plan, the human requested the next Layer C increment.
> `git diff HEAD~1..HEAD -- tractor/spawn/_netns.py tests/test_netns_spawn.py ai/tpt-backends/03_wg_tunnel_bindspace.md`
Added a private synchronous `enter_netns()` bootstrap primitive. It
validates the inherited FD and expected inode, constrains `setns()` to
`CLONE_NEWNET`, verifies `/proc/self/ns/net` afterward, and leaves FD
closure to the future spawn-bootstrap caller.
Tests use real stand-in FDs while replacing only the privileged syscall
and post-entry procfs observation. They prove stale FDs fail before
entry, the syscall receives the exact namespace type, and bootstrap
rejects an unexpected post-entry namespace. Ruff and all three focused
tests passed.

View File

@ -0,0 +1,45 @@
---
model: gpt-5.6-sol
service: opencode
session: addr-unpacking-rebase-conflict
timestamp: 2026-08-28T00:38:05Z
git_ref: e40c39da
scope: code
substantive: true
raw_file: 20260828T003805Z_e40c39da_prompt_io.raw.md
---
## Prompt
The human asked the agent to resolve the first conflict in the
`wkt/addr_unpacking` rebase because the replayed history was primarily
agent-generated.
## Response summary
Preserved the lazy-import and platform-aware behavior from the new
base while restoring the replayed WireGuard address parsing and
composition behavior. Reconciled the changed dependency lock with the
replayed direct dependency contract and validated the combined result.
## Files changed
- `tractor/discovery/_addr.py` - combined address-platform and tunnel
typing changes.
- `tractor/discovery/_multiaddr.py` - combined lazy imports with WG
parse and composition dispatch.
- `tractor/discovery/_tunnel.py` - retained cold-import behavior for
the newly added WG implementation.
- `tests/discovery/test_multiaddr.py` - adapted missing-codec
monkeypatching to the lazily imported upstream protocol module.
- `uv.lock` - restored the WG-capable multiaddr source and direct
multibase metadata.
## Human edits
The human chose to abort the first rebase attempt, restart with an
explicit old-base boundary, and delegated the conflict resolution to
the agent. The human also chose not to preserve the superseded
module-level protocol-lookup test seam; the agent redirected the test
to the upstream protocol module. No direct manual source edits were
observed during this resolution.

View File

@ -0,0 +1,32 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T00:38:05Z
git_ref: e40c39da
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - addr-unpacking rebase resolution
The human delegated resolution of the first conflict while rebasing
`wkt/addr_unpacking` from `d9a6e2e9` onto `85a44588`.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/discovery/_addr.py tractor/discovery/_multiaddr.py tractor/discovery/_tunnel.py`
Combined the rebased base's platform-aware address registries and lazy
optional-dependency imports with the replayed `TunnelledAddress` typing
and WireGuard multiaddr dispatch. Adapted the new tunnel module so
importing tractor does not eagerly load `multiaddr`.
> `git diff HEAD~1..HEAD -- uv.lock`
Regenerated the lockfile to retain the replayed WG-capable
`py-multiaddr` revision and direct `py-multibase` dependency after the
new base's independently changed lockfile silently omitted that hunk.
## Verification
Conflict-marker, syntax, lock consistency, lint, collection, focused
discovery, and cold-import checks were selected for the resolution.

View File

@ -0,0 +1,38 @@
---
model: gpt-5.6-sol
service: opencode
session: wg-netns-bootstrap-entry
timestamp: 2026-08-28T01:22:05Z
git_ref: 9ae7cd86
scope: code
substantive: true
raw_file: 20260828T012205Z_9ae7cd86_prompt_io.raw.md
---
## Prompt
After completing and auditing the stacked rebase, the human approved
the next WireGuard Layer C slice: integrate the verified network
namespace entry primitive into child spawn bootstrap.
## Response summary
Wire an optional child-owned namespace FD and expected inode into both
child entrypoints. Keep namespace entry and FD closure synchronous and
ordered before actor runtime initialization, while deliberately
deferring backend FD transport and public API plumbing.
## Files changed
- `tractor/spawn/_entry.py` - enter and release an optional inherited
namespace capability before multiprocessing child runtime startup.
- `tractor/_child.py` - consume the capability at the start of the
Trio child bootloader, before patching or actor construction.
- `tests/test_netns_spawn.py` - prove entry ordering, failure isolation,
and exact FD ownership for both child entrypoints.
## Human edits
The human selected spawn-time namespace integration as the next work
slice and authorized implementation. No direct manual source edits
were observed during generation.

View File

@ -0,0 +1,36 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T01:22:05Z
git_ref: 9ae7cd86
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - enter inherited netns during child bootstrap
The human asked to continue the WireGuard Layer C implementation after
the rebased branch landed the verified `spawn._netns.enter_netns()`
primitive.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/_child.py tractor/spawn/_entry.py tests/test_netns_spawn.py`
Add one optional atomic `(namespace_fd, expected_inode)` bootstrap
capability to the multiprocessing entrypoint and Trio child bootloader.
Enter and verify the namespace before Trio patching, actor construction,
runtime state, logging, Trio startup, parent connection, sockets, or
worker threads, then close the child-owned inherited FD before
continuing.
Add privilege-free direct-entrypoint tests using real stand-in FDs and
a fake namespace syscall boundary. Cover helper-level no-op behavior,
successful entry ordering, exact FD closure, malformed-FD isolation,
primary-error preservation, and failure cleanup for both child
bootstrap paths.
## Scope boundary
This increment does not transfer FDs through spawn backends or expose a
public actor API. Backend-specific FD duplication, bootstrap failure
reporting, and root-process namespace entry remain follow-up work.

View File

@ -0,0 +1,47 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: pr490-landing-review
timestamp: 2026-08-28T01:42:47Z
git_ref: wkt/fix_broadcast_lag_count
scope: code
substantive: true
raw_file: 20260828T014247Z_d06ee34b_prompt_io.raw.md
---
## Prompt
Prime PR #490 for landing, review its concurrent broadcast changes,
fix valid findings, and keep the landing pass focused.
## Response summary
Rejected an invalid mutable-default finding under `msgspec.Struct`
semantics. Found and fixed a close-time race where concurrent receives
on one `BroadcastReceiver` could overwrite its sole cancellation scope
and strand an earlier waiter. The receiver now rejects overlapping
calls with `trio.BusyResourceError`. Public docs now define each
receiver as one logical subscription cursor and require one receiver
per concurrent lossless consumer. Busy errors identify the active Trio
task which owns the receiver.
## Files changed
- `tractor/trionics/_broadcast.py` - reject concurrent receives on one
receiver handle.
- `tests/test_task_broadcasting.py` - prove the second call fails while
the original receive remains usable.
- `docs/guide/streaming.rst` - document per-task receiver ownership and
explicit lag behavior.
## Human edits
The human selected PR #490 as the next landing target, rebased it onto
current `main`, directed the agent to continue the landing pass, and
required a faster, tightly scoped execution after the review expanded.
The human then confirmed the per-receiver guard and required the public
method, type, and surrounding guide to emphasize lossless per-consumer
semantics. The human also required the regression's checkpoint loop to
explain its synchronization and the busy error to report the active
consumer task. The agent applied the source, test, and documentation
changes; no direct manual edits to these files were identified.

View File

@ -0,0 +1,56 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T01:42:47Z
git_ref: wkt/fix_broadcast_lag_count
diff_cmd: git diff
---
## Prompt
Get PR #490 primed and landed. Continue after rebasing, but keep the
review and fix pass focused.
## Response
The mutable `{}` default is safe because `msgspec.Struct` creates a
fresh dictionary per instance. A separate concurrency defect remains:
two tasks can call `receive()` through one `BroadcastReceiver`, and the
second call overwrites the only `_wait_scope`. Closing that receiver can
then leave the first task blocked indefinitely.
> `git diff -- tractor/trionics/_broadcast.py tests/test_task_broadcasting.py`
Added a per-receiver in-progress guard that raises
`trio.BusyResourceError` for an overlapping receive. Added a regression
test that blocks the first receive, verifies the second call is
rejected, and releases the source to prove the first call still works.
The complete broadcast test file passes: 28 tests in 4.57 seconds.
## Follow-up prompt
Clarify that the guard is per receiver, preserves lossless consumer
semantics, and document the contract on the public method, type, and
surrounding guide.
## Follow-up response
> `git diff -- tractor/trionics/_broadcast.py docs/guide/streaming.rst`
Documented that each `BroadcastReceiver` owns one logical cursor, each
concurrent consumer needs its own subscribed receiver, overlapping
calls on one handle raise `BusyResourceError`, and strict lag handling
never skips values silently.
## Second follow-up prompt
Explain that the polling loop waits until the background receive is
blocked before making the concurrent call, and include the first
consumer task's runtime information in the busy exception.
## Second follow-up response
Replaced the boolean guard with the active `trio.lowlevel.Task`, added
its name and representation to `BusyResourceError`, named the fixture
task for a deterministic assertion, and documented the checkpoint-loop
interleaving directly above the poll.

View File

@ -0,0 +1,47 @@
---
model: gpt-5.6-sol
service: opencode
session: trio-pre-handshake-child-death
timestamp: 2026-08-28T04:51:19Z
git_ref: fb6d81d3
scope: code
substantive: true
raw_file: 20260828T045119Z_fb6d81d3_prompt_io.raw.md
---
## Prompt
The human approved the next WireGuard Layer C implementation step after
committing child-side network namespace bootstrap.
## Response summary
Restore and harden the boot-race design originally implemented by
`3b0724eba85b4014170ed95773e1e41a60d5c513` before transferring
namespace FDs. Race the initial peer handshake against child-process
death so a pre-connection bootstrap failure surfaces instead of
parking `ActorNursery.start_actor()` forever.
## Files changed
- `tractor/spawn/_spawn.py` - provide the generic handshake/process
death race.
- `tractor/spawn/_trio.py` - use the race during Trio child startup.
- `tests/test_spawning.py` - cover handshake-first and death-first
schedules, simultaneous outcomes, waiter errors, and backend cleanup.
## Human edits
The human approved continued Layer C implementation. The agent narrowed
the next slice to failure detection after analysis showed FD transfer
would otherwise introduce a parent hang. The human then identified the
earlier `wait_for_peer_or_proc_death()` implementation and required
explicit credit; the source docstring and commit message now cite its
original commit rather than a later cherry-pick. The human directly
reflowed `_trio.py` to the project line limit and normalized nearby
quotes. The human also required naming the race helper in its lifecycle
comment, explicit `trio.Cancelled` handling in waiter tests, expected
cancellation diagnostics, and a `trio.Process`-constrained test double.
After mainline portability verification, the human chose to commit the
fix on the current WireGuard branch and require its cherry-pick onto
`main` as a blocker in the eventual WireGuard pull request.

View File

@ -0,0 +1,34 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T04:51:19Z
git_ref: fb6d81d3
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - detect child death before parent handshake
The human asked to continue WireGuard Layer C after child entrypoints
learned to consume inherited network namespace capabilities.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/spawn/_spawn.py tractor/spawn/_trio.py tests/test_spawning.py`
Race a spawned child's initial parent handshake against its process
wait operation. Return the established channel when the handshake wins;
raise `ActorFailure` with the exit status when the process dies first.
Use the race in the Trio exec backend instead of waiting indefinitely
on `IPCServer.wait_for_peer()`.
Add deterministic Trio tests for handshake-first, death-first,
simultaneous, and waiter-error schedules, controlling readiness with
events rather than sleeps. Exercise the full Trio backend with a fake
dead process to prove pre-publication event registration and exact
failed-startup cleanup.
## Scope boundary
This prerequisite prevents pre-connection namespace-entry failures
from hanging their parent. Namespace FD transfer itself remains the
next commit.

View File

@ -0,0 +1,53 @@
---
model: gpt-5.6-sol
service: opencode
session: trio-bindspace-fd-transport
timestamp: 2026-08-28T17:29:43Z
git_ref: 2ca8c570
scope: code
substantive: true
raw_file: 20260828T172943Z_2ca8c570_prompt_io.raw.md
---
## Prompt
After committing generic pre-handshake child-death detection, the human
approved the proposed next Layer C slice: transport a live bindspace
capability through the Trio spawn backend.
## Response summary
Plumb an optional process-local `Bindspace` through actor spawn APIs.
Give a Trio exec child its own inherited namespace descriptor, preserve
existing caller `pass_fds`, transport the FD/inode pair through the CLI,
and close the temporary parent descriptor on every open-process outcome.
Reject unsupported multiprocessing transport explicitly. Exercise the
complete Trio path with a real actor that enters a distinct network
namespace and reports its namespace and inherited-FD inodes over RPC.
## Files changed
- `tractor/runtime/_supervise.py` - accept and relay a child bindspace.
- `tractor/to_actor/_api.py` - expose bindspace spawn configuration.
- `tractor/spawn/_spawn.py` - relay the bindspace to spawn backends.
- `tractor/spawn/_trio.py` - duplicate, transfer, and close the FD.
- `tractor/spawn/_mp.py` - reject unsupported MP transport.
- `tractor/_child.py` - parse and forward the atomic CLI capability.
- `tests/test_netns_spawn.py` - cover transport and cleanup schedules.
## Human edits
The human approved the Trio-first transport boundary after reviewing
the proposed dependency order, then supplied staged-diff comments across
multiple review passes. Human-directed revisions replaced abstract
ownership wording with concrete FD and process terminology, clarified
descriptor cleanup, preserved existing `pass_fds` explicitly, and
replaced the mocked success-path test with a real child actor, real
`setns()`, handshake, and RPC round trip inside unprivileged user/network
namespaces. Follow-up review added direct parent FD-table verification
that the temporary child-bound duplicate is closed after successful
spawn.
Multiprocessing FD reduction and root-process namespace entry remain
separate follow-up slices. No direct manual source edits were observed;
the review-driven source revisions were applied by the agent.

View File

@ -0,0 +1,34 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-28T17:29:43Z
git_ref: 2ca8c570
diff_cmd: git diff HEAD~1..HEAD
---
# Raw output - transport bindspace FD through Trio spawn
The human approved the next WireGuard Layer C increment after landing
pre-handshake child-death detection.
## Generated code
> `git diff HEAD~1..HEAD -- tractor/_child.py tractor/runtime/_supervise.py tractor/spawn/_spawn.py tractor/spawn/_trio.py tractor/spawn/_mp.py tractor/to_actor/_api.py tests/test_netns_spawn.py`
Thread an optional live `Bindspace` through actor spawn APIs. For the
Trio backend, duplicate its namespace FD, append that temporary
descriptor to existing `pass_fds`, and send the atomic descriptor/inode
pair through the child CLI. Close the parent temporary duplicate as
soon as `open_process()` returns, raises, or is cancelled, while the
child consumes its inherited descriptor before runtime startup.
Reject bindspace transport explicitly in multiprocessing backends until
their descriptor-reduction slice is implemented. Add privilege-free
tests for CLI forwarding, merged pass-FD state, caller immutability,
source/duplicate ownership, exec failure, cancellation, absent live
authority, death-before-handshake cleanup, and MP rejection.
## Scope boundary
Multiprocessing FD reduction and root-process namespace entry remain
follow-up work.

View File

@ -0,0 +1,409 @@
# `tractor.ipc` next-gen transport backends: the shared contract
Status: design doc / implementation spec.
Audience: any model or human implementing one of the three
sibling plans in this directory.
- [`01_tipc_backend.md`](./01_tipc_backend.md) — `AF_TIPC`
(gh #378)
- [`02_quic_iroh_backend.md`](./02_quic_iroh_backend.md) — QUIC
via `iroh` FFI, uniffi-async rewritten onto `trio` (gh #353)
- [`03_wg_tunnel_bindspace.md`](./03_wg_tunnel_bindspace.md) —
WireGuard (and other shuttle-able) tunnels as a *nested
bindspace* layer via `pyroute2` (gh #482, #443)
This doc is the **normative** description of what a `tractor`
transport backend *is* as of `main@83b34884`. Each sibling plan
assumes it and only documents its own deltas. Read this first;
do not re-derive it from the code.
---
## 0. Why a shared contract doc
The three plans are meant to be implementable *independently and
concurrently* by different models/providers without design
drift. Everything they share — the backend duck-type, the
registration tables, the test harness plumbing, the naming and
code-style rules — lives here exactly once. If an implementer
finds this doc disagrees with `main`, **the code wins**; fix this
doc in the same PR.
---
## 1. The backend duck-type (empirical, from `_tcp.py`/`_uds.py`)
A transport backend is **one module** under `tractor/ipc/`
exposing exactly four things. There is no ABC to subclass and no
plugin entrypoint; wiring is by explicit table registration
(§2) plus one piece of reflection (§1.3).
### 1.1 `class <Proto>Address(msgspec.Struct, frozen=True)`
Structurally conforms to the `Address` `Protocol` in
`tractor/discovery/_addr.py:82`. Required surface:
| member | kind | notes |
| --- | --- | --- |
| `proto_key` | `ClassVar[str]` | the wire/registry key, e.g. `'tcp'`, `'uds'` |
| `unwrapped_type` | `ClassVar[type]` | the primitive tuple shape |
| `def_bindspace` | `ClassVar` | default bindspace value |
| `is_valid` | `@property -> bool` | "is this a *dialable/bindable* addr" |
| `bindspace` | `@property` | the "set of hosts"-ish scope (see below) |
| `from_addr(cls, addr)` | `@classmethod` | primitive -> wrapped, `match`-based |
| `unwrap(self)` | method | wrapped -> primitive (must be msgpack-native!) |
| `get_random(cls, bindspace=...)` | `@classmethod` | per-subactor ephemeral addr |
| `get_root(cls)` | `@classmethod` | host-singleton default registrar addr |
| `__repr__` | method | `f'{type(self).__name__}[{...}]'` house style |
Hard constraints learned from the existing two:
- **`frozen=True`.** Addresses are dict keys
(`Server.epsdict()`, `Endpoint.peer_tpts`) and are compared by
value all over the runtime.
- **`.unwrap()` output must round-trip through `msgspec` and
through `wrap_address()`.** It is what actually crosses the
wire in `SpawnSpec`/`_root_mailbox`/`_registry_addrs`, and it
is what `Actor.reg_addrs` and every test compares against. If
your unwrapped form is not *uniquely* pattern-matchable
against the other backends' forms in
`wrap_address()` (`_addr.py:230`), you have a bug that
manifests as the wrong transport being loaded — the file's own
`XXX NOTE` warns about precisely this.
⚠️ **and shape-matching does not survive 4 backends.** Adding
TIPC and iroh breaks it outright: TIPC's natural form is a
`(str, int)` — indistinguishable from `TCPAddress` — and
iroh's is a `(str, str)`, which the *existing* UDS case
(`case (_, filename) if type(filename) is str`) already
swallows. Ordering hacks and prefix-tagging (an earlier
revision of plan 01 proposed `('tipc:<stype>:<scope>', inst)`)
paper over it at best.
**The fix, and the recommended prerequisite for all three
backends: make the unwrapped form carry an explicit
proto-key, using the `multiaddr` protocol name as the
canonical spelling** — `('tcp', host, port)`,
`('unix', path)`, `('udp', ...)`, `('tipc', stype, inst,
scope)`. Then `wrap_address()` collapses from an
order-sensitive `match` to `_address_types[addr[0]]`, and the
whole collision class stops existing. Note this *also* aligns
the on-wire form with `mk_maddr()`/`parse_maddr()`, so the two
representations stop being independent inventions.
Two consequences to plan for:
- it's a **wire-format change** (`SpawnSpec`,
`_root_mailbox`, `_registry_addrs`) plus every test fixture
and downstream config (`piker`'s `[network]` table). It
wants its **own migration commit, landed before any new
backend**, not smuggled into one.
- it's the moment to **stop handing raw unwrapped tuples to
users at all.** The long-term shape is: `Address` subtypes
are the public currency and `UnwrappedAddress` becomes an
internal serialization detail — the same discipline
`ipaddress` uses (you pass `IPv4Address`, not a 4-tuple).
Public API should accept `Address|maddr-str` and treat bare
tuples as legacy-tolerated input, ideally deprecated.
- **`.get_random()` must be collision-free without a live
runtime.** See the `UDSAddress.get_random()` uuid-token
comment (`_uds.py:207-220`): with no `current_actor()` the
sockname degenerates to a pure fn of `(prefix, pid)` and two
calls in one proc alias. Mix in a `uuid4().hex[:8]` token.
- **`.bindspace` semantics**: "the address' bindable space" —
ip/host for `tcp`, the socket-file *directory* for `uds`. For
the new backends: the TIPC *scope* (§1 of plan 01), the iroh
*ALPN + relay/discovery realm* (plan 02). Do not overload this
transport-level bind selector with process namespace lifecycle.
Plan 03 augments an maddr/address declaration with a serializable
`BindspaceSpec` request and host-local `BindspaceRef`. A scoped,
non-serializable `Bindspace` carries that ref and owns the FD/lifetime
used during spawn bootstrap before a concrete address reaches
transport bind.
`Address.namespace` is already spec'd in the Protocol as
"the if-available OS-specific network namespace key" and is
currently unimplemented by both backends — plan 03 is the
first real consumer.
### 1.2 module-level listener lifecycle
```python
async def start_listener(
addr: <Proto>Address,
**kwargs,
) -> trio.SocketListener # or a trio.abc.Listener, see §3
...
def close_listener( # OPTIONAL
addr: <Proto>Address,
lstnr: trio.abc.Listener,
) -> None:
...
```
`close_listener()` is optional; `Endpoint.close_listener()`
(`_server.py:674`) `getattr`s it and treats absence as "closing
is implicit". `uds` needs it (unlinks the sock-file), `tcp`
does not.
### 1.3 the ONE piece of reflection you must not break
`Endpoint.start_listener()` (`_server.py:656`):
```python
tpt_mod: ModuleType = inspect.getmodule(self.addr)
lstnr = await tpt_mod.start_listener(addr=self.addr)
```
The transport module is found by `inspect.getmodule()` **on the
`Address` instance**. Therefore: *the `Address` class and its
`start_listener()`/`close_listener()` MUST live in the same
module.* Do not define the address type in `_types.py` or a
`_addrs.py` and the listener elsewhere.
Immediately after, the same method does:
```python
if (unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap():
self.addr = self.addr.from_addr(unwrapped)
```
i.e. it assumes `lstnr.socket.getsockname()` exists and that its
return value is a valid `from_addr()` input. This is fine for
TIPC (§3 of plan 01) and **is the main integration hazard for
iroh** (§3 of plan 02) — plans that break it must say so
explicitly and propose the upstream `_server.py` patch.
### 1.4 `class Msgpack<Proto>Stream(MsgpackTransport)`
Subclass `tractor.ipc._transport.MsgpackTransport`. You inherit
all framing (`<I` 4-byte little-endian length prefix),
`msgspec` codec ctx-var lookup, `TransportClosed` normalization,
`.drain()`, `__aiter__`. You implement only:
| member | notes |
| --- | --- |
| `address_type` | the `<Proto>Address` class |
| `layer_key: int` | OSI-ish layer, `4` for both current backends |
| `maddr` `@property` | `-> Multiaddr\|str`, via `mk_maddr(self.raddr)` |
| `connected(self) -> bool` | `tcp`/`uds` both use `self.stream.socket.fileno() != -1` |
| `connect_to(cls, addr, prefix_size=4, codec=None, **kw)` | `@classmethod`, returns an instance |
| `get_stream_addrs(cls, stream) -> (laddr, raddr)` | `@classmethod`, called from `MsgpackTransport.__init__` |
`MsgpackTransport.__init__` requires the object passed as
`stream` to satisfy:
- `await stream.send_all(bytes)`
- usable as `tricycle.BufferedReceiveStream(transport_stream=stream)`,
i.e. `await stream.receive_some(n)`
- `trio.BrokenResourceError` / `trio.ClosedResourceError` /
`ValueError('...unclean EOF...')` on the failure paths that
`_iter_packets()` and `send()` already `match` on
(`_transport.py:221-304`, `:436-499`).
That is **`trio.abc.Stream`, not `trio.SocketStream`**. The
`MsgTransport` Protocol's `stream: trio.SocketStream`
annotation (`_transport.py:83`) is a lie of convenience — the
actual `MsgpackTransport.__init__` param is typed
`trio.abc.Stream` and nothing in the msg path touches
`.socket`. Only `connected()` (which each backend defines) and
`Endpoint.start_listener()`'s `getsockname()` do.
### 1.5 verified-good news for socket-family backends
Both `trio.SocketStream` and `trio.SocketListener` are
**address-family agnostic**. Verified against the installed
`trio` (`trio/_highlevel_socket.py`): the only constructor
checks are
- `isinstance(socket, trio.socket.SocketType)`
- `socket.type == SOCK_STREAM`
- (listener) `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)` is truthy,
with `OSError` **suppressed** (the macOS carve-out, which
also covers exotic families that reject the opt)
There is no `AF_*` check and no `IPPROTO_TCP` hard dependency
(`TCP_NODELAY`/`TCP_NOTSENT_LOWAT` are set under
`suppress(OSError)`). Consequence: **any `SOCK_STREAM` family
CPython can create — including `AF_TIPC` — drops straight into
the existing `trio.SocketStream` + `trio.serve_listeners()`
path.** This is why plan 01 is small and plan 02 is not.
---
## 2. Registration tables (the full wiring checklist)
Adding a backend touches these and only these:
1. `tractor/runtime/_state.py:46`
`TransportProtocolKey = Literal['tcp', 'uds', ...]` — add the
key. This `Literal` is the canonical set; `_testing/pytest.py`
drives `--tpt-proto` validation off `_addr._address_types`,
and the spawn-backend fixture already models the
"drive-the-set-from-the-Literal" pattern
(`pytest.py:870-880`) — do the same rather than hardcoding.
2. `tractor/discovery/_addr.py:173` `_address_types: bidict`
`{'<key>': <Proto>Address}`. Note it is a **`bidict`**, so
the mapping must stay 1:1.
3. `tractor/discovery/_addr.py:181` `_default_lo_addrs`
`'<key>': <Proto>Address.get_root().unwrap()`.
⚠️ this dict is built at **import time**, so
`get_root()` must not require a live runtime, a loaded kernel
module, or network I/O. (`UDSAddress.def_bindspace =
get_rt_dir()` is the precedent for "cheap, pure, filesystem-
ish".) A backend whose root addr needs I/O must make this
entry lazy — propose that refactor explicitly.
4. `tractor/discovery/_addr.py:230` `wrap_address()` `match`
add a case iff your `unwrapped_type` isn't already uniquely
matched. **Preferably do the proto-key migration in §1.1
first**, after which this step becomes a one-line
`_address_types` entry instead of an order-sensitive `case`.
5. `tractor/ipc/_types.py``Address` union alias,
`_msg_transports` list, `_key_to_transport[('msgpack', key)]`,
`_addr_to_transport[<Proto>Address]`.
6. `tractor/ipc/_types.py:92` `transport_from_stream()` — the
`sock.family` `match`. For a non-socket stream type (iroh)
this needs a different discriminator; see plan 02 §3.3.
7. `tractor/discovery/_multiaddr.py`
`_tpt_proto_to_maddr`, and a `case` in both `mk_maddr()` and
`parse_maddr()`.
8. `tractor/ipc/__init__.py` — re-export if the backend has a
public surface.
9. `tractor/_testing/addr.py::get_rando_addr()` — per-proto
branch so the whole suite can run under `--tpt-proto <key>`.
10. `pyproject.toml` — new deps go in an **optional extra**, never
in `[project].dependencies`. See §5.
## 3. Where the `trio.SocketListener` assumption is load-bearing
`_serve_ipc_eps()` (`_server.py:1041`) annotates
`listener: trio.abc.Listener` and hands the list to
`trio.serve_listeners(handler=handle_stream_from_peer,
listeners=..., handler_nursery=stream_handler_tn)`.
`trio.serve_listeners` itself is generic over
`trio.abc.Listener`. So the *only* `SocketListener`-specific
code in the server path is the `getsockname()` reconciliation in
`Endpoint.start_listener()` (§1.3) and the type annotations.
`handle_stream_from_peer()` (`_server.py:298`) then does
`Channel.from_stream(stream)`
`transport_from_stream(stream)``sock.family` match (§2.6).
**Therefore**: a non-socket backend needs (a) a
`trio.abc.Listener` subclass, (b) a change to
`Endpoint.start_listener()` to not blindly `getsockname()`, and
(c) a change to `transport_from_stream()`'s discrimination.
All three are small, upstream-able, and *should be landed as
their own prep PR* before the backend itself — see plan 02 §3.
## 4. Handshake / discovery invariants you inherit
- Every accepted stream immediately does
`chan._do_handshake(aid=actor.aid)`; a peer that fails it is
logged at `runtime` and dropped, **not** raised
(`_server.py:334-365`). Discovery-sys "pings" rely on this,
so your `connect_to()` must raise something that normalizes
to `TransportClosed`/`ConnectionError` on a dead peer, never
a novel exception type.
- `_root.py:381-406` fail-fasts when a `registry_addrs` entry's
`proto_key` is not in `enable_transports`. Your key must be
spellable in both.
- `_root.py:256` currently enforces `len(enable_transports) == 1`.
Multi-tpt actors are a separate work item; none of these three
plans may depend on lifting it.
- Sub-actor bind addrs come from
`_runtime.py:1600-1610`: for each key in the parent-supplied
`enable_transports`, `get_address_cls(key).get_random()`.
So `get_random()` runs *in the child, post-fork, pre-listen*.
Anything it needs (kernel module, netns membership, an iroh
secret key) must already be true at that moment.
## 5. Dependency policy
`[project].dependencies` stays lean (see the boot-latency work,
gh #470: `import tractor` is budgeted at ~0.145s). Every new
backend dep is an extra:
```toml
[project.optional-dependencies]
tipc = [] # stdlib-only!
quic = ["iroh>=0.35"] # pin per plan 02 §1
wg = ["pyroute2>=0.9"] # pin per plan 03 §1
```
and every backend module must be **import-lazy**: a
`tractor/ipc/_<proto>.py` that imports its 3rd-party dep at
module scope must not be imported by `tractor/__init__.py`,
`tractor/ipc/__init__.py`, or `tractor/discovery/_addr.py`'s
import-time table construction. The `_addr._default_lo_addrs`
eager-dict (§2.3) is the trap: keep the backend's `get_root()`
dep-free, or make that table lazy.
## 6. Test-harness plumbing (identical for all three)
- `--tpt-proto <key>` (`_testing/pytest.py:409`) selects the
session-wide proto; the `tpt_proto` fixture mutates
`_state._def_tpt_proto` + `_runtime_vars['_enable_tpts']`
(`pytest.py:807-835`). Adding the key to `_address_types` is
what makes `--tpt-proto <key>` legal (`pytest.py:795-800`
asserts the lookup).
- The **acceptance bar** for every backend is: the *entire*
existing suite passes under `--tpt-proto <key>`, unmodified.
That is the whole point of the abstraction. Backend-specific
unit tests go in `tests/ipc/test_each_tpt.py` (the existing
`test_uds_bindspace_created_implicitly` /
`test_uds_double_listen_raises_connerr` are the model).
- Capability gating: each backend needs a **cheap, pure
predicate** + a `pytest.mark.skipif`, because these are all
environment-dependent. Verified example: on this dev box
`socket.socket(AF_TIPC, SOCK_STREAM)` raises
`OSError(97, 'Address family not supported by protocol')`
because the `tipc` module isn't loaded. Put the predicate in
the backend module (so apps can use it too), not in the test.
- New pytest marks must be registered in `pyproject.toml`, per
the project's fix-warnings-at-source rule (gh #469).
## 7. Code style (non-negotiable, matches the repo)
- module header tagline: `# tractor: distributed structured
concurrency.` for **new** files (not the legacy
`structured concurrent "actors".` form the existing `_tcp.py`
carries).
- AGPL header block copied verbatim from `_tcp.py`.
- `from __future__ import annotations` first.
- annotate *everything*, including locals:
`sockpath: Path = addr.sockpath`.
- `match`/`case` over `isinstance` chains for address and
error dispatch.
- multi-line call/`import` style with trailing commas.
- never emit a whitespace-only line.
- error messages are multi-line f-strings ending in `\n`, with
the `f'...\n' f'...\n'` implicit-concat layout and the
`>[`/`[>`/`<=(` nested-op sigils where a `nest_from_op()` is
in play.
- prefer pure functions + module-level helpers over methods;
keep `Address` types data-only. Where a helper needs
scoped setup/teardown, it's an `@acm` — not a class with
`.start()`/`.stop()`.
- pure getters: no `get_*(..., mutate=True)` flags; split into
a read-only getter and an explicit sibling setter.
---
## 8. Cross-plan sequencing
The three are independent *except*:
- plan 02 (iroh) needs the `_server.py` /
`transport_from_stream()` generalization (§3) — plan 01 does
**not**, and should therefore land first as the cheap proof
that the table-registration story works for a genuinely new
proto.
- plan 03 (wg) composes *under* whatever L4 tpt is in use and
its netns work is what finally implements
`Address.namespace`. It can land before or after 02, but its
`TunnelledAddress` design must be reviewed against plan 02's
address shape so the "tunnelled maddr" grammar (gh #443)
covers `/…/quic-v1/p2p/…` inner addrs too.
- All three want first-class `wg`/`quic`/`tipc` protos in
`py-multiaddr`; that upstream track is gh #483 and
multiformats/py-multiaddr#107/#108.

View File

@ -0,0 +1,707 @@
# Plan 01 — `TIPC` transport backend (`tractor/ipc/_tipc.py`)
Tracks gh [#378]. Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: TIPC is the *cheapest* new backend we can add and
simultaneously the only one that gives us cluster-wide service
discovery **for free, in the kernel**, replacing (for
TIPC-capable deployments) the whole `tractor.discovery`
registrar round-trip with a `bind()`/`connect()` on a
*service name*. It is stdlib-only: zero new dependencies.
[#378]: https://github.com/goodboy/tractor/issues/378
---
## 1. Why this is small: three verified facts
1. **CPython already speaks TIPC.** `socket.AF_TIPC` plus 23
`TIPC_*` constants are present in the stdlib on Linux
(verified on the dev box, py3.13):
`AF_TIPC, SOL_TIPC, TIPC_ADDR_ID, TIPC_ADDR_NAME,
TIPC_ADDR_NAMESEQ, TIPC_CFG_SRV, TIPC_CLUSTER_SCOPE,
TIPC_CONN_TIMEOUT, TIPC_{CRITICAL,HIGH,MEDIUM,LOW}_IMPORTANCE,
TIPC_DEST_DROPPABLE, TIPC_IMPORTANCE, TIPC_NODE_SCOPE,
TIPC_PUBLISHED, TIPC_SRC_DROPPABLE, TIPC_SUBSCR_TIMEOUT,
TIPC_SUB_CANCEL, TIPC_SUB_PORTS, TIPC_SUB_SERVICE,
TIPC_TOP_SRV, TIPC_WAIT_FOREVER, TIPC_WITHDRAWN,
TIPC_ZONE_SCOPE`.
`sock.bind()/connect()/getsockname()` take/return the
5-tuple `(addr_type, v1, v2, v3, scope)` — the last element
is optional on input and defaults to `0`.
2. **`trio` doesn't care about the address family.** Per
contract §1.5, `trio.SocketStream` and `trio.SocketListener`
only require a trio socket object of type `SOCK_STREAM`.
TIPC's `SOCK_STREAM` is a real connection-oriented reliable
byte stream. So we reuse `trio.SocketStream`,
`trio.SocketListener`, `trio.serve_listeners()`,
`MsgpackTransport`'s framing — *all of it*.
3. **It is not available by default.** On this box
`socket.socket(AF_TIPC, SOCK_STREAM)`
`OSError(97, 'Address family not supported by protocol')`
with no `tipc` in `/proc/modules`. `modprobe tipc` is
required; cross-node needs a bearer
(`tipc bearer enable media eth device <if>` or
`media udp name <n> localip <ip>`). Everything about this
plan's testability hinges on gating (§7).
Non-goals: `SOCK_RDM`/`SOCK_DGRAM`/`SOCK_SEQPACKET` message
modes, multicast fan-out, and TIPC group messaging. They are
genuinely interesting for a future `tractor` broadcast/pubsub
transport but they do **not** fit `MsgTransport`'s
stream-of-length-prefixed-msgs shape. Note them in the
follow-up issue, do not build them here.
---
## 2. `TIPCAddress`
### 2.1 the three TIPC address flavours, and which we use
| flavour | tuple | meaning |
| --- | --- | --- |
| `TIPC_ADDR_NAMESEQ` | `(type, lower, upper, scope)` | a *published range* — what a server `bind()`s |
| `TIPC_ADDR_NAME` | `(type, instance, domain, scope)` | a *lookup* — what a client `connect()`s |
| `TIPC_ADDR_ID` | `(node, ref, 0, scope)` | a concrete port id — the "physical" address |
The design decision that makes this backend coherent:
> **A `tractor` actor's TIPC address is a *service name*
> `(type, instance)`; `bind()` publishes the singleton range
> `(type, instance, instance)`; peers `connect()` by name and
> the kernel resolves + load-balances. `TIPC_ADDR_ID` is only
> ever an *observed* address (`getpeername()`), never a
> user-facing one.**
This is exactly the "leverage the built-in discovery machinery"
ask in #378: publishing a bind *is* registration, and
`connect()` on a name *is* a lookup, with no registrar actor in
the loop.
### 2.2 the struct
```python
class TIPCAddress(
msgspec.Struct,
frozen=True,
):
_stype: int # TIPC "type" == service class
_instance: int # service instance within the type
_scope: int = TIPC_CLUSTER_SCOPE
# observed-only, never part of identity/equality-by-intent
maybe_node: int|None = None # from TIPC_ADDR_ID getpeername()
maybe_ref: int|None = None
proto_key: ClassVar[str] = 'tipc'
unwrapped_type: ClassVar[type] = tuple[str, int]
def_bindspace: ClassVar[int] = TIPC_CLUSTER_SCOPE
```
**Unwrapped form** (the wire/`SpawnSpec` shape).
TIPC's natural form is `(stype, instance, scope)` — but a
2-tuple squeeze of it is a `(str, int)`, i.e. *the same coarse
shape as `TCPAddress`*, so `wrap_address()`'s
`case (str(), int())` steals it. This backend is therefore the
forcing function for the contract-doc's conclusion (§1.1):
> **make the unwrapped form carry an explicit proto-key, spelled
> with the `multiaddr` protocol name.**
```python
def unwrap(self) -> tuple[str, int, int, int]:
return ('tipc', self._stype, self._instance, self._scope)
```
`wrap_address()` then dispatches `_address_types[addr[0]]` and
the collision class disappears. **This is a prerequisite
migration commit, not part of this backend** — see contract §1.1
for its blast radius (wire format + every fixture + `piker`
config) and for the follow-on "stop handing raw tuples to users
at all, à la `ipaddress`" direction.
⚠️ an earlier revision of this plan proposed a self-tagging
`('tipc:<stype>:<scope>', instance)` string-prefix hack with an
ordered `case` guard. **Dropped** — it papers over the problem,
keeps `wrap_address()` order-sensitive, and doesn't help iroh's
`(str, str)`-vs-UDS collision at all. Do not resurrect it.
Note `TIPCAddress` is the first backend where `.unwrap()` is
**not** a lossless view of the live socket — `maybe_node`/
`maybe_ref` are observed metadata, exactly like
`UDSAddress.maybe_pid` (which is likewise excluded from
`.unwrap()`). Follow that precedent, including its `__repr__`
treatment (`_uds.py:242`).
### 2.3 how to pick `_stype` and `_instance`
- `_stype` = a `tractor`-reserved service class. TIPC reserves
0..63 for internal use (`TIPC_TOP_SRV == 1`,
`TIPC_CFG_SRV == 0`). Use a module constant
`TRACTOR_STYPE: int = 0x74_72_00_00` ("tr\0\0") as the default
and make it overridable via `TIPCAddress._stype` so an app
can partition service classes. Document that two `tractor`
trees sharing a cluster **and** a `_stype` share a namespace.
- `_instance` for `get_root()`: `1616` — mirrors the
`TCPAddress.get_root()` port and the `registry@1616.sock`
UDS filename, so the "1616 is tractor's registrar" idiom
holds across all backends.
- `_instance` for `get_random()`: TIPC gives us no
kernel-assigned-instance analogue of `port=0`, so we must
choose. Use a *pure* fn of the actor identity so it is
reproducible and collision-free:
```python
# 32-bit instance derived from the actor's uuid4 (+ pid when
# there's no live runtime, per the UDS precedent).
inst: int = int.from_bytes(
blake2b(seed.encode(), digest_size=4).digest(),
'big',
)
```
where `seed = f'{actor.aid.name}@{pid}'` if
`current_actor(err_on_no_runtime=False)` else
`f'{prefix}.{uuid4().hex[:8]}@{pid}'`. Must avoid the reserved
low range: `inst = 64 + (inst % (2**32 - 64))`.
⚠️ *unlike* `port=0`, a collision here surfaces as a
successful-but-shared publication (TIPC allows multiple
binders on the same name and round-robins!) rather than
`EADDRINUSE`. That is a silent-crosstalk failure mode; §7 has
the test that proves the 4-byte digest is enough and §9 has
the mitigation if it isn't.
- `_scope`: `TIPC_NODE_SCOPE` for a same-host-only actor (the
UDS-equivalent), `TIPC_CLUSTER_SCOPE` (default) for
cluster-visible. **This is `.bindspace`**:
```python
@property
def bindspace(self) -> int:
return self._scope
```
It is the honest analogue of "the set of hosts this bind is
reachable from", which is precisely the docstring in
`Address.bindspace`. (`TIPC_ZONE_SCOPE` is deprecated/aliased
to cluster in modern kernels — accept it on input, normalize
to cluster, log at `transport` level.)
### 2.4 `is_valid`
```python
@property
def is_valid(self) -> bool:
return (
self._instance != 0
and
self._stype not in _tipc_reserved_stypes # {0, 1, ...}
and
self._scope in (TIPC_NODE_SCOPE, TIPC_CLUSTER_SCOPE)
)
```
---
## 3. Listener + stream
### 3.1 `start_listener()`
```python
async def start_listener(
addr: TIPCAddress,
backlog: int = 128,
**kwargs,
) -> SocketListener:
sock = trio.socket.socket(
socket.AF_TIPC,
socket.SOCK_STREAM,
)
# publish the singleton name-range == "register the service"
await sock.bind((
socket.TIPC_ADDR_NAMESEQ,
addr._stype,
addr._instance,
addr._instance,
addr._scope,
))
sock.listen(backlog)
return SocketListener(sock)
```
Notes / hazards:
- `bind()` on `AF_TIPC` is **not** a filesystem or port-table
operation and can't block on DNS, but keep it `await`ed
through `trio.socket` anyway for uniformity.
- `backlog=128` matching `_uds.start_listener()`'s hard-won
value (see its comment at `_uds.py:317-331` re: concurrent
deregistration storms). Do not use `1`.
- **no `close_listener()` needed** — nothing to unlink. Omit the
function entirely (contract §1.2: absence means implicit).
Withdrawal of the published name happens on socket close.
- ⚠️ `SocketListener.__init__` will try
`getsockopt(SOL_SOCKET, SO_ACCEPTCONN)`. If TIPC rejects it,
trio's `except OSError: pass` covers us. Assert this in a
unit test rather than assuming.
- Wrap the bind in a `_reraise_as_connerr()`-style `@cm` (copy
the `_uds.py:256` pattern) so `EADDRINUSE`-ish and
`EAFNOSUPPORT` become `ConnectionError` with the addr in the
message. `EAFNOSUPPORT` here means "kernel module not
loaded" and deserves a *specifically actionable* message:
`'TIPC unavailable — try `sudo modprobe tipc`\n'`.
### 3.2 the `getsockname()` reconciliation
`Endpoint.start_listener()` does
`if lstnr.socket.getsockname() != self.addr.unwrap(): self.addr =
self.addr.from_addr(unwrapped)`.
For TIPC, `getsockname()` on a bound-but-listening socket
returns a `TIPC_ADDR_ID`-flavoured 5-tuple (the port id), *not*
the name-seq we bound. So the `!=` is **always true** and
`from_addr()` will be handed a 5-tuple.
Handle it inside `TIPCAddress.from_addr()` — do **not** patch
`_server.py`:
```python
@classmethod
def from_addr(cls, addr) -> TIPCAddress:
match addr:
# our own unwrapped form
case (str() as tag, int() as inst) if tag.startswith('tipc:'):
_, stype, scope = tag.split(':')
return TIPCAddress(int(stype), inst, int(scope))
# a kernel-observed TIPC_ADDR_ID 5-tuple: keep the
# *service* identity we already know and only annotate
# the observed port-id.
case (int() as atype, *rest) if atype == socket.TIPC_ADDR_ID:
...
```
The `TIPC_ADDR_ID` case cannot reconstruct `(stype, instance)`
— that info isn't in a port id. So `from_addr()` alone is
insufficient for the reconciliation path. **Resolution**: make
`from_addr()` raise a clear `ValueError` for the bare
`TIPC_ADDR_ID` case, and instead prevent the reconciliation
from firing by having `start_listener()` return a listener
whose `getsockname()` we never need — i.e. land this two-line
upstream fix in `_server.py:664`:
```python
if (
(unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap()
and
self.addr.rebind_from_sockname # ClassVar[bool] = True on tcp/uds
):
```
with `TIPCAddress.rebind_from_sockname: ClassVar[bool] = False`
(and `True` on `TCPAddress`/`UDSAddress`, preserving today's
behaviour exactly). Rationale: the reconciliation exists *only*
to learn the kernel-assigned port for `port=0` TCP binds (its
own comment says so, `_server.py:662`); TIPC has no such
late-binding, so opting out is semantically right rather than a
hack. **Land this as its own commit, ahead of the backend**,
with a test that `tcp`'s `port=0` behaviour is unchanged.
Keep the observed port-id available anyway: annotate
`ep.addr = ep.addr.with_port_id(*getsockname()[1:3])` (a pure
`msgspec.structs.replace()` helper) purely for logging/repr.
### 3.3 `MsgpackTIPCStream`
```python
class MsgpackTIPCStream(MsgpackTransport):
address_type = TIPCAddress
layer_key: int = 4
@property
def maddr(self) -> Multiaddr|str:
return mk_maddr(self.raddr)
def connected(self) -> bool:
return self.stream.socket.fileno() != -1
@classmethod
async def connect_to(
cls,
destaddr: TIPCAddress,
prefix_size: int = 4,
codec: MsgCodec|None = None,
**kwargs,
) -> MsgpackTIPCStream:
sock = trio.socket.socket(AF_TIPC, SOCK_STREAM)
with close_on_error(sock):
# NOTE: connect by *name* -> kernel does the lookup,
# so this is our "discovery" call.
await sock.connect((
socket.TIPC_ADDR_NAME,
destaddr._stype,
destaddr._instance,
0, # domain: 0 == "anywhere in scope"
destaddr._scope,
))
return cls(
trio.SocketStream(sock),
prefix_size=prefix_size,
codec=codec,
)
```
- reuse `trio._highlevel_open_unix_stream.close_on_error` (the
UDS backend already imports it) or inline the equivalent
`try/except: sock.close(); raise`.
- `SO_/TIPC_` opts worth setting and documenting:
- `setsockopt(SOL_TIPC, TIPC_IMPORTANCE, TIPC_HIGH_IMPORTANCE)`
for the *parent<->child* lifetime channel — this is a real
win TIPC gives us that TCP can't: the runtime's
supervision channel can outrank bulk app traffic under
congestion. Wire it as a `connect_to(..., importance=...)`
kwarg defaulted from a module constant, and have
`_runtime.py`'s parent-chan path pass the high value **in a
follow-up** (don't couple it to this PR).
- `TIPC_CONN_TIMEOUT` — the kernel-side connect timeout;
leave at default, we have `trio` cancel scopes.
- `TIPC_DEST_DROPPABLE = 0` on the connection so undeliverable
msgs come back as errors rather than being silently dropped.
- **`connect_to()` on a name with no publisher**: TIPC returns
`ECONNREFUSED`/`EHOSTUNREACH` promptly (no SYN-timeout wait),
which is *better* discovery-ping behaviour than TCP. Confirm
the errno and make sure it surfaces as `ConnectionError`
(contract §4 — the registrar ping path depends on it).
### 3.4 `get_stream_addrs()`
```python
@classmethod
def get_stream_addrs(cls, stream) -> tuple[TIPCAddress, TIPCAddress]:
sock = stream.socket
# both return TIPC_ADDR_ID 5-tuples for a connected sock
l_id = sock.getsockname()
r_id = sock.getpeername()
...
```
Problem: neither end's port-id tells us the *service name*. The
`laddr`/`raddr` are used for logging, `Channel.raddr`,
`Server._peers` keying-adjacent repr, and `maddr`. Design:
- the **connecting** side knows the destaddr it dialled →
`connect_to()` overrides `_raddr` after construction with the
known-good `TIPCAddress`, exactly as
`MsgpackUDSStream.connect_to()` does for the peer-pid case
(`_uds.py:539-543`).
- the **accepting** side does not know the peer's service name
from the socket. Two honest options:
- **(a) accept it: `raddr` carries only `(node, ref)`** via
`maybe_node`/`maybe_ref`, `_stype/_instance` set to a
sentinel `-1`, and `__repr__` renders
`TIPCAddress[<peer-node:0x...>:<ref>]`. The `Aid` from the
handshake already gives us the peer's logical identity, so
nothing in the runtime actually *needs* the peer's service
name. **Recommended.**
- (b) piggyback the peer's own bound name in the handshake.
Rejected for this PR: touches `Aid`/msg-spec.
- `laddr` on the accepting side: the `Endpoint` knows its own
`addr`; but `get_stream_addrs()` is a `@classmethod` with only
the stream. Use `TIPC_ADDR_ID` for `laddr` too and let
`Endpoint.peer_tpts` keying (which is by *peer* addr) still
work. Verify nothing asserts `laddr == ep.addr` — grep for
`.laddr` uses before committing (`_server.py`'s
`con_status` logging, `Channel.pformat()`).
---
## 4. Multiaddr representation
There is no `/tipc` in the multiaddr protocol table. Interim
grammar, mirroring how `uds` maps to the spec-legal `/unix`:
```
/tipc/<stype>/<instance> # scope implied = cluster
/tipc/<stype>/<instance>/<scope> # explicit
```
- `_tpt_proto_to_maddr['tipc'] = 'tipc'` and a `mk_maddr()`
`case 'tipc':` building the above.
- `parse_maddr()` gets `case ['tipc']:` — but note
`py-multiaddr` will reject an unregistered protocol name
outright, so this **requires an upstream registration** (same
track as the `wg` work, gh #483 /
multiformats/py-multiaddr#107). Until that lands:
- `MsgpackTIPCStream.maddr` returns the **`str`** form (the
`MsgTransport.maddr` return type is already
`Multiaddr|str`, and `MsgpackUDSStream.maddr` already
exercises the `str` branch), and
- `parse_maddr()` special-cases the `/tipc/` prefix *before*
handing the string to `Multiaddr()`.
Document this as the reason gh #443's "standardize on
returning `Multiaddr` everywhere" item stays blocked.
Propose `/tipc/` upstream as: name `tipc`, code TBD, size
variable, value `<stype>:<instance>:<scope>` — or as three
composed protos. Prefer *one* proto with a structured value so
the maddr stays 2-segment like `/unix/...`.
---
## 5. Discovery: the actually-interesting part
Two independently-shippable layers. **Layer A is in scope for
the first PR; layer B is a fast-follow.**
### 5.1 Layer A — "discovery by bind" (free)
Because `bind(TIPC_ADDR_NAMESEQ)` publishes and
`connect(TIPC_ADDR_NAME)` resolves, a `tractor` tree whose
`registry_addrs` are TIPC service names needs **no registrar
liveness at all** for the connect path: `find_actor()`'s
"connect to the registrar and ask" becomes "connect to the
service name directly". Concretely:
- `tractor.discovery._api.find_actor()` etc. keep working
unchanged (they go through the registrar), *and*
- a new, TIPC-only fast path becomes possible: derive an actor's
service name from its `(name, uuid)` and dial it without any
registrar hop.
Do **not** build the fast path in PR 1. Instead, prove the
property with a test (§7.4) and file the follow-up: it changes
`discovery` semantics (name→instance derivation must be a
documented, stable, cross-language-able hash) and deserves its
own design.
### 5.2 Layer B — the topology service (`TIPC_TOP_SRV`)
This is what makes #378's "end game cluster proto" claim real:
a *subscription* to name-table events, i.e. push-based
`register`/`deregister` for free, replacing the registrar's
polled `find_actor()`.
Mechanics (verify each field against
`linux/include/uapi/linux/tipc.h` + `net/tipc/topsrv.c` at
implementation time — the struct layout below is from the uapi
header and the byte-order caveat is real):
```python
# SOCK_SEQPACKET connected to the topology server
sock = trio.socket.socket(AF_TIPC, SOCK_SEQPACKET)
await sock.connect((
socket.TIPC_ADDR_NAME,
socket.TIPC_TOP_SRV, # == 1
socket.TIPC_TOP_SRV,
0,
))
# struct tipc_subscr {
# struct tipc_name_seq seq; /* 3 * __u32: type, lower, upper */
# __u32 timeout; /* TIPC_WAIT_FOREVER == ~0 */
# __u32 filter; /* TIPC_SUB_{PORTS,SERVICE,CANCEL} */
# char usr_handle[8];
# } /* == 28 bytes */
_SUBSCR_FMT: str = '=IIIII8s' # ⚠ 5*I is 20 -> use '=5I8s'
```
- **byte order**: the topology server historically accepts both
host and swapped order and auto-detects; modern kernels are
strict-ish. Pack native (`'='`) first, and if the server
closes the connection immediately, retry with `'>'`. Encode
that as a one-time probe helper
`_detect_topsrv_endianness()` cached at module level — and
put a `# ?TODO` pointing at `net/tipc/topsrv.c` for someone
to make it deterministic.
- **events**: `struct tipc_event` is `event: u32`,
`found_lower: u32`, `found_upper: u32`,
`port: {ref: u32, node: u32}`, then the 28-byte subscription
echo → 40 bytes. `event ∈ {TIPC_PUBLISHED, TIPC_WITHDRAWN,
TIPC_SUBSCR_TIMEOUT}`.
- **trio shape** — this is where the "nearly-functional,
modern-async" style pays off; expose it as an `@acm` yielding
a `trio` receive-channel of typed events, *not* a class:
```python
@acm
async def open_topology_events(
stype: int = TRACTOR_STYPE,
lower: int = 0,
upper: int = 0xFFFFFFFF,
filter: int = TIPC_SUB_SERVICE,
timeout: int = TIPC_WAIT_FOREVER,
buf_size: int = 64,
) -> AsyncGenerator[
trio.MemoryReceiveChannel[TIPCNameEvent],
None,
]:
...
```
with `TIPCNameEvent(msgspec.Struct, frozen=True)` fields
`kind: Literal['published','withdrawn','timeout']`,
`addr: TIPCAddress`, `node: int`, `ref: int`. One
`trio.lowlevel`-free implementation: a nursery-spawned reader
task doing `await sock.recv(40)` in a loop and
`send_nowait()`ing decoded events, with the `@acm` closing the
socket on exit → reader gets `ClosedResourceError` → cancel
scope collapses. Standard `tractor` `@acm` discipline.
- **consumer**: `tractor/discovery/_registry.py` gains an
optional "watch" mode so a registrar (or any actor) can keep
a live view of the actor set without polling. Sketch the
integration in the follow-up issue; do not wire it in PR 1.
- **`SOCK_SEQPACKET` is fine here** because this socket never
goes through `MsgpackTransport` — it's a plain trio socket
used with `recv()`. The contract's "`SOCK_STREAM` only"
constraint applies to `MsgTransport` streams, not to this.
---
## 6. Commit sequencing (each independently reviewable + green)
1. `_server.py`: add `Address.rebind_from_sockname:
ClassVar[bool]`, gate the `getsockname()` reconciliation on
it, `True` for tcp/uds. Test: tcp `port=0` unchanged.
2. `tractor/ipc/_tipc.py`: `TIPCAddress` + `is_tipc_available()`
predicate + `start_listener()`. No transport yet.
Tests: address round-trip (`unwrap`/`from_addr`/`wrap_address`),
`get_random()` uniqueness, bind/listen + `SO_ACCEPTCONN`
tolerance, `EAFNOSUPPORT` → actionable `ConnectionError`.
3. `MsgpackTIPCStream` + `connect_to()` + `get_stream_addrs()`.
Test: two `trio` tasks in one proc exchange a msg over
`Msgpack` framing (no `tractor` runtime).
4. registration tables (contract §2 items 1-6, 9) +
`pyproject.toml` mark/extra. Test: full suite under
`--tpt-proto tipc` (§7.3).
5. maddr support (`str` form + prefix special-case) + docs.
6. `open_topology_events()` @acm + its tests (layer B).
7. docs page + `docs/` example.
Per project convention, a reproducing/guard test lands in its
own commit **before** the fix it guards.
---
## 7. Testing
### 7.1 the capability predicate (in `_tipc.py`, public)
```python
def is_tipc_available() -> bool:
'''
True iff this kernel can create an `AF_TIPC` socket, i.e.
the `tipc` module is loaded.
'''
try:
socket.socket(socket.AF_TIPC, socket.SOCK_STREAM).close()
return True
except OSError:
return False
```
Cache it in a module global (it can't change without a
`modprobe`, and a cold call costs a syscall). Pure predicate, no
side effects, no logging.
### 7.2 gating
- `pytest.mark.tipc` registered in `pyproject.toml`.
- module-level
`pytestmark = pytest.mark.skipif(not is_tipc_available(),
reason='`tipc` kernel module not loaded (`modprobe tipc`)')`
in `tests/ipc/test_tipc.py`.
- `--tpt-proto tipc` with no module must fail **loudly and
early** with the actionable message, not with 400 confusing
timeouts. Add the check to the `tpt_protos` fixture's existing
per-proto validation loop (`_testing/pytest.py:795`): if the
chosen `Address` type exposes an `is_available()`-style
classmethod, call it and `pytest.fail()` with its reason.
Generalize (don't special-case tipc) — plans 02/03 need the
same hook.
### 7.3 CI
- add a job matrix entry `--tpt-proto tipc` that runs
`sudo modprobe tipc` in a `before` step. GH's
`ubuntu-latest` runners do allow `modprobe tipc` (the module
ships with the standard Ubuntu kernel package); verify in a
throwaway workflow before wiring the matrix. If it turns out
to be unavailable, fall back to a container job with
`--privileged`/`--cap-add NET_ADMIN`, and mark the job
`continue-on-error` until it's proven stable.
- cross-node TIPC (bearer) cannot be CI'd; cover it with a
documented manual smoke test in the docs page, in the style
of gh #482's LAN examples.
### 7.4 backend-specific tests worth writing
- **name-publication is discovery**: bind a listener on
`(stype, inst)`, then from a second task `connect()` by name
and assert it lands — *without* any `tractor` registrar.
- **`get_random()` collision resistance**: 10k `get_random()`
calls with no live runtime → 10k distinct `_instance`s.
(This is the silent-crosstalk risk from §2.3; if the 4-byte
digest ever collides in this test, escalate to §9.)
- **round-robin surprise**: two listeners bound to the *same*
`(stype, inst)` both succeed (TIPC allows it) and connects
distribute. Assert the observed behaviour and reference it
from the `get_random()` docstring so the next reader knows
why the hash matters.
- **scope isolation**: a `TIPC_NODE_SCOPE` bind is not visible
to a cluster-scope lookup from another node (manual/marked).
- **importance opt** round-trips via `getsockopt`.
- **graceful + abrupt close** produce `TransportClosed` with the
same `loglevel` classification as tcp/uds — i.e. re-run the
relevant `tests/ipc/test_each_tpt.py` cases parametrized over
the new proto rather than writing new ones.
---
## 8. Deployment / docs deliverable
A `docs/` page (and/or an `examples/` script) covering:
```bash
# single host, node-scope only
sudo modprobe tipc
tipc node get addr
# multi-host over ethernet (pairs beautifully with plan 03's wg)
sudo tipc bearer enable media eth device eth0
# ...or over UDP when L2 isn't available:
sudo tipc bearer enable media udp name uc localip 10.0.11.1
tipc link list
tipc nametable show # <- see tractor's published services!
```
`tipc nametable show` displaying live `tractor` actors is the
single best demo this backend has; lead with it.
---
## 9. Known risks + escalations
| risk | mitigation |
| --- | --- |
| `_instance` hash collision → silent crosstalk (two actors share a service name, TIPC round-robins connects between them) | §7.4 test; if it bites, add a post-bind verification handshake, or bump to a 6-byte digest folded into `(stype_low, instance)` |
| kernel/module unavailability everywhere (dev boxes, macOS, CI) | hard gating (§7.2); TIPC is explicitly an *opt-in cluster* transport, never a default |
| `getsockname()` returns port-id not name | the `rebind_from_sockname` opt-out (§3.2), landed first |
| unregistered `/tipc` multiaddr proto | `str` maddr fallback (§4) + upstream track gh #483 |
| stale docs (#378 notes tipc.io docs may be out of date) | treat `include/uapi/linux/tipc.h` + `net/tipc/` as the only normative source; cite file+symbol in code comments |
| `SOCK_SEQPACKET` topology framing byte-order | probe helper + `?TODO` (§5.2) |
## 10. Follow-up issue seeds
- **register `/tipc` in the multiaddr spec**, mirroring the `wg`
track (multiformats/py-multiaddr#107/#108 + gh #483). Same
shape of work: propose the proto + code, land a codec in
`py-multiaddr`, then drop our `str`-maddr fallback (§4). Worth
filing *alongside* the `wg` spec-submission issue so both
proposals go up together rather than as one-offs.
- registrar-less discovery fast path via name derivation (§5.1)
- `TIPC_TOP_SRV`-driven push registry in
`discovery/_registry.py` (§5.2)
- `TIPC_IMPORTANCE` for the parent<->child lifetime channel
(§3.3) — genuinely novel supervision QoS, no other backend
can do it
- TIPC multicast / group messaging as a *broadcast* transport
for `tractor.trionics` fan-out (explicitly not `MsgTransport`)
- dual-link resiliency / multi-homing (#378's "hybrid dual link")
once bearers are scripted in the docs

View File

@ -0,0 +1,566 @@
# Plan 02 — QUIC backend via `iroh` FFI, uniffi-async rewritten onto `trio`
Tracks gh [#353]. Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: the value of `iroh` over "just QUIC" is
`NodeId`-addressed, NAT-traversing, relay-fallback endpoints —
i.e. a `tractor` actor tree that spans hosts *without* a
reachable listening socket. The cost is that `iroh`'s python
surface is `uniffi`-generated **asyncio** and its listener is not
a socket. This plan spends its complexity budget in exactly two
places: a `trio`-native uniffi future bridge, and a
`trio.abc.Listener`/`Stream` adapter pair. Everything else is
contract boilerplate.
[#353]: https://github.com/goodboy/tractor/issues/353
---
## 1. Library selection (decided, with the rejected alternatives)
**Chosen: `iroh` (PyPI, from `n0-computer/iroh-ffi`), pinned to
a single minor.** The `iroh` python package is a `uniffi`
binding over the rust `iroh` crate (QUIC via `quinn`/`noq`).
Rejected, and why — record these so the next implementer doesn't
relitigate:
- **`aioquic`** (sans-io + asyncio): genuinely trio-portable
(`hypercorn` already pairs its sans-io core with a trio UDP
server, see the links in #353) and dependency-light. But it
gives us *only* QUIC — no NodeId identity, no hole punching,
no relay. We'd be reimplementing iroh's whole reason for
existing. **Keep as the documented fallback** if the FFI
bridge (§2) proves unmaintainable; the `MsgTransport` and
`Listener` adapters from §3 are ~90% reusable against an
`aioquic` core, which is a deliberate design property of this
plan.
- **`quiche` / `quinn` via a hand-rolled PyO3 ext**: strictly
more work than reusing `iroh-ffi`, and puts us in the
build-wheels business.
- **`trio-asyncio`**: viable *shortcut* to run the asyncio-shaped
bindings under trio, and `tractor` already ships
infected-asyncio machinery (`tractor.to_asyncio`,
`tests/test_infected_asyncio.py`). Rejected as the *primary*
design because it makes every IPC send/recv cross a
loop-boundary shim in the hot path, and because #353 asks
explicitly for the asyncio support to be "rewritten for trio".
**But**: build it first as the throwaway spike (§6 step 0) to
de-risk the iroh API surface before writing the bridge.
Version pinning: `iroh` moves fast and has had breaking
API renames across minors. Pin `iroh>=X.Y,<X.Y+1` in a `quic`
extra, and **write down the exact resolved version + the
generated `iroh/_uniffi*` module layout** in the module
docstring, because §2 depends on generated-code internals.
**Step 0 of implementation is an API-truth pass**: install the
pinned `iroh`, `python -c "import iroh; help(iroh)"`, and record
in this doc's §1.1 the real names of: endpoint builder, secret
key type, `connect`/`accept`, bi-stream open/accept, the
send/recv methods and their exact signatures/return types, and
whether they're `async def`. Everything below uses *provisional*
names and must be reconciled. Do not skip this; do not guess
from memory.
### 1.1 API-truth table (fill in during step 0)
| concept | provisional name | actual (fill in) |
| --- | --- | --- |
| secret key | `iroh.SecretKey.generate()` | |
| endpoint builder | `iroh.Endpoint.builder(...).bind()` | |
| node id | `endpoint.node_id() -> str` | |
| node addr (relay + direct) | `iroh.NodeAddr` | |
| dial | `await endpoint.connect(node_addr, alpn)` | |
| accept conn | `await endpoint.accept()` | |
| open bi-stream | `await conn.open_bi()` | |
| accept bi-stream | `await conn.accept_bi()` | |
| send | `await send_stream.write_all(b)` | |
| recv | `await recv_stream.read(n) -> bytes\|None` | |
| half-close | `await send_stream.finish()` | |
---
## 2. The `trio`-native uniffi future bridge (`tractor/ipc/_uniffi_trio.py`)
### 2.1 what uniffi actually generates
`uniffi`'s async support does not use asyncio *semantically*
it uses asyncio only as the *executor* for a poll loop. The
generated python for an `async fn` is, in shape:
1. call `_uniffi_..._<method>(...)` → returns an opaque
`RustFuture` handle (a `void*`/`u64`).
2. loop: call
`ffi_..._rust_future_poll_<T>(handle, callback, callback_data)`.
The callback is a C-ABI fn pointer invoked **from an
arbitrary rust thread** with a poll-result code
(`READY`/`MAYBE_READY`).
3. the generated glue's callback resolves an
`asyncio.Future` via `loop.call_soon_threadsafe(...)`; the
coroutine awaits it, then re-polls.
4. on ready: `ffi_..._rust_future_complete_<T>(handle,
&call_status)` → the value; then
`ffi_..._rust_future_free_<T>(handle)`.
**The asyncio dependency is confined to step 3.** That is the
whole insight: the bridge is ~40 lines.
### 2.2 the trio version
```python
async def await_rust_future(
poll: Callable, # ffi_..._rust_future_poll_<T>
complete: Callable, # ffi_..._rust_future_complete_<T>
free: Callable, # ffi_..._rust_future_free_<T>
handle: int,
lift: Callable[[Any], Any],
) -> Any:
'''
Drive a `uniffi` rust-future to completion on the current
`trio` task, bridging rust-thread wakeups via
`TrioToken.run_sync_soon()`.
'''
token = trio.lowlevel.current_trio_token()
while True:
wake = trio.Event()
# NOTE, invoked from a *rust* thread!
def _cb(_data, poll_code):
token.run_sync_soon(wake.set)
cb = _UNIFFI_FUTURE_CALLBACK(_cb) # keep a strong ref!
poll(handle, cb, 0)
await wake.wait()
if <poll_code was READY>:
break
try:
status = _UniffiRustCallStatus.default()
res = complete(handle, status)
_uniffi_check_call_status(status) # reuse generated helper
return lift(res)
finally:
free(handle)
```
Critical details, each a real bug if missed:
- **`token.run_sync_soon()` is the only trio API callable from a
foreign thread**, and it is documented as such. Use it; do
*not* use `trio.from_thread.run_sync` (requires a trio thread
context) and do not touch the `Event` directly from the
callback.
- **the poll code must reach the trio side.** Capture it in a
`nonlocal`/1-slot list written by the callback *before*
`run_sync_soon`, since the callback owns the value. Handle
`MAYBE_READY` by re-polling (the loop above does).
- **keep the `ctypes` callback object alive** across the await —
a GC'd `CFUNCTYPE` trampoline is a segfault. Bind it to a
local *and* make sure the local outlives the `poll()` call
window.
- **cancellation.** `await wake.wait()` is a trio checkpoint, so
a `Cancelled` can fire while rust still owns the future. On
cancel we must still `free(handle)` — and per uniffi, the
correct sequence is to call the generated
`ffi_..._rust_future_cancel_<T>(handle)` then continue
polling to completion before `free`. Wrap the whole thing so
the cancel path does:
`with trio.CancelScope(shield=True): cancel(handle); <drain
poll loop>; free(handle)`. **Bounded** shield (add a
`trio.move_on_after()` with a module-level constant) so a
wedged rust future can't make an actor un-cancellable —
`tractor` is SC-first and an unbounded shield here would
violate that.
- **`trio.lowlevel.current_trio_token()`** must be captured on
the trio side (not in the callback).
### 2.3 how to apply it to the generated bindings
Do **not** fork/vendor the generated `iroh` python. Instead ship
a *narrow* re-dispatch shim:
- write `tractor/ipc/_uniffi_trio.py` with `await_rust_future()`
plus a `@cm patch_uniffi_for_trio()` that monkey-patches the
generated module's single async-driver entrypoint (in current
uniffi that's `_uniffi_rust_call_async` / `_rust_call_async`,
one function) to the trio implementation.
- verify at import time that the expected symbol exists and
raise a clear, actionable error naming the pinned `iroh`
version if not. A silent fallback to asyncio would be a
nightmare to debug.
- **plan for this to break on `iroh`/`uniffi` upgrades.** Mitigate
with (a) a unit test that drives one trivial `iroh` async call
under bare `trio.run()` and asserts no event loop was ever
created (`asyncio.get_event_loop_policy()` untouched /
`asyncio._get_running_loop() is None`), and (b) a docstring
pointing at the uniffi codegen template this mirrors.
If step 0 reveals the generated code is *structurally* hostile
to this (e.g. `asyncio` imported and used at module scope for
more than the driver), fall back to option (b): run iroh under
`tractor.to_asyncio` infected mode and open the follow-up to
revisit. Say so in the PR rather than fighting it.
---
## 3. Mapping QUIC onto `MsgTransport`
### 3.1 the layering decision
QUIC natively multiplexes streams inside one connection. The
mapping that preserves *all* existing `tractor` semantics with
the least new code:
```
iroh Endpoint == one per actor (process) -> the "listener"
iroh Connection == one per peer actor -> pooled
iroh bi-stream == one `Channel`/`MsgTransport` -> 1:1
```
- keep the 4-byte `<I` length-prefix framing **unchanged**. It's
redundant-ish over a QUIC stream but it means
`MsgpackTransport` is reused verbatim, and framing is cheap.
Revisit only after it works.
- **one-task-per-stream** falls out naturally, which is exactly
the #353 note about QUIC sub-stream QoS/cancellation fitting
`trio`.
- `layer_key: int = 4` still (QUIC is L4-ish); note in a comment
that this backend is really 4+security+multiplex.
**Connection pooling** is the one place we add state the other
backends don't have: dialing the same peer twice should reuse
the `Connection` and open a second bi-stream. Implement as a
module-level `dict[NodeId, Connection]` guarded by a
`trio.Lock`... **no** — that's a per-process cache with
lifetime/teardown hazards. Instead reuse the codebase's existing
idiom: `tractor.trionics.maybe_open_context()` keyed on the
node-id, which already solves exactly this (one-cached-resource-
per-key, refcounted, teardown-on-last-exit) and whose teardown
semantics were just hardened (gh #488). Use it; do not hand-roll
a cache. Anything concurrency-subtle here should get the
`conc-anal` skill run over it.
### 3.2 `IrohAddress`
```python
class IrohAddress(
msgspec.Struct,
frozen=True,
):
_node_id: str # 32B ed25519 pubkey, hex or z32
_alpn: str = 'tractor/0' # the bindspace!
# optional dial hints; NOT part of identity
maybe_relay_url: str|None = None
maybe_direct_addrs: tuple[str, ...] = ()
proto_key: ClassVar[str] = 'iroh' # ?or 'quic'; see §3.2.1
unwrapped_type: ClassVar[type] = tuple[str, str]
def_bindspace: ClassVar[str] = 'tractor/0'
```
- **`.unwrap() -> (node_id_str, alpn_str)`** — a `(str, str)`
tuple, which is *unambiguously distinct* from
`TCPAddress`'s `(str, int)`. But careful:
`wrap_address()`'s UDS case is
`case (_, filename) if type(filename) is str` — which
**already catches `(str, str)`**. So the iroh `case` MUST be
ordered *before* the UDS case and guarded, e.g.
`case (str() as nid, str() as alpn) if _is_node_id(nid):`
with `_is_node_id()` a cheap length+alphabet check. Add a
regression test asserting a UDS `(dir, filename)` pair still
wraps to `UDSAddress` — this is the exact "wrong transport
loaded" hazard `_addr.py:214` warns about.
- `.bindspace``self._alpn`. This is the honest analogue:
the ALPN is the set of endpoints willing to talk to you, and
two `tractor` deployments sharing an iroh network are
separated by ALPN exactly as two UDS deployments are
separated by directory. Include a `tractor` version/proto
epoch in the default ALPN so incompatible runtimes can't
handshake.
- `.is_valid` → node-id parses, alpn non-empty.
- **`get_root()` is the hard one.** There is no
well-known-port analogue: an iroh node id is a *keypair*, so
"the host's default registrar addr" requires a *persisted
secret key*. Design:
- the root/registrar's secret key lives at
`get_rt_dir() / 'iroh_registrar.key'` (0600), created on
first use.
- `get_root()` must stay **pure and import-time-safe**
(contract §2.3: `_default_lo_addrs` is built at import!).
So `get_root()` *reads* the key file if present and
otherwise returns an `IrohAddress` with
`_node_id=''`/sentinel, and the **generation** happens in
an explicit sibling — `ensure_registrar_key() ->
IrohAddress` — called from the listen path. Pure getter,
explicit setter; do not smuggle key generation into
`get_root()`.
- this almost certainly means `_default_lo_addrs` must become
lazy for this backend. **Land that refactor as its own prep
commit** (a `default_lo_addrs()` that computes per-call
instead of the import-time dict) — it also unblocks plan
03's netns-scoped defaults.
- `get_random()`: generate a fresh `SecretKey` per subactor and
return its node-id. Note this runs post-fork pre-listen
(contract §4) and costs an ed25519 keygen (~µs, fine). The
*secret* can't live in a frozen `Address`, so it must be
stashed where the listen path can find it: a module-level
`dict[node_id, SecretKey]` populated by `get_random()` and
consumed+popped by `start_listener()`. Ugly but honest;
document it and note the alternative (thread the key through
`Endpoint`) as a follow-up.
#### 3.2.1 `proto_key`: `'iroh'` vs `'quic'`
Use **`'quic'`** for the `proto_key`/`--tpt-proto` name and
name the module `_quic.py`, with `iroh` as the *implementation*.
Rationale: it keeps the door open for the `aioquic` fallback
(§1) without a user-visible rename, and it matches how `uds` is
a proto name rather than a lib name. Put `iroh`-specific bits
behind an internal `_iroh` submodule if the file gets big.
### 3.3 the `trio.abc` adapters — where the real work is
Contract §3 says a non-socket backend needs three upstream
generalizations. Land them **as a prep PR, before any iroh
code**, so they can be reviewed on their own merits with
tcp/uds still the only backends:
1. **`Endpoint.start_listener()` must not assume
`.socket.getsockname()`.** Use the same
`Address.rebind_from_sockname: ClassVar[bool]` gate that
plan 01 §3.2 introduces — coordinate so it lands once. (If
plan 01 lands first, this is free.)
2. **`transport_from_stream()` (`_types.py:92`) must not assume
`trio.SocketStream`.** Replace the `sock.family` match with:
check `isinstance(stream, trio.SocketStream)` → existing
family match; else look for a
`stream.tpt_key: ClassVar[MsgTransportKey]` attribute on the
adapter and use it. Keeps the existing path byte-identical
and makes new stream types self-describing (a much better
shape than growing an `isinstance` ladder).
3. **type annotations**: `handle_stream_from_peer(stream:
trio.SocketStream)` → `trio.abc.Stream`; `Endpoint._listener:
SocketListener|None` → `trio.abc.Listener|None`;
`MsgTransport.stream: trio.SocketStream`
`trio.abc.Stream`. Annotation-only, zero behaviour change.
Then the adapters:
```python
class QuicMsgStream(trio.abc.HalfCloseableStream):
'''
A single `iroh` bi-directional QUIC stream presented as
a `trio` byte-stream so `MsgpackTransport` can frame over
it unmodified.
'''
tpt_key: ClassVar[MsgTransportKey] = ('msgpack', 'quic')
def __init__(self, conn, send, recv) -> None: ...
async def send_all(self, data: bytes) -> None: ...
async def wait_send_all_might_not_block(self) -> None: ...
async def receive_some(self, max_bytes: int|None = None) -> bytes: ...
async def send_eof(self) -> None: ...
async def aclose(self) -> None: ...
```
Non-negotiable behaviours (each maps to a `match` case that
already exists in `_transport.py` and must keep working):
- `receive_some()` returns `b''` at clean EOF →
`MsgpackTransport._iter_packets()` sees `header == b''` and
raises `TransportClosed(loglevel='transport')`. **This is the
graceful-disconnect path the whole runtime relies on**; get it
right first.
- a reset/aborted stream → raise `trio.BrokenResourceError`.
- use after local close → raise `trio.ClosedResourceError`
(ideally with `'another task closed this fd'`-equivalent text
absent, so the `raise_on_report` branch at
`_transport.py:290` stays quiet).
- `send_all()` on a closed peer → `trio.BrokenResourceError`.
- honour `trio`'s one-task-per-direction rule: guard with
`trio._util.ConflictDetector` equivalents (or just document +
assert), because `MsgpackTransport` already serializes sends
with a `StrictFIFOLock` but recvs are single-task by
construction.
- **buffering**: if iroh's `read()` doesn't support
"read up to n", `receive_some()` must maintain an internal
leftover buffer. Note `MsgpackTransport` wraps us in
`tricycle.BufferedReceiveStream` anyway, so `receive_some()`
just needs *some* nonzero-progress contract.
```python
class QuicListener(trio.abc.Listener):
'''
Accepts iroh `Connection`s and yields one `QuicMsgStream`
per accepted bi-stream, so `trio.serve_listeners()` spawns
one `handle_stream_from_peer()` per `Channel`.
'''
async def accept(self) -> QuicMsgStream: ...
async def aclose(self) -> None: ...
```
The accept-side subtlety: `trio.abc.Listener.accept()` yields
one stream per call, but iroh gives us *connections* which then
yield *streams*. So `QuicListener` needs an internal
`trio.MemoryReceiveChannel[QuicMsgStream]` fed by a background
task-pair (one task accepting connections, one per connection
accepting bi-streams). `trio.abc.Listener` has no nursery, so:
make the listener **constructed by an `@acm`** that owns the
nursery, and have `start_listener()` be that `@acm`'s driver.
⚠️ this collides with `Endpoint.start_listener()` being a plain
`async def` returning a listener. Two options:
- **(a)** hang the nursery off the `Endpoint`'s existing
`listen_tn``_serve_ipc_eps()` already creates `listen_tn`
and passes it into every `Endpoint` (`_server.py:1063-1074`),
and `Endpoint.listen_tn` is right there. So
`start_listener()` can `self.listen_tn.start_soon(...)` the
acceptor tasks. **Recommended**: no upstream signature change,
correct lifetime (dies with the ep group), and it's why
`listen_tn` is on the struct in the first place.
- (b) change `start_listener()` to a `@acm`. Bigger blast
radius; only if (a) proves insufficient.
Since `start_listener()` is called via
`inspect.getmodule(addr)` with only `addr=` (contract §1.3),
option (a) needs the `Endpoint` itself. Either add `ep=` to the
module-level `start_listener()` call signature (all backends
ignore it except quic → small upstream change, do it as part of
the prep PR and make it keyword-only with a default) or have
`QuicListener.accept()` lazily spawn via
`trio.lowlevel.current_task().parent_nursery` (**rejected** —
fragile, implicit). Do the explicit `ep=` kwarg.
### 3.4 `maddr`
Multiaddr already standardizes the pieces:
```
/ip4/<h>/udp/<p>/quic-v1 # direct
/ip4/<h>/udp/<p>/quic-v1/p2p/<node-id> # direct + identity
/dns/<relay-host>/tcp/443/tls/ws/p2p/<node> # relay-ish
```
- primary form: `/p2p/<node-id>` alone is a legal maddr and is
the *only* required component for iroh dialling — relay +
direct addrs are discovery hints. So `mk_maddr()` emits
`/p2p/<node_id>` and, when known, prefixes the direct
`/ip4/../udp/../quic-v1/`.
- `/p2p/` values are multihash-encoded peer ids; an iroh node-id
is a raw ed25519 key. Converting requires the identity
multihash + libp2p key protobuf wrapper. **Decide**: emit the
raw node-id under a *tractor-local* `/iroh/<node-id>` segment
(needs upstream registration, same track as `wg`/`tipc`,
gh #483) rather than pretending to be a libp2p peer-id we
can't round-trip. Return the `str` form until upstream lands
(`MsgTransport.maddr` is `Multiaddr|str`).
- this backend is the strongest argument for gh #443's
**tunnelled/composed maddr** item: `/ip4/../udp/../quic-v1/..`
*is* a composed stack. Cross-reference plan 03 §5 so the two
grammars land compatibly.
---
## 4. Discovery integration
- iroh's node-id addressing means the `tractor` registrar can
hold `IrohAddress`es that are **reachable from anywhere** with
no port-forwarding — that is the headline feature. The
registrar itself works unchanged.
- iroh has its own discovery (DNS/pkarr/mdns). **Out of scope**;
note in the follow-up that `tractor.discovery` could
eventually delegate to it, which would be the direct analogue
of plan 01's TIPC-topology idea.
- relay servers: default to n0's public relays for the demo,
document self-hosting (docs.iroh.computer's dedicated-infra
page is linked from #353), and make the relay set a
`start_listener()` kwarg.
## 5. Security note
QUIC is TLS-1.3-always and iroh authenticates by node-id, so
this backend is the first `tractor` transport with real
transport security and peer authentication. Two things follow:
1. an **allowlist hook** — an actor should be able to reject
inbound connections from unknown node-ids *before* the
`Aid` handshake. Natural home: a predicate kwarg on
`start_listener()`, evaluated in `QuicListener`'s connection
acceptor task. Sketch it; ship it in PR 1 if cheap (it is).
2. do **not** claim any security property for the other
backends by association. `tcp`/`uds`/`tipc` remain
unauthenticated; that's what plan 03 (wg) is for.
## 6. Commit sequencing
0. **spike (throwaway, not committed)**: drive iroh under
`trio-asyncio`/`tractor.to_asyncio`, echo bytes over a
bi-stream between two procs. Fills in §1.1. Timebox it.
1. prep PR: annotation widening + `rebind_from_sockname` gate +
`transport_from_stream()` `tpt_key` dispatch + `ep=` kwarg on
`start_listener()` + lazy `default_lo_addrs()`. **No new
backend.** Full suite green on tcp *and* uds.
2. `_uniffi_trio.py` + its tests (drive one iroh async call
under bare `trio.run()`; assert no asyncio loop; assert
cancellation frees the future).
3. `QuicMsgStream` + tests against a *loopback* iroh endpoint
pair in one process (no `tractor` runtime): send/recv, clean
EOF → `b''`, reset → `BrokenResourceError`, use-after-close
`ClosedResourceError`.
4. `QuicListener` + `start_listener()` + `IrohAddress` +
key-file mgmt.
5. `MsgpackQuicStream(MsgpackTransport)` + `connect_to()` +
`maybe_open_context()` connection pooling.
6. registration tables + `--tpt-proto quic` + full suite.
7. maddr + docs + a two-host example (pairs with #482's format).
## 7. Testing
- capability predicate `is_quic_available()``iroh` importable
*and* the uniffi driver symbol present at the pinned version.
Same `pytest.fail`-early hook as plan 01 §7.2.
- **the acceptance bar is the same**: whole suite green under
`--tpt-proto quic`. Expect this to shake out real bugs in the
adapters (esp. teardown ordering and `TransportClosed`
classification) — that's the point.
- expect to need **timeout headroom**: iroh endpoint bind +
first connect (relay discovery) is orders of magnitude slower
than a UDS bind. Before touching any test deadline, rule out
the CPU-throttle false-positive (see the project's
`env_cpu_throttle_masquerades_as_regression` note); then, if
real, add a per-proto timeout multiplier to the test harness
rather than editing individual tests.
- a no-network test mode: iroh with relays disabled +
loopback direct addrs only, so CI doesn't depend on n0's
infra. **Make this the default in CI**; mark the relay tests
`pytest.mark.net` and keep them out of the default run.
- leak checks: assert every `SecretKey`/`Endpoint` is closed on
actor teardown (an `Endpoint` left open holds UDP sockets and
relay connections; a leak here shows up as hung tests, not
errors).
## 8. Risks
| risk | mitigation |
| --- | --- |
| uniffi codegen internals shift on upgrade | pinned minor, symbol assertion at import, the "no asyncio loop" test, documented fallback to `to_asyncio` |
| rust-thread callback → trio wakeup mishandled (segfault / lost wakeup / un-cancellable task) | strong ref on the ctypes trampoline; `run_sync_soon` only; **bounded** shielded cancel-drain; run the `conc-anal` skill over the bridge |
| `iroh` wheel availability for 3.13/3.14 on linux+macos | verify in step 0; if missing, that alone may force the `aioquic` fallback |
| QUIC latency/jitter destabilizes the existing suite's timing assumptions | per-proto timeout multiplier, relay-less CI mode |
| `(str, str)` unwrapped form collides with UDS in `wrap_address()` | guarded case ordered first + explicit regression test (§3.2) |
| scope creep into iroh's docs/blobs/gossip crates | this backend is `Endpoint`+`Connection`+bi-streams only; anything else is a separate issue |
## 9. Follow-up issue seeds
- `tractor.discovery` delegating to iroh discovery (DNS/pkarr/mdns)
- per-`Context` QUIC sub-streams: today one `Channel` == one
stream; QUIC would let each `tractor.Context` own its own
stream with independent flow-control and cancellation — this
is the genuinely novel win #353 gestures at, and it's a
runtime-layer change, not a transport one
- unreliable QUIC datagrams for a lossy-ok broadcast transport
(pairs with plan 01's TIPC-multicast seed)
- node-id allowlist → a real `tractor` authz story
- `aioquic` sans-io backend reusing §3's adapters

View File

@ -0,0 +1,757 @@
# Plan 03 — WireGuard (and other tunnels) as a *nested bindspace* via `pyroute2`
Tracks gh [#482] + the tunnelled-maddr item of [#443].
Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: WireGuard is **not** a `MsgTransport`. It is an
interface-layer tunnel that is transparent to `socket(2)`, so
the correct abstraction is a *bindspace* — a scoped,
`@acm`-managed network context that an existing L4 transport
(`tcp`, and later `quic`/`tipc`-over-UDP-bearer) binds *inside*.
This plan implements `Address.namespace` (spec'd but unused
since day one) and the composed/tunnelled maddr grammar, with
`pyroute2` as the netlink codec and as much of the I/O moved
onto `trio` as the library's sans-io layer allows.
[#482]: https://github.com/goodboy/tractor/issues/482
[#443]: https://github.com/goodboy/tractor/issues/443
---
## 1. What exists today (verified, per #482)
- `wrap_address()` accepts maddr `str`s (leading-`/` dispatch,
`_addr.py:262`). `parse_maddr()` and `mk_maddr()` support plain
TCP/UDS addresses plus nested, canonical bearer-first `/wg/`
stacks represented locally as `TunnelledAddress` wrappers.
- there is no `wg` proto in the multiaddr *spec* yet, but
multiformats/py-multiaddr#108 (key form `u<base64url>`) is
**merged** as of 2026-07-28 (`f86519da`) — and unreleased, the
latest `0.2.0` predating it. Spec registration is still tracked
by multiformats/py-multiaddr#107 and gh #483.
- **today's deployable story remains declarative**: run `wg-quick`
out-of-band, parse the maddr, strip its wrapper to the overlay
`(host, port)`, verify the pubkey against the live tunnel,
hand the overlay addr to `registry_addrs=`/`tpt_bind_addrs=`.
#482 already contains working example code for exactly this.
- `Address.namespace` exists in the Protocol
(`_addr.py:94-101`, "the if-available OS-specific network
namespace key"). `TunnelledAddress` implements it from its spec;
no concrete transport backend implements it yet.
## 2. Three layers, three PRs
| layer | what | dep | ships |
| --- | --- | --- | --- |
| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u<key>` → overlay `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first |
| **B. `pyroute2` read/verify** | replace the `subprocess.run(['sudo','wg','show'])` shelling with netlink queries | `pyroute2` extra | second |
| **C. `@acm` lifecycle** | create/configure/tear down wg ifaces + netns *from the runtime*, as nested bindspaces; implement `Address.namespace` | `pyroute2` + `CAP_NET_ADMIN` | third |
Each is independently valuable and independently reviewable.
**Do not attempt C first** — the interesting design (nested
bindspace `@acm`s) is only well-posed once A has pinned the
address grammar and B has proven the netlink path under trio.
---
## 3. Layer A — declarative `wg` maddrs
### 3.1 the address shape
The decision: **a wg segment annotates an existing address, it
does not create a new address type.** Two candidate encodings;
**pick (a)**:
- **(a) `TunnelledAddress` wrapper** (recommended):
```python
class TunnelledAddress(
msgspec.Struct,
frozen=True,
):
overlay: Address # e.g. TCPAddress
tunnel: WGTunnelSpec # proto-specific, frozen
```
with `.proto_key` **delegating to `overlay.proto_key`** so every
existing table lookup (`_addr_to_transport`,
`enable_transports` guard at `_root.py:391`,
`transport_from_addr()`) keeps working untouched, and
`.unwrap()` delegating to `overlay.unwrap()` so **nothing new
crosses the wire**. `.namespace` and `.bindspace` come from
the tunnel spec. The wrapper is stripped (`→ .overlay`) at the
moment of bind/connect.
- ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests
`type(addr) in _address_types.values()` — a `bidict` of
proto_key→type. `TunnelledAddress` isn't in it and must not
be (it's not 1:1 with a proto). So either add an explicit
`isinstance(addr, TunnelledAddress)` clause there, or give
the wrapper a marker and test structurally. Do the former;
it's two lines and honest.
- the reflection in `Endpoint.start_listener()`
(`inspect.getmodule(self.addr)`) would resolve to the
*wrapper's* module, not the transport's. **So the wrapper
must be unwrapped before it reaches `Endpoint`** — i.e. by
the bindspace `@acm` (layer C) or by `parse_maddr()`
(layer A). State this loudly in the docstring; it's the #1
way to get this wrong.
- (b) add fields to each existing `Address` type. Rejected:
duplicates tunnel logic per-backend and pollutes `.unwrap()`.
```python
class WGTunnelSpec(
msgspec.Struct,
frozen=True,
):
peer_pubkey: str # std-base64 `wg(8)` form
iface: str = 'wg0'
netns: str|None = None
# layer-C-only fields, unset in layer A
maybe_endpoint: tuple[str, int]|None = None
maybe_allowed_ips: tuple[str, ...] = ()
```
### 3.2 `parse_maddr()`/`mk_maddr()`
Grammar — **verified** against py-multiaddr#108, first on the
`baudco/py-multiaddr@wg_support` branch and re-verified after it
merged upstream (`multiformats/py-multiaddr@f86519da`); all three
forms below parse *and* round-trip. Note the codec also validates
that the key decodes to exactly 32 bytes, so a truncated key is a
`StringParseError`, not a silently-mangled parse:
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\_______ bearer __________/\__ key __/\______ overlay ______/
underlay, wg `ListenPort` the `MsgTransport` bind
```
The `/wg/` segment is **infix, not suffix** — the segments
*before* it are the wg **bearer** (the underlay `(ip, udp-port)`
that `wg(8)` itself listens on, per the codec docstring's own
`/ip4/1.2.3.4/udp/51820/wg/{key}` example), and the segments
*after* are the **overlay** endpoint that `tractor` binds.
⚠️ **CORRECTION** — an earlier revision of this plan (and the
examples in gh #482) used a *suffix* form
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. That parses, but it is
semantically inverted: it puts the overlay addr where the bearer
belongs, `tcp` where wg's `udp` `ListenPort` goes, and declares
no overlay endpoint at all. `parse_wg_maddr()` in
`examples/multihost/wg_lan/` now rejects it with an actionable
error.
Observed protocol-name lists, for writing the `match`:
| maddr | `[p.name for p in m.protocols()]` |
| --- | --- |
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
- so the three parts have **three different owners**, and only the
third is an `Endpoint`:
| part | socket owner / provisioner | runtime role |
| --- | --- | --- |
| bearer | kernel-owned; externally provisioned in layer A, tractor bindspace-provisioned in layer C | control-plane metadata, never an `Endpoint` |
| `/wg/u<key>` | nothing — it's an identity | parsed and explicitly verified |
| overlay | `tractor`'s `IPCServer` | application `MsgTransport`, as `.overlay` |
This owner-split is the real axis of the design, *not* whether
the maddr stack is "composed" (it is).
- ⚠️ **CORRECTION**, an earlier draft of this section specced a
hand-rolled `_peel_tunnel_segs(proto_names) -> (bearer_names,
tunnel_specs, overlay_names)`. **Do not write it.**
`py-multiaddr` already ships the whole tunnel compose/peel API
and it was simply missed here — see its README "En/decapsulate"
and "Tunneling" sections, and gh #443's 2nd bullet which links
them. Verified against the pinned rev:
| need | API |
| --- | --- |
| isolate the bearer | `ma.decapsulate_code(P_WG)` |
| drop the overlay, keep bearer+key | `ma.decapsulate(overlay_ma)` |
| per-seg maddrs | `ma.split()` |
| rejoin a seg tail | `Multiaddr.join(*segs)` |
| read the key | `ma.value_for_protocol('wg')` |
| recompose | `bearer.encapsulate(key).encapsulate(overlay)` |
`.decapsulate_code()` handles the infix `/wg/` seg cleanly
*because* it cuts on proto-code and never tries to match an
addr value — the key seg has no addr of its own. This is the
same NIH trap gh #429 existed to close, one layer up.
- ⚠️ `value_for_protocol('ip4')` on a *full* tunnelled maddr
silently returns the **first** match, i.e. the bearer's host.
Always call it on a peeled sub-maddr, never the whole stack.
- `parse_maddr()` gains a case on
`[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), <overlay-l4>]`
peel w/ the API above, decode the multibase key to std-base64,
and return `TunnelledAddress(overlay=..., tunnel=WGTunnelSpec(
...))` w/ the bearer recorded in the spec.
- keep the existing 2-proto cases byte-identical; add the new
case *after* them.
- nesting (wg-in-wg) falls out of `.decapsulate_code()` cutting
at the *last* occurrence — peel repeatedly rather than
recursing through a bespoke splitter.
- `mk_maddr()` inverse for `TunnelledAddress` is just
`.encapsulate()` composition; don't rebuild `str`s by hand.
- **pending an upstream release**: py-multiaddr#108 is merged, so
`Multiaddr('/…/wg/u…')` parses off a PEP 621 direct-revision pin,
since no release carries the codec. Gate parser entry on
`_wg_proto_code()`, implemented as
`protocols.protocol_with_name('wg')` under
`except ProtocolNotFoundError`. Do **not** probe by parsing a
dummy like `Multiaddr('/wg/uAAAA')` — the codec enforces a
32-byte key, so that raises even when the proto *is* known. Do
**not** hand-roll a `wg` parser in `tractor` — the whole point
of #429 was dropping the NIH parser.
### 3.3 pure codecs + explicit verification
Port #482 §2's pure helpers into
`tractor/discovery/_tunnel.py`, keeping the impure probe cleanly
separated until layer B:
```python
def parse_wg_maddr(maddr: str) -> TunnelledAddress: ... # pure
def wg8_pubkey(multibase_key: str) -> str: ... # pure
async def verify_wg_peer(spec: WGTunnelSpec) -> bool: ... # layer B
```
Layer A's example-local `verify_wg_peer()` may shell out (`wg show
<if> peers`), but layer B replaces that probe with one explicit async
function backed by pyroute2. Never call it implicitly from
`wrap_address()`/`parse_maddr()` — parsing must stay pure and
side-effect-free; verification is the *caller's* explicit step
(and later, the bindspace `@acm`'s).
### 3.4 deliverables
- `examples/` scripts distilled from #482 §§3-5 (this is the
unchecked "commit examples from ^" bullet in #443). They live
under `examples/multihost/``test_docs_examples.py` walks
`examples/` recursively and runs every collected file as a
subproc asserting `rc == 0` (it doesn't even filter by
extension, so a stray `README.md` would be `python`-run too),
and `'multihost' not in p[0]` is already in its exclusion
list. Anything needing a real second host or a live tunnel
belongs there.
- a `docs/` page: tunnel setup, the maddr form, the two-host
run. Keep prose in the docs; keep the examples runnable and
minimal.
- tests: maddr round-trip, `TunnelledAddress` delegation
(`proto_key`/`unwrap` identical to overlay), `wrap_address()`
regression (a tunnelled maddr `str``TunnelledAddress`; a
plain one → unchanged), and **a real end-to-end over a
locally-created wg pair** gated on `CAP_NET_ADMIN` (see §5.4).
---
## 4. Layer B — `pyroute2` under `trio`
### 4.1 the library situation (verify at implementation time)
`pyroute2` ≥0.9 rewrote its core onto **asyncio**
(`AsyncIPRoute`; the sync `IPRoute` wraps it with its own loop).
It also ships a `WireGuard` netlink (generic-netlink) module
supporting `.set(iface, private_key=..., peer={...})` and
`.info(iface)`, plus `pyroute2.netns` / `NetNS` for namespaces,
and `IPRoute.link('add', kind='wireguard', ifname=...)`.
Three integration options, in increasing trio-nativeness:
- **(1) `trio.to_thread.run_sync()` around the sync API.**
Netlink ops here are one-shot, sub-millisecond, and happen at
bind/teardown time only — *not* in the msg hot path. This is
the **correct default**: it's ~10 lines, uses a battle-tested
API, and costs nothing where it's used.
- **(2) sans-io: `trio.socket` + pyroute2's message codecs.**
`pyroute2`'s message classes
(`pyroute2.netlink.rtnl.*`, `pyroute2.netlink.generic.wireguard.wgmsg`)
encode/decode independently of its I/O core. So a
`tractor/ipc/_netlink.py` with a small trio `NetlinkSocket`
(`trio.socket.socket(AF_NETLINK, SOCK_RAW|SOCK_DGRAM, proto)`,
`sendto`/`recv`, seq/pid matching, `NLMSG_DONE`/`NLMSG_ERROR`
handling) + pyroute2 codecs is very achievable and is the
honest reading of "as much trio wrapping as possible where any
other async support can be replaced".
**Do this for the paths we actually need** (link add/del,
addr add, wg get/set, netns bind) and *only* those — a
general netlink client is out of scope.
- (3) reimplement the codecs. Never.
**Recommended split**: ship (1) first so layer B is a small,
reviewable, behaviour-preserving swap of `verify_wg_peer()`'s
body; then land (2) as a follow-up commit for the read path
(`wg get`, `link get`) where the sans-io surface is smallest,
and keep (1) for the privileged mutating ops. Measure before
converting anything else — there is no perf argument here, only
a "no foreign event loop in a trio actor" argument, which (1)
already satisfies (a thread is not an event loop).
Explicitly **do not** pull in `trio-asyncio` for pyroute2 or infect
every wg-using actor merely to service one-shot netlink calls. A
dedicated `wgman` actor (§5.1) is the one plausible asyncio-hosted
shape: it can use tractor's own `.to_asyncio` task linkage while
keeping the foreign loop and provisioning authority out of ordinary
actor processes.
### 4.2 API shape
Pure-ish, functional, `@acm` for anything with teardown:
```python
async def read_wg_peers(
iface: str = 'wg0',
netns: str|None = None,
) -> tuple[str, ...]: ... # base64 pubkeys
async def read_wg_pubkey(iface: str = 'wg0', ...) -> str: ...
```
and `verify_wg_peer()` becomes a thin composition over one shared key
snapshot.
Note the pure-getter rule: no `read_wg_peers(..., create=True)`.
---
## 5. Layer C — nested bindspace `@acm`s + `Address.namespace`
This is the part #443 and `multiaddr_declare_eps.md` actually
ask for: *"for any tunneled maddr-`str`-entry we deliver a
data-structure which can easily be passed to nested `@acm`s
which consecutively setup nested net bindspaces for binding the
endpoint addrs"*.
Layer C is where tractor takes ownership of bindspace orchestration.
For a fully bootstrapped deployment it may create the netns and wg
iface, configure peers/routes, and ask the kernel to establish the
bearer's UDP `ListenPort` through netlink/`pyroute2`. "Kernel-owned"
describes the data-plane socket, not who provisions it: tractor owns
the lifecycle while `Endpoint`/`MsgTransport` remain responsible only
for the overlay application socket.
### 5.1 candidate default: first-child `wgman`
For a WG-enabled deployment profile, consider eagerly spawning one
private **WireGuard manager** (`wgman`) as the root actor's logical
first child. It is a narrow network-control-plane service, not a
general worker and not an application-visible transport endpoint.
The naive profile gets one manager for the actor tree; advanced
deployments may disable it for pre-provisioned networking or place
one manager in each capability/bindspace security domain.
"First child" describes supervision and teardown ordering, not a
serial startup barrier. Submit the `wgman` spawn in the same startup
wave as ordinary children, start its pyroute2 import, generic-netlink
discovery and declared-tunnel reconciliation immediately, and publish
a readiness signal separately. Sibling processes can boot in parallel;
only their first WG-dependent bind/dial waits for manager readiness.
This overlaps setup with actor-tree startup and avoids every sibling
paying its own pyroute2/loop/socket initialization latency.
The initial manager can still call the sync helpers from §4.1. A
natural follow-up is to spawn it with `infect_asyncio=True` and keep
`AsyncWireGuard` clients alive on asyncio's host loop through
`tractor.to_asyncio.run_task()`. Tractor then owns cross-loop task
linkage, cancellation and error propagation, while normal siblings
remain plain Trio actors. Keep one client per realized namespace or
other kernel control domain; do not share a pyroute2 socket across
domains merely to reduce object count.
Keep the authority surface deliberately small:
- accept structured inspect/verify/ensure/release requests derived
from `WGTunnelSpec`, `BindspaceSpec` and explicit `role`; never
expose arbitrary pyroute2 calls, shell commands or `setns()` RPC;
- let the root/supervisor mediate access initially, or hand siblings
a scoped manager capability; do not register a privileged `wgman`
endpoint for unrestricted cluster-wide discovery;
- never return private keys or namespace FDs to application actors;
pass secrets and live capabilities into the manager through the
supervisor-owned bootstrap path;
- grant only the capabilities required for the manager's assigned
domain. Prefer a manager already placed in that user/net namespace
over one process holding ambient authority across every namespace;
- make ensure/release idempotent and reference-count ownership so one
sibling cannot tear down a tunnel still borrowed by another.
The root owns the manager's lifetime. `wgman` must outlive all
siblings borrowing its tunnels and exit before the root drops the
underlying namespace capabilities. A manager crash fails closed:
dependent operations receive an explicit service error; restart, if
enabled, reconciles declared state idempotently before advertising
readiness again. Do not silently let siblings fall back to privileged
local provisioning, since that defeats both the security boundary and
the single warm control-plane benefit.
Treat eager `wgman` as a measured deployment-profile choice. Compare
root startup with no WG declarations, pre-provisioned read-only WG,
and runtime-managed tunnels before making it unconditional whenever
the `wg` extra is installed. The intended invariant is "one warm
manager per simple WG actor tree", not "every tractor program spawns
a privileged child".
### 5.2 the composition
The maddr describes the composed network path and can be used as
either a source/listen or destination/dial handle. It does **not**
select the local instance of that network stack. A netns, VRF,
interface, user namespace, or equivalent platform resource is
orthogonal augmentation carried alongside/below the maddr.
Keep three bindspace representations with deliberately different roles
and lifetimes:
```python
class BindspaceSpec(msgspec.Struct, frozen=True):
'''Serializable spawn/config declaration.'''
kind: str # `netns`, later `vrf`, ...
key: str|None # requested name/key, if any
lifecycle: Literal['attach', 'open']
class BindspaceRef(msgspec.Struct, frozen=True):
'''Wire-safe, non-owning ref to the realized resource.'''
kind: str
key: str|None # mutable name, absent after unlink
inode: int # host-local Linux nsfs fingerprint
class Bindspace(ProcessLocal):
'''Scoped, non-serializable capability for one live bindspace.'''
spec: BindspaceSpec
ref: BindspaceRef
namespace_fd: int|None
ownership: Literal['owned', 'borrowed']
@acm
async def open_bindspace(
spec: BindspaceSpec,
) -> AsyncGenerator[Bindspace, None]:
'''
Provision/borrow one bindspace and yield its live capability.
'''
```
The initial model limits `BindspaceKind` to `netns` while preserving
the required role split. `BindspaceSpec` is the requested resource and
lifecycle policy. `BindspaceRef` is a serializable, non-owning,
host-local record of the resource that was actually opened; it can be
compared or logged, but cannot reopen, pin or enter that resource.
`Bindspace` is the live capability and uses msgspec's generic struct
storage by inheriting the global `tractor.msg.ProcessLocal` marker. Its
hidden unsupported sentinel blocks direct and nested default msgspec
encoding without a recursive IPC hot-path scan. The live bindspace
validates any supplied FD against `BindspaceRef.inode`; explicit FD
transfer belongs to the supervisor bootstrap path. An FD avoids
name-resolution TOCTOU,
survives rename/unlink, and identifies the exact namespace the parent
provisioned. Extend the kind/field union only when a second platform
resource is implemented.
`BindspaceSpec.lifecycle` is explicit serialized policy:
`'attach'` borrows an existing resource and `'open'` creates/owns one.
`open_bindspace()` dispatches that policy by bindspace kind. Never
infer it from a listen/dial role: either role may use pre-provisioned
or locally owned networking.
The first lifecycle implementation is deliberately borrow-only:
`attach_netns()` opens either `/proc/self/ns/net` when
`BindspaceSpec.key = CURRENT_NETNS`, or a named entry beneath
`/var/run/netns`. It derives a `BindspaceRef` from the opened FD, yields
`ownership='borrowed'`, and closes only that FD on exit. "Attach" does
not call `setns()`; it never creates, enters or removes a namespace.
Future `open_netns()` creation and owned teardown remain a separate
privileged supervisor change.
`open_netns()` is that owned counterpart: it requires a named spec,
creates through pyroute2 in a shielded worker call, attaches the live
FD, and yields `ownership='owned'`. FD closure precedes another
shielded pyroute2 removal call on every post-creation exit, including
cancellation. It still never calls `setns()`; process entry remains a
spawn/bootstrap operation.
`open_bindspace()` is **not** an address factory and does not return a
`TunnelledAddress`. At the declaration layer, listener allocation can
use the live bindspace to replace an overlay while preserving every
tunnel:
```python
async with open_bindspace(
bindspace_spec,
) as bindspace:
listen_decl = declared_addr.get_random(
bindspace=bindspace,
)
transport_addr = strip_tunnels(listen_decl)
```
That sketch intentionally leaves the `.get_random()`/bindspace value
contract open. A concrete transport call returns a concrete overlay;
a declaration-level call may replace the overlay and return a new
`TunnelledAddress`. In either case wrappers remain until the final
transport bind/dial boundary, where `strip_tunnels()` is mandatory.
At the listener boundary, keep the split explicit:
`Endpoint.addr` is the peeled concrete address used for transport
reflection, while `Endpoint.declared_addr` retains the original
wrapper for namespace diagnostics and later bindspace orchestration.
Per-platform provisioning still composes one resource context per
tunnel/bindspace layer:
```python
@acm
async def open_netns(
spec: BindspaceSpec,
) -> AsyncGenerator[Bindspace, None]: ...
@acm
async def open_wg_iface(
spec: WGTunnelSpec,
config: WGInterfaceConfig,
bindspace: Bindspace,
role: Literal['listen', 'dial'],
) -> AsyncGenerator[WGTunnelSpec, None]: ...
```
`WGInterfaceConfig` and each `WGPeerConfig` are process-local and
rejected by the global `ProcessLocal` wire guard. The interface config
owns its private key, local addresses and listen port; each peer owns
its public key, allowed CIDRs, optional endpoint, preshared key and
keepalive. Reprs redact private/preshared keys. `WGTunnelSpec` remains
serializable public maddr-derived identity/endpoint data. This split
supports multi-peer listeners without overloading the tunnel maddr.
The initial `open_wg_iface()` lifecycle is owned and Linux-only. It
validates role-dependent bearer policy before side effects, creates the
iface and addresses through `IPRoute`, configures keys/peers through
`WireGuard`, raises the link, and removes it on every post-creation
exit. Creation/removal run in shielded Trio worker calls. A listen
bearer supplies the local listen port; a dial bearer supplies an
omitted endpoint only for the selected maddr peer.
The composition driver folds a list of specs into nested contexts with
`contextlib.AsyncExitStack` for the N-deep case. The
`parse_endpoints()` API (`_multiaddr.py:153`) is the front door:
it already returns
`dict[name, list[Address|TunnelledAddress]]` and the
`multiaddr_declare_eps.md` sketch anticipates the recursive
`dict[str, list[Address]]|dict[...]` return for tunnelled
entries. Extend it to carry the tunnel stack, not to *enter* it.
`open_wg_bindspace()` is the initial driver for one bindspace and an
ordered sequence of `(WGTunnelSpec, WGInterfaceConfig)` layers. It
opens the bindspace first, enters WG interfaces outermost-first through
`AsyncExitStack`, and yields the live `Bindspace` for endpoint
allocation. Exit is inside-out, so every interface is removed while the
namespace FD remains pinned; only then can an owned namespace be
removed. Endpoint/channel lifetimes belong inside the yielded scope.
The caller supplies `role` to tunnel-resource contexts such as
`open_wg_iface()`; do not infer it from maddr shape. Bindspace
lifecycle remains the independent explicit policy above. The same
composed maddr can name a server source or client destination (§5.4).
### 5.3 `Address.namespace`, at last
- an unrealized `TunnelledAddress.namespace` reports its declared name
as `(kind, key)`, e.g. `('netns', 'tractor-wg0')`;
- `TunnelledAddress.with_bindspace_ref()` returns a frozen declaration
annotated with `bindspace.ref`, never the FD-bearing `Bindspace`.
Its `.namespace` reports `(kind, inode)` so the
realized ref remains stable across rename or unlink;
- existing plain backends implement it explicitly as `None`, so the
Protocol does not lie and tunnel delegation needs no `getattr()`
fallback.
- `Endpoint.namespace` reads the retained declaration rather than its
peeled transport addr; both `Endpoint.pformat()` and
`Server.pformat()` expose that value as the cheapest proof the layer
is wired.
Deferred follow-ups:
- add native tagged encoding for the complete `TunnelledAddress` graph,
including its concrete overlay-address union, tunnel-spec union and
optional `BindspaceRef`. Once that codec exists, tests should perform
typed roundtrips instead of inspecting an untyped decoded payload.
Use `github/ns_aware@e4688cad` as prototype evidence, not code to
cherry-pick unchanged. Its `/proc/<pid>/ns/<type>` inode reader and
`ip netns identify` probe establish the useful `(key, inode)` reference
record. Layer C should move that shape into `BindspaceRef`, avoid a
subprocess where netlink/procfs suffices, and hold the namespace FD in
`Bindspace` to pin the referenced resource.
### 5.4 the netns/process reality — read this before designing
**The headline consequence, stated up front**: netns is a
**runtime-level config API, not an actor-app-code API.** It is
declared as part of how an actor process is *brought up* — a
spawn-time/boot-time input alongside `enable_transports` and
`tpt_bind_addrs` — and it is **not** dynamically re-enterable by
app code once the actor is live. There is deliberately no
`await actor.enter_netns(...)`. Two hard reasons, both below:
`setns(2)` doesn't retroactively move existing sockets, and it's
per-thread rather than per-process. Anything that *looks* like a
mid-life API here would be a footgun that silently leaves the IPC
server bound in the old namespace.
- `setns(2)` with `CLONE_NEWNET` affects **the calling thread
only**, and sockets already created keep their original netns.
A trio actor is effectively single-threaded for our purposes,
so "enter the netns, *then* bind" works — but any
`to_thread` worker (§4.1 option 1!) is in the **original**
netns unless it also `setns`. Concretely: a wg query issued
via `trio.to_thread` will hit the wrong namespace. Either
pass `netns=` down to `pyroute2` (which does the
fork/setns dance itself) or pin a dedicated worker. **This is
the single subtlest bug in this plan — write the test first.**
- entering a netns is *process-global-ish and irreversible-ish*
in practice. Therefore: **netns membership belongs to the
actor process, decided before the runtime binds**, not to a
mid-life actor API. Design:
- the root/parent decides the `BindspaceSpec`, provisions or
borrows it, and passes the spec plus an inherited/transferred
namespace-FD capability through the spawn backend (there's already
`enable_transports`/`accept_addrs` plumbing at
`_runtime.py:1595-1615` — the netns rides alongside).
- the child spawn/bootstrap trampoline calls `setns()` **before**
`_runtime.async_main()`, `IPCServer.listen_on()`, parent-channel
connection, or creation of any worker thread/socket.
- `spawn._netns.enter_netns()` is the first private bootstrap
primitive: it checks the inherited FD against the expected inode,
calls `setns(fd, CLONE_NEWNET)`, and verifies
`/proc/self/ns/net` before returning. It deliberately does not own
or close the FD; spawn propagation and status reporting remain the
caller's next integration boundary.
- only after successful entry does the child drop namespace-entry
privileges and initialize the actor runtime.
- a root/single-actor process follows the same ordering: enter during
root bootstrap, never after actor runtime startup.
- iface/route/WG provisioning is genuinely scoped and remains under
the parent/supervisor's `Bindspace` context.
- document the constraint rather than hiding it; a
`RuntimeError` if namespace entry is attempted after bootstrap.
- capabilities: iface/netns creation/config needs `CAP_NET_ADMIN`;
entering an existing Linux namespace normally requires
`CAP_SYS_ADMIN` in the owning user namespace. Never `sudo` from
inside the runtime. A privileged parent/helper should provision the
stack and open the namespace FD; the child receives only the scoped
capability and temporary authority needed to enter it, then drops
that authority before actor code runs. This separates create/config
authority from enter/use authority and fits user-namespace/capability
deployments without granting every actor broad ambient caps.
Two supported modes remain:
(i) pre-provisioned out-of-band (layers A/B — the default,
and what #482 documents), (ii) runtime-managed when the supervising
process/helper holds the required caps. Probe exact required caps and
*fail loudly with an actionable message* otherwise.
- role semantics are explicit:
- `listen`: may create/own the local bindspace, iface, routes, WG
peer/listener state, and random local overlay; lifetime normally
extends through all listeners and the actor process.
- `dial`: may borrow an actor-wide bindspace or ensure local routing
and tunnel state reaches the remote stack; it does not own the
remote maddr and may need no new local resource at all.
- source/destination use is an operation property, never permanently
encoded into the maddr or inferred from segment ordering.
- teardown follows capability ownership, not just address type:
- owned listener bindspaces tear down after endpoints/channels and
the actor process have exited;
- borrowed dial/actor-wide bindspaces only release their capability;
- nested resources exit inside-out, but shared resources remain until
their owning supervisor drops the final capability.
- teardown must be idempotent and tolerant: an iface/netns
already gone must not strand the rest of the teardown — the
exact lesson `_uds.close_listener()`'s `FileNotFoundError`
tolerance and `_serve_ipc_eps()`'s per-ep `try/except`
encode. Mirror both.
### 5.5 tests for layer C
- unit: fold-N-tunnel-specs-into-nested-`@acm`s, with fakes; assert
enter/exit ordering (outermost-last-out) via a trace list.
- integration, gated on `CAP_NET_ADMIN` (skip otherwise, and in
CI run it in a `--cap-add NET_ADMIN` container job): create two
netns + a wg pair entirely in-process, boot a `tractor` root in
one and a subactor in the other, `find_actor()` across the
tunnel. This is a *fantastic* test to have and is fully
self-contained — no second host, no `sudo` in the test body.
- the `to_thread`-netns-mismatch regression from §5.4, written
**first** (red), then the fix (green), per project convention.
- bootstrap ordering: assert the child reports the expected namespace
inode before parent-channel connect and listener creation.
- FD capability: rename/unlink the namespace name after opening its FD
and prove child entry still selects the pinned inode.
- privilege drop: prove actor code lacks provisioning caps after entry.
- role/ownership: fake listen/dial resources and assert owned listener
teardown versus borrowed dial-handle release.
- `wgman` bootstrap: prove sibling process startup overlaps manager
reconciliation while the first WG operation still waits for its
readiness signal.
- `wgman` authority: reject arbitrary callers/operations and prove an
unprivileged sibling cannot receive secrets, FDs or provisioning
authority through the manager API.
- `wgman` lifetime: prove it outlives tunnel borrowers, fails pending
requests explicitly on crash and reconciles before restart-ready.
---
## 6. "Other shuttle-able tpts"
The generalization the #482 follow-up gestures at: once
`TunnelledAddress` + `open_bindspace()` exist, the same
machinery covers any iface-layer tunnel `pyroute2` can drive —
`ipip`/`gre`/`sit`/`vxlan`/`geneve`/`bridge`/`veth`. Keep
`WGTunnelSpec` as *one* frozen struct among a
`TunnelSpec = WGTunnelSpec|VxlanTunnelSpec|...` union with a
`kind: ClassVar[str]`, and dispatch `open_*` by `match` on it.
Design for it now (union + `match`), implement only `wg` +
`netns`. `veth`-pairs-in-netns is the natural second one because
it makes the §5.5 integration test possible without wg at all —
consider doing it *first* for exactly that reason.
## 7. Non-goals
- no wg userspace implementation, no key exchange, no
`wg-quick` reimplementation (config-file parsing is
out of scope; take structured input).
- no persistence of private keys beyond what layer C's iface
creation needs (and that stays in `get_rt_dir()`, 0600).
- macOS/Windows: layers B/C are Linux-only. Layer A (declarative)
works anywhere `wg` does. Gate accordingly and say so in the
docs — do not silently no-op.
## 8. Risks
| risk | mitigation |
| --- | --- |
| `to_thread` worker runs in the wrong netns | §5.4; pass `netns=` to pyroute2 or pin a worker; test-first |
| namespace name is renamed/replaced between provision and spawn | pass an open namespace FD; verify `(key, inode)` after child entry |
| child starts sockets/threads before `setns()` | enter in the spawn bootstrap trampoline before `_runtime.async_main()`; assert inode ordering |
| ambient capabilities leak into actor app code | split provision/enter authority and drop caps before runtime initialization |
| dial path tears down a shared actor bindspace | encode ownership in `Bindspace`; borrowed bindspaces never remove resources |
| py-multiaddr#108 merged but unreleased | PEP 621 direct-revision pin + `_wg_proto_code()` gate; replace with a release floor once published |
| `TunnelledAddress` leaks into transport reflection/type dispatch | keep wrappers through declaration/bindspace handling, call `strip_tunnels()` at channel/endpoint boundaries, and retain the boundary regressions |
| privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default |
| pyroute2 0.9 asyncio core drags a loop into every actor | use a worker for one-shots; confine persistent asyncio to an infected `wgman` (§4.1, §5.1) |
| eager `wgman` serializes or slows root bootstrap | spawn it in parallel; gate only WG-dependent operations on readiness; measure before making the profile unconditional |
| `wgman` becomes a cluster-wide privilege oracle | keep it private/scoped, expose structured verbs only and split managers by capability domain |
| netns teardown strands actor teardown | idempotent/tolerant teardown mirroring `_uds.close_listener()` |
## 9. Follow-up issue seeds
- `veth`-in-netns bindspace (unblocks capless-ish integration
testing, and is a great local multi-"host" test rig)
- composed/tunnelled maddr grammar shared with plan 02's
`/…/quic-v1/…` stacks (gh #443)
- `wg` proto into the multiaddr **spec** (gh #483), then flip
`MsgTransport.maddr` to always return `Multiaddr` (the third
#443 bullet)
- first-child `wgman` prototype: concurrent bootstrap, scoped sibling
access, infected-asyncio pyroute2 ownership and restart reconciliation
- runtime-managed wg key rotation / peer add-remove through `wgman`
the natural "actor that owns the network" demo

View File

@ -0,0 +1,54 @@
# next-gen `tractor.ipc` transport backend plans
Implementation specs for three prospective `.ipc` transport
backends, written so each can be worked independently (by a
different model/provider) without design or lib-selection drift.
**Read [`00_shared_backend_contract.md`](./00_shared_backend_contract.md)
first** — it is the normative description of what a `tractor`
transport backend *is* as of `main@83b34884` (the backend
duck-type, the 10-item registration checklist, the test-harness
plumbing, the code-style rules). The three plans assume it and
document only their own deltas.
| plan | issue | dep | size | lands |
| --- | --- | --- | --- | --- |
| [01 — TIPC](./01_tipc_backend.md) | [#378] | **none** (stdlib) | small | first |
| [02 — QUIC/`iroh`](./02_quic_iroh_backend.md) | [#353] | `iroh` (uniffi FFI) | large | needs a prep PR |
| [03 — `wg` bindspace](./03_wg_tunnel_bindspace.md) | [#482], [#443] | `pyroute2` | medium, 3 layers | layer A now |
Headline conclusions:
- **TIPC is the cheap win.** Verified: `trio.SocketStream` and
`trio.SocketListener` are address-family agnostic (only
`SOCK_STREAM` + a trio socket), and CPython ships `AF_TIPC` +
23 `TIPC_*` constants. So the backend is ~one module of
contract boilerplate, zero new deps, and it buys
*kernel-native* service discovery: `bind()` publishes,
`connect()`-by-name resolves — no registrar in the loop.
(`modprobe tipc` is required; hard-gate everything.)
- **QUIC's cost is entirely in two adapters**, not in QUIC. The
`iroh` python bindings are `uniffi`-generated asyncio, but the
asyncio dependency is confined to *one* future-poll callback —
a ~40-line `trio` bridge (`TrioToken.run_sync_soon`) replaces
it. The second cost is that an iroh listener isn't a socket,
which needs a small, independently-reviewable prep PR to
`_server.py`/`_types.py`.
- **WireGuard is not a transport.** It's an iface-layer tunnel,
so it belongs as a *nested bindspace* (`TunnelledAddress` +
`open_bindspace()` `@acm`s) wrapping whatever L4 tpt is in
use — which is also what finally implements the long-spec'd
`Address.namespace`, and what generalizes to
`veth`/`vxlan`/`gre`.
Ordering rationale: plan 01 first as the cheap proof the
table-registration story generalizes to a genuinely new proto;
plan 03 layer A is already deployable-today doc/example work;
plan 02 last (and gated on its prep PR). Plans 01 and 02 both
want the same `Address.rebind_from_sockname` gate — whichever
lands first ships it.
[#378]: https://github.com/goodboy/tractor/issues/378
[#353]: https://github.com/goodboy/tractor/issues/353
[#482]: https://github.com/goodboy/tractor/issues/482
[#443]: https://github.com/goodboy/tractor/issues/443

View File

@ -40,6 +40,9 @@ Broadcast fan-out
.. autoexception:: Lagged
:show-inheritance:
.. autoexception:: BroadcastReceiveError
:show-inheritance:
A single-producer, many-consumer broadcast layer over any
``trio``-style receive channel: non-lossy for the *fastest*
consumer while slower consumers raise :class:`Lagged` (a
@ -48,6 +51,13 @@ internal ring. This is exactly the machinery behind
:meth:`tractor.MsgStream.subscribe` — see
``examples/streaming_broadcast_fanout.py``.
If the shared underlying receiver raises an ordinary exception, the
subscriber which owned that receive gets the original failure.
Waiting peers drain their retained values and then raise
:class:`BroadcastReceiveError`, with the original failure available
as ``__cause__``. Later subscribers observe the same terminal state
without retrying the failed underlying receiver.
ExceptionGroup helpers
----------------------

View File

@ -209,6 +209,11 @@ The underlying broadcast machinery is lazily allocated on first
use and is *not* reversible for the channel's remaining lifetime,
so only reach for it when you actually want the fan-out.
As with ``MsgStream``, pass ``raise_on_lag=False`` for a consumer
which may warn, drop old values and resume from the retained window.
Each child chooses independently; the first subscription also fixes
the linked channel's root receive policy.
One-shot calls with ``run_task()``
----------------------------------
When you just want a single ``asyncio`` result and no streaming

View File

@ -171,6 +171,20 @@ keeps pace with the *fastest* subscriber; a task falling more
than the buffered window behind has its next receive raise
``tractor.trionics.Lagged`` to say it lost data.
Each ``BroadcastReceiver`` is one logical subscription cursor, so
give every concurrent consumer task its own receiver. Overlapping
``receive()`` calls on the same handle raise
``trio.BusyResourceError`` instead of racing that cursor. In strict
mode values are never skipped silently: the consumer either reads
each retained value in sequence or receives an explicit ``Lagged``
error after exceeding the buffer window.
Pass ``raise_on_lag=False`` when a consumer may drop old values and
resume from the oldest retained item instead. The receiver logs the
overrun rather than raising. Each child subscription chooses its own
policy; the first call also fixes the policy of the stream's root
receive handle because broadcaster allocation is irreversible.
The broadcast handle stays duplex btw: it proxies ``send()``
through to the underlying stream, so each subscriber task can
keep talking upstream while consuming its fan-out copy.

View File

@ -0,0 +1,172 @@
# `tractor` over a WireGuard tunnel, declared as one maddr
A two-host LAN setup: a `tractor` actor tree on host A, dialed
from host B, with the endpoint declared as a single `wg`
multiaddr.
Supersedes the example set in gh
[#482](https://github.com/goodboy/tractor/issues/482) — see
[what changed](#what-changed-vs-482).
> **Why `examples/multihost/`?** `tests/test_docs_examples.py`
> walks `examples/` recursively and runs everything it collects
> as a subproc, asserting `rc == 0`. These need a real second
> host and a live `wg` tunnel, so they can't satisfy that;
> `'multihost' not in p[0]` is already in the test's exclusion
> list, which is what keeps them out of CI.
## the maddr form
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\____ wg bearer ___________/\__ key __/\____ tractor ep _____/
underlay, wg `ListenPort` overlay, on the wg iface
(kernel owns the socket) (`MsgTransport` binds this)
```
Three parts, three different owners:
| part | socket owner / provisioner | runtime role |
| --- | --- | --- |
| `/ip4/../udp/51820` bearer | kernel-owned; `wg-quick` now, tractor bindspace later | control-plane metadata |
| `/wg/u<key>` | nothing — it's an identity | parsed, verified explicitly |
| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | application `MsgTransport` |
Verified against py-multiaddr
[#108](https://github.com/multiformats/py-multiaddr/pull/108):
this composed form parses and round-trips
(`['ip4','udp','wg','ip4','tcp']`).
## requirements
py-multiaddr #108 is **merged** (2026-07-28) but ships in no
release yet — the latest `0.2.0` (2026-03-17) predates it and has
no `wg` codec. So `pyproject.toml` temporarily pins the merge commit
in its PEP 621 dependency metadata, and a plain
```bash
uv sync --extra wg
```
gets you a `wg`-aware `multiaddr` plus pyroute2's Linux netlink API.
The multiaddr pin goes away once a release carries the codec.
`py-multibase` is a direct dependency.
Without the codec `parse_wg_maddr()` raises immediately with an
actionable message — there is deliberately **no** degraded
hand-split fallback. `_wg_proto_code()` performs the capability
check before parsing.
Every peel and re-compose here goes through `py-multiaddr`'s own
tunnel API (`.decapsulate_code()`, `.split()`, `.join()`,
`.encapsulate()`, `.value_for_protocol()`) rather than any
bespoke segment slicing — see its README "En/decapsulate" and
"Tunneling" sections. gh #429 was about *dropping* our NIH
parser, and that applies to peeling a tunnel stack just as much
as to decoding one proto.
## 0. tunnel setup (out-of-band, both hosts)
Host A is the service host (underlay e.g. `192.168.1.50`), host B
your workstation. Overlay net `10.0.11.0/24`.
```bash
umask 077
wg genkey | tee wg_priv.key | wg pubkey > wg_pub.key
```
`/etc/wireguard/wg0.conf` on **host A**:
```ini
[Interface]
PrivateKey = <A_priv>
Address = 10.0.11.1/24
ListenPort = 51820
```
```ini
[Peer]
PublicKey = <B_pub>
AllowedIPs = 10.0.11.2/32
```
on **host B**:
```ini
[Interface]
PrivateKey = <B_priv>
Address = 10.0.11.2/24
```
```ini
[Peer]
PublicKey = <A_pub>
Endpoint = 192.168.1.50:51820
AllowedIPs = 10.0.11.1/32
PersistentKeepalive = 25
```
Note how `ListenPort` and `Endpoint` are exactly the maddr's
bearer segment, and `[Interface] Address` is its overlay host.
```bash
sudo wg-quick up wg0 # both hosts
ping -c1 10.0.11.1 # from B
```
## 1. get your pubkey into the maddr
```bash
python -c "
from tractor.discovery import mb_pubkey
key = open('wg_pub.key').read().strip()
print(mb_pubkey(key))
"
```
Paste the `u...` output into `WG_MADDR` in both scripts (they use
the same string — A's bearer, A's key, A's overlay ep).
## 2. run
```bash
# host A
python host_a_srv.py
# host B
python host_b_client.py
```
`host_a_srv.py` must be importable on host B too, since
`portal.run()` refs the fn by module path — standard `tractor`
RPC semantics.
## what changed vs #482
Four corrections, all from
`ai/tpt-backends/03_wg_tunnel_bindspace.md`:
1. **the maddr semantics were inverted.** #482 used
`/ip4/10.0.11.1/tcp/1616/wg/u<key>` — that parses, but it puts
the *overlay* addr where the bearer belongs and `tcp` where
wg's `udp` `ListenPort` goes, and it declares no overlay ep at
all. `parse_wg_maddr()` now rejects it with an actionable
error.
2. **parsing is pure.** #482's helper had the key-check adjacent
to the parse; async `verify_wg_peer()` is now a separate,
explicitly composed step that the caller invokes. Implicit
kernel inspection from a parser is a nasty surprise.
3. **no `sudo` or subprocess.** #482 ran `sudo wg show`; tractor's
helper reads generic netlink through pyroute2 and never attempts
privilege escalation or namespace creation.
4. **no new `Address` proto-type.** The tunnel rides *beside* the
overlay addr in a frozen `TunnelledAddress`, and only `.overlay`
crosses into `open_nursery()`. #482 §6 floated a `WGAddress`
registered in `_address_types` — that table is a `bidict`
(1:1 proto-key↔type) and `_addr_to_transport` wants a
`MsgTransport` per addr-type, which `wg` doesn't have.
## next
Layer A's `TunnelledAddress` and native maddr parser plus Layer B's
explicit pyroute2 verification now live in `tractor.discovery`. Next,
add `open_bindspace()` `@acm`s which create/tear down the iface and
netns.

View File

@ -0,0 +1,61 @@
# tractor: distributed structured concurrency.
'''
Host A: the service host, reachable over a `wg` tunnel.
Binds `tractor`'s registrar + an `echo_srv` sub-actor on the
tunnel's *overlay* addr, declared as a single `wg` maddr.
'''
from __future__ import annotations
import tractor
import trio
from tractor.discovery import (
TunnelledAddress,
mk_maddr,
parse_wg_maddr,
verify_wg_peer,
)
# bearer = host A's underlay `(ip, wg ListenPort)`
# key = host A's OWN tunnel pubkey
# overlay = the ep `tractor` binds, on the wg iface's addr
WG_MADDR: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/u<A_pub_b64url>'
'/ip4/10.0.11.1/tcp/1616'
)
async def echo(msg: str) -> str:
actor = tractor.current_actor()
return f'{actor.aid.name!r} echoes: {msg}'
async def main():
addr: TunnelledAddress = parse_wg_maddr(WG_MADDR)
assert await verify_wg_peer(addr.tunnel), (
f'wg pubkey from maddr not active on wg0 !\n'
f'maddr: {WG_MADDR}\n'
f'key: {addr.tunnel.peer_pubkey}\n'
)
print(
f'wg bearer (kernel-owned): {addr.tunnel.bearer}\n'
f'tractor overlay ep: {addr.overlay}\n'
)
async with tractor.open_nursery(
# XXX only `.overlay` crosses into the runtime; the bearer
# + key are bindspace metadata, never `Endpoint` addrs.
registry_addrs=[addr.overlay],
enable_transports=[addr.overlay.proto_key],
) as an:
await an.start_actor(
'echo_srv',
enable_modules=[__name__],
)
print(f'echo_srv up on\n {mk_maddr(addr)}\n')
await trio.sleep_forever()
if __name__ == '__main__':
trio.run(main)

View File

@ -0,0 +1,51 @@
# tractor: distributed structured concurrency.
'''
Host B: workstation dialing host A's actor tree through the
`wg` tunnel.
'''
from __future__ import annotations
import tractor
import trio
from tractor.discovery import (
TunnelledAddress,
parse_wg_maddr,
verify_wg_peer,
)
from host_a_srv import echo # noqa: F401 (RPC refs it by mod path)
# same maddr as host A: A's bearer, A's key, A's overlay ep
WG_MADDR: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/u<A_pub_b64url>'
'/ip4/10.0.11.1/tcp/1616'
)
async def main():
addr: TunnelledAddress = parse_wg_maddr(WG_MADDR)
assert await verify_wg_peer(addr.tunnel), (
f'wg pubkey from maddr not a peer on wg0 !\n'
f'maddr: {WG_MADDR}\n'
)
async with (
tractor.open_root_actor(
name='wg_client',
registry_addrs=[addr.overlay],
enable_transports=[addr.overlay.proto_key],
),
tractor.find_actor(
'echo_srv',
registry_addrs=[addr.overlay],
) as portal,
):
res: str = await portal.run(
echo,
msg='hello over wg!',
)
print(res)
if __name__ == '__main__':
trio.run(main)

View File

@ -48,7 +48,10 @@ dependencies = [
# typed IPC msging
"msgspec>=0.20.0",
"bidict>=0.23.1",
"multiaddr>=0.2.0",
# unreleased `/wg/` codec from py-multiaddr#108
"multiaddr @ git+https://github.com/multiformats/py-multiaddr.git@f86519daaa21699023d0037c58cdff600313dd09",
# encode/decode `wg` pubkeys carried by multiaddrs
"py-multibase>=2.0.0,<3",
"platformdirs>=4.4.0",
# per-actor `argv[0]` proc-title for OS-level diag tools
# (`ps`, `top`, `psutil`-backed tooling like `acli.pytree`).
@ -59,6 +62,12 @@ dependencies = [
"setproctitle>=1.3,<2",
]
[project.optional-dependencies]
wg = [
# read/provision Linux WireGuard state through netlink
"pyroute2>=0.9.6,<0.10 ; sys_platform == 'linux'",
]
# ------ project ------
[dependency-groups]
@ -165,9 +174,8 @@ sync_pause = {requires-python = ">=3.13, <3.14"}
# editable = true
# ------ tool.uv.sources ------
# TODO, distributed (multi-host) extensions
# linux kernel networking
# 'pyroute2
# Linux kernel networking is provided by the optional `wg` extra.
# Add any temporary `pyroute2` source overrides here.
# ------ tool.uv.sources ------
@ -183,6 +191,9 @@ python-preference = 'system'
# ------ tool.uv ------
[tool.hatch.metadata]
allow-direct-references = true
[tool.hatch.build.targets.sdist]
include = ["tractor"]

View File

@ -0,0 +1,100 @@
'''
Canonical tagged-address decoding and legacy input compatibility.
'''
from pathlib import Path
import pytest
from tractor.discovery._addr import wrap_address
from tractor.ipc._tcp import TCPAddress
from tractor.ipc._uds import UDSAddress
@pytest.mark.parametrize(
'value',
[
('tcp', '127.0.0.1', 1616),
['tcp', '127.0.0.1', 1616],
],
)
def test_decode_tagged_tcp_address(value):
'''
Shape-only decoding cannot distinguish future transport address
forms. Feed canonical tuple and msgpack-style list values through
the compatibility boundary and prove the explicit `tcp` tag
selects the TCP backend and emits the canonical tagged form.
'''
addr = wrap_address(value)
assert type(addr) is TCPAddress
assert addr.unwrap() == ('tcp', '127.0.0.1', 1616)
@pytest.mark.parametrize(
'tag',
['unix', 'uds'],
)
@pytest.mark.parametrize('container', [tuple, list])
def test_decode_tagged_unix_address(
tag: str,
container: type,
):
'''
Multiaddr calls the protocol `unix` while tractor's transport key
remains `uds`. Decode both spellings from tuple/list containers,
normalize them to one `UDSAddress`, and emit the canonical `unix`
spelling.
'''
value = container((tag, '/tmp/tractor/registry.sock'))
addr = wrap_address(value)
assert type(addr) is UDSAddress
assert addr.sockpath == Path('/tmp/tractor/registry.sock')
assert addr.unwrap() == (
'unix',
'/tmp/tractor/registry.sock',
)
@pytest.mark.parametrize(
'value, expected_type',
[
(('127.0.0.1', 1616), TCPAddress),
(['127.0.0.1', 1616], TCPAddress),
(('/tmp/tractor', 'registry.sock'), UDSAddress),
(['/tmp/tractor', 'registry.sock'], UDSAddress),
],
)
def test_decode_legacy_address_forms(
value,
expected_type: type,
):
'''
Existing callers, config, and older msgpack payloads still
provide untagged pairs. Keep tuple/list forms readable while
canonical tagged emission is introduced, proving the writer
migration does not break shipped input behavior.
'''
addr = wrap_address(value)
assert type(addr) is expected_type
assert addr.unwrap()[0] in {'tcp', 'unix'}
def test_tcp_from_native_ipv6_sockname():
'''
`socket.getsockname()` returns a four-item IPv6 sockaddr which is
neither a wire form nor a legacy two-item pair. Preserve it as an
OS compatibility boundary and intentionally ignore unsupported
flow-info/scope-id fields when constructing `TCPAddress`.
'''
addr = TCPAddress.from_addr(
('::1', 1616, 0, 0)
)
assert addr.unwrap() == ('tcp', '::1', 1616)

View File

@ -0,0 +1,568 @@
'''
Bindspace declaration, reference and live-capability contracts.
'''
from __future__ import annotations
from pathlib import Path
import os
import sys
from typing import BinaryIO
import msgspec
import pytest
import trio
from tractor.discovery import (
Bindspace,
BindspaceOwnership,
BindspaceRef,
BindspaceSpec,
CURRENT_NETNS,
attach_netns,
open_bindspace,
open_netns,
)
from tractor.discovery import _bindspace
from tractor.msg import ProcessLocal
def test_bindspace_declarations_roundtrip() -> None:
'''
Spawn configuration and realized refs must cross actor IPC.
Encode both frozen structs through msgpack and decode with their
concrete types, proving names and stable inode refs survive
without carrying any process-local capability state.
'''
values: tuple[
BindspaceSpec|BindspaceRef,
...,
] = (
BindspaceSpec(
kind='netns',
key='tractor-wg0',
lifecycle='open',
),
BindspaceRef(
kind='netns',
key='tractor-wg0',
inode=1234,
),
)
value: BindspaceSpec|BindspaceRef
for value in values:
encoded: bytes = msgspec.msgpack.encode(value)
decoded: BindspaceSpec|BindspaceRef = (
msgspec.msgpack.decode(
encoded,
type=type(value),
)
)
assert decoded == value
def test_bindspace_pins_local_capability(
tmp_path: Path,
) -> None:
'''
A live bindspace pins one exact FD and realized ref.
Open a stand-in platform FD, record its inode in the realized
ref and construct an owned capability. Prove the generic
msgspec struct retains that exact local state. Its ability to
encode ordinary fields is not authority to transfer the
bindspace.
'''
token_path: Path = tmp_path / 'bindspace'
token_path.touch()
namespace_file: BinaryIO
with token_path.open('rb') as namespace_file:
namespace_fd: int = namespace_file.fileno()
inode: int = token_path.stat().st_ino
spec: BindspaceSpec = BindspaceSpec(
kind='netns',
key='tractor-wg0',
lifecycle='open',
)
ref: BindspaceRef = BindspaceRef(
kind='netns',
key='tractor-wg0',
inode=inode,
)
bindspace: Bindspace = Bindspace(
spec=spec,
ref=ref,
namespace_fd=namespace_fd,
ownership='owned',
)
assert bindspace.spec is spec
assert bindspace.ref is ref
assert bindspace.namespace_fd == namespace_file.fileno()
assert bindspace.ownership == 'owned'
assert isinstance(bindspace, msgspec.Struct)
assert isinstance(bindspace, ProcessLocal)
with pytest.raises(
TypeError,
match='_ProcessLocalToken.*unsupported',
):
msgspec.msgpack.encode(bindspace)
def test_bindspace_rejects_mismatched_ref(
tmp_path: Path,
) -> None:
'''
A name or inode mismatch would make a bindspace stale authority.
Construct a requested named spec, then prove both a different
realized name and an inode not belonging to the supplied FD are
rejected before either can become a live capability.
'''
token_path: Path = tmp_path / 'bindspace'
token_path.touch()
spec: BindspaceSpec = BindspaceSpec(
kind='netns',
key='tractor-wg0',
)
# Keep ownership and FD fixed so only the ref changes below.
ownership: BindspaceOwnership = 'borrowed'
namespace_file: BinaryIO
with token_path.open('rb') as namespace_file:
namespace_fd: int = namespace_file.fileno()
wrong_name: BindspaceRef = BindspaceRef(
kind='netns',
key='other-wg',
inode=token_path.stat().st_ino,
)
with pytest.raises(
ValueError,
match='Spec.key.*Ref.key',
):
Bindspace(
spec=spec,
ref=wrong_name,
namespace_fd=namespace_fd,
ownership=ownership,
)
wrong_inode: BindspaceRef = BindspaceRef(
kind='netns',
key='tractor-wg0',
inode=token_path.stat().st_ino + 1,
)
with pytest.raises(
ValueError,
match='FD inode.*reference inode',
):
Bindspace(
spec=spec,
ref=wrong_inode,
namespace_fd=namespace_fd,
ownership=ownership,
)
@pytest.mark.parametrize(
('model', 'kwargs', 'match'),
(
pytest.param(
BindspaceRef,
{
'kind': 'netns',
'key': None,
'inode': None,
},
'must be a positive `int`',
id='ref-requires-inode',
),
pytest.param(
BindspaceSpec,
{'kind': 'vrf'},
'Unsupported bindspace kind',
id='spec-rejects-kind',
),
pytest.param(
BindspaceRef,
{
'kind': 'vrf',
'key': 'blue',
'inode': 1234,
},
'Unsupported bindspace kind',
id='ref-rejects-kind',
),
pytest.param(
BindspaceSpec,
{
'kind': 'netns',
'key': '../outside',
},
'Invalid netns name',
id='spec-rejects-path',
),
pytest.param(
BindspaceSpec,
{
'kind': 'netns',
'key': '',
},
'BindspaceSpec.key',
id='spec-rejects-empty-key',
),
pytest.param(
BindspaceSpec,
{
'kind': 'netns',
'key': 'tractor-wg0',
'lifecycle': 'replace',
},
'Unsupported bindspace lifecycle',
id='spec-rejects-lifecycle',
),
pytest.param(
BindspaceRef,
{
'kind': 'netns',
'key': '',
'inode': 1234,
},
'BindspaceRef.key',
id='ref-rejects-empty-key',
),
),
)
def test_bindspace_models_reject_invalid_values(
model: type[BindspaceSpec]|type[BindspaceRef],
kwargs: dict[str, object],
match: str,
) -> None:
'''
Direct msgspec construction does not enforce field annotations.
Parameterize the missing stable inode and future, unimplemented
kinds. Prove neither serializable model can carry invalid
refs or provisioning instructions into spawn configuration.
'''
with pytest.raises(ValueError, match=match):
model(**kwargs) # type: ignore[arg-type]
@pytest.mark.skipif(
sys.platform != 'linux',
reason='Linux netns API',
)
def test_open_bindspace_attaches_current_netns() -> None:
'''
The unnamed spec must borrow and pin the caller's current netns.
Open `/proc/self/ns/net`, prove the yielded bindspace records its
stable inode and borrowed ownership, then exit the context and
prove the exact descriptor was closed without altering the
namespace itself.
'''
async def main() -> int:
'''
Borrow the current netns and return its descriptor number.
'''
spec: BindspaceSpec = BindspaceSpec(
kind='netns',
)
assert spec.key is CURRENT_NETNS
async with open_bindspace(spec) as bindspace:
namespace_fd: int|None = bindspace.namespace_fd
assert namespace_fd is not None
assert bindspace.spec is spec
assert bindspace.ref.key is None
assert bindspace.ref.inode == os.fstat(
namespace_fd
).st_ino
assert bindspace.ownership == 'borrowed'
return namespace_fd
namespace_fd: int = trio.run(main)
with pytest.raises(OSError):
os.fstat(namespace_fd)
@pytest.mark.skipif(
sys.platform != 'linux',
reason='Linux netns API',
)
def test_attach_named_netns_uses_run_directory(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
'''
A named spec must resolve only beneath the configured netns dir.
Replace the run directory with a temporary stand-in, borrow its
named inode, and prove the context neither deletes the existing
resource nor leaves its descriptor open after exit.
'''
netns_path: Path = tmp_path / 'tractor-wg0'
netns_path.touch()
monkeypatch.setattr(
_bindspace,
'_NETNS_RUN_DIR',
tmp_path,
)
async def main() -> int:
'''
Borrow the named stand-in and return its descriptor number.
'''
spec: BindspaceSpec = BindspaceSpec(
kind='netns',
key='tractor-wg0',
)
async with attach_netns(spec) as bindspace:
namespace_fd: int|None = bindspace.namespace_fd
assert namespace_fd is not None
assert bindspace.ref.key == 'tractor-wg0'
assert bindspace.ref.inode == netns_path.stat().st_ino
assert bindspace.ownership == 'borrowed'
return namespace_fd
namespace_fd: int = trio.run(main)
assert netns_path.exists()
with pytest.raises(OSError):
os.fstat(namespace_fd)
@pytest.mark.skipif(
sys.platform != 'linux',
reason='Linux netns API',
)
def test_attach_named_netns_never_creates(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
'''
Borrow-only lookup must fail without creating a missing resource.
Point the run directory at an empty location, request one named
netns, and prove the open error propagates while no path appears.
'''
monkeypatch.setattr(
_bindspace,
'_NETNS_RUN_DIR',
tmp_path,
)
missing_path: Path = tmp_path / 'missing'
async def main() -> None:
'''
Attempt to borrow one absent named netns.
'''
spec: BindspaceSpec = BindspaceSpec(
kind='netns',
key='missing',
)
async with attach_netns(spec):
raise AssertionError('Missing netns unexpectedly opened')
with pytest.raises(FileNotFoundError):
trio.run(main)
assert not missing_path.exists()
@pytest.mark.skipif(
sys.platform != 'linux',
reason='Linux netns API',
)
def test_open_netns_owns_lifecycle(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
'''
Successful creation must yield ownership and remove on exit.
Fake pyroute2 creation with a named stand-in file, verify the
yielded FD and ref while it exists, then prove FD closure
precedes resource removal when the context exits.
'''
events: list[str] = []
namespace_fds: list[int] = []
netns_path: Path = tmp_path / 'tractor-wg0'
def create(key: str) -> None:
'''
Create the named stand-in and record lifecycle order.
'''
assert key == 'tractor-wg0'
netns_path.touch()
events.append('create')
def remove(key: str) -> None:
'''
Remove the stand-in after its FD has closed.
'''
assert key == 'tractor-wg0'
events.append('fd-closed')
with pytest.raises(OSError):
os.fstat(namespace_fds[0])
netns_path.unlink()
events.append('remove')
monkeypatch.setattr(
_bindspace,
'_NETNS_RUN_DIR',
tmp_path,
)
monkeypatch.setattr(
_bindspace,
'_create_netns',
create,
)
monkeypatch.setattr(
_bindspace,
'_remove_netns',
remove,
)
async def main() -> None:
'''
Open the fake netns and publish its live descriptor number.
'''
spec: BindspaceSpec = BindspaceSpec(
kind='netns',
key='tractor-wg0',
lifecycle='open',
)
async with open_bindspace(spec) as bindspace:
fd: int|None = bindspace.namespace_fd
assert fd is not None
assert bindspace.ownership == 'owned'
assert bindspace.ref.inode == os.fstat(fd).st_ino
namespace_fds.append(fd)
events.append('yield')
trio.run(main)
assert events == [
'create',
'yield',
'fd-closed',
'remove',
]
assert not netns_path.exists()
@pytest.mark.skipif(
sys.platform != 'linux',
reason='Linux netns API',
)
def test_open_netns_shields_cancelled_cleanup(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
'''
Cancellation after creation must not leak an owned namespace.
Cancel the caller inside the yielded context and checkpoint.
Prove shielded teardown still removes the stand-in before
cancellation leaves the enclosing scope.
'''
netns_path: Path = tmp_path / 'tractor-wg0'
removed: list[str] = []
def create(key: str) -> None:
'''
Create the named stand-in before cancellation.
'''
netns_path.touch()
def remove(key: str) -> None:
'''
Remove the stand-in despite caller cancellation.
'''
netns_path.unlink()
removed.append(key)
monkeypatch.setattr(
_bindspace,
'_NETNS_RUN_DIR',
tmp_path,
)
monkeypatch.setattr(
_bindspace,
'_create_netns',
create,
)
monkeypatch.setattr(
_bindspace,
'_remove_netns',
remove,
)
async def main() -> None:
'''
Cancel while borrowing the newly owned namespace.
'''
spec: BindspaceSpec = BindspaceSpec(
kind='netns',
key='tractor-wg0',
lifecycle='open',
)
with trio.CancelScope() as scope:
async with open_netns(spec):
scope.cancel()
await trio.sleep_forever()
trio.run(main)
assert removed == ['tractor-wg0']
assert not netns_path.exists()
@pytest.mark.skipif(
sys.platform != 'linux',
reason='Linux netns API',
)
def test_open_netns_requires_name() -> None:
'''
Creation cannot target the caller's current netns.
Pass `CURRENT_NETNS` and prove validation rejects it before any
privileged pyroute2 operation can run.
'''
spec: BindspaceSpec = BindspaceSpec(
kind='netns',
key=CURRENT_NETNS,
lifecycle='open',
)
async def main() -> None:
'''
Attempt to create the unnamed current namespace.
'''
async with open_netns(spec):
raise AssertionError(
'Current netns unexpectedly created'
)
with pytest.raises(
ValueError,
match='requires a named',
):
trio.run(main)

View File

@ -10,6 +10,14 @@ from types import SimpleNamespace
import pytest
from multiaddr import Multiaddr
from tractor.discovery import (
TunnelledAddress,
WGTunnelSpec,
mb_pubkey,
mk_wg_maddr,
parse_wg_maddr,
tunnels_of,
)
from tractor.ipc._tcp import TCPAddress
from tractor.ipc._uds import UDSAddress
from tractor.discovery._multiaddr import (
@ -22,6 +30,19 @@ from tractor.discovery._multiaddr import (
from tractor.discovery._addr import wrap_address
_WG_PUBKEY: str = (
'g3x7z0AdV1rM6UQU22CC7IL3/ivn4DzrE7ikDhCZ/Dc='
)
_WG_PUBKEY_2: str = (
'AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8='
)
_WG_MADDR: str = (
f'/ip4/192.168.1.50/udp/51820'
f'/wg/{mb_pubkey(_WG_PUBKEY)}'
f'/ip4/10.0.11.1/tcp/1616'
)
def test_tpt_proto_to_maddr_mapping():
'''
`_tpt_proto_to_maddr` maps all supported `proto_key`
@ -167,7 +188,7 @@ def test_parse_maddr_tcp_ipv4():
result = parse_maddr('/ip4/127.0.0.1/tcp/1234')
assert isinstance(result, TCPAddress)
assert result.unwrap() == ('127.0.0.1', 1234)
assert result.unwrap() == ('tcp', '127.0.0.1', 1234)
def test_parse_maddr_tcp_ipv6():
@ -179,7 +200,7 @@ def test_parse_maddr_tcp_ipv6():
result = parse_maddr('/ip6/::1/tcp/5678')
assert isinstance(result, TCPAddress)
assert result.unwrap() == ('::1', 5678)
assert result.unwrap() == ('tcp', '::1', 5678)
def test_parse_maddr_uds():
@ -192,9 +213,10 @@ def test_parse_maddr_uds():
result = parse_maddr('/unix/tmp/tractor_test/test.sock')
assert isinstance(result, UDSAddress)
filedir, filename = result.unwrap()
assert filename == 'test.sock'
assert str(filedir) == '/tmp/tractor_test'
assert result.unwrap() == (
'unix',
'/tmp/tractor_test/test.sock',
)
def test_parse_maddr_unsupported():
@ -210,6 +232,181 @@ def test_parse_maddr_unsupported():
parse_maddr('/ip4/127.0.0.1/udp/1234')
def test_parse_wg_maddr():
'''
`parse_maddr()` previously rejected the canonical infix `/wg/`
grammar even though `py-multiaddr` parsed it. Feed a bearer,
identity, and TCP overlay through both the WG-specific and public
parsers, then prove they produce the same local-only tunnel
annotation without changing the bindable overlay.
'''
parsed = parse_wg_maddr(_WG_MADDR)
assert parse_maddr(_WG_MADDR) == parsed
assert isinstance(parsed, TunnelledAddress)
assert parsed.tunnel == WGTunnelSpec(
peer_pubkey=_WG_PUBKEY,
bearer=('192.168.1.50', 51820),
)
assert isinstance(parsed.overlay, TCPAddress)
assert parsed.overlay.unwrap() == ('tcp', '10.0.11.1', 1616)
def test_mk_wg_maddr_roundtrip():
'''
`mk_maddr()` previously saw only the wrapper's delegated TCP
proto-key and silently dropped all tunnel metadata. Parse the
canonical maddr, compose it through both public entry points, and
prove bearer, key, and overlay survive byte-for-byte.
'''
parsed = parse_wg_maddr(_WG_MADDR)
assert str(mk_wg_maddr(parsed)) == _WG_MADDR
assert str(mk_maddr(parsed)) == _WG_MADDR
def test_nested_wg_maddr_roundtrip():
'''
A single first-match lookup confuses nested WG keys and bearers.
Arrange an IPv4 outer bearer around an IPv6 inner bearer, parse
from the last `/wg/` outward, and assert tunnel ordering plus an
exact re-composition of the original stack.
'''
nested_maddr: str = (
f'/ip4/192.168.1.50/udp/51820'
f'/wg/{mb_pubkey(_WG_PUBKEY)}'
f'/ip6/2001:db8::2/udp/51821'
f'/wg/{mb_pubkey(_WG_PUBKEY_2)}'
f'/ip4/10.0.11.1/tcp/1616'
)
parsed = parse_maddr(nested_maddr)
specs = tunnels_of(parsed)
assert len(specs) == 2
assert specs[0].peer_pubkey == _WG_PUBKEY
assert specs[0].bearer == ('192.168.1.50', 51820)
assert specs[1].peer_pubkey == _WG_PUBKEY_2
assert specs[1].bearer == ('2001:db8::2', 51821)
assert str(mk_maddr(parsed)) == nested_maddr
@pytest.mark.parametrize(
'maddr, match',
[
pytest.param(
(
f'/ip4/192.168.1.50/tcp/51820'
f'/wg/{mb_pubkey(_WG_PUBKEY)}'
f'/ip4/10.0.11.1/tcp/1616'
),
'Bad `wg` bearer',
id='non-udp-bearer',
),
pytest.param(
(
f'/ip4/192.168.1.50/udp/51820'
f'/wg/{mb_pubkey(_WG_PUBKEY)}'
),
'no overlay endpoint',
id='missing-overlay',
),
pytest.param(
(
f'/ip4/192.168.1.50/udp/51820'
f'/wg/{mb_pubkey(_WG_PUBKEY)}'
f'/ip4/10.0.11.1/udp/1616'
),
'Unsupported `wg` overlay',
id='non-tcp-overlay',
),
],
)
def test_parse_wg_maddr_rejects_bad_grammar(
maddr: str,
match: str,
):
'''
Accepting an invalid bearer or overlay assigns an endpoint to the
wrong runtime owner. Exercise parseable but unsupported protocol
combinations and prove each fails before constructing a wrapper,
with an error identifying the violated WG grammar boundary.
'''
with pytest.raises(ValueError, match=match):
parse_maddr(maddr)
def test_parse_wg_maddr_rejects_malformed_key():
'''
A truncated multibase key used to be vulnerable to silent
identity corruption in hand-written parsers. Give the upstream
`/wg/` codec a short key and prove `Multiaddr()` rejects it
before tractor's wrapper parser runs.
'''
maddr: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/udG9vIHNob3J0'
'/ip4/10.0.11.1/tcp/1616'
)
with pytest.raises(ValueError):
parse_maddr(maddr)
def test_parse_wg_maddr_reports_missing_codec(
monkeypatch,
):
'''
Released `multiaddr==0.2.0` does not know `/wg/` and emits an
opaque unknown-protocol parse error. Simulate that registry and
prove an actual WG stack reports the dependency action while a
Unix path containing a `wg` directory remains ordinary UDS data.
'''
from multiaddr.exceptions import ProtocolNotFoundError
from multiaddr import protocols
def no_wg_proto(name: str):
raise ProtocolNotFoundError(name)
monkeypatch.setattr(
protocols,
'protocol_with_name',
no_wg_proto,
)
uds = parse_maddr('/unix/tmp/wg/service.sock')
assert isinstance(uds, UDSAddress)
with pytest.raises(
RuntimeError,
match='py-multiaddr#108',
):
parse_maddr(_WG_MADDR)
def test_mk_wg_maddr_requires_bearer():
'''
A key-only tunnel spec relies on local configuration and cannot
be reconstructed as the canonical bearer-first maddr. Build that
incomplete annotation and prove composition raises instead of
emitting a misleading overlay-only address.
'''
addr = TunnelledAddress(
overlay=TCPAddress('10.0.11.1', 1616),
tunnel=WGTunnelSpec(peer_pubkey=_WG_PUBKEY),
)
with pytest.raises(
ValueError,
match='without a bearer',
):
mk_maddr(addr)
@pytest.mark.parametrize(
'addr',
[
@ -249,7 +446,22 @@ def test_wrap_address_maddr_str():
result = wrap_address('/ip4/127.0.0.1/tcp/9999')
assert isinstance(result, TCPAddress)
assert result.unwrap() == ('127.0.0.1', 9999)
assert result.unwrap() == ('tcp', '127.0.0.1', 9999)
def test_wrap_address_wg_maddr_str():
'''
`wrap_address()` delegates slash-prefixed strings to
`parse_maddr()`. Pass a canonical WG maddr through that public
boundary and prove it preserves the tunnel annotation rather than
rejecting the protocol stack or returning only its TCP overlay.
'''
result = wrap_address(_WG_MADDR)
assert isinstance(result, TunnelledAddress)
assert result.tunnel.peer_pubkey == _WG_PUBKEY
assert result.overlay.unwrap() == ('tcp', '10.0.11.1', 1616)
# ------ parse_endpoints() tests ------
@ -270,11 +482,11 @@ def test_parse_endpoints_tcp_only():
reg_addr = result['registry'][0]
assert isinstance(reg_addr, TCPAddress)
assert reg_addr.unwrap() == ('127.0.0.1', 1616)
assert reg_addr.unwrap() == ('tcp', '127.0.0.1', 1616)
feed_addr = result['data_feed'][0]
assert isinstance(feed_addr, TCPAddress)
assert feed_addr.unwrap() == ('0.0.0.0', 5555)
assert feed_addr.unwrap() == ('tcp', '0.0.0.0', 5555)
def test_parse_endpoints_mixed_tpts():
@ -294,12 +506,37 @@ def test_parse_endpoints_mixed_tpts():
assert len(addrs) == 2
assert isinstance(addrs[0], TCPAddress)
assert addrs[0].unwrap() == ('127.0.0.1', 4040)
assert addrs[0].unwrap() == ('tcp', '127.0.0.1', 4040)
assert isinstance(addrs[1], UDSAddress)
filedir, filename = addrs[1].unwrap()
assert filename == 'broker.sock'
assert str(filedir) == '/tmp/tractor'
assert addrs[1].unwrap() == (
'unix',
'/tmp/tractor/broker.sock',
)
def test_parse_endpoints_wg_maddr():
'''
Service endpoint tables previously rejected WG protocol stacks.
Put a tunnelled maddr beside a plain TCP address and prove
`parse_endpoints()` retains input order while delivering the
wrapper needed by the future bindspace lifecycle.
'''
table = {
'registry': [
_WG_MADDR,
'/ip4/127.0.0.1/tcp/1616',
],
}
addrs = parse_endpoints(table)['registry']
assert isinstance(addrs[0], TunnelledAddress)
assert addrs[0].tunnel.bearer == (
'192.168.1.50',
51820,
)
assert isinstance(addrs[1], TCPAddress)
def test_parse_endpoints_unwrapped_tuples():
@ -315,7 +552,7 @@ def test_parse_endpoints_unwrapped_tuples():
addr = result['ems'][0]
assert isinstance(addr, TCPAddress)
assert addr.unwrap() == ('127.0.0.1', 6666)
assert addr.unwrap() == ('tcp', '127.0.0.1', 6666)
def test_parse_endpoints_mixed_str_and_tuple():
@ -335,10 +572,10 @@ def test_parse_endpoints_mixed_str_and_tuple():
assert len(addrs) == 2
assert isinstance(addrs[0], TCPAddress)
assert addrs[0].unwrap() == ('127.0.0.1', 7777)
assert addrs[0].unwrap() == ('tcp', '127.0.0.1', 7777)
assert isinstance(addrs[1], TCPAddress)
assert addrs[1].unwrap() == ('127.0.0.1', 8888)
assert addrs[1].unwrap() == ('tcp', '127.0.0.1', 8888)
def test_parse_endpoints_unsupported_proto():

View File

@ -371,7 +371,7 @@ def test_non_registrar_root_tpt_bind_addrs(
for uw_addr in bound:
w = wrap_address(uw_addr)
if w.proto_key == 'tcp':
_host, port = uw_addr
_, _host, port = uw_addr
assert port > 0
trio.run(_main)
@ -443,7 +443,7 @@ def test_tpt_bind_addrs_as_maddr_str(
for uw_addr in actor.accept_addrs:
w = wrap_address(uw_addr)
if w.proto_key == 'tcp':
_host, port = uw_addr
_, _host, port = uw_addr
assert port > 0
trio.run(_main)
@ -475,7 +475,7 @@ def test_registrar_merge_binds_union(
# actually differ (always true for TCP, may
# collide for UDS).
expect_disjoint: bool = (
tuple(reg_addr) != rando.unwrap()
reg_wrapped.unwrap() != rando.unwrap()
)
async def _main():

View File

@ -0,0 +1,298 @@
'''
`TunnelledAddress` delegation + peeling semantics.
A tunnel annotates an existing L4 addr rather than being its own
transport, so the contract under test is mostly *delegation*: the
runtime must not be able to tell a tunnelled addr from its
overlay, and **nothing** about the tunnel may cross the wire.
See `ai/tpt-backends/03_wg_tunnel_bindspace.md` §3.1/§3.4.
'''
from __future__ import annotations
import msgspec
import pytest
from tractor.discovery import (
BindspaceRef,
TunnelledAddress,
WGTunnelSpec,
mb_pubkey,
strip_tunnels,
tunnels_of,
wg8_pubkey,
)
from tractor.discovery._addr import (
is_wrapped_addr,
wrap_address,
)
from tractor.ipc._tcp import TCPAddress
from tractor.ipc._uds import UDSAddress
# a valid-looking std-base64 `wg(8)` pubkey (32B -> 44 chars)
_PUBKEY: str = 'g3x7z0AdV1rM6UQU22CC7IL3/ivn4DzrE7ikDhCZ/Dc='
@pytest.fixture
def overlay() -> TCPAddress:
return TCPAddress('10.0.11.1', 1616)
@pytest.fixture
def spec() -> WGTunnelSpec:
return WGTunnelSpec(
peer_pubkey=_PUBKEY,
bearer=('192.168.1.50', 51820),
)
@pytest.fixture
def tunnelled(
overlay: TCPAddress,
spec: WGTunnelSpec,
) -> TunnelledAddress:
return TunnelledAddress(overlay=overlay, tunnel=spec)
def test_wg_pubkey_codec_roundtrip():
'''
Standard `wg(8)` base64 keys can contain `/`, which cannot be
embedded unchanged in a slash-delimited maddr. Prove the helper
emits `u`-prefixed multibase base64url without `/` and decodes
it back to the exact original 32-byte key.
'''
mb_key: str = mb_pubkey(_PUBKEY)
assert mb_key.startswith('u')
assert '/' not in mb_key
assert wg8_pubkey(mb_key) == _PUBKEY
@pytest.mark.parametrize(
'key, converter',
[
pytest.param(
'dG9vIHNob3J0',
mb_pubkey,
id='wg8-base64',
),
pytest.param(
'udG9vIHNob3J0',
wg8_pubkey,
id='multibase',
),
],
)
def test_wg_pubkey_codec_rejects_wrong_size(
key: str,
converter,
):
'''
WireGuard silently-corrupt key handling would let an invalid
identity reach peer verification. Exercise both input encodings
with a short payload and prove conversion rejects it before a
tunnel spec or maddr can be constructed.
'''
with pytest.raises(
ValueError,
match='must decode to 32 bytes',
):
converter(key)
def test_proto_key_delegates(
tunnelled: TunnelledAddress,
overlay: TCPAddress,
):
'''
A tunnel has no transport of its own, so every table lookup
must see the *overlay's* proto-key.
'''
assert tunnelled.proto_key == overlay.proto_key == 'tcp'
def test_unwrap_is_identical_to_overlay(
tunnelled: TunnelledAddress,
overlay: TCPAddress,
):
'''
The whole point: nothing new crosses the wire, so a peer
never has to understand tunnels.
'''
assert tunnelled.unwrap() == overlay.unwrap()
# and it must survive msgpack as-is
enc: bytes = msgspec.msgpack.encode(tunnelled.unwrap())
assert msgspec.msgpack.decode(enc) == list(overlay.unwrap())
def test_unwrap_roundtrips_back_to_plain_overlay(
tunnelled: TunnelledAddress,
overlay: TCPAddress,
):
'''
`wrap_address()` on a tunnelled addr's unwrapped form yields
the *plain* overlay type the tunnel is simply absent, which
is correct: it was never on the wire.
'''
rewrapped = wrap_address(tunnelled.unwrap())
assert type(rewrapped) is TCPAddress
assert rewrapped == overlay
assert not isinstance(rewrapped, TunnelledAddress)
def test_bindspace_and_validity_delegate(
tunnelled: TunnelledAddress,
overlay: TCPAddress,
):
assert tunnelled.bindspace == overlay.bindspace
assert tunnelled.is_valid == overlay.is_valid
def test_is_wrapped_addr_accepts_tunnelled(
tunnelled: TunnelledAddress,
overlay: TCPAddress,
):
'''
`TunnelledAddress` is deliberately absent from
`_address_types`, so `is_wrapped_addr()` needs its own
clause.
'''
assert is_wrapped_addr(overlay)
assert is_wrapped_addr(tunnelled)
# the unwrapped form is NOT a wrapped addr
assert not is_wrapped_addr(tunnelled.unwrap())
def test_namespace_comes_from_the_tunnel(
overlay: TCPAddress,
):
'''
Plain transport addresses explicitly select no namespace, while
a tunnel can select one for the same concrete overlay.
'''
uds_addr: UDSAddress = UDSAddress('/tmp', 'tractor-test.sock')
assert overlay.namespace is None
assert uds_addr.namespace is None
no_ns = TunnelledAddress(
overlay=overlay,
tunnel=WGTunnelSpec(peer_pubkey=_PUBKEY),
)
assert no_ns.namespace is None
in_ns = TunnelledAddress(
overlay=overlay,
tunnel=WGTunnelSpec(peer_pubkey=_PUBKEY, netns='wg-test'),
)
assert in_ns.namespace == ('netns', 'wg-test')
def test_realized_namespace_uses_stable_ref(
overlay: TCPAddress,
) -> None:
'''
Realization must retain a stable ref without mutating the maddr.
Build an unrealized named declaration, annotate it with the
matching realized key and inode, and prove the frozen original is
unchanged. The annotated copy must preserve transport delegation
and expose the stable inode through `.namespace`. Direct msgspec
encoding also proves only serializable ref metadata was retained.
'''
declared: TunnelledAddress = TunnelledAddress(
overlay=overlay,
tunnel=WGTunnelSpec(
peer_pubkey=_PUBKEY,
netns='wg-test',
),
)
ref: BindspaceRef = BindspaceRef(
kind='netns',
key='wg-test',
inode=1234,
)
realized: TunnelledAddress = declared.with_bindspace_ref(
ref,
)
assert declared.bindspace_ref is None
assert realized.bindspace_ref is ref
assert realized.namespace == ('netns', 1234)
assert realized.overlay is declared.overlay
assert realized.tunnel is declared.tunnel
assert realized.unwrap() == declared.unwrap()
assert realized.bindspace == declared.bindspace
declared_payload: dict[str, object] = msgspec.msgpack.decode(
msgspec.msgpack.encode(declared)
)
assert 'bindspace_ref' not in declared_payload
decoded: dict[str, object] = msgspec.msgpack.decode(
msgspec.msgpack.encode(realized)
)
assert decoded['bindspace_ref'] == {
'kind': 'netns',
'key': 'wg-test',
'inode': 1234,
}
mismatched: BindspaceRef = BindspaceRef(
kind='netns',
key='other-netns',
inode=5678,
)
with pytest.raises(
ValueError,
match='wg-test.*other-netns',
):
declared.with_bindspace_ref(
mismatched,
)
def test_strip_tunnels(
tunnelled: TunnelledAddress,
overlay: TCPAddress,
spec: WGTunnelSpec,
):
# idempotent on a plain addr
assert strip_tunnels(overlay) is overlay
# peels one
assert strip_tunnels(tunnelled) is overlay
# and collapses a nested stack in one call
nested = TunnelledAddress(overlay=tunnelled, tunnel=spec)
assert strip_tunnels(nested) is overlay
def test_tunnels_of_is_outermost_first(
tunnelled: TunnelledAddress,
overlay: TCPAddress,
):
assert tunnels_of(overlay) == ()
assert tunnels_of(tunnelled) == (tunnelled.tunnel,)
inner_spec = WGTunnelSpec(peer_pubkey=_PUBKEY, iface='wg1')
nested = TunnelledAddress(
overlay=tunnelled,
tunnel=inner_spec,
)
assert tunnels_of(nested) == (inner_spec, tunnelled.tunnel)
def test_frozen(
tunnelled: TunnelledAddress,
):
with pytest.raises(AttributeError):
tunnelled.overlay = TCPAddress('127.0.0.1', 1)

View File

@ -0,0 +1,163 @@
'''
Process-local WireGuard interface configuration contracts.
'''
from __future__ import annotations
import msgspec
import pytest
from tractor.discovery import (
WGInterfaceConfig,
WGPeerConfig,
)
from tractor.msg import ProcessLocal
_PRIVATE_KEY: str = 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
_PEER_KEY: str = 'r1LKM1pqhuY9Z6L4y5jQ2fGX67kJSrq5kRV5Jk2ywEo='
_PRESHARED_KEY: str = 'BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBA='
def test_wg_config_is_process_local_and_redacted() -> None:
'''
Private WireGuard configuration must neither print nor cross IPC.
Construct a complete local config and prove its public routing
policy remains inspectable while both keys are absent from repr.
Verify `ProcessLocal` blocks default msgpack encoding.
'''
peer: WGPeerConfig = WGPeerConfig(
public_key=_PEER_KEY,
allowed_ips=('10.1.0.0/16', 'fd01::/64'),
endpoint=('192.0.2.1', 51820),
preshared_key=_PRESHARED_KEY,
persistent_keepalive=25,
)
config: WGInterfaceConfig = WGInterfaceConfig(
private_key=_PRIVATE_KEY,
addresses=('10.0.0.1/24', 'fd00::1/64'),
listen_port=51820,
peers=(peer,),
)
config_repr: str = repr(config)
assert isinstance(config, ProcessLocal)
assert _PRIVATE_KEY not in config_repr
assert _PRESHARED_KEY not in config_repr
assert config.addresses[0] in config_repr
assert peer.allowed_ips[0] in config_repr
with pytest.raises(
TypeError,
match='_ProcessLocalToken.*unsupported',
):
msgspec.msgpack.encode(config)
@pytest.mark.parametrize(
('kwargs', 'error'),
(
pytest.param(
{'private_key': 'not-base64'},
ValueError,
id='private-key',
),
pytest.param(
{
'private_key': _PRIVATE_KEY,
'addresses': ('not-an-interface',),
},
ValueError,
id='address',
),
pytest.param(
{
'private_key': _PRIVATE_KEY,
'listen_port': 65536,
},
ValueError,
id='listen-port',
),
pytest.param(
{
'private_key': _PRIVATE_KEY,
'peers': (
WGPeerConfig(public_key=_PEER_KEY),
WGPeerConfig(public_key=_PEER_KEY),
),
},
ValueError,
id='duplicate-peer',
),
),
)
def test_wg_config_rejects_invalid_values(
kwargs: dict[str, object],
error: type[Exception],
) -> None:
'''
Invalid config must fail before kernel mutation.
Parameterize every validated input class and prove direct msgspec
construction cannot carry malformed configuration into a future
pyroute2 interface lifecycle.
'''
with pytest.raises(error):
WGInterfaceConfig(**kwargs) # type: ignore[arg-type]
@pytest.mark.parametrize(
'kwargs',
(
pytest.param(
{'public_key': 'not-base64'},
id='public-key',
),
pytest.param(
{
'public_key': _PEER_KEY,
'preshared_key': 'not-base64',
},
id='preshared-key',
),
pytest.param(
{
'public_key': _PEER_KEY,
'allowed_ips': ('not-a-network',),
},
id='allowed-ip',
),
pytest.param(
{
'public_key': _PEER_KEY,
'endpoint': ('not-an-ip', 51820),
},
id='endpoint-host',
),
pytest.param(
{
'public_key': _PEER_KEY,
'endpoint': ('192.0.2.1', 65536),
},
id='endpoint-port',
),
pytest.param(
{
'public_key': _PEER_KEY,
'persistent_keepalive': -1,
},
id='keepalive',
),
),
)
def test_wg_peer_config_rejects_invalid_values(
kwargs: dict[str, object],
) -> None:
'''
Invalid peer policy must fail before kernel mutation.
'''
with pytest.raises(ValueError):
WGPeerConfig(**kwargs) # type: ignore[arg-type]

View File

@ -0,0 +1,378 @@
'''
WireGuard interface policy and owned lifecycle contracts.
'''
from __future__ import annotations
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager as acm
import os
from pathlib import Path
from typing import BinaryIO
import pytest
import trio
from tractor.discovery import (
Bindspace,
BindspaceRef,
BindspaceSpec,
WGInterfaceConfig,
WGPeerConfig,
WGTunnelSpec,
open_wg_bindspace,
open_wg_iface,
)
from tractor.discovery import _tunnel
_LOCAL_KEY: str = 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
_PEER_KEY: str = 'r1LKM1pqhuY9Z6L4y5jQ2fGX67kJSrq5kRV5Jk2ywEo='
def test_wg_iface_settings_follow_role() -> None:
'''
Listen and dial roles interpret the tunnel bearer differently.
Prove listen derives its local port from the bearer while dial
applies the bearer as the selected peer's omitted endpoint. Other
explicit peers retain their own endpoint and routing policy.
'''
selected: WGPeerConfig = WGPeerConfig(
public_key=_PEER_KEY,
allowed_ips=('10.1.0.0/16',),
)
other: WGPeerConfig = WGPeerConfig(
public_key=_LOCAL_KEY,
endpoint=('198.51.100.2', 51821),
)
config: WGInterfaceConfig = WGInterfaceConfig(
private_key=_LOCAL_KEY,
peers=(selected, other),
)
spec: WGTunnelSpec = WGTunnelSpec(
peer_pubkey=_PEER_KEY,
bearer=('192.0.2.1', 51820),
)
listen_port: int|None
listen_peers: tuple[dict[str, object], ...]
listen_port, listen_peers = _tunnel._wg_iface_settings(
spec,
config,
'listen',
)
assert listen_port == 51820
# A listen bearer configures the local port, not a peer endpoint.
assert 'endpoint_addr' not in listen_peers[0]
dial_port: int|None
dial_peers: tuple[dict[str, object], ...]
dial_port, dial_peers = _tunnel._wg_iface_settings(
spec,
config,
'dial',
)
# No local dial listen port was declared; the bearer is remote.
assert dial_port is None
assert dial_peers[0]['endpoint_addr'] == '192.0.2.1'
assert dial_peers[0]['endpoint_port'] == 51820
assert dial_peers[0]['allowed_ips'] == ['10.1.0.0/16']
assert dial_peers[1]['endpoint_addr'] == '198.51.100.2'
assert dial_peers[1]['endpoint_port'] == 51821
@pytest.mark.parametrize(
('config', 'role', 'match'),
(
pytest.param(
WGInterfaceConfig(
private_key=_LOCAL_KEY,
),
'dial',
'not in.*configured peer keys',
id='missing-dial-peer',
),
pytest.param(
WGInterfaceConfig(
private_key=_LOCAL_KEY,
listen_port=51821,
),
'listen',
'51821.*51820',
id='listen-port-51821-vs-bearer-51820',
),
pytest.param(
WGInterfaceConfig(
private_key=_LOCAL_KEY,
peers=(
WGPeerConfig(
public_key=_PEER_KEY,
endpoint=('198.51.100.1', 51820),
),
),
),
'dial',
'198.51.100.1.*192.0.2.1',
id='dial-endpoint-conflict',
),
),
)
def test_wg_iface_settings_reject_conflicts(
config: WGInterfaceConfig,
role: str,
match: str,
) -> None:
'''
Role-dependent conflicts must fail before pyroute2 side effects.
'''
spec: WGTunnelSpec = WGTunnelSpec(
peer_pubkey=_PEER_KEY,
bearer=('192.0.2.1', 51820),
)
with pytest.raises(ValueError, match=match):
_tunnel._wg_iface_settings(
spec,
config,
role, # type: ignore[arg-type]
)
def test_open_wg_iface_shields_cancelled_cleanup(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
'''
Cancellation after creation must still remove the owned WG iface.
Pin a stand-in namespace FD and fake privileged create/remove
calls. Cancel inside the yielded context and prove shielded
teardown runs before cancellation leaves the enclosing scope.
'''
token_path: Path = tmp_path / 'netns'
token_path.touch()
events: list[str] = []
def create(
spec: WGTunnelSpec,
config: WGInterfaceConfig,
bindspace: Bindspace,
listen_port: int|None,
peers: tuple[dict[str, object], ...],
) -> None:
'''
Record the validated create request.
'''
assert bindspace.namespace_fd is not None
assert listen_port is None
assert peers[0]['public_key'] == _PEER_KEY
events.append('create')
def remove(
spec: WGTunnelSpec,
bindspace: Bindspace,
) -> None:
'''
Record shielded removal after cancellation.
'''
events.append('remove')
monkeypatch.setattr(
_tunnel,
'_sync_create_wg_iface',
create,
)
monkeypatch.setattr(
_tunnel,
'_sync_remove_wg_iface',
remove,
)
namespace_file: BinaryIO
with token_path.open('rb') as namespace_file:
namespace_fd: int = namespace_file.fileno()
bindspace_spec: BindspaceSpec = BindspaceSpec(
kind='netns',
key='tractor-wg0',
)
bindspace: Bindspace = Bindspace(
spec=bindspace_spec,
ref=BindspaceRef(
kind='netns',
key='tractor-wg0',
inode=os.fstat(namespace_fd).st_ino,
),
namespace_fd=namespace_fd,
ownership='borrowed',
)
peer: WGPeerConfig = WGPeerConfig(
public_key=_PEER_KEY,
)
config: WGInterfaceConfig = WGInterfaceConfig(
private_key=_LOCAL_KEY,
peers=(peer,),
)
spec: WGTunnelSpec = WGTunnelSpec(
peer_pubkey=_PEER_KEY,
)
async def main() -> None:
'''
Cancel while the fake WG iface is owned.
'''
with trio.CancelScope() as scope:
async with open_wg_iface(
spec,
config,
bindspace,
'dial',
):
events.append('yield')
scope.cancel()
await trio.sleep_forever()
trio.run(main)
assert events == ['create', 'yield', 'remove']
def test_open_wg_bindspace_nests_resource_lifetimes(
monkeypatch: pytest.MonkeyPatch,
) -> None:
'''
Nested WG interfaces must exit before their bindspace capability.
Fake two interface layers over one bindspace. Clear the caller's
mutable layer list at bindspace entry, then cancel from inside
the yielded application scope and checkpoint. The trace proves
stack snapshots its declaration before entry, layers enter
outermost-first, cancellation exits them inside-out, and the live
bindspace remains available through every interface exit.
'''
events: list[str] = []
calls: list[
tuple[
WGTunnelSpec,
WGInterfaceConfig,
Bindspace,
_tunnel.WGRole,
]
] = []
bindspace_spec: BindspaceSpec = BindspaceSpec(
kind='netns',
)
bindspace: Bindspace = Bindspace(
spec=bindspace_spec,
ref=BindspaceRef(
kind='netns',
key=None,
inode=1,
),
namespace_fd=None,
ownership='borrowed',
)
outer_spec: WGTunnelSpec = WGTunnelSpec(
peer_pubkey=_PEER_KEY,
iface='wg-outer',
)
inner_spec: WGTunnelSpec = WGTunnelSpec(
peer_pubkey=_LOCAL_KEY,
iface='wg-inner',
)
outer_config: WGInterfaceConfig = WGInterfaceConfig(
private_key=_LOCAL_KEY,
)
inner_config: WGInterfaceConfig = WGInterfaceConfig(
private_key=_PEER_KEY,
)
layers: list[
tuple[WGTunnelSpec, WGInterfaceConfig]
] = [
(outer_spec, outer_config),
(inner_spec, inner_config),
]
@acm
async def fake_open_bindspace(
spec: BindspaceSpec,
) -> AsyncIterator[Bindspace]:
'''
Yield the stand-in bindspace and record its full lifetime.
'''
assert spec is bindspace_spec
events.append('bindspace-enter')
layers.clear()
try:
yield bindspace
finally:
events.append('bindspace-exit')
@acm
async def fake_open_wg_iface(
spec: WGTunnelSpec,
config: WGInterfaceConfig,
bindspace_arg: Bindspace,
role: _tunnel.WGRole,
) -> AsyncIterator[WGTunnelSpec]:
'''
Record one interface's arguments and nested lifetime.
'''
calls.append((spec, config, bindspace_arg, role))
events.append(f'{spec.iface}-enter')
try:
yield spec
finally:
assert bindspace_arg is bindspace
events.append(f'{spec.iface}-exit')
monkeypatch.setattr(
_tunnel,
'open_bindspace',
fake_open_bindspace,
)
monkeypatch.setattr(
_tunnel,
'open_wg_iface',
fake_open_wg_iface,
)
async def main() -> None:
'''
Cancel while both interface layers are live.
'''
with trio.CancelScope() as scope:
async with open_wg_bindspace(
bindspace_spec,
layers,
'dial',
) as opened_bindspace:
assert opened_bindspace is bindspace
events.append('yield')
scope.cancel()
await trio.sleep_forever()
trio.run(main)
assert calls == [
(outer_spec, outer_config, bindspace, 'dial'),
(inner_spec, inner_config, bindspace, 'dial'),
]
assert events == [
'bindspace-enter',
'wg-outer-enter',
'wg-inner-enter',
'yield',
'wg-inner-exit',
'wg-outer-exit',
'bindspace-exit',
]

View File

@ -0,0 +1,305 @@
'''
Read-only WireGuard netlink inspection tests.
'''
from __future__ import annotations
import threading
from typing import (
Any,
NoReturn,
)
import pytest
import trio
from tractor.discovery import (
read_wg_peers,
read_wg_pubkey,
verify_wg_peer,
WGTunnelSpec,
)
from tractor.discovery import _tunnel
pyroute2: Any = pytest.importorskip('pyroute2')
_PUBKEY: str = 'g3x7z0AdV1rM6UQU22CC7IL3/ivn4DzrE7ikDhCZ/Dc='
_PEER_1: str = '7PClzcj8o1yAjyPJb0zL2Gt0s2J7yZ6c0JXYqNBGr0E='
_PEER_2: str = 'H7bJbl1bpY7VzDlB5wI3KjA7JsiYoMWGDJd8dYgc5iw='
_MISSING_KEY: str = 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
class Attrs:
'''
Minimal pyroute2 netlink-attribute message fake.
'''
def __init__(
self,
**attrs: Any,
) -> None:
'''
Store attributes for `.get_attr()` lookups.
'''
self._attrs: dict[str, Any] = attrs
def get_attr(
self,
name: str,
) -> Any:
'''
Return the named fake netlink attribute.
'''
return self._attrs.get(name)
def test_read_wg_keys_in_worker_thread(
monkeypatch: pytest.MonkeyPatch,
) -> None:
'''
Pyroute2's synchronous `WireGuard` API owns a private asyncio
loop. Running it in the Trio thread would either block Trio or
introduce that foreign loop into the actor runtime.
Replace `pyroute2.WireGuard` with a fake which records thread,
iface, netns and close state. Return a multipart dump containing
duplicate peers, then prove both public helpers execute
off-thread, preserve named-netns selection, validate keys,
deduplicate peers in kernel order and close every netlink client.
'''
trio_thread: int = threading.get_ident()
class FakeWireGuard:
'''
Record each read-only `pyroute2.WireGuard` interaction.
'''
def __init__(
self,
*,
netns: str|None,
flags: int,
) -> None:
'''
Record namespace selection without opening netlink.
'''
self.netns = netns
self.flags = flags
self.closed = False
self.thread_id: int|None = None
self.iface: str|None = None
instances.append(self)
def info(
self,
iface: str,
) -> tuple[Attrs, Attrs]:
'''
Return a multipart WireGuard device dump.
'''
self.thread_id = threading.get_ident()
self.iface = iface
peer_1: Attrs = Attrs(
WGPEER_A_PUBLIC_KEY=_PEER_1.encode(),
)
peer_2: Attrs = Attrs(
WGPEER_A_PUBLIC_KEY=_PEER_2.encode(),
)
return (
Attrs(
WGDEVICE_A_PUBLIC_KEY=_PUBKEY.encode(),
WGDEVICE_A_PEERS=[peer_1],
),
Attrs(
WGDEVICE_A_PUBLIC_KEY=_PUBKEY.encode(),
WGDEVICE_A_PEERS=[peer_2, peer_1],
),
)
def close(self) -> None:
'''
Record netlink-client cleanup.
'''
self.closed = True
instances: list[FakeWireGuard] = []
monkeypatch.setattr(
pyroute2,
'WireGuard',
FakeWireGuard,
)
async def main() -> None:
'''
Read both key views from Trio's run thread.
'''
assert await read_wg_pubkey(
iface='wg-test',
netns='actor-net',
) == _PUBKEY
assert await read_wg_peers(
iface='wg-test',
netns='actor-net',
) == (_PEER_1, _PEER_2)
trio.run(main)
assert len(instances) == 2
instance: FakeWireGuard
for instance in instances:
assert instance.netns == 'actor-net'
assert instance.flags == 0
assert instance.iface == 'wg-test'
assert instance.thread_id != trio_thread
assert instance.closed
def test_wg_client_closes_when_read_fails(
monkeypatch: pytest.MonkeyPatch,
) -> None:
'''
A failed netlink read must not leak pyroute2's socket or private
event loop. Raise from the fake `.info()` call and prove the same
error reaches the Trio caller only after `.close()` runs.
'''
class FakeWireGuard:
'''
Raise during device inspection and record cleanup.
'''
def __init__(
self,
*,
netns: str|None,
flags: int,
) -> None:
'''
Publish this fake instance for the cleanup assertion.
'''
nonlocal instance
self.closed = False
instance = self
def info(self, iface: str) -> NoReturn:
'''
Simulate a failing netlink device read.
'''
raise OSError('netlink read failed')
def close(self) -> None:
'''
Record cleanup after the failed read.
'''
self.closed = True
instance: FakeWireGuard|None = None
monkeypatch.setattr(
pyroute2,
'WireGuard',
FakeWireGuard,
)
with pytest.raises(
OSError,
match='netlink read failed',
):
trio.run(read_wg_pubkey)
assert instance is not None
assert instance.closed
@pytest.mark.parametrize(
('declared_key', 'expected'),
(
(_PUBKEY, True),
(_PEER_2, True),
(_MISSING_KEY, False),
),
)
def test_verify_wg_peer(
monkeypatch: pytest.MonkeyPatch,
declared_key: str,
expected: bool,
) -> None:
'''
A tunnel declaration can identify either side of one local iface.
Return one stable key snapshot from the async reader, then prove
a local interface key and configured peer both verify while an
absent key does not. Also prove the spec selects the iface/netns
supplied to the read instead of silently using process defaults.
'''
reads: list[tuple[str, str|None]] = []
async def read_keys(
iface: str,
netns: str|None,
) -> tuple[str, tuple[str, ...]]:
'''
Return one deterministic WireGuard key snapshot.
'''
reads.append((iface, netns))
return _PUBKEY, (_PEER_1, _PEER_2)
monkeypatch.setattr(
_tunnel,
'_read_wg_keys',
read_keys,
)
spec: WGTunnelSpec = WGTunnelSpec(
peer_pubkey=declared_key,
iface='wg-test',
netns='actor-net',
)
assert trio.run(verify_wg_peer, spec) is expected
assert reads == [('wg-test', 'actor-net')]
def test_verify_wg_peer_validates_before_read(
monkeypatch: pytest.MonkeyPatch,
) -> None:
'''
A directly constructed tunnel spec can contain a malformed key.
Install a reader which would fail if called, pass malformed
base64, and prove validation rejects the declaration before any
kernel-state inspection occurs.
'''
async def unexpected_read(
iface: str,
netns: str|None,
) -> NoReturn:
'''
Fail if malformed-key validation reaches the read boundary.
'''
raise AssertionError('WireGuard read must not run')
monkeypatch.setattr(
_tunnel,
'_read_wg_keys',
unexpected_read,
)
spec: WGTunnelSpec = WGTunnelSpec(
peer_pubkey='not-base64',
)
with pytest.raises(ValueError):
trio.run(verify_wg_peer, spec)

View File

@ -0,0 +1,91 @@
'''
Tunnel annotation peeling at the outbound IPC transport boundary.
'''
from __future__ import annotations
import pytest
import trio
from tractor.discovery import (
TunnelledAddress,
WGTunnelSpec,
tunnels_of,
)
from tractor.ipc import _chan
from tractor.ipc._tcp import TCPAddress
_PUBKEY: str = 'g3x7z0AdV1rM6UQU22CC7IL3/ivn4DzrE7ikDhCZ/Dc='
@pytest.fixture
def overlay() -> TCPAddress:
return TCPAddress('127.0.0.1', 0)
@pytest.fixture
def tunnelled(
overlay: TCPAddress,
) -> TunnelledAddress:
return TunnelledAddress(
overlay=overlay,
tunnel=WGTunnelSpec(
peer_pubkey=_PUBKEY,
bearer=('192.168.1.50', 51820),
),
)
@pytest.mark.parametrize('use_tunnel', [False, True])
def test_channel_peels_before_transport_dispatch(
monkeypatch,
overlay: TCPAddress,
tunnelled: TunnelledAddress,
use_tunnel: bool,
):
'''
Exact-type transport lookup cannot dispatch a `TunnelledAddress`,
and passing one onward would make TCP dial the wrong object. Feed
both a plain overlay and its annotated wrapper into
`Channel.from_addr()`, capture lookup and connect arguments, and
prove both transport operations receive only the same bindable
TCP address while the caller's tunnel metadata remains intact.
'''
seen: list[tuple[str, TCPAddress]] = []
class FakeTransport:
@classmethod
async def connect_to(
cls,
addr: TCPAddress,
**kwargs,
) -> FakeTransport:
seen.append(('connect', addr))
return cls()
def fake_transport_from_addr(
addr: TCPAddress,
) -> type[FakeTransport]:
seen.append(('lookup', addr))
return FakeTransport
monkeypatch.setattr(
_chan,
'transport_from_addr',
fake_transport_from_addr,
)
async def main() -> None:
declared = tunnelled if use_tunnel else overlay
chan = await _chan.Channel.from_addr(declared)
assert isinstance(chan.transport, FakeTransport)
trio.run(main)
assert seen == [
('lookup', overlay),
('connect', overlay),
]
assert tunnels_of(tunnelled) == (tunnelled.tunnel,)

View File

@ -703,17 +703,20 @@ def test_uds_bindspace_created_implicitly(
root: Actor = tractor.current_actor()
assert root.is_registrar
canonical_addr = _addr.wrap_address(
registry_addr,
).unwrap()
assert registry_addr in root.reg_addrs
assert canonical_addr in root.reg_addrs
assert (
registry_addr
canonical_addr
in
_state._runtime_vars['_registry_addrs']
)
assert (
_addr.wrap_address(registry_addr)
canonical_addr
in
root.registry_addrs
[addr.unwrap() for addr in root.registry_addrs]
)
trio.run(main)

View File

@ -0,0 +1,81 @@
'''
Tunnel annotation peeling at the inbound IPC transport boundary.
'''
from __future__ import annotations
import trio
from tractor.discovery import (
BindspaceRef,
TunnelledAddress,
WGTunnelSpec,
tunnels_of,
)
from tractor.ipc._server import open_ipc_server
from tractor.ipc._tcp import TCPAddress
_PUBKEY: str = 'g3x7z0AdV1rM6UQU22CC7IL3/ivn4DzrE7ikDhCZ/Dc='
def test_server_peels_before_endpoint_construction() -> None:
'''
`Endpoint.start_listener()` reflects on its address's declaring
module, so retaining a tunnel wrapper there selects `._tunnel`
instead of the TCP backend. Start a real listener from the
wrapper, assert the resulting `Endpoint` contains only a resolved
`TCPAddress`. Prove `Endpoint.declared_addr` still retains the
original declaration and realized bindspace ref for
diagnostics. This retained metadata does not claim the listener
process entered that namespace.
'''
overlay = TCPAddress('127.0.0.1', 0)
tunnelled = TunnelledAddress(
overlay=overlay,
tunnel=WGTunnelSpec(
peer_pubkey=_PUBKEY,
bearer=('192.168.1.50', 51820),
netns='actor-net',
),
)
ref: BindspaceRef = BindspaceRef(
kind='netns',
key='actor-net',
inode=1234,
)
declared: TunnelledAddress = tunnelled.with_bindspace_ref(
ref,
)
async def main() -> None:
async with open_ipc_server() as server:
eps = await server.listen_on(
accept_addrs=[declared],
)
assert len(eps) == 1
endpoint = eps[0]
assert type(endpoint.addr) is TCPAddress
_, host, port = endpoint.addr.unwrap()
assert host == overlay.unwrap()[1]
assert port > 0
assert endpoint.addr is not declared
assert endpoint.declared_addr is declared
namespace: tuple[str, int] = ('netns', 1234)
assert endpoint.namespace == namespace
endpoint_repr: str = endpoint.pformat()
server_repr: str = server.pformat()
expected_namespace: str = f'namespace: {namespace!r}'
assert expected_namespace in endpoint_repr
assert ' |_namespaces:' in server_repr
assert 'netns' in server_repr
assert '1234' in server_repr
assert tunnels_of(declared) == (
declared.tunnel,
)
server.cancel()
trio.run(main)

View File

@ -0,0 +1,57 @@
'''
Process-local struct wire-encoding guards.
'''
from __future__ import annotations
import msgspec
import pytest
from tractor.msg import ProcessLocal
class LocalHandle(ProcessLocal):
'''
Minimal process-local struct used to exercise the global marker.
'''
resource_id: int
@pytest.mark.parametrize(
'nested',
(
pytest.param(
False,
id='direct',
),
pytest.param(
True,
id='nested',
),
),
)
def test_process_local_rejects_default_encoding(
nested: bool,
) -> None:
'''
Process-local values can appear directly or deep in a payload.
Embed the same marked struct at both depths and prove msgspec's
normal traversal reaches the unsupported sentinel without a
tractor-specific recursive payload scan.
'''
handle: LocalHandle = LocalHandle(resource_id=1)
value: object = (
{'nested': [handle]}
if nested
else handle
)
assert repr(handle) == 'LocalHandle(resource_id=1)'
with pytest.raises(
TypeError,
match='_ProcessLocalToken.*unsupported',
):
msgspec.msgpack.encode(value)

View File

@ -8,6 +8,7 @@ import trio
import tractor
from tractor._testing import tractor_test
from tractor.discovery._addr import wrap_address
def test_no_runtime():
@ -48,7 +49,7 @@ async def test_self_is_registered_localportal(reg_addr):
with trio.fail_after(0.2):
sockaddr = await portal.run_from_ns(
'self', 'wait_for_actor', name='root')
assert sockaddr[0] == reg_addr
assert sockaddr[0] == wrap_address(reg_addr).unwrap()
def test_local_actor_async_func(reg_addr):

File diff suppressed because it is too large Load Diff

View File

@ -10,15 +10,25 @@ API design.
"""
from functools import partial
from types import SimpleNamespace
from typing import (
Any,
)
from unittest.mock import (
AsyncMock,
MagicMock,
)
import pytest
import trio
import tractor
from tractor._exceptions import ActorFailure
from tractor._testing import tractor_test
from tractor.spawn import (
_spawn,
_trio,
)
data_to_pass_down = {
'doggy': 10,
@ -26,6 +36,396 @@ data_to_pass_down = {
}
def test_peer_handshake_wins_child_boot_race() -> None:
'''
A connected child must cancel process-death monitoring cleanly.
Start the death waiter first and hold it at a checkpoint. Let the
fake server then return one peer event and channel. The helper must
preserve the normal handshake result and cancel the losing process
waiter before its nursery exits. The fake explicitly catches and
re-raises `trio.Cancelled` to prove cancellation caused its exit.
'''
async def main() -> None:
'''
Control the handshake-first schedule with Trio events.
'''
uid: tuple[str, str] = ('handshake-child', 'test')
death_started = trio.Event()
death_cancelled = trio.Event()
peer_event = trio.Event()
channel = object()
async def wait_for_peer(
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Return the peer only after death monitoring is active.
'''
assert child_uid == uid
await death_started.wait()
return (peer_event, channel)
async def wait_for_death() -> int:
'''
Block until the winning handshake cancels this waiter.
'''
death_started.set()
try:
await trio.sleep_forever()
except trio.Cancelled:
death_cancelled.set()
raise
result = await _spawn.wait_for_peer_or_proc_death(
ipc_server=SimpleNamespace(
wait_for_peer=wait_for_peer,
),
uid=uid,
proc_wait=wait_for_death,
proc_repr='handshake-proc',
)
assert result == (peer_event, channel)
assert death_cancelled.is_set()
trio.run(main)
def test_child_death_wins_peer_handshake_race() -> None:
'''
Pre-handshake child death must fail startup instead of hanging.
Start the peer waiter first and leave it parked like
`IPCServer.wait_for_peer()` on an unset event. Return a distinctive
process status from the competing waiter, then prove the helper
cancels the handshake and raises `ActorFailure` with child identity,
status, and process diagnostics.
'''
async def main() -> None:
'''
Control the death-first schedule with Trio events.
'''
uid: tuple[str, str] = ('dead-child', 'test')
handshake_started = trio.Event()
handshake_cancelled = trio.Event()
async def wait_for_peer(
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Park until process death cancels this handshake waiter.
'''
assert child_uid == uid
handshake_started.set()
try:
await trio.sleep_forever()
except trio.Cancelled:
handshake_cancelled.set()
raise
async def wait_for_death() -> int:
'''
Report child death after handshake monitoring is active.
'''
await handshake_started.wait()
return 23
with pytest.raises(ActorFailure) as exc_info:
await _spawn.wait_for_peer_or_proc_death(
ipc_server=SimpleNamespace(
wait_for_peer=wait_for_peer,
),
uid=uid,
proc_wait=wait_for_death,
proc_repr='dead-proc',
)
message: str = str(exc_info.value)
assert repr(uid) in message
assert 'died during boot' in message
assert '(rc=23)' in message
assert 'parent-handshake' in message
assert 'dead-proc' in message
assert handshake_cancelled.is_set()
trio.run(main)
def test_child_death_wins_simultaneous_boot_results() -> None:
'''
Observed process death must outrank a simultaneous handshake.
Hold both fake waits behind one barrier with cancellation shielding,
then release them together so both publish a committed result before
sibling cancellation takes effect. Because the child has exited
before receiving its `SpawnSpec`, bootstrap must raise `ActorFailure`
rather than return its briefly established channel.
'''
async def main() -> None:
'''
Release both boot outcomes from one controlled barrier.
'''
uid: tuple[str, str] = ('simultaneous-child', 'test')
handshake_ready = trio.Event()
death_ready = trio.Event()
release = trio.Event()
peer_event = trio.Event()
async def wait_for_peer(
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Publish a handshake despite sibling cancellation.
'''
assert child_uid == uid
handshake_ready.set()
with trio.CancelScope(shield=True):
await release.wait()
return (peer_event, object())
async def wait_for_death() -> int:
'''
Publish process death despite sibling cancellation.
'''
death_ready.set()
with trio.CancelScope(shield=True):
await release.wait()
return 0
async def release_both() -> None:
'''
Open the barrier only after both waiters are parked.
'''
await handshake_ready.wait()
await death_ready.wait()
release.set()
async with trio.open_nursery() as nursery:
nursery.start_soon(release_both)
with pytest.raises(
ActorFailure,
match=r'simultaneous-child.*rc=0',
):
await _spawn.wait_for_peer_or_proc_death(
ipc_server=SimpleNamespace(
wait_for_peer=wait_for_peer,
),
uid=uid,
proc_wait=wait_for_death,
)
trio.run(main)
@pytest.mark.parametrize('failing_waiter', ('handshake', 'death'))
def test_child_boot_race_preserves_waiter_error(
failing_waiter: str,
) -> None:
'''
Waiter failures must retain their original exception identity.
Park the non-failing sibling and raise one unique error from either
the peer or process waiter. The helper's internal nursery must
cancel the sibling and re-raise that exact exception instead of
wrapping it in an `ExceptionGroup`.
'''
async def main() -> None:
'''
Trigger one selected waiter after its sibling starts.
'''
uid: tuple[str, str] = ('errored-child', 'test')
sibling_started = trio.Event()
wait_error = RuntimeError(f'{failing_waiter} failed')
async def wait_for_peer(
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Raise or park according to the selected peer schedule.
'''
assert child_uid == uid
if failing_waiter == 'handshake':
await sibling_started.wait()
raise wait_error
sibling_started.set()
await trio.sleep_forever()
async def wait_for_death() -> int:
'''
Raise or park according to the selected process schedule.
'''
if failing_waiter == 'death':
await sibling_started.wait()
raise wait_error
sibling_started.set()
await trio.sleep_forever()
with pytest.raises(RuntimeError) as exc_info:
await _spawn.wait_for_peer_or_proc_death(
ipc_server=SimpleNamespace(
wait_for_peer=wait_for_peer,
),
uid=uid,
proc_wait=wait_for_death,
)
assert exc_info.value is wait_error
trio.run(main)
def test_trio_proc_cleans_failed_child_peer_event(
monkeypatch: pytest.MonkeyPatch,
) -> None:
'''
Death-first Trio startup must release provisional peer state.
Return one already-dead fake process while its server handshake
parks forever. The fake nursery proves the peer event exists before
provisional child publication. After `ActorFailure`, both that
exact event and the provisional child record must be gone so repeated
failed spawns cannot leak server state.
'''
uid: tuple[str, str] = ('dead-trio-child', 'test')
proc: trio.Process = MagicMock(spec=trio.Process)
proc.pid = 1234
proc.wait = AsyncMock(return_value=23)
proc.poll.return_value = 23
proc.__str__.return_value = 'dead-trio-proc'
class FakeServer:
'''
Hold the peer registry used during Trio child startup.
'''
def __init__(self) -> None:
self._peer_connected: dict[
tuple[str, str],
trio.Event,
] = {}
async def wait_for_peer(
self,
child_uid: tuple[str, str],
) -> tuple[trio.Event, object]:
'''
Park like a child that never reaches its handshake.
'''
assert child_uid == uid
await trio.sleep_forever()
server = FakeServer()
class FakeNursery:
'''
Track provisional child publication and cleanup.
'''
def __init__(self) -> None:
self._actor = SimpleNamespace(ipc_server=server)
self._children: dict[tuple[str, str], tuple] = {}
def _register_child(
self,
subactor: object,
proc: object,
portal: object|None,
) -> tuple[trio.Event, trio.Event, bool]:
'''
Require peer-event registration before child publication.
'''
assert uid in server._peer_connected
assert portal is None
self._children[uid] = (subactor, proc, portal)
return (trio.Event(), trio.Event(), False)
async def fake_open_process(
command: list[str],
**kwargs: object,
) -> trio.Process:
'''
Return a process whose death wins the bootstrap race.
'''
assert command
return proc
async def fake_wait_for_debugger(**kwargs: object) -> None:
'''
Keep hard-reap cleanup deterministic and non-interactive.
'''
return None
monkeypatch.setattr(
_trio.trio.lowlevel,
'open_process',
fake_open_process,
)
monkeypatch.setattr(
_trio.debug,
'maybe_wait_for_debugger',
fake_wait_for_debugger,
)
nursery = FakeNursery()
subactor = SimpleNamespace(
aid=tractor.msg.Aid(
name=uid[0],
uuid=uid[1],
),
loglevel=None,
pformat=lambda: 'dead-trio-child',
)
async def main() -> None:
'''
Run the full Trio backend through death-first cleanup.
'''
with pytest.raises(
ActorFailure,
match=r'dead-trio-child.*rc=23',
):
await _trio.trio_proc(
name=uid[0],
actor_nursery=nursery,
subactor=subactor,
errors={},
bind_addrs=[],
parent_addr=('127.0.0.1', 1616),
_runtime_vars={},
)
trio.run(main)
assert server._peer_connected == {}
assert nursery._children == {}
async def run_same_func_in_child(
should_be_root: bool,
data: dict,

View File

@ -8,14 +8,18 @@ from contextlib import (
from functools import partial
from itertools import cycle
import time
from types import SimpleNamespace
from typing import Optional
import warnings
import pytest
import trio
from trio.lowlevel import current_task
import tractor
from tractor.to_asyncio import LinkedTaskChannel
from tractor.trionics import (
broadcast_receiver,
BroadcastReceiveError,
Lagged,
collapse_eg,
)
@ -307,6 +311,847 @@ def test_subscribe_errors_after_close():
trio.run(main)
@pytest.mark.parametrize(
('size', 'sent', 'dropped'),
[
(1, 2, 1),
(3, 5, 2),
],
)
def test_lagged_reports_exact_drop_count(
size: int,
sent: int,
dropped: int,
) -> None:
'''
`Lagged` must report every value outside the retained window.
`BroadcastReceiver.receive_nowait()` previously subtracted the
queue length from an already-invalid deque index without counting
that first displaced value. A one-slot queue therefore claimed it
dropped zero values after two sends. Keep one root receiver idle
while a child subscriber drains every produced value, then prove
the lag error reports the exact overrun and positions the root at
the oldest value still retained by `BroadcastState.queue`.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(size)
brx = broadcast_receiver(rx, size)
async with brx.subscribe() as fast:
for value in range(sent):
await tx.send(value)
assert await fast.receive() == value
match = rf'dropped `{dropped}` values'
with pytest.raises(Lagged, match=match):
await brx.receive()
assert await brx.receive() == sent - size
trio.run(main)
def test_broadcast_statistics_report_queued_counts() -> None:
'''
`BroadcastState.statistics()` must report counts, not indexes.
Each `BroadcastState.subs` value is the deque index of a
receiver's next unread value, with `-1` meaning caught up. The
statistics API returned these indexes directly, so one queued
value appeared as zero and every positive count was one short.
Keep one root receiver idle while a child synchronously receives
four produced values. Prove the root count advances through one
and three retained values, then remains clamped to the three-slot
retention window after lagging.
Finally install an actual unwaited `trio.Event` in
`BroadcastState.recv_ready` while treating deprecations as errors.
This proves statistics checks `None` explicitly instead of using
deprecated `trio.Event` truthiness.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(3)
brx = broadcast_receiver(rx, 3)
async with brx.subscribe() as child:
state = brx._state
assert state.statistics()['queued_len_by_task'] == {
brx.key: 0,
child.key: 0,
}
await tx.send(0)
assert await child.receive() == 0
assert state.statistics()['queued_len_by_task'] == {
brx.key: 1,
child.key: 0,
}
for value in range(1, 4):
await tx.send(value)
assert await child.receive() == value
state.recv_ready = (child.key, trio.Event())
with warnings.catch_warnings():
warnings.simplefilter('error', DeprecationWarning)
stats = state.statistics()
assert stats['queued_len_by_task'] == {
brx.key: 3,
child.key: 0,
}
assert stats['tasks_waiting'] == 0
state.recv_ready = None
trio.run(main)
def test_cancelled_reader_diagnostics_are_transient() -> None:
'''
Cancelled-reader diagnostics must not retain stale `Task`s.
`BroadcastState.cancelled` previously accumulated every source
owner cancelled during `BroadcastReceiver.receive()`. Even after
that receiver successfully read again or its subscription closed,
`BroadcastState.statistics()` retained the old `Task`, reporting
stale state and keeping the completed task alive.
Cancel one child's source read under a receiver-local scope and
verify its task is reported. Reuse that same receiver for one
successful read to prove progress clears the entry. Cancel it once
more, then leave the subscription and prove close also removes the
diagnostic while the root receiver remains registered.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(1)
brx = broadcast_receiver(rx, 1)
cancel_scope = trio.CancelScope()
child_key: int
child_task = None
async with brx.subscribe() as child:
child_key = child.key
async def cancel_source_read() -> None:
nonlocal child_task
child_task = current_task()
with cancel_scope:
await child.receive()
assert cancel_scope.cancelled_caught
async with trio.open_nursery() as nursery:
nursery.start_soon(cancel_source_read)
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
cancel_scope.cancel()
stats = brx._state.statistics()
assert child_task is not None
assert stats['tasks_cancelled'] == {
child_key: child_task,
}
await tx.send(1)
assert await child.receive() == 1
assert not brx._state.cancelled
cancel_scope = trio.CancelScope()
async with trio.open_nursery() as nursery:
nursery.start_soon(cancel_source_read)
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
cancel_scope.cancel()
assert child_key in brx._state.cancelled
assert child_key not in brx._state.cancelled
assert brx.key in brx._state.subs
trio.run(main)
@pytest.mark.parametrize(
'terminal_exc',
[
trio.EndOfChannel(),
RuntimeError('terminal source failure'),
],
ids=['end-of-channel', 'receive-error'],
)
def test_terminal_broadcast_clears_cancelled_tasks(
terminal_exc: Exception,
) -> None:
'''
Terminal broadcast state must release every cancelled `Task`.
A receiver which owned and cancelled a source read can leave its
task in `BroadcastState.cancelled`. If another receiver later gets
EOC or a terminal source failure, no subscriber can make source
progress to clear that stale diagnostic. Clearing only the terminal
owner's key therefore retained the first receiver's completed task.
Cancel a child during the first controlled source read, then let
the root own a second read which raises EOC or `RuntimeError`.
Prove each terminal path clears the other receiver's diagnostic
before propagating its exact source outcome.
'''
class TerminalReceiver:
'''
Block one cancellable read, then raise a terminal outcome.
'''
def __init__(self) -> None:
self.calls = 0
self.first_started = trio.Event()
async def receive(self) -> None:
'''
Drive cancellation followed by terminal source state.
'''
self.calls += 1
if self.calls == 1:
self.first_started.set()
await trio.sleep_forever()
raise terminal_exc
async def main() -> None:
source = TerminalReceiver()
brx = broadcast_receiver(source, 1)
cancel_scope = trio.CancelScope()
async with brx.subscribe() as child:
async def cancel_child_read() -> None:
with cancel_scope:
await child.receive()
assert cancel_scope.cancelled_caught
async with trio.open_nursery() as nursery:
nursery.start_soon(cancel_child_read)
await source.first_started.wait()
cancel_scope.cancel()
assert child.key in brx._state.cancelled
with pytest.raises(type(terminal_exc)) as exc_info:
await brx.receive()
assert exc_info.value is terminal_exc
assert not brx._state.cancelled
trio.run(main)
def test_end_of_channel_is_terminal_for_waiting_peer() -> None:
'''
EOC must not let an awakened peer re-enter the closed source.
`BroadcastState.eoc` was set when one source owner received EOC,
but neither receive path consulted it. A peer waiting behind that
owner therefore woke, saw no queued value, and started a second
source read. Cancellation at that checkpoint could repopulate
`BroadcastState.cancelled` after the broadcast became terminal.
Block one child in the sole source read while the root waits on its
event, then release EOC. Both receivers must terminate from that
one source call, and the root's later receive must replay EOC
immediately without retaining cancellation diagnostics.
'''
class EOCReceiver:
'''
Publish one controlled EOC and reject any second source read.
'''
def __init__(self) -> None:
self.calls = 0
self.started = trio.Event()
self.release = trio.Event()
async def receive(self) -> None:
'''
Block the only valid source read until EOC release.
'''
self.calls += 1
assert self.calls == 1
self.started.set()
await self.release.wait()
raise trio.EndOfChannel
async def main() -> None:
source = EOCReceiver()
brx = broadcast_receiver(source, 1)
outcomes: list[str] = []
async with brx.subscribe() as child:
async def receive_eoc(
receiver,
name: str,
) -> None:
with pytest.raises(trio.EndOfChannel):
await receiver.receive()
outcomes.append(name)
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_eoc, child, 'child')
await source.started.wait()
nursery.start_soon(receive_eoc, brx, 'root')
_, event = brx._state.recv_ready
while not event.statistics().tasks_waiting:
await trio.lowlevel.checkpoint()
source.release.set()
with pytest.raises(trio.EndOfChannel):
await brx.receive()
assert sorted(outcomes) == ['child', 'root']
assert source.calls == 1
assert not brx._state.cancelled
trio.run(main)
def test_msgstream_eoc_close_preserves_aclose_override() -> None:
'''
Internal EOC cleanup must preserve the public `aclose()` contract.
Passing a new private keyword from `MsgStream.receive()` to
`self.aclose()` broke subclasses whose compatible override kept
the original zero-argument signature. Use a minimal subclass which
records virtual dispatch and delegates to the base implementation.
Drive graceful EOC through the real root broadcaster and prove the
override runs without closing that active root re-entrantly.
'''
class Stream(tractor.MsgStream):
'''
Record public close dispatch with the established signature.
'''
close_calls = 0
async def aclose(self):
'''
Delegate closure without accepting private arguments.
'''
self.close_calls += 1
return await super().aclose()
class PldRx:
'''
Delegate source receive and terminate the close drain.
'''
def __init__(self, rx) -> None:
self._rx = rx
async def recv_pld(self, **kwargs):
'''
Receive directly from the test source channel.
'''
return await self._rx.receive()
def recv_msg_nowait(self, **kwargs):
'''
Report EOC to finish `MsgStream.aclose()` draining.
'''
raise trio.EndOfChannel
async def main() -> None:
tx, rx = trio.open_memory_channel(1)
ctx = SimpleNamespace(
cid='test-context',
_pld_rx=PldRx(rx),
send_stop=lambda: trio.lowlevel.checkpoint(),
side='caller',
peer_side='callee',
maybe_raise=lambda **kwargs: None,
)
stream = Stream(ctx, rx)
async with stream.subscribe():
await tx.aclose()
with pytest.raises(trio.EndOfChannel):
await stream.receive()
assert stream.close_calls == 1
assert not stream._broadcaster._closed
trio.run(main)
@pytest.mark.parametrize(
'close_wrapper',
[
tractor.MsgStream.aclose,
LinkedTaskChannel.aclose,
],
ids=['msg-stream', 'linked-task-channel'],
)
def test_wrapper_close_clears_root_cancelled_task(
close_wrapper,
) -> None:
'''
Public stream close must release root cancellation diagnostics.
Root broadcasters allocated by `MsgStream.subscribe()` and
`LinkedTaskChannel.subscribe()` are private implementation state.
If their source receive was cancelled, callers had no public way
to close the root, so wrapper teardown retained the completed
`Task` in `BroadcastState.cancelled` indefinitely.
Cancel a root source read, attach that broadcaster to a minimal
public wrapper, and close it through each real `aclose()` method.
The root receiver and its task diagnostic must both be removed;
for `MsgStream`, pre-close the source to cover its idempotent early
return path.
'''
async def main() -> None:
_, rx = trio.open_memory_channel(1)
brx = broadcast_receiver(rx, 1)
cancel_scope = trio.CancelScope()
async def cancel_source_read() -> None:
with cancel_scope:
await brx.receive()
assert cancel_scope.cancelled_caught
async with trio.open_nursery() as nursery:
nursery.start_soon(cancel_source_read)
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
cancel_scope.cancel()
assert brx.key in brx._state.cancelled
if close_wrapper is tractor.MsgStream.aclose:
ctx = SimpleNamespace(cid='test-context')
wrapper = tractor.MsgStream(ctx, rx)
wrapper._broadcaster = brx
await rx.aclose()
else:
wrapper = SimpleNamespace(
_broadcaster=brx,
_from_aio=rx,
)
await close_wrapper(wrapper)
assert brx.key not in brx._state.subs
assert brx.key not in brx._state.cancelled
trio.run(main)
def test_broadcast_rejects_zero_buffer_size() -> None:
'''
A broadcaster must retain at least one value for peer fan-out.
`collections.deque(maxlen=0)` silently discards every appended
value, so `broadcast_receiver(..., 0)` allowed the source owner to
receive while peer cursors advanced into an always-empty queue.
Their lag recovery then reset to index `-1` and recursively retried
without any retained value to consume.
Construct a rendezvous memory channel and prove broadcaster setup
rejects its zero capacity synchronously with a clear public error,
before any receiver is registered or source receive can begin.
'''
_, rx = trio.open_memory_channel(0)
with pytest.raises(
ValueError,
match='`max_buffer_size` must be greater than zero',
):
broadcast_receiver(rx, 0)
def test_underlying_receive_failure_wakes_all_subscribers() -> None:
'''
A shared receive failure must terminate every broadcast receiver.
Previously, only `EndOfChannel` and receiver cancellation woke
peer tasks waiting on `BroadcastState.recv_ready`. If the shared
underlying receiver raised another error, its owner propagated
the failure and cleared the event while every peer remained
blocked forever.
Script one successful receive followed by a controlled
`RuntimeError`. Let a fast child own both underlying receives
while the root first drains its retained value and then waits on
the child's second receive. Release the failure only after both
tasks have reached those positions. Both exact errors prove the
peer was awakened without losing buffered data. A later
subscriber proves the terminal failure remains published for new
receivers instead of retrying the failed underlying channel.
'''
class FailingReceiver:
'''
Return one value, then fail after deterministic release.
'''
def __init__(self) -> None:
self.calls: int = 0
self.failure_started = trio.Event()
self.release_failure = trio.Event()
async def receive(self) -> int:
'''
Drive the scripted success-then-failure sequence.
'''
self.calls += 1
if self.calls == 1:
return 1
self.failure_started.set()
await self.release_failure.wait()
raise RuntimeError('underlying receive failed')
async def main() -> None:
source = FailingReceiver()
brx = broadcast_receiver(source, 3)
child_error: list[RuntimeError] = []
root_error: list[BroadcastReceiveError] = []
late_error: list[BroadcastReceiveError] = []
root_drained = trio.Event()
async def receive_child() -> None:
async with brx.subscribe() as child:
assert await child.receive() == 1
try:
await child.receive()
except RuntimeError as exc:
child_error.append(exc)
async def receive_root() -> None:
assert await brx.receive() == 1
root_drained.set()
try:
await brx.receive()
except BroadcastReceiveError as exc:
root_error.append(exc)
with trio.fail_after(1):
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_child)
await source.failure_started.wait()
nursery.start_soon(receive_root)
await root_drained.wait()
source.release_failure.set()
assert source.calls == 2
assert [str(exc) for exc in child_error] == [
'underlying receive failed',
]
assert [str(exc) for exc in root_error] == [
'Shared broadcast receiver failed',
]
assert child_error[0] is not root_error[0]
assert root_error[0].__cause__ is child_error[0]
async with brx.subscribe() as late:
with pytest.raises(
BroadcastReceiveError,
match='Shared broadcast receiver failed',
) as exc_info:
await late.receive()
late_error.append(exc_info.value)
assert late_error[0] is not child_error[0]
assert late_error[0] is not root_error[0]
assert late_error[0].__cause__ is child_error[0]
assert source.calls == 2
trio.run(main)
def test_control_flow_exit_wakes_broadcast_peer() -> None:
'''
Non-terminal control flow must wake peers without being retained.
Process-control and cancellation-like `BaseException` values
should remain local to the task which receives them, but the old
owner still has to wake subscribers blocked on its shared event.
Make one child own a controlled `BaseException` receive while the
root waits behind it. After release, prove the child gets that
exact exit and the root takes ownership of the next underlying
receive instead of hanging or replaying the control-flow event.
'''
class ReceiveExit(BaseException):
'''
Model a non-terminal process-control receive exit.
'''
class ControlFlowReceiver:
'''
Raise one controlled exit, then return a value.
'''
def __init__(self) -> None:
self.calls: int = 0
self.exit_started = trio.Event()
self.release_exit = trio.Event()
async def receive(self) -> int:
'''
Drive the scripted control-flow-then-value sequence.
'''
self.calls += 1
if self.calls == 1:
self.exit_started.set()
await self.release_exit.wait()
raise ReceiveExit
return 2
async def main() -> None:
source = ControlFlowReceiver()
brx = broadcast_receiver(source, 3)
child_exit: list[ReceiveExit] = []
root_value: list[int] = []
async def receive_child() -> None:
async with brx.subscribe() as child:
try:
await child.receive()
except ReceiveExit as exc:
child_exit.append(exc)
async def receive_root() -> None:
root_value.append(await brx.receive())
with trio.fail_after(1):
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_child)
await source.exit_started.wait()
nursery.start_soon(receive_root)
while True:
_, event = brx._state.recv_ready
if event.statistics().tasks_waiting:
break
await trio.lowlevel.checkpoint()
source.release_exit.set()
assert len(child_exit) == 1
assert root_value == [2]
assert source.calls == 2
assert brx._state.receive_exc is None
trio.run(main)
def test_closing_non_owner_preserves_source_wait() -> None:
'''
Closing one subscriber must not wake another receiver's peers.
`BroadcastReceiver.aclose()` previously set the one shared
`BroadcastState.recv_ready` event even when a different receiver
owned the source read. Waiting peers then repeatedly awaited an
already-set event until the source produced another value,
creating a runnable hot loop on idle streams.
Block one child in the source receive, then place both the root
and a closing child behind its event. Close only that waiting
child and prove it gets `ClosedResourceError` without setting the
shared event. Both remaining receivers must still get the same
value after the source is released.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(1)
brx = broadcast_receiver(rx, 3)
owner_value: list[int] = []
root_value: list[int] = []
closing_closed = trio.Event()
async with (
brx.subscribe() as owner,
brx.subscribe() as closing,
):
async def receive_owner() -> None:
owner_value.append(await owner.receive())
async def receive_root() -> None:
root_value.append(await brx.receive())
async def receive_closing() -> None:
with pytest.raises(trio.ClosedResourceError):
await closing.receive()
closing_closed.set()
with trio.fail_after(1):
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_owner)
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
nursery.start_soon(receive_root)
nursery.start_soon(receive_closing)
_, event = brx._state.recv_ready
while event.statistics().tasks_waiting < 2:
await trio.lowlevel.checkpoint()
await closing.aclose()
await closing_closed.wait()
assert not event.is_set()
await tx.send(1)
assert owner_value == [1]
assert root_value == [1]
trio.run(main)
def test_concurrent_receive_raises_busy() -> None:
'''
Reject concurrent receives on one broadcast handle.
A receiver stores one private peer-wait cancellation scope. If two
tasks receive through the same handle, the second task can replace
that scope and prevent `BroadcastReceiver.aclose()` from waking the
first task. Block one task in the shared source receive, then prove
a second call raises `BusyResourceError` before it can mutate any
per-receiver wait state. Releasing the source proves the original
receive remains usable.
'''
async def main() -> None:
tx, rx = trio.open_memory_channel(1)
brx = broadcast_receiver(rx, 1)
values: list[int] = []
async def receive() -> None:
values.append(await brx.receive())
async with trio.open_nursery() as nursery:
nursery.start_soon(
receive,
name='first broadcast consumer',
)
# Synchronize with the background task after it blocks in
# the shared source `.receive()`, ensuring the next call is
# concurrent with an already-active receive on this handle.
while brx._state.recv_ready is None:
await trio.lowlevel.checkpoint()
with pytest.raises(
trio.BusyResourceError,
match='first broadcast consumer',
):
await brx.receive()
await tx.send(1)
assert values == [1]
trio.run(main)
@pytest.mark.parametrize(
'first_outcome',
[
1,
RuntimeError('discarded source error'),
trio.EndOfChannel(),
],
)
def test_closing_source_owner_hands_read_to_peer(
first_outcome: int|Exception,
) -> None:
'''
Closing the source-read owner must transfer ownership to a peer.
Merely suppressing the old shared-event wake would leave peers
blocked behind an externally closed receiver that still owned an
idle source read. Script a first receive which blocks until its
private scope is cancelled and a second which returns immediately.
Close that owner only after the root is waiting behind it. Cover
a shielded value, ordinary error and EOC from the cancelled source
read. The owner must always get `ClosedResourceError`, while the
awakened root takes the second source read without publishing the
discarded source outcome.
'''
class HandoffReceiver:
'''
Block the first source read and satisfy the second.
'''
def __init__(self) -> None:
self.calls: int = 0
self.first_started = trio.Event()
self.release_first = trio.Event()
async def receive(self) -> int:
'''
Drive one cancelled read followed by one value.
'''
self.calls += 1
if self.calls == 1:
self.first_started.set()
with trio.CancelScope(shield=True):
await self.release_first.wait()
if isinstance(first_outcome, BaseException):
raise first_outcome
return first_outcome
return 2
async def main() -> None:
source = HandoffReceiver()
brx = broadcast_receiver(source, 3)
owner_closed = trio.Event()
root_value: list[int] = []
async with brx.subscribe() as owner:
async def receive_owner() -> None:
with pytest.raises(trio.ClosedResourceError):
await owner.receive()
owner_closed.set()
async def receive_root() -> None:
root_value.append(await brx.receive())
with trio.fail_after(1):
async with trio.open_nursery() as nursery:
nursery.start_soon(receive_owner)
await source.first_started.wait()
nursery.start_soon(receive_root)
_, event = brx._state.recv_ready
while not event.statistics().tasks_waiting:
await trio.lowlevel.checkpoint()
await owner.aclose()
source.release_first.set()
await owner_closed.wait()
assert source.calls == 2
assert root_value == [2]
assert brx._state.receive_exc is None
assert not brx._state.eoc
trio.run(main)
def test_ensure_slow_consumers_lag_out(
reg_addr,
start_method,
@ -448,6 +1293,7 @@ def test_first_recver_is_cancelled():
async with brx.subscribe() as bc:
async for value in bc:
print(value)
assert cs.cancelled_caught
async def cancel_and_send():
await trio.sleep(0.2)
@ -519,3 +1365,74 @@ def test_no_raise_on_lag():
with pytest.raises(KeyboardInterrupt):
trio.run(main)
@pytest.mark.parametrize(
('subscribe', 'chan_attr'),
[
(tractor.MsgStream.subscribe, '_rx_chan'),
(LinkedTaskChannel.subscribe, '_from_aio'),
],
ids=['msg-stream', 'linked-task-channel'],
)
def test_stream_subscribe_forwards_lag_policy(
subscribe,
chan_attr: str,
) -> None:
'''
Stream wrappers must expose per-subscriber lag policy.
`MsgStream.subscribe()` and `LinkedTaskChannel.subscribe()`
previously omitted `BroadcastReceiver.raise_on_lag`, forcing
downstream users to mutate a private receiver attribute. Invoke
each public wrapper against a minimal receive-compatible handle.
Prove the first non-raising subscription configures both the
irreversible root broadcaster and its child, while a later strict
child selects its own policy without changing that root.
'''
class StreamHandle:
'''
Provide the wrapper fields needed for local fan-out.
'''
def __init__(self) -> None:
self._broadcaster = None
setattr(
self,
chan_attr,
SimpleNamespace(
_state=SimpleNamespace(max_buffer_size=1),
),
)
async def receive(self):
'''
Block if a regression unexpectedly enters source receive.
'''
await trio.sleep_forever()
async def send(self, value) -> None:
'''
Satisfy `MsgStream` duplex-handle patching.
'''
async def main() -> None:
stream = StreamHandle()
async with subscribe(
stream,
raise_on_lag=False,
) as first:
assert not stream._broadcaster._raise_on_lag
assert not first._raise_on_lag
async with subscribe(
stream,
raise_on_lag=True,
) as second:
assert not stream._broadcaster._raise_on_lag
assert second._raise_on_lag
trio.run(main)

View File

@ -26,7 +26,10 @@ from ast import literal_eval
from typing import TYPE_CHECKING
from .runtime._runtime import Actor
from .spawn._entry import _trio_main
from .spawn._entry import (
_consume_netns_bootstrap,
_trio_main,
)
if TYPE_CHECKING:
from .discovery._addr import UnwrappedAddress
@ -46,12 +49,41 @@ def parse_ipaddr(arg):
return arg
def parse_netns_bootstrap(arg: str) -> tuple[int, int]:
'''
Parse one atomic inherited namespace capability.
Descriptor and inode validation remains in
`_consume_netns_bootstrap()` so every valid descriptor-shaped
input reaches its exact child-owned cleanup boundary.
'''
try:
value: object = literal_eval(arg)
except (ValueError, SyntaxError) as exc:
raise argparse.ArgumentTypeError(
'netns bootstrap must be an `(fd, inode)` tuple'
) from exc
if (
not isinstance(value, tuple)
or
len(value) != 2
):
raise argparse.ArgumentTypeError(
'netns bootstrap must be an `(fd, inode)` tuple'
)
return value
def _actor_child_main(
uid: tuple[str, str],
loglevel: str | None,
parent_addr: UnwrappedAddress | None,
infect_asyncio: bool,
spawn_method: SpawnMethodKey = 'trio',
netns_bootstrap: tuple[int, int]|None = None,
) -> None:
'''
@ -62,7 +94,13 @@ def _actor_child_main(
invokes this from inside a fresh `concurrent.interpreters`
sub-interpreter via `Interpreter.call()`.
Consume `netns_bootstrap` before Trio patching, actor construction,
process-title setup, or actor-runtime entry. The spawn backend must
supply an exclusively child-owned FD duplicate.
'''
_consume_netns_bootstrap(netns_bootstrap)
# Apply defensive monkey-patches for upstream `trio`
# bugs we've encountered while running tractor — see
# `tractor.trionics.patches` for the catalog +
@ -113,7 +151,11 @@ def _actor_child_main(
)
if __name__ == "__main__":
def main(argv: list[str]|None = None) -> None:
'''
Parse Trio child-bootstrap arguments and enter actor runtime.
'''
__tracebackhide__: bool = True
parser = argparse.ArgumentParser()
@ -121,7 +163,11 @@ if __name__ == "__main__":
parser.add_argument("--loglevel", type=str)
parser.add_argument("--parent_addr", type=parse_ipaddr)
parser.add_argument("--asyncio", action='store_true')
args = parser.parse_args()
parser.add_argument(
'--netns_bootstrap',
type=parse_netns_bootstrap,
)
args = parser.parse_args(argv)
_actor_child_main(
uid=args.uid,
@ -129,4 +175,9 @@ if __name__ == "__main__":
parent_addr=args.parent_addr,
infect_asyncio=args.asyncio,
spawn_method='trio',
netns_bootstrap=args.netns_bootstrap,
)
if __name__ == "__main__":
main()

Some files were not shown because too many files have changed in this diff Show More