1
0
Fork 0
tractor/tractor/_addr.py

283 lines
7.8 KiB
Python
Raw Normal View History

# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from __future__ import annotations
from uuid import uuid4
from typing import (
Protocol,
ClassVar,
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
Type,
TYPE_CHECKING,
)
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
from bidict import bidict
from trio import (
SocketListener,
)
from .log import get_logger
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
from ._state import (
_def_tpt_proto,
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
)
from .ipc._tcp import TCPAddress
from .ipc._uds import UDSAddress
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
if TYPE_CHECKING:
from ._runtime import Actor
log = get_logger(__name__)
# TODO, maybe breakout the netns key to a struct?
# class NetNs(Struct)[str, int]:
# ...
# TODO, can't we just use a type alias
# for this? namely just some `tuple[str, int, str, str]`?
#
# -[ ] would also just be simpler to keep this as SockAddr[tuple]
# or something, implying it's just a simple pair of values which can
# presumably be mapped to all transports?
# -[ ] `pydoc socket.socket.getsockname()` delivers a 4-tuple for
# ipv6 `(hostaddr, port, flowinfo, scope_id)`.. so how should we
# handle that?
# -[ ] as a further alternative to this wrap()/unwrap() approach we
# could just implement `enc/dec_hook()`s for the `Address`-types
# and just deal with our internal objs directly and always and
# leave it to the codec layer to figure out marshalling?
# |_ would mean only one spot to do the `.unwrap()` (which we may
# end up needing to call from the hook()s anyway?)
# -[x] rename to `UnwrappedAddress[Descriptor]` ??
# seems like the right name as per,
# https://www.geeksforgeeks.org/introduction-to-address-descriptor/
#
UnwrappedAddress = (
# tcp/udp/uds
tuple[
str, # host/domain(tcp), filesys-dir(uds)
int|str, # port/path(uds)
]
# ?TODO? should we also include another 2 fields from
# our `Aid` msg such that we include the runtime `Actor.uid`
# of `.name` and `.uuid`?
# - would ensure uniqueness across entire net?
# - allows for easier runtime-level filtering of "actors by
# service name"
)
# TODO, maybe rename to `SocketAddress`?
class Address(Protocol):
proto_key: ClassVar[str]
unwrapped_type: ClassVar[UnwrappedAddress]
# TODO, i feel like an `.is_bound()` is a better thing to
# support?
# Lke, what use does this have besides a noop and if it's not
# valid why aren't we erroring on creation/use?
@property
def is_valid(self) -> bool:
...
# TODO, maybe `.netns` is a better name?
@property
def namespace(self) -> tuple[str, int]|None:
'''
The if-available, OS-specific "network namespace" key.
'''
...
@property
def bindspace(self) -> str:
'''
Deliver the socket address' "bindable space" from
a `socket.socket.bind()` and thus from the perspective of
specific transport protocol domain.
I.e. for most (layer-4) network-socket protocols this is
normally the ipv4/6 address, for UDS this is normally
a filesystem (sub-directory).
For (distributed) network protocols this is normally the routing
layer's domain/(ip-)address, though it might also include a "network namespace"
key different then the default.
For local-host-only transports this is either an explicit
namespace (with types defined by the OS: netns, Cgroup, IPC,
pid, etc. on linux) or failing that the sub-directory in the
filesys in which socket/shm files are located *under*.
'''
...
@classmethod
def from_addr(cls, addr: UnwrappedAddress) -> Address:
...
def unwrap(self) -> UnwrappedAddress:
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
'''
Deliver the underying minimum field set in
a primitive python data type-structure.
'''
...
@classmethod
def get_random(
cls,
current_actor: Actor,
bindspace: str|None = None,
) -> Address:
...
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
# TODO, this should be something like a `.get_def_registar_addr()`
# or similar since,
# - it should be a **host singleton** (not root/tree singleton)
# - we **only need this value** when one isn't provided to the
# runtime at boot and we want to implicitly provide a host-wide
# registrar.
# - each rooted-actor-tree should likely have its own
# micro-registry (likely the root being it), also see
@classmethod
def get_root(cls) -> Address:
...
def __repr__(self) -> str:
...
def __eq__(self, other) -> bool:
...
async def open_listener(
self,
**kwargs,
) -> SocketListener:
...
async def close_listener(self):
...
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
_address_types: bidict[str, Type[Address]] = {
'tcp': TCPAddress,
'uds': UDSAddress
}
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
# TODO! really these are discovery sys default addrs ONLY useful for
# when none is provided to a root actor on first boot.
_default_lo_addrs: dict[
str,
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
UnwrappedAddress
] = {
'tcp': TCPAddress.get_root().unwrap(),
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
'uds': UDSAddress.get_root().unwrap(),
}
def get_address_cls(name: str) -> Type[Address]:
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
return _address_types[name]
def is_wrapped_addr(addr: any) -> bool:
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
return type(addr) in _address_types.values()
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
def mk_uuid() -> str:
'''
Encapsulate creation of a uuid4 as `str` as used
for creating `Actor.uid: tuple[str, str]` and/or
`.msg.types.Aid`.
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
'''
return str(uuid4())
def wrap_address(
addr: UnwrappedAddress
) -> Address:
More `._addr` boxing refinements The more I think about it, it seems @guille's orig approach of unwrapping UDS socket-file addresses to strings (or `Path`) is making the most sense. I had originally thought that pairing it with the listening side's pid would add clarity (and it definitely does for introspection/debug/logging) but since we don't end up passing that pid to the eventual `.connect()` call on the client side, it doesn't make much sense to wrap it for the wire just to discard.. Further, the `tuple[str, int]` makes `wrap_address()` break for TCP since it will always match on uds first. So, on that note this patch refines a few things in prep for going back to that original `UnwrappedAddress` as `str` type though longer run i think the more "builtin approach" would be to add `msgspec` codec hooks for these types to avoid all the `.wrap()`/`.unwrap()` calls throughout the runtime. Down-low deats, - add `wrap_address()` doc string, detailed (todo) comments and handle the `[None, None]` case that can come directly from `._state._runtime_vars['_root_mailbox']`. - buncha adjustments to `UDSAddress`, - add a `filedir`, chng `filepath` -> `filename` and mk `maybe_pid` optional. - the intent `filedir` is act as the equivalent of the host part in a network proto's socket address and when it's null use the `.def_bindspace = get_rt_dir()`. - always ensure the `filedir / filename` is an absolute path and expose it as a new `.sockpath: Path` property. - mk `.is_valid` actually verify the `.sockpath` is in the valid `.bindspace: namely just checking it's in the expected dir. - add pedantic `match:`ing to `.from_addr()` such that we error on unexpected `type(addr)` inputs and otherwise parse any `sockpath: Path` inputs using a new `unwrap_sockpath()` which simply splits an abs file path to dir, file-name parts. - `.unwrap()` now just `str`-ifies the `.sockpath: Path` - adjust `.open/close_listener()` to use `.sockpath`.
2025-04-03 14:10:18 +00:00
'''
Wrap an `UnwrappedAddress` as an `Address`-type based
on matching builtin python data-structures which we adhoc
use for each.
XXX NOTE, careful care must be placed to ensure
`UnwrappedAddress` cases are **definitely unique** otherwise the
wrong transport backend may be loaded and will break many
low-level things in our runtime in a not-fun-to-debug way!
More `._addr` boxing refinements The more I think about it, it seems @guille's orig approach of unwrapping UDS socket-file addresses to strings (or `Path`) is making the most sense. I had originally thought that pairing it with the listening side's pid would add clarity (and it definitely does for introspection/debug/logging) but since we don't end up passing that pid to the eventual `.connect()` call on the client side, it doesn't make much sense to wrap it for the wire just to discard.. Further, the `tuple[str, int]` makes `wrap_address()` break for TCP since it will always match on uds first. So, on that note this patch refines a few things in prep for going back to that original `UnwrappedAddress` as `str` type though longer run i think the more "builtin approach" would be to add `msgspec` codec hooks for these types to avoid all the `.wrap()`/`.unwrap()` calls throughout the runtime. Down-low deats, - add `wrap_address()` doc string, detailed (todo) comments and handle the `[None, None]` case that can come directly from `._state._runtime_vars['_root_mailbox']`. - buncha adjustments to `UDSAddress`, - add a `filedir`, chng `filepath` -> `filename` and mk `maybe_pid` optional. - the intent `filedir` is act as the equivalent of the host part in a network proto's socket address and when it's null use the `.def_bindspace = get_rt_dir()`. - always ensure the `filedir / filename` is an absolute path and expose it as a new `.sockpath: Path` property. - mk `.is_valid` actually verify the `.sockpath` is in the valid `.bindspace: namely just checking it's in the expected dir. - add pedantic `match:`ing to `.from_addr()` such that we error on unexpected `type(addr)` inputs and otherwise parse any `sockpath: Path` inputs using a new `unwrap_sockpath()` which simply splits an abs file path to dir, file-name parts. - `.unwrap()` now just `str`-ifies the `.sockpath: Path` - adjust `.open/close_listener()` to use `.sockpath`.
2025-04-03 14:10:18 +00:00
XD
'''
if is_wrapped_addr(addr):
return addr
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
cls: Type|None = None
More `._addr` boxing refinements The more I think about it, it seems @guille's orig approach of unwrapping UDS socket-file addresses to strings (or `Path`) is making the most sense. I had originally thought that pairing it with the listening side's pid would add clarity (and it definitely does for introspection/debug/logging) but since we don't end up passing that pid to the eventual `.connect()` call on the client side, it doesn't make much sense to wrap it for the wire just to discard.. Further, the `tuple[str, int]` makes `wrap_address()` break for TCP since it will always match on uds first. So, on that note this patch refines a few things in prep for going back to that original `UnwrappedAddress` as `str` type though longer run i think the more "builtin approach" would be to add `msgspec` codec hooks for these types to avoid all the `.wrap()`/`.unwrap()` calls throughout the runtime. Down-low deats, - add `wrap_address()` doc string, detailed (todo) comments and handle the `[None, None]` case that can come directly from `._state._runtime_vars['_root_mailbox']`. - buncha adjustments to `UDSAddress`, - add a `filedir`, chng `filepath` -> `filename` and mk `maybe_pid` optional. - the intent `filedir` is act as the equivalent of the host part in a network proto's socket address and when it's null use the `.def_bindspace = get_rt_dir()`. - always ensure the `filedir / filename` is an absolute path and expose it as a new `.sockpath: Path` property. - mk `.is_valid` actually verify the `.sockpath` is in the valid `.bindspace: namely just checking it's in the expected dir. - add pedantic `match:`ing to `.from_addr()` such that we error on unexpected `type(addr)` inputs and otherwise parse any `sockpath: Path` inputs using a new `unwrap_sockpath()` which simply splits an abs file path to dir, file-name parts. - `.unwrap()` now just `str`-ifies the `.sockpath: Path` - adjust `.open/close_listener()` to use `.sockpath`.
2025-04-03 14:10:18 +00:00
# if 'sock' in addr[0]:
# import pdbp; pdbp.set_trace()
match addr:
More `._addr` boxing refinements The more I think about it, it seems @guille's orig approach of unwrapping UDS socket-file addresses to strings (or `Path`) is making the most sense. I had originally thought that pairing it with the listening side's pid would add clarity (and it definitely does for introspection/debug/logging) but since we don't end up passing that pid to the eventual `.connect()` call on the client side, it doesn't make much sense to wrap it for the wire just to discard.. Further, the `tuple[str, int]` makes `wrap_address()` break for TCP since it will always match on uds first. So, on that note this patch refines a few things in prep for going back to that original `UnwrappedAddress` as `str` type though longer run i think the more "builtin approach" would be to add `msgspec` codec hooks for these types to avoid all the `.wrap()`/`.unwrap()` calls throughout the runtime. Down-low deats, - add `wrap_address()` doc string, detailed (todo) comments and handle the `[None, None]` case that can come directly from `._state._runtime_vars['_root_mailbox']`. - buncha adjustments to `UDSAddress`, - add a `filedir`, chng `filepath` -> `filename` and mk `maybe_pid` optional. - the intent `filedir` is act as the equivalent of the host part in a network proto's socket address and when it's null use the `.def_bindspace = get_rt_dir()`. - always ensure the `filedir / filename` is an absolute path and expose it as a new `.sockpath: Path` property. - mk `.is_valid` actually verify the `.sockpath` is in the valid `.bindspace: namely just checking it's in the expected dir. - add pedantic `match:`ing to `.from_addr()` such that we error on unexpected `type(addr)` inputs and otherwise parse any `sockpath: Path` inputs using a new `unwrap_sockpath()` which simply splits an abs file path to dir, file-name parts. - `.unwrap()` now just `str`-ifies the `.sockpath: Path` - adjust `.open/close_listener()` to use `.sockpath`.
2025-04-03 14:10:18 +00:00
# classic network socket-address as tuple/list
case (
(str(), int())
|
[str(), int()]
):
cls = TCPAddress
Unwrap `UDSAddress` as `tuple[str, str]`, i.e. sin pid Since in hindsight the real analog of a net-proto's "bindspace" (normally its routing layer's addresses-port-set) is more akin to the "location in the file-system" for a UDS socket file (aka the file's parent directory) determines whether or not the "port" (aka it's file-name) collides with any other. So the `._filedir: Path` is like the allocated "address" and, the `._filename: Path|str` is basically the "port", at least in my mind.. Bp Thinking about fs dirs like a "host address" means you can get essentially the same benefits/behaviour of say an (ip) addresses-port-space but using the (current process-namespace's) filesys-tree. Note that for UDS sockets in particular the network-namespace is what would normally isolate so called "abstract sockets" (i.e. UDS sockets that do NOT use file-paths by setting `struct sockaddr_un.sun_path = 'abstract', see `man unix`); using directories is even easier and definitely more explicit/readable/immediately-obvious as a human-user. As such this reworks all the necessary `UDSAddress` meths, - `.unwrap()` now returns a `tuple(str(._filedir, str(._filename))`, - `wrap_address()` now matches UDS on a 2nd tuple `str()` element, - `.get_root()` no longer passes `maybe_pid`. AND adjusts `MsgpackUDSStream` to, - use the new `unwrap_sockpath()` on the `socket.get[sock/peer]name()` output before passing directly as `UDSAddress.__init__(filedir, filename)` instead of via `.from_addr()`. - also pass `maybe_pid`s to init since no longer included in the unwrapped-type form.
2025-04-04 02:24:24 +00:00
case (
# (str()|Path(), str()|Path()),
# ^TODO? uhh why doesn't this work!?
(_, filename)
) if type(filename) is str:
cls = UDSAddress
More `._addr` boxing refinements The more I think about it, it seems @guille's orig approach of unwrapping UDS socket-file addresses to strings (or `Path`) is making the most sense. I had originally thought that pairing it with the listening side's pid would add clarity (and it definitely does for introspection/debug/logging) but since we don't end up passing that pid to the eventual `.connect()` call on the client side, it doesn't make much sense to wrap it for the wire just to discard.. Further, the `tuple[str, int]` makes `wrap_address()` break for TCP since it will always match on uds first. So, on that note this patch refines a few things in prep for going back to that original `UnwrappedAddress` as `str` type though longer run i think the more "builtin approach" would be to add `msgspec` codec hooks for these types to avoid all the `.wrap()`/`.unwrap()` calls throughout the runtime. Down-low deats, - add `wrap_address()` doc string, detailed (todo) comments and handle the `[None, None]` case that can come directly from `._state._runtime_vars['_root_mailbox']`. - buncha adjustments to `UDSAddress`, - add a `filedir`, chng `filepath` -> `filename` and mk `maybe_pid` optional. - the intent `filedir` is act as the equivalent of the host part in a network proto's socket address and when it's null use the `.def_bindspace = get_rt_dir()`. - always ensure the `filedir / filename` is an absolute path and expose it as a new `.sockpath: Path` property. - mk `.is_valid` actually verify the `.sockpath` is in the valid `.bindspace: namely just checking it's in the expected dir. - add pedantic `match:`ing to `.from_addr()` such that we error on unexpected `type(addr)` inputs and otherwise parse any `sockpath: Path` inputs using a new `unwrap_sockpath()` which simply splits an abs file path to dir, file-name parts. - `.unwrap()` now just `str`-ifies the `.sockpath: Path` - adjust `.open/close_listener()` to use `.sockpath`.
2025-04-03 14:10:18 +00:00
# likely an unset UDS or TCP reg address as defaulted in
# `_state._runtime_vars['_root_mailbox']`
#
# TODO? figure out when/if we even need this?
More `._addr` boxing refinements The more I think about it, it seems @guille's orig approach of unwrapping UDS socket-file addresses to strings (or `Path`) is making the most sense. I had originally thought that pairing it with the listening side's pid would add clarity (and it definitely does for introspection/debug/logging) but since we don't end up passing that pid to the eventual `.connect()` call on the client side, it doesn't make much sense to wrap it for the wire just to discard.. Further, the `tuple[str, int]` makes `wrap_address()` break for TCP since it will always match on uds first. So, on that note this patch refines a few things in prep for going back to that original `UnwrappedAddress` as `str` type though longer run i think the more "builtin approach" would be to add `msgspec` codec hooks for these types to avoid all the `.wrap()`/`.unwrap()` calls throughout the runtime. Down-low deats, - add `wrap_address()` doc string, detailed (todo) comments and handle the `[None, None]` case that can come directly from `._state._runtime_vars['_root_mailbox']`. - buncha adjustments to `UDSAddress`, - add a `filedir`, chng `filepath` -> `filename` and mk `maybe_pid` optional. - the intent `filedir` is act as the equivalent of the host part in a network proto's socket address and when it's null use the `.def_bindspace = get_rt_dir()`. - always ensure the `filedir / filename` is an absolute path and expose it as a new `.sockpath: Path` property. - mk `.is_valid` actually verify the `.sockpath` is in the valid `.bindspace: namely just checking it's in the expected dir. - add pedantic `match:`ing to `.from_addr()` such that we error on unexpected `type(addr)` inputs and otherwise parse any `sockpath: Path` inputs using a new `unwrap_sockpath()` which simply splits an abs file path to dir, file-name parts. - `.unwrap()` now just `str`-ifies the `.sockpath: Path` - adjust `.open/close_listener()` to use `.sockpath`.
2025-04-03 14:10:18 +00:00
case (
None
|
[None, None]
):
cls: Type[Address] = get_address_cls(_def_tpt_proto)
addr: UnwrappedAddress = cls.get_root().unwrap()
case _:
More `._addr` boxing refinements The more I think about it, it seems @guille's orig approach of unwrapping UDS socket-file addresses to strings (or `Path`) is making the most sense. I had originally thought that pairing it with the listening side's pid would add clarity (and it definitely does for introspection/debug/logging) but since we don't end up passing that pid to the eventual `.connect()` call on the client side, it doesn't make much sense to wrap it for the wire just to discard.. Further, the `tuple[str, int]` makes `wrap_address()` break for TCP since it will always match on uds first. So, on that note this patch refines a few things in prep for going back to that original `UnwrappedAddress` as `str` type though longer run i think the more "builtin approach" would be to add `msgspec` codec hooks for these types to avoid all the `.wrap()`/`.unwrap()` calls throughout the runtime. Down-low deats, - add `wrap_address()` doc string, detailed (todo) comments and handle the `[None, None]` case that can come directly from `._state._runtime_vars['_root_mailbox']`. - buncha adjustments to `UDSAddress`, - add a `filedir`, chng `filepath` -> `filename` and mk `maybe_pid` optional. - the intent `filedir` is act as the equivalent of the host part in a network proto's socket address and when it's null use the `.def_bindspace = get_rt_dir()`. - always ensure the `filedir / filename` is an absolute path and expose it as a new `.sockpath: Path` property. - mk `.is_valid` actually verify the `.sockpath` is in the valid `.bindspace: namely just checking it's in the expected dir. - add pedantic `match:`ing to `.from_addr()` such that we error on unexpected `type(addr)` inputs and otherwise parse any `sockpath: Path` inputs using a new `unwrap_sockpath()` which simply splits an abs file path to dir, file-name parts. - `.unwrap()` now just `str`-ifies the `.sockpath: Path` - adjust `.open/close_listener()` to use `.sockpath`.
2025-04-03 14:10:18 +00:00
# import pdbp; pdbp.set_trace()
raise TypeError(
Unwrap `UDSAddress` as `tuple[str, str]`, i.e. sin pid Since in hindsight the real analog of a net-proto's "bindspace" (normally its routing layer's addresses-port-set) is more akin to the "location in the file-system" for a UDS socket file (aka the file's parent directory) determines whether or not the "port" (aka it's file-name) collides with any other. So the `._filedir: Path` is like the allocated "address" and, the `._filename: Path|str` is basically the "port", at least in my mind.. Bp Thinking about fs dirs like a "host address" means you can get essentially the same benefits/behaviour of say an (ip) addresses-port-space but using the (current process-namespace's) filesys-tree. Note that for UDS sockets in particular the network-namespace is what would normally isolate so called "abstract sockets" (i.e. UDS sockets that do NOT use file-paths by setting `struct sockaddr_un.sun_path = 'abstract', see `man unix`); using directories is even easier and definitely more explicit/readable/immediately-obvious as a human-user. As such this reworks all the necessary `UDSAddress` meths, - `.unwrap()` now returns a `tuple(str(._filedir, str(._filename))`, - `wrap_address()` now matches UDS on a 2nd tuple `str()` element, - `.get_root()` no longer passes `maybe_pid`. AND adjusts `MsgpackUDSStream` to, - use the new `unwrap_sockpath()` on the `socket.get[sock/peer]name()` output before passing directly as `UDSAddress.__init__(filedir, filename)` instead of via `.from_addr()`. - also pass `maybe_pid`s to init since no longer included in the unwrapped-type form.
2025-04-04 02:24:24 +00:00
f'Can not wrap unwrapped-address ??\n'
f'type(addr): {type(addr)!r}\n'
f'addr: {addr!r}\n'
)
return cls.from_addr(addr)
Rework/simplify transport addressing A few things that can fundamentally change, - UDS addresses now always encapsulate the local and remote pid such that it denotes each side's process much like a TCP *port*. |_ `.__init__()` takes a new `maybe_pid: int`. |_ this required changes to the `.ipc._uds` backend which will come in an subsequent commit! |_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the TCP case. |_ adjust `wrap_address()` to match. - use a new `_state.get_rt_dir() -> Path` as the default location for UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by default. - re-implement `USDAddress.get_random()` to use both the local `Actor.uid` (if available) and at least the pid for its socket file name. Removals, - drop the loop generated `_default_addrs`, simplify to just `_default_lo_addrs` for per-transport default registry addresses. |_ change to `_address_types: dict[str, Type[Address]]` instead of separate types `list`. |_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`. - comment out `Address.open_stream()` it's unused and i think the wrong place for this API. Renames, - from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type union and all this type set is, is the simple python data-structures we encode to for the wire. |_ see note about possibly implementing the `.[un]wrap()` stuff as `msgspec` codec `enc/dec_hook()`s instead! Additions, - add a `mk_uuid()` to be used throughout the runtime including for generating the `Aid.uuid` part. - tons of notes around follow up refinements!
2025-03-30 22:30:43 +00:00
def default_lo_addrs(
transports: list[str],
) -> list[Type[Address]]:
'''
Return the default, host-singleton, registry address
for an input transport key set.
'''
return [
_default_lo_addrs[transport]
for transport in transports
]