As per the outstanding TODO just above the redic `setattr()` loop in
`Actor._from_parent()`!!
Instead of all that risk-ay monkeying, add detailed comment-sections
around each explicit assignment of each `SpawnSpec` field, including
those that were already being explicitly set.
Those and other deats,
- ONLY enable the `.devx.debug._tty_lock` module from `Actor.__init__()`
in the root actor.
- add a new `get_mod_nsps2fps()` to replace the loop in init and assign
the initial `.enable_modules: dict[str, str]` from it.
- do `self.enable_modules.update(spawnspec.enable_modules)` instead of
an overwrite and assert the table is by default empty in all
subs.
From what was originall the `.devx._debug` monolith module, since that
file was way out of ctl in terms of LoC!
New modules so far include,
- ._repl: our `pdb[p]` ext type/lowlevel-APIs and `mk_pdb()` factory.
- ._sigint: just our REPL-interaction shield-handler.
- ._tty_lock: containing all the root-actor TTY mutex machinery
including the `Lock`/`DebugStatus` primitives/APIs as well as the
inter-tree IPC context eps:
* the server-side `lock_stdio_for_peer()` which pairs with the,
* client-(subactor)-side `request_root_stdio_lock()` via the,
* pld-msg-spec of `LockStatus/LockRelease`.
AND the `any_connected_locker_child()` predicate.
Factoring the (basically duplicate) content from both use spots into
a common `@cm` which delivers a `bool` signalling whether the REPL
should be engaged. Fixes a lingering bug with `nullcontext()` calling
btw..
By supporting a new optional param to `open_crash_handler()`,
`raise_on_exit: bool|Sequence[Type[BaseException]] = True` which
determines whether, after the REPL interaction completes, the handled
exception is raised upward. This is **very** handy for writing bits of
"debug-able but resilient code" as is the case in (many) dependent
projects/apps.
Impl,
- `raise_on_exit` can be a `bool` or (set) sequence of types which will
always be raised.
- also add a `BoxedMaybeException.raise_on_exit` equiv which (for now)
we check matches (in case down the road we want to offer dynamic ctls).
- rename both crash-handler cm's `tb_hide` -> `hide_tb`.
It turns out to be fairly useful to allow hooking into a given actor's
entry-and-exit around `.devx._debug._pause/._post_mortem()` calls which
engage the `pdbp.Pdb` REPL (really our `._debug.PdbREPL` but yeah).
Some very handy use cases include,
- swapping out-of-band (config) state that may otherwise halt the
user's app since the actor normally handles kb&mouse input, in thread,
which means that the handler will be blocked while the REPL is in use.
- (remotely) reporting actor-runtime state for monitoring purposes
around crashes or pauses in normal operation.
- allowing for crash-handling to be hard-disabled via
`._state._runtime_vars` say for when you never want a debugger to be
entered in a production instance where you're not-sure-if/don't-want
per-actor `debug_mode: bool` settings to always be unset, say bc
you're still debugging some edge cases that ow you'd normally want to
REPL up.
Impl details,
- add a new optional `._state._runtime_vars['repl_fixture']` field which
for now can be manually set; i saw no reason for a formal API yet
since we want to convert the `dict` to a struct anyway (first).
- augment both `.devx._debug._pause()/._post_mortem()` with a new
optional `repl_fixture: AbstractContextManager[bool]` kwarg which
when provided is `with repl_fixture()` opened around the lowlevel
REPL interaction calls; if the enter-result, an expected `bool`, is
`False` then the interaction is hard-bypassed.
* for the `._pause()` case the `@cm` is opened around the entire body
of the embedded `_enter_repl_sync()` closure (for now) though
ideally longer term this entire routine is factored to be a lot less
"nested" Bp
* in `_post_mortem()` the entire previous body is wrapped similarly
and also now excepts an optional `boxed_maybe_exc: BoxedMaybeException`
only passed in the `open_crash_handler()` caller case.
- when the new runtime-var is overridden, (only manually atm) it is used
instead but only whenever the above `repl_fixture` kwarg is left null.
- add a `BoxedMaybeException.pformat() = __repr__()` which when
a `.value: Exception` is set renders a more "objecty" repr of the exc.
Obviously tests for all this should be coming soon!
Particularly on a get-attr of `StackLevelAdapter.handlers` which, when
a `logger: StackLevelAdapter` is passed, we need to *not call* our own
`get_logger()` and just set is as the `log`. Fix the typing to match.
Discovered this bug while testing `modden`'s daemon under various
cancelled-while-booting race conditions where sequential tests would
fail a lingering `assert 0` inside `.to_asyncio.run_as_asyncio_guest()`
to (oddly) catch redundant greenback-re-inits..
XD
Needs a test likely ;P
Such that the default is `None` and in the case where the caller *does
not* set the `pdb` arg to an explicit `bool` we instead determine it via
the output from `._state.is_debug_mode()` allowing for more "nonchalant"
usage throughout a (test) code base which passes the `debug_mode: bool`
as runtime config; allows delegation to the per-actor proc-global state.
Namely any CLI driven runtime-config fixtures such as,
- `--spawn-backend` and `start_method`,
- `--tpdb` and `debug_mode`,
- `--tpt-proto` and `tpt_protos`/`tpt_proto`,
- `reg_addr` as driven by the above.
This moves all fixtures and necessary hook funcs (CLI parsing,
configuring and test-gen) to the `._testing.pytest` module and thus
allows any dependent project to leverage these fixtures in their own
test suites after pointing to that plugin mod using,
```python
# conftest.py
pytest_plugins: tuple[str] = (
"tractor._testing.pytest",
)
```
Also, add a new `._testing.addr` helper mod which now contains
a factored `get_rando_addr()` helper for creating test-sesh unique
tpt-specific registry (or other) IPC endpoint addrs.
As much as is possible given we currently do some graceful
cancellation join-waiting on any connected sub-actors whenever an active
`local_nursery: AcrtorNursery` in the post-rpc teardown sequence of
`handle_stream_from_peer()` is detected. In such cases we try to allow
the higher level inter-actor (task) context(s) to fully cancelled-ack
before conducting IPC machinery shutdown.
The main immediate motivation for all this is to support unit testing
the `.ipc._server` APIs but in the future may be useful for anyone
wanting to use our modular IPC transport layer sin-"actors".
Impl deats,
- drop passing an `actor: Actor` ref from as many routines in
`.ipc._server` as possible instead opting to use
`._state.current_actor()` where abs needed; thus the fns dropping an
`actor` input param are:
- `open_ipc_server()`
- `IPCServer.listen_on()`
- `._serve_ipc_eps()`
- `.handle_stream_from_peer()`
- factor the above mentioned graceful remote-cancel-ack waiting into
a new `maybe_wait_on_canced_subs()` which is called from
`handle_stream_from_peer()` and delivers a
maybe-`local_nursery: ActorNursery` for downstream logic; it's this
new fn which primarily still needs to call `current_actor()`.
- in `handle_stream_from_peer()` also use `current_actor()` to check if
a handshake is needed (or if it was called as part of some
actor-runtime-less operation like our unit test suite!).
- also don't pass an `actor` to `._rpc.process_messages()` see how-n-why
below..
Surrounding ipc-server client/caller adjustments,
- `._rpc.process_messages()` no longer takes an `actor` input and
now calls `current_actor()` instead.
- `._portal.open_portal()` is adjusted to ^.
- `._runtime.async_main()` is adjusted to the `.ipc._server`'s removal
of `actor` ref passing.
Also,
- drop some server `log.info()`s to `.runtime()`
Namely transferring the `Actor` peer-`Channel` tracking attrs,
- `._peers` which maps the uids to client channels (with duplicates
apparently..)
- the `._peer_connected: dict[tuple[str, str], trio.Event]` child-peer
syncing table mostly used by parent actors to wait on sub's to connect
back during spawn.
- the `._no_more_peers = trio.Event()` level triggered state signal.
Further we move over with some minor reworks,
- `.wait_for_peer()` verbatim (adjusting all dependants).
- factor the no-more-peers shielded wait branch-block out of
the end of `async_main()` into 2 new server meths,
* `.has_peers()` with optional chan-connected checking flag.
* `.wait_for_no_more_peers()` which *just* does the
maybe-shielded `._no_more_peers.wait()`
Call it `handle_stream_from_peer()` and bind in the `actor: Actor` via
a `handler=partial()` to `trio.serve_listeners()`.
With this (minus the `Actor._peers/._peer_connected/._no_more_peers`
attrs ofc) we get nearly full separation of IPC-connection-processing
(concerns) from `Actor` state. Thus it's a first look at modularizing
the low-level runtime into isolated subsystems which will hopefully
improve the entire code base's grok-ability and ease any new feature
design discussions especially pertaining to introducing and/or
composing-together any new transport protocols.
Just like we *were* for the `trio`-resource-errors it normally wraps
since we now also do the same wrapping in `MsgpackTransport.send()`
and we don't normally care to raise tpt-closure-errors on graceful actor
cancel requests.
Also, warn-report any non-tpt-closed low-level `trio` errors we haven't
yet re-wrapped (likely bc they haven't shown up).
Such that re-wrapping/raising from a low-level `trio` resource error is
simpler and includes the `.src_exc` in the `__repr__()` and
`.message/.args` rendered at higher layers (like from `Channel` and
`._rpc` machinery).
Impl deats,
- mainly leverages packing in a new cls-method `.repr_src_exc() -> str:`
repr of the underlying error before an optional `body: str` all as
handled by the previously augmented `.pformat()`'s delegation to
`pformat_exc()`.
- change `.src_exc` to be a property around a renamed `._src_exc`.
But wait, why?
- use it inside `MsgpackTransport.send()` to rewrap any
`trio.BrokenResourceError`s so we always see the underlying
`trio`-src-exc just like in the `.recv()._iter_packets()` handlers.
Primarily moving the `Actor._serve_forever()`-task-as-method and
supporting actor-instance attributes to a new `.ipo._server` sub-mod
which now encapsulates,
- the coupling various `trio.Nursery`s (and their independent lifetime mgmt)
to different `trio.serve_listener()`s tasks and `SocketStream`
handler scopes.
- `Address` and `SocketListener` mgmt and tracking through the idea of
an "IPC endpoint": each "bound-and-active instance" of a served-listener
for some (varied transport protocol's socket) address.
- start and shutdown of the entire server's lifetime via an `@acm`.
- delegation of starting/stopping tpt-protocol-specific `trio.abc.Listener`s
to the corresponding `.ipc._<proto_key>` sub-module (newly defined
mod-top-level instead of `Address` method) `start/close_listener()`
funcs.
Impl details of the `.ipc._server` sub-sys,
- add new `IPCServer`, allocated with `open_ipc_server()`, and which
encapsulates starting multiple-transport-proto-`trio.abc.Listener`s
from an input set of `._addr.Address`s using,
|_`IPCServer.listen_on()` which internally spawns tasks that delegate to a new
`_serve_ipc_eps()`, a rework of what was (effectively)
`Actor._serve_forever()` and which now,
* allocates a new `IPCEndpoint`-struct (see below) for each
address-listener pair alongside the specified
listener-serving/stream-handling `trio.Nursery`s provided by the
caller.
* starts and stops each transport (socket's) listener by calling
`IPCEndpoint.start/close_listener()` which in turn delegates to
the underlying `inspect.getmodule(IPCEndpoint.addr)` backend tpt
module's equivalent impl.
* tracks all created endpoints in a `._endpoints: list[IPCEndpoint]`
which is further exposed through public properties for
introspection of served transport-protocols and their addresses.
|_`IPCServer._[parent/stream_handler]_tn: Nursery`s which are either
allocated (in which case, as the same instance) or provided by the
caller of `open_ipc_server()` such that the same nursery-cancel-scope
controls offered by `trio.serve_listeners(handler_nursery=)` are
offered where the `._parent_tn` is used to spawn `_serve_ipc_eps()`
tasks, and `._stream_handler_tn` is passed verbatim as `handler_nursery`.
- a new `IPCEndpoint`-struct (as mentioned) which wraps each
transport-proto's address + listener + allocated-supervising-nursery
to encapsulate the "lifetime of a server IPC endpoint" such that
eventually we can track and managed per-protocol/address/`.listen_on()`-call
scoped starts/stops/restarts for the purposes of filtering/banning
peer traffic.
|_ also included is an unused `.peer_tpts` table which we can
hopefully use to replace `Actor._peers` in a `Channel`-tracking
transport-proto-aware way!
Surrounding changes to `.ipc.*` primitives to match,
- make `[TCP|UDS]Address` types `msgspec.Struct(frozen=True)` and thus
drop any-and-all `addr._host =` style mutation throughout.
|_ as such also drop their `.__init__()` and `.__eq__()` meths.
|_ UDS tweaks to field names and thus `.__repr__()`.
- move `[TCP|UDS]Address.[start/close]_listener()` meths to be mod-level
equiv `start|close_listener()` funcs.
- just hard code the `.ipc._types._key_to_transport/._addr_to_transport`
table entries instead of all the prior fancy dynamic class property
reading stuff (remember, "explicit is better then implicit").
Modified in `._runtime.Actor` internals,
- drop the `._serve_forever()` and `.cancel_server()`, methods and
`._server_down` waiting logic from `.cancel_soon()`
- add `.[_]ipc_server` which is opened just after the `._service_n` and
delegate to it for any equivalent publicly exposed instance
attributes/properties.
That is moving from `._addr`,
- `TCPAddress` to `.ipc._tcp`
- `UDSAddress` to `.ipc._uds`
Obviously this requires adjusting a buncha stuff in `._addr` to avoid
import cycles (the original reason the module was not also included in
the new `.ipc` subpkg) including,
- avoiding "unnecessary" imports of `[Unwrapped]Address` in various modules.
* since `Address` is a protocol and the main point is that it **does
not need to be inherited** per
(https://typing.python.org/en/latest/spec/protocol.html#terminology)
thus I removed the need for it in both transport submods.
* and `UnwrappedAddress` is a type alias for tuples.. so we don't
really always need to be importing it since it also kinda obfuscates
what the underlying pairs are.
- not exporting everything in submods at the `.ipc` top level and
importing from specific submods by default.
- only importing various types under a `if typing.TYPE_CHECKING:` guard
as needed.
By borrowing from the implementation of `RemoteActorError.pformat()`
which is now factored into a new `.devx.pformat_exc()` and re-used for
both error types while maintaining the same func-sig. Obviously delegate
`RemoteActorError.pformat()` to the new helper accordingly and keeping
the prior `body` generation from `.devx.pformat_boxed_tb()` as before.
The new helper allows for,
- passing any of a `header|message|body: str` which are all combined in
that order in the final output.
- getting the `exc.message` as the default `message` part.
- generating an objecty-looking "type-name" header to be rendered by
default when `header` is not overridden.
- "first-line-of `message`" processing which we split-off and then
re-inject as a `f'<{type(exc).__name__}( {first} )>'` top line header.
- an optional `tail: str = '>'` to "close the object"-look only added
when `with_type_header: bool = True`.
Adjustments to `TransportClosed` around this include,
- replacing the init `cause` arg for a `src_exc` which is now always
assigned to a same named instance var.
- displaying that new `.src_exc` in the `body: str` arg to the
`.devx.pformat.pformat_exc()` call so you can always see the
underlying (normally `trio`) source error.
- just make it inherit from `Exception` not `trio.BrokenResourceError`
to avoid handlers catching `TransportClosed` as the former
particularly in testing when we want to sometimes to distinguish them.
Since in hindsight the real analog of a net-proto's "bindspace"
(normally its routing layer's addresses-port-set) is more akin to the
"location in the file-system" for a UDS socket file (aka the file's
parent directory) determines whether or not the "port" (aka it's
file-name) collides with any other.
So the `._filedir: Path` is like the allocated "address" and,
the `._filename: Path|str` is basically the "port",
at least in my mind.. Bp
Thinking about fs dirs like a "host address" means you can get
essentially the same benefits/behaviour of say an (ip)
addresses-port-space but using the (current process-namespace's)
filesys-tree. Note that for UDS sockets in particular the
network-namespace is what would normally isolate so called "abstract
sockets" (i.e. UDS sockets that do NOT use file-paths by setting `struct
sockaddr_un.sun_path = 'abstract', see `man unix`); using directories is
even easier and definitely more explicit/readable/immediately-obvious as
a human-user.
As such this reworks all the necessary `UDSAddress` meths,
- `.unwrap()` now returns a `tuple(str(._filedir, str(._filename))`,
- `wrap_address()` now matches UDS on a 2nd tuple `str()` element,
- `.get_root()` no longer passes `maybe_pid`.
AND adjusts `MsgpackUDSStream` to,
- use the new `unwrap_sockpath()` on the `socket.get[sock/peer]name()`
output before passing directly as `UDSAddress.__init__(filedir, filename)`
instead of via `.from_addr()`.
- also pass `maybe_pid`s to init since no longer included in the
unwrapped-type form.
Such that the "global-ish" setting (actor-local) is managed with the
others per actor-process and type it as a `Literal['tcp', 'uds']` of the
currently support protocol keys.
Here obvi `_tpt` is some kinda shorthand for "transport" and `_proto` is
for "protocol" Bp
Change imports and refs in all dependent modules.
Oh right, and disable UDS in `wrap_address()` for the moment while
i figure out how to avoid the unwrapped type collision..
There was a very strange legacy test
`test_spawning.test_local_arbiter_subactor_global_state` which was
causing unforseen hangs/errors on the UDS tpt and looking deeper this
test was already doing root-actor things that should never have been
valid XD
So rework that test to properly demonstrate something of value
(i guess..) and add a new suite which start more rigorously auditing our
`open_root_actor()` permitted usage.
For the old test,
- since the main point of this test seemed to be the ability to invoke
the same function in both the parent and child actor (using the very
legacy `ActorNursery.run_in_actor()`.. due to be deprecated) rename it
to `test_run_in_actor_same_func_in_child`,
- don't re-enter `.open_root_actor()` since that's invalid usage (tested
in new suite see below),
- adjust some `spawn()` arg/var naming and ensure we only return in the
child.
For the new suite add tests for,
- ensuring the implicit `open_root_actor()` call under `open_nursery()`.
- double open of `open_root_actor()` from within the same process tree
both from a root and sub.
Intro some new `_exceptions` used in the new suite,
- a top level `RuntimeFailure` for generically expressing faults not of
our own doing that prevent successful operation; this is what we now
(changed in this commit) raise on attempts to open a 2nd root.
- mk `ActorFailure` derive from the former; it's already used from
`._spawn` when subprocs fail to boot.
- aggregate the `MsgStream.aclose()` "reader tasks" stats content into a
common `message: str` before emit.
- tweak an `_rpc.process_messages()` emit per new `Channel.__repr__()`.
Throughout `_context` and `_spawn` where it causes no big disruption.
Still lots to work out for things like how to pass `--uid
<tuple-as-str>` to spawned subactors and whether we want a diff name for
the minimum `tuple` required to distinguish a subactor pre-process-ID
allocation by the OS.
Finally.. i've been meaning todo this for ages since the
actor-id-swap-as-handshake is better layered as part of the IPC msg-ing
machinery and then let's us encapsulate the connection-time-assignment
of a remote peer's `Aid` as a new `Channel.aid: Aid`. For now we
continue to offer the `.uid: tuple[str, str]` attr (by delegating to the
`.uid` field) since there's still a few things relying on it in the
runtime and ctx layers
Nice bonuses from this,
- it's very easy to get the peer's `Aid.pid: int` from anywhere in an
IPC ctx by just reading it from the chan.
- we aren't saving more then the wire struct-msg received.
Also add deprecation warnings around usage to get us moving on porting
the rest of consuming runtime code to the new attr!
For the case where there's clearly no socket file created/bound
obviously the `trio.socket.connect()` call will raise
`FileNotFoundError`, so just translate this to
a builtin-`ConnectionError` at the transport layer so we can report the
guilty `UDSAddress`.
The more I think about it, it seems @guille's orig approach of
unwrapping UDS socket-file addresses to strings (or `Path`) is making
the most sense. I had originally thought that pairing it with the
listening side's pid would add clarity (and it definitely does for
introspection/debug/logging) but since we don't end up passing that pid
to the eventual `.connect()` call on the client side, it doesn't make
much sense to wrap it for the wire just to discard.. Further, the
`tuple[str, int]` makes `wrap_address()` break for TCP since it will
always match on uds first.
So, on that note this patch refines a few things in prep for going back
to that original `UnwrappedAddress` as `str` type though longer run
i think the more "builtin approach" would be to add `msgspec` codec
hooks for these types to avoid all the `.wrap()`/`.unwrap()` calls
throughout the runtime.
Down-low deats,
- add `wrap_address()` doc string, detailed (todo) comments and handle
the `[None, None]` case that can come directly from
`._state._runtime_vars['_root_mailbox']`.
- buncha adjustments to `UDSAddress`,
- add a `filedir`, chng `filepath` -> `filename` and mk `maybe_pid` optional.
- the intent `filedir` is act as the equivalent of the host part in a network proto's
socket address and when it's null use the `.def_bindspace = get_rt_dir()`.
- always ensure the `filedir / filename` is an absolute path and
expose it as a new `.sockpath: Path` property.
- mk `.is_valid` actually verify the `.sockpath` is in the valid
`.bindspace: namely just checking it's in the expected dir.
- add pedantic `match:`ing to `.from_addr()` such that we error on
unexpected `type(addr)` inputs and otherwise parse any `sockpath:
Path` inputs using a new `unwrap_sockpath()` which simply splits an
abs file path to dir, file-name parts.
- `.unwrap()` now just `str`-ifies the `.sockpath: Path`
- adjust `.open/close_listener()` to use `.sockpath`.
Call it `maybe_block_bp()` can wrap the `open_root_actor()` body with
it. Main reason is to guarantee we can bp inside actor runtime bootup as
needed when debugging internals! Prolly should factor this to another
module tho?
ALSO, ensure we RTE on recurrent entries to `open_root_actor()` from
within an existing tree! There was actually `test_spawning` test somehow
getting away with this!? Should never be possible or allowed!
And map `.__repr__/__str__` to it and add various new fields to fill it
out,
- drop `self.uid` as var and instead add `Actor._aid: Aid` and proxy to
it for the various `.name/.uid/.pid` properties as well as a new
`.aid` field.
|_ the `Aid.pid` addition is also included.
Other improvements,
- flip to a sync call to `Address.close_listener()`.
- track the `async_main()` parent task as `Actor._task`.
- add exception logging around failure to bind due to already-in-use
when calling `add.open_listener()` in `._stream_forever()`; sometimes
the error might be overridden by something else during the
runtime-failure unwind..
Namely reducing the duplication of class-fields and `TypeVar`s used
for parametrizing the `Address` protocol type,
- drop all of the `TypeVar` types and just stick with all concrete addrs
types inheriting from `Address` only.
- rename `Address.name_key` -> `.proto_key`.
- rename `Address.address_type` -> `.unwrapped_type`
- rename `.namespace` -> `.bindspace` to better reflect that this "part"
of the address represents the possible "space for binding endpoints".
|_ also linux already uses "namespace" to mean the `netns` and i'd
prefer to stick with their semantics for that.
- add `TCPAddress/UDSAddress.def_bindspace` values.
- drop commented `.open_stream()` method; never used.
- simplify `UnwrappedAdress` to just a `tuple` of union types.
- add logging to `USDAddress.open_listener()` for now.
- adjust `tractor.ipc/_uds/tcp` transport to use new addr field names.
Much like we already do in the `._iter_packets()` async-generator which
delivers to `.recv()` and `async for`, handle the `''[Errno 32] Broken
pipe'` case that can show up with unix-domain-socket usage.
Seems like the cause is due to how fast the socket can be torn down
during a registry addr channel ping where,
- the sending side can break the connection faster then the pong side
can prep its handshake msg,
- the pong side tries to send it's handshake pkt via
`.SocketStream.send_all()` after the breakage and then raises
`trio.BrokenResourceError`.
Such that whenev the `self._ctx.chan._exc is trans_err` we suppress.
I.e. when the `Channel._exc: Exception|None` error **is the same as**
set by the `._rpc.process_messages()` loop (that is, set to the
underlying transport layer error), we suppress the lowlevel tb,
otherwise we deliver the full tb since likely something at the lowlevel
that we aren't detecting changed/signalled/is-relevant!
Much like how `Context` has been implemented, try to give tons of high
level details on all the lower level encapsulated primitives, namely the
`.msgstream/.transport` and any useful runtime state.
B)
Impl deats,
- adjust `.from_addr()` to only call `._addr.wrap_address()` when we
detect `addr` is unwrapped.
- add another `log.runtime()` using the new `.__repr__()` in
`Channel.from_addr()`.
- change to `UnwrappedAddress` as in prior commits.
Previously whenever an `ActorNursery.start_actor()` call did not receive
a `bind_addrs` arg we would allocate the default `(localhost, 0)` pairs
in the parent, for UDS this obviously won't work nor is it ideal bc it's
nicer to have the actor to be a socket server (who calls
`Address.open_listener()`) define the socket-file-name containing their
unique ID info such as pid, actor-uuid etc.
As such this moves "random" generation of server addresses to the
child-side of a subactor's spawn-sequence when it's sin-`bind_addrs`;
i.e. we do the allocation of the `Address.get_random()` addrs inside
`._runtime.async_main()` instead of `Portal.start_actor()` and **only
when** `accept_addrs`/`bind_addrs` was **not provided by the spawning
parent**.
Further this patch get's way more rigorous about the `SpawnSpec`
processing in the child inside `Actor._from_parent()` such that we
handle any invalid msgs **very loudly and pedantically!**
Impl deats,
- do the "random addr generation" in an explicit `for` loop (instead of
prior comprehension) to allow for more detailed typing of the layered
calls to the new `._addr` mod.
- use a `match:/case:` for process any invalid `SpawnSpec` payload case
where we can instead receive a `MsgTypeError` from the `chan.recv()`
call in `Actor._from_parent()` to raise it immediately instead of
triggering downstream type-errors XD
|_ as per the big `#TODO` we prolly want to take from other callers
of `Channel.recv()` (like in the `._rpc.process_messages()` loop).
|_ always raise `InternalError` on non-match/fall-through case!
|_ add a note about not being able to use `breakpoint()` in this
section due to causality of `SpawnSpec._runtime_vars` not having
been processed yet..
|_ always return a third element from `._from_rent()` eventually to be
the `preferred_transports: list[str]` from the spawning rent.
- use new `._addr.mk_uuid()` and pass to new `Actor.__init__(uuid: str)`
for all actor creation (including in all the mods tweaked here).
- Move to new type-alias-name `UnwrappedAddress` throughout.
Such that any UDS socket pair is represented (and with the recent
updates to) a `USDAddress` via a similar pair-`tuple[str, int]` as TCP
sockets, a pair of the `.filepath: Path` & the peer proc's `.pid: int`
which we read from the underlying `socket.socket` using
`.set/getsockopt()` calls
Impl deats,
- using the Linux specific APIs, we add a `get_peer_info()` which reads
the `(pid, uid, gid)` using the `SOL_SOCKET` and `SOL_PEECRED` opts to
`sock.getsockopt()`.
|_ this presumes the client has been correspondingly configured to
deliver the creds via a `sock.setsockopt(SOL_SOCKET, SO_PASSCRED,
1)` call - this required us to override `trio.open_unix_socket()`.
- override `trio.open_unix_socket()` as per the above bullet to ensure
connecting peers always transmit "credentials" options info to the
listener.
- update `.get_stream_addrs()` to always call `get_peer_info()` and
extract the peer's pid for the `raddr` and use `os.getpid()` for
`laddr` (obvi).
|_ as part of the new impl also `log.info()` the creds-info deats and
socket-file path.
|_ handle the oddity where it depends which of `.getpeername()` or
`.getsockname()` will return the file-path; i think it's to do with
who is client vs. server?
Related refinements,
- set `.layer_key: int = 4` for the "transport layer" ;)
- tweak some typing and multi-line unpacking in `.ipc/_tcp`.
A few things that can fundamentally change,
- UDS addresses now always encapsulate the local and remote pid such
that it denotes each side's process much like a TCP *port*.
|_ `.__init__()` takes a new `maybe_pid: int`.
|_ this required changes to the `.ipc._uds` backend which will come in
an subsequent commit!
|_ `UDSAddress.address_type` becomes a `tuple[str, int]` just like the
TCP case.
|_ adjust `wrap_address()` to match.
- use a new `_state.get_rt_dir() -> Path` as the default location for
UDS socket file: now under `XDG_RUNTIME_DIR'/tractor/` subdir by
default.
- re-implement `USDAddress.get_random()` to use both the local
`Actor.uid` (if available) and at least the pid for its socket file
name.
Removals,
- drop the loop generated `_default_addrs`, simplify to just
`_default_lo_addrs` for per-transport default registry addresses.
|_ change to `_address_types: dict[str, Type[Address]]` instead of
separate types `list`.
|_ adjust `is_wrapped_addr()` to just check `in _addr_types.values()`.
- comment out `Address.open_stream()` it's unused and i think the wrong
place for this API.
Renames,
- from `AddressTypes` -> `UnwrappedAddress`, since it's a simple type
union and all this type set is, is the simple python data-structures
we encode to for the wire.
|_ see note about possibly implementing the `.[un]wrap()` stuff as
`msgspec` codec `enc/dec_hook()`s instead!
Additions,
- add a `mk_uuid()` to be used throughout the runtime including for
generating the `Aid.uuid` part.
- tons of notes around follow up refinements!
EventFD class now expects the fd to already be init with open_eventfd
RingBuff Sender and Receiver fully manage SharedMemory and EventFD lifecycles, no aditional ctx mngrs needed
Separate ring buf tests into its own test bed
Add parametrization to test and cancellation
Add docstrings
Add simple testing data gen module .samples
First attempt at getting `multiprocessing.shared_memory.ShareableList`
working; we wrap the stdlib type with a readonly attr and a `.key` for
cross-actor lookup. Also, rename all `numpy` specific routines to have
a `ndarray` suffix in the func names.
More or less a verbatim copy-paste minus some edgy variable naming and
internal `piker` module imports. There is a bunch of OHLC related
defaults that need to be dropped and we need to adjust to an optional
dependence on `numpy` by supporting shared lists as per the mp docs.
In particular ensuring we use `ctx._pld_rx.recv_msg_nowait()` from
`.receive_nowait()` (which is called from `.aclose()`) such that we
ALWAYS (can) set the surrounding `Context._result/._outcome_msg` attrs
on reception of a final `Return`!!
This fixes a final stream-teardown-race-condition-bug where prior we
normally didn't set the `Context._result/._outcome_msg` in such cases.
This is **precisely because** `.receive_nowait()` only returns the
`pld` and when called from `.aclose()` this value is discarded, meaning
so is its boxing `Return` despite consuming it from the underlying
`._rx_chan`..
Longer term this should be solved differently by ensuring such races
cases are handled at a higher scope like inside `Context._deliver_msg()`
or the `Portal.open_context()` enter/exit blocks? Add a detailed warning
note and todos for all this around the special case block!
Such that any `Return` is always capture for each ctx instance and set
in `._deliver_msg()` normally; ensures we can at least introspect for it
when missing (like in a recently discovered stream teardown race bug).
Yes this augments the already existing `._result` which is dedicated for
the `._outcome_msg.pld` in the non-error case; we might want to see if
there's a nicer way to directly proxy ref to that without getting the
pre-pld-decoded `Raw` form with `msgspec`?
Also use the new `ctx._pld_rx.recv_msg()` and drop assigning
`pld_rx._ctx`.
Namely renaming and tweaking the `MsgType` receiving methods,
- `.recv_msg()` from what was `.recv_msg_w_pld()` which both receives
the IPC msg from the underlying `._rx_chan` and then decodes its
payload with `.decode_pld()`; it now also log reports on the different
"stage of SC dialog protocol" msg types via a `match/case`.
- a new `.recv_msg_nowait()` sync equivalent of ^ (*was*
`.recv_pld_nowait()`) who's use was the source of a recently
discovered bug where any final `Return.pld` is being
consumed-n-discarded by by `MsgStream.aclose()` depending on
ctx/stream teardown race conditions..
Also,
- remove all the "instance persistent" ipc-ctx attrs, specifically the
optional `_ipc`, `_ctx` and the `.wraps_ipc()` cm, since none of them
were ever really needed/used; all methods which require
a `Context/MsgStream` are explicitly always passed.
- update a buncha typing namely to use the more generic-styled
`PayloadT` over `Any` and obviously `MsgType[PayloadT]`.
Repairs a bug in `drain_to_final_msg()` where in the `Yield()` case
block we weren't guarding against the `ctx._stream is None` edge case
which should be treated a `continue`-draining (not a `break` or
attr-error!!) situation since the peer task maybe be continuing to send
`Yield` but has not yet sent an outcome msg (one of
`Return/Error/ContextCancelled`) to terminate the loop. Ensure we
explicitly warn about this case as well as `.cancel()` emit on a taskc.
Thanks again to @guille for discovering this!
Also add temporary `.info()`s around rxed `Return` msgs as part of
trying to debug a different bug discovered while updating the
context-semantics test suite (in a prior commit).
Just like we do from the `.devx._debug.open_crash_handler()`, this
allows checking various attrs on the raised `ContextCancelled` much like
`with pytest.raises() as excinfo:`.
Seems this can happen in particular when we raise a `MessageTypeError`
on the sender side of a `Context`, since there isn't any msg relayed
from the other side (though i'm wondering if MTE should derive from RAE
then considering this case?).
Means `RemoteActorError.boxed_type = None` in such cases instead of
raising an attr-error for the `None.boxed_type_str`.
Since it should only be used from within a `Portal.open_context()`
scope, make sure the caller knows that!
Also don't hide the frame in tb if the immediate function errors..
By using our new `PldRx` design we can,
- pass through the pld-spec & a `dec_hook()` to our `MsgDec` which is
used to configure the underlying `.dec: msgspec.msgpack.Decoder`
- pass through a `enc_hook()` to `mk_codec()` and use it to conf the
equiv `MsgCodec.enc` such that sent msg-plds are converted prior
to transport.
The trick ended up being just to always union the `mk_dec()`
extension-types spec with the normaly with the `msgspec.Raw` pld-spec
such that the `dec_hook()` is only invoked for payload types tagged
by the encoder/sender side B)
A variety of impl tweaks to make it all happen as well as various
cleanups in the `.msg._codec` mod include,
- `mk_dec()` no defaul `spec` arg, better doc string, accept the new
`ext_types` arg, doing the union of that with `msgspec.Raw`.
- proto-ed a now unused `mk_boxed_ext_struct()` which will likely get
removed since it ended up that our `PayloadMsg` structs already cover
the ext-type-hook requirement that the decoder is passed
a `.type=msgspec.Struct` of some sort in order for `.dec_hook` to be
used.
- add a `unpack_spec_types()` util fn for getting the `set[Type]` from
from a `Union[Type]` annotation instance.
- mk the default `mk_codec(pc_pld_spec = Raw,)` since the `PldRx` design
was already passing/overriding it and it doesn't make much sense to
use `Any` anymore for the same reason; it will cause various `Context`
apis to now break.
|_ also accept a `enc_hook()` and `ext_types` which are used to maybe
config the `.msgpack.Encoder`
- generally tweak a bunch of comments-as-docs and todos namely the ones
that are completed after the pld-rx design was implemented.
Also,
- mask the non-functioning `'defstruct'` approach `inside
`.msg.types.mk_msg_spec()` to prep for its removal.
Adjust the test suite (rn called `test_caps_based_msging`),
- add a new suite `test_custom_extension_types` and move and
use the `enc/dec_nsp()` hooks to the mod level for its use.
- prolly planning to drop the `test_limit_msgspec` suite since it's
mostly replaced by the `test_pldrx_limiting` mod's version?
- originally was tweaking a bunch in `test_codec_hooks_mod` but likely
it will get mostly rewritten to be simpler and simply verify that
ext-typed fields can be used over IPC `Context`s between actors (as
originally intended for this sub-suite).
Since it isn't required until the landing of the new service-manager
stuff in #12; was an oversight
from commit `0607a31dddeba032a2cf7d9fe605edd9d7bb4846`.
Like any `bdb.BdbQuit` that might be relayed from a remote context after
a REPl exit with the `quit` cmd. This fixes various issues while
debugging where it may not be clear to the parent task that the child
was terminated with a purposefully unrecoverable error.
Tyler Goodlet
Guillermo Rodriguez