Factoring the (basically duplicate) content from both use spots into
a common `@cm` which delivers a `bool` signalling whether the REPL
should be engaged. Fixes a lingering bug with `nullcontext()` calling
btw..
By supporting a new optional param to `open_crash_handler()`,
`raise_on_exit: bool|Sequence[Type[BaseException]] = True` which
determines whether, after the REPL interaction completes, the handled
exception is raised upward. This is **very** handy for writing bits of
"debug-able but resilient code" as is the case in (many) dependent
projects/apps.
Impl,
- `raise_on_exit` can be a `bool` or (set) sequence of types which will
always be raised.
- also add a `BoxedMaybeException.raise_on_exit` equiv which (for now)
we check matches (in case down the road we want to offer dynamic ctls).
- rename both crash-handler cm's `tb_hide` -> `hide_tb`.
It turns out to be fairly useful to allow hooking into a given actor's
entry-and-exit around `.devx._debug._pause/._post_mortem()` calls which
engage the `pdbp.Pdb` REPL (really our `._debug.PdbREPL` but yeah).
Some very handy use cases include,
- swapping out-of-band (config) state that may otherwise halt the
user's app since the actor normally handles kb&mouse input, in thread,
which means that the handler will be blocked while the REPL is in use.
- (remotely) reporting actor-runtime state for monitoring purposes
around crashes or pauses in normal operation.
- allowing for crash-handling to be hard-disabled via
`._state._runtime_vars` say for when you never want a debugger to be
entered in a production instance where you're not-sure-if/don't-want
per-actor `debug_mode: bool` settings to always be unset, say bc
you're still debugging some edge cases that ow you'd normally want to
REPL up.
Impl details,
- add a new optional `._state._runtime_vars['repl_fixture']` field which
for now can be manually set; i saw no reason for a formal API yet
since we want to convert the `dict` to a struct anyway (first).
- augment both `.devx._debug._pause()/._post_mortem()` with a new
optional `repl_fixture: AbstractContextManager[bool]` kwarg which
when provided is `with repl_fixture()` opened around the lowlevel
REPL interaction calls; if the enter-result, an expected `bool`, is
`False` then the interaction is hard-bypassed.
* for the `._pause()` case the `@cm` is opened around the entire body
of the embedded `_enter_repl_sync()` closure (for now) though
ideally longer term this entire routine is factored to be a lot less
"nested" Bp
* in `_post_mortem()` the entire previous body is wrapped similarly
and also now excepts an optional `boxed_maybe_exc: BoxedMaybeException`
only passed in the `open_crash_handler()` caller case.
- when the new runtime-var is overridden, (only manually atm) it is used
instead but only whenever the above `repl_fixture` kwarg is left null.
- add a `BoxedMaybeException.pformat() = __repr__()` which when
a `.value: Exception` is set renders a more "objecty" repr of the exc.
Obviously tests for all this should be coming soon!
Discovered this bug while testing `modden`'s daemon under various
cancelled-while-booting race conditions where sequential tests would
fail a lingering `assert 0` inside `.to_asyncio.run_as_asyncio_guest()`
to (oddly) catch redundant greenback-re-inits..
XD
Needs a test likely ;P
Such that the default is `None` and in the case where the caller *does
not* set the `pdb` arg to an explicit `bool` we instead determine it via
the output from `._state.is_debug_mode()` allowing for more "nonchalant"
usage throughout a (test) code base which passes the `debug_mode: bool`
as runtime config; allows delegation to the per-actor proc-global state.
Adding an underlying `iter_struct_ppfmt_lines()` (which can also be
called directly) to iter-render field-lines-pairs; it's now called from
the top level `.pformat()`. The use case is to allow more granular
control for rendering runtime primitive (like `Actor.pformat()`) reprs
depending on log-level/config, oh and using `textwrap` for indenting.
Particularly on a get-attr of `StackLevelAdapter.handlers` which, when
a `logger: StackLevelAdapter` is passed, we need to *not call* our own
`get_logger()` and just set is as the `log`. Fix the typing to match.
Since I'd like to use some `reprlib` formatting which `modden` already
implemented (and it's a main dependee project), figured I'd just bring
it all into `.devx.pformat` for now.
Moving it from where i (oddly) first wrote it up in `._entry` to a more
proper place with its pals in `.devx.pformat` ;p
Iface summary, what caller provides:
- `input_op`: a "sclang" chars-symbol to represent the conc "operation",
- `text`: the "entity" being *operated on*,
- `nest_prefix/indent`: what the ^ will be prefixed with.
- `prefix_op: bool` so that, when unset, the `input_op` is instead
used as a suffix to the first line of `text`.
- `next_indent: int|None = None` such that when set (and not null) we
use that exact ws-indent instead of calculating it from the
`len(nest_prefix)` allowing for specifying a `0`-indent easily.
- includes logic where we either get a non-zero value and
apply it strictly to both the `nest_prefix` and `text`, OR we
auto-calc it from any `nest_prefix`, NOT a conflation of both..
- `op_suffix: str = '\n'` for instead of assuming `f'{input_op}\n'`.
- `rm_from_first_ln: str` which allows removing chars from the
first line of `text` after a `str.strip()` (handy for removing
the '<Channel' first chevron from type-reprs).
There's also a huge comment-doc for "sclang" into the fn body which is
the terrible "primer" on this whole idea for the moment XD
For now just as sanity that we're not breaking anything on that
transport backend (since just a little while back there were issues with
crash handling in subs..) when it comes to crash-REPLing.
Like it sounds, verifying that when that param is passed to the runtime
startup eps (`.open_root_actor()/.open_nursery()`), the appropriate
tpt-protocol is deployed for IPC (both the server and bound endpoints)
in both the root and any sub-actors (as passed down from rent to child
via the `.msg.types.SpawnSpec`).
We already have the `.ipc` sub-pkg name so it seems a bit
redundant/noisy for a namespace path Bp
Leave an alias for the `Server` rn since it's already used in a few
other internal mods.. will likely rename later if everyone is cool with
it..
Namely any CLI driven runtime-config fixtures such as,
- `--spawn-backend` and `start_method`,
- `--tpdb` and `debug_mode`,
- `--tpt-proto` and `tpt_protos`/`tpt_proto`,
- `reg_addr` as driven by the above.
This moves all fixtures and necessary hook funcs (CLI parsing,
configuring and test-gen) to the `._testing.pytest` module and thus
allows any dependent project to leverage these fixtures in their own
test suites after pointing to that plugin mod using,
```python
# conftest.py
pytest_plugins: tuple[str] = (
"tractor._testing.pytest",
)
```
Also, add a new `._testing.addr` helper mod which now contains
a factored `get_rando_addr()` helper for creating test-sesh unique
tpt-specific registry (or other) IPC endpoint addrs.
For now it just boots a server, parametrized over all tpt-protos, sin
any actor runtime bootup. Obvi the future todo is ensuring it all works
with a client connecting via the equivalent lowlevel
`.ipc._chan._connect_chan()` API(s).
Namely while what I was actually trying to solve was why
`TransportClosed` was getting raised from `Portal.cancel_actor()` but
still useful edge case auditing either way. Also opts into the
`debug_mode` fixture with apprope timeout adjustment B)
In `tests/test_advanced_faults.py` that is.
Since instead of zero-responses like we'd expect from a network-socket
we actually can get a few differences from the OS when "everything IPC
is known"
XD
Namely it's about underlying `trio` exceptions versus how we wrap them
and how we expect to box them. A `TransportClosed` boxing improvement
is coming in follow up btw to make this all work!
B)
Via a new accumulative `--tpt-proto` arg you can select which
`tpt_protos: list[str]`-fixture protocol keys will be delivered to
opting in tests!
B)
Also includes,
- CLI quote handling/stripping.
- default of 'tcp'.
- only support one selection per session at the moment (until we figure
out how we want to support multiples, either simultaneously or
sequentially).
- draft a (masked) dynamic-`metafunc` parametrization in the
`pytest_generate_tests()` hook.
- first proven and working use in the `test_advanced_faults`-suite (and
thus its underlying
`examples/advanced_faults/ipc_failure_during_stream.py` script)!
|_ actually needed this to prove that the suite only has 2 failures on
'uds' seemingly due to low-level `trio` error semantics translation
differences to do with with calling `socket.close()`..
On a very nearly related topic,
- draft an (also commented out) `set_script_runtime_args()` fixture idea
for a std way of `partial`-ling in runtime args to `examples/`
scripts-as-modules defining a `main()` which would proxy to
`tractor.open_nursery()`.
Such that we can run (opting-in) tests on both TCP and UDS backends and
ensure the `reg_addr` fixture and various timeouts are adjusted
accordingly.
Impl deats,
- add a new `tpc_proto` CLI option and fixture to allow choosing which
"transport protocol" will be used in the test suites (either globally
or contextually).
- rm `_reg_addr` instead opting for a `_rando_port` which will only be
used for `reg_addr`s which are net-tpt-protos.
- rejig `reg_addr` fixture to set a ideally session-unique `testrun_reg_addr`
based on the `tpt_proto` setting making appropriate calls to `._addr`
APIs as needed.
- refine `daemon` fixture a bit with typing, `tpt_proto` timings, and
stderr capture.
- in `test_discovery` do a ton of type-annots, add `debug_mode` fixture
opt ins, augment `spawn_and_check_registry()` with `psutil.Process`
passing for introspection (when things go wrong..).
Oddly my env was borked bc a missing sub-dep (`typing-extensions`
apparently not added by `uv` for `stackscope`?) and then `stackscope`
was silently failing import and caused the shield-pause test to also
fail (since it couldn't match the expected `log.devx()` on console). The
import failure is not very explanatory due to the `log.warning()`;
change it to `.error()` level.
Also, explicitly import `_sync_pause_from_builtin` in
`examples/debugging/restore_builtin_breakpoint.py` to ensure the ref is
exported properly from `.devx.debug` (which it wasn't during dev of the
prior commit Bp).
See docs: https://docs.astral.sh/uv/guides/integration/github/
Summary,
- drop `mypy` job for now since I'd like to move to trying `ty`.
- convert sdist built to `uv build`
- just run test suite on py3.13 for now, not sure if 3.12 will break due
to the eg stuff or not?
Actually applying the input it in the root as well as all sub-actors by
passing it down to sub-actors through runtime-vars as delivered by the
initial `SpawnSpec` msg during child runtime init.
Impl deats,
- add a new `_state._runtime_vars['_enable_tpts']: list[str]` field set
by the input param (if provided) to `.open_root_actor()`.
- mk `current_ipc_protos()` return the runtime-var entry with instead
the default in the `_runtime_vars: dict` set to `[_def_tpt_proto]`.
- in `.open_root_actor()`, still error on this being a >1 `list[str]`
until we have more testing infra/suites to audit multi-protos per
actor.
- return the new value (as 3rd element) from `Actor._from_parent()` as
per the todo note; means `_runtime.async_main()` will allocate
`accept_addrs` as tpt-specific `Address` entries and pass them to
`IPCServer.listen_on()`.
Also,
- also add a new `_state._runtime_vars['_root_addrs']: list = []` field
with the intent of fully replacing the `'_root_mailbox'` field since,
* it will need to be a collection to support multi-tpt,
* it's a more cohesive field name alongside `_registry_addrs`,
* the root actor of every tree needs to have a dedicated addr set
(separate from any host-singleton registry actor) so that all its
subs can contact it for capabilities mgmt including debugger
access/locking.
- in the root, populate the field in `._runtime.async_main()` and for
now just set '_root_mailbox' to the first entry in that list in
anticipation of future multi-homing/transport support.
As per the outstanding TODO just above the redic `setattr()` loop in
`Actor._from_parent()`!!
Instead of all that risk-ay monkeying, add detailed comment-sections
around each explicit assignment of each `SpawnSpec` field, including
those that were already being explicitly set.
Those and other deats,
- ONLY enable the `.devx._debug` (CHERRY-CONFLICT: later changed to
`.debug._tty_lock`) module from `Actor.__init__()` in the root actor.
- ONLY enable the `.devx.debug._tty_lock` module from `Actor.__init__()`
in the root actor.
- add a new `get_mod_nsps2fps()` to replace the loop in init and assign
the initial `.enable_modules: dict[str, str]` from it.
- do `self.enable_modules.update(spawnspec.enable_modules)` instead of
an overwrite and assert the table is by default empty in all
subs.
As much as is possible given we currently do some graceful
cancellation join-waiting on any connected sub-actors whenever an active
`local_nursery: AcrtorNursery` in the post-rpc teardown sequence of
`handle_stream_from_peer()` is detected. In such cases we try to allow
the higher level inter-actor (task) context(s) to fully cancelled-ack
before conducting IPC machinery shutdown.
The main immediate motivation for all this is to support unit testing
the `.ipc._server` APIs but in the future may be useful for anyone
wanting to use our modular IPC transport layer sin-"actors".
Impl deats,
- drop passing an `actor: Actor` ref from as many routines in
`.ipc._server` as possible instead opting to use
`._state.current_actor()` where abs needed; thus the fns dropping an
`actor` input param are:
- `open_ipc_server()`
- `IPCServer.listen_on()`
- `._serve_ipc_eps()`
- `.handle_stream_from_peer()`
- factor the above mentioned graceful remote-cancel-ack waiting into
a new `maybe_wait_on_canced_subs()` which is called from
`handle_stream_from_peer()` and delivers a
maybe-`local_nursery: ActorNursery` for downstream logic; it's this
new fn which primarily still needs to call `current_actor()`.
- in `handle_stream_from_peer()` also use `current_actor()` to check if
a handshake is needed (or if it was called as part of some
actor-runtime-less operation like our unit test suite!).
- also don't pass an `actor` to `._rpc.process_messages()` see how-n-why
below..
Surrounding ipc-server client/caller adjustments,
- `._rpc.process_messages()` no longer takes an `actor` input and
now calls `current_actor()` instead.
- `._portal.open_portal()` is adjusted to ^.
- `._runtime.async_main()` is adjusted to the `.ipc._server`'s removal
of `actor` ref passing.
Also,
- drop some server `log.info()`s to `.runtime()`
Namely transferring the `Actor` peer-`Channel` tracking attrs,
- `._peers` which maps the uids to client channels (with duplicates
apparently..)
- the `._peer_connected: dict[tuple[str, str], trio.Event]` child-peer
syncing table mostly used by parent actors to wait on sub's to connect
back during spawn.
- the `._no_more_peers = trio.Event()` level triggered state signal.
Further we move over with some minor reworks,
- `.wait_for_peer()` verbatim (adjusting all dependants).
- factor the no-more-peers shielded wait branch-block out of
the end of `async_main()` into 2 new server meths,
* `.has_peers()` with optional chan-connected checking flag.
* `.wait_for_no_more_peers()` which *just* does the
maybe-shielded `._no_more_peers.wait()`
Call it `handle_stream_from_peer()` and bind in the `actor: Actor` via
a `handler=partial()` to `trio.serve_listeners()`.
With this (minus the `Actor._peers/._peer_connected/._no_more_peers`
attrs ofc) we get nearly full separation of IPC-connection-processing
(concerns) from `Actor` state. Thus it's a first look at modularizing
the low-level runtime into isolated subsystems which will hopefully
improve the entire code base's grok-ability and ease any new feature
design discussions especially pertaining to introducing and/or
composing-together any new transport protocols.
Tyler Goodlet
Bd