- only generate a repr in `.from_addr()` when log level is >= 'runtime'.
|_ add a todo about supporting this optimization more generally on our
adapter.
- fix `Channel.pformat()` to show unknown peer field line fmt correctly.
- add a `Channel.maddr: str` which just delegates directly to the
`._transport` like other pass-thru property fields.
To start trying out,
- using in the `Start`-msg handler-block to repr the msg coming
*from* a `repr(Channel)` using '<=)` sclang op.
- for a completed RPC task in `_invoke_non_context()`.
- for the msg loop task's termination report.
Such as the `MsgTransport.stream` and `.drain` attrs since they're
rarely that important at the chan level. Also start adopting
a `.<attr>=` style for actual attrs of the type versus a `<name>:
` style for meta-field info lines.
To simplify `.pformat()` output when the new `privates: bool` is unset
(the default) this adds new public attrs to wrap an actor's
cancellation status as well as provide a `.repr_state: str` (similar to
our equiv on `Context`). Rework `.pformat()` to render a much simplified
repr using all these new refinements.
Further, port the `.cancel()` method to use `.msg.types.Aid` for all
internal `requesting_uid` refs (now renamed with `_aid`) and in all
called downstream methods.
New cancel-state iface deats,
- rename `._cancel_called_by_remote` -> `._cancel_called_by` and expect
it to be set as an `Aid`.
- add `.cancel_complete: bool` which flags whether `.cancel()` ran to
completion.
- add `.cancel_called: bool` which just wraps `._cancel_called` (and
which likely will just be dropped since we already have
`._cancel_called_by`).
- add `.cancel_caller: Aid|None` which wraps `._cancel_called_by`.
In terms of using `Aid` in cancel methods,
- rename vars with `_aid` suffix in `.cancel()` (and wherever else).
- change `.cancel_rpc_tasks()` input param to `req_aid: msgtypes.Aid`.
- do the same for `._cancel_task()` and (for now until we adjust its
internals as well) use the `Aid.uid` remap property when assigning
`Context._canceller`.
- adjust all log msg refs to match obvi.
Providing the legacy `.uid -> tuple` style id (since still used for the
`Actor._contexts` table) and a `repr-one-line` method `.reprol() -> str`
for rendering a compact unique actor ID summary (useful in
logging/.pformat()s at the least).
Since it's for beg filtering, the current impl should be renamed anyway;
it's not just for filtering cancelled excs.
Deats,
- added a real doc string, links to official eg docs and fixed the
return typing.
- adjust all internal imports to match.
To resolve the recently added and failing
`test_remote_exc_relay::test_unmasked_remote_exc`: never allow
`trio.Cancelled` to mask an underlying user-code exception, ever.
Our first real-world (runtime internal) use case for the new
`.trionics.maybe_raise_from_masking_exc()` such that the failing
test now passes with an properly relayed remote RTE unmasking B)
Details,
- flip the `Context._scope_nursery` to the default strict-eg behaviour
and instead stack its outer scope with a `.trionics.collapse_eg()`.
- wrap the inner-most scope (after `msgops.maybe_limit_plds()`) with
a `maybe_raise_from_masking_exc()` to ensure user-code errors are
never masked by `trio.Cancelled`s.
Some err-reporting refinement,
- always capture any `scope_err` from the entire block for debug
purposes; report it in the `finally` block's log.
- always capture any suppressed `maybe_re`, output from
`ctx.maybe_raise()`, and `log.cancel()` report it.
To handle captured non-egs (when the now optional `tn` isn't provided)
as well as yield up a `BoxedMaybeException` which contains any detected
and un-masked `exc_ctx` as its `.value`.
Also add some additional tooling,
- a `raise_unmasked: bool` toggle for when the caller just wants to
report the masked exc and not raise-it-in-place of the masker.
- `extra_note: str` which by default is tuned to the default
`unmask_from = (trio.Cancelled,)` but which can be used to deliver
custom exception msg content.
- `always_warn_on: tuple[BaseException]` which will always emit
a warning log of what would have been the raised-in-place-of
`ctx_exc`'s msg for special cases where you want to report
a masking case that might not be otherwise noticed by the runtime
(cough like a `Cancelled` masking another `Cancelled) but which
you'd still like to warn the caller about.
- factor out the masked-`ext_ctx` predicate logic into
a `find_masked_excs()` and also use it for non-eg cases.
Still maybe todo?
- rewrapping multiple masked sub-excs in an eg back into an eg? left in
#TODOs and a pause-point where applicable.
Turns out there were some subtle internal bugs discovered by the just
added `tests/devx/test_tooling::test_crash_handler_cms` suite.
So this fixes,
- a mis-ordering around `rt_repl_fixture :=` in the logic of
`DebugStatus.maybe_enter_repl_fixture()`.
- `.devx.debug._post_mortem._post_mortem()` ensuring we now **always**
call `DebugStatus.release()`, and thus unwind the exist-stack managing
the `repl_fixture` exit/teardown, **even in the case** where
`yield False` is delivered from the user-fixture-fn (meaning
`dnter_repl=False`) thus triggering an early `return` (as is done in
the new test suite).
Opting for performance over broad multi-actor "debug-ability" from
sync-function-contexts when `debug_mode=True` is set;
IOW prefer no behind-the-scenes `greenlet` perf impact over being
able to use an actor-safe `breakpoint()` wherever as per,
https://greenback.readthedocs.io/en/latest/principle.html#performance
Adjust the breakpoint restore ex script to match.
Dropping the `_maybe_open_repl_fixture()` approach and instead using
a `DebugStatus._fixture_stack = ExitStack()` which provides for much
simpler support around both sync and async pausing APIs thanks to only
invoking `repl_fixture.__exit__()` on actual `PdbREPL` interaction being
complete!
Deats,
- all `repl_fixture` detection logic still happens in one place (the new
method) but we aren't limited to closing it via an immediate post REPL
`.__exit__()` call which instead is triggered by,
- `DebugStatus.release()` which now calls `._fixture_stack.close()` and
thus only invokes `repl_fixture.__exit__()` when user REPL-ing is
**actually complete** an arbitrary amount of debugging time later.
- include the notes for `@acm` support above the new method, though not
sure if they're as relevant any more?
Benefits,
- we can drop the previously added indent levels from
`_enter_repl_sync()` and `_post_mortem()`.
- now we automatically have support for the `.pause_from_sync()` API
since `_enter_repl_sync()` doesn't close the prior
`_maybe_open_repl_fixture()` immediately when `debug_func=None`; the
user's `__exit__()` is only ever called once `.release()` is.
Other,
- add big 'CASE' comments around the various blocks in
`.pause_from_sync()`, i was having trouble figuring out which i was
using from a `breakpoint()` in a dependent app..
Which cleans out the pkg-mod to just the expected exports with (its
longstanding todo comment list) and thus a separation-of-concerns
and smaller mod-file sizes via the following new sub-mods:
- `._trace` for the `.pause()`/`breakpoint()`/`pdb.set_trace()`-style
APIs including all sync-caller variants.
- `._post_mortem` to contain our async `.post_mortem()` and all other
public crash handling APIs for use from sync callers.
- `._sync` for the high-level syncing helper-routines used throughout the
runtime to avoid multi-proc TTY use collisions.
And also,
- remove `hide_runtime_frames()` since moved to `.devx._frame_stack`.
Orig commit was,
"9c0de24 Be explicit with `SpawnSpec` processing in subs"
The commit was picked onto an upstream branch but at that time there was
no `.devx.debug` subpkg yet, hence this revert to the original patch's
module path.
From what was originall the `.devx._debug` monolith module, since that
file was way out of ctl in terms of LoC!
New modules so far include,
- ._repl: our `pdb[p]` ext type/lowlevel-APIs and `mk_pdb()` factory.
- ._sigint: just our REPL-interaction shield-handler.
- ._tty_lock: containing all the root-actor TTY mutex machinery
including the `Lock`/`DebugStatus` primitives/APIs as well as the
inter-tree IPC context eps:
* the server-side `lock_stdio_for_peer()` which pairs with the,
* client-(subactor)-side `request_root_stdio_lock()` via the,
* pld-msg-spec of `LockStatus/LockRelease`.
AND the `any_connected_locker_child()` predicate.
Factoring the (basically duplicate) content from both use spots into
a common `@cm` which delivers a `bool` signalling whether the REPL
should be engaged. Fixes a lingering bug with `nullcontext()` calling
btw..
By supporting a new optional param to `open_crash_handler()`,
`raise_on_exit: bool|Sequence[Type[BaseException]] = True` which
determines whether, after the REPL interaction completes, the handled
exception is raised upward. This is **very** handy for writing bits of
"debug-able but resilient code" as is the case in (many) dependent
projects/apps.
Impl,
- `raise_on_exit` can be a `bool` or (set) sequence of types which will
always be raised.
- also add a `BoxedMaybeException.raise_on_exit` equiv which (for now)
we check matches (in case down the road we want to offer dynamic ctls).
- rename both crash-handler cm's `tb_hide` -> `hide_tb`.
It turns out to be fairly useful to allow hooking into a given actor's
entry-and-exit around `.devx._debug._pause/._post_mortem()` calls which
engage the `pdbp.Pdb` REPL (really our `._debug.PdbREPL` but yeah).
Some very handy use cases include,
- swapping out-of-band (config) state that may otherwise halt the
user's app since the actor normally handles kb&mouse input, in thread,
which means that the handler will be blocked while the REPL is in use.
- (remotely) reporting actor-runtime state for monitoring purposes
around crashes or pauses in normal operation.
- allowing for crash-handling to be hard-disabled via
`._state._runtime_vars` say for when you never want a debugger to be
entered in a production instance where you're not-sure-if/don't-want
per-actor `debug_mode: bool` settings to always be unset, say bc
you're still debugging some edge cases that ow you'd normally want to
REPL up.
Impl details,
- add a new optional `._state._runtime_vars['repl_fixture']` field which
for now can be manually set; i saw no reason for a formal API yet
since we want to convert the `dict` to a struct anyway (first).
- augment both `.devx._debug._pause()/._post_mortem()` with a new
optional `repl_fixture: AbstractContextManager[bool]` kwarg which
when provided is `with repl_fixture()` opened around the lowlevel
REPL interaction calls; if the enter-result, an expected `bool`, is
`False` then the interaction is hard-bypassed.
* for the `._pause()` case the `@cm` is opened around the entire body
of the embedded `_enter_repl_sync()` closure (for now) though
ideally longer term this entire routine is factored to be a lot less
"nested" Bp
* in `_post_mortem()` the entire previous body is wrapped similarly
and also now excepts an optional `boxed_maybe_exc: BoxedMaybeException`
only passed in the `open_crash_handler()` caller case.
- when the new runtime-var is overridden, (only manually atm) it is used
instead but only whenever the above `repl_fixture` kwarg is left null.
- add a `BoxedMaybeException.pformat() = __repr__()` which when
a `.value: Exception` is set renders a more "objecty" repr of the exc.
Obviously tests for all this should be coming soon!
Discovered this bug while testing `modden`'s daemon under various
cancelled-while-booting race conditions where sequential tests would
fail a lingering `assert 0` inside `.to_asyncio.run_as_asyncio_guest()`
to (oddly) catch redundant greenback-re-inits..
XD
Needs a test likely ;P
Such that the default is `None` and in the case where the caller *does
not* set the `pdb` arg to an explicit `bool` we instead determine it via
the output from `._state.is_debug_mode()` allowing for more "nonchalant"
usage throughout a (test) code base which passes the `debug_mode: bool`
as runtime config; allows delegation to the per-actor proc-global state.
Adding an underlying `iter_struct_ppfmt_lines()` (which can also be
called directly) to iter-render field-lines-pairs; it's now called from
the top level `.pformat()`. The use case is to allow more granular
control for rendering runtime primitive (like `Actor.pformat()`) reprs
depending on log-level/config, oh and using `textwrap` for indenting.
Particularly on a get-attr of `StackLevelAdapter.handlers` which, when
a `logger: StackLevelAdapter` is passed, we need to *not call* our own
`get_logger()` and just set is as the `log`. Fix the typing to match.
Since I'd like to use some `reprlib` formatting which `modden` already
implemented (and it's a main dependee project), figured I'd just bring
it all into `.devx.pformat` for now.
Moving it from where i (oddly) first wrote it up in `._entry` to a more
proper place with its pals in `.devx.pformat` ;p
Iface summary, what caller provides:
- `input_op`: a "sclang" chars-symbol to represent the conc "operation",
- `text`: the "entity" being *operated on*,
- `nest_prefix/indent`: what the ^ will be prefixed with.
- `prefix_op: bool` so that, when unset, the `input_op` is instead
used as a suffix to the first line of `text`.
- `next_indent: int|None = None` such that when set (and not null) we
use that exact ws-indent instead of calculating it from the
`len(nest_prefix)` allowing for specifying a `0`-indent easily.
- includes logic where we either get a non-zero value and
apply it strictly to both the `nest_prefix` and `text`, OR we
auto-calc it from any `nest_prefix`, NOT a conflation of both..
- `op_suffix: str = '\n'` for instead of assuming `f'{input_op}\n'`.
- `rm_from_first_ln: str` which allows removing chars from the
first line of `text` after a `str.strip()` (handy for removing
the '<Channel' first chevron from type-reprs).
There's also a huge comment-doc for "sclang" into the fn body which is
the terrible "primer" on this whole idea for the moment XD
We already have the `.ipc` sub-pkg name so it seems a bit
redundant/noisy for a namespace path Bp
Leave an alias for the `Server` rn since it's already used in a few
other internal mods.. will likely rename later if everyone is cool with
it..
Namely any CLI driven runtime-config fixtures such as,
- `--spawn-backend` and `start_method`,
- `--tpdb` and `debug_mode`,
- `--tpt-proto` and `tpt_protos`/`tpt_proto`,
- `reg_addr` as driven by the above.
This moves all fixtures and necessary hook funcs (CLI parsing,
configuring and test-gen) to the `._testing.pytest` module and thus
allows any dependent project to leverage these fixtures in their own
test suites after pointing to that plugin mod using,
```python
# conftest.py
pytest_plugins: tuple[str] = (
"tractor._testing.pytest",
)
```
Also, add a new `._testing.addr` helper mod which now contains
a factored `get_rando_addr()` helper for creating test-sesh unique
tpt-specific registry (or other) IPC endpoint addrs.
Oddly my env was borked bc a missing sub-dep (`typing-extensions`
apparently not added by `uv` for `stackscope`?) and then `stackscope`
was silently failing import and caused the shield-pause test to also
fail (since it couldn't match the expected `log.devx()` on console). The
import failure is not very explanatory due to the `log.warning()`;
change it to `.error()` level.
Also, explicitly import `_sync_pause_from_builtin` in
`examples/debugging/restore_builtin_breakpoint.py` to ensure the ref is
exported properly from `.devx.debug` (which it wasn't during dev of the
prior commit Bp).
Actually applying the input it in the root as well as all sub-actors by
passing it down to sub-actors through runtime-vars as delivered by the
initial `SpawnSpec` msg during child runtime init.
Impl deats,
- add a new `_state._runtime_vars['_enable_tpts']: list[str]` field set
by the input param (if provided) to `.open_root_actor()`.
- mk `current_ipc_protos()` return the runtime-var entry with instead
the default in the `_runtime_vars: dict` set to `[_def_tpt_proto]`.
- in `.open_root_actor()`, still error on this being a >1 `list[str]`
until we have more testing infra/suites to audit multi-protos per
actor.
- return the new value (as 3rd element) from `Actor._from_parent()` as
per the todo note; means `_runtime.async_main()` will allocate
`accept_addrs` as tpt-specific `Address` entries and pass them to
`IPCServer.listen_on()`.
Also,
- also add a new `_state._runtime_vars['_root_addrs']: list = []` field
with the intent of fully replacing the `'_root_mailbox'` field since,
* it will need to be a collection to support multi-tpt,
* it's a more cohesive field name alongside `_registry_addrs`,
* the root actor of every tree needs to have a dedicated addr set
(separate from any host-singleton registry actor) so that all its
subs can contact it for capabilities mgmt including debugger
access/locking.
- in the root, populate the field in `._runtime.async_main()` and for
now just set '_root_mailbox' to the first entry in that list in
anticipation of future multi-homing/transport support.
Tyler Goodlet