Commit Graph

2012 Commits (4e5742a0567c0f57ebab90306e37eae6c832d134)

Author SHA1 Message Date
Tyler Goodlet 4e5742a056 Extend ctx semantics suite for streaming edge cases!
Muchas grax to @guilledk for finding the first issue which kicked of
this further scrutiny of the `tractor.Context` and `MsgStream` semantics
test suite with a strange edge case where,
- if the parent opened and immediately closed a stream while the remote
  child task started and continued (without terminating) to send msgs
  the parent's `open_context().__aexit__()` would **not block** on the
  child to complete!
=> this was seemingly due to a bug discovered inside the
  `.msg._ops.drain_to_final_msg()` stream handling case logic where we
  are NOT checking if `Context._stream` is non-`None`!

As such this,
- extends the `test_caller_closes_ctx_after_callee_opens_stream` (now
  renamed, see below) to include cases for all combinations of the child
  and parent sending before receiving on the stream as well as all
  placements of `Context.cancel()` in the parent before, around and after
  the stream open.
- uses the new `expect_ctxc()` for expecting the taskc (`trio.Task`
  cancelled)` cases.
- also extends the `test_callee_closes_ctx_after_stream_open` (also
  renamed) to include the case where the parent sends a msg before it
  receives.
=> this case has unveiled yet-another-bug where somehow the underlying
  `MsgStream._rx_chan: trio.ReceiveMemoryChannel` is allowing the
  child's `Return[None]` msg be consumed and NOT in a place where it is
  correctly set as `Context._result` resulting in the parent hanging
  forever inside `._ops.drain_to_final_msg()`..

Alongside,
- start renaming using the new "remote-task-peer-side" semantics
  throughout the test module: "caller" -> "parent", "callee" -> "child".
2025-03-27 15:51:34 -04:00
Tyler Goodlet 3e1d18497f Deliver a `MaybeBoxedError` from `.expect_ctxc()`
Just like we do from the `.devx._debug.open_crash_handler()`, this
allows checking various attrs on the raised `ContextCancelled` much like
`with pytest.raises() as excinfo:`.
2025-03-27 15:51:34 -04:00
Tyler Goodlet f3b10a8032 Support `ctx: UnionType` annots for `@tractor.context` eps 2025-03-27 15:51:34 -04:00
Tyler Goodlet 305e1d0347 Avoid attr-err when `._ipc_msg==None`
Seems this can happen in particular when we raise a `MessageTypeError`
on the sender side of a `Context`, since there isn't any msg relayed
from the other side (though i'm wondering if MTE should derive from RAE
then considering this case?).

Means `RemoteActorError.boxed_type = None` in such cases instead of
raising an attr-error for the `None.boxed_type_str`.
2025-03-27 15:51:34 -04:00
Tyler Goodlet 80fa61af61 Facepalm, fix logic misstep on child side
Namely that `add_hooks: bool` should be the same as on the rent side..
Also, just drop the now unused `iter_maybe_sends`.

This makes the suite entire greeeeen btw, including the new sub-suite
which i hadn't runt before Bo
2025-03-27 15:51:33 -04:00
Tyler Goodlet 048a232a6e Rework IPC-using `test_caps_basesd_msging` tests
Namely renaming and massively simplifying it to a new
`test_ext_types_over_ipc` which avoids all the wacky "parent dictates
what sender should be able to send beforehand"..

Instead keep it simple and just always try to send the same small set of
types over the wire with expect-logic to handle each case,

- use the new `dec_hook`/`ext_types` args to `mk_[co]dec()` routines for
  pld-spec ipc transport.
- always try to stream a small set of types from the child with logic to
  handle the cases expected to error.

Other,
- draft a `test_pld_limiting_usage` to check runtime raising of bad API
  usage; haven't run it yet tho.
- move `test_custom_extension_types` to top of mod so that the
  `enc/dec_nsp()` hooks can be reffed from test parametrizations.
- comment out (and maybe remove) the old routines for
  `iter_maybe_sends`, `test_limit_msgspec`, `chk_pld_type`.

XXX TODO, turns out the 2 failing cases from this suite have exposed an
an actual bug with `MsgTypeError` unpacking where the `ipc_msg=` input
is being set to `None` ?? -> see the comment at the bottom of
`._exceptions._mk_recv_mte()` which seems to describe the likely
culprit?
2025-03-27 15:51:33 -04:00
Tyler Goodlet 337385762b Raise RTE from `limit_plds()` on no `curr_ctx`
Since it should only be used from within a `Portal.open_context()`
scope, make sure the caller knows that!

Also don't hide the frame in tb if the immediate function errors..
2025-03-27 15:51:33 -04:00
Tyler Goodlet fbb8c7ecd0 Offer a `mods: list` to `dec_type_union()`; drop importing this-mod 2025-03-27 15:51:33 -04:00
Tyler Goodlet f8cc034630 Tweak type-error messages for when `ext_types` is missing 2025-03-27 15:51:33 -04:00
Tyler Goodlet 56d0f10195 Move `Union` serializers to new `msg.` mod
Namely moving `enc/dec_type_union()` from the test mod to a new
`tractor.msg._exts` for general use outside the test suite.
2025-03-27 15:51:33 -04:00
Tyler Goodlet 49aeae4150 Finally get type-extended `msgspec` fields workinn
By using our new `PldRx` design we can,
- pass through the pld-spec & a `dec_hook()` to our `MsgDec` which is
  used to configure the underlying `.dec: msgspec.msgpack.Decoder`
- pass through a `enc_hook()` to `mk_codec()` and use it to conf the
  equiv `MsgCodec.enc` such that sent msg-plds are converted prior
  to transport.

The trick ended up being just to always union the `mk_dec()`
extension-types spec with the normaly with the `msgspec.Raw` pld-spec
such that the `dec_hook()` is only invoked for payload types tagged
by the encoder/sender side B)

A variety of impl tweaks to make it all happen as well as various
cleanups in the `.msg._codec` mod include,

- `mk_dec()` no defaul `spec` arg, better doc string, accept the new
  `ext_types` arg, doing the union of that with `msgspec.Raw`.
- proto-ed a now unused `mk_boxed_ext_struct()` which will likely get
  removed since it ended up that our `PayloadMsg` structs already cover
  the ext-type-hook requirement that the decoder is passed
  a `.type=msgspec.Struct` of some sort in order for `.dec_hook` to be
  used.
- add a `unpack_spec_types()` util fn for getting the `set[Type]` from
  from a `Union[Type]` annotation instance.
- mk the default `mk_codec(pc_pld_spec = Raw,)` since the `PldRx` design
  was already passing/overriding it and it doesn't make much sense to
  use `Any` anymore for the same reason; it will cause various `Context`
  apis to now break.
  |_ also accept a `enc_hook()` and `ext_types` which are used to maybe
     config the `.msgpack.Encoder`
- generally tweak a bunch of comments-as-docs and todos namely the ones
  that are completed after the pld-rx design was implemented.

Also,
- mask the non-functioning `'defstruct'` approach `inside
  `.msg.types.mk_msg_spec()` to prep for its removal.

Adjust the test suite (rn called `test_caps_based_msging`),
- add a new suite `test_custom_extension_types` and move and
  use the `enc/dec_nsp()` hooks to the mod level for its use.
- prolly planning to drop the `test_limit_msgspec` suite since it's
  mostly replaced by the `test_pldrx_limiting` mod's version?
- originally was tweaking a bunch in `test_codec_hooks_mod` but likely
  it will get mostly rewritten to be simpler and simply verify that
  ext-typed fields can be used over IPC `Context`s between actors (as
  originally intended for this sub-suite).
2025-03-27 15:51:33 -04:00
goodboy 1e86722357 Merge pull request 'Python 3.13 support' (#18) from py313_support into main
Reviewed-on: #18
2025-03-27 19:50:43 +00:00
Tyler Goodlet eda48c8021 Move bp to-match-comments on same line for py3.13
In the `examples/debugging/restore_builtin_breakpoint.py` i had put the
pattern-comment lines on the line following the `breakpoint()` bc it
seems that's where `pdb` would always "stop" and print the line to
console? So the test would only pass by actually ensuring that in the
`pexpect` capture..

Now on 3.13 it seems that the `pdb` line halting must have been fixed;
it now renders to console the same `breakpoint()` line?
Anyway it works as you'd expect now but **only** on 3.13 so after this
change we might have to adjust the tests to `pytest.xfail()` on earlier
versions.
2025-03-27 13:38:47 -04:00
Tyler Goodlet ceda1e466d Drop explicit `tabcompleter` dep, `pdpp` already sub-depends on it? 2025-03-27 13:38:47 -04:00
Tyler Goodlet d14d29ae8c Bump up to `pytest>=8.3.5` to match "GH actions"
Ensure it's only for the `--dev` optional deps.
2025-03-27 13:38:47 -04:00
Tyler Goodlet f068782e74 Bump to `msgspec>=0.19.0` for py 3.13 support! 2025-03-27 13:38:47 -04:00
Tyler Goodlet 84b04639f8 Bind another `_bexc` for debuggin 2025-03-27 13:38:47 -04:00
Tyler Goodlet 4aa7e8c022 Unpack errors from `pdb.bdb`
Like any `bdb.BdbQuit` that might be relayed from a remote context after
a REPl exit with the `quit` cmd. This fixes various issues while
debugging where it may not be clear to the parent task that the child
was terminated with a purposefully unrecoverable error.
2025-03-27 13:38:47 -04:00
Tyler Goodlet b46a886449 Show frames when decode is handed bad input 2025-03-27 13:38:47 -04:00
Tyler Goodlet a26f817ed1 Another loosie in the trioisms suite 2025-03-27 13:38:47 -04:00
Tyler Goodlet 2d18e6a4be Match `maybe_open_crash_handler()` to non-maybe version
Such that it will deliver a `BoxedMaybeException` to the caller
regardless whether `pdb` is set, and proxy through all `**kwargs`.
2025-03-27 13:38:47 -04:00
Tyler Goodlet e815dcd3c8 Use `collapse_eg()` in broadcaster suite
Around the test embedded `trio.open_nursery()` calls as expected. Also
tidy up the various nursery var names.
2025-03-27 13:38:47 -04:00
Tyler Goodlet 0d7b3f1ac5 Draft some eg collapsing helpers
Inside a new `.trionics._beg` and exposed from the subpkg ns in
anticipation of the `strict_exception_groups=False` being removed by
`trio` in py 3.15.

Notes,
- mk an embedded single-exc "extractor" using a `BaseExceptionGroup.exceptions` length
  check, when 1 return the lone child.
- use the above in a new `@acm`, async bc it's most likely to be composed in an
  `async with` tuple-style sequence block, called `collapse_eg()` which
  acts a one line "absorber" for when the above mentioned flag is no
  logner supported by `trio.open_nursery()`.

All untested atm fwiw.. but soon to be used in our test suite(s) likely!
2025-03-27 13:38:47 -04:00
Tyler Goodlet 3ad558230a Fix docs tests with yet another loosie-goosie
So the KBI propagates up to the actor nursery scope and also avoid
running any `examples/multihost/` subdir scripts.
2025-03-27 13:38:47 -04:00
Tyler Goodlet 22f405a707 Another couple loose-ifies for discovery and advanced fault suites 2025-03-27 13:38:47 -04:00
Tyler Goodlet e5bcefb575 Add (masked) meta-debug-fixture for determining if `debug_mode` is set in harness.. 2025-03-27 13:38:47 -04:00
Tyler Goodlet 8f7c022afe Various test tweaks related to 3.13 egs
Including changes like,
- loose eg flagging in various test emedded `trio.open_nursery()`s.
- changes to eg handling (like using `except*`).
- added `debug_mode` integration to tests that needed some REPLin
  in order to figure out appropriate updates.
2025-03-27 13:38:47 -04:00
Tyler Goodlet c453623b9b Go to loose egs in `Actor` root & service nurseries (for now..) 2025-03-27 13:38:47 -04:00
Tyler Goodlet 6e68f51617 Fix `roundtripped` ref error in `validate_payload_msg()` 2025-03-27 13:38:47 -04:00
Tyler Goodlet fdf934d02d Hide `open_nursery()` frame by def 2025-03-27 13:38:47 -04:00
Tyler Goodlet 13572151aa Moar sclang log fmting tweaks 2025-03-27 13:38:47 -04:00
Tyler Goodlet 87342696a1 Expose `._state.debug_mode()` predicate at top level 2025-03-27 13:38:47 -04:00
Tyler Goodlet 8f774f52b1 Another loose-egs flag in `test_child_manages_service_nursery` 2025-03-27 13:38:47 -04:00
Tyler Goodlet 8b4ed31d3b Handle egs on failed `request_root_stdio_lock()`
Namely when the subactor fails to lock the root, in which case we
try to be very verbose about how/what failed in logging as well
as ensure we cancel the employed IPC ctx.

Implement the outer `BaseException` handler to handle both styles,
- match on an eg (or the prior std cancel excs) only raising a lone
  sub-exc from for former.
- always `as _req_err:` and assign to a new func-global `req_err`
  to enable the above matching.

Other,
- raise `DebugStateError` on `status.subactor_uid != actor_uid`.
- fix a `_repl_fail_report` ref error due to making silly assumptions
  about the `_repl_fail_msg` global; now copy from global as default.
- various log-fmt and logic expression styling tweaks.
- ignore `trio.Cancelled` by default in `open_crash_handler()`.
2025-03-27 13:38:47 -04:00
Tyler Goodlet eb18168a4e A couple more loose-egs flag flips
Namely inside,
- `ActorNursery.open_portal()` which uses
  `.trionics.maybe_open_nursery()` and is now adjusted to
  pass-through `**kwargs` for at least this flag.
- inside the `.trionics.gather_contexts()`.
2025-03-27 13:38:47 -04:00
Tyler Goodlet 6b2809b82e Disable tb colors in `._testing.mk_cmd()`
Unset the appropriate cpython osenv var such that our `pexpect` script
runs in the test suite can maintain original matching logic.
2025-03-27 13:38:47 -04:00
Tyler Goodlet aa80b55567 Log format tweaks for sclang reprs
A space here, a newline there..
2025-03-27 13:38:47 -04:00
Tyler Goodlet 4186541724 Expose `hide_tb: bool` from `.open_nursery()`
Such that it gets passed through to `.open_root_actor()` in the
`implicit_runtime==True` case - useful for debugging cases where
`.devx._debug` APIs might be used to avoid REPL clobbering in subactors.
2025-03-27 13:38:47 -04:00
Tyler Goodlet f0deda1fda Flip to `strict_exception_groups=False` in core tns
Since it'll likely need a bit of detailing to get the test suite running
identically with strict egs (exception groups), i've opted to just flip
the switch on a few core nursery scopes for now until as such a time
i can focus enough to port the matching internals.. Xp
2025-03-27 13:38:47 -04:00
Tyler Goodlet 8f369b5132 Clean up some imports in `._clustering` 2025-03-27 13:38:47 -04:00
Tyler Goodlet aa3432f2a4 Bump various (dev) deps and prefer sys python
Since it turns out there's a few gotchas moving to python 3.13,
- we need to pin to new(er) `trio` which now flips to strict exception
  groups (something to be handled in a follow up patch).
- since we're now using `uv` we should (at least for now) prefer the
  system `python` (over astral's distis) since they compile for
  `libedit` in terms of what the (new) `readline.backend: str` will read
  as; this will break our tab-completion and vi-mode settings in
  the `pdbp` REPL without a user configuring a `~/.editrc`
  appropriately.
- go back to using latest `pdbp` (not a local dev version) since it
  should work fine presuming the previous bullet is addressed.

Lock bumps,
- for now use latest `trio==0.29.0` (which i gotta feeling might have
  broken some existing attempts at strict-eg handling i've tried..)
- update to latest `xonsh`, `pdbp` and its dep `tabcompleter`

Other cleaning,
- put back in various deps "comments" from `poetry` content.
- drop the `xonsh-vox` and `xontrib-vox` dev deps; no `vox` support with
  `uv` rn anyway..
2025-03-27 13:38:47 -04:00
goodboy 222b90940c Merge pull request 'Prevent `asyncio` from abandoning guest-runs, `.pause_from_sync()` support via `.to_asyncio`' (#2) from aio_abandons into main
Reviewed-ish-on: #2
2025-03-27 17:37:57 +00:00
Tyler Goodlet c91373148a Comment-tag pause points in `asycnio_bp.py`
Thought i already did this but, obvi needed these to make the expect
matches pass in our test.
2025-03-27 13:24:25 -04:00
Tyler Goodlet f1af87007e Add equiv of `AsyncioCancelled` for aio side
Such that a `TrioCancelled` is raised in the aio task via
`.set_exception()` to explicitly indicate and allow that task to handle
a taskc request from the parent `trio.Task`.
2025-03-27 13:24:25 -04:00
Tyler Goodlet 13adaa110a Drop `asyncio`-canc error from `._exceptions` 2025-03-27 13:24:25 -04:00
Tyler Goodlet 9e10064bda Continue supporting py3.11+
Apparently the only thing needing a guard was use of
`asyncio.Queue.shutdown()` and the paired `QueueShutDown` exception?

Cool.
2025-03-27 13:24:25 -04:00
Tyler Goodlet bde355dcd5 Fix an `aio_err` ref bug 2025-03-27 13:24:25 -04:00
Tyler Goodlet b021772a1e Mask ctlc borked REPL tests
Namely the `tractor.pause_from_sync()` examples using both bg threads
and `asyncio` which seem to go into bad states where SIGINT is ignored..

Deats,
- add `maybe_expect_timeout()` cm to ensure the EOF hangs get
  `.xfail()`ed instead.
- @pytest.mark.ctlcs_bish` `test_pause_from_sync` and don't expect the
  greenback prompt msg.
- also mark `test_sync_pause_from_aio_task`.
2025-03-27 13:24:25 -04:00
Tyler Goodlet 03406e020c Repair/update `stackscope` test
Seems that on 3.13 it's not showing our script code in the output now?
Gotta get an example for @oremanj to see what's up but really it'd be
nice to just custom format stuff above `trio`'s runtime by def..

Anyway, update the `.devx._stackscope`,
- log formatting to be a little more "sclangy" lookin.
- change the per-actor "delimiter" lines style.
- report the `signal.getsignal(SIGINT)` which i needed in the
  `sync_bp.py` with ctl-c causing a hang..
- mask the `_tree_dumped` duplicator log report as well as the "dumped
  fine" one.
- add an example `pkill --signal SIGUSR1` cmdline.

Tweak the test to cope with,
- not showing our script lines now.. which i've commented in the
  `assert_before()` patts..
- to expect the newly formatted delimiter (ascii) lines to separate the
  root vs. hanger sub-actor sections.
2025-03-27 13:24:25 -04:00
Tyler Goodlet b0acc9ffe8 Add a mark to `pytest.xfail()` questionably conc py stuff (ur mam `.xfail()`s bish!) 2025-03-27 13:24:25 -04:00