Commit Graph

1160 Commits (b8117dad2aa030b89e81d44bc3399d2dadd4f6b9)

Author SHA1 Message Date
Tyler Goodlet b8117dad2a Make `Actor._process_messages()` report disconnects
The method now returns a `bool` which flags whether the transport died
to the caller and allows for reporting a disconnect in the
channel-transport handler task. This is something a user will normally
want to know about on the caller side especially after seeing
a traceback from the peer (if in tree) on console.
2022-02-15 09:09:33 -05:00
Tyler Goodlet a80591b914 Only cancel/get-result from a ctx if transport is up
There's no point in sending a cancel message to the remote linked task
and especially no reason to block waiting on a result from that task if
the transport layer is detected to be disconnected. We expect that the
transport shouldn't go down at the layer of the message loop
(reconnection logic should be handled in the transport layer itself) so
if we detect the channel is not connected we don't bother requesting
cancels nor waiting on a final result message.

Why?

- if the connection goes down in error the caller side won't have a way
  to know "how long" it should block to wait for a cancel ack or result
  and causes a potential hang that may require an additional ctrl-c from
  the user especially if using the debugger or if the traceback is not
  seen on console.
- obviously there's no point in waiting for messages when there's no
  transport to deliver them XD

Further, add some more detailed cancel logging detailing the task and
actor ids.
2022-02-15 09:08:50 -05:00
Tyler Goodlet 52ad597c20 Drop high log level in ctx example 2022-02-15 09:08:50 -05:00
Tyler Goodlet 4973deb55f Typing fixes, simplify `_set_trace()` 2022-02-15 09:08:50 -05:00
Tyler Goodlet 872c47213a Add notes around py3.10 stdlib bug from `pdb++`
There's a bug that's triggered in the stdlib without latest `pdb++`
installed; add a note for that.

Further inside `wait_for_parent_stdin_hijack()` don't `.started()` until
the interactor stream has been opened to avoid races when debugging this
`._debug.py` module (at the least) since we usually don't want the
spawning (parent) task to resume until we know for sure the tty lock has
been acquired. Also, drop the random checkpoint we had inside
`_breakpoint()`, not sure it was actually adding anything useful since
we're (mostly) carefully shielded throughout this func.
2022-02-15 09:08:50 -05:00
Tyler Goodlet fddaaa289c Add and use a pdb instance factory 2022-02-15 09:08:50 -05:00
Tyler Goodlet 85aa8899b3 A `.open_context()` example that causes a hang!
Finally! I think this may be the root issue we've been seeing in
production in a client project.

No idea yet why this is happening but the fault-causing sequence seems
to be:
- `.open_context()` in a child actor
- enter the debugger via `tractor.breakpoint()`
- continue from that entry via `c` command in REPL
- raise an error just after inside the context task's body

Looking at logging it appears as though the child thinks it has the tty
but no input is accepted on the REPL and a further `ctrl-c` results in
some teardown but also a further hang where both parent and child become
unresponsive..
2022-02-15 09:08:50 -05:00
Tyler Goodlet eb017c8da6 Drop all the `@cm.__exit__()` override attempts..
None of it worked (you still will see `.__exit__()` frames on debugger
entry - you'd think this would have been solved by now but, shrug) so
instead wrap the debugger entry-point in a `try:` and put the SIGINT
handler restoration inside `MultiActorPdb` teardown hooks.

This seems to restore the UX as it was prior but with also giving the
desired SIGINT override handler behaviour.
2022-02-15 09:08:50 -05:00
Tyler Goodlet 05e41ecd85 Try overriding `_GeneratorContextManager.__exit__()`; didn't work..
Using either of `@pdb.hideframe` or `__tracebackhide__` on stdlib
methods doesn't seem to work either.. This all seems to have something
to do with async generator usage I think ?
2022-02-15 09:08:50 -05:00
Tyler Goodlet 8fda172ae2 Fix example name typo 2022-02-15 09:08:50 -05:00
Tyler Goodlet a6de29517a Handle a context cancel? Might be a noop 2022-02-15 09:08:50 -05:00
Tyler Goodlet 0183c5bf9b Add a pre-started breakpoint example 2022-02-15 09:08:50 -05:00
Tyler Goodlet 741dff6795 Make `mypy` happy 2022-02-15 09:08:50 -05:00
Tyler Goodlet 94dc5f9114 Refine the handler for child vs. root cases
This gets very close to avoiding any possible hangs to do with tty
locking and SIGINT handling minus a special case that will be detailed
below.

Summary of implementation changes:

- convert `_mk_pdb()` -> `with _open_pdb() as pdb:` which implicitly
  handles the `bdb.BdbQuit` case such that debugger teardown hooks are
  always called.
- rename the handler to `shield_sigint()` and handle a variety of new
  cases:
  * the root is in debug but hasn't been cancelled -> call
    `Actor.cancel_soon()`
  * the root is in debug but *has* been called (`Actor.cancel_soon()`
    already called) -> raise KBI
  * a child is in debug *and* has a task locking the debugger -> ignore
    SIGINT in child *and* the root actor.
- if the debugger instance is provided to the handler at acquire time,
  on SIGINT handling completion re-print the last pdb++ REPL output so
  that the user realizes they are still actively in debug.
- ignore the unlock case where a race condition of "no task" holding the
  lock causes the `RuntimeError` normally associated with the "wrong
  task" doing so (not sure if this is a `trio` bug?).
- change debug logs to runtime level.

Unhandled case(s):

- a child is maybe in debug mode but does not itself have any task using
  the debugger.
    * ToDo: we need a way to decide what to do with
      "intermediate" child actors who themselves either are not in
      `debug_mode=True` but have children who *are* such that a SIGINT
      won't cause cancellation of that child-as-parent-of-another-child
      **iff** any of their children are in in debug mode.
2022-02-15 09:08:50 -05:00
Tyler Goodlet cb77216110 (facepalm) Reraise `BdbQuit` and discard ownerless lock releases 2022-02-15 09:08:50 -05:00
Tyler Goodlet 28513bc601 Add WIP while-debugger-active SIGINT ignore handler 2022-02-15 09:08:50 -05:00
goodboy 6e5590dad6
Merge pull request #300 from goodboy/msgpack_lists_by_default
Use lists by default like `msgspec`, update to latest `msgspec`  and `msgpack` releases
2022-02-15 09:08:20 -05:00
Tyler Goodlet 76a0492028 Fix type annot 2022-02-15 08:52:04 -05:00
Tyler Goodlet 4eab4a0213 Type fix 2022-02-15 08:51:25 -05:00
Tyler Goodlet 0edc6a26bc Go back to strict map keys 2022-02-15 08:48:43 -05:00
Tyler Goodlet c5acc3b969 Pack tuple keys as . delim strs in registry tests 2022-02-15 08:48:07 -05:00
Tyler Goodlet 17e195aacf They renamed to `msgpack` and the version is 1.0.3 2022-02-14 16:03:54 -05:00
Tyler Goodlet c65756ed80 Add nooz 2022-02-14 16:03:10 -05:00
Tyler Goodlet 927decc88d Pin to latest `msgspec` version 2022-02-14 14:14:05 -05:00
Tyler Goodlet 17bfa120cc Port to msgpec `0.4.0` imports 2022-02-14 14:05:55 -05:00
Tyler Goodlet 77ddc073e8 Use lists by default like `msgspec` 2022-02-09 10:07:33 -05:00
goodboy 26bebc42b7
Merge pull request #295 from goodboy/nspaths
`NamespacePath`: a message compatible "object reference" type
2022-01-30 12:40:05 -05:00
Tyler Goodlet 87de28fd88 Slight doc string update 2022-01-30 12:21:41 -05:00
Tyler Goodlet 56b29c27de Add msg serialization coding todo resources list 2022-01-30 12:19:21 -05:00
Tyler Goodlet adf9a1d0aa Add nooz 2022-01-30 12:17:32 -05:00
Tyler Goodlet 25a27e780d Add todo resources for eventual capability-based module filtering 2022-01-30 11:28:10 -05:00
Tyler Goodlet c265f3f94e Move namespace path type into `msg` mod 2022-01-30 11:27:34 -05:00
Tyler Goodlet 2900ceb003 Not all objects have a `.__name__` 2022-01-30 11:26:34 -05:00
Tyler Goodlet b6ae77b5ac Use `pkgutils.resolve_name()` and a `str` subtype
Python 3.9's new object resolver + a `str` is much simpler then mucking
with tuples (and easier to serialize). Include a `.to_tuple()` formatter
since we still are passing the module namespace and function name
separately inside the runtime's message format but in theory we might be
able to simplify this depending on how we would change the support for
`enable_modules:list[str]` in the spawn API.

Thanks to @Fuyukai for pointing `resolve_name()` which I didn't know
about before!
2022-01-30 11:26:34 -05:00
Tyler Goodlet 949cb2c9fe First draft "namespace path" named tuple; probably will discard 2022-01-30 11:26:34 -05:00
goodboy 094206ee9d
Merge pull request #298 from goodboy/experimental_subpkg
Add `tractor.experimental` subpkg
2022-01-29 19:19:50 -05:00
Tyler Goodlet debbf64d58 Add nooz 2022-01-29 17:58:58 -05:00
Tyler Goodlet 070e6ba459 Add `.experimental` subpkg to setup.py 2022-01-29 14:30:39 -05:00
Tyler Goodlet 7e004c0688 Add back blank `msg.py` 2022-01-29 14:22:15 -05:00
Tyler Goodlet ffe88de53b Better idea: start a `tractor.experimental` subpkg 2022-01-29 14:03:55 -05:00
Tyler Goodlet d29a915d48 Update mod doc string 2022-01-29 14:02:04 -05:00
Tyler Goodlet be87caa99b Move legacy pubsub stuff from `msg.py` to trionics mod 2022-01-29 14:02:04 -05:00
goodboy 0b51ebfe11
Merge pull request #284 from goodboy/maybe_cancel_the_cancel_
Cancel the `.cancel_actor()` request on proc death
2022-01-21 14:22:48 -05:00
Tyler Goodlet 4bf7992200 Bump to alpha 5 dev 2022-01-21 13:05:26 -05:00
Tyler Goodlet 41296448e8 Add nooz 2022-01-21 12:49:26 -05:00
Tyler Goodlet 9650055519 Use `.exitcode` which is poll + error handling 2022-01-21 12:49:26 -05:00
Tyler Goodlet 532974fb90 Drop leftover print 2022-01-21 12:49:26 -05:00
Tyler Goodlet b1d72b77c9 Patch mp procs with a `.poll()`
Not sure why they don't already expose this from the `Popen` backends
but, k.
2022-01-21 12:49:26 -05:00
Tyler Goodlet a2171c7e71 Cancel the `.cancel_actor()` request on proc death
Adjust the `soft_wait()` strategy to avoid sending needless cancel
requests if it is known that a child process is already terminated or
does so before the cancel request times out. This should be no slower
and should avoid needless waits on either closure-in-progress or already
closed channels.

Basic strategy is,
- request child actor to cancel
- if process termination is detected, cancel the cancel
- if the process is still alive after a cancel request timeout warn the
  user and yield back to the hard reap handling
2022-01-21 12:49:26 -05:00
goodboy 30986d6b64
Merge pull request #292 from goodboy/moar_timeoutz
Moar timeoutz
2022-01-21 12:49:01 -05:00