jc211
/
tractor
forked from goodboy/tractor
1
0
Fork 0
Code Pull Requests Activity
695 Commits
211 Branches
0 Tags
9.3 MiB
Commit Graph

695 Commits (wip_fix_asyncio_gen_streaming)

Author SHA1 Message Date
Tyler Goodlet c6e52b0387 Cancel scope on stream consumer completion 2021-05-31 09:25:38 -04:00
Tyler Goodlet f792aa06b5 Expose `@context` decorator at top level 2021-05-31 09:25:38 -04:00
Tyler Goodlet 15fa777ddf Add initial bi-directional streaming 
This mostly adds the api described in
https://github.com/goodboy/tractor/issues/53#issuecomment-806258798

The first draft summary:
- formalize bidir steaming using the `trio.Channel` style interface
  which we derive as a `MsgStream` type.
- add `Portal.open_context()` which provides a `trio.Nursery.start()`
  remote task invocation style for setting up and tearing down tasks
  contexts in remote actors.
- add a distinct `'started'` message to the ipc protocol to facilitate
  `Context.start()` with a first return value.
- for our `ReceiveMsgStream` type, don't cancel the remote task in
  `.aclose()`; this is now done explicitly by the surrounding `Context`
   usage: `Context.cancel()`.
- streams in either direction still use a `'yield'` message keeping the
  proto mostly symmetric without having to worry about which side is the
  caller / portal opener.
- subtlety: only allow sending a `'stop'` message during a 2-way
  streaming context from `ReceiveStream.aclose()`, detailed comment
  with explanation is included.

Relates to #53
 2021-05-31 09:25:38 -04:00
goodboy f48548ab94
Merge pull request #197 from goodboy/drop_run 
Drop run
 2021-05-07 12:02:23 -04:00
Tyler Goodlet 73e123bac7 Fix line length 2021-05-07 11:21:40 -04:00
Tyler Goodlet 2b4cf59ee1 Drop sleep 2021-05-07 11:21:40 -04:00
Tyler Goodlet 0551756e22 Use trio.run() in windows tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet 5ca963148e Disable leftover debug mode 2021-05-07 11:21:40 -04:00
Tyler Goodlet 4798d3b5db Drop lingering rpc_module_paths refs 2021-05-07 11:21:40 -04:00
Tyler Goodlet 247483ee93 Drop run and rpc_module_paths from streaming tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet 9e64161538 Drop run and rpc_module_paths from rpc tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet 3bddf9a94b Drop run and rpc_module_paths from spawning tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet 1eedd463cb Drop run and rpc_module_paths from pubsub tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet b46e60ab9d Drop run from multi prog tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet cb527c2562 Mostly drop run from local tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet 1584c547cd Drop run and rpc_module_paths from discovery tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet 2efd8ed167 Drop run and rpc_module_paths from cancel tests 2021-05-07 11:21:40 -04:00
Tyler Goodlet a54260a67e Drop tractor.run() from docs 2021-05-07 11:21:40 -04:00
Tyler Goodlet 98a0594c26 Drop `tractor.run()` from all examples 2021-05-07 11:21:40 -04:00
goodboy ffd10e193e
Merge pull request #208 from goodboy/mp_teardown_hardening 
Mp teardown hardening
 2021-05-06 19:59:50 -04:00
Tyler Goodlet 87971de1d9 Re-raise any sidestepped `trio.Cancelled` 2021-05-06 12:05:17 -04:00
Tyler Goodlet 9f38406e85 Appease mypy 2021-05-06 12:05:17 -04:00
Tyler Goodlet c4b42000eb Shield around root actor cancel 2021-05-06 12:05:17 -04:00
Tyler Goodlet 607c48f1ac Distinctly separate and harden mp spawning 
It's clear now that special attention is needed to handle the case where
a spawned `multiprocessing` proc is started but then the parent is
cancelled before the child can connect back; in this case we need to be
sure to kill the near-zombie child asap. This may end up being the
solution to other resiliency issues seen around mp with nested process
trees too. More testing is needed to be sure.

Relates to #84 #89 #134 #146
 2021-05-06 12:05:17 -04:00
goodboy af93b8532a
Merge pull request #206 from goodboy/stream_contexts 
Explicit stream contexts
 2021-05-04 10:41:03 -04:00
Tyler Goodlet fc36e73628 Comment out `MsgStream` for now 2021-04-28 16:40:38 -04:00
Tyler Goodlet b1f657e246 De-compact async with tuple on 3.8- 
Turns out can't use the nicer syntax before python 3.9 (even though it
doesn't seem documented anywhere?).

Relates to #207
 2021-04-28 16:35:15 -04:00
Tyler Goodlet 2498a4963b Update all tests to new streaming API 2021-04-28 12:23:14 -04:00
Tyler Goodlet 5a5e6baad1 Update all examples to new streaming API 2021-04-28 12:23:08 -04:00
Tyler Goodlet f59346d854 Add func type checking to `.run_in_actor()` 2021-04-28 12:23:08 -04:00
Tyler Goodlet 86fc418050 Error on bad registry pops 2021-04-28 12:23:08 -04:00
Tyler Goodlet 83af295b45 Fix func type checking 2021-04-28 12:23:08 -04:00
Tyler Goodlet ad9256bcdb Drop stream exhaustion; no longer needed 2021-04-28 12:23:08 -04:00
Tyler Goodlet 3e19fd311b Move debugger locking to new stream api 2021-04-28 12:23:08 -04:00
Tyler Goodlet 80c96cab01 Add a warning for soon to be deprecated `ctx` use in `@stream` func 2021-04-28 12:23:08 -04:00
Tyler Goodlet 36251357b3 Add a new one-way stream API 
NB: this is a breaking change removing support for `Portal.run()` being
able to invoke remote streaming functions and instead replacing the
method call with an async context manager api `Portal.open_stream_from()`
This style explicitly defines stream teardown at the call site instead
of expecting the user to handle tricky things correctly themselves: eg.
`async_geneartor.aclosing()`. Going forward `Portal.run()` can be used
only for invoking async functions.
 2021-04-28 12:23:08 -04:00
Tyler Goodlet 81f3558494 Formatting 2021-04-28 12:23:08 -04:00
Tyler Goodlet 897ab79946 Add a no runtime error 2021-04-28 12:23:08 -04:00
Tyler Goodlet 7f38b7225d Aggregate and organize streaming components 
Move receive stream into streaming modules and rebrand as a "message
stream".  Factor out cancellation mechanics in `.aclose()` into the
`Context` type which will soon provide the api for for cancelling portal
invocations.  Comment-stage a few methods on both types in anticipation
of a new bi-directional streaming api.  Add a `MsgStream` bidirectional
channel type which will be the eventual type yielded from
`Context.open_stream()`.  Adjust the response/dialog types to be the set
`{'asyncfun', 'asyncgen', 'context'}`. OH, and add async func checking
in `Portal.run()` to catch and error on sync funcs early.
 2021-04-28 12:23:08 -04:00
goodboy a5a88e2f64
Merge pull request #205 from goodboy/drop_sync_funcs 
Drop sync func invocation support.
 2021-04-28 12:14:41 -04:00
Tyler Goodlet d0eacc3fd6 Appease mypy 2021-04-27 12:08:30 -04:00
Tyler Goodlet 89ce1a63e4 Only accept asyncfunc response type 2021-04-27 12:08:30 -04:00
Tyler Goodlet 1f1619c730 Convert all test suite sync funcs 2021-04-27 12:08:30 -04:00
Tyler Goodlet 5798ef6796 Enforce async funcs on callee side, convert arbiter methods 2021-04-27 12:08:30 -04:00
Tyler Goodlet c2a1612bf5 Drop sync function support 
You can always wrap a sync function in an async one and there seems to
be no good reason to support invoking them directly especially since
cancellation won't work without some thread hackery. If it's requested
we'll point users to `trio-parallel`.

Resolves #77
 2021-04-27 12:08:30 -04:00
Tyler Goodlet be22a2526a Add `Actor.cancel_soon()` for sync self destruct 
Add a sync method that can be used to cancel the current actor from
a synchronous context. This is useful in debugging situations where
sync debugger code may need to kill the process tree.

Also, make the internal "lifetime stack" a global var; easier to manage
from client code that may was to add callbacks prior to the actor
runtime being fully setup.
 2021-04-27 11:35:28 -04:00
Tyler Goodlet a13b3fe0a5 Bump alpha version 2021-04-27 11:35:16 -04:00
goodboy 0f4f7f05cb
Merge pull request #202 from goodboy/first_pypi_release 
First alpha
 2021-02-28 22:02:00 -05:00
Tyler Goodlet 34064cd2c7 Tweak classifiers 2021-02-28 20:44:45 -05:00
goodboy ac30374a15
Merge pull request #201 from goodboy/single_func_example 
Single func example
 2021-02-27 18:04:15 -05:00