forked from goodboy/tractor
1
0
Fork 0
Commit Graph

699 Commits (8371621e57e37324eba6619a9aa553ab44789a72)

Author SHA1 Message Date
Tyler Goodlet c3eee1f228 Move "treat_as_gen" detection into `_invoke()` 2018-08-22 11:51:22 -04:00
Tyler Goodlet b0ceb308ba Add type annotations to most functions
This is purely for documentation purposes for now as it should be
obvious a bunch of the signatures aren't using the correct "generics"
syntax (i.e. the use of `(str, int)` instead of `typing.Tuple[str, int])`)
in a bunch of places. We're also not using a type checker yet and besides,
`trio` doesn't really expose a lot of its internal types very well.

2SQASH
2018-08-22 11:50:45 -04:00
goodboy d65cdc0153
Merge pull request #29 from tgoodlet/py3.7_tweaks
Python 3.7!
2018-08-20 08:50:01 -04:00
Tyler Goodlet 81c284272b Run py3.7 in CI 2018-08-20 08:28:14 -04:00
goodboy 5e23ed20e5
Merge pull request #34 from tgoodlet/reliable_cancel_tests
Reliable cancel tests
2018-08-20 08:27:38 -04:00
Tyler Goodlet 996ad891f4 py3.6 is missing this attr 2018-08-19 16:11:57 -04:00
Tyler Goodlet a64c2a70bd Base cancel delays on speed of a non-cancelled run 2018-08-19 16:08:02 -04:00
Tyler Goodlet 328e5bd597 Import our `forkserver.main()` in server cmd
Something changed in 3.7 (likely to do with changes to the core
import system) that requires explicitly importing our version
of `forkserver.main()` in order to guarantee the server runs our
module code. Override `forkserver.ensure_running()`; specifically,
modify the python launch command.
2018-08-19 15:37:01 -04:00
goodboy 1264cae218
Merge pull request #33 from tgoodlet/wait_for_actor
Add wait_for_actor() helper
2018-08-17 23:31:57 -04:00
Tyler Goodlet 8c110c79fb A teensy more lax on the speed test 2018-08-17 16:55:00 -04:00
Tyler Goodlet 3202462cd5 Attach remote internal errors to channels
This ensures that internal errors received from a remote actor are
indeed raised even in the `MainProcess` **before** comms tasks are
cancelled. Internal error in this case means any error packet received
on a channel that doesn't have a `cid` header. RPC errors (which **do**
have a `cid` header) are still forwarded to the consuming caller as usual.
2018-08-17 14:49:17 -04:00
Tyler Goodlet 901f99bbec Throw internal errors into the main coroutine
If an internal error is bubbled up from some sub-actor throw that error
into the `MainProcess` "main" async function / coro in order to trigger
nursery teardowns (i.e. cancellations) that need to be done.

I'll likely change this shortly back to where we run a "main task"
inside `actor._async_main()`...
2018-08-16 00:22:16 -04:00
Tyler Goodlet f8111e51cd Maybe wait for actor result(s) after proc join 2018-08-16 00:21:49 -04:00
Tyler Goodlet d4da80c558 Store remote errors on each portal 2018-08-16 00:21:00 -04:00
Tyler Goodlet 73e8aac36c Always allow and enable rpc prior to task start 2018-08-15 01:24:06 -04:00
Tyler Goodlet 09e3a94060 Cancel result waiter once proc terminates 2018-08-15 01:24:06 -04:00
Tyler Goodlet b1f17dea1f Update readme and upgrade all packages on travis 2018-08-13 12:08:26 -04:00
Tyler Goodlet ea60a3dff9 Test the `wait_for_actor()` api 2018-08-13 00:06:22 -04:00
Tyler Goodlet 3f0c644768 Add `tractor.wait_for_actor()` helper
Allows for waiting on another actor (by name) to register with the
arbiter. This makes synchronized actor spawning and consecutive task
coordination easier to accomplish from within sub-actors.

Resolves #31
2018-08-12 23:59:19 -04:00
Tyler Goodlet 8e027ff571 Be honest about the event loop
Thanks to @pquentin for calling this one out.
2018-08-08 13:25:04 -04:00
goodboy 199b9030a5
Merge pull request #11 from tgoodlet/draft_readme
Initial readme documenting most features
2018-08-07 18:16:10 -04:00
Tyler Goodlet 99e2cf9a13 Draft v2 after new `run_in_actor()` API
Revamp the docs after some feedback from @vodik.
See #24 #25 for additional details.
2018-08-07 18:05:35 -04:00
Tyler Goodlet d4a6cbbc34 Add CI badge 2018-08-07 17:59:27 -04:00
Tyler Goodlet a7315f91ee Document per-func actors for clarity 2018-08-07 17:59:27 -04:00
Tyler Goodlet 8f342ad0dd Add simpler intro
As per a suggestion from @njsmith I've added a little less verbose intro
which mentions *actors* up front.

More,
- add install cmd
- reorg spawning and portal sections a wee bit
2018-08-07 17:59:27 -04:00
Tyler Goodlet 168ff190b3 Initial readme documenting most features
There is a slew of tests to match to verify everything documented thus
far. Hopefully it's a decent little start :)
2018-08-07 17:59:27 -04:00
goodboy a51fbbcf9f
Merge pull request #28 from tgoodlet/reg_with_uid
Reg with uid and drop aiter_recv()
2018-08-07 15:07:34 -04:00
Tyler Goodlet e4ef973be9 Add discovery testing
Add a new test to verify actors register with their `.uid` tuple as
per #7. Break off relevant "discovery" tests into a new test module.
2018-08-07 14:30:25 -04:00
Tyler Goodlet 1bd5582d8a Register each actor using its unique ID tuple
This allows for registering more then one actor with the same "name"
when you have multiple actors fulfilling the same role. Eventually
we'll need support for looking up all actors registered under a given
"service name" (or whatever we decide to call it).

Also, a fix to the arbiter such that each new instance refers to a
separate `_registry` dict (found an issue with duplicate names during
testing).

Resolves #7
2018-08-07 14:03:01 -04:00
Tyler Goodlet 758fbc6790 Drop `Channel.aiter_recv()`
Internalize the implementation of this and expect client code to
iterate the `Channel` directly.

Resolves #16
2018-08-07 14:03:01 -04:00
Tyler Goodlet bd14cbe082 Port to trio's new resource error 2018-08-07 14:03:01 -04:00
Tyler Goodlet 163f747afb Drop legacy write_unsigned() 2018-08-07 14:02:42 -04:00
goodboy 7f0f2e52a9
Merge pull request #21 from tgoodlet/forkserver_singleton
Forkserver singleton
2018-08-04 18:18:21 -04:00
Tyler Goodlet e7c7391497 Drop needless tuple unpack 2018-08-04 18:15:24 -04:00
Tyler Goodlet 4fc7edf466 Change class names, use errno constant 2018-08-04 18:15:24 -04:00
Tyler Goodlet 4b875f0ade Be more explicit with naming and stdlib override 2018-08-04 18:15:24 -04:00
Tyler Goodlet 7017f68503 3.6.3 fix - missing older attr 2018-08-04 18:15:24 -04:00
Tyler Goodlet 50517c9488 Manage a `multiprocessing.forkserver` manually
Start a forkserver once in the main (parent-most) process
and pass ipc info (fds) to subprocesses manually such that embedded
calls to `multiprocessing.Process.start()` just work. Note that this
relies on our overridden version of the stdlib's
`multiprocessing.forkserver` module.

Resolves #6
2018-08-04 18:15:24 -04:00
Tyler Goodlet f46d5b2b62 Hackery to override the stdlib's forkserver
The stdlib insists on creating multiple forkservers and semaphore trackers
for each sub-sub-process launched. This isn't ideal since it costs each
`tractor` sub-actor an additional 2 more processes then necessary and is
confusing when viewed as a process tree (eg. via `pstree`).

The majority of the change is simply avoiding the call to
`forkserver.ensure_running()` and `semaphore_tracker.ensure_running()`
in `ForkServer.connect_new_process()` and instead treating the user like
an adult and expecting those calls to be made *once* in the parent most
process (i.e. what `multiprocessing` calls the `MainProcess`).

Really a proper patch should be made against cpython which allows for
similar manual management of the server along with a mechanism to communicate
forkserver and semaphore tracker fd info to sub-processes such that
further calls to `Process.start()` work as expected.

Relates to #6
2018-08-04 18:15:24 -04:00
Tyler Goodlet d6d7fea708 Use plain func for __aiter__() 2018-08-04 18:15:24 -04:00
goodboy 140956aedb
Merge pull request #25 from tgoodlet/drop_main_kwarg
Drop the "main" task via kwarg idea
2018-08-04 18:13:44 -04:00
Tyler Goodlet f7706074a2 Drop needless if check 2018-08-04 18:10:31 -04:00
Tyler Goodlet 1da69b1396 Allow daemonizing top level actor; don't require main func 2018-08-03 09:41:18 -04:00
Tyler Goodlet db08dbad3b Streaming is too fast, cancel sooner 2018-08-02 16:33:42 -04:00
Tyler Goodlet 41914f3118 Drop console logging 2018-08-02 16:29:01 -04:00
Tyler Goodlet dadea87451 Update test to new api 2018-08-02 15:27:09 -04:00
Tyler Goodlet bb13b79df5 Drop the "main" task via kwarg idea
Stop worrying about a "main task" in each actor and instead add an
additional `ActorNursery.run_in_actor()` method which wraps calls
to create an actor and run a lone RPC task inside it. Note this
adjusts the public API of `ActorNursery.start_actor()` to drop
its `main` kwarg.

The dirty deats of making this possible:
- each spawned RPC task is now tracked with a specific cancel scope such
  that when the actor is cancelled all ongoing responders are cancelled
  before any IPC/channel machinery is closed (turns out that spawning
  new actors from `outlive_main=True` actors was probably borked before
  finally getting this working).
- make each initial RPC response be a packet which describes the
  `functype` (eg. `{'functype': 'asyncfunction'}`) allowing for async
  calls/submissions by client actors (this was required to make
  `run_in_actor()` work - `Portal._submit()` is the new async method).
- hooray we can stop faking "main task" results for daemon actors
- add better handling/raising of internal errors caught in the bowels of
  the `Actor` itself.
- drop the rpc spawning nursery; just use the `Actor._root_nursery`
- only wait on `_no_more_peers` if there are existing peer channels that
  are actually still connected.
- an `ActorNursery.__aexit__()` now implicitly waits on `Portal.result()` on close
  for each `run_in_actor()` spawned actor.
- handle cancelling partial started actors which haven't yet connected
  back to the parent

Resolves #24
2018-08-02 15:24:28 -04:00
goodboy f726bd81da
Merge pull request #15 from tgoodlet/reorg
Reorg everything into private modules
2018-07-17 23:02:29 -04:00
Tyler Goodlet 9571f60a6d Expose channel in public api 2018-07-17 11:57:27 -04:00
Tyler Goodlet 64cbb922dc Reorg everything into private modules 2018-07-14 16:09:05 -04:00