forked from goodboy/tractor
1
0
Fork 0
Commit Graph

695 Commits (e1f128a79cd7d4487987ee1d50c96d47cabcbdc5)

Author SHA1 Message Date
Tyler Goodlet e1a55a6f4f Importing happens once locally now so expect a local error 2020-01-21 10:28:32 -05:00
Tyler Goodlet 3c86aa2ab5 Add trio-run-in-process` as dep 2020-01-21 10:28:32 -05:00
Tyler Goodlet 6c45416016 Drop ActorNusery.wait(); it's no longer necessary really 2020-01-21 10:27:53 -05:00
Tyler Goodlet c074aea030 Support TRIP for process launching
This took a ton of tinkering and a rework of the actor nursery tear down
logic. The main changes include:

- each subprocess is now spawned from inside a trio task
from one of two containing nurseries created in the body of
`tractor.open_nursery()`: one for `run_in_actor()` processes and one for
`start_actor()` "daemons". This is to address the need for
`trio-run-in_process.open_in_process()` opening a nursery which must
be closed from the same task that opened it. Using this same approach
for `multiprocessing` seems to work well. The nurseries are waited in
order (rip actors then daemon actors) during tear down which allows
for avoiding the recursive re-entry of `ActorNursery.wait()` handled
prior.

- pull out all the nested functions / closures that were in
`ActorNursery.wait()` and move into the `_spawn` module such that
that process shutdown logic takes place in each containing task's
code path. This allows for vastly simplifying `.wait()` to just contain an
event trigger which initiates process waiting / result collection.
Likely `.wait()` should just be removed since it can no longer be used
to synchronously wait on the actor nursery.

- drop `ActorNursery.__aenter__()` / `.__atexit__()` and move this
"supervisor" tear down logic into the closing block of `open_nursery()`.
This not only cleans makes the code more comprehensible it also
makes our nursery implementation look more like the one in `trio`.

Resolves #93
2020-01-21 10:27:53 -05:00
Tyler Goodlet 91c3716968 Do module abspath loading in actor init 2020-01-21 10:27:53 -05:00
Tyler Goodlet afa640dcab More trip WIP stuff working.. kinda
Get a few more things working:
- fail reliably when remote module loading goes awry
- do a real hacky job of module loading using `sys.path` stuffsies
- we're still totally borked when trying to spin up and quickly cancel
a bunch of subactors...

It's a small move forward I guess.
2020-01-21 10:27:53 -05:00
Tyler Goodlet 1b7cdfe512 WIP trying out trio_run_in_process 2020-01-21 10:27:53 -05:00
goodboy 7c0efce84b
Merge pull request #94 from goodboy/log_task_context
Well...after enough `# type: ignore`s `mypy` is happy, and after enough clicking of *rerun build* the windows CI passed so I think this is prolly gtg peeps!
2020-01-15 21:58:11 -05:00
Tyler Goodlet 698951c515 More mypy apeasement on 3.7 2020-01-15 21:06:13 -05:00
Tyler Goodlet e2c9477122 Allow overriding the root logger name
Handy if other dependent projects want to use the logging system but
also want to slap their own root "branding" onto the record prefix.
2019-12-20 16:37:17 -05:00
Tyler Goodlet 79c152fe38 Make latest mpypy happy 2019-12-10 00:55:03 -05:00
Tyler Goodlet 7947eeebff Use trio_typing stubs 2019-12-09 22:56:13 -05:00
Tyler Goodlet 14bfef0df7 Update types for log adapter 2019-12-09 22:10:15 -05:00
Tyler Goodlet cf73283586 Make info object a mapping type
Make the info object a `Mapping` to play nicer with static type
checking. Simplify the task or actor context method lookup using a dict.
2019-12-09 00:03:22 -05:00
Tyler Goodlet 52efbfc2cd Log task and actor names where possible
Prepend the actor and task names in each log emission. This makes
debugging much more sane since you can see from which process and
running task the log message originates from!

Resolves #13
2019-12-01 23:26:25 -05:00
goodboy 8d2a05e788
Merge pull request #92 from goodboy/drop_event_clear
Drop use of `trio.Event.clear()`
2019-11-25 13:52:20 -05:00
Tyler Goodlet 915bf17a9a Add process tree depth control to nested multierror test
Another step toward having a complete test for #89.
Subactor breadth still seems to cause the most havoc and is why I've
kept that value to just 2 for now.
2019-11-25 12:05:15 -05:00
Tyler Goodlet d2a01e8b81 Drop use of `trio.Event.clear()`
Just spin up new events instead; because apparently they're
so cheap (rolls eyes).

Resolves #78
2019-11-23 11:29:23 -05:00
goodboy 4d43f2564c
Merge pull request #91 from goodboy/more_thorough_super_tests
More thorough basic supervision tests
2019-11-23 11:24:18 -05:00
Tyler Goodlet 2d4b6de4f4 Spawn even less suba-actors in Windows CI
Seems like we've probably got some greater limitations
with Windows and "nested" spawned sub-processes...
2019-11-22 21:23:25 -05:00
Tyler Goodlet f977d37cee Add nursery self-destruct logic on cancel failure
If a nursery fails to cancel (some sub-actors presumably) then hard kill
the whole process tree to avoid hangs during a catastrophic failure.
This logic may get factored out (and changed) as we introduce custom
supervisor strategies.
2019-11-22 17:11:48 -05:00
Tyler Goodlet 42978bf9ac Readme description bump after talks with multiple would-be users 2019-11-22 16:43:49 -05:00
Tyler Goodlet f8adbd73df Add windows and py3.8 support to setup script 2019-11-16 09:58:06 -05:00
Tyler Goodlet 5e056bae71 Expose trio exceptions to `RemoteActorError` 2019-10-30 00:32:10 -04:00
Tyler Goodlet 97df927714 Run first example test under both start methods 2019-10-30 00:31:28 -04:00
Tyler Goodlet 6d9ac53bd5 Add nested multierror testing
Add a test to verify that `trio.MultiError`s are properly propagated up
a simple actor nursery tree. We don't have any exception marshalling
between processes (yet) so we can't validate much more then a simple
2-depth tree. This satisfies the final bullet in #43.

Note I've limited the number of subactors per layer to around 5 since
any more then this seems to break the `multiprocessing` forkserver;
zombie subprocesses seem to be blocking teardown somehow...

Also add a single depth fast fail test just to verify that it's the
nested spawning that triggers this forkserver bug.
2019-10-30 00:30:40 -04:00
Tyler Goodlet 95e8f3d306 Propagate `trio.MultiError`s up the actor tree
`trio.MultiError` isn't an `Exception` (derived instead from
`BaseException`) so we have to specially catch it in the task
invocation machinery and ship it upwards (like regular errors)
since nurseries running in sub-actors can raise them.
2019-10-28 00:47:06 -04:00
Tyler Goodlet d406383cd3 Add a preliminary nested subactor `MultiError` test
This exemplifies the undefined behaviour in #88 and begins to test for
the last bullet in #43.
2019-10-26 15:04:13 -04:00
Tyler Goodlet 6dbb3f7ae6 Extend cancellation tests
In an effort towards #43. This completes the first major bullet's worth of tests
described in that issue.
2019-10-26 09:55:07 -04:00
goodboy ab349cdb8d
Merge pull request #86 from goodboy/pip_ci_fix
Make pip a keener
2019-10-20 16:42:42 -04:00
Tyler Goodlet 1127e3b579 Make pip a keener 2019-10-20 16:24:01 -04:00
Tyler Goodlet c5074f5a60 Always upgrade pip before CI run 2019-10-20 14:06:28 -04:00
goodboy 07d54110c0
Merge pull request #82 from goodboy/windows_support
Windows and Python 3.8 support
2019-10-17 09:11:40 -04:00
Tyler Goodlet e0072f925d Add back a py3.7 run on windows 2019-10-16 21:31:09 -04:00
Tyler Goodlet 6ec9752f46 Slight slowdown on windows / py3.8? 2019-10-16 11:02:18 -04:00
Tyler Goodlet 5f11072442 Add more detailed Windows gotchas section
Fill out with the solution from #79 and move the section further down.
This should hopefully suffice what's left to fulfil for #59

Resolves #79
2019-10-16 09:47:58 -04:00
Tyler Goodlet 7e8b7091cb Check for proper SIGINT return code
They finally got https://bugs.python.org/issue1054041 in Python 3.8
2019-10-15 23:22:48 -04:00
Tyler Goodlet da4796749f Continue hacking the forkserver in Python 3.8
They got all fancy and added shared memory segment tracking and then
had to "generalize" the tracker name...hooray

Fixes #81
2019-10-15 22:37:47 -04:00
Tyler Goodlet 6ff32347bf Don't hardcode python version in path for Windows
Also add Python 3.8 testing on Linux.
2019-10-15 21:18:48 -04:00
goodboy 22b5c1c207
Merge pull request #75 from goodboy/rename_forkserver_mod
Rename override module
2019-10-15 01:05:45 -04:00
Tyler Goodlet 7da95a806d Rename override module 2019-10-14 12:58:10 -04:00
goodboy ee9a71f4bf
Merge pull request #76 from goodboy/user_update
User name and email bump
2019-04-28 11:28:38 -04:00
Tyler Goodlet 24a4d6df4b User name and email bump 2019-04-28 10:42:39 -04:00
goodboy f2b08b5565
Merge pull request #74 from goodboy/win_ci
Add windows CI using choco
2019-04-07 22:27:16 -04:00
Tyler Goodlet 5760bb1b7c Adjust test timeout/sync handling for windows 2019-03-31 15:34:44 -04:00
Tyler Goodlet 3af58d129d Add windows CI using choco
Resolves #62
2019-03-30 20:47:17 -04:00
goodboy e0f4894071
Merge pull request #73 from goodboy/stream_functions
Stream functions
2019-03-29 19:41:50 -04:00
Tyler Goodlet b965d20cba Add stream func tests 2019-03-29 19:10:56 -04:00
Tyler Goodlet f885b02c73 Validate stream functions at decorate time 2019-03-29 19:10:32 -04:00
Tyler Goodlet 5c0ae47cf5 Fix type annotation 2019-03-26 08:03:12 -04:00