forked from goodboy/tractor
1
0
Fork 0
Commit Graph

849 Commits (10f66e5141906823f7c3b10a699e5fe568bc9afe)

Author SHA1 Message Date
Tyler Goodlet 8e32199509 Get entry points reorg without asyncio compat
This is an edit to factor out changes needed for the `asyncio` in guest mode
integration (which currently isn't tested well) so that later more pertinent
changes (which are tested well) can be rebased off of this branch and
merged into mainline sooner. The *infect_asyncio* branch will need to be
rebased onto this branch as well before merge to mainline.
2020-07-24 17:02:03 -04:00
Tyler Goodlet 8054bc7c70 Support "infected asyncio" actors
This is an initial solution for #120.

Allow spawning `asyncio` based actors which run `trio` in guest
mode. This enables spawning `tractor` actors on top of the `asyncio`
event loop whilst still leveraging the SC focused internal actor
supervision machinery. Add a `tractor.to_syncio.run()` api to allow
spawning tasks on the `asyncio` loop from an embedded (remote) `trio`
task and return or stream results all the way back through the `tractor`
IPC system using a very similar api to portals.

One outstanding problem is getting SC around calls to
`asyncio.create_task()`. Currently a task that crashes isn't able to
easily relay the error to the embedded `trio` task without us fully
enforcing the portals based message protocol (which seems superfluous
given the error ref is in process). Further experiments using `anyio`
task groups may alleviate this.
2020-07-24 16:48:06 -04:00
goodboy 2b2cf2e001
Merge pull request #110 from goodboy/init_sphinx_docs
Initial sphinx docs
2020-02-10 18:07:37 -06:00
Tyler Goodlet d62610c44e Search for guard and strip instead of hardcoding 2020-02-10 12:59:44 -05:00
Tyler Goodlet cfc97c4204 Set correct master doc name 2020-02-10 12:26:19 -05:00
Tyler Goodlet 3dcdc9181e Include our `__main__.py` script ex for windows 2020-02-10 12:22:14 -05:00
Tyler Goodlet 20f9ccfa9e Move two more examples out of docs for testing 2020-02-10 12:14:16 -05:00
Tyler Goodlet 63bcd99323 Only error the exs test when "Error" in last line of output 2020-02-10 12:14:16 -05:00
Tyler Goodlet 5a19826bd3 Drop sphinx toctree from readme 2020-02-10 12:14:16 -05:00
Tyler Goodlet 802f47b4ca Drop uneeded import 2020-02-10 12:14:16 -05:00
Tyler Goodlet 03d07cb12a Mirror readme off docs intro 2020-02-10 12:14:16 -05:00
Tyler Goodlet cd06298476 Simplify and re-org the intro section 2020-02-10 12:14:16 -05:00
Tyler Goodlet d6abfa774a Drop toc from sidebar 2020-02-10 12:14:16 -05:00
Tyler Goodlet 66b803780f Replace examples with ..literalinclude directives
This should address both #98 and #108 by using our now tested examples
scripts directly in the documentation (so we know they must work or CI
will fail).

Resolves #98 #108
2020-02-10 12:14:16 -05:00
Tyler Goodlet 5d2fd0eb05 Remove duplicate docs from readme 2020-02-10 12:14:16 -05:00
Tyler Goodlet 6e7d57c01d Add initial sphinx docs draft 2020-02-10 12:14:16 -05:00
goodboy 3b3d563ac9
Merge pull request #102 from goodboy/example_tests
Test docs examples as scripts
2020-02-10 11:13:32 -06:00
Tyler Goodlet f2030a2714 Better document the window's gotcha solution in test code 2020-02-09 14:59:22 -05:00
Tyler Goodlet 7880934505 Add tests for all docs examples
Parametrize our docs example test to include all (now fixed) examples
from the `READ.rst`. The examples themselves have been fixed/corrected
to run but they haven't yet been updated in the actual docs. Once #99
lands these example scripts will be directly included in our
documentation so there will be no possibility of presenting incorrect
examples to our users! This technically fixes #108 even though the new
example aren't going to be included directly in our docs until #99
lands.
2020-02-09 02:01:39 -05:00
Tyler Goodlet 30f8dd8be4 Pass a `Channel` to `LocalPortal` for compat purposes 2020-02-09 01:59:39 -05:00
Tyler Goodlet 9fb05d8849 Drop uneeded import 2020-02-09 01:07:14 -05:00
Tyler Goodlet 596aca8097 Alias __mp_main__ at import time 2020-02-09 01:07:14 -05:00
Tyler Goodlet 70636a98f6 Use the windows "gotchcas" fix for example tests
Apply the fix from @chrizzFTD where we invoke the entry point using
module exec mode on a ``__main__.py`` and import the
``test_example::`main()` from within that entry point script.
2020-02-09 01:07:07 -05:00
Tyler Goodlet 00fc734580 Fix missing `_ctx` define when on Windows 2020-02-07 20:01:41 -05:00
Tyler Goodlet c6f3ab5ae2 Initial examples testing attempt
A per #98 we need tests for examples from the docs as they would be run
by a user copy and pasting the code. This adds a small system for loading
examples from an "examples/" directory and executing them in
a subprocess while checking the output. We can use this to also verify
end-to-end expected logging output on std streams (ex. logging on
stderr).

To expand this further we can parameterize the test list using the
contents of the examples directory instead of hardcoding the script
names as I've done here initially.

Also, fix up the current readme examples to have the required/proper `if
__name__ == '__main__'` script guard.
2020-02-07 19:36:52 -05:00
goodboy 4bd3a14a68
Merge pull request #106 from goodboy/fix_examples_in_docs
Add script "guards" to docs examples
2020-02-07 10:09:38 -06:00
Tyler Goodlet d8daa57a33 Add script "guards" to docs examples
This was originally bundled in #102 but the windows CI there has blocked
that from landing quickly. These examples need to be fixed stat since
I've had at least a couple people notice it now when first trying out
the project.
2020-02-07 00:23:48 -05:00
goodboy 5741bd5209
Merge pull request #95 from goodboy/try_trip
Support trio-run-in-process as process spawning backend
2020-01-31 14:30:54 -06:00
Tyler Goodlet e671cb4f3b Fixup _spawn.py comments to incorporate trip 2020-01-31 12:05:15 -05:00
Tyler Goodlet 8264b7d136 Drop old module loading from abspath cruft 2020-01-31 12:04:46 -05:00
Tyler Goodlet ee4b014092 Fix typo 2020-01-31 12:04:13 -05:00
Tyler Goodlet d64508e1a6 Add more detailed docs around nursery logic
The logic in the `ActorNursery` block is critical to cancellation
semantics and in particular, understanding how supervisor strategies are
invoked. Stick in a bunch of explanatory comments to clear up these
details and also prepare to introduce more supervisor strats besides
the current one-cancels-all approach.
2020-01-31 09:50:25 -05:00
Tyler Goodlet 6348121d23 Do __main__ fixups like ``mulitprocessing does``
Instead of hackery trying to map modules manually from the filesystem
let Python do all the work by simply copying what ``multiprocessing``
does to "fixup the __main__ module" in spawned subprocesses. The new
private module ``_mp_fixup_main.py`` is simply cherry picked code from
``multiprocessing.spawn`` which does just that. We only need these
"fixups" when using a backend other then ``multiprocessing``; for
now just when using ``trio_run_in_process``.
2020-01-29 21:14:48 -05:00
Tyler Goodlet 2a4307975d Fix that thing where the first example in your docs is supposed to work
Thanks to @salotz for pointing out that the first example in the docs
was broken. Though it's somewhat embarrassing this might also explain
the problem in #79 and certain issues in #59...

The solution here is to import the target RPC module using the its
unique basename and absolute filepath in the sub-actor that requires it.
Special handling for `__main__` and `__mp_main__` is needed since the
spawned subprocess will have no knowledge about these parent-
-state-specific module variables. Solution: map the modules name to the
respective module file basename in the child process since the module
variables will of course have different values in children.
2020-01-29 12:16:14 -05:00
Tyler Goodlet 7feef44798 Document available process spawning backends 2020-01-27 16:03:51 -05:00
Tyler Goodlet 43cca122f5 Handle windows in `@tractor_test` as well 2020-01-26 23:44:47 -05:00
Tyler Goodlet a6b249cd52 Forkserver just can't seem to cut it... 2020-01-26 23:17:06 -05:00
Tyler Goodlet 5fd38d4618 Force `mp` backend if option is blank? 2020-01-26 23:16:43 -05:00
Tyler Goodlet b4cb7439a1 Drop useless fork error branch 2020-01-26 22:46:48 -05:00
Tyler Goodlet e57811a602 Fork isn't present on windows... 2020-01-26 22:35:42 -05:00
Tyler Goodlet 7c1bc1fce4 Make windows job names explicit 2020-01-26 22:17:38 -05:00
Tyler Goodlet e18fec9b17 Always force mp backend on Windows 2020-01-26 22:09:06 -05:00
Tyler Goodlet 87948bde3d Add per backend test runs for each Python version 2020-01-26 21:50:03 -05:00
Tyler Goodlet ecced3d09a Allow choosing the spawn backend per test session
Add a `--spawn-backend` option which can be set to one of {'mp',
'trio_run_in_process'} which will either run the test suite using the
`multiprocessing` or `trio-run-in-process` backend respectively.
Currently trying to run both in the same session can result in hangs
seemingly due to a lack of cleanup of forkservers / resource trackers
from `multiprocessing` which cause broken pipe errors on occasion (no
idea on the details).

For `test_cancellation.py::test_nested_multierrors`, use less nesting
when mp is used since it breaks if we push it too hard with the
whole recursive subprocess spawning thing...
2020-01-26 21:36:08 -05:00
Tyler Goodlet 27c9760f96 Be explicit about the spawning backend default
Set `trio-run-in-process` as the default on *nix systems and
`multiprocessing`'s spawn method on Windows. Enable overriding the
default choice using `tractor._spawn.try_set_start_method()`. Allows
for easy runs of the test suite using a user chosen backend.
2020-01-26 21:13:29 -05:00
Tyler Goodlet 783fe53b06 Don't mix trip with multiprocessing for now
It seems that mixing the two backends in the test suite results in hangs
due to lingering forkservers and resource managers from
`multiprocessing`? Likely we'll need either 2 separate CI runs to work
or someway to be sure that these lingering servers are killed in between
tests.
2020-01-24 00:55:40 -05:00
Tyler Goodlet bc259b7eab Use trip as default in all tests for now 2020-01-24 00:54:19 -05:00
Tyler Goodlet d9803ca906 Be explicit with the real name for trip 2020-01-24 00:47:01 -05:00
Tyler Goodlet 4837595e36 Fake out mypy again 2020-01-23 01:32:02 -05:00
Tyler Goodlet 4c5a60d06a Don't import trip on Windows 2020-01-23 01:23:26 -05:00