forked from goodboy/tractor
1
0
Fork 0

Document the mp start method more explicitly

spawn_method_support
Tyler Goodlet 2019-03-08 19:54:27 -05:00
parent 8eb138b8a7
commit c3daf73112
3 changed files with 40 additions and 24 deletions

View File

@ -26,12 +26,12 @@ An async-native "`actor model`_" built on trio_ and multiprocessing_.
``tractor`` lets you spawn ``trio`` *"actors"*: processes which each run a ``trio`` scheduler and task ``tractor`` lets you spawn ``trio`` *"actors"*: processes which each run a ``trio`` scheduler and task
tree (also known as an `async sandwich`_). *Actors* communicate by exchanging asynchronous messages_ over tree (also known as an `async sandwich`_). *Actors* communicate by exchanging asynchronous messages_ over
channels_ and avoid sharing any state. This model allows for highly distributed software architecture channels_ and avoid sharing any state. This model allows for highly distributed software architecture
which works just as well on multiple cores as it does over many hosts. ``tractor`` is an actor-model-*like* which works just as well on multiple cores as it does over many hosts.
system in the sense that it adheres to the `3 axioms`_ but not does not (yet) fufill all "unrequirements_" in
practice.
``tractor`` takes inspiration from pulsar_ and execnet_ but attempts to be more focussed on sophistication of ``tractor`` is an actor-model-*like* system in the sense that it adheres to the `3 axioms`_ but not does
the lower level distributed architecture as well as have first class support for streaming using `async generators`_. not (yet) fufill all "unrequirements_" in practice. The API and design takes inspiration from pulsar_ and
execnet_ but attempts to be more focussed on sophistication of the lower level distributed architecture as
well as have first class support for streaming using `async generators`_.
The first step to grok ``tractor`` is to get the basics of ``trio`` down. The first step to grok ``tractor`` is to get the basics of ``trio`` down.
A great place to start is the `trio docs`_ and this `blog post`_. A great place to start is the `trio docs`_ and this `blog post`_.
@ -84,8 +84,8 @@ No PyPi release yet!
Windows "gotchas" Windows "gotchas"
***************** *****************
`tractor` uses the stdlib's `multiprocessing` module internally which `tractor` internally uses the stdlib's `multiprocessing` package which
*can* have some *gotchas* on Windows, namely the need for calling *can* have some gotchas on Windows. Namely, the need for calling
`freeze_support()`_ inside the ``__main__`` context. See `#61`_ for the `freeze_support()`_ inside the ``__main__`` context. See `#61`_ for the
deats. deats.
@ -647,6 +647,9 @@ multiple tasks streaming responses concurrently:
The context notion comes from the context_ in nanomsg_. The context notion comes from the context_ in nanomsg_.
.. _context: https://nanomsg.github.io/nng/man/tip/nng_ctx.5
.. _msgpack: https://en.wikipedia.org/wiki/MessagePack
Running actors standalone Running actors standalone
************************* *************************
@ -662,6 +665,16 @@ need to hop into a debugger. You just need to pass the existing
tractor.run(main, arbiter_addr=('192.168.0.10', 1616)) tractor.run(main, arbiter_addr=('192.168.0.10', 1616))
Choosing a ``multiprocessing`` *start method*
*********************************************
``tractor`` supports selection of the `multiprocessing start method`_ via
a ``start_method`` kwarg to ``tractor.run()``. Note that on Windows
*spawn* it the only supported method and on nix systems *forkserver* is
selected by default for speed.
.. _multiprocessing start method: https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
Enabling logging Enabling logging
**************** ****************
Considering how complicated distributed software can become it helps to know Considering how complicated distributed software can become it helps to know
@ -692,6 +705,14 @@ Stuff I'd like to see ``tractor`` do real soon:
- support for reactive programming primitives and native support for asyncitertools_ like libs - support for reactive programming primitives and native support for asyncitertools_ like libs
- introduction of a `capability-based security`_ model - introduction of a `capability-based security`_ model
.. _supervisors: https://github.com/tgoodlet/tractor/issues/22
.. _nanomsg: https://nanomsg.github.io/nng/index.html
.. _gossip protocol: https://en.wikipedia.org/wiki/Gossip_protocol
.. _celery: http://docs.celeryproject.org/en/latest/userguide/debugging.html
.. _asyncitertools: https://github.com/vodik/asyncitertools
.. _pdb++: https://github.com/antocuni/pdb
.. _capability-based security: https://en.wikipedia.org/wiki/Capability-based_security
Feel like saying hi? Feel like saying hi?
-------------------- --------------------
@ -700,14 +721,4 @@ This project is very much coupled to the ongoing development of
community). If you want to help, have suggestions or just want to community). If you want to help, have suggestions or just want to
say hi, please feel free to ping me on the `trio gitter channel`_! say hi, please feel free to ping me on the `trio gitter channel`_!
.. _supervisors: https://github.com/tgoodlet/tractor/issues/22
.. _nanomsg: https://nanomsg.github.io/nng/index.html
.. _context: https://nanomsg.github.io/nng/man/tip/nng_ctx.5
.. _gossip protocol: https://en.wikipedia.org/wiki/Gossip_protocol
.. _trio gitter channel: https://gitter.im/python-trio/general .. _trio gitter channel: https://gitter.im/python-trio/general
.. _celery: http://docs.celeryproject.org/en/latest/userguide/debugging.html
.. _pdb++: https://github.com/antocuni/pdb
.. _msgpack: https://en.wikipedia.org/wiki/MessagePack
.. _asyncitertools: https://github.com/vodik/asyncitertools
.. _capability-based security: https://en.wikipedia.org/wiki/Capability-based_security

View File

@ -11,7 +11,7 @@ import trio # type: ignore
from trio import MultiError from trio import MultiError
from .log import get_console_log, get_logger, get_loglevel from .log import get_console_log, get_logger, get_loglevel
from ._ipc import _connect_chan, Channel, Context from ._ipc import _connect_chan, Channel
from ._actor import ( from ._actor import (
Actor, _start_actor, Arbiter, get_arbiter, find_actor, wait_for_actor Actor, _start_actor, Arbiter, get_arbiter, find_actor, wait_for_actor
) )
@ -93,14 +93,16 @@ def run(
name: str = None, name: str = None,
arbiter_addr: Tuple[str, int] = ( arbiter_addr: Tuple[str, int] = (
_default_arbiter_host, _default_arbiter_port), _default_arbiter_host, _default_arbiter_port),
spawn_method: str = 'forkserver', # the `multiprocessing` start method:
# https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
start_method: str = 'forkserver',
**kwargs: typing.Dict[str, typing.Any], **kwargs: typing.Dict[str, typing.Any],
) -> Any: ) -> Any:
"""Run a trio-actor async function in process. """Run a trio-actor async function in process.
This is tractor's main entry and the start point for any async actor. This is tractor's main entry and the start point for any async actor.
""" """
_spawn.try_set_start_method(spawn_method) _spawn.try_set_start_method(start_method)
return trio.run(_main, async_fn, args, kwargs, name, arbiter_addr) return trio.run(_main, async_fn, args, kwargs, name, arbiter_addr)

View File

@ -24,12 +24,15 @@ def try_set_start_method(name: str) -> mp.context.BaseContext:
global _ctx global _ctx
allowed = mp.get_all_start_methods() allowed = mp.get_all_start_methods()
if name not in allowed:
name == 'spawn'
assert name in allowed assert name in allowed
if name == 'forkserver': if name not in allowed:
name == 'spawn'
elif name == 'fork':
raise ValueError(
"`fork` is unsupported due to incompatibility with `trio`"
)
elif name == 'forkserver':
_forkserver_hackzorz.override_stdlib() _forkserver_hackzorz.override_stdlib()
_ctx = mp.get_context(name) _ctx = mp.get_context(name)