forked from goodboy/tractor
Document the mp start method more explicitly
parent
8eb138b8a7
commit
c3daf73112
45
README.rst
45
README.rst
|
@ -26,12 +26,12 @@ An async-native "`actor model`_" built on trio_ and multiprocessing_.
|
||||||
``tractor`` lets you spawn ``trio`` *"actors"*: processes which each run a ``trio`` scheduler and task
|
``tractor`` lets you spawn ``trio`` *"actors"*: processes which each run a ``trio`` scheduler and task
|
||||||
tree (also known as an `async sandwich`_). *Actors* communicate by exchanging asynchronous messages_ over
|
tree (also known as an `async sandwich`_). *Actors* communicate by exchanging asynchronous messages_ over
|
||||||
channels_ and avoid sharing any state. This model allows for highly distributed software architecture
|
channels_ and avoid sharing any state. This model allows for highly distributed software architecture
|
||||||
which works just as well on multiple cores as it does over many hosts. ``tractor`` is an actor-model-*like*
|
which works just as well on multiple cores as it does over many hosts.
|
||||||
system in the sense that it adheres to the `3 axioms`_ but not does not (yet) fufill all "unrequirements_" in
|
|
||||||
practice.
|
|
||||||
|
|
||||||
``tractor`` takes inspiration from pulsar_ and execnet_ but attempts to be more focussed on sophistication of
|
``tractor`` is an actor-model-*like* system in the sense that it adheres to the `3 axioms`_ but not does
|
||||||
the lower level distributed architecture as well as have first class support for streaming using `async generators`_.
|
not (yet) fufill all "unrequirements_" in practice. The API and design takes inspiration from pulsar_ and
|
||||||
|
execnet_ but attempts to be more focussed on sophistication of the lower level distributed architecture as
|
||||||
|
well as have first class support for streaming using `async generators`_.
|
||||||
|
|
||||||
The first step to grok ``tractor`` is to get the basics of ``trio`` down.
|
The first step to grok ``tractor`` is to get the basics of ``trio`` down.
|
||||||
A great place to start is the `trio docs`_ and this `blog post`_.
|
A great place to start is the `trio docs`_ and this `blog post`_.
|
||||||
|
@ -84,8 +84,8 @@ No PyPi release yet!
|
||||||
|
|
||||||
Windows "gotchas"
|
Windows "gotchas"
|
||||||
*****************
|
*****************
|
||||||
`tractor` uses the stdlib's `multiprocessing` module internally which
|
`tractor` internally uses the stdlib's `multiprocessing` package which
|
||||||
*can* have some *gotchas* on Windows, namely the need for calling
|
*can* have some gotchas on Windows. Namely, the need for calling
|
||||||
`freeze_support()`_ inside the ``__main__`` context. See `#61`_ for the
|
`freeze_support()`_ inside the ``__main__`` context. See `#61`_ for the
|
||||||
deats.
|
deats.
|
||||||
|
|
||||||
|
@ -647,6 +647,9 @@ multiple tasks streaming responses concurrently:
|
||||||
|
|
||||||
The context notion comes from the context_ in nanomsg_.
|
The context notion comes from the context_ in nanomsg_.
|
||||||
|
|
||||||
|
.. _context: https://nanomsg.github.io/nng/man/tip/nng_ctx.5
|
||||||
|
.. _msgpack: https://en.wikipedia.org/wiki/MessagePack
|
||||||
|
|
||||||
|
|
||||||
Running actors standalone
|
Running actors standalone
|
||||||
*************************
|
*************************
|
||||||
|
@ -662,6 +665,16 @@ need to hop into a debugger. You just need to pass the existing
|
||||||
tractor.run(main, arbiter_addr=('192.168.0.10', 1616))
|
tractor.run(main, arbiter_addr=('192.168.0.10', 1616))
|
||||||
|
|
||||||
|
|
||||||
|
Choosing a ``multiprocessing`` *start method*
|
||||||
|
*********************************************
|
||||||
|
``tractor`` supports selection of the `multiprocessing start method`_ via
|
||||||
|
a ``start_method`` kwarg to ``tractor.run()``. Note that on Windows
|
||||||
|
*spawn* it the only supported method and on nix systems *forkserver* is
|
||||||
|
selected by default for speed.
|
||||||
|
|
||||||
|
.. _multiprocessing start method: https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
|
||||||
|
|
||||||
|
|
||||||
Enabling logging
|
Enabling logging
|
||||||
****************
|
****************
|
||||||
Considering how complicated distributed software can become it helps to know
|
Considering how complicated distributed software can become it helps to know
|
||||||
|
@ -692,6 +705,14 @@ Stuff I'd like to see ``tractor`` do real soon:
|
||||||
- support for reactive programming primitives and native support for asyncitertools_ like libs
|
- support for reactive programming primitives and native support for asyncitertools_ like libs
|
||||||
- introduction of a `capability-based security`_ model
|
- introduction of a `capability-based security`_ model
|
||||||
|
|
||||||
|
.. _supervisors: https://github.com/tgoodlet/tractor/issues/22
|
||||||
|
.. _nanomsg: https://nanomsg.github.io/nng/index.html
|
||||||
|
.. _gossip protocol: https://en.wikipedia.org/wiki/Gossip_protocol
|
||||||
|
.. _celery: http://docs.celeryproject.org/en/latest/userguide/debugging.html
|
||||||
|
.. _asyncitertools: https://github.com/vodik/asyncitertools
|
||||||
|
.. _pdb++: https://github.com/antocuni/pdb
|
||||||
|
.. _capability-based security: https://en.wikipedia.org/wiki/Capability-based_security
|
||||||
|
|
||||||
|
|
||||||
Feel like saying hi?
|
Feel like saying hi?
|
||||||
--------------------
|
--------------------
|
||||||
|
@ -700,14 +721,4 @@ This project is very much coupled to the ongoing development of
|
||||||
community). If you want to help, have suggestions or just want to
|
community). If you want to help, have suggestions or just want to
|
||||||
say hi, please feel free to ping me on the `trio gitter channel`_!
|
say hi, please feel free to ping me on the `trio gitter channel`_!
|
||||||
|
|
||||||
|
|
||||||
.. _supervisors: https://github.com/tgoodlet/tractor/issues/22
|
|
||||||
.. _nanomsg: https://nanomsg.github.io/nng/index.html
|
|
||||||
.. _context: https://nanomsg.github.io/nng/man/tip/nng_ctx.5
|
|
||||||
.. _gossip protocol: https://en.wikipedia.org/wiki/Gossip_protocol
|
|
||||||
.. _trio gitter channel: https://gitter.im/python-trio/general
|
.. _trio gitter channel: https://gitter.im/python-trio/general
|
||||||
.. _celery: http://docs.celeryproject.org/en/latest/userguide/debugging.html
|
|
||||||
.. _pdb++: https://github.com/antocuni/pdb
|
|
||||||
.. _msgpack: https://en.wikipedia.org/wiki/MessagePack
|
|
||||||
.. _asyncitertools: https://github.com/vodik/asyncitertools
|
|
||||||
.. _capability-based security: https://en.wikipedia.org/wiki/Capability-based_security
|
|
||||||
|
|
|
@ -11,7 +11,7 @@ import trio # type: ignore
|
||||||
from trio import MultiError
|
from trio import MultiError
|
||||||
|
|
||||||
from .log import get_console_log, get_logger, get_loglevel
|
from .log import get_console_log, get_logger, get_loglevel
|
||||||
from ._ipc import _connect_chan, Channel, Context
|
from ._ipc import _connect_chan, Channel
|
||||||
from ._actor import (
|
from ._actor import (
|
||||||
Actor, _start_actor, Arbiter, get_arbiter, find_actor, wait_for_actor
|
Actor, _start_actor, Arbiter, get_arbiter, find_actor, wait_for_actor
|
||||||
)
|
)
|
||||||
|
@ -93,14 +93,16 @@ def run(
|
||||||
name: str = None,
|
name: str = None,
|
||||||
arbiter_addr: Tuple[str, int] = (
|
arbiter_addr: Tuple[str, int] = (
|
||||||
_default_arbiter_host, _default_arbiter_port),
|
_default_arbiter_host, _default_arbiter_port),
|
||||||
spawn_method: str = 'forkserver',
|
# the `multiprocessing` start method:
|
||||||
|
# https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
|
||||||
|
start_method: str = 'forkserver',
|
||||||
**kwargs: typing.Dict[str, typing.Any],
|
**kwargs: typing.Dict[str, typing.Any],
|
||||||
) -> Any:
|
) -> Any:
|
||||||
"""Run a trio-actor async function in process.
|
"""Run a trio-actor async function in process.
|
||||||
|
|
||||||
This is tractor's main entry and the start point for any async actor.
|
This is tractor's main entry and the start point for any async actor.
|
||||||
"""
|
"""
|
||||||
_spawn.try_set_start_method(spawn_method)
|
_spawn.try_set_start_method(start_method)
|
||||||
return trio.run(_main, async_fn, args, kwargs, name, arbiter_addr)
|
return trio.run(_main, async_fn, args, kwargs, name, arbiter_addr)
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -24,12 +24,15 @@ def try_set_start_method(name: str) -> mp.context.BaseContext:
|
||||||
global _ctx
|
global _ctx
|
||||||
|
|
||||||
allowed = mp.get_all_start_methods()
|
allowed = mp.get_all_start_methods()
|
||||||
if name not in allowed:
|
|
||||||
name == 'spawn'
|
|
||||||
|
|
||||||
assert name in allowed
|
assert name in allowed
|
||||||
|
|
||||||
if name == 'forkserver':
|
if name not in allowed:
|
||||||
|
name == 'spawn'
|
||||||
|
elif name == 'fork':
|
||||||
|
raise ValueError(
|
||||||
|
"`fork` is unsupported due to incompatibility with `trio`"
|
||||||
|
)
|
||||||
|
elif name == 'forkserver':
|
||||||
_forkserver_hackzorz.override_stdlib()
|
_forkserver_hackzorz.override_stdlib()
|
||||||
|
|
||||||
_ctx = mp.get_context(name)
|
_ctx = mp.get_context(name)
|
||||||
|
|
Loading…
Reference in New Issue