jc211
/
tractor
forked from goodboy/tractor
1
0
Fork 0
Code Pull Requests Activity
822 Commits
211 Branches
0 Tags
9.3 MiB
Commit Graph

822 Commits (22383d1ed9a30f5d1a55d5f77fe882832d2c729b)

Author SHA1 Message Date
Tyler Goodlet 22383d1ed9 Add `msgspec` mentions to readme 2021-09-07 21:27:05 -04:00
Tyler Goodlet 0d41f1410f Add a stream type factory 2021-09-07 21:07:33 -04:00
Tyler Goodlet 6cf4a80fe4 Don't expect list value from registry 2021-09-07 20:25:40 -04:00
Tyler Goodlet c188008844 Ugh, appease mypy yet again 2021-09-07 20:24:02 -04:00
Tyler Goodlet 593fd24a9e Attempt to gracefully handle channel breakage? 2021-09-07 16:59:40 -04:00
Tyler Goodlet bb8452dbdb Ensure tuple for passed in arbiter addr 2021-09-06 12:07:09 -04:00
Tyler Goodlet 82999d10df Pin to latest and greatest `msgspec` 2021-09-06 11:53:50 -04:00
Tyler Goodlet a085111173 Call registry getter method in test 2021-09-06 11:53:50 -04:00
Tyler Goodlet c46bf6b3c4 Cast `defaultdict` to `dict` for registry get 2021-09-06 11:53:50 -04:00
Tyler Goodlet b8b264ae54 Map broken stream errs to transport closed; msgspec seems to be racy 2021-09-05 18:48:25 -04:00
Tyler Goodlet c27b00687c Convert actor UIDs to hashable tuples 
`msgspec` sends python lists over the wire
(https://github.com/jcrist/msgspec/issues/30) which is fine and dandy
but we use them as lookup keys so we need to be sure we tuple-cast
first.
 2021-09-05 18:27:50 -04:00
Tyler Goodlet fa6d9bef52 Fix log levels 2021-09-05 16:29:16 -04:00
Tyler Goodlet bdde646d4c Add msgspec installs, drop py3.7 2021-09-05 16:26:36 -04:00
Tyler Goodlet 7d0541d864 Mypy fixes to enforce uid tuple 2021-09-05 16:26:36 -04:00
Tyler Goodlet 7888de6070 Fix py version classifier 2021-09-05 16:26:36 -04:00
Tyler Goodlet 3b2598a060 Pkg `msgpec` as optional dep, load transport type if importable 2021-09-05 16:26:34 -04:00
Tyler Goodlet eb44244f24 Accept transport closed error during handshake and msg loop 2021-09-05 16:25:41 -04:00
Tyler Goodlet 7b902b7e9c Drop happy eyeballs inf delay 2021-09-05 16:23:33 -04:00
Tyler Goodlet fdd2da238a Add our own "transport closed" signal 
This change some super old (and bad) code from the project's very early
days. For some redic reason i must have thought masking `trio`'s
internal stream / transport errors and a TCP EOF as `StopAsyncIteration`
somehow a good idea. The reality is you probably
want to know the difference between an unexpected transport error
and a simple EOF lol. This begins to resolve that by adding our own
special `TransportClosed` error to signal the "graceful" termination of
a channel's underlying transport. Oh, and this builds on the `msgspec`
integration which helped shed light on the core issues here B)
 2021-09-05 16:23:10 -04:00
Tyler Goodlet bc6af2219e Add streaming decode support for `msgspec` 
Add a `tractor._ipc.MsgspecStream` type which can be swapped in for
`msgspec` serialization transparently. A small msg-length-prefix framing
is implemented as part of the type and we use
`tricycle.BufferedReceieveStream` to handle buffering logic for the
underlying transport.

Notes:
- had to force cast a few more list  -> tuple spots due to no native
  `tuple`decode-by-default in `msgspec`: https://github.com/jcrist/msgspec/issues/30
- the framing can be understood by this protobuf walkthrough:
  https://eli.thegreenplace.net/2011/08/02/length-prefix-framing-for-protocol-buffers
- `tricycle` becomes a new dependency
 2021-09-05 16:18:42 -04:00
Tyler Goodlet 5e03108211 Always cast arbiter addr to tuple 2021-09-05 15:27:08 -04:00
Tyler Goodlet 132b9651dd Add `tricycle` and `msgspec` deps 2021-09-05 15:27:07 -04:00
Tyler Goodlet adc77861bb Try out `msgspec` in our msgpack stream channel 
Can only really use an encoder currently since there is no streaming api
in `msgspec` as of currently. See jcrist/msgspec#27.

Not sure if any encoding speedups are currently noticeable especially
without any validation going on yet XD.

First experiments toward #196
 2021-09-05 15:26:00 -04:00
Tyler Goodlet 93a83eab1c Cast to tuples for all uids explicitly 2021-09-05 15:22:40 -04:00
goodboy 3f1bc37143
Merge pull request #229 from goodboy/live_on_air_from_tokio 
`tokio`-style broadcast channels
 2021-09-03 07:29:29 -04:00
Tyler Goodlet 1137a9e7ac Fix 404ed tokio urls 2021-09-02 21:12:54 -04:00
Tyler Goodlet bcf5b9fd18 Add news fragment 2021-09-02 21:12:54 -04:00
Tyler Goodlet 2745a2b1dc Solve first-recv-cancelled by recursive `.receive()` on wake 2021-09-02 21:12:54 -04:00
Tyler Goodlet 5881a82d2a Add a first receiver is cancelled test 2021-09-02 21:12:54 -04:00
Tyler Goodlet b7b489dd07 Drop shielded stream api usage 2021-09-02 21:12:54 -04:00
Tyler Goodlet d9bb52fe7b Store array `maxlen` in state singleton 
The `collections.deque` takes care of array length truncation of values
for us implicitly but in the future we'll likely want this value exposed
to alternate array implementations. This patch is to provide for that as
well as make `mypy` happy since the `dequeu.maxlen` can also be `None`.
 2021-09-02 21:12:54 -04:00
Tyler Goodlet 9258f79510 Don't wake sibling bcast consumers on a cancelled call 2021-09-02 21:12:54 -04:00
Tyler Goodlet 5c6355062c Shorten sequence length for test speedup 2021-09-02 21:12:54 -04:00
Tyler Goodlet 44ef26bb18 Shorten default feeder mem chan size to 64 2021-09-02 21:12:54 -04:00
Tyler Goodlet d9e793d4ba Can't use built-in generics till 3.9... 2021-09-02 21:12:54 -04:00
Tyler Goodlet 7857a9ac6d Add `shield: bool` kwarg to `Portal.open_stream_from()` 2021-09-02 21:12:54 -04:00
Tyler Goodlet 5182ee7782 Add a "faster task is cancelled" test 2021-09-02 21:12:54 -04:00
Tyler Goodlet 39cf9af9fc Rename test module 2021-09-02 21:12:54 -04:00
Tyler Goodlet 63ec740e27 Add some bcaster ref sanity asserts around subscriptions 2021-09-02 21:12:54 -04:00
Tyler Goodlet 0d70e3081a Add laggy parent stream tests 
Add a couple more tests to check that a parent and sub-task stream can
be lagged and recovered (depending on who's slower). Factor some of the
test machinery into a new ctx mngr to make it all happen.
 2021-09-02 21:12:54 -04:00
Tyler Goodlet 093e7d921c Instance ids are ints 2021-09-02 21:12:54 -04:00
Tyler Goodlet d7ad8982ff Add subscribe after close test 2021-09-02 21:12:54 -04:00
Tyler Goodlet bec3f5999d Drop uuid4 keys, raise closed error on subscription after close 2021-09-02 21:12:54 -04:00
Tyler Goodlet 2bad2bac50 Don't enable debug mode..it borks CI 2021-09-02 21:12:54 -04:00
Tyler Goodlet a4cb0ef21f Fix `.receive()` re-assignment, drop `.clone()` 2021-09-02 21:12:54 -04:00
Tyler Goodlet 236ed0b0dd Initial broadcaster tests including one to test our `MsgStream.subscribe()` api 2021-09-02 21:12:54 -04:00
Tyler Goodlet 346b5d2eda Blade runner it 
Get rid of all the (requirements for) clones of the underlying
receivable. We can just use a uuid generated key for each instance
(thinking now this can probably just be `id(self)`). I'm fully convinced
now that channel cloning is only a source of confusion and anti-patterns
when we already have nurseries to define resource lifetimes. There is no
benefit in particular when you allocate subscriptions using a context
manager (not sure why `trio.open_memory_channel()` doesn't enforce
this).

Further refinements:
- add a `._closed` state that will error the receiver on reuse
- drop module script section;  it's been moved to a real test
- call the "receiver" duck-type stub a new name
 2021-09-02 21:12:54 -04:00
Tyler Goodlet 6c17c7367a Store handle to underlying channel's `.receive()` 
This allows for wrapping an existing stream by re-assigning its receive
method to the allocated broadcaster's `.receive()` so as to avoid
expecting any original consumer(s) of the stream to now know about the
broadcaster; this instead mutates the stream to delegate to the new
receive call behind the scenes any time `.subscribe()` is called.

Add a `typing.Protocol` for so called "cloneable channels" until we
decide/figure out a better keying system for each subscription and
mask all undesired typing failures.
 2021-09-02 21:12:54 -04:00
Tyler Goodlet 2d1c24112b Add subscription support to message streams 
Add `ReceiveMsgStream.subscribe()` which allows allocating a broadcast
receiver around the stream for use by multiple actor-local consumer
tasks. Entering this context manager idempotently mutates the stream's
receive machinery which for now can not be undone. Move `.clone()` to
the receive stream type.

Resolves #204
 2021-09-02 21:12:54 -04:00
Tyler Goodlet a12b1fc631 Drop optimization check, binance made its point 2021-09-02 21:12:54 -04:00