forked from goodboy/tractor
1
0
Fork 0
Commit Graph

759 Commits (a27aca070e7866621bf7f719de718a07be8dbed1)

Author SHA1 Message Date
Tyler Goodlet a27aca070e Drop py3.7 from CI; cut run to 5mins 2021-09-01 06:49:09 -04:00
Tyler Goodlet 3ba01e7e40 Fix `.receive()` re-assignment, drop `.clone()` 2021-09-01 06:49:07 -04:00
Tyler Goodlet 843a713f5a Initial broadcaster tests including one to test our `MsgStream.subscribe()` api 2021-09-01 06:46:43 -04:00
Tyler Goodlet e9b038e87d Blade runner it
Get rid of all the (requirements for) clones of the underlying
receivable. We can just use a uuid generated key for each instance
(thinking now this can probably just be `id(self)`). I'm fully convinced
now that channel cloning is only a source of confusion and anti-patterns
when we already have nurseries to define resource lifetimes. There is no
benefit in particular when you allocate subscriptions using a context
manager (not sure why `trio.open_memory_channel()` doesn't enforce
this).

Further refinements:
- add a `._closed` state that will error the receiver on reuse
- drop module script section;  it's been moved to a real test
- call the "receiver" duck-type stub a new name
2021-09-01 06:46:43 -04:00
Tyler Goodlet 43820e194e Store handle to underlying channel's `.receive()`
This allows for wrapping an existing stream by re-assigning its receive
method to the allocated broadcaster's `.receive()` so as to avoid
expecting any original consumer(s) of the stream to now know about the
broadcaster; this instead mutates the stream to delegate to the new
receive call behind the scenes any time `.subscribe()` is called.

Add a `typing.Protocol` for so called "cloneable channels" until we
decide/figure out a better keying system for each subscription and
mask all undesired typing failures.
2021-09-01 06:46:40 -04:00
Tyler Goodlet eaa761b0c7 Add subscription support to message streams
Add `ReceiveMsgStream.subscribe()` which allows allocating a broadcast
receiver around the stream for use by multiple actor-local consumer
tasks. Entering this context manager idempotently mutates the stream's
receive machinery which for now can not be undone. Move `.clone()` to
the receive stream type.

Resolves #204
2021-09-01 06:40:25 -04:00
Tyler Goodlet db2f3f787a Drop optimization check, binance made its point 2021-09-01 06:39:21 -04:00
Tyler Goodlet b9863fc4ab Add common state delegate type for all consumers
For every set of broadcast receivers which pull from the same producer,
we need a singleton state for all of,
- subscriptions
- the sender ready event
- the queue

Add a `BroadcastState` dataclass for this and pass it to all
subscriptions. This makes the design much more like the built-in memory
channels which do something very similar with `MemoryChannelState`.

Use a `filter()` on the subs list in the sequence update step, plus some
other commented approaches we can try for speed.
2021-09-01 06:39:21 -04:00
Tyler Goodlet 9d12cc80dd Facepalm: use single `_subs` per clone set 2021-09-01 06:39:21 -04:00
Tyler Goodlet 3f9b860210 Obviously keying on tasks isn't going to work
Using the current task as a subscription key fails horribly as soon as
you hand off new subscription receiver to another task you've spawned..

Instead use the underlying ``trio.abc.ReceiveChannel.clone()`` as a key
(so i guess we're assuming cloning is supported by the underlying?)
which makes this all work just like default mem chans. As a bonus, now
we can just close the underlying rx (which may be a clone) on
`.aclose()` and everything should just work in terms of the underlying
channels lifetime (i think?).

Change `.subscribe()` to be async since the receive channel type
interface only expects `.aclose()` and it actually ends up being
nicer for 3.9+ style `async with` parentheses style anyway.
2021-09-01 06:39:21 -04:00
Tyler Goodlet eeca3d0d50 Rename to broadcast mod, don't expect mem chan specifically 2021-09-01 06:39:21 -04:00
Tyler Goodlet e1e3e6918c `Task` is hashable, so key on it 2021-09-01 06:39:21 -04:00
Tyler Goodlet dfc4082ad2 Simplify api around receive channel
Buncha improvements:
- pass in the queue via constructor
- tracking over all underlying memory channel closure using cloning
- do it like `tokio` and set lagged consumers to the last sequence
  before raising
- copy the subs on first receiver wakeup for iteration instead of
  iterating the table directly (and being forced to skip the current
  tasks sequence increment)
- implement `.aclose()` to close the underlying clone for this task
- make `broadcast_receiver()` just take the recv chan since it doesn't
  need anything on the send side.
2021-09-01 06:39:21 -04:00
Tyler Goodlet af6e8a64ad Ultra naive broadcast channel prototype 2021-09-01 06:39:21 -04:00
Tyler Goodlet 0c6e7ca351 Drop stream shielding; it was from a legacy design
The whole origin was not having an explicit open/close semantic for
streams. We have that now so this internal mechanic isn't needed and
further our streams become more correct by having `.aclose()` be
independent of cancellation.
2021-09-01 06:37:53 -04:00
goodboy a105e32e34
Merge pull request #226 from goodboy/debugger_test_tweaks
Debugger tests determinism
2021-08-03 08:59:02 -04:00
Tyler Goodlet ace1b1312c Terminate async gen example caller to avoid (benign) errors in console output 2021-08-02 21:49:15 -04:00
Tyler Goodlet 7431e8ea01 Don't log cancelled inceptions seen by the root 2021-08-02 21:15:42 -04:00
Tyler Goodlet 82999801a6 Drop leftover noisy exception logging.. 2021-08-02 16:56:00 -04:00
Tyler Goodlet c5c7e694ec Better early timeout handling, continue on child re-lock 2021-08-01 13:10:51 -04:00
goodboy b01f594025
Merge pull request #225 from goodboy/fix_news_links
Facepalm: fix rst hyperlinks
2021-08-01 11:52:39 -04:00
Tyler Goodlet a84a27c6d3 Facepalm: fix rst hyperlinks 2021-08-01 11:29:41 -04:00
goodboy 9cfec2d3b5
Merge pull request #224 from goodboy/wats_da_nooz
Add .alpha1 news flash
2021-08-01 11:24:12 -04:00
Tyler Goodlet 8a4a11b885 Add .alpha1 news flash 2021-08-01 10:58:41 -04:00
goodboy 14379a0f46
Merge pull request #220 from goodboy/ctx_debugger
Ctx debugger
2021-08-01 10:56:57 -04:00
Tyler Goodlet 674fbbc6b3 Docs and comments tidying 2021-08-01 10:44:13 -04:00
Tyler Goodlet f173012fea Handle repeat child tty-acquires race 2021-07-31 15:01:26 -04:00
Tyler Goodlet 6006adc0de Hide `_invoke()` tb, move actor error to exceptions mod 2021-07-31 13:56:26 -04:00
Tyler Goodlet 0afa7f0f8e Fix lock context manager return type 2021-07-31 12:50:58 -04:00
Tyler Goodlet b3d28a1ee4 Drop debugger path and duplicate func from rebasing 2021-07-31 12:46:40 -04:00
Tyler Goodlet 13b76c9439 Add fast fail test using the context api 2021-07-31 12:46:40 -04:00
Tyler Goodlet 632c666a7d Adjust debug tests to accomodate no more root clobbering
We may get multiple re-entries to debugger by `bp_forever` sub-actor
now since the root will incrementally try to cancel it only when the tty
lock is not held.
2021-07-31 12:46:40 -04:00
Tyler Goodlet 09f00a5a00 Go back to only logging tbs on no debugger 2021-07-31 12:46:40 -04:00
Tyler Goodlet 44bfacc0c2 Comment hard-kill-sidestep for now since nursery version covers it? 2021-07-31 12:46:40 -04:00
Tyler Goodlet 551816e80d Solve the root-cancels-child-in-tty-lock race
Finally this makes a cancelled root actor nursery not clobber child
tasks which request and lock the root's tty for the debugger repl.

Using an edge triggered event which is set after all fifo-lock-queued
tasks are complete, we can be sure that no lingering child tasks are
going to get interrupted during pdb use and tty lock acquisition.
Further, even if new tasks do queue up to get the lock, the root will
incrementally send cancel msgs to each sub-actor only once the tty is
not locked by a (set of) child request task(s). Add shielding around all
the critical sections where the child attempts to allocate the lock from
the root such that it won't be disrupted from cancel messages from the
root after the acquire lock transaction has started.
2021-07-31 12:46:40 -04:00
Tyler Goodlet be1fcb2a5b Distinguish between a local pdb unlock and the tty unlock in root 2021-07-31 12:46:40 -04:00
Tyler Goodlet ef89ed947a Fix hard kill in debug mode; only do it when debug lock is empty 2021-07-31 12:46:40 -04:00
Tyler Goodlet 5b3894827f Move some infos to runtime level 2021-07-31 12:46:40 -04:00
Tyler Goodlet 0fdcfa0ba1 Move debugger wait inside OCA nursery 2021-07-31 12:46:40 -04:00
Tyler Goodlet 37a1897c47 Don't shield debugger status wait; it causes hangs 2021-07-31 12:46:40 -04:00
Tyler Goodlet 0f2a39a311 Catch and delay errors in the root if debugger is active 2021-07-31 12:46:40 -04:00
Tyler Goodlet 23a1622256 Don't kill root's immediate children when in debug
If the root calls `trio.Process.kill()` on immediate child proc teardown
when the child is using pdb, we can get stdstreams clobbering that
results in a pdb++ repl where the user can't see what's been typed. Not
killing such children on cancellation / error seems to resolve this
issue whilst still giving reliable termination. For now, code that
special path until a time it becomes a problem for ensuring zombie
reaps.
2021-07-31 12:46:40 -04:00
Tyler Goodlet 63bdddd0c9 Add debug example that causes pdb stdin clobbering 2021-07-31 12:46:40 -04:00
Tyler Goodlet 49d439b681 Add some brief todo notes on idea of shielded breakpoint 2021-07-31 12:46:40 -04:00
Tyler Goodlet 6f05f5d5e6 Wait for debugger lock task context termination 2021-07-31 12:46:40 -04:00
Tyler Goodlet b369b91357 Fix up var naming and typing 2021-07-31 12:46:40 -04:00
Tyler Goodlet 969bce3aa4 Use context for remote debugger locking
A context is the natural fit (vs. a receive stream) for locking the root
proc's tty usage via it's `.started()` sync point. Simplify the
`_breakpoin()` routine to be a simple async func instead of all this
"returning a coroutine" stuff from before we decided that
`tractor.breakpoint()` must be async. Use `runtime` level for locking
logging making it easier to trace.
2021-07-31 12:46:40 -04:00
goodboy 54d8c93f1b
Merge pull request #219 from goodboy/bi_streaming_no_debugger_stuff
Initial bi-directional streaming support!
2021-07-31 12:27:53 -04:00
Tyler Goodlet 240f591234 Add 2-way streaming example to readme and scripts 2021-07-31 12:10:25 -04:00
Tyler Goodlet 69bbf6a957 Install test deps and py3.9 for type check job 2021-07-08 13:53:28 -04:00