Commit Graph

16 Commits (54712827eeacab404f9e8a779c482ad9b9e850d5)

Author SHA1 Message Date
Tyler Goodlet 54712827ee Fix context attr lookup.. 2022-01-25 07:57:01 -05:00
Tyler Goodlet bcc8d8a0d5 Simplify throttle loop to a single while block
This should in theory result in increased burstiness since we remove
the plain `trio.sleep()` and instead always wait on the receive channel
as much as possible until the `trio.move_on_after()` (+ time diffing
calcs) times out and signals the next throttled send cycle. This also is
slightly easier to grok code-wise instead of the `try, except` and
another tight while loop until a `trio.WouldBlock`. The only simpler
way i can think to do it is with 2 tasks: 1 to collect ticks and the
other to read and send at the throttle rate.

Comment out the log msg for now to avoid latency and add much more
detailed comments. Add an overrun log msg to the main sample loop.
2021-12-09 08:23:59 -05:00
Tyler Goodlet 92d7ffd332 WIP fsp output throttling - not working yet 2021-11-01 13:28:57 -04:00
Tyler Goodlet e95589e5b0 Ignore ohlc step stream subs lookup errors 2021-09-06 09:28:11 -04:00
Tyler Goodlet fa88d91b8d Add breakpoint on bcast lag for testing 2021-09-06 09:28:11 -04:00
Tyler Goodlet 2a9d24ccac Remove dead OHLC index consumers from subs list on error 2021-08-26 17:04:59 -04:00
Tyler Goodlet 2f5abaa47a Add njs token bucket gist url 2021-08-26 17:04:59 -04:00
Tyler Goodlet f4a998655b Feed detach must explicitly unsub throttled streams
If a client attaches to a quotes data feed and requests a throttle rate,
be sure to unsub that side-band memchan + task when it detaches and
especially so on any transport connection error.

Also, use an explicit `tractor.Context.cancel()` on the client feed
block exit since we removed the implicit cancel option from the
`tractor` api.
2021-07-07 07:51:09 -04:00
Tyler Goodlet 8b6fb83257 Pop subscriber streams on connection errors 2021-07-07 07:51:09 -04:00
Tyler Goodlet ccf81520cb First attempt data feed side quote throttling
Adding binance's "hft" ws feeds has resulted in a lot of context
switching in our Qt charts, so much so it's chewin CPU and definitely
worth it to throttle to the detected display rate as per discussion in
issue #192.

This is a first very very naive attempt at throttling L1 tick feeds on
the `brokerd` end (producer side) using a constant and uniform delivery
rate by way of a `trio` task + mem chan.  The new func is
`data._sampling.uniform_rate_send()`. Basically if a client request
a feed and provides a throttle rate we just spawn a task and queue up
ticks until approximately the next display rate's worth period of time
has passed before forwarding. It's definitely nothing fancy but does
provide fodder and a start point for an up and coming queueing eng to
start digging into both #107 and #109 ;)
2021-06-14 21:43:19 -04:00
Tyler Goodlet 57a35a3c6c Port feed bus endpoint to a `@tractor.context` 2021-06-14 10:55:01 -04:00
Tyler Goodlet 59475cfd81 Store lowercase symbols within piker data internals 2021-05-27 13:05:23 -04:00
Tyler Goodlet 0b36bacfb6 Avoid weird `pydantic` runtime warning 2021-05-24 12:22:17 -04:00
Tyler Goodlet 9c3f8ff050 Only do context unsubs in main feed bus path 2021-05-24 12:16:26 -04:00
Tyler Goodlet 3147a49384 Move sample-broadcast routine into sampling module 2021-04-10 14:18:40 -04:00
Tyler Goodlet 71d02db126 Rename "buffer" to "sampling" 2021-04-10 14:18:40 -04:00