Commit Graph

93 Commits (9f897ba75c371aad91f1d1f04c4d5fc0e9019591)

Author SHA1 Message Date
Tyler Goodlet 8b6fb83257 Pop subscriber streams on connection errors 2021-07-07 07:51:09 -04:00
Tyler Goodlet df2f6487ff Apply `brokerd` quote rate throttling when requested in `open_feed()` 2021-06-14 21:55:51 -04:00
Tyler Goodlet ccf81520cb First attempt data feed side quote throttling
Adding binance's "hft" ws feeds has resulted in a lot of context
switching in our Qt charts, so much so it's chewin CPU and definitely
worth it to throttle to the detected display rate as per discussion in
issue #192.

This is a first very very naive attempt at throttling L1 tick feeds on
the `brokerd` end (producer side) using a constant and uniform delivery
rate by way of a `trio` task + mem chan.  The new func is
`data._sampling.uniform_rate_send()`. Basically if a client request
a feed and provides a throttle rate we just spawn a task and queue up
ticks until approximately the next display rate's worth period of time
has passed before forwarding. It's definitely nothing fancy but does
provide fodder and a start point for an up and coming queueing eng to
start digging into both #107 and #109 ;)
2021-06-14 21:43:19 -04:00
Tyler Goodlet 57a35a3c6c Port feed bus endpoint to a `@tractor.context` 2021-06-14 10:55:01 -04:00
Tyler Goodlet 3455ebc60c Cast back to tuples after msgspec strips them... 2021-06-14 00:03:05 -04:00
Tyler Goodlet f4c9e20f0d Avoid `numpy` type usage on the wire 2021-06-01 10:48:23 -04:00
Tyler Goodlet edf3af9777 Drop waits to half-seconds 2021-06-01 10:48:23 -04:00
Tyler Goodlet 89dc3dde61 Move no bs websocket api into its own data module 2021-05-28 14:11:02 -04:00
Tyler Goodlet 50aff72f8e Don't require map (yet) in backend modules 2021-05-27 13:05:23 -04:00
Tyler Goodlet 59475cfd81 Store lowercase symbols within piker data internals 2021-05-27 13:05:23 -04:00
Tyler Goodlet 9bfc230dde Speedup: load provider searches async at startup 2021-05-27 13:05:23 -04:00
Tyler Goodlet 42fda2a9e6 Drop old code 2021-05-27 13:05:23 -04:00
Tyler Goodlet 1bd0ee8746 Support loading multi-brokerds search at startup 2021-05-27 13:05:23 -04:00
Tyler Goodlet 59377da0ad Load pause configs from backends on feed opens 2021-05-27 13:05:23 -04:00
Tyler Goodlet c9c686c98d Register context-stream with multi-search for each feed 2021-05-27 13:05:23 -04:00
Tyler Goodlet f19f4348e0 Decouple symbol search from feed type 2021-05-27 13:05:22 -04:00
Tyler Goodlet 5766dd518d Enforce lower case symbols across providers 2021-05-27 13:05:22 -04:00
Tyler Goodlet 534553a6f5 Add client side multi-provider feed symbol search 2021-05-27 13:05:22 -04:00
Tyler Goodlet dcc60524cb Add remote context allocation api to service daemon
This allows for more deterministically managing long running sub-daemon
services under `pikerd` using the new context api from `tractor`.
The contexts are allocated in an async exit stack and torn down at root
daemon termination. Spawn brokerds using this method by changing the
persistence entry point to be a `@tractor.context`.
2021-05-24 12:26:11 -04:00
Tyler Goodlet 0b36bacfb6 Avoid weird `pydantic` runtime warning 2021-05-24 12:22:17 -04:00
Tyler Goodlet 9c3f8ff050 Only do context unsubs in main feed bus path 2021-05-24 12:16:26 -04:00
Tyler Goodlet 2ef5a52521 Fix broken import in ib backend 2021-05-20 19:34:45 -04:00
Tyler Goodlet 0d9f091a34 Port data feed to new tractor stream api 2021-04-29 09:10:18 -04:00
Tyler Goodlet 7d6bc4d856 Move feed api(s) into new submodule
Also add a --pdb flag to chart app.
2021-04-15 10:43:29 -04:00
Tyler Goodlet 598aec579f Avoid token checking type mismatches 2021-04-10 14:18:40 -04:00
Tyler Goodlet fa7fadebac Report sym on unexpected open 2021-04-10 14:18:40 -04:00
Tyler Goodlet 3147a49384 Move sample-broadcast routine into sampling module 2021-04-10 14:18:40 -04:00
Tyler Goodlet 71d02db126 Rename "buffer" to "sampling" 2021-04-10 14:18:40 -04:00
Tyler Goodlet 100e27ac12 Task lock bus loading, always close feed stream on disconnect 2021-04-10 14:18:40 -04:00
Tyler Goodlet ce4144aace Deliver and utilise broker backend OHLC sample rate in init msg 2021-04-10 14:18:40 -04:00
Tyler Goodlet c05fc8991a Rework ohlc sampling to launch from .start()
Avoid bothering with a trio event and expect the caller to do manual shm
registering with the write loop. Provide OHLC sample period indexing
through a re-branded pub-sub func ``iter_ohlc_periods()``.
2021-04-10 14:18:40 -04:00
Tyler Goodlet 30dabbab44 Support backend volume summing; handle disconnects 2021-04-10 14:18:40 -04:00
Tyler Goodlet 4f51ca74f4 Broadcast all tick types to subs, not just trades 2021-04-10 14:18:40 -04:00
Tyler Goodlet 8ccf987d52 Fix typo 2021-04-10 14:18:40 -04:00
Tyler Goodlet a82f43e3a5 Rework data feed API to allow for caching streams
Move all feed/stream agnostic logic and shared mem writing into a new
set of routines inside the ``data`` sub-package. This lets us move
toward a more standard API for broker and data backends to provide
cache-able persistent streams to client apps.

The data layer now takes care of
- starting a single background brokerd task to start a stream for as
  symbol if none yet exists and register that stream for later lookups
- the existing broker backend actor is now always re-used if possible
  if it can be found in a service tree
- synchronization with the brokerd stream's startup sequence is now
  oriented around fast startup concurrency such that client code gets
  a handle to historical data and quote schema as fast as possible
- historical data loading is delegated to the backend more formally by
  starting a ``backfill_bars()`` task
- write shared mem in the brokerd task and only destruct it once requested
  either from the parent actor or further clients
- fully de-duplicate stream data by using a dynamic pub-sub strategy
  where new clients register for copies of the same quote set per symbol

This new API is entirely working with the IB backend; others will need
to be ported. That's to come shortly.
2021-04-10 14:18:40 -04:00
Tyler Goodlet 65e7680cdd Draft a feed cacheing sub-system 2021-04-10 14:18:40 -04:00
goodboy feb8d5bbad
Merge pull request #153 from wattygetlood/windows_patches
Windows patches
2021-04-06 11:55:51 -04:00
Guillermo Rodriguez 189c56c012 Add piker root daemon spawning machinery
Refactor maybe_spawn_brokerd to adapt to new process tree structure
and add a ``maybe_open_pikerd``.
2021-04-03 12:22:14 -04:00
wattygetlood 583cafc947 Avoid loading posix unlinker, use shm bytes size 2021-03-28 17:23:59 -04:00
Tyler Goodlet e1dc2b9225 Enable daemon debug through top level kwarg 2021-03-17 08:36:34 -04:00
Tyler Goodlet 624617d8e1 Don't run brokerds in debug mode by default 2021-03-11 21:44:10 -05:00
Tyler Goodlet e6ea053d40 Get kivy/questrade shit working again 2021-02-21 12:32:40 -05:00
Tyler Goodlet 8c757d0bdd Accept a symbol type key from broker 2021-02-21 11:42:19 -05:00
Tyler Goodlet ead2f77d40 Add a symbol/asset type key 2021-02-19 18:43:56 -05:00
Tyler Goodlet bf66eb0b3d Support lot tick size (mostly for crypto) 2021-02-08 06:42:59 -05:00
Tyler Goodlet c3fa31e731 Convert symbol type to use pydantic 2021-02-06 14:38:25 -05:00
Tyler Goodlet a8c4829cb6 Start using `tick_size` throughout charting
The min tick size is the smallest step an instrument can move in value
(think the number of decimals places of precision the value can have).

We start leveraging this in a few places:
- make our internal "symbol" type expose it as part of it's api
  so that it can be passed around by UI components
- in y-axis view box scaling, use it to keep the bid/ask spread (L1 UI)
  always on screen even in the case where the spread has moved further
  out of view then the last clearing price
- allows the EMS to determine dark order live order submission offsets
2021-02-06 11:35:12 -05:00
Tyler Goodlet 92efb8fd8e Expect new init message in feed from brokers 2021-01-22 22:56:42 -05:00
Tyler Goodlet 5327d7be5e Add screen acquire timeout loop 2021-01-22 16:46:39 -05:00
Tyler Goodlet f3ae8db04b Big refactor; start paper client 2021-01-18 19:56:35 -05:00