Commit Graph

2102 Commits (2679d6261e7e5187372cb958d6fd0c143081585c)

Author SHA1 Message Date
Tyler Goodlet 2679d6261e Disable re-connect for now in ib script 2022-04-16 15:39:24 -04:00
Tyler Goodlet 48c989d400 Ensure bfqsn is lower cased for feed api consumers
Also, Start tinkering with `tractor.trionics.ipython_embed()`

In effort to get back to a usable REPL around the mkts client
this adds usage of the new `tractor` integration api as well as logic
for skipping backfilling if existing tsdb arrays are found.
2022-04-16 15:39:24 -04:00
Tyler Goodlet 831c23872c Try downsampling mkts data 2022-04-16 15:39:24 -04:00
Tyler Goodlet 008e153f36 Drop old type annot 2022-04-16 15:39:24 -04:00
Tyler Goodlet 38be93a009 Comment each special key combo 2022-04-16 15:39:24 -04:00
Tyler Goodlet 3bf3f450f4 Load any symbol-matching shm array if no `marketstored` found 2022-04-16 15:39:24 -04:00
Tyler Goodlet 4e1ab378f4 Get ib key hack script to work with reconnect 2022-04-16 15:39:24 -04:00
Tyler Goodlet 8d77d177bf Move ib data reset script into a new `scripts/` dir 2022-04-16 15:39:24 -04:00
Tyler Goodlet 36463dddbe Use new `tractor.query_actor()` for service checking 2022-04-16 15:39:24 -04:00
Tyler Goodlet e399b74b67 Return all timeframe arrays if `timeframe` not passed as input 2022-04-16 15:39:24 -04:00
Tyler Goodlet 15b609bf13 Make `pikerd` work again without `--tsdb` flag 2022-04-16 15:39:24 -04:00
Tyler Goodlet dfc3fb76fd Add a service checker predicate 2022-04-16 15:39:24 -04:00
Tyler Goodlet 881b1afc12 Allow kill-child-proc-with-root-perms to fail silently in `tractor` reaping 2022-04-16 15:39:24 -04:00
Tyler Goodlet 9878735ff3 Proxy `marketstore` container log level to our own 2022-04-16 15:39:24 -04:00
Tyler Goodlet 1389ee51f3 Prototype a high level `Storage` api
Starts a wrapper around the `marketstore` client to do basic ohlcv query
and retrieval and prototypes out write methods for ohlc and tick.
Try to connect to `marketstore` automatically (which will fail if not
started currently) but we will eventually first do a service query.

Further:

- get `pikerd` working with and without `--tsdb` flag.
- support spawning `brokerd` with no real-time quotes.
- bring back in "fqsn" support that was originally not
  in this history before commits factoring.
2022-04-16 15:39:24 -04:00
Tyler Goodlet 6225e8a58e Doc str formatting 2022-04-16 15:39:24 -04:00
Tyler Goodlet 2b48943066 Add latency measures around diffs/writes to mkts 2022-04-16 15:39:24 -04:00
Tyler Goodlet 92d074f308 Prototype out writing `1Sec` OHLCV data 2022-04-16 15:39:24 -04:00
Tyler Goodlet 49a280ce14 Better doc string 2022-04-16 15:39:24 -04:00
Tyler Goodlet 42e5f48345 Persist backing `/data/` filesystem across container runs 2022-04-16 15:39:24 -04:00
Tyler Goodlet 18344603e6 Get basic OHLCV writes working with `anyio` client 2022-04-16 15:39:24 -04:00
Tyler Goodlet ae1d9efa97 Pass in daemon name to `start_ahab()` 2022-04-16 15:39:24 -04:00
Tyler Goodlet f1747749e7 Map the grpc port and add graceful container teardown
Not sure how I missed mapping the 5995 grpc port 🤦; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-04-16 15:39:24 -04:00
Tyler Goodlet cdd03759bf Revive `ms-shell` sub-cmd 2022-04-16 15:39:24 -04:00
Tyler Goodlet 00a36826c2 Add WIP backfiller from data feed helper 2022-04-16 15:39:24 -04:00
Tyler Goodlet 3aa8044aa1 Better handle nested erros from docker client 2022-04-16 15:39:24 -04:00
Tyler Goodlet f0ae7edb00 Add back in legacy write loop for reference 2022-04-16 15:39:24 -04:00
Tyler Goodlet 797ba15923 Add back in OHLCV dtype template and client side ws streamer 2022-04-16 15:39:24 -04:00
Tyler Goodlet b8f86d4599 Drop ununsed `Services` ref 2022-04-16 15:39:24 -04:00
Tyler Goodlet 908093264a Py3.9+ type updates 2022-04-16 15:39:24 -04:00
Tyler Goodlet 3ef68f78f7 Add `--tsdb` flag to start `marketstore` with `pikerd` 2022-04-16 15:39:24 -04:00
Tyler Goodlet d095c21885 De-escalate sudo perms in `pikerd` once docker spawns 2022-04-16 15:39:24 -04:00
Tyler Goodlet 52dc0989ac Handle the non-root perms case specifically too 2022-04-16 15:39:24 -04:00
Tyler Goodlet 6f749c340d Add explicit no-docker error and supervisor start task-func 2022-04-16 15:39:24 -04:00
Tyler Goodlet dcff80131f Type annot updates 2022-04-16 15:39:24 -04:00
Tyler Goodlet 6cd829d1fd Drop old client instantiate line 2022-04-16 15:39:24 -04:00
Tyler Goodlet 76fff18b1a Drop import, it's got madness with and SIGINT? 2022-04-16 15:39:24 -04:00
Tyler Goodlet 88e1befffe Add a super simple `marketstore` container supervisor 2022-04-16 15:39:24 -04:00
Tyler Goodlet c1579b630e Extract non-sudo user for config dir path 2022-04-16 15:39:24 -04:00
Tyler Goodlet 7dfdd0f00b Basic module-script for spawning `marketstore`, needs correct bind mount usage 2022-04-16 15:39:24 -04:00
Guillermo Rodriguez 14a893a5e5 Still WIP, switch to using new marketstore client, missing streaming from marketstore 2022-04-16 15:39:24 -04:00
Guillermo Rodriguez c91599472d Simplify and optimize tick format, similar to techtonicdb's 2022-04-16 15:39:24 -04:00
Guillermo Rodriguez 9e4dec98da Add multi ingestor support and update to new feed API 2022-04-16 15:39:24 -04:00
Tyler Goodlet 8800ee0661 Only update y mxmn from L1 when last index in view 2022-04-16 15:39:24 -04:00
Tyler Goodlet 885116ae46 Only update y-range from L1 mxmn when last index in view
We still have to always keep track of the last max and min
though.
2022-04-16 15:38:26 -04:00
Tyler Goodlet eab9127170 Drop old `pyqtgraph` downsample code 2022-04-16 15:38:26 -04:00
Tyler Goodlet c675773de4 Tweak log-scaler for more detail 2022-04-16 15:38:26 -04:00
Tyler Goodlet b20245398d Drop `._ic` debugging prints 2022-04-16 15:38:26 -04:00
Tyler Goodlet 2abb3ec84d Display loop mega-cleanup
The most important changes include:
- iterating the new `Flow` type and updating graphics
- adding detailed profiling
- increasing the min uppx before graphics updates are throttled
- including the L1 spread in y-range calcs so that you never have the
  bid/ask go "out of view"..
- pass around `Flow`s instead of shms
- drop all the old prototyped downsampling code
2022-04-16 15:38:26 -04:00
Tyler Goodlet 6843f9a515 Allow passing a `plotItem` to `.draw_curve()`
If manually managing an overlay you'll likely call `.overlay_plotitem()`
and then a plotting method so we need to accept a plot item input so
that the chart's pi doesn't get assigned incorrectly in the `Flow` entry
(though it is by default if no input is provided).

More,
- add a `Flow.graphics` field and set it to the `pg.GraphicsObject`.
- make `Flow.maxmin()` return `None` in the "can't calculate" cases.
2022-04-16 15:38:26 -04:00