Commit Graph

2110 Commits (842142276848a7a2819953ed16b0eb7ef5e783c1)

Author SHA1 Message Date
Tyler Goodlet 8421422768 Ensure bfqsn is lower cased for feed api consumers
Also, Start tinkering with `tractor.trionics.ipython_embed()`

In effort to get back to a usable REPL around the mkts client
this adds usage of the new `tractor` integration api as well as logic
for skipping backfilling if existing tsdb arrays are found.
2022-04-30 11:38:24 -04:00
Tyler Goodlet 23d386e0f5 Try downsampling mkts data 2022-04-30 11:38:24 -04:00
Tyler Goodlet f0c4261aa4 Drop old type annot 2022-04-30 11:38:24 -04:00
Tyler Goodlet 2be8f63487 Comment each special key combo 2022-04-30 11:38:24 -04:00
Tyler Goodlet 24a07fd5e5 Load any symbol-matching shm array if no `marketstored` found 2022-04-30 11:38:24 -04:00
Tyler Goodlet a893537ade Get ib key hack script to work with reconnect 2022-04-30 11:38:24 -04:00
Tyler Goodlet 15d15fdfbf Move ib data reset script into a new `scripts/` dir 2022-04-30 11:38:24 -04:00
Tyler Goodlet d1b05246f8 Use new `tractor.query_actor()` for service checking 2022-04-30 11:38:24 -04:00
Tyler Goodlet 96ec4ba28b Return all timeframe arrays if `timeframe` not passed as input 2022-04-30 11:38:24 -04:00
Tyler Goodlet fa0be47d66 Make `pikerd` work again without `--tsdb` flag 2022-04-30 11:38:24 -04:00
Tyler Goodlet 65609a35dc Add a service checker predicate 2022-04-30 11:38:24 -04:00
Tyler Goodlet d8d7757e88 Allow kill-child-proc-with-root-perms to fail silently in `tractor` reaping 2022-04-30 11:38:24 -04:00
Tyler Goodlet c439e99f8a Proxy `marketstore` container log level to our own 2022-04-30 11:38:24 -04:00
Tyler Goodlet 5c2b9a01e9 Prototype a high level `Storage` api
Starts a wrapper around the `marketstore` client to do basic ohlcv query
and retrieval and prototypes out write methods for ohlc and tick.
Try to connect to `marketstore` automatically (which will fail if not
started currently) but we will eventually first do a service query.

Further:

- get `pikerd` working with and without `--tsdb` flag.
- support spawning `brokerd` with no real-time quotes.
- bring back in "fqsn" support that was originally not
  in this history before commits factoring.
2022-04-30 11:38:24 -04:00
Tyler Goodlet 15d3f99410 Doc str formatting 2022-04-30 11:38:24 -04:00
Tyler Goodlet ae8170204f Add latency measures around diffs/writes to mkts 2022-04-30 11:38:24 -04:00
Tyler Goodlet 7d628c4059 Prototype out writing `1Sec` OHLCV data 2022-04-30 11:38:24 -04:00
Tyler Goodlet 6dfe59cce6 Better doc string 2022-04-30 11:38:24 -04:00
Tyler Goodlet a465a11782 Persist backing `/data/` filesystem across container runs 2022-04-30 11:38:24 -04:00
Tyler Goodlet 807685d27e Get basic OHLCV writes working with `anyio` client 2022-04-30 11:38:24 -04:00
Tyler Goodlet aea42ccbd9 Pass in daemon name to `start_ahab()` 2022-04-30 11:38:24 -04:00
Tyler Goodlet 8fb9308e21 Map the grpc port and add graceful container teardown
Not sure how I missed mapping the 5995 grpc port 🤦; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-04-30 11:38:24 -04:00
Tyler Goodlet bbae8ad426 Revive `ms-shell` sub-cmd 2022-04-30 11:38:24 -04:00
Tyler Goodlet 893ac7a986 Add WIP backfiller from data feed helper 2022-04-30 11:38:24 -04:00
Tyler Goodlet 91856ddda8 Better handle nested erros from docker client 2022-04-30 11:38:24 -04:00
Tyler Goodlet 25e2e13bd7 Add back in legacy write loop for reference 2022-04-30 11:38:24 -04:00
Tyler Goodlet 2427c96336 Add back in OHLCV dtype template and client side ws streamer 2022-04-30 11:38:24 -04:00
Tyler Goodlet 8732b2bd5e Drop ununsed `Services` ref 2022-04-30 11:38:24 -04:00
Tyler Goodlet ff9208c15b Py3.9+ type updates 2022-04-30 11:38:24 -04:00
Tyler Goodlet a2547a548f Add `--tsdb` flag to start `marketstore` with `pikerd` 2022-04-30 11:38:24 -04:00
Tyler Goodlet 3873b8619e De-escalate sudo perms in `pikerd` once docker spawns 2022-04-30 11:38:24 -04:00
Tyler Goodlet 326d05ac82 Handle the non-root perms case specifically too 2022-04-30 11:38:24 -04:00
Tyler Goodlet dcab99e3d2 Add explicit no-docker error and supervisor start task-func 2022-04-30 11:38:24 -04:00
Tyler Goodlet 5a5df21f94 Type annot updates 2022-04-30 11:38:24 -04:00
Tyler Goodlet d7c1286e5d Drop old client instantiate line 2022-04-30 11:38:24 -04:00
Tyler Goodlet 8acaa28df0 Drop import, it's got madness with and SIGINT? 2022-04-30 11:38:24 -04:00
Tyler Goodlet bb45100168 Add a super simple `marketstore` container supervisor 2022-04-30 11:38:24 -04:00
Tyler Goodlet d9ded54e10 Extract non-sudo user for config dir path 2022-04-30 11:38:24 -04:00
Tyler Goodlet 1bde86a7b2 Basic module-script for spawning `marketstore`, needs correct bind mount usage 2022-04-30 11:38:24 -04:00
Guillermo Rodriguez 5031892dcf Still WIP, switch to using new marketstore client, missing streaming from marketstore 2022-04-30 11:38:24 -04:00
Guillermo Rodriguez afe6f0b42b Simplify and optimize tick format, similar to techtonicdb's 2022-04-30 11:38:24 -04:00
Guillermo Rodriguez 5d539b7c49 Add multi ingestor support and update to new feed API 2022-04-30 11:38:24 -04:00
Tyler Goodlet e2ce341f93 Only update y mxmn from L1 when last index in view 2022-04-30 11:38:24 -04:00
goodboy 84399e8131
Merge pull request #289 from pikers/big_data_lines
"Big data" lines
2022-04-30 11:37:50 -04:00
Tyler Goodlet 5921d18d66 Only update y-range from L1 mxmn when last index in view
We still have to always keep track of the last max and min
though.
2022-04-30 11:36:23 -04:00
Tyler Goodlet cdc882657a Drop old `pyqtgraph` downsample code 2022-04-30 11:36:23 -04:00
Tyler Goodlet 62d08eaf85 Tweak log-scaler for more detail 2022-04-30 11:36:23 -04:00
Tyler Goodlet f2f00dcc52 Drop `._ic` debugging prints 2022-04-30 11:36:23 -04:00
Tyler Goodlet ee831baeb3 Display loop mega-cleanup
The most important changes include:
- iterating the new `Flow` type and updating graphics
- adding detailed profiling
- increasing the min uppx before graphics updates are throttled
- including the L1 spread in y-range calcs so that you never have the
  bid/ask go "out of view"..
- pass around `Flow`s instead of shms
- drop all the old prototyped downsampling code
2022-04-30 11:36:23 -04:00
Tyler Goodlet 7c615a403b Allow passing a `plotItem` to `.draw_curve()`
If manually managing an overlay you'll likely call `.overlay_plotitem()`
and then a plotting method so we need to accept a plot item input so
that the chart's pi doesn't get assigned incorrectly in the `Flow` entry
(though it is by default if no input is provided).

More,
- add a `Flow.graphics` field and set it to the `pg.GraphicsObject`.
- make `Flow.maxmin()` return `None` in the "can't calculate" cases.
2022-04-30 11:36:23 -04:00