Commit Graph

2091 Commits (cd3dbc92759a73a25c2f03fc5edd327e0d015cc8)

Author SHA1 Message Date
Tyler Goodlet cd3dbc9275 Ensure bfqsn is lower cased for feed api consumers
Also, Start tinkering with `tractor.trionics.ipython_embed()`

In effort to get back to a usable REPL around the mkts client
this adds usage of the new `tractor` integration api as well as logic
for skipping backfilling if existing tsdb arrays are found.
2022-04-13 08:11:57 -04:00
Tyler Goodlet 87fab6c5b1 Try downsampling mkts data 2022-04-13 08:11:57 -04:00
Tyler Goodlet 2d719f88d2 Drop old type annot 2022-04-13 08:11:57 -04:00
Tyler Goodlet d3fb9f228d Comment each special key combo 2022-04-13 08:11:57 -04:00
Tyler Goodlet 58f6aa4308 Load any symbol-matching shm array if no `marketstored` found 2022-04-13 08:11:57 -04:00
Tyler Goodlet 1eb62e7678 Get ib key hack script to work with reconnect 2022-04-13 08:11:57 -04:00
Tyler Goodlet 965ea120e9 Move ib data reset script into a new `scripts/` dir 2022-04-13 08:11:57 -04:00
Tyler Goodlet fbdcbc9304 Use new `tractor.query_actor()` for service checking 2022-04-13 08:11:57 -04:00
Tyler Goodlet 7f87d09bf0 Return all timeframe arrays if `timeframe` not passed as input 2022-04-13 08:11:57 -04:00
Tyler Goodlet f7a0f117a0 Make `pikerd` work again without `--tsdb` flag 2022-04-13 08:11:57 -04:00
Tyler Goodlet ae986e9fcf Add a service checker predicate 2022-04-13 08:11:57 -04:00
Tyler Goodlet cad6508b28 Allow kill-child-proc-with-root-perms to fail silently in `tractor` reaping 2022-04-13 08:11:57 -04:00
Tyler Goodlet 46ad8557cd Proxy `marketstore` container log level to our own 2022-04-13 08:11:57 -04:00
Tyler Goodlet 85658a165c Prototype a high level `Storage` api
Starts a wrapper around the `marketstore` client to do basic ohlcv query
and retrieval and prototypes out write methods for ohlc and tick.
Try to connect to `marketstore` automatically (which will fail if not
started currently) but we will eventually first do a service query.

Further:

- get `pikerd` working with and without `--tsdb` flag.
- support spawning `brokerd` with no real-time quotes.
- bring back in "fqsn" support that was originally not
  in this history before commits factoring.
2022-04-13 08:11:57 -04:00
Tyler Goodlet 387cb51ca2 Doc str formatting 2022-04-13 08:11:57 -04:00
Tyler Goodlet 3c6f412a5b Add latency measures around diffs/writes to mkts 2022-04-13 08:11:57 -04:00
Tyler Goodlet 852f9e3e64 Prototype out writing `1Sec` OHLCV data 2022-04-13 08:11:57 -04:00
Tyler Goodlet 0af809849b Better doc string 2022-04-13 08:11:57 -04:00
Tyler Goodlet 5871aba04c Persist backing `/data/` filesystem across container runs 2022-04-13 08:11:57 -04:00
Tyler Goodlet 0904b48a23 Get basic OHLCV writes working with `anyio` client 2022-04-13 08:11:57 -04:00
Tyler Goodlet c2b5a3d225 Pass in daemon name to `start_ahab()` 2022-04-13 08:11:57 -04:00
Tyler Goodlet 4c5ea60035 Map the grpc port and add graceful container teardown
Not sure how I missed mapping the 5995 grpc port 🤦; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-04-13 08:11:57 -04:00
Tyler Goodlet abf7b14717 Revive `ms-shell` sub-cmd 2022-04-13 08:11:57 -04:00
Tyler Goodlet a6613161c3 Add WIP backfiller from data feed helper 2022-04-13 08:11:57 -04:00
Tyler Goodlet 6d38f3d0cb Better handle nested erros from docker client 2022-04-13 08:11:57 -04:00
Tyler Goodlet 0877d09bbf Add back in legacy write loop for reference 2022-04-13 08:11:57 -04:00
Tyler Goodlet 98ec2b71ae Add back in OHLCV dtype template and client side ws streamer 2022-04-13 08:11:57 -04:00
Tyler Goodlet 00cd77a0ad Drop ununsed `Services` ref 2022-04-13 08:11:57 -04:00
Tyler Goodlet cbf1dca0b1 Py3.9+ type updates 2022-04-13 08:11:57 -04:00
Tyler Goodlet 7cdf7f5d13 Add `--tsdb` flag to start `marketstore` with `pikerd` 2022-04-13 08:11:57 -04:00
Tyler Goodlet 4130a1d2d6 De-escalate sudo perms in `pikerd` once docker spawns 2022-04-13 08:11:57 -04:00
Tyler Goodlet 17aa59b1f9 Handle the non-root perms case specifically too 2022-04-13 08:11:57 -04:00
Tyler Goodlet c27ce637c8 Add explicit no-docker error and supervisor start task-func 2022-04-13 08:11:56 -04:00
Tyler Goodlet 86b9c9a8f9 Type annot updates 2022-04-13 08:11:56 -04:00
Tyler Goodlet 54741ad9c6 Drop old client instantiate line 2022-04-13 08:11:56 -04:00
Tyler Goodlet 4670e66348 Drop import, it's got madness with and SIGINT? 2022-04-13 08:11:56 -04:00
Tyler Goodlet a3620a93ca Add a super simple `marketstore` container supervisor 2022-04-13 08:11:56 -04:00
Tyler Goodlet ddd92bb2ba Extract non-sudo user for config dir path 2022-04-13 08:11:56 -04:00
Tyler Goodlet 12a1074f0e Basic module-script for spawning `marketstore`, needs correct bind mount usage 2022-04-13 08:11:56 -04:00
Guillermo Rodriguez eb41ecb4db Still WIP, switch to using new marketstore client, missing streaming from marketstore 2022-04-13 08:11:56 -04:00
Guillermo Rodriguez bec76bfe59 Simplify and optimize tick format, similar to techtonicdb's 2022-04-13 08:11:56 -04:00
Guillermo Rodriguez ea1d7e4598 Add multi ingestor support and update to new feed API 2022-04-13 08:11:56 -04:00
Tyler Goodlet 681ea18c41 Only update y mxmn from L1 when last index in view 2022-04-13 08:11:56 -04:00
Tyler Goodlet b02a43466f Only update y-range from L1 mxmn when last index in view
We still have to always keep track of the last max and min
though.
2022-04-13 08:11:34 -04:00
Tyler Goodlet 229d269ae4 Drop old `pyqtgraph` downsample code 2022-04-13 08:11:34 -04:00
Tyler Goodlet 6f4216e102 Tweak log-scaler for more detail 2022-04-13 08:11:34 -04:00
Tyler Goodlet fadc7a7b10 Drop `._ic` debugging prints 2022-04-13 08:11:34 -04:00
Tyler Goodlet 985acb30a8 Display loop mega-cleanup
The most important changes include:
- iterating the new `Flow` type and updating graphics
- adding detailed profiling
- increasing the min uppx before graphics updates are throttled
- including the L1 spread in y-range calcs so that you never have the
  bid/ask go "out of view"..
- pass around `Flow`s instead of shms
- drop all the old prototyped downsampling code
2022-04-13 08:11:34 -04:00
Tyler Goodlet 91b4816c88 Allow passing a `plotItem` to `.draw_curve()`
If manually managing an overlay you'll likely call `.overlay_plotitem()`
and then a plotting method so we need to accept a plot item input so
that the chart's pi doesn't get assigned incorrectly in the `Flow` entry
(though it is by default if no input is provided).

More,
- add a `Flow.graphics` field and set it to the `pg.GraphicsObject`.
- make `Flow.maxmin()` return `None` in the "can't calculate" cases.
2022-04-13 08:11:34 -04:00
Tyler Goodlet 6fa00cc2ce Fsp UI initialization updates
- set shm refs on `Flow` entries.
- don't run a graphics cycle on 'update' msgs from the engine
  if the containing chart is hidden.
- drop `volume` from flows map and disable auto-yranging
  once $vlm comes up.
2022-04-13 08:11:34 -04:00