Commit Graph

2134 Commits (06f311c1828c2c09fabbbcd10be4202e9b2162b0)

Author SHA1 Message Date
Tyler Goodlet 49a280ce14 Better doc string 2022-04-16 15:39:24 -04:00
Tyler Goodlet 42e5f48345 Persist backing `/data/` filesystem across container runs 2022-04-16 15:39:24 -04:00
Tyler Goodlet 18344603e6 Get basic OHLCV writes working with `anyio` client 2022-04-16 15:39:24 -04:00
Tyler Goodlet ae1d9efa97 Pass in daemon name to `start_ahab()` 2022-04-16 15:39:24 -04:00
Tyler Goodlet f1747749e7 Map the grpc port and add graceful container teardown
Not sure how I missed mapping the 5995 grpc port 🤦; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-04-16 15:39:24 -04:00
Tyler Goodlet cdd03759bf Revive `ms-shell` sub-cmd 2022-04-16 15:39:24 -04:00
Tyler Goodlet 00a36826c2 Add WIP backfiller from data feed helper 2022-04-16 15:39:24 -04:00
Tyler Goodlet 3aa8044aa1 Better handle nested erros from docker client 2022-04-16 15:39:24 -04:00
Tyler Goodlet f0ae7edb00 Add back in legacy write loop for reference 2022-04-16 15:39:24 -04:00
Tyler Goodlet 797ba15923 Add back in OHLCV dtype template and client side ws streamer 2022-04-16 15:39:24 -04:00
Tyler Goodlet b8f86d4599 Drop ununsed `Services` ref 2022-04-16 15:39:24 -04:00
Tyler Goodlet 908093264a Py3.9+ type updates 2022-04-16 15:39:24 -04:00
Tyler Goodlet 3ef68f78f7 Add `--tsdb` flag to start `marketstore` with `pikerd` 2022-04-16 15:39:24 -04:00
Tyler Goodlet d095c21885 De-escalate sudo perms in `pikerd` once docker spawns 2022-04-16 15:39:24 -04:00
Tyler Goodlet 52dc0989ac Handle the non-root perms case specifically too 2022-04-16 15:39:24 -04:00
Tyler Goodlet 6f749c340d Add explicit no-docker error and supervisor start task-func 2022-04-16 15:39:24 -04:00
Tyler Goodlet dcff80131f Type annot updates 2022-04-16 15:39:24 -04:00
Tyler Goodlet 6cd829d1fd Drop old client instantiate line 2022-04-16 15:39:24 -04:00
Tyler Goodlet 76fff18b1a Drop import, it's got madness with and SIGINT? 2022-04-16 15:39:24 -04:00
Tyler Goodlet 88e1befffe Add a super simple `marketstore` container supervisor 2022-04-16 15:39:24 -04:00
Tyler Goodlet c1579b630e Extract non-sudo user for config dir path 2022-04-16 15:39:24 -04:00
Tyler Goodlet 7dfdd0f00b Basic module-script for spawning `marketstore`, needs correct bind mount usage 2022-04-16 15:39:24 -04:00
Guillermo Rodriguez 14a893a5e5 Still WIP, switch to using new marketstore client, missing streaming from marketstore 2022-04-16 15:39:24 -04:00
Guillermo Rodriguez c91599472d Simplify and optimize tick format, similar to techtonicdb's 2022-04-16 15:39:24 -04:00
Guillermo Rodriguez 9e4dec98da Add multi ingestor support and update to new feed API 2022-04-16 15:39:24 -04:00
Tyler Goodlet 8800ee0661 Only update y mxmn from L1 when last index in view 2022-04-16 15:39:24 -04:00
Tyler Goodlet 885116ae46 Only update y-range from L1 mxmn when last index in view
We still have to always keep track of the last max and min
though.
2022-04-16 15:38:26 -04:00
Tyler Goodlet eab9127170 Drop old `pyqtgraph` downsample code 2022-04-16 15:38:26 -04:00
Tyler Goodlet c675773de4 Tweak log-scaler for more detail 2022-04-16 15:38:26 -04:00
Tyler Goodlet b20245398d Drop `._ic` debugging prints 2022-04-16 15:38:26 -04:00
Tyler Goodlet 2abb3ec84d Display loop mega-cleanup
The most important changes include:
- iterating the new `Flow` type and updating graphics
- adding detailed profiling
- increasing the min uppx before graphics updates are throttled
- including the L1 spread in y-range calcs so that you never have the
  bid/ask go "out of view"..
- pass around `Flow`s instead of shms
- drop all the old prototyped downsampling code
2022-04-16 15:38:26 -04:00
Tyler Goodlet 6843f9a515 Allow passing a `plotItem` to `.draw_curve()`
If manually managing an overlay you'll likely call `.overlay_plotitem()`
and then a plotting method so we need to accept a plot item input so
that the chart's pi doesn't get assigned incorrectly in the `Flow` entry
(though it is by default if no input is provided).

More,
- add a `Flow.graphics` field and set it to the `pg.GraphicsObject`.
- make `Flow.maxmin()` return `None` in the "can't calculate" cases.
2022-04-16 15:38:26 -04:00
Tyler Goodlet 5002e78b81 Fsp UI initialization updates
- set shm refs on `Flow` entries.
- don't run a graphics cycle on 'update' msgs from the engine
  if the containing chart is hidden.
- drop `volume` from flows map and disable auto-yranging
  once $vlm comes up.
2022-04-16 15:38:26 -04:00
Tyler Goodlet 5446cbd335 Disconnect signals in `ChartView.disable_auto_yrange()`
Allows for removing resize callbacks for a flow/overlay that you wish to
remove from view (eg. unit volume after dollar volume is up) and thus
less general interaction callback overhead for any plot you don't wish
to show or resize.

Further,
- drop the `autoscale_linked_plots` block for now since with
  multi-view-box overlays each register their own vb resize slots
- pull the graphics object from the chart's `Flow` map inside
  `.maybe_downsample_graphics()`
2022-04-16 15:38:26 -04:00
Tyler Goodlet 7870e759ff Fix bug where if `yrange` was passed the mxmin callback was still used.. 2022-04-16 15:38:26 -04:00
Tyler Goodlet dfed6cb8e9 Guard against zero px width 2022-04-16 15:38:26 -04:00
Tyler Goodlet b30b4bb555 Add `Flow` type with a real chitty mxmn cacheing method
This new type wraps a shm data flow and will eventually include things
like incremental path-graphics updates and serialization + bg downsampling
techniques. The main immediate motivation was to get a cached y-range max/min
calc going since profiling revealed the `numpy` equivalents were
actually quite slow as the data set grows large. Likely we can use all
this to drive a streaming mx/mn routine that's always launched as part
of each on-host flow.

This is our official foray into use of `msgspec.Struct` B) and I have to
say, pretty impressed; we'll likely completely ditch `pydantic` from
here on out.
2022-04-16 15:38:26 -04:00
Tyler Goodlet 514bbb1a98 Loop for first graphic with xvec 2022-04-16 15:38:26 -04:00
Tyler Goodlet 918789d1a3 Only `.maybe_downsample_graphics()` on manual changes
We don't need update graphics on every x-range change since that's what
the display loop does. Instead, only on manual changes do we make manual
calls into `.update_graphics_from_array()` and be sure to iterate all
linked subplots and all their embedded graphics.
2022-04-16 15:38:26 -04:00
Tyler Goodlet 127294d39c Don't return early on ds line render to avoid breaking profiling
The pg profiler seems to have trouble with early `return`s in function
calls (likely muckery with the GC/`.__delete__()`) so let's just try
to avoid it for now until we either fix it (probably by implementing as
a ctx mngr) or use diff one.
2022-04-16 15:38:26 -04:00
Tyler Goodlet d3a30a272d Use HL tracer by default, seems to be faster? 2022-04-16 15:38:26 -04:00
Tyler Goodlet 9477c7e66c Allow passing "ms slower then" value on cli to `--profile` 2022-04-16 15:38:26 -04:00
Tyler Goodlet abab8d3451 Only bail up pan updates if uppx > 16 2022-04-16 15:38:26 -04:00
Tyler Goodlet c4ad4e089e Delegate `BarItems.x_uppx()` to internal ds curve 2022-04-16 15:38:26 -04:00
Tyler Goodlet 36224eac5a Downsample curves even less frequently 2022-04-16 15:38:26 -04:00
Tyler Goodlet 1cdec55725 Only pass vr for bars, allow source vb in autorange 2022-04-16 15:38:26 -04:00
Tyler Goodlet 576263dc4d Drop the unit-volume chart once $vlm is fully drawn 2022-04-16 15:38:26 -04:00
Tyler Goodlet 3998cb70a6 Ensure we update the volume array, not graphics
Ugh, turns out the wacky `ChartView.maxmin` callback stuff we did (for
determining y-range sizings) currently requires that the volume array
has a "bars in view" result.. so let's make that keep working without
rendering the graphics for the curve (since we're disabling them once
$vlm comes up).
2022-04-16 15:38:26 -04:00
Tyler Goodlet 096d8553e6 Add `.update_graphics_from_array()` flags for setting view-range use and graphics rendering 2022-04-16 15:38:26 -04:00
Tyler Goodlet ec877f38a0 Guard against zero px width 2022-04-16 15:38:26 -04:00