Commit Graph

2072 Commits (0904b48a230b60a9625d107f11a6776663374d21)

Author SHA1 Message Date
Tyler Goodlet 0904b48a23 Get basic OHLCV writes working with `anyio` client 2022-04-13 08:11:57 -04:00
Tyler Goodlet c2b5a3d225 Pass in daemon name to `start_ahab()` 2022-04-13 08:11:57 -04:00
Tyler Goodlet 4c5ea60035 Map the grpc port and add graceful container teardown
Not sure how I missed mapping the 5995 grpc port 🤦; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-04-13 08:11:57 -04:00
Tyler Goodlet abf7b14717 Revive `ms-shell` sub-cmd 2022-04-13 08:11:57 -04:00
Tyler Goodlet a6613161c3 Add WIP backfiller from data feed helper 2022-04-13 08:11:57 -04:00
Tyler Goodlet 6d38f3d0cb Better handle nested erros from docker client 2022-04-13 08:11:57 -04:00
Tyler Goodlet 0877d09bbf Add back in legacy write loop for reference 2022-04-13 08:11:57 -04:00
Tyler Goodlet 98ec2b71ae Add back in OHLCV dtype template and client side ws streamer 2022-04-13 08:11:57 -04:00
Tyler Goodlet 00cd77a0ad Drop ununsed `Services` ref 2022-04-13 08:11:57 -04:00
Tyler Goodlet cbf1dca0b1 Py3.9+ type updates 2022-04-13 08:11:57 -04:00
Tyler Goodlet 7cdf7f5d13 Add `--tsdb` flag to start `marketstore` with `pikerd` 2022-04-13 08:11:57 -04:00
Tyler Goodlet 4130a1d2d6 De-escalate sudo perms in `pikerd` once docker spawns 2022-04-13 08:11:57 -04:00
Tyler Goodlet 17aa59b1f9 Handle the non-root perms case specifically too 2022-04-13 08:11:57 -04:00
Tyler Goodlet c27ce637c8 Add explicit no-docker error and supervisor start task-func 2022-04-13 08:11:56 -04:00
Tyler Goodlet 86b9c9a8f9 Type annot updates 2022-04-13 08:11:56 -04:00
Tyler Goodlet 54741ad9c6 Drop old client instantiate line 2022-04-13 08:11:56 -04:00
Tyler Goodlet 4670e66348 Drop import, it's got madness with and SIGINT? 2022-04-13 08:11:56 -04:00
Tyler Goodlet a3620a93ca Add a super simple `marketstore` container supervisor 2022-04-13 08:11:56 -04:00
Tyler Goodlet ddd92bb2ba Extract non-sudo user for config dir path 2022-04-13 08:11:56 -04:00
Tyler Goodlet 12a1074f0e Basic module-script for spawning `marketstore`, needs correct bind mount usage 2022-04-13 08:11:56 -04:00
Guillermo Rodriguez eb41ecb4db Still WIP, switch to using new marketstore client, missing streaming from marketstore 2022-04-13 08:11:56 -04:00
Guillermo Rodriguez bec76bfe59 Simplify and optimize tick format, similar to techtonicdb's 2022-04-13 08:11:56 -04:00
Guillermo Rodriguez ea1d7e4598 Add multi ingestor support and update to new feed API 2022-04-13 08:11:56 -04:00
Tyler Goodlet 681ea18c41 Only update y mxmn from L1 when last index in view 2022-04-13 08:11:56 -04:00
Tyler Goodlet b02a43466f Only update y-range from L1 mxmn when last index in view
We still have to always keep track of the last max and min
though.
2022-04-13 08:11:34 -04:00
Tyler Goodlet 229d269ae4 Drop old `pyqtgraph` downsample code 2022-04-13 08:11:34 -04:00
Tyler Goodlet 6f4216e102 Tweak log-scaler for more detail 2022-04-13 08:11:34 -04:00
Tyler Goodlet fadc7a7b10 Drop `._ic` debugging prints 2022-04-13 08:11:34 -04:00
Tyler Goodlet 985acb30a8 Display loop mega-cleanup
The most important changes include:
- iterating the new `Flow` type and updating graphics
- adding detailed profiling
- increasing the min uppx before graphics updates are throttled
- including the L1 spread in y-range calcs so that you never have the
  bid/ask go "out of view"..
- pass around `Flow`s instead of shms
- drop all the old prototyped downsampling code
2022-04-13 08:11:34 -04:00
Tyler Goodlet 91b4816c88 Allow passing a `plotItem` to `.draw_curve()`
If manually managing an overlay you'll likely call `.overlay_plotitem()`
and then a plotting method so we need to accept a plot item input so
that the chart's pi doesn't get assigned incorrectly in the `Flow` entry
(though it is by default if no input is provided).

More,
- add a `Flow.graphics` field and set it to the `pg.GraphicsObject`.
- make `Flow.maxmin()` return `None` in the "can't calculate" cases.
2022-04-13 08:11:34 -04:00
Tyler Goodlet 6fa00cc2ce Fsp UI initialization updates
- set shm refs on `Flow` entries.
- don't run a graphics cycle on 'update' msgs from the engine
  if the containing chart is hidden.
- drop `volume` from flows map and disable auto-yranging
  once $vlm comes up.
2022-04-13 08:11:34 -04:00
Tyler Goodlet 6f0fba30c8 Disconnect signals in `ChartView.disable_auto_yrange()`
Allows for removing resize callbacks for a flow/overlay that you wish to
remove from view (eg. unit volume after dollar volume is up) and thus
less general interaction callback overhead for any plot you don't wish
to show or resize.

Further,
- drop the `autoscale_linked_plots` block for now since with
  multi-view-box overlays each register their own vb resize slots
- pull the graphics object from the chart's `Flow` map inside
  `.maybe_downsample_graphics()`
2022-04-13 08:11:34 -04:00
Tyler Goodlet 5b7f33f60b Fix bug where if `yrange` was passed the mxmin callback was still used.. 2022-04-13 08:11:34 -04:00
Tyler Goodlet f2a9580d5c Guard against zero px width 2022-04-13 08:11:34 -04:00
Tyler Goodlet 1066d78918 Add `Flow` type with a real chitty mxmn cacheing method
This new type wraps a shm data flow and will eventually include things
like incremental path-graphics updates and serialization + bg downsampling
techniques. The main immediate motivation was to get a cached y-range max/min
calc going since profiling revealed the `numpy` equivalents were
actually quite slow as the data set grows large. Likely we can use all
this to drive a streaming mx/mn routine that's always launched as part
of each on-host flow.

This is our official foray into use of `msgspec.Struct` B) and I have to
say, pretty impressed; we'll likely completely ditch `pydantic` from
here on out.
2022-04-13 08:11:34 -04:00
Tyler Goodlet b3fde388dd Loop for first graphic with xvec 2022-04-13 08:11:34 -04:00
Tyler Goodlet d6b7ba2127 Only `.maybe_downsample_graphics()` on manual changes
We don't need update graphics on every x-range change since that's what
the display loop does. Instead, only on manual changes do we make manual
calls into `.update_graphics_from_array()` and be sure to iterate all
linked subplots and all their embedded graphics.
2022-04-13 08:11:34 -04:00
Tyler Goodlet f367f1a095 Don't return early on ds line render to avoid breaking profiling
The pg profiler seems to have trouble with early `return`s in function
calls (likely muckery with the GC/`.__delete__()`) so let's just try
to avoid it for now until we either fix it (probably by implementing as
a ctx mngr) or use diff one.
2022-04-13 08:11:34 -04:00
Tyler Goodlet 228b2998d0 Use HL tracer by default, seems to be faster? 2022-04-13 08:11:34 -04:00
Tyler Goodlet 1618428eed Allow passing "ms slower then" value on cli to `--profile` 2022-04-13 08:11:34 -04:00
Tyler Goodlet 75d5371a81 Only bail up pan updates if uppx > 16 2022-04-13 08:11:33 -04:00
Tyler Goodlet c15f737f76 Delegate `BarItems.x_uppx()` to internal ds curve 2022-04-13 08:11:33 -04:00
Tyler Goodlet e7ee93f6df Downsample curves even less frequently 2022-04-13 08:11:33 -04:00
Tyler Goodlet a48edb08f3 Only pass vr for bars, allow source vb in autorange 2022-04-13 08:11:33 -04:00
Tyler Goodlet 81f74ea44b Drop the unit-volume chart once $vlm is fully drawn 2022-04-13 08:11:33 -04:00
Tyler Goodlet ebf41281f5 Ensure we update the volume array, not graphics
Ugh, turns out the wacky `ChartView.maxmin` callback stuff we did (for
determining y-range sizings) currently requires that the volume array
has a "bars in view" result.. so let's make that keep working without
rendering the graphics for the curve (since we're disabling them once
$vlm comes up).
2022-04-13 08:11:33 -04:00
Tyler Goodlet f8368e31e2 Add `.update_graphics_from_array()` flags for setting view-range use and graphics rendering 2022-04-13 08:11:33 -04:00
Tyler Goodlet 7746f09c3b Guard against zero px width 2022-04-13 08:11:33 -04:00
Tyler Goodlet dd8443be7b Given in-view rendering, make bars downsample on uppx >= 8 2022-04-13 08:11:33 -04:00
Tyler Goodlet bf14fea9e9 Make `FastAppendCurve` optionally view range aware
As with the `BarItems` graphics, this makes it possible to pass in a "in
view" range of array data that can be *only* rendered improving
performance for large(r) data sets. All the other normal behaviour is
kept (i.e a persistent, (pre/ap)pendable path can still be maintained)
if a ``view_range`` is not provided.

Further updates,
- drop the `.should_ds_or_redraw()` and `.maybe_downsample()` predicates
 instead moving all that logic inside `.update_from_array()`.
- disable the "cache flipping", which doesn't seem to be needed to avoid
  artifacts any more?
- handle all redraw/dowsampling logic in `.update_from_array()`.
- even more profiling.
- drop path `.reserve()` stuff until we better figure out how it's
  supposed to work.
2022-04-13 08:11:33 -04:00