Commit Graph

3475 Commits (a7ad63ce4e5978f59afaaeb8f10c7d5aa630714b)

Author SHA1 Message Date
Tyler Goodlet a7ad63ce4e Add note around docker client detach flag 2023-03-08 11:24:01 -05:00
Tyler Goodlet 58f39d1829 First draft storage layer cli
Adds a `piker storage` subcmd with a `-d` flag to wipe a particular
fqsn's time series (both 1s and 60s). Obviously this needs to be
extended much more but provides a start point.
2023-03-08 11:23:33 -05:00
Tyler Goodlet 4379bfe760 Read `Symbol` tick precision fields when no entry in `.broker_info` 2023-03-08 09:06:50 -05:00
Tyler Goodlet 73089e5612 Always show a minimum bars during startup
This is particularly more "good looking" when we boot with a pair that
doesn't have historical 1s OHLC and thus the fast chart is empty from
outset. In this case it's a lot nicer to be already zoomed to
a comfortable preset number of "datums in view" even when the history
isn't yet filled in.

Adjusts the chart display `Viz.default_view()` startup to explicitly
ensure this happens via the `do_min_bars=True` flag B)
2023-03-07 20:40:21 -05:00
Tyler Goodlet 5bf40ceb79 Catch `KeyError` on bcast errors which pop the sub
Not sure how i missed this (and left in handling of `list.remove()` and
it ever worked for that?) after the `samplerd` impl in 5ec1a72 but, this
adjusts the remove-broken-subscriber loop to catch the correct
`set.remove()` exception type on a missing (likely already removed)
subscription entry.
2023-03-07 15:42:06 -05:00
Tyler Goodlet 6f3a6bcb42 Remove leftover debug print in cache reset meth 2023-03-07 15:41:38 -05:00
Tyler Goodlet ddd1722a0a Add (commented) draft 1min OHLC time index logging
For the purposes of eventually trying to resolve last-step indexing
synchronization (an intermittent but still existing) issue(s) that can
happen due to races during history frame query and shm writing during
startup. In fact, here we drop all `hist_viz` info queries from the main
display loop for now anticipating that this code will either be removed
or improved later.
2023-03-07 15:35:07 -05:00
Tyler Goodlet 0cfad6838d Always pass step to `slice_from_time()` in view mode
Again, as per the signature change, never expect implicit time step
calcs from overlay processing/machinery code. Also, extend the debug
printing (yet again) to include better details around
"rescale-due-to-minor-range-out-of-view" cases and a detailed msg for
the transform/scaling calculation (inputs/outputs), particularly for the
cases when one of the curves has a lesser support.
2023-03-07 15:18:34 -05:00
Tyler Goodlet c0d2baaaaa Always pass `step` to `slice_from_time()` in the `Viz`
As per the change to `slice_from_time()` this ensures this `Viz` always
passes its self-calculated time indexing step size to the time slicing
routine(s).

Further this contains a slight impl tweak to `.scalars_from_index()` to
slice the actual view range from `xref` to `Viz.ViewState.xrange[1]` and
then reading the corresponding `yref` from the first entry in that
array; this should be no slower in theory and makes way for further
caching of x-read-range to `ViewState` opportunities later.
2023-03-07 15:05:42 -05:00
Tyler Goodlet b9c2c254dc Require `step: float` input to `slice_from_time()`
There's been way too many issues when trying to calculate this
dynamically from the input array, so just expect the caller to know what
it's doing and don't bother with ever hitting the error case of
calculating and incorrect value internally.
2023-03-07 14:36:19 -05:00
Tyler Goodlet 4da772292f Handle "target-is-shorter-then-pinned" case
When the target pinning curve (by default, the dispersion major) is
shorter then the pinned curve, we need to make sure we find still find
the x-intersect for computing returns scalars! Use `Viz.i_from_t()` to
accomplish this as well and, augment that method with a `return_y: bool`
to allow the caller to also retrieve the equivalent y-value at the
requested input time `t: float` for convenience.

Also tweak a few more internals around the 'loglin_ref_to_curve'
method:
- only solve / adjust for the above case when the major's xref is
  detected as being "earlier" in time the current minor's.
- pop the major viz entry from the overlay table ahead of time to avoid
  a needless iteration and simplify the transform calc phase loop to
  avoid handling that needless cycle B)
- add much better "organized" debug printing with more clear headers
  around which "phase"/loop the message pertains and well as more
  explicit details in terms of x and y-range values on each cycle of
  each loop.
2023-03-06 19:03:04 -05:00
Tyler Goodlet 7a37c700e5 Don't `@lru_cache` on `Viz.i_from_t()`, since view state.. 2023-03-06 18:30:58 -05:00
Tyler Goodlet f3fd627a75 Tweak debug printing to display y-mxmn per viz 2023-03-06 10:37:26 -05:00
Tyler Goodlet 1d649e55ca Fix curve up-sampling on `'r'` hotkey
Previously when very zoomed out and using the `'r'` hotkey the
interaction handler loop wouldn't trigger a re-(up)sampling to get
a more detailed curve graphic and instead the previous downsampled
(under-detailed) graphic would show. Fix that by ensuring we yield back
to the Qt event loop and do at least a couple render cycles with paired
`.interact_graphics_cycle()` calls.

Further this flips the `.start/signal_ic()` methods to use the new
`.reset_graphics_caches()` ctr-mngr method.
2023-03-05 21:23:42 -05:00
Tyler Goodlet 08f1f569d0 Facepalm: set `Viz.ViewState.yrange` even on cache hits.. 2023-03-05 21:22:55 -05:00
Tyler Goodlet 80af431c2a Drop remaining usage of `ChartPlotWidget.default_view()`
Instead delegate directly to `Viz.default_view()` throughout charting
startup and interaction handlers.

Also add a `ChartPlotWidget.reset_graphics_caches()` context mngr which
resets all managed graphics object's cacheing modes on enter and
restores them on exit for simplified use in interaction handling code.
2023-03-05 21:14:22 -05:00
Tyler Goodlet d32382f831 Add `do_min_bars: bool` flag to `Viz.default_view()` 2023-03-04 17:07:46 -05:00
Tyler Goodlet d2e64accf6 Drop remaining non-usage of `ChartPlotWidget.maxmin()` 2023-03-04 16:49:20 -05:00
Tyler Goodlet 0ac2e4e027 Expand mxmn view y-margins back to 0.06 2023-03-03 18:36:46 -05:00
Tyler Goodlet 82797a097b Handle yrange not set on view vase for vlm fsp plot 2023-03-03 18:36:46 -05:00
Tyler Goodlet eecae69076 Disable coordinate caching during interaction
This finally seems to mitigate all the "smearing" and "jitter" artifacts
when using Qt's "coordinate cache" graphics-mode:

- whenever we're in a mouse interaction (as per calls to
  `ChartView.start/signal_ic()`) we simply disable the caching mode (set
  `.NoCache` until the interaction is complete.
- only do this (for now) during a pan since it doesn't seem to be an
  issue when zooming?
- ensure disabling all `Viz.graphics` and `.ds_graphics` to be agnostic
  to any case where there's both a zoom and a pan simultaneously (not
  that it's easy to do manually XD) as well as solving the problem
  whenever an OHLC series is in traced-and-downsampled mode (during low
  zoom).

Impl deatz:
- rename `ChartView._ic` -> `._in_interact: trio.Event`
- add `.ChartView._interact_stack: ExitStack` which  we use to open.
  and close the `FlowGraphics.reset_cache()` mngrs from mouse handlers.
- drop all the commented per-subtype overrides for `.cache_mode: int`.
- write up much better doc strings for `FlattenedOHLC` and `StepCurve`
  including some very basic ASCII-art diagrams.
2023-03-03 18:36:46 -05:00
Tyler Goodlet e0dd8ae3cf Add per-chart `Viz`/overlay graphics iterator method 2023-03-03 18:36:46 -05:00
Tyler Goodlet 5c08b5658f Move cache-reset ctx mngr to parent type: `FlowGraphics.reset_cache()` 2023-03-03 18:36:46 -05:00
Tyler Goodlet a81b51b142 Fix focal min calc after switching to `Viz.datums_range()`.. 2023-03-03 18:36:46 -05:00
Tyler Goodlet 6df2c3d009 Simplify `FlowGraphics.x_last()` logics 2023-03-03 18:36:46 -05:00
Tyler Goodlet e2cb1aca8e Rename overlay technique var to `method` 2023-03-03 18:36:46 -05:00
Tyler Goodlet aa9bd9994d Repair x-label datetime labels when in array-index mode 2023-03-03 18:36:46 -05:00
Tyler Goodlet 1d76586701 Skip overlay handling when `N < 2` are detected 2023-03-03 18:36:46 -05:00
Tyler Goodlet 94682ed9d9 Drop passing overlay method from viewbox to view-mode handler 2023-03-03 18:36:46 -05:00
Tyler Goodlet 8149b25732 Drop a bunch of commented/uneeded cruft 2023-03-03 18:36:46 -05:00
Tyler Goodlet 2fd36d27f6 Solve a final minor-should-rescale edge case
When the minor has the same scaling as the major in a given direction we
should still do back-scaling against the major-target and previous
minors to avoid strange edge cases where only the target-major might not
be shifted correctly to show an matched intersect point? More or less
this just meant making the y-mxmn checks interval-inclusive with
`>=`/`<=` operators.

Also adds a shite ton of detailed comments throughout the pin-to-target
method blocks and moves the final major y-range call outside the final
`scaled: dict` loop.
2023-03-03 18:36:46 -05:00
Tyler Goodlet f8727251f9 Better doc string, use `Viz.vs: ViewState` 2023-03-03 18:36:46 -05:00
Tyler Goodlet d3c85bc925 Back-rescale previous (minor) curves from latest
For the "pin to target major/target curve" overlay method, this finally
solves the longstanding issue of ensuring that any new minor curve,
which requires and increase in the major/target curve y-range, also
re-scales all previously scaled minor curves retroactively. Thus we now
guarantee that all minor curves are correctly "pinned" to their
target/major on their earliest available datum **and** are all kept in
view.
2023-03-03 18:36:46 -05:00
Tyler Goodlet b118954bf7 Support "pin-to-target-curve" overlay method again
Yah yah, i know it's the same as before (the N > 2 curves case with
out-of-range-minor rescaling the previously scaled curves isn't fixed
yet...) but, this is a much better and optional implementation in less
code. Further we're now better leveraging various new cached properties
and methods on `Viz`.

We now handle different `overlay_technique: str` options using `match:`
syntax in the 2ndary scaling loop, stash the returns scalars per curve
in `overlay_table`, and store and iterate the curves by dispersion
measure sort order.

Further wrt "pin-to-target-curve" mode, which currently still pins to
the largest measured dispersion curve in the overlay set:

- pop major Ci overlay table entries at start for sub-calcs usage when
  handling the "minor requires major rescale after pin" case.
- (finally) correctly rescale the major curve y-mxmn to whatever the
  latest minor overlay curve by calcing the inverse transform from the
  minor *at that point*:
  - the intersect point being that which the minor has starts support on
    the major's x-domain* using the new `Viz.scalars_from_index()` and,
  - checking that the minor is not out of range (versus what the major's
    transform calcs it to be, in which case,
  - calc the inverse transform from the current out-of-range minor and
    use it to project the new y-mxmn for the major/target based on the
    same intersect-reference point in the x-domain used by the minor.
  - always handle the target-major Ci specially by only setting the
    `mx_ymn` / `mx_ymn` value when iterating that entry in the overlay
    table.
  - add todos around also doing the last sub-sub bullet for all previously
    major-transform scaled minor overlays (this is coming next..i hope).
- add a final 3rd overlay loop which goes through a final `scaled: dict`
  to apply all range values to each view; this is where we will
  eventually solve that last edge case of an out-of-range minor's
  scaling needing to be used to rescale already scaled minors XD
2023-03-03 18:36:46 -05:00
Tyler Goodlet 55ec9ef5a0 Add cached dispersion methods to `Viz`
In an effort to make overlay calcs cleaner and leverage caching of view
range -> dispersion measures, this adds the following new methods:

- `._dispersion()` an lru cached returns scalar calculator given input
  y-range and y-ref values.
- `.disp_from_range()` which calls the above method and returns variable
  output depending on requested calc `method: str`.
- `.i_from_t()` a currently unused cached method for slicing the
  in-view's array index from time stamp (though not working yet due to
  needing to parameterize the cache by the input `.vs.xrange`).

Further refinements/adjustments:
- rename `.view_state: ViewState` -> `.vs`.
- drop the `.bars_range()` method as it's no longer used anywhere else
  in the code base.
- always set the `ViewState.in_view: np.ndarray` inside `.read()`.
- return the start array index (from slice) and `yref` value @ `xref`
  from `.scalars_from_index()` to aid with "pin to curve" rescaling
  caused by out-of-range pinned-minor curves.
2023-03-03 18:36:46 -05:00
Tyler Goodlet c5a9cc22c2 Avoid index-from-time slicing including gaps
Not sure why this was ever allowed but, for slicing to the sample
*before* whatever target time stamp is passed in we should definitely
not return the prior index as for the slice start since that might
include a very large gap prior to whatever sample is scanned to have
the earliest matching time stamp.

This was essential to fixing overlay intersect points searching in our
``ui.view_mode`` machinery..
2023-03-03 18:36:46 -05:00
Tyler Goodlet 5ec873fa2a Drop last lingering usage of `Viz.bars_range()` 2023-03-03 18:36:46 -05:00
Tyler Goodlet 247a77857f Add `Viz.view_state: ViewState`
Adds a small struct which is used to track the most recently viewed
data's x/y ranges as well as the last `Viz.read()` "in view" array data
for fast access by chart related graphics processing code, namely view
mode overlay handling.

Also adds new `Viz` interfaces:
- `Viz.ds_yrange: tuple[float, float]' which replaces the previous
  `.yrange` (now set by `.datums_range()` on manual y-range calcs) so
  that the m4 downsampler can set this field specifically and then it
  get used (when available) by `Viz.maxmin()`.
- `Viz.scalars_from_index()` a new returns-scalar generator which can be
  used to calc the up and down returns values (used for scaling overlay
  y-ranges) from an input `xref` x-domain index which maps to some
  `Ci(xref) = yref`.
2023-03-03 18:36:46 -05:00
Tyler Goodlet ca80b3b808 Make slow chart a teensie bit smaller 2023-03-03 18:36:46 -05:00
Tyler Goodlet d2b99c6889 Drop (now) unused major curve mx/mn variables 2023-03-03 18:36:46 -05:00
Tyler Goodlet cab335ef2f Move overlay transform logic to new `.ui.view_mode`
It was getting waayy to long to be jammed in a method XD

This moves all the chart-viz iteration and transform logic into a new
`piker.ui.view_mode.overlay_viewlists()` core routine which will make it
a lot nicer for,

- AOT compilation via `numba` / `cython` / `mypyc`.
- decoupling from the `pyqtgraph.ViewBox` APIs if we ever decide to get
  crazy and go without another graphics engine.
- keeping your head clear when trying to rework the code B)
2023-03-03 18:36:46 -05:00
Tyler Goodlet 1346058a48 Adjust `.ui` modules to new set-style "optional" annots 2023-03-03 18:36:46 -05:00
Tyler Goodlet 9b43639416 Remove vlm chart again, drop lotsa fsp cruft 2023-03-03 18:36:46 -05:00
Tyler Goodlet 22efd05d8c Rework overlay pin technique: "align to first"
As part of solving a final bullet-issue in #455, which is specifically
a case:
- with N > 2 curves, one of which is the "major" dispersion curve" and
  the others are "minors",
- we can run into a scenario where some minor curve which gets pinned to
  the major (due to the original "pinning technique" -> "align to
  major") at some `P(t)` which is *not* the major's minimum / maximum
  due to the minor having a smaller/shorter support and thus,
- requires that in order to show then max/min on the minor curve we have
  to expand the range of the major curve as well but,
- that also means any previously scaled (to the major) minor curves need
  to be adjusted as well or they'll not be pinned to the major the same
  way!

I originally was trying to avoid doing the recursive iteration back
through all previously scaled minor curves and instead decided to try
implementing the "per side" curve dispersion detection (as was
originally attempted when first starting this work). The idea is to
decide which curve's up or down "swing in % returns" would determine the
global y-range *on that side*. Turns out I stumbled on the "align to
first" technique in the process: "for each overlay curve we align its
earliest sample (in time) to the same level of the earliest such sample
for whatever is deemed the major (directionally disperse) curve in
view".

I decided (with help) that this "pin to first" approach/style is equally
as useful and maybe often more so when wanting to view support-disjoint
time series:

- instead of compressing the y-range on "longer series which have lesser
  sigma" to make whatever "shorter but larger-sigma series" pin to it at
  an intersect time step, this instead will expand the price ranges
  based on the earliest time step in each series.
- the output global-returns-overlay-range for any N-set of series is equal to
  the same in the previous "pin to intersect time" technique.
- the only time this technique seems less useful is for overlaying
  market feeds which have the same destination asset but different
  source assets (eg. btceur and btcusd on the same chart since if one
  of the series is shorter it will always be aligned to the earliest
  datum on the longer instead of more naturally to the intersect sample
  level as was in the previous approach).

As such I'm going to keep this technique as discovered and will later
add back optional support for the "align to intersect" approach from
previous (which will again require detecting the highest dispersion
curve direction-agnostic) and pin all minors to the price level at which
they start on the major.

Further details of the implementation rework in
`.interact_graphics_cycle()` include:

- add `intersect_from_longer()` to detect and deliver a common datum
  from 2 series which are different in length: the first time-index
  sample in the longer.
- Rewrite the drafted `OverlayT` to only compute (inversed log-returns)
  transforms for a single direction and use 2 instances, one for each
  direction inside the `Viz`-overlay iteration loop.
- do all dispersion-per-side major curve detection in the first pass of
  all `Viz`s on a plot, instead updating the `OverlayT` instances for
  each side and compensating for any length mismatch and
  rescale-to-minor cases in each loop cycle.
2023-03-03 18:36:46 -05:00
Tyler Goodlet 73912ab9a8 Try to hide all axes even when removed 2023-03-03 18:36:46 -05:00
Tyler Goodlet 7f91cda899 Add hack-zone UI REPL access via `ctl-u` 2023-03-03 18:36:46 -05:00
Tyler Goodlet fd1fd8d49b Facepalm, align overlay plot view exactly to parent
Previously we were aligning the child's `PlotItem` to the "root" (top
most) overlays `ViewBox`..smh. This is why there was a weird gap on the
LHS next to the 'left' price axes: something weird in the implied axes
offsets was getting jammed in that rect.

Also comments out "the-skipping-of" moving axes from the overlay's
`PlotItem.layout` to the root's linear layout(s) when an overlay's axis
is read as not visible; this isn't really necessary nor useful and if we
want to remove the axes entirely we should do it explicitly and/or
provide a way through the `ComposeGridLayout` API.
2023-03-03 18:36:46 -05:00
Tyler Goodlet a2934b7d18 Go back to caching on all curves
Despite there being artifacts when interacting, the speedups when
cross-hair-ing are just too good to ignore. We can always play with
disabling caches when interaction takes place much like we do with feed
pausing.
2023-03-03 18:36:46 -05:00
Tyler Goodlet 2a4a5588a8 Dynamically adjust y-range margin in display loop
When zoomed in alot, and thus a quote driven y-range resize takes place,
it makes more sense to increase the `range_margin: float` input to
`._set_yrange()` to ensure all L1 labels stay in view; generally the
more zoomed in,
- the smaller the y-range is and thus the larger the needed margin (on
  that range's dispersion diff) would be,
- it's more likely to get a last datum move outside the previous range.

Also, always do overlayT style scaling on the slow chart whenever it
treads.
2023-03-03 18:36:46 -05:00
Tyler Goodlet 1e85668bc2 Expose `._set_yrange()` kwargs via `yrange_kwargs: dict`
Since it can be desirable to dynamically adjust inputs to the y-ranging
method (such as in the display loop when a chart is very zoomed in), this
adds such support through a new `yrange_kwargs: dict[Viz, dict]` which
replaces the `yrange` tuple we were passing through prior. Also, adjusts
the y-range margin back to the original 0.09 of the diff now that we can
support dynamic control.
2023-03-03 18:36:46 -05:00