Commit Graph

3544 Commits (d6634e9b0225380ec49113733fe25eb2ccb1f36e)

Author SHA1 Message Date
Tyler Goodlet eeca9eb4c7 Add `.update_graphics_from_array()` flags for setting view-range use and graphics rendering 2022-04-30 11:36:23 -04:00
Tyler Goodlet 9bbfa4be02 Guard against zero px width 2022-04-30 11:36:23 -04:00
Tyler Goodlet ce85031ef2 Given in-view rendering, make bars downsample on uppx >= 8 2022-04-30 11:36:23 -04:00
Tyler Goodlet b6f852e0ad Make `FastAppendCurve` optionally view range aware
As with the `BarItems` graphics, this makes it possible to pass in a "in
view" range of array data that can be *only* rendered improving
performance for large(r) data sets. All the other normal behaviour is
kept (i.e a persistent, (pre/ap)pendable path can still be maintained)
if a ``view_range`` is not provided.

Further updates,
- drop the `.should_ds_or_redraw()` and `.maybe_downsample()` predicates
 instead moving all that logic inside `.update_from_array()`.
- disable the "cache flipping", which doesn't seem to be needed to avoid
  artifacts any more?
- handle all redraw/dowsampling logic in `.update_from_array()`.
- even more profiling.
- drop path `.reserve()` stuff until we better figure out how it's
  supposed to work.
2022-04-30 11:36:23 -04:00
Tyler Goodlet fdd5aa33d2 Fix view range array to include most recent (facepalm) 2022-04-30 11:36:23 -04:00
Tyler Goodlet 82732e3f17 TOQUASH: drop display loop old .update_ohlc_.. 2022-04-30 11:36:23 -04:00
Tyler Goodlet 2c1daab990 Port to new `.update_graphics_from_array()`, pause quote updates on chart interaction 2022-04-30 11:36:23 -04:00
Tyler Goodlet a9e1c6c50e Make panning pause feeds, call into update method from downsampler cb loop 2022-04-30 11:36:23 -04:00
Tyler Goodlet ef03b8e987 Attempt only rendering ohlc bars in view and ds-ing otherwise 2022-04-30 11:36:23 -04:00
Tyler Goodlet 3b90b1f960 Unify into a single update method: `.update_graphics_from_array()` 2022-04-30 11:36:23 -04:00
Tyler Goodlet 1cf6ba789c Remove units vlm cuve once the $vlm one comes up 2022-04-30 11:36:23 -04:00
Tyler Goodlet 49c25eeef4 Index must be int bro.. 2022-04-30 11:36:23 -04:00
Tyler Goodlet 5bcd6ac494 Move px width log scaling into `ds_m4()` 2022-04-30 11:36:23 -04:00
Tyler Goodlet 5da9f7fdb4 Add more frequent ds steps when zooming out; use profiler gt 2022-04-30 11:36:23 -04:00
Tyler Goodlet 5128e4c304 Make `BarItems` use our line curve for downsampling
Drop all the logic originally in `.update_ds_line()` which is now done
internal to our `FastAppendCurve`. Add incremental update of the
flattened OHLC -> line curve (unfortunately using `np.concatenate()` for
the moment) and maintain a new `._ds_line_xy` arrays tuple which keeps
the internal state. Add `.maybe_downsample()` as per the new interaction
update method requirement. Draft out some fast path curve stuff like in
our line graphic. Short-circuit bars path updates when we downsample to
line. Oh, and add a ton more profiling in prep for getting
all this stuff faf.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 6410c68e2e Add global profile timeout var 2022-04-30 11:36:23 -04:00
Tyler Goodlet 947a514153 Add "native" downsampling to our `FastAppendCurve`
Build out an interface that makes it super easy to downsample curves
using the m4 algorithm while keeping our incremental `QPainterPath`
update feature. A lot of hard work and tinkering went into getting this
working all in-thread correctly and there are quite a few details..

New interface methods:
- `.x_uppx()` which returns the x-axis "view units per pixel"
- `.px_width()` which returns the total (rounded) x-axis pixels spanned
    by the curve in view.
- `.should_ds_or_redraw()` a predicate which checks internal state to
  see if either downsampling of the curve should take place, or the curve
  should have all downsampling removed and be redrawn with source array
  data.
- `.downsample()` the actual ds processing routine which delegates into
  the m4 algo impl.
- `.maybe_downsample()` a simple update method which can be called by
  the view box when the user changes the zoom level.

Implementation details/changes:

- make `.update_from_array()` check for downsample (or revert to source
  aka de-downsample) conditions exist and then downsample and re-draw
  path graphics accordingly.
- in order to even further speed up path appends (since our main
  bottleneck is measured to be `QPainter.drawPath()` calls with large
  paths which are frequently updates), add a secondary path `.fast_path`
  which is the path that is real-time updates by incremental appends and
  which is painted separately for speed in `.pain()`.
- drop all the `QPolyLine` stuff since it was tested to be much slower
  in general and especially so for append-updates.
- stop disabling the cache settings on updates since it doesn't seem to
  be required any more?
- more move toward deprecating and removing all lingering interface
  requirements from `pg.PlotCurveItem` (like `.xData`/`.yData`).
- adjust `.paint()` and `.boundingRect()` to compensate for the new
  `.fast_path`
- add a butt-load of profiling B)
2022-04-30 11:36:23 -04:00
Tyler Goodlet 8627f6f6c5 Add no-path guard now that we can use a poly 2022-04-30 11:36:23 -04:00
Tyler Goodlet 5800c10901 First try, drop `FastAppendCurve` inheritance from `pg.PlotCurveItem` 2022-04-30 11:36:23 -04:00
Tyler Goodlet 28bf8853aa Drop commented line from pq method copy/paste 2022-04-30 11:36:23 -04:00
Tyler Goodlet 86da64c2c2 Show baseline bars length on in view read < 6 2022-04-30 11:36:23 -04:00
Tyler Goodlet d59442e3b1 Bump up resolution log scaling a mag 2022-04-30 11:36:23 -04:00
Tyler Goodlet 5e161aa251 Always clear previous downsample curve on switch
Pretty sure this was most of the cause of the stale (more downsampled)
curves showing when zooming in and out from bars mode quickly. All this
stuff needs to get factored out into a new abstraction anyway, but
i think this get's mostly correct functionality.

Only draw new ds curve on uppx steps >= 4 and stop adding/removing
graphics objects from the scene; doesn't seem to speed anything up
afaict. Add better reporting of ds scale changes.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 9b2ec871a0 Clear ds line graphics on switch back to bars 2022-04-30 11:36:23 -04:00
Tyler Goodlet 2b12742992 More ems resiliency: discard broken client dialogs 2022-04-30 11:36:23 -04:00
Tyler Goodlet b262532fd4 Allocate m4 output arrays in `numba` code, avoid segfaults? 2022-04-30 11:36:23 -04:00
Tyler Goodlet 561d7e0349 Only clear/redraw curve on uppx diffs > 2
Only if the uppx increases by more then 2 we redraw the entire line
otherwise just ds with previous params and update the current curve.
This *should* avoid strange lower sample rate artefacts from showing on
updates.

Summary:
- stash both uppx and px width in `._dsi` (downsample info)
- use the new `ohlc_to_m4_line()` flags
- add notes about using `.reserve()` and friends
- always delete last `._array` ref prior to line updates
2022-04-30 11:36:23 -04:00
Tyler Goodlet 3a6c5a2fbd Try supporting reuse of path allocation 2022-04-30 11:36:23 -04:00
Tyler Goodlet 88a7314bd0 Add optional mxmn HL tracer support to m4 sampler 2022-04-30 11:36:23 -04:00
Tyler Goodlet 1abe513ecb Add our own `FastAppendCurve.clear()`, try mem reso
In an effort to try and make `QPainterPath.reserve()` work, add internal
logic to use the same object without de-allocating memory from
a previous path write/creation.

Note this required the addition of a `._redraw` flag (to be used in
`.clear()` and a small patch to `pyqtgraph.functions.arrayToQPath` to
allow passing in an existing path (thus reusing the same underlying mem
alloc) which will likely be first pushed to our fork.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 44f3a08ef1 Add optional uppx log scaling to m4 sampler
We were previously ad-hoc scaling up the px count/width to get more
detail at lower uppx values. Add a log scaling sigmoid that range scales
between 1 < px_width < 16.

Add in a flag to use the mxmn OH tracer in `ohlc_flatten()` if desired.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 03e0e3e76b Delegate to m4 ohlc helper for curve, only ds on uppx steps > 2 2022-04-30 11:36:23 -04:00
Tyler Goodlet 08f90c275c Add OHLC to m4 line converters
Helpers to quickly convert ohlc struct-array sequences into lines
for consumption by the m4 downsampler. Strip trailing zero entries
from the `ds_m4()` output if found (avoids lines back to origin).
2022-04-30 11:36:23 -04:00
Tyler Goodlet 7edfe68d4d M4 workin bishhhhh 2022-04-30 11:36:23 -04:00
Tyler Goodlet ff00993412 Call default view on symbol switch 2022-04-30 11:36:23 -04:00
Tyler Goodlet ed03d77e6e Make a derivs intrustment type table for alloc config checks 2022-04-30 11:36:23 -04:00
Tyler Goodlet 1a0e89d07e Even more correct "default view" snap-to-pp-marker
This makes the `'r'` hotkey snap the last bar to the middle of the pp
line arrow marker no matter the zoom level. Now we also boot with
approximately the most number of x units on screen that keep the bars
graphics drawn in full (just before downsampling to a line).

Moved some internals around to get this all in place,
- drop `_anchors.marker_right_points()` and move it to a chart method.
- change `.pre_l1_x()` -> `.pre_l1_xs()` and just have it return the
  two view-mapped x values from the former method.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 56c163cdd7 Make `ChartPlotWidget.default_view()` pin to L1
Instead of using a guess about how many x-indexes to reset the last
datum in-view to, calculate and shift the latest index such that it's
just before any L1 spread labels on the y-axis. This makes the view
placement "widget aware" and gives a much more cross-display UX.

Summary:
- add `ChartPlotWidget.pre_l1_x()` which returns a `tuple` of
  x view-coord points for the absolute x-pos and length of any L1
  line/labels
- make `.default_view()` only shift to see the xlast just outside
  the l1 but keep whatever view range xfirst as the first datum in view
- drop `LevelLine.right_point()` since this is now just a
  `.pre_l1_x()` call and can be retrieved from the line's internal chart
  ref
- drop `._style.bars_from/to_..` vars since we aren't using hard coded
  offsets any more
2022-04-30 11:36:23 -04:00
Tyler Goodlet c4242acc21 Pass in fqsn from chart UI components 2022-04-30 11:36:23 -04:00
Tyler Goodlet 772f871272 Use units by default for continuous futes 2022-04-30 11:36:23 -04:00
Tyler Goodlet 1ad83e4556 WIP add non-working m4 ds code to ohlc graphic 2022-04-30 11:36:23 -04:00
Tyler Goodlet bedb55b79d Use service cancel method for graceful teardown 2022-04-30 11:36:23 -04:00
Tyler Goodlet 03a08b5f63 Add curve px width getter
`ChartPlotWidget.curve_width_pxs()` now can be used to get the total
horizontal (x) pixels on screen that are occupied by the current curve
graphics for a given chart. This will be used for downsampling large
data sets to the pixel domain using M4.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 8f26335aea Add display loop profiling
Probably the best place to root the profiler since we can get a better
top down view of bottlenecks in the graphics stack.

More,
- add in draft M4 downsampling code (commented) after getting it mostly
  working; next step is to move this processing into an FSP subactor.
- always update the vlm chart last y-axis sticky
- set call `.default_view()` just before inf sleep on startup
2022-04-30 11:36:23 -04:00
Tyler Goodlet f1f257d4a2 Profiler format, code stretch 2022-04-30 11:36:23 -04:00
Tyler Goodlet d02b1a17ad Fix x-range -> # of frames calculation
Obviously determining the x-range from indices was wrong and was the
reason for the incorrect (downsampled) output size XD. Instead correctly
determine the x range and start value from the *values of* the input
x-array. Pretty sure this makes the implementation nearly production
ready.

Relates to #109
2022-04-30 11:36:23 -04:00
Tyler Goodlet 4d4f745918 Add `ChartPlotWidget.in_view()` shm-compatible array slicer 2022-04-30 11:36:23 -04:00
Tyler Goodlet 39b7c9340d Add (ostensibly) working first attempt at M4 algo
All the refs are in the comments and original sample code from infinite
has been reworked to expect the input x/y arrays to already be sliced
(though we can later support passing in the start-end indexes if
desired).

The new routines are `ds_m4()` the python top level API and `_m4()` the
fast `numba` implementation.
2022-04-30 11:36:23 -04:00
Tyler Goodlet e7481b1634 Array diff lengths must be int 2022-04-30 11:36:23 -04:00
Tyler Goodlet 09d95157dc Limit real-time chart updates in "big data" cases
- the chart's uppx (units-per-pixel) is > 4 (i.e. zoomed out a lot)
- don't shift the chart (to keep the most recent step in view) if the
  last datum isn't in view (aka the user is probably looking at history)
2022-04-30 11:36:23 -04:00