Commit Graph

3142 Commits (44a328536735e71f7282b4f42d8cd4872b56c6c5)

Author SHA1 Message Date
Tyler Goodlet 432d4545c2 Fix last values, must be pulled from source data in step mode 2022-06-05 22:13:08 -04:00
Tyler Goodlet fa30df36ba Simplify default xy formatter 2022-06-05 22:13:08 -04:00
Tyler Goodlet 17456d96e0 Drop tons of old cruft, move around some commented ideas 2022-06-05 22:13:08 -04:00
Tyler Goodlet 167ae96566 Move graphics update logic into `Renderer.render()`
Finally this gets us much closer to a generic incremental update system
for graphics wherein the input array diffing, pre-graphical format data
processing, downsampler activation and incremental update and storage of
any of these data flow stages can be managed in one modular sub-system
:surfer_boi:.

Dirty deatz:
- reorg and move all path logic into `Renderer.render()` and have it
  take in pretty much the same flags as the old
  `FastAppendCurve.update_from_array()` and instead storing all update
  state vars (even copies of the downsampler related ones) on the
  renderer instance:
    - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr`
    - `.render()` input bools: `new_sample_rate, should_redraw,
      should_ds, showing_src_data`
    - add a hack-around for passing in incremental update data (for now)
    via a `input_data: tuple` of numpy arrays
    - a default `uppx: float = 1`

- add new render interface attrs:
 - `.format_xy()` which takes in the source data array and produces out
   x, y arrays (and maybe a `connect` array) that can be passed to
   `.draw_path()` (the default for this is just to slice out the index
   and `array_key: str` columns from the input struct array),
 - `.draw_path()` which takes in the x, y, connect arrays and generates
   a `QPainterPath`
 - `.fast_path`, for "appendable" updates like there was on the fast
   append curve
 - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger
   via `redraw: bool` flag.

- our graphics objects no longer set their own `.path` state, it's done
  by the `Flow.update_graphics()` method using output from
  `Renderer.render()` (and it's state if necessary)
2022-06-05 22:13:08 -04:00
Tyler Goodlet aa0efe1523 Drop `BarItems.draw_from_data()` 2022-06-05 22:13:08 -04:00
Tyler Goodlet 664a208ae5 Drop path generation from `gen_ohlc_qpath()` 2022-06-05 22:13:08 -04:00
Tyler Goodlet 876add4fc2 Drop `.update()` call from `.draw_last()` 2022-06-05 22:13:08 -04:00
Tyler Goodlet 72e849c651 Drop commented cruft from update logic 2022-06-05 22:13:08 -04:00
Tyler Goodlet b3ae562e4f Fully drop `.update_from_array()` 2022-06-05 22:13:08 -04:00
Tyler Goodlet b5b9ecf4b1 Treat paths like input/output vars 2022-06-05 22:13:08 -04:00
Tyler Goodlet 1dab77ca0b Rect wont show on step curves unless we avoid `.draw_last()` 2022-06-05 22:13:08 -04:00
Tyler Goodlet 4c7661fc23 Factor `.update_from_array()` into `Flow.update_graphics()`
A bit hacky to get all graphics types working but this is hopefully the
first step toward moving all the generic update logic into `Renderer`
types which can be themselves managed more compactly and cached per
uppx-m4 level.
2022-06-05 22:13:08 -04:00
Tyler Goodlet e258654c86 Just drop "line dot" updates for now.. 2022-06-05 22:13:08 -04:00
Tyler Goodlet 81be0b4bd0 Dont pass `px_width` to m4, add some commented path cap tracking 2022-06-05 22:13:08 -04:00
Tyler Goodlet df1c89e811 Drop all "pixel width" refs (`px_width`) from m4 impl 2022-06-05 22:13:08 -04:00
Tyler Goodlet f67fd11a29 Little formattito 2022-06-05 22:13:08 -04:00
Tyler Goodlet 1f95ba4fd8 Drop input xy from constructor, only keep state for cursor stuff.. 2022-06-05 22:13:08 -04:00
Tyler Goodlet 1dca7766d2 Add notes about how to do mkts "trimming"
Which is basically just "deleting" rows from a column series.
You can only use the trim command from the `.cmd` cli and only with a so
called `LocalClient` currently; it's also sketchy af and caused
a machine to hang due to mem usage..

Ideally we can patch in this functionality for use by the rpc api
and have it not hang like this XD

Pertains to https://github.com/alpacahq/marketstore/issues/264
2022-06-05 22:13:08 -04:00
Tyler Goodlet b236dc72e4 Make vlm a float; discrete is so 80s 2022-06-05 22:13:08 -04:00
Tyler Goodlet 27ee9fdc81 Drop old non-working flatten routine 2022-06-05 22:13:08 -04:00
Tyler Goodlet 5d294031f2 Factor step format data gen into `to_step_format()`
Yet another path ops routine which converts a 1d array into a data
format suitable for rendering a "step curve" graphics path (aka a "bar
graph" but implemented as a continuous line).

Also, factor the `BarItems` rendering logic (which determines whether to
render the literal bars lines or a downsampled curve) into a routine
`render_baritems()` until we figure out the right abstraction layer for
it.
2022-06-05 22:13:08 -04:00
Tyler Goodlet 537b725bf3 Factor ohlc to line data conversion into `._pathops.ohlc_to_line()` 2022-06-05 22:13:08 -04:00
Tyler Goodlet ca5a25f921 Drop commented `numba` imports 2022-06-05 22:13:08 -04:00
Tyler Goodlet 037300ced0 Move ohlc lines-curve generators into pathops mod 2022-06-05 22:13:08 -04:00
Tyler Goodlet 9c5bc6deda Add `.ui._pathops` module
Starts a module for grouping together all our `QPainterpath` related
generation and data format operations for creation of fast curve
graphics. To start, drops `FastAppendCurve.downsample()` and moves
it to a new `._pathops.xy_downsample()`.
2022-06-05 22:13:08 -04:00
Tyler Goodlet bc50db5925 Rename `._ohlc.gen_qpath()` -> `.gen_ohlc_qpath()` 2022-06-05 22:13:08 -04:00
Tyler Goodlet e8e26232ea Drop `BarItems.update_from_array()`; moved into `Flow` 2022-06-05 22:13:08 -04:00
Tyler Goodlet f6909ae395 Drop legacy step mode data formatter 2022-06-05 22:13:08 -04:00
Tyler Goodlet b609f46d26 Always delay interaction update profiling 2022-06-05 22:13:08 -04:00
Tyler Goodlet 5d26609693 Add "no-tsdb-found" history load length defaults 2022-06-05 22:13:08 -04:00
Tyler Goodlet 09e988ec3e Use `ms_threshold` throughout remaining profilers 2022-06-05 22:13:08 -04:00
Tyler Goodlet 5e602214be Use new flag, add more marks through display loop 2022-06-05 22:13:08 -04:00
Tyler Goodlet cfc4198837 Use new profiler arg name, add more marks throughout flow update 2022-06-05 22:13:08 -04:00
Tyler Goodlet c455df7fa8 Drop legacy step path gen, always slice full data
Mostly just dropping old commented code for "step mode" format
generation. Always slice the tail part of the input data and move to the
new `ms_threshold` in the `pg` profiler'
2022-06-05 22:13:08 -04:00
Tyler Goodlet 47cf4aa4f7 Error log brokerd msgs that have `.reqid == None`
Relates to the bug discovered in #310, this should avoid out-of-order
msgs which do not have a `.reqid` set to be error logged to console.
Further, add `pformat()` to kraken logging of ems msging.
2022-06-05 22:13:08 -04:00
Tyler Goodlet 4f36743f64 Only udpate prepended graphics when actually in view 2022-06-05 22:13:08 -04:00
Tyler Goodlet 1fcb9233b4 Add back mx/mn updates for L1-in-view, lost during rebase 2022-06-05 22:13:08 -04:00
Tyler Goodlet fb38265199 Clean out legacy code from `Flow.update_graphics()` 2022-06-05 22:13:08 -04:00
Tyler Goodlet e163a7e336 Drop `bar_wap` curve for now, seems to also be causing hangs?! 2022-06-05 22:13:08 -04:00
Tyler Goodlet 36a10155bc Add profiler passthrough type annot, comments about appends vs. uppx 2022-06-05 22:13:08 -04:00
Tyler Goodlet 7a3437348d An absolute uppx diff of >= 1 seems more then fine 2022-06-05 22:13:08 -04:00
Tyler Goodlet 0744dd0415 Up the display throttle rate to 22Hz 2022-06-05 22:13:08 -04:00
Tyler Goodlet 0770a39125 Only do curve appends on low uppx levels 2022-06-05 22:13:08 -04:00
Tyler Goodlet 2b6041465c Startup up with 3k bars 2022-06-05 22:13:08 -04:00
Tyler Goodlet 859eaffa29 Drop vwap fsp for now; causes hangs.. 2022-06-05 22:13:08 -04:00
Tyler Goodlet b12921678b Drop step routine import 2022-06-05 22:13:08 -04:00
Tyler Goodlet 186658ab09 Drop uppx guard around downsamples on interaction
Since downsampling with the more correct version of m4 (uppx driven
windows sizing) is super fast now we don't need to avoid downsampling
on low uppx values. Further all graphics objects now support in-view
slicing so make sure to use it on interaction updates. Pass in the view
profiler to update method calls for more detailed measuring.

Even moar,
- Add a manual call to `.maybe_downsample_graphics()` inside the mouse
  wheel event handler since it seems that sometimes trailing events get
  lost from the `.sigRangeChangedManually` signal which can result in
  "non-downsampled-enough" graphics on chart given the scroll amount;
  this manual call seems to entirely fix this?
- drop "max zoom" guard since internals now support (near) infinite
  scroll out to graphics becoming a single pixel column line XD
- add back in commented xrange signal connect code for easy testing to
  verify against range updates not happening without it
2022-06-05 22:13:08 -04:00
Tyler Goodlet 12d60e6d9c WIP get incremental step curve updates working
This took longer then i care to admit XD but it definitely adds a huge
speedup and with only a few outstanding correctness bugs:

- panning from left to right causes strange trailing artifacts in the
  flows fsp (vlm) sub-plot but only when some data is off-screen on the
  left but doesn't appear to be an issue if we keep the `._set_yrange()`
  handler hooked up to the `.sigXRangeChanged` signal (but we aren't
  going to because this makes panning way slower). i've got a feeling
  this is a bug todo with the device coordinate cache stuff and we may
  need to report to Qt core?
- factoring out the step curve logic from
  `FastAppendCurve.update_from_array()` (un)fortunately required some
  logic branch uncoupling but also meant we needed special input controls
  to avoid things like redraws and curve appends for special cases,
  this will hopefully all be better rectified in code when the core of
  this method is moved into a renderer type/implementation.
- the `tina_vwap` fsp curve now somehow causes hangs when doing erratic
  scrolling on downsampled graphics data. i have no idea why or how but
  disabling it makes the issue go away (ui will literally just freeze
  and gobble CPU on a `.paint()` call until you ctrl-c the hell out of
  it). my guess is that something in the logic for standard line curves
  and appends on large data sets is the issue?

Code related changes/hacks:
- drop use of `step_path_arrays_from_1d()`, it was always a bit hacky
  (being based on `pyqtgraph` internals) and was generally hard to
  understand since it returns 1d data instead of the more expected (N,2)
  array of "step levels"; instead this is now implemented (uglily) in
  the `Flow.update_graphics()` block for step curves (which will
  obviously get cleaned up and factored elsewhere).
- add a bunch of new flags to the update method on the fast append
  curve:  `draw_last: bool`, `slice_to_head: int`, `do_append: bool`,
  `should_redraw: bool` which are all controls to aid with previously
  mentioned issues specific to getting step curve updates working
  correctly.
- add a ton of commented tinkering related code (that we may end up
  using) to both the flow and append curve methods that was written as
  part of the effort to get this all working.
- implement all step curve updating inline in `Flow.update_graphics()`
  including prepend and append logic for pre-graphics incremental step
  data maintenance and in-view slicing as well as "last step" graphics
  updating.

Obviously clean up commits coming stat B)
2022-06-05 22:13:08 -04:00
Tyler Goodlet c5beecf8a1 Drop cursor debounce delay, decrease rate limit 2022-06-05 22:13:08 -04:00
Tyler Goodlet 629ea8ba9d Downsample on every uppx inrement since it's way faster 2022-06-05 22:13:08 -04:00