Commit Graph

3113 Commits (a4eb2d0febfd9ebc988610b86ebcbc026c12ba1d)

Author SHA1 Message Date
Tyler Goodlet a4eb2d0feb Drop `index_field` input to renders, add `.read()` profiling 2023-01-13 13:24:25 -05:00
Tyler Goodlet 7a71282d7e Delegate formatter `.index_field` to the parent `Viz` 2023-01-13 13:24:25 -05:00
Tyler Goodlet 342b3e5817 Facepalm**2: fix array-read-slice, like actually..
We need to subtract the first index in the array segment read, not the
first index value in the time-sliced output, to get the correct offset
into the non-absolute (`ShmArray.array` read) array..

Further we **do** need the `&` between the advance indexing conditions
and this adds profiling to see that it is indeed real slow (like 20ms
ish even when using `np.where()`).
2023-01-13 13:24:25 -05:00
Tyler Goodlet a681cd9870 Markup OHLC->path gen with `numba` issue # 2023-01-13 13:24:25 -05:00
Tyler Goodlet cbd2c73dd1 Facepalm: put graphics cycle in `do_ds: bool` block.. 2023-01-13 13:24:25 -05:00
Tyler Goodlet 1ba92a026f TOSQUASH: 552a8c298cd (return index for arrow..) 2023-01-13 13:24:25 -05:00
Tyler Goodlet c0a521358f Facepalm: actually return latest index on time slice fail.. 2023-01-13 13:24:25 -05:00
Tyler Goodlet 3cebfb07dc Go with explicit `.data._m4` mod name
Since it's a notable and self-contained graphics compression algo, might
as well give it a dedicated module B)
2023-01-13 13:24:25 -05:00
Tyler Goodlet 87e47f9eed Move (unused) path gen routines to `.ui._pathops` 2023-01-13 13:24:25 -05:00
Tyler Goodlet eda7b7f3c7 Move qpath-ops routines back to separate mod 2023-01-13 13:24:25 -05:00
Tyler Goodlet ce8279b6b1 Rename `.ui._pathops.py` -> `.ui._formatters.py 2023-01-13 13:24:25 -05:00
Tyler Goodlet 1144b3f2f5 Look up "index field" in display cycles
Again, to make epoch indexing a flip-of-switch for testing look up the
`Viz.index_field: str` value when updating labels.

Also, drops the legacy tick-type set tracking which we no longer use
thanks to the new throttler subsys and it's framing msgs.
2023-01-13 13:24:25 -05:00
Tyler Goodlet a7cebc0026 Fix from-time index slicing?
Apparently we want an `|` for the advanced indexing logic?
Also, fix `read_slc` start to not always be 0 XD
2023-01-13 13:24:25 -05:00
Tyler Goodlet e85e9f71ad Move old label sizing cruft to label mod 2023-01-13 13:24:25 -05:00
Tyler Goodlet 7c863b50e9 Move path ops routines to top of mod
Planning to put the formatters into a new mod and aggregate all path
gen/op helpers into this module.

Further tweak include:
- moving `path_arrays_from_ohlc()` back to module level
- slice out the last xy datum for `OHLCBarsAsCurveFmtr` 1d formatting
- always copy the new x-value from the source to `.x_nd`
2023-01-13 13:24:25 -05:00
Tyler Goodlet 366310b124 Drop diff state tracking in formatter
This was a major cause of error (particularly trying to get epoch
indexing working) and really isn't necessary; instead just have
`.diff()` always read from the underlying source array for current
index-step diffing and append/prepend slice construction.

Allows us to,
- drop `._last_read` state management and thus usage.
- better handle startup indexing by setting `.xy_nd_start/stop` to
  `None` initially so that the first update can be done in one large
  prepend.
- better understand and document the step curve "slice back to previous
  level" logic which is now heavily commented B)
- drop all the `slice_to_head` stuff from and instead allow each
  formatter to choose it's 1d segmenting.
2023-01-13 13:24:25 -05:00
Tyler Goodlet f403deb7d0 Explicitly enable chart widget yranging in display init 2023-01-13 13:24:25 -05:00
Tyler Goodlet 6ec30a2a6c Enable/disable vlm chart yranging (TO SQUASH) 2023-01-13 13:24:25 -05:00
Tyler Goodlet 733a6e3f8a Don't disable non-enabled vlm chart y-autoranging 2023-01-13 13:24:25 -05:00
Tyler Goodlet 483567797d Comment out bps for time indexing 2023-01-13 13:24:25 -05:00
Tyler Goodlet f980e2eddd Call `Viz.bars_range()` from display loop 2023-01-13 13:24:25 -05:00
Tyler Goodlet 1d3b03c937 TOSQUASH: f5dcf1dc (viz index field) 2023-01-13 13:24:25 -05:00
Tyler Goodlet 77c472f300 Fix `.default_view()` to view-left-of-data 2023-01-13 13:24:25 -05:00
Tyler Goodlet a63dd6c4f2 Add `Viz.index_field: str`, pass to graphics objs
In an effort to make it easy to override the indexing scheme.

Further, this repairs the `.datums_range()` special case to handle when
the view box is to-the-right-of the data set (i.e. l > datum_start).
2023-01-13 13:24:25 -05:00
Tyler Goodlet fd3c72b277 Expect `index_field: str` in all graphics objects 2023-01-13 13:24:25 -05:00
Tyler Goodlet 7e3a1720fc TOSQUASH: 2dc706aa (.default_view w time) 2023-01-13 13:24:25 -05:00
Tyler Goodlet ea368496eb Facepalm: pass correct flume to each FSP chart group.. 2023-01-13 13:24:25 -05:00
Tyler Goodlet fc993146b6 Attempt to make `.default_view()` time-index ready
As in make the call to `Flume.slice_from_time()` to try and convert any
time index values from the view range to array-indices; all untested
atm.

Also drop some old/unused/moved methods:
- `._set_xlimits()`
- `.bars_range()`
- `.curve_width_pxs()`

and fix some `flow` -> `viz` var naming.
2023-01-13 13:24:25 -05:00
Tyler Goodlet 301bfa2463 Simplify formatter update methodology
Don't expect values (array + slice) to be returned and applied by
`.incr_update_xy_nd()` and instead presume this will implemented
internally in each (sub)formatter.

Attempt to simplify some incr-update routines, (particularly in the step
curve formatter, though most of it was reverted to just a simpler form
of the original implementation XD) including:
- dropping the need for the `slice_to_head: int` control.
- using the `xy_nd_start/stop` index counters over custom lookups.
2023-01-13 13:24:25 -05:00
Tyler Goodlet a9da11451f TOSQUASH: f3d757c2 (flow->viz) 2023-01-13 13:24:25 -05:00
Tyler Goodlet 99b1230442 First attempt, field-index agnostic formatting
Remove harcoded `'index'` field refs from all formatters in a first
attempt at moving towards epoch-time alignment (though don't actually
use it it yet).

Adjustments to the formatter interface:
- property for `.xy_nd` the x/y nd arrays.
- property for and `.xy_slice` the nd format array(s) start->stop index
  slice.

Internal routine tweaks:
- drop `read_src_from_key` and always pass full source array on updates
  and adjust handlers to expect to have to index the data field of
  interest.
- set `.last_read` right after update calls instead of after 1d
  conversion.
- drop `slice_to_head` array read slicing.
- add some debug points for testing 'time' indexing (though not used
  here yet).
- add `.x_nd` array update logic for when the `.index_field` is not
  'index' - i.e. when we begin to try and support epoch time.
- simplify some new y_nd updates to not require use of `np.broadcast()`
  where possible.
2023-01-13 13:24:25 -05:00
Tyler Goodlet 05a7a06416 Pepper render routines with time-slice calls 2023-01-13 13:24:25 -05:00
Tyler Goodlet e3b9926e6f Add `Viz.bars_range()` (moved from chart API)
Call it from view kb loop.
2023-01-13 13:24:25 -05:00
Tyler Goodlet 0b596bf5a4 Make `Viz.slice_from_time()` take input array
Probably means it doesn't need to be a `Flume` method but it's
convenient to expect the caller to pass in the `np.ndarray` with
a `'time'` field instead of a `timeframe: str` arg; also, return the
slice mask instead of the sliced array as output (again allowing the
caller to do any slicing). Also, handle the slice-outside-time-range
case by just returning the entire index range with a `None` mask.

Adjust `Viz.view_data()` to instead do timeframe (for rt vs. hist shm
array) lookup and equiv array slicing with the returned mask.
2023-01-13 13:24:25 -05:00
Tyler Goodlet ba43b751ed Add breakpoint on -ve range for now 2023-01-13 13:24:25 -05:00
Tyler Goodlet f5f476f964 `Order.symbol` is a `str`.. 2023-01-13 13:23:49 -05:00
Tyler Goodlet b0a8728d28 Go back to hard-coded index field
Turns out https://github.com/numba/numba/issues/8622 is real
and the suggested `numba.literally` hack doesn't seem to work..
2023-01-13 13:23:49 -05:00
Tyler Goodlet 6ec113659b Move `ui._compression`/`._pathops` to `.data` subpkg
Since these modules no longer contain Qt specific code we might
as well include them in the data sub-package.

Also, add `IncrementalFormatter.index_field` as single point to def the
indexing field that should be used for all x-domain graphics-data
rendering.
2023-01-13 13:23:49 -05:00
Tyler Goodlet b8622d87a4 Rename `.ui._flows.py` -> `.ui._render.py` 2023-01-13 13:23:49 -05:00
Tyler Goodlet 47b9e59655 Rename `Flow` -> `Viz`
The type is better described as a "data visualization":
https://en.wikipedia.org/wiki/Data_and_information_visualization

Add `ChartPlotWidget.get_viz()` to start working towards not accessing
the private table directly XD

We'll probably end up using the name `Flow` for a type that tracks
a collection of composed/cascaded `Flume`s:
https://en.wikipedia.org/wiki/Two-port_network#Cascade_connection
2023-01-13 13:23:49 -05:00
Tyler Goodlet 42e0048b7c Adjust order mode to use `Flume.get_index()` 2023-01-13 13:23:49 -05:00
Tyler Goodlet b92ff7caf9 Pass `Flume`s throughout FSP-ui and charting APIs
Since higher level charting and fsp management need access to the
new `Flume` indexing apis this adjusts some func sigs to pass through
(and/or create) flume instances:
- `LinkedSplits.add_plot()` and dependents.
- `ChartPlotWidget.draw_curve()` and deps, and it now returns a `Flow`.
- `.ui._fsp.open_fsp_admin()` and `FspAdmin.open_fsp_ui()` related
  methods => now we wrap the destination fsp shm in a flume on the admin
  side and is returned from `.start_engine_method()`.

Drop a bunch of (unused) chart widget methods including some already
moved to flume methods: `.get_index()`, `.in_view()`,
`.last_bar_in_view()`, `.is_valid_index()`.
2023-01-13 13:23:49 -05:00
Tyler Goodlet 437fc511a3 Drop px-cache-resets, failed try at path appends
Comments out the pixel-cache resetting since it doesn't seem we need it
any more to avoid draw oddities?

For `.fast_path` appends, this nearly got it working except the new path
segments are either not being connected correctly (step curve) or not
being drawn in full since the history path (plain line).

Leaving the attempted code commented in for a retry in the future; my
best guesses are that maybe,
- `.connectPath()` call is being done with incorrect segment length
  and/or start point.
- the "appended" data: `appended = array[-append_len-1:slice_to_head]`
  (done inside the formatter) isn't correct (i.e. endpoint handling
  considering a path append) and needs special handling for different
  curve types?
2023-01-13 13:23:49 -05:00
Tyler Goodlet c6b0eaa347 Mask profile points and drop rect `.united()` attempts 2023-01-13 13:23:49 -05:00
Tyler Goodlet d71045400e Make curve graphics timeframe agnostic
Ensure `.boundingRect()` calcs and `.draw_last_datum()` do geo-sizing
based on source data instead of presuming some `1.0` unit steps in some
spots; we need this to support an epoch index as is needed for overlays.

Further, clean out a bunch of old bounding rect calc code and add some
commented code for trying out `QRectF.united()` on the path + last datum
curve segment. Turns out that approach is slower as per eyeballing the
added profiler points.
2023-01-13 13:23:49 -05:00
Tyler Goodlet d6ae75d743 Add graphics incr-updated "formatter" subsys
After trying to hack epoch indexed time series and failing miserably,
decided to properly factor out all formatting routines into a common
subsystem API: ``IncrementalFormatter`` which provides the interface for
incrementally updating and tracking pre-path-graphics formatted data.

Previously this functionality was mangled into our `Renderer` (which
also does the work of `QPath` generation and update) but splitting it
out also preps for being able to do graphics-buffer downsampling and
caching on a remote host B)

The ``IncrementalFormatter`` (parent type) has the default behaviour of
tracking a single field-array on some source `ShmArray`, updating
a flattened `numpy.ndarray` in-mem allocation, and providing a default
1d conversion for pre-downsampling and path generation.

Changed out of `Renderer`,
- `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to
  more explicitly named formatter methods.
- all `.x/y_data` nd array management and update
- "last view range" tracking
- `.last_read`, `.diff()`
- now calls `IncrementalFormatter.format_to_1d()` inside `.render()`

The new API gets,
- `.diff()`, `.last_read`
- all view range diff tracking through `.track_inview_range()`.
- better nd format array names: `.x/y_nd`, `xy_nd_start/stop`.
- `.format_to_1d()` which renders pre-path formatted arrays ready for
  both m4 sampling and path gen.
- better explicit overloadable formatting method names:
  * `.allocate_xy()` -> `.allocate_xy_nd()`
  * `.update_xy()` -> `.incr_update_xy_nd()`
  * `.format_xy()` -> `.format_xy_nd_to_1d()`

Finally this implements per-graphics-type formatters which define
each set up related formatting routines:
- `OHLCBarsFmtr`: std multi-line style bars
- `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data
- `StepCurveFmtr`: handles vlm style curves
2023-01-13 13:23:49 -05:00
Tyler Goodlet 4c799386c6 Max out per symbol throttle @ 22Hz 2023-01-13 13:23:49 -05:00
Tyler Goodlet 0945033d38 Move all pre-path formatting routines to `._pathops`, proto formatter type 2023-01-13 13:23:49 -05:00
Tyler Goodlet 18857ac077 Expect and update from by-type tick frames
Move to expect and process new by-tick-event frames where the display
loop can now just iterate the most recent tick events by type instead of
the entire tick history sequence - thus we reduce iterations inside the
update loop.

Also, go back to use using the detected display's refresh rate (minus 6)
as the default feed requested throttle rate since we can now handle
much more bursty-ness in display updates thanks to the new framing
format B)
2023-01-13 13:23:49 -05:00
Tyler Goodlet 5a9e985024 Factor info print into func 2023-01-13 13:23:49 -05:00