Just like for the source OHLC, we now have the chart parent actor create
an fsp shm array and use it to read back signal data for plotting.
Some tweaks to get the price chart (and sub-charts) to load historical
datums immediately instead of waiting on an initial quote.
Added a comment to clarify, ish.
Add `ChartPlotWidget._overlays` as registry of curves added on top of
main graphics. Hackishly (ad-hoc-ishly?) update the curve assuming the
data resides in the same `._array` for now (which it does for historical
vwap).
Allow passing a fixed ylow, yhigh tuple to `._set_yrange()` which avoids
recomputing the range from data if desired (eg. rsi-like bounded
signals). Add support for overlay curves to the OHLC chart and add basic
support to brokers which provide a historical 'vwap`. The data array
increment logic had to be tweaked to copy the vwap from the last bar.
Oh, and hack the subchart curves with two extra prepended datums to make
them align "better" with the ohlc main chart; need to talk to
`pyqtgraph` core about how to do this more correctly.
By mapping any in view "contents labels" to the range of the
``ViewBox``'s data we can avoid having graphics overlap with labels.
Take this approach instead of specifying a min y-range using the std
and activate the range compute on resize and mouser scrolling.
Also, add y-sticky update for signal plots.
Use two separate `QPicture` instances:
- one for the 3 lines for the last bar
- one for all the historical bars lines
On price changes update the last bar and only update historical bars
when the current bar's period expires (when a new bar is "added").
Add a flag `just_history` for this `BarItems.draw_lines()`.
Also, switch the internal lines array/buffer to a 2D numpy array to avoid
the type-cast step and instead just flatten using `numpy.ravel()`.
Overall this should avoid the problem of draws getting slower over time
as new bars are added to the history since price updates only redraw
a single bar to the "last" `QPicture` instance. Ideally in the future we
can make the `history` `QPicture` a `QPixmap` but it looks like this
will require some internal work in `pyqtgraph` to support it.
Use a ``rec2array`` struct array converter to generate lines sequence
faster. Start looking into using a `QPixmap` to avoid redrawing all
bars every update.
Add a default "contents label" (eg. OHLC values for bar charts) to each
chart and update on crosshair interaction.
Few technical changes to make this happen:
- adjust bar graphics to have the HL line be in the "middle" of the
underlying arrays' "index range" in the containing view.
- add a label dict each chart's graphics name to a label + update routine
- use symbol names instead of this "main" identifier crap for referring to
particular price curves/graphics
This is a first attempt at a financial signal processing subsystem which
utilizes async generators for streaming frames of numpy array data
between actors. In this initial attempt the focus is on processing price
data and relaying it to the chart app for real-time display. So far this
seems to work (with decent latency) but much more work is likely needed
around improving the data model for even better latency and less data
duplication.
Surprisingly (or not?) a lot of simplifications to the charting code
came out of this in terms of conducting graphics updates in streaming
tasks instead of hiding them inside the obfuscated mess that is the
Qt-style-inheritance-OO-90s-trash. The goal from here on wards will be
to enforce strict semantics around reading and writing of data such that
state is kept outside "object trees" as much as possible and streaming
function semantics guide our flow model. Unsurprisingly, this reduction
in "instance state" is happening wherever we use `trio` ;)
A little summary on the technical changes:
- not going to explain the fsp system yet; it's too nascent and
probably going to get some heavy editing.
- drop any "update" methods from the `LinkedCharts` type since each
sub-chart will have it's own update task and thus a separate update
loop; further individual graphics (per chart) may eventually require
this same design.
- delete `ChartView`; moved into separate mod.
- add "stream from fsp" task to start our foray into real-time actor
processed numpy streaming.
Wait for a first actual real-time quote before starting graphics update
tasks. Use the new normalized tick format brokers are expected to emit
as a `quotes['ticks']` list. Auto detect time frame from historical
bars.
Add `ChartPlotWidget.add_plot()` to add sub charts for indicators which
can be updated independently. Clean up rt bar update code and drop some
legacy ohlc loading cruft.
Stop with all this "main chart" special treatment.
Manage all lines in the same way across all referenced plots.
Add `CrossHair.add_plot()` for adding new plots dynamically.
Just, smh.
There's really nothing coupling it to the graphics class (which frankly
also seems like it doesn't need to be a class.. Qt).
Add support to `.update_from_array()` for diffing with the input array
and creating additional bar-lines where necessary. Note, there are still
issues with the "correctness" here in terms of bucketing open/close
values in the time frame / bar range. Also, this jamming of each bar's 3
lines into a homogeneous array seems like it could be better done with
struct arrays and avoid all this "index + 3" stuff.
Flat bars have a rendering issue we work around by hacking values in `QLineF`
but we have to revert those on any last bar that is being updated in
real-time. Comment out candle implementations for now; we can get back
to it if/when the tinas unite. Oh, and make bars have a little space
between them.
Don't allow zooming to less then a min number of data points. Allow
panning "outside" the data set (i.e. moving one of the sequence "ends"
to the middle of the view. Start adding logging.
For whatever reason if the `QLineF` high/low values are the same a weird
little rectangle is drawn (my guess is a `float` precision error of some
sort). Instead, if they're the same just use one of the values.
Also, store local vars to avoid so many lookups.
`pg.PlotCurveItem.setData()` is normally used for real-time updates to
curves and takes in a whole new array of data to graphics.
It makes sense to stick with this interface especially if
the current datum graphic will originally be drawn from tick quotes and
later filled in when bars data is available (eg. IB has this option in
TWS charts for volume). Additionally, having a data feed api where the push
process/task can write to shared memory and the UI task(s) can read from
that space is ideal. It allows for indicator and algo calculations to be
run in parallel (via actors) with initial price draw instructions
such that plotting of downstream metrics can be "pipelined" into the
chart UI's render loop. This essentially makes the chart UI async
programmable from multiple remote processes (or at least that's the
goal).
Some details:
- Only store a single ref to the source array data on the
`LinkedSplitCharts`. There should only be one reference since the main
relation is **that** x-time aligned sequence.
- Add `LinkedSplitCharts.update_from_quote()` which takes in a quote
dict and updates the OHLC array from it's contents.
- Add `ChartPlotWidget.update_from_array()` method to trigger graphics
updates per chart with consideration for overlay curves.
This makes a OHLC graphics "sequence" update very similar (actually API
compatible) with `pg.PlotCurveItem.setData()`. The difference here is
that only latest OHLC datum is used to update the charts last bar.
This was a mess before with a weird loop using the parent split charts
to update all "indicators". Instead just have each plot do its own
yrange updates since the signals are being handled just fine per plot.
Handle both the OHLC and plane line chart cases with a hacky `try:,
except IndexError:` for now.
Oh, and move the main entry point for the chart app to the relevant
module. I added some WIP bar update code for the moment.
Speed up the lines array creation using proper slice assignment.
This gives another 10% speedup to the historical price rendering.
Drop ``_tina_mode`` support for now since we're not testing it.
Previously graphics were loaded and rendered implicitly during the
import and creation of certain objects. Remove all this and instead
expect client code to pass in the OHLC sequence to plot. Speed up
the bars graphics rendering by simplifying to a single iteration of
the input array; gives about a 2x speedup.
Move chart resize code into our ``ViewBox`` subtype (a ``ChartView``)
in an effort to start organizing interaction behaviour closer to the
appropriate underlying objects. Add some docs for all this and do some
renaming.
Modify the default ``ViewBox`` scroll to zoom behaviour such that
whatever right-most point is visible is used as the "center" for
zooming. Add a "traditional" cross-hair cursor.
- Move out equity plotting to new module.
- Make axis margins and fonts look good on i3.
- Adjust axis labels colors to gray.
- Start commenting a lot of the code after figuring out what it all does
when cross referencing with ``pyqtgraph``.
- Add option to move date axis to middle.
Hand select necessary components to get real-time charting with
`pyqtgraph` from the `Quantdom` projects:
https://github.com/constverum/Quantdom
We've offered to collaborate with the author but have received no
response and the project has not been updated in over a year.
Given this, we are moving forward with taking the required components to
make further improvements upon especially since the `pyqtgraph` project
is now being actively maintained again.
If the author comes back we will be more then happy to contribute
modified components upstream:
https://github.com/constverum/Quantdom/issues/18
Relates to #80
This required some copy-paste of code from @matham's branch:
https://github.com/kivy/kivy/pull/5241
namely, the stuff in the `utils_async.py` module. I've added all that as
a standalone file for now.
Update the pipfile to use `kivy`'s master branch (since there seems to
be some lingering cython issues in the current release wheels).
- stop displaying search bar widget on <ctrl-c>
- if there's existing search bar content highlight it automatically
to allow user to start typing new content right away
- when activated allow search bar to insert its own set of keybinding
controls; restore prior bindings on exit
Look up the broker module and set up the loglevel locally.
Ask the arbiter for a portal to the data daemon since we can't
pass one to a subactor by reference.
This allows for using a monitor to select the current option chain
symbol!
The deats:
- start a bg task which streams the monitor selected symbol
- dynamically repopulate expiry buttons on a newly published symbol
- move static widget creation into a chain method to avoid multiple
quotes requests at startup
- rename a bunch of methods
This is an optimization to improve performance when the UI is fed real
time data. Instead of resorting all rows on every quote update, only
re-render when the sort key appears in the quote data, and further, only
resort rows which are changed using bisection based widget insertion to
avoid having `kivy` re-add widgets (and thus re-render graphics) more
often than absolutely necessary.
There's still a ton to polish (and some bugs to fix) but this is a first
working draft of a real-time option chain!
Insights and todos:
- `kivy` widgets need to be cached and reused (eg. rows, cells, etc.)
for speed since it seems creating new ones constantly is quite taxing
on the CPU
- the chain will tear down and re-setup the option data feed stream each
time a different contract expiry button set is clicked
- there's still some weird bug with row highlighting where it seems rows
added from a new expiry set (which weren't previously rendered) aren't
being highlighted reliably
`Row`:
- `no_cell`: support a list of keys for which no cells will be created
- allow passing in a `cell_type` at instantiation
`TickerTable`:
- keep track of rendered rows via a private `_rendered` set
- don't create rows inside `append_row()` expect caller to do it
- never render already rendered widgets in `render_rows()`
Miscellaneous:
- generalize `update_quotes()` to not be tied to specific quote fields
and allow passing in a quote `formatter()` func
- don't bother creating a nursery block until necessary in main
- more commenting
Copy out `kivy.clock.triggered` from version 1.10.1 since it isn't yet
available in the `trio`/async branch and use it to throttle the callback
rate. Use a `collections.deque` to LIFO iterate widgets each call
using the heuristic that it's more likely the mouse is still within the
currently highlighted (or it's adjacent neighbors) widget as opposed
to some far away widget (the case when the mouse is moved very
drastically across the window).
Thanks yet again to @tshirtman for suggesting this.
Instead of defining a `on_mouse_pos()` on every widget simply
register and track each widget and loop through them all once (or as much
as is necessary) in a single callback. The assumption here is that we
get a performance boost by looping widgets instead of having `kivy` loop
and call back each widget thus avoiding costly python function calls.
Instead of all this adding/removing of canvas instructions nonsense
simple add a static "highlighted" rectangle to each row and make its
size very small when there's no mouse over.
Mad props to @tshirtman for showing me the light :D
It's still a bit of a shit show, and I've left a lot of commented tweaks
that need to be further played with, but I think this is a much
better look for what I'm considering to be one of the main "entry point"
apps for `piker`. To get any more serious fine tuning the way I want
I may have to talk to some kivy experts as I'm having some headaches
with button borders, padding, and the header row height..
Some of the new changes include:
- port to the new `brokers.data` module
- much darker theme with a stronger terminal vibe
- last trade price and volume amount flash on each trade
- fixed the symbol search bar to be a static height; before it was
getting squashed oddly when using stacked windows
- make all the cells transparent (for now) such that I can just use
a row color (relates to cell padding/spacing - can't seem to ditch it)
- start adding type annotations
- make the % daily change use the previous days close as the reference
price
- color each cell on every change (results in "pulsed" colors on changes)
- tweak some quote fields
- redraw and sort all rows on every quotes update cycle
- error when the QT api is returning None values
Tyler Goodlet
Tyler Goodlet