Commit Graph

565 Commits (247b5fa2ec65f315dfcf994116644568105eb5ee)

Author SHA1 Message Date
Tyler Goodlet 94a8ee6270 Optimize axis labels using `QPicture`
This is likely a marginal improvement but is slightly less execution and
adds a coolio black border around the label. Drop all the legacy code
from quantdom which was quite a convoluted mess for "coloring".
Had to tweak sticky offsets to get the crosshair to line up right; not
sure what that's all about yet.
2020-10-24 20:12:12 -04:00
Tyler Goodlet 13f32acfdf Only update history on bar increment
With the improved update logic on `BarsItems` it doesn't seem to be
necessary. Remove y sticky for overlays for now to avoid clutter that
looks like double draws when the last overlay value is close to the last
price.
2020-10-22 21:21:14 -04:00
Tyler Goodlet 8eb4344d86 More "thematic" default view values ;) 2020-10-22 20:42:46 -04:00
Tyler Goodlet 8c25892521 Fix (really sidestep) flat bar rendering issue(s)
It seems a plethora of problems (including drawing performance) are due
to trying to hack around the strange rendering bug in Qt with `QLineF`
with y1 == y2. There was all sorts of weirdness that would show up with
trying (a hack) to just set all 4 points to the same value including
strange infinite diagonal ghost lines randomly on charts. Instead, just
place hold these flat bar's 'body' line with a `None` and filter the
null values out before calling `QPainter.drawLines()`. This results
in simply no body lines drawn for these datums. We can probably `numba`
the filtering too if it turns out to be a bottleneck.
2020-10-22 20:35:51 -04:00
Tyler Goodlet 7be624de39 Implement `tickStrings` for price axis; use float_digits() 2020-10-22 20:22:21 -04:00
Tyler Goodlet 18dc809acb Add naive digits count routine 2020-10-22 14:05:35 -04:00
Tyler Goodlet f2c4a46c94 Center bars around index, adjust curves back to match... 2020-10-21 11:19:41 -04:00
Tyler Goodlet cd828db9e9 Show 24 bars to left on startup 2020-10-21 10:51:54 -04:00
Tyler Goodlet 875bc8be24 Add line dots cursors to curves by default 2020-10-21 10:46:56 -04:00
Tyler Goodlet 1f6b5da17e Add support for curve "cursors" using a filled dot
Add a new graphic `LineDot` which is a `pg.CurvePoint` that draws
a simple filled dot over a curve at the specified index.

Add support for adding these cursor-dots to the crosshair/mouse through
a new `CrossHair.add_curve_cursor()`. Discretized the vertical line
updates on the crosshair such that it's only drawn in the middle of
the current bar in the main chart.
2020-10-21 10:40:51 -04:00
Tyler Goodlet 88583d999a Add "follow mode"
Makes the chart act like tws where each new time step increment the
chart shifts to the right so that the last bar stays in place. This
gets things looking like a proper auto-trading UX.

Added a couple methods to ``ChartPlotWidget`` to make this work:
- ``.default_view()`` to set the preferred view based on user settings
- ``.increment_view()`` to shift the view one time frame right

Also, split up the `.update_from_array()` method to be curve/ohlc
specific allowing for passing in a struct array with a named field
containing curve data more straightforwardly. This also simplifies the
contest label update functions.
2020-10-20 09:13:27 -04:00
Tyler Goodlet 1902507703 Change scroll "center" to rightmost bar on screen 2020-10-19 21:32:50 -04:00
Tyler Goodlet c57f678295 Fix contents labels issues
Lookup overlay contents from the OHLC struct array (for now / to make
things work) and fix anchoring logic with better offsets to keep
contents labels super tight to the edge of the view box. Unfortunately,
had to hack the label-height-calc thing for avoiding overlap of graphics
with the label; haven't found a better solution yet and pyqtgraph seems
to require more rabbit holing to figure out something better. Slap in
some inf lines for over[sold/bought] rsi conditions thresholding.
2020-10-19 14:18:06 -04:00
Tyler Goodlet 851104dd31 Add an inf horizontal line helper 2020-10-19 14:01:57 -04:00
Tyler Goodlet 1706b67e00 Note the issues with the shared fsp array index.. 2020-10-19 14:01:25 -04:00
Tyler Goodlet 32974a118c Add hidpi comments, 300 bars in view at startup 2020-10-19 11:37:50 -04:00
Tyler Goodlet d3dc8fb219 Differentiate array schema by close field 2020-10-19 11:37:28 -04:00
Tyler Goodlet c7d5ea6e15 Fix static yrange and last bar double draw issues 2020-10-16 12:18:14 -04:00
Tyler Goodlet fc23b2180d Pass through fonts to axes 2020-10-16 12:15:33 -04:00
Tyler Goodlet 58d3234f74 Drop lingering print 2020-10-16 12:15:07 -04:00
Tyler Goodlet f4c38621d5 Add a default "bars from right" style setting 2020-10-15 15:08:56 -04:00
Tyler Goodlet 6f429b1104 These seem to be "faster" crosshair settings? 2020-10-15 15:08:16 -04:00
Tyler Goodlet cb72662350 Add warnings for shm cache misses 2020-10-15 15:07:56 -04:00
Tyler Goodlet 454b445b4b Add better shared mem writer task checking
If you have a common broker feed daemon then likely you don't want to
create superfluous shared mem buffers for the same symbol. This adds an
ad hoc little context manger which keeps a bool state of whether
a buffer writer task currently is running in this process. Before we
were checking the shared array token cache and **not** clearing it when
the writer task exited, resulting in incorrect writer/loader logic on
the next entry..

Really, we need a better set of SC semantics around the shared mem stuff
presuming there's only ever one writer per shared buffer at given time.
Hopefully that will come soon!
2020-10-15 15:02:42 -04:00
Tyler Goodlet db273e1cd7 Up the mouse re-draw rate limit 2020-10-12 09:36:46 -04:00
Tyler Goodlet 18097fc33b Scale for hidpi pixmaps too 2020-10-02 12:13:50 -04:00
Tyler Goodlet 2302e59f1d 3k bars for now, ignore rtTime 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8d9a6845c6 Add a naive maxmin calc to avoid frequent resizes
If we know the max and min in view then on datum updates we can avoid
resizing the y-range when a new max/min has not yet arrived.

This adds a very naive numpy calc in the drawing thread which we can
likely improve with a more efficient streaming alternative which can
also likely be run in a fsp subactor. Also, since this same calc is
essentially done inside `._set_yrange()` we will likely want to allow
passing the result into the method to avoid duplicate work.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 6d5ccc6c3f Specify font size in pixels 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8276b02f92 Further label and axis sizing tweaks for hidpi 2020-10-02 12:13:50 -04:00
Tyler Goodlet e524ee9045 Resize everything with HiDPI scaling on 2020-10-02 12:13:50 -04:00
Tyler Goodlet bceeaa56ff Drop ringbuf, didn't end up using 2020-10-02 12:13:50 -04:00
Tyler Goodlet acc8dd66f5 Add data._normalize.py ... 2020-10-02 12:13:50 -04:00
Tyler Goodlet bc65040601 Add shm support to kraken backend 2020-10-02 12:13:50 -04:00
Tyler Goodlet d4eb5ccca4 Handle vwap overlay with shm 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8a4528c006 Always ask backend for ohlc dtype 2020-10-02 12:13:50 -04:00
Tyler Goodlet 47d4ec5985 Move _source under data package 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8e2e695ba8 Revert project name removal; breaks shit elsewhere somehow.. 2020-10-02 12:13:50 -04:00
Tyler Goodlet 155c3eef2a Convert timeit thing to use print() 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8832804bab Sub each new symbol to shm incrementing 2020-10-02 12:13:50 -04:00
Tyler Goodlet e3e219aa4b Add multi-symbol-buffer increment support 2020-10-02 12:13:50 -04:00
Tyler Goodlet 41e85ccaa9 Break wma calc into sync func 2020-10-02 12:13:50 -04:00
Tyler Goodlet 3f0e175011 Get bar oriented RSI working correctly 2020-10-02 12:13:50 -04:00
Tyler Goodlet 268e748417 Drop extra prefix in logs 2020-10-02 12:13:50 -04:00
Tyler Goodlet bfa7839370 Adopt official color 2020-10-02 12:13:50 -04:00
Tyler Goodlet 2fcbefa6e1 Use shm in fsp cascading
This kicks off what will be the beginning of hopefully a very nice
(soft) real-time financial signal processing system. We're keeping the
hack to "time align" curves (for now) with the bars for now by slapping
in an extra datum at index 0.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 4383579cd0 Use shm array in chart-fsp task
Just like for the source OHLC, we now have the chart parent actor create
an fsp shm array and use it to read back signal data for plotting.
Some tweaks to get the price chart (and sub-charts) to load historical
datums immediately instead of waiting on an initial quote.
2020-10-02 12:13:50 -04:00
Tyler Goodlet ba4261f974 Add timeit prints 2020-10-02 12:13:50 -04:00
Tyler Goodlet 561cafbe55 Another black 2020-10-02 12:13:50 -04:00
Tyler Goodlet efb52f2292 Make shared array buffer incrementer a message pub
Drop ctx manager api and use `tractor.msg.pub`.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 373ff90229 Only need UTC offset hacking if time w broker is messed.. 2020-10-02 12:13:50 -04:00
Tyler Goodlet d93ce84a99 Variety of IB backend improvements
- Move to new shared mem system only writing on the first (by process)
  entry to `stream_quotes()`.
- Deliver bars before first quote arrives so that chart can populate and
  then wait for initial arrival.
- Allow caching clients per actor.
- Load bars using the same (cached) client that starts the quote stream
  thus speeding up initialization.
2020-10-02 12:13:50 -04:00
Tyler Goodlet b1093dc71d Add a `data.Feed` type
Wraps the growing tuple of items being delivered by `open_feed()`.
Add lazy loading of the broker's signal step stream with
a `Feed.index_stream()` method.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 38469bd6ef Slight rework: shm API
Add an internal `_Token` to do interchange (un)packing for passing
"references" to shm blocks between actors.  Part of the token involves
providing the `numpy.dtype` in a cross-actor format.  Add a module
variable for caching "known tokens" per actor.  Drop use of context
managers since they tear down shm blocks too soon in debug mode and
there seems to be no reason to unlink/close shm before the process has
terminated; if code needs it torn down explicitly, it can.
2020-10-02 12:13:50 -04:00
Tyler Goodlet cd540fd07e Cleanups 2020-10-02 12:13:50 -04:00
Tyler Goodlet 6fa4f6e943 Port charts to new shm arrays 2020-10-02 12:13:50 -04:00
Tyler Goodlet f872fbecf8 Hook IB up to shared memory system
Adjust the `data.open_feed()` api to take a shm token so the
broker-daemon can attach a previously created (by the parent actor) mem
buf and push real-time tick data. There's still some sloppiness here in
terms of ensuring only one mem buf per symbol (can be seen in
`stream_quotes()`) which should really managed at the data api level.
Add a bar incrementing stream-task which delivers increment msgs to any
consumers.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 17491ba819 Disconnect stdlib's resource_tracker, fix .push()
Logic in `SharedArray.push()` was totally wrong.
Remove all the `multiprocessing.resource_tracker` crap such that we
aren't loading an extra process at every layer and we don't get tons of
errors when cleaning on in an SC way.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 712e36b9d5 First draft of a shared numpy array sub-system
This adds a shared memory "incrementing array" sub-sys interface
for single writer, multi-reader style data passing. The main motivation
is to avoid multiple copies of the same `numpy` array across actors
(plus now we can start being fancy like ray).

There still seems to be some odd issues with the "resource tracker"
complaining at teardown (likely partially to do with SIGINT stuff) so
some further digging in the stdlib code is likely coming.

Pertains to #107 and #98
2020-10-02 12:13:50 -04:00
Tyler Goodlet 07beec59bf Hopefully fix datum offset on signal sub-plots
Added a comment to clarify, ish.
Add `ChartPlotWidget._overlays` as registry of curves added on top of
main graphics. Hackishly (ad-hoc-ishly?) update the curve assuming the
data resides in the same `._array` for now (which it does for historical
vwap).
2020-10-02 12:13:28 -04:00
Tyler Goodlet 2cc2b32805 Fix dbz with `np.divide()` 2020-10-02 12:13:28 -04:00
Tyler Goodlet eb5d64ceef Add support for overlay curves and fixed y-range
Allow passing a fixed ylow, yhigh tuple to `._set_yrange()` which avoids
recomputing the range from data if desired (eg. rsi-like bounded
signals). Add support for overlay curves to the OHLC chart and add basic
support to brokers which provide a historical 'vwap`. The data array
increment logic had to be tweaked to copy the vwap from the last bar.

Oh, and hack the subchart curves with two extra prepended datums to make
them align "better" with the ohlc main chart; need to talk to
`pyqtgraph` core about how to do this more correctly.
2020-10-02 12:13:28 -04:00
Tyler Goodlet e91ba55d68 Always draw any history bars on update 2020-10-02 12:13:28 -04:00
Tyler Goodlet a4a5bff3fa Yes, even more grays 2020-10-02 12:13:28 -04:00
Tyler Goodlet da2325239c Copy non-base dtype fields on bar increment 2020-10-02 12:13:28 -04:00
Tyler Goodlet 80f191c57d Adjust range logic to avoid overlap with labels
By mapping any in view "contents labels" to the range of the
``ViewBox``'s data we can avoid having graphics overlap with labels.
Take this approach instead of specifying a min y-range using the std
and activate the range compute on resize and mouser scrolling.
Also, add y-sticky update for signal plots.
2020-10-02 12:13:28 -04:00
Tyler Goodlet fc0a03d597 Handle OHLC *and* signal indexing 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9a59f2408d Start fsp subpackage, separate momo stuff 2020-10-02 12:13:28 -04:00
Tyler Goodlet 7d24e8eeb0 First draft of real-time rsi using numba 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9d8a867767 Bar graphics update speed improvements
Use two separate `QPicture` instances:
- one for the 3 lines for the last bar
- one for all the historical bars lines

On price changes update the last bar and only update historical bars
when the current bar's period expires (when a new bar is "added").
Add a flag `just_history` for this  `BarItems.draw_lines()`.
Also, switch the internal lines array/buffer to a 2D numpy array to avoid
the type-cast step and instead just flatten using `numpy.ravel()`.

Overall this should avoid the problem of draws getting slower over time
as new bars are added to the history since price updates only redraw
a single bar to the "last" `QPicture` instance. Ideally in the future we
can make the `history` `QPicture` a `QPixmap` but it looks like this
will require some internal work in `pyqtgraph` to support it.
2020-10-02 12:13:28 -04:00
Tyler Goodlet b1591e3ee1 Start mucking with faster bars updates
Use a ``rec2array`` struct array converter to generate lines sequence
faster. Start looking into using a `QPixmap` to avoid redrawing all
bars every update.
2020-10-02 12:13:28 -04:00
Tyler Goodlet 0e513599eb Info log fsp output for now 2020-10-02 12:13:28 -04:00
Tyler Goodlet f1b72dfd6b Better bg color, tweak margins. 2020-10-02 12:13:28 -04:00
Tyler Goodlet 17d205f773 Add proper x-axis time-stamping 2020-10-02 12:13:28 -04:00
Tyler Goodlet ea2a675adf Use dashed crosshair, simplify x-axis alloc 2020-10-02 12:13:28 -04:00
Tyler Goodlet 387a696232 Even more colors 2020-10-02 12:13:28 -04:00
Tyler Goodlet fca6257152 Use dashed lines for crosshair 2020-10-02 12:13:28 -04:00
Tyler Goodlet 7a245ddda4 Add and update y-sticky labels on new price data 2020-10-02 12:13:28 -04:00
Tyler Goodlet 363d4cf609 Start color map 2020-10-02 12:13:28 -04:00
Tyler Goodlet d7466a58b4 Add updateable y-sticky label 2020-10-02 12:13:28 -04:00
Tyler Goodlet 8d29338174 Cleanup latency tracker 2020-10-02 12:13:28 -04:00
Tyler Goodlet 58b2e7e395 Refer to main chart's data for date axis 2020-10-02 12:13:28 -04:00
Tyler Goodlet f46fa99a6e Add "contents" labels to charts
Add a default "contents label" (eg. OHLC values for bar charts) to each
chart and update on crosshair interaction.

Few technical changes to make this happen:
- adjust bar graphics to have the HL line be in the "middle" of the
  underlying arrays' "index range" in the containing view.
- add a label dict each chart's graphics name to a label + update routine
- use symbol names instead of this "main" identifier crap for referring to
  particular price curves/graphics
2020-10-02 12:13:28 -04:00
Tyler Goodlet 61e460a422 Start brokers.api module 2020-10-02 12:13:28 -04:00
Tyler Goodlet fd21f4b0fe WIP initial draft of FSP subsystem
This is a first attempt at a financial signal processing subsystem which
utilizes async generators for streaming frames of numpy array data
between actors. In this initial attempt the focus is on processing price
data and relaying it to the chart app for real-time display. So far this
seems to work (with decent latency) but much more work is likely needed
around improving the data model for even better latency and less data
duplication.

Surprisingly (or not?) a lot of simplifications to the charting code
came out of this in terms of conducting graphics updates in streaming
tasks instead of hiding them inside the obfuscated mess that is the
Qt-style-inheritance-OO-90s-trash. The goal from here on wards will be
to enforce strict semantics around reading and writing of data such that
state is kept outside "object trees" as much as possible and streaming
function semantics guide our flow model. Unsurprisingly, this reduction
in "instance state" is happening wherever we use `trio` ;)

A little summary on the technical changes:
- not going to explain the fsp system yet; it's too nascent and
  probably going to get some heavy editing.
- drop any "update" methods from the `LinkedCharts` type since each
  sub-chart will have it's own update task and thus a separate update
  loop; further individual graphics (per chart) may eventually require
  this same design.
- delete `ChartView`; moved into separate mod.
- add "stream from fsp" task to start our foray into real-time actor
  processed numpy streaming.
2020-10-02 12:13:28 -04:00
Tyler Goodlet edb32e8c2b Drop weird chart type enum 2020-10-02 12:13:28 -04:00
Tyler Goodlet 04e21a96da Use partial, pass kwargs to `tractor._main()` 2020-10-02 12:13:28 -04:00
Tyler Goodlet a29b7d9be5 Start "interaction" module 2020-10-02 12:13:28 -04:00
Tyler Goodlet 8a46f8d6ed Port monitor to normalized streams 2020-10-02 12:13:28 -04:00
Tyler Goodlet 241b2374e8 Port `DataFeed` api to broker specific normalizer routine 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9bbf0e0d7a Add a normalizer routine which emits quote differentials/ticks 2020-10-02 12:13:28 -04:00
Tyler Goodlet 81fb327fe1 Add `services` cmd for monitoring actors 2020-10-02 12:13:28 -04:00
Tyler Goodlet 65fb92eaff Flatten out chart tasks 2020-10-02 12:13:28 -04:00
Tyler Goodlet 6b572eb0ef Add ravel() reference link 2020-10-02 12:13:28 -04:00
Tyler Goodlet 971b871647 Handle "mouse-not-on-plot" edge cases 2020-10-02 12:13:28 -04:00
Tyler Goodlet b2506b04f6 Attempt more reliable chart startup
Wait for a first actual real-time quote before starting graphics update
tasks. Use the new normalized tick format brokers are expected to emit
as a `quotes['ticks']` list. Auto detect time frame from historical
bars.
2020-10-02 12:13:28 -04:00
Tyler Goodlet d81f6620f5 Passthrough loglevel from qtractor 2020-10-02 12:13:28 -04:00
Tyler Goodlet 2eea946e5b Drop forkserver usage.
We've got the sweet and realable `trio` spawner now :)
2020-10-02 12:13:28 -04:00
Tyler Goodlet 1a143f6b16 Pass piker log level through to tractor for chart app 2020-10-02 12:13:28 -04:00
Tyler Goodlet dc919fa676 Set tractor loglevel in cli config 2020-10-02 12:13:28 -04:00