Commit Graph

3176 Commits (efbb8e86d47de3465a0848b378c72e0279f7c194)

Author SHA1 Message Date
Tyler Goodlet 2302e59f1d 3k bars for now, ignore rtTime 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8d9a6845c6 Add a naive maxmin calc to avoid frequent resizes
If we know the max and min in view then on datum updates we can avoid
resizing the y-range when a new max/min has not yet arrived.

This adds a very naive numpy calc in the drawing thread which we can
likely improve with a more efficient streaming alternative which can
also likely be run in a fsp subactor. Also, since this same calc is
essentially done inside `._set_yrange()` we will likely want to allow
passing the result into the method to avoid duplicate work.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 6d5ccc6c3f Specify font size in pixels 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8276b02f92 Further label and axis sizing tweaks for hidpi 2020-10-02 12:13:50 -04:00
Tyler Goodlet e524ee9045 Resize everything with HiDPI scaling on 2020-10-02 12:13:50 -04:00
Tyler Goodlet bceeaa56ff Drop ringbuf, didn't end up using 2020-10-02 12:13:50 -04:00
Tyler Goodlet acc8dd66f5 Add data._normalize.py ... 2020-10-02 12:13:50 -04:00
Tyler Goodlet bc65040601 Add shm support to kraken backend 2020-10-02 12:13:50 -04:00
Tyler Goodlet d4eb5ccca4 Handle vwap overlay with shm 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8a4528c006 Always ask backend for ohlc dtype 2020-10-02 12:13:50 -04:00
Tyler Goodlet 47d4ec5985 Move _source under data package 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8e2e695ba8 Revert project name removal; breaks shit elsewhere somehow.. 2020-10-02 12:13:50 -04:00
Tyler Goodlet 155c3eef2a Convert timeit thing to use print() 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8832804bab Sub each new symbol to shm incrementing 2020-10-02 12:13:50 -04:00
Tyler Goodlet e3e219aa4b Add multi-symbol-buffer increment support 2020-10-02 12:13:50 -04:00
Tyler Goodlet 41e85ccaa9 Break wma calc into sync func 2020-10-02 12:13:50 -04:00
Tyler Goodlet 3f0e175011 Get bar oriented RSI working correctly 2020-10-02 12:13:50 -04:00
Tyler Goodlet 268e748417 Drop extra prefix in logs 2020-10-02 12:13:50 -04:00
Tyler Goodlet bfa7839370 Adopt official color 2020-10-02 12:13:50 -04:00
Tyler Goodlet 2fcbefa6e1 Use shm in fsp cascading
This kicks off what will be the beginning of hopefully a very nice
(soft) real-time financial signal processing system. We're keeping the
hack to "time align" curves (for now) with the bars for now by slapping
in an extra datum at index 0.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 4383579cd0 Use shm array in chart-fsp task
Just like for the source OHLC, we now have the chart parent actor create
an fsp shm array and use it to read back signal data for plotting.
Some tweaks to get the price chart (and sub-charts) to load historical
datums immediately instead of waiting on an initial quote.
2020-10-02 12:13:50 -04:00
Tyler Goodlet ba4261f974 Add timeit prints 2020-10-02 12:13:50 -04:00
Tyler Goodlet 561cafbe55 Another black 2020-10-02 12:13:50 -04:00
Tyler Goodlet efb52f2292 Make shared array buffer incrementer a message pub
Drop ctx manager api and use `tractor.msg.pub`.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 373ff90229 Only need UTC offset hacking if time w broker is messed.. 2020-10-02 12:13:50 -04:00
Tyler Goodlet d93ce84a99 Variety of IB backend improvements
- Move to new shared mem system only writing on the first (by process)
  entry to `stream_quotes()`.
- Deliver bars before first quote arrives so that chart can populate and
  then wait for initial arrival.
- Allow caching clients per actor.
- Load bars using the same (cached) client that starts the quote stream
  thus speeding up initialization.
2020-10-02 12:13:50 -04:00
Tyler Goodlet b1093dc71d Add a `data.Feed` type
Wraps the growing tuple of items being delivered by `open_feed()`.
Add lazy loading of the broker's signal step stream with
a `Feed.index_stream()` method.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 38469bd6ef Slight rework: shm API
Add an internal `_Token` to do interchange (un)packing for passing
"references" to shm blocks between actors.  Part of the token involves
providing the `numpy.dtype` in a cross-actor format.  Add a module
variable for caching "known tokens" per actor.  Drop use of context
managers since they tear down shm blocks too soon in debug mode and
there seems to be no reason to unlink/close shm before the process has
terminated; if code needs it torn down explicitly, it can.
2020-10-02 12:13:50 -04:00
Tyler Goodlet cd540fd07e Cleanups 2020-10-02 12:13:50 -04:00
Tyler Goodlet 6fa4f6e943 Port charts to new shm arrays 2020-10-02 12:13:50 -04:00
Tyler Goodlet f872fbecf8 Hook IB up to shared memory system
Adjust the `data.open_feed()` api to take a shm token so the
broker-daemon can attach a previously created (by the parent actor) mem
buf and push real-time tick data. There's still some sloppiness here in
terms of ensuring only one mem buf per symbol (can be seen in
`stream_quotes()`) which should really managed at the data api level.
Add a bar incrementing stream-task which delivers increment msgs to any
consumers.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 17491ba819 Disconnect stdlib's resource_tracker, fix .push()
Logic in `SharedArray.push()` was totally wrong.
Remove all the `multiprocessing.resource_tracker` crap such that we
aren't loading an extra process at every layer and we don't get tons of
errors when cleaning on in an SC way.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 712e36b9d5 First draft of a shared numpy array sub-system
This adds a shared memory "incrementing array" sub-sys interface
for single writer, multi-reader style data passing. The main motivation
is to avoid multiple copies of the same `numpy` array across actors
(plus now we can start being fancy like ray).

There still seems to be some odd issues with the "resource tracker"
complaining at teardown (likely partially to do with SIGINT stuff) so
some further digging in the stdlib code is likely coming.

Pertains to #107 and #98
2020-10-02 12:13:50 -04:00
Tyler Goodlet 2f1e9ee760 Update dev deps to current state of things.. 2020-10-02 12:13:49 -04:00
Tyler Goodlet 07beec59bf Hopefully fix datum offset on signal sub-plots
Added a comment to clarify, ish.
Add `ChartPlotWidget._overlays` as registry of curves added on top of
main graphics. Hackishly (ad-hoc-ishly?) update the curve assuming the
data resides in the same `._array` for now (which it does for historical
vwap).
2020-10-02 12:13:28 -04:00
Tyler Goodlet 2cc2b32805 Fix dbz with `np.divide()` 2020-10-02 12:13:28 -04:00
Tyler Goodlet eb5d64ceef Add support for overlay curves and fixed y-range
Allow passing a fixed ylow, yhigh tuple to `._set_yrange()` which avoids
recomputing the range from data if desired (eg. rsi-like bounded
signals). Add support for overlay curves to the OHLC chart and add basic
support to brokers which provide a historical 'vwap`. The data array
increment logic had to be tweaked to copy the vwap from the last bar.

Oh, and hack the subchart curves with two extra prepended datums to make
them align "better" with the ohlc main chart; need to talk to
`pyqtgraph` core about how to do this more correctly.
2020-10-02 12:13:28 -04:00
Tyler Goodlet e91ba55d68 Always draw any history bars on update 2020-10-02 12:13:28 -04:00
Tyler Goodlet a4a5bff3fa Yes, even more grays 2020-10-02 12:13:28 -04:00
Tyler Goodlet da2325239c Copy non-base dtype fields on bar increment 2020-10-02 12:13:28 -04:00
Tyler Goodlet 80f191c57d Adjust range logic to avoid overlap with labels
By mapping any in view "contents labels" to the range of the
``ViewBox``'s data we can avoid having graphics overlap with labels.
Take this approach instead of specifying a min y-range using the std
and activate the range compute on resize and mouser scrolling.
Also, add y-sticky update for signal plots.
2020-10-02 12:13:28 -04:00
Tyler Goodlet fc0a03d597 Handle OHLC *and* signal indexing 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9a59f2408d Start fsp subpackage, separate momo stuff 2020-10-02 12:13:28 -04:00
Tyler Goodlet ee6e4d2207 Add colorama for numba tracebacks 2020-10-02 12:13:28 -04:00
Tyler Goodlet 7d24e8eeb0 First draft of real-time rsi using numba 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9d8a867767 Bar graphics update speed improvements
Use two separate `QPicture` instances:
- one for the 3 lines for the last bar
- one for all the historical bars lines

On price changes update the last bar and only update historical bars
when the current bar's period expires (when a new bar is "added").
Add a flag `just_history` for this  `BarItems.draw_lines()`.
Also, switch the internal lines array/buffer to a 2D numpy array to avoid
the type-cast step and instead just flatten using `numpy.ravel()`.

Overall this should avoid the problem of draws getting slower over time
as new bars are added to the history since price updates only redraw
a single bar to the "last" `QPicture` instance. Ideally in the future we
can make the `history` `QPicture` a `QPixmap` but it looks like this
will require some internal work in `pyqtgraph` to support it.
2020-10-02 12:13:28 -04:00
Tyler Goodlet b1591e3ee1 Start mucking with faster bars updates
Use a ``rec2array`` struct array converter to generate lines sequence
faster. Start looking into using a `QPixmap` to avoid redrawing all
bars every update.
2020-10-02 12:13:28 -04:00
Tyler Goodlet aed310dd8b Kill pipenv 2020-10-02 12:13:28 -04:00
Tyler Goodlet 48c70e8ae4 better readme 2020-10-02 12:13:28 -04:00
Tyler Goodlet 0e513599eb Info log fsp output for now 2020-10-02 12:13:28 -04:00