Commit Graph

835 Commits (4774881812c1cc6baa3db409cacd722147bc13e3)

Author SHA1 Message Date
Tyler Goodlet dba8457be9 Trigger kraken sample increment after startup 2021-01-07 12:17:10 -05:00
Tyler Goodlet 282cc85ba0 Create an "order mode"
Our first major UI "mode" (yes kinda like the modes in emacs) that has
handles to a client side order book api, line and arrow editors, and
interacts with a spawned `emsd` (the EMS daemon actor).

Buncha cleaning and fixes in here for various thingers as well.
2021-01-07 12:03:18 -05:00
Tyler Goodlet 8d66a17daf Add a "lines editor" api/component 2021-01-05 13:37:03 -05:00
Tyler Goodlet 268f207a6c Shadow 'close' field with 'last' 2021-01-05 08:02:48 -05:00
Tyler Goodlet 24536ad769 Get roundtrip alert uuids workin; stage order book api 2021-01-05 08:02:14 -05:00
Tyler Goodlet d0a3deae09 Pass symbol type further down chart stack 2021-01-04 14:46:47 -05:00
Tyler Goodlet 4d6b1d4bb1 Add brokers list field to symbol type 2021-01-04 14:45:59 -05:00
Tyler Goodlet 3c424a153f Port to new `Portal.run()` api 2021-01-04 14:45:34 -05:00
Tyler Goodlet 267c8c6bd3 Go back to manual label contents check for now 2021-01-04 14:45:12 -05:00
Tyler Goodlet 9478adf600 Use event sync for increment task launch 2021-01-04 14:44:39 -05:00
Tyler Goodlet c7ff0804db Graph snap quote immediately on ib spin up 2021-01-04 14:43:59 -05:00
Tyler Goodlet c030b63101 Rejig state with dataclasses; prep for numba 2021-01-04 14:42:35 -05:00
Tyler Goodlet 7b5a72909e Start a "real" alert looking msg 2021-01-03 17:23:45 -05:00
Tyler Goodlet 616a74865b Better highlighted yellow 2021-01-03 17:23:23 -05:00
Tyler Goodlet 6e2328d351 More comments? 2021-01-03 17:23:06 -05:00
Tyler Goodlet a3468fb915 Barebones level based alerts are working! 2021-01-03 17:19:16 -05:00
Tyler Goodlet 97b2f86cfe Add preliminary support for alert line management 2021-01-03 11:12:51 -05:00
Tyler Goodlet 88d48bd188 Port to `Cursor` 2021-01-03 11:10:08 -05:00
Tyler Goodlet 08aa5984ec Rename crosshair (type) to cursor
Since the "crosshair" is growing more and more UX implementation details
it probably makes sense to call it what it is; a python level mouse
abstraction. Add 2 internal sets: `_hovered` for allowing mouse hovered
objects to register themselves to other cursor aware components, and
`_trackers` for allowing scene items to "track" cursor movements via
a `on_tracked_source()` callback.
2021-01-03 10:46:28 -05:00
Tyler Goodlet 14dad08d98 Extend level line interaction
Support tracking the mouse cursor using a new `on_tracked_sources()`
callback method. Make hovered highlight a bit thicker and highlight when
click-dragged. Add a delete method for removing from the scene along
with label.
2021-01-03 10:43:53 -05:00
Tyler Goodlet edbbcbf954 Add alert yellow 2021-01-03 10:39:06 -05:00
Tyler Goodlet d492f5c35a Define our own mouse clicked handler 2021-01-01 17:49:23 -05:00
Tyler Goodlet a55d72f8d6 Create and "EMS" module for order execution/routing actor(s) 2021-01-01 17:49:02 -05:00
Tyler Goodlet 80d48e5ece Dynamically override .boundingRect() after startup 2021-01-01 13:23:05 -05:00
Tyler Goodlet e474d8c309 Remove horizontal crosshair on "a" hotkey 2021-01-01 13:05:50 -05:00
Tyler Goodlet 7a53f19eeb Highlight level line label on hover 2021-01-01 13:05:16 -05:00
Tyler Goodlet d602cbc985 Avoid active_plot race 2021-01-01 12:59:31 -05:00
Tyler Goodlet bd85214017 Add draft order actor architecture 2021-01-01 12:40:05 -05:00
Tyler Goodlet 5fddb581ab Kill backfiller on "no history" error 2021-01-01 12:34:50 -05:00
Tyler Goodlet 81b10b9dfc Add initial y-axis zoom 2020-12-30 12:55:02 -05:00
Tyler Goodlet f3a0b1e91e Use pixel buffers on axis labels 2020-12-29 18:05:36 -05:00
Tyler Goodlet 1e09b0f08f Type annot paint methods 2020-12-29 18:05:36 -05:00
Tyler Goodlet 98c3bb8aee Add a pyqtgraph profiling toggle to chart CLI 2020-12-29 18:05:36 -05:00
Tyler Goodlet a68fff4139 Break specialized graphics into specific mods 2020-12-29 18:05:36 -05:00
Tyler Goodlet 575b3a0605 Adjust imports 2020-12-29 18:05:36 -05:00
Tyler Goodlet 916396fbfe Move fast curve graphics to new mod 2020-12-29 18:05:36 -05:00
Tyler Goodlet 19e3db8f61 Move OHLC items into new module 2020-12-29 18:05:36 -05:00
Tyler Goodlet b4c28cb086 Start a new graphics sub-package 2020-12-29 18:05:36 -05:00
Tyler Goodlet 5d30f351af Remove use of `tractor.run()` 2020-12-29 18:05:36 -05:00
Tyler Goodlet d7f80d8ddb Add note about shm limit 2020-12-29 18:05:36 -05:00
Tyler Goodlet b467e8fd8d Try cache mode on a bunch of "static" graphics 2020-12-29 18:05:36 -05:00
Tyler Goodlet 588213a230 Add an experimental "fast appendable curve" graphic 2020-12-29 18:05:36 -05:00
Tyler Goodlet 93e76fa12c Comment some unneeded PDI optimization flags 2020-12-29 18:05:36 -05:00
Tyler Goodlet cac797a7fc Add initial pixel buffer caching usage
Leverages `QGraphicsItem.cacheMode` to speed up interactivity via
less `.paint()` calls (on mouse interaction) and redraws of the
underlying path when there are no transformations (other then a shift).
In order to keep the "flat bar on new time period" UX, a couple special
methods have to be triggered to get a redraw of the pixel buffer when
appending new data.

Use `QPainterPath.controlPointRect()` over `.boundingRect()` since
supposedly it's a lot faster. Drop all use of `QPicture` (since it seems
to conflict with the pixel buffer stuff?) and it doesn't give any
measurable speedup when drawing the "last bar" lines.

Oh, and add some profiling for now.
2020-12-29 18:05:36 -05:00
Tyler Goodlet bd73d02cfb Avoid loading volume FSPs on symbols without any data 2020-12-28 14:43:17 -05:00
Tyler Goodlet 42761438f6 Pass backfiller explicit symbol str, don't require volume ticks 2020-12-26 18:04:32 -05:00
Tyler Goodlet bc96c5847c Port to new tractor api 2020-12-21 13:02:43 -05:00
Tyler Goodlet 714c757e3e Add vwap back to fsp conf on feature branch 2020-12-21 13:02:43 -05:00
Tyler Goodlet 48da729364 Use .shield() meth name from tractor 2020-12-21 13:02:43 -05:00
Tyler Goodlet c01382294e Add signal backfilling via trio task respawn 2020-12-21 13:02:43 -05:00
Tyler Goodlet e89d3f9560 Use `numpy.divide()` to avoid divide-by-zero 2020-12-21 13:02:43 -05:00
Tyler Goodlet 7c7b31ebbe Break hist calc into wap func, use hlc3. 2020-12-21 13:02:43 -05:00
Tyler Goodlet 33515f45cc Add vwap to exposed fsp map 2020-12-21 13:02:43 -05:00
Tyler Goodlet ef2be16d97 Add initial tina (ohl3) vwap fsp 2020-12-21 13:02:43 -05:00
Tyler Goodlet 833a442ac0 Disable vwap in conf, feature delay 2020-12-19 16:57:58 -05:00
Tyler Goodlet f27d639552 Port kraken to declare "wap" field 2020-12-19 16:53:51 -05:00
Tyler Goodlet b72d7d3085 Drop profile calls on OHLC bars for now 2020-12-19 16:52:04 -05:00
Tyler Goodlet df686755da Drop legacy "historical" QPicture cruft 2020-12-19 16:51:11 -05:00
Tyler Goodlet 618c2f8e0a More general salutation 2020-12-19 16:51:03 -05:00
Tyler Goodlet db6f77481b Stick with slightly smaller fonts 2020-12-19 16:50:54 -05:00
Tyler Goodlet 7ce27107df Fix axes for shm primary indexing 2020-12-19 16:50:49 -05:00
Tyler Goodlet c8537d59a8 Port charting to new shm primary indexing 2020-12-19 16:50:38 -05:00
Tyler Goodlet 9930430721 Close app on last window exit
Use a system triggered SIGINT on app close to tear down the streaming
stack and terminate the `trio`/`tractor` runtimes deterministically.
2020-12-19 16:50:29 -05:00
Tyler Goodlet 599b5276b4 Port data apis to not touch primary index 2020-12-19 16:50:20 -05:00
Tyler Goodlet fda9fcbc55 Add historical backfilling to ib backend 2020-12-19 16:50:08 -05:00
Tyler Goodlet 6d50ad75a7 Ensure right bar x index is an int 2020-12-19 16:49:57 -05:00
Tyler Goodlet 2568a2d2e9 First draft, make graphics work on shm primary index
This is a bit hacky (what with array indexing semantics being relative
to the primary index's "start" value but it works. We'll likely want
to somehow wrap this index finagling into an API soon.
2020-12-19 16:49:43 -05:00
Tyler Goodlet 02b7d6cd19 Add prepend support to shm system 2020-12-19 16:49:35 -05:00
Tyler Goodlet 9710685508 Left align yaxis label 2020-12-19 16:49:27 -05:00
Tyler Goodlet 1f8f2eb8b3 Font size tweaks for low dpi 2020-12-19 16:06:54 -05:00
Tyler Goodlet 247b5fa2ec Tidy up doc string 2020-12-19 16:06:28 -05:00
Tyler Goodlet 3e16840566 Attempt to add numba typing and use `QGraphicsPathItem`
Failed at using either.

Quirks in numba's typing require specifying readonly arrays by
composing types manually.

The graphics item path thing, while it does take less time to write on
bar appends, seems to be slower in general in calculating the
``.boundingRect()`` value. Likely we'll just add manual max/min tracking
on array updates like ``pg.PlotCurveItem`` to squeeze some final juices
on this.
2020-12-19 16:06:22 -05:00
Tyler Goodlet 949e9d6cd1 Drop commented pixmap cruft
See #124 as to why we'll probably never need this.
2020-12-19 16:06:07 -05:00
Tyler Goodlet f083f537b1 Get `QPainterPath` "append" working
Pertains further to #109.

Instead of redrawing the entire `QPainterPath` every time there's
a historical bars update just use `.addPath()` to slap in latest
history. It seems to work and is fast. This also seems like it will be
a great strategy for filling in earlier data, woot!
2020-12-19 16:05:59 -05:00
Tyler Goodlet 413c703e34 Draw bars using `QPainterPath` magic
This gives a massive speedup when viewing large bar sets (think a day's
worth of 5s bars) by using the `pg.functions.arrayToQPath()` "magic"
binary array writing that is also used in `PlotCurveItem`.  We're using
this same (lower level) function directly to draw bars as part of one
large path and it seems to be painting 15k (ish) bars with around 3ms
`.paint()` latency. The only thing still a bit slow is the path array
generation despite doing it with `numba`. Likely, either having multiple
paths or, only regenerating the missing backing array elements should
speed this up further to avoid slight delays when incrementing the bar
step.

This is of course a first draft and more cleanups are coming.
2020-12-19 16:05:53 -05:00
Tyler Goodlet 8aede3cbcb Add field diffing on failed push 2020-12-19 16:05:22 -05:00
Tyler Goodlet f9e4e9507d Tweak axis text offset and margins 2020-12-19 15:05:37 -05:00
Tyler Goodlet c625dc90f1 Use new global var stack from tractor 2020-12-19 15:05:29 -05:00
Tyler Goodlet 9572280553 Kill the tractor tree on window close.
This makes it so you don't have to ctrl-c kill apps.
Add in the experimental openGL support even though I'm pretty sure it's
not being used much for curve plotting (but could be wrong).
2020-12-19 15:05:21 -05:00
Tyler Goodlet bfcf5170dc Add commented ex. code for line price charts 2020-12-19 15:05:14 -05:00
Tyler Goodlet acf8aeb33e Allocate space for 2d worth of 5s bars 2020-12-19 15:05:03 -05:00
Tyler Goodlet daa429f7ca Put fsp plotting into a couple tasks, startup speedups.
Break the chart update code for fsps into a new task (add a nursery) in
new `spawn_fsps` (was `chart_from_fsps`) that async requests actor
spawning and initial historical data (all CPU bound work).  For multiple
fsp subcharts this allows processing initial output in parallel
(multi-core). We might want to wrap this in a "feed" like api
eventually. Basically the fsp startup sequence is now:
- start all requested fsp actors in an async loop and wait for
  historical data to arrive
- loop through them all again to start update tasks which do chart
  graphics rendering

Add separate x-axis objects for each new subchart (required by
pyqtgraph); still need to fix hiding unnecessary ones.
Add a `ChartPlotWidget._arrays: dict` for holding overlay data distinct
from ohlc. Drop the sizing yrange to label heights for now since it's
pretty much all gone to hell since adding L1 labels. Fix y-stickies to
look up correct overly arrays.
2020-12-19 15:04:51 -05:00
Tyler Goodlet 49699a16ff Skip base metrics on no basis 2020-12-17 10:50:05 -05:00
Tyler Goodlet af61611801 Add initial selection box interaction
Requires decent modification of the built-in ``ViewBox``.
We do away with the zoom functionality for now and instead just add
a label full of some simple stats on the bounded data.
2020-11-08 16:15:34 -05:00
Tyler Goodlet 1972740d0c Make salotz stop grumbling like an old man 2020-11-06 15:35:10 -05:00
Tyler Goodlet 0f458f8263 Add min tick setting to axis type 2020-11-06 15:24:01 -05:00
Tyler Goodlet c1109ee3fb Add license headers to pertinent files 2020-11-06 12:23:14 -05:00
Tyler Goodlet be4a3df7ba Add L1 spread streaming to kraken 2020-11-06 11:35:40 -05:00
Tyler Goodlet 043bc985df Configure L1 queue size precisions from history 2020-11-06 11:35:10 -05:00
Tyler Goodlet e27fece4e6 Add L1 queue size precision controls 2020-11-06 11:34:32 -05:00
Tyler Goodlet 205bedce85 Spec dpi aware font size in inches 2020-11-05 20:32:35 -05:00
Tyler Goodlet 1e491fb1bb Use pyqtgraph default pen for level lines 2020-11-05 13:23:29 -05:00
Tyler Goodlet cc88300ac5 Fix L1 updates to be like TWS
I think this gets us to the same output as TWS both on booktrader and
the quote details pane. In theory there might be logic needed to
decreases an L1 queue size on trades but can't seem to get it without
getting -ves displayed occasionally - thus leaving it for now.

Also, fix the max-min streaming logic to actually do its job, lel.
2020-11-05 12:08:29 -05:00
Tyler Goodlet db075b81ac Specialize `LevelLabel` for orientation-around-axis gymnastics 2020-11-05 12:08:02 -05:00
Tyler Goodlet 9c3850874d Add all L1 tick types for ib 2020-11-05 08:18:55 -05:00
Tyler Goodlet f438139ad7 Update L1 labels in price loop 2020-11-03 22:03:49 -05:00
Tyler Goodlet 73e54a2259 Add L1 labels wrapper type
Start a simple API for L1 bid/ask labels.
Make `LevelLabel` draw a line above/below it's text (instead of the
rect fill we had before) since it looks much simpler/slicker.
Generalize the label text orientation through bounding rect
geometry positioning.
2020-11-03 19:48:27 -05:00
Tyler Goodlet 987c13c584 Classify L1 tick types 2020-11-03 16:22:04 -05:00
Tyler Goodlet a5d5208cfa Add sticky "orientation", allow overriding label draw instructions. 2020-11-03 16:21:21 -05:00
Tyler Goodlet e65f511648 Draw flat line on every new time step
Until we get a better datum "cursor" figured out just draw the flat bar
despite the extra overhead. The reason to do this in 2 separate calls is
detailed in the comment but basic gist is that there's a race between
writer and reader of the last shm index.

Oh, and toss in some draft symbol search label code.
2020-11-03 12:25:08 -05:00
Tyler Goodlet d92e02db86 Add back min window size i guess 2020-11-03 12:22:57 -05:00
Tyler Goodlet 22f1b56b36 Always update left (open) arm 2020-11-03 08:15:00 -05:00
Tyler Goodlet 1640906b09 Write shm volume (facepalm), update open=close on first volume 2020-11-03 08:14:37 -05:00
Tyler Goodlet b23e459027 Cleanup unneeded commented stuff 2020-11-02 15:43:19 -05:00
Tyler Goodlet 119196f2ff Scale level label correctly to small(er) fonts
Not sure what fixed it exactly, and I guess we didn't need any relative
DPI scaling factor after all. Using the 3px margin on the level label
seems to make it look nice for any font size (i think) as well.

Gonna need some cleanup after this one.
2020-11-02 15:27:48 -05:00
Tyler Goodlet 96f700a762 Add level line type with custom label 2020-11-02 13:33:19 -05:00
Tyler Goodlet 9e7aa3f9bf Add a dpi aware font wrapper type 2020-11-02 12:02:05 -05:00
Tyler Goodlet 307c501763 Add symbol field to ib quotes 2020-10-29 17:21:41 -04:00
Tyler Goodlet da0789e184 Add symbol field to kraken quotes 2020-10-29 17:08:25 -04:00
Tyler Goodlet 416f027c5f Flip contents label stuff into a type 2020-10-29 17:08:03 -04:00
Tyler Goodlet 68304b79bc More thematic max datums on screen 2020-10-28 09:28:37 -04:00
Tyler Goodlet 031eab28c7 Adjust contents label font for DPI 2020-10-28 08:52:08 -04:00
Tyler Goodlet 20a4aed672 Update font metrics after dpi calc; facepalm. 2020-10-28 08:12:56 -04:00
Tyler Goodlet 751cca35e1 Attempt to calculate font size by DPI 2020-10-27 15:15:31 -04:00
Tyler Goodlet 23672fc22b Rework axes types, sizing stuff
Make our own ``Axis`` and have it call an impl specific ``.resize()``
such that different axes can size to their own spec. Allow passing in a
"typical maximum value string" which will be used by default for sizing
the axis' minor dimension; a common value should be passed to all axes
in a linked split charts widget. Add size hinting for axes labels such
that they can check their parent (axis) for desired dimensions if
needed.
2020-10-27 10:50:28 -04:00
Tyler Goodlet 89d48afb6c Size axis labels based on text contents
Compute the size in pixels the label based on the label's contents.
Eventually we want to have an update system that can iterate through
axes and labels to do this whenever needed (eg. after widget is moved
to a new screen with a different DPI).
2020-10-26 23:34:48 -04:00
Tyler Goodlet 55f34dfed0 Start a profiling mod 2020-10-26 14:08:42 -04:00
Tyler Goodlet bed6a631c0 Don't pass color down to axis 2020-10-25 10:49:31 -04:00
Tyler Goodlet f32763d992 Only move x-axis sticky when we mouse-over a new index
Avoid drawing a new new sticky position if the mouse hasn't moved to the
next (rounded) index in terms of the scene's coordinates. This completes
the "discrete-ization" of the mouse/cursor UX.

Finalizing this feature helped discover and solve
pyqtgraph/pyqtgraph#1418 which masssively improves interaction
performance throughout the whole lib!

Hide stickys on startup until cursor shows up on plot.
2020-10-24 20:18:21 -04:00
Tyler Goodlet ece57b2a1d Rename our main color 2020-10-24 20:18:03 -04:00
Tyler Goodlet 94a8ee6270 Optimize axis labels using `QPicture`
This is likely a marginal improvement but is slightly less execution and
adds a coolio black border around the label. Drop all the legacy code
from quantdom which was quite a convoluted mess for "coloring".
Had to tweak sticky offsets to get the crosshair to line up right; not
sure what that's all about yet.
2020-10-24 20:12:12 -04:00
Tyler Goodlet 13f32acfdf Only update history on bar increment
With the improved update logic on `BarsItems` it doesn't seem to be
necessary. Remove y sticky for overlays for now to avoid clutter that
looks like double draws when the last overlay value is close to the last
price.
2020-10-22 21:21:14 -04:00
Tyler Goodlet 8eb4344d86 More "thematic" default view values ;) 2020-10-22 20:42:46 -04:00
Tyler Goodlet 8c25892521 Fix (really sidestep) flat bar rendering issue(s)
It seems a plethora of problems (including drawing performance) are due
to trying to hack around the strange rendering bug in Qt with `QLineF`
with y1 == y2. There was all sorts of weirdness that would show up with
trying (a hack) to just set all 4 points to the same value including
strange infinite diagonal ghost lines randomly on charts. Instead, just
place hold these flat bar's 'body' line with a `None` and filter the
null values out before calling `QPainter.drawLines()`. This results
in simply no body lines drawn for these datums. We can probably `numba`
the filtering too if it turns out to be a bottleneck.
2020-10-22 20:35:51 -04:00
Tyler Goodlet 7be624de39 Implement `tickStrings` for price axis; use float_digits() 2020-10-22 20:22:21 -04:00
Tyler Goodlet 18dc809acb Add naive digits count routine 2020-10-22 14:05:35 -04:00
Tyler Goodlet f2c4a46c94 Center bars around index, adjust curves back to match... 2020-10-21 11:19:41 -04:00
Tyler Goodlet cd828db9e9 Show 24 bars to left on startup 2020-10-21 10:51:54 -04:00
Tyler Goodlet 875bc8be24 Add line dots cursors to curves by default 2020-10-21 10:46:56 -04:00
Tyler Goodlet 1f6b5da17e Add support for curve "cursors" using a filled dot
Add a new graphic `LineDot` which is a `pg.CurvePoint` that draws
a simple filled dot over a curve at the specified index.

Add support for adding these cursor-dots to the crosshair/mouse through
a new `CrossHair.add_curve_cursor()`. Discretized the vertical line
updates on the crosshair such that it's only drawn in the middle of
the current bar in the main chart.
2020-10-21 10:40:51 -04:00
Tyler Goodlet 88583d999a Add "follow mode"
Makes the chart act like tws where each new time step increment the
chart shifts to the right so that the last bar stays in place. This
gets things looking like a proper auto-trading UX.

Added a couple methods to ``ChartPlotWidget`` to make this work:
- ``.default_view()`` to set the preferred view based on user settings
- ``.increment_view()`` to shift the view one time frame right

Also, split up the `.update_from_array()` method to be curve/ohlc
specific allowing for passing in a struct array with a named field
containing curve data more straightforwardly. This also simplifies the
contest label update functions.
2020-10-20 09:13:27 -04:00
Tyler Goodlet 1902507703 Change scroll "center" to rightmost bar on screen 2020-10-19 21:32:50 -04:00
Tyler Goodlet c57f678295 Fix contents labels issues
Lookup overlay contents from the OHLC struct array (for now / to make
things work) and fix anchoring logic with better offsets to keep
contents labels super tight to the edge of the view box. Unfortunately,
had to hack the label-height-calc thing for avoiding overlap of graphics
with the label; haven't found a better solution yet and pyqtgraph seems
to require more rabbit holing to figure out something better. Slap in
some inf lines for over[sold/bought] rsi conditions thresholding.
2020-10-19 14:18:06 -04:00
Tyler Goodlet 851104dd31 Add an inf horizontal line helper 2020-10-19 14:01:57 -04:00
Tyler Goodlet 1706b67e00 Note the issues with the shared fsp array index.. 2020-10-19 14:01:25 -04:00
Tyler Goodlet 32974a118c Add hidpi comments, 300 bars in view at startup 2020-10-19 11:37:50 -04:00
Tyler Goodlet d3dc8fb219 Differentiate array schema by close field 2020-10-19 11:37:28 -04:00
Tyler Goodlet c7d5ea6e15 Fix static yrange and last bar double draw issues 2020-10-16 12:18:14 -04:00
Tyler Goodlet fc23b2180d Pass through fonts to axes 2020-10-16 12:15:33 -04:00
Tyler Goodlet 58d3234f74 Drop lingering print 2020-10-16 12:15:07 -04:00
Tyler Goodlet f4c38621d5 Add a default "bars from right" style setting 2020-10-15 15:08:56 -04:00
Tyler Goodlet 6f429b1104 These seem to be "faster" crosshair settings? 2020-10-15 15:08:16 -04:00
Tyler Goodlet cb72662350 Add warnings for shm cache misses 2020-10-15 15:07:56 -04:00
Tyler Goodlet 454b445b4b Add better shared mem writer task checking
If you have a common broker feed daemon then likely you don't want to
create superfluous shared mem buffers for the same symbol. This adds an
ad hoc little context manger which keeps a bool state of whether
a buffer writer task currently is running in this process. Before we
were checking the shared array token cache and **not** clearing it when
the writer task exited, resulting in incorrect writer/loader logic on
the next entry..

Really, we need a better set of SC semantics around the shared mem stuff
presuming there's only ever one writer per shared buffer at given time.
Hopefully that will come soon!
2020-10-15 15:02:42 -04:00
Tyler Goodlet db273e1cd7 Up the mouse re-draw rate limit 2020-10-12 09:36:46 -04:00
Tyler Goodlet 18097fc33b Scale for hidpi pixmaps too 2020-10-02 12:13:50 -04:00
Tyler Goodlet 2302e59f1d 3k bars for now, ignore rtTime 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8d9a6845c6 Add a naive maxmin calc to avoid frequent resizes
If we know the max and min in view then on datum updates we can avoid
resizing the y-range when a new max/min has not yet arrived.

This adds a very naive numpy calc in the drawing thread which we can
likely improve with a more efficient streaming alternative which can
also likely be run in a fsp subactor. Also, since this same calc is
essentially done inside `._set_yrange()` we will likely want to allow
passing the result into the method to avoid duplicate work.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 6d5ccc6c3f Specify font size in pixels 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8276b02f92 Further label and axis sizing tweaks for hidpi 2020-10-02 12:13:50 -04:00
Tyler Goodlet e524ee9045 Resize everything with HiDPI scaling on 2020-10-02 12:13:50 -04:00
Tyler Goodlet bceeaa56ff Drop ringbuf, didn't end up using 2020-10-02 12:13:50 -04:00
Tyler Goodlet acc8dd66f5 Add data._normalize.py ... 2020-10-02 12:13:50 -04:00
Tyler Goodlet bc65040601 Add shm support to kraken backend 2020-10-02 12:13:50 -04:00
Tyler Goodlet d4eb5ccca4 Handle vwap overlay with shm 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8a4528c006 Always ask backend for ohlc dtype 2020-10-02 12:13:50 -04:00
Tyler Goodlet 47d4ec5985 Move _source under data package 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8e2e695ba8 Revert project name removal; breaks shit elsewhere somehow.. 2020-10-02 12:13:50 -04:00
Tyler Goodlet 155c3eef2a Convert timeit thing to use print() 2020-10-02 12:13:50 -04:00
Tyler Goodlet 8832804bab Sub each new symbol to shm incrementing 2020-10-02 12:13:50 -04:00
Tyler Goodlet e3e219aa4b Add multi-symbol-buffer increment support 2020-10-02 12:13:50 -04:00
Tyler Goodlet 41e85ccaa9 Break wma calc into sync func 2020-10-02 12:13:50 -04:00
Tyler Goodlet 3f0e175011 Get bar oriented RSI working correctly 2020-10-02 12:13:50 -04:00
Tyler Goodlet 268e748417 Drop extra prefix in logs 2020-10-02 12:13:50 -04:00
Tyler Goodlet bfa7839370 Adopt official color 2020-10-02 12:13:50 -04:00
Tyler Goodlet 2fcbefa6e1 Use shm in fsp cascading
This kicks off what will be the beginning of hopefully a very nice
(soft) real-time financial signal processing system. We're keeping the
hack to "time align" curves (for now) with the bars for now by slapping
in an extra datum at index 0.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 4383579cd0 Use shm array in chart-fsp task
Just like for the source OHLC, we now have the chart parent actor create
an fsp shm array and use it to read back signal data for plotting.
Some tweaks to get the price chart (and sub-charts) to load historical
datums immediately instead of waiting on an initial quote.
2020-10-02 12:13:50 -04:00
Tyler Goodlet ba4261f974 Add timeit prints 2020-10-02 12:13:50 -04:00
Tyler Goodlet 561cafbe55 Another black 2020-10-02 12:13:50 -04:00
Tyler Goodlet efb52f2292 Make shared array buffer incrementer a message pub
Drop ctx manager api and use `tractor.msg.pub`.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 373ff90229 Only need UTC offset hacking if time w broker is messed.. 2020-10-02 12:13:50 -04:00
Tyler Goodlet d93ce84a99 Variety of IB backend improvements
- Move to new shared mem system only writing on the first (by process)
  entry to `stream_quotes()`.
- Deliver bars before first quote arrives so that chart can populate and
  then wait for initial arrival.
- Allow caching clients per actor.
- Load bars using the same (cached) client that starts the quote stream
  thus speeding up initialization.
2020-10-02 12:13:50 -04:00
Tyler Goodlet b1093dc71d Add a `data.Feed` type
Wraps the growing tuple of items being delivered by `open_feed()`.
Add lazy loading of the broker's signal step stream with
a `Feed.index_stream()` method.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 38469bd6ef Slight rework: shm API
Add an internal `_Token` to do interchange (un)packing for passing
"references" to shm blocks between actors.  Part of the token involves
providing the `numpy.dtype` in a cross-actor format.  Add a module
variable for caching "known tokens" per actor.  Drop use of context
managers since they tear down shm blocks too soon in debug mode and
there seems to be no reason to unlink/close shm before the process has
terminated; if code needs it torn down explicitly, it can.
2020-10-02 12:13:50 -04:00
Tyler Goodlet cd540fd07e Cleanups 2020-10-02 12:13:50 -04:00
Tyler Goodlet 6fa4f6e943 Port charts to new shm arrays 2020-10-02 12:13:50 -04:00
Tyler Goodlet f872fbecf8 Hook IB up to shared memory system
Adjust the `data.open_feed()` api to take a shm token so the
broker-daemon can attach a previously created (by the parent actor) mem
buf and push real-time tick data. There's still some sloppiness here in
terms of ensuring only one mem buf per symbol (can be seen in
`stream_quotes()`) which should really managed at the data api level.
Add a bar incrementing stream-task which delivers increment msgs to any
consumers.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 17491ba819 Disconnect stdlib's resource_tracker, fix .push()
Logic in `SharedArray.push()` was totally wrong.
Remove all the `multiprocessing.resource_tracker` crap such that we
aren't loading an extra process at every layer and we don't get tons of
errors when cleaning on in an SC way.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 712e36b9d5 First draft of a shared numpy array sub-system
This adds a shared memory "incrementing array" sub-sys interface
for single writer, multi-reader style data passing. The main motivation
is to avoid multiple copies of the same `numpy` array across actors
(plus now we can start being fancy like ray).

There still seems to be some odd issues with the "resource tracker"
complaining at teardown (likely partially to do with SIGINT stuff) so
some further digging in the stdlib code is likely coming.

Pertains to #107 and #98
2020-10-02 12:13:50 -04:00
Tyler Goodlet 07beec59bf Hopefully fix datum offset on signal sub-plots
Added a comment to clarify, ish.
Add `ChartPlotWidget._overlays` as registry of curves added on top of
main graphics. Hackishly (ad-hoc-ishly?) update the curve assuming the
data resides in the same `._array` for now (which it does for historical
vwap).
2020-10-02 12:13:28 -04:00
Tyler Goodlet 2cc2b32805 Fix dbz with `np.divide()` 2020-10-02 12:13:28 -04:00
Tyler Goodlet eb5d64ceef Add support for overlay curves and fixed y-range
Allow passing a fixed ylow, yhigh tuple to `._set_yrange()` which avoids
recomputing the range from data if desired (eg. rsi-like bounded
signals). Add support for overlay curves to the OHLC chart and add basic
support to brokers which provide a historical 'vwap`. The data array
increment logic had to be tweaked to copy the vwap from the last bar.

Oh, and hack the subchart curves with two extra prepended datums to make
them align "better" with the ohlc main chart; need to talk to
`pyqtgraph` core about how to do this more correctly.
2020-10-02 12:13:28 -04:00
Tyler Goodlet e91ba55d68 Always draw any history bars on update 2020-10-02 12:13:28 -04:00
Tyler Goodlet a4a5bff3fa Yes, even more grays 2020-10-02 12:13:28 -04:00
Tyler Goodlet da2325239c Copy non-base dtype fields on bar increment 2020-10-02 12:13:28 -04:00
Tyler Goodlet 80f191c57d Adjust range logic to avoid overlap with labels
By mapping any in view "contents labels" to the range of the
``ViewBox``'s data we can avoid having graphics overlap with labels.
Take this approach instead of specifying a min y-range using the std
and activate the range compute on resize and mouser scrolling.
Also, add y-sticky update for signal plots.
2020-10-02 12:13:28 -04:00
Tyler Goodlet fc0a03d597 Handle OHLC *and* signal indexing 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9a59f2408d Start fsp subpackage, separate momo stuff 2020-10-02 12:13:28 -04:00
Tyler Goodlet 7d24e8eeb0 First draft of real-time rsi using numba 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9d8a867767 Bar graphics update speed improvements
Use two separate `QPicture` instances:
- one for the 3 lines for the last bar
- one for all the historical bars lines

On price changes update the last bar and only update historical bars
when the current bar's period expires (when a new bar is "added").
Add a flag `just_history` for this  `BarItems.draw_lines()`.
Also, switch the internal lines array/buffer to a 2D numpy array to avoid
the type-cast step and instead just flatten using `numpy.ravel()`.

Overall this should avoid the problem of draws getting slower over time
as new bars are added to the history since price updates only redraw
a single bar to the "last" `QPicture` instance. Ideally in the future we
can make the `history` `QPicture` a `QPixmap` but it looks like this
will require some internal work in `pyqtgraph` to support it.
2020-10-02 12:13:28 -04:00
Tyler Goodlet b1591e3ee1 Start mucking with faster bars updates
Use a ``rec2array`` struct array converter to generate lines sequence
faster. Start looking into using a `QPixmap` to avoid redrawing all
bars every update.
2020-10-02 12:13:28 -04:00
Tyler Goodlet 0e513599eb Info log fsp output for now 2020-10-02 12:13:28 -04:00
Tyler Goodlet f1b72dfd6b Better bg color, tweak margins. 2020-10-02 12:13:28 -04:00
Tyler Goodlet 17d205f773 Add proper x-axis time-stamping 2020-10-02 12:13:28 -04:00
Tyler Goodlet ea2a675adf Use dashed crosshair, simplify x-axis alloc 2020-10-02 12:13:28 -04:00
Tyler Goodlet 387a696232 Even more colors 2020-10-02 12:13:28 -04:00
Tyler Goodlet fca6257152 Use dashed lines for crosshair 2020-10-02 12:13:28 -04:00
Tyler Goodlet 7a245ddda4 Add and update y-sticky labels on new price data 2020-10-02 12:13:28 -04:00
Tyler Goodlet 363d4cf609 Start color map 2020-10-02 12:13:28 -04:00
Tyler Goodlet d7466a58b4 Add updateable y-sticky label 2020-10-02 12:13:28 -04:00
Tyler Goodlet 8d29338174 Cleanup latency tracker 2020-10-02 12:13:28 -04:00
Tyler Goodlet 58b2e7e395 Refer to main chart's data for date axis 2020-10-02 12:13:28 -04:00
Tyler Goodlet f46fa99a6e Add "contents" labels to charts
Add a default "contents label" (eg. OHLC values for bar charts) to each
chart and update on crosshair interaction.

Few technical changes to make this happen:
- adjust bar graphics to have the HL line be in the "middle" of the
  underlying arrays' "index range" in the containing view.
- add a label dict each chart's graphics name to a label + update routine
- use symbol names instead of this "main" identifier crap for referring to
  particular price curves/graphics
2020-10-02 12:13:28 -04:00
Tyler Goodlet 61e460a422 Start brokers.api module 2020-10-02 12:13:28 -04:00
Tyler Goodlet fd21f4b0fe WIP initial draft of FSP subsystem
This is a first attempt at a financial signal processing subsystem which
utilizes async generators for streaming frames of numpy array data
between actors. In this initial attempt the focus is on processing price
data and relaying it to the chart app for real-time display. So far this
seems to work (with decent latency) but much more work is likely needed
around improving the data model for even better latency and less data
duplication.

Surprisingly (or not?) a lot of simplifications to the charting code
came out of this in terms of conducting graphics updates in streaming
tasks instead of hiding them inside the obfuscated mess that is the
Qt-style-inheritance-OO-90s-trash. The goal from here on wards will be
to enforce strict semantics around reading and writing of data such that
state is kept outside "object trees" as much as possible and streaming
function semantics guide our flow model. Unsurprisingly, this reduction
in "instance state" is happening wherever we use `trio` ;)

A little summary on the technical changes:
- not going to explain the fsp system yet; it's too nascent and
  probably going to get some heavy editing.
- drop any "update" methods from the `LinkedCharts` type since each
  sub-chart will have it's own update task and thus a separate update
  loop; further individual graphics (per chart) may eventually require
  this same design.
- delete `ChartView`; moved into separate mod.
- add "stream from fsp" task to start our foray into real-time actor
  processed numpy streaming.
2020-10-02 12:13:28 -04:00
Tyler Goodlet edb32e8c2b Drop weird chart type enum 2020-10-02 12:13:28 -04:00
Tyler Goodlet 04e21a96da Use partial, pass kwargs to `tractor._main()` 2020-10-02 12:13:28 -04:00
Tyler Goodlet a29b7d9be5 Start "interaction" module 2020-10-02 12:13:28 -04:00
Tyler Goodlet 8a46f8d6ed Port monitor to normalized streams 2020-10-02 12:13:28 -04:00
Tyler Goodlet 241b2374e8 Port `DataFeed` api to broker specific normalizer routine 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9bbf0e0d7a Add a normalizer routine which emits quote differentials/ticks 2020-10-02 12:13:28 -04:00
Tyler Goodlet 81fb327fe1 Add `services` cmd for monitoring actors 2020-10-02 12:13:28 -04:00
Tyler Goodlet 65fb92eaff Flatten out chart tasks 2020-10-02 12:13:28 -04:00
Tyler Goodlet 6b572eb0ef Add ravel() reference link 2020-10-02 12:13:28 -04:00
Tyler Goodlet 971b871647 Handle "mouse-not-on-plot" edge cases 2020-10-02 12:13:28 -04:00
Tyler Goodlet b2506b04f6 Attempt more reliable chart startup
Wait for a first actual real-time quote before starting graphics update
tasks. Use the new normalized tick format brokers are expected to emit
as a `quotes['ticks']` list. Auto detect time frame from historical
bars.
2020-10-02 12:13:28 -04:00
Tyler Goodlet d81f6620f5 Passthrough loglevel from qtractor 2020-10-02 12:13:28 -04:00
Tyler Goodlet 2eea946e5b Drop forkserver usage.
We've got the sweet and realable `trio` spawner now :)
2020-10-02 12:13:28 -04:00
Tyler Goodlet 1a143f6b16 Pass piker log level through to tractor for chart app 2020-10-02 12:13:28 -04:00
Tyler Goodlet dc919fa676 Set tractor loglevel in cli config 2020-10-02 12:13:28 -04:00
Tyler Goodlet cb8215c203 Also log the payload 2020-10-02 12:13:28 -04:00
Tyler Goodlet 46c804db0b Support the `stream_quotes()` api in questrade backend 2020-10-02 12:13:28 -04:00
Tyler Goodlet ad519c10a9 Always just look up the current plot on mouse handling 2020-10-02 12:13:28 -04:00
Tyler Goodlet bbe02570b3 Allow for dynamically added plots
Add `ChartPlotWidget.add_plot()` to add sub charts for indicators which
can be updated independently. Clean up rt bar update code and drop some
legacy ohlc loading cruft.
2020-10-02 12:13:28 -04:00
Tyler Goodlet b4f1ec7960 Massively simplify the cross-hair monstrosity
Stop with all this "main chart" special treatment.
Manage all lines in the same way across all referenced plots.
Add `CrossHair.add_plot()` for adding new plots dynamically.

Just, smh.
2020-10-02 12:13:28 -04:00
Tyler Goodlet c56aee6347 Use array of names for lookup 2020-10-02 12:13:28 -04:00
Tyler Goodlet 788771bd75 Change name to qtractor 2020-10-02 12:13:28 -04:00
Tyler Goodlet 3aebeb5801 Standardize ohlc dtype 2020-10-02 12:13:28 -04:00
Tyler Goodlet e5bca1e089 Fix import error 2020-10-02 12:13:28 -04:00
Tyler Goodlet 88fb7a8951 Handle overloaded arg 2020-10-02 12:13:28 -04:00
Tyler Goodlet 1a1e768126 Port to new data apis 2020-10-02 12:13:28 -04:00
Tyler Goodlet 6802675637 Add kraken to backend list 2020-10-02 12:13:28 -04:00
Tyler Goodlet f9084b8650 Store lines graphics in struct array to simplify indexing 2020-10-02 12:13:28 -04:00
Tyler Goodlet b9224cd396 Add WIP real-time 5s bar charting 2020-10-02 12:13:28 -04:00
Tyler Goodlet 4c5bc19ec7 Always convert to posix time 2020-10-02 12:13:28 -04:00
Tyler Goodlet 5f89a2bf08 Make run_qtrio invoke tractor at top level 2020-10-02 12:13:28 -04:00
Tyler Goodlet 2dec32e41f Move bar generation into func; support bar appends
There's really nothing coupling it to the graphics class (which frankly
also seems like it doesn't need to be a class.. Qt).

Add support to `.update_from_array()` for diffing with the input array
and creating additional bar-lines where necessary. Note, there are still
issues with the "correctness" here in terms of bucketing open/close
values in the time frame / bar range. Also, this jamming of each bar's 3
lines into a homogeneous array seems like it could be better done with
struct arrays and avoid all this "index + 3" stuff.
2020-10-02 12:13:28 -04:00
Tyler Goodlet 3c55f7c6e2 Use structure array indexing syntax 2020-10-02 12:13:28 -04:00
Tyler Goodlet 9c1d64413e Handle flat bar updates
Flat bars have a rendering issue we work around by hacking values in `QLineF`
but we have to revert those on any last bar that is being updated in
real-time. Comment out candle implementations for now; we can get back
to it if/when the tinas unite. Oh, and make bars have a little space
between them.
2020-10-02 12:13:28 -04:00
Tyler Goodlet 6b1bdbe3ea Docs the ui pkg mod 2020-10-02 12:13:28 -04:00
Tyler Goodlet 013c0fef15 Fix a bunch of scrolling / panning logic
Don't allow zooming to less then a min number of data points. Allow
panning "outside" the data set (i.e. moving one of the sequence "ends"
to the middle of the view. Start adding logging.
2020-10-02 12:13:28 -04:00
Tyler Goodlet 4c753b5ee6 Add ib 2020-10-02 12:13:28 -04:00
Tyler Goodlet ecfa6d33aa Use msgpack-numpy 2020-10-02 12:13:28 -04:00
Tyler Goodlet 16e2e27cb8 Handle high = low bars
For whatever reason if the `QLineF` high/low values are the same a weird
little rectangle is drawn (my guess is a `float` precision error of some
sort). Instead, if they're the same just use one of the values.
Also, store local vars to avoid so many lookups.
2020-10-02 12:13:28 -04:00
Tyler Goodlet 7a660b335d Make search work with ib backend 2020-10-02 12:13:28 -04:00
Tyler Goodlet 99c18abfea Add symbol search to broker api 2020-10-02 12:13:28 -04:00
Tyler Goodlet 2f1fdaf9e5 Rework charting internals for real-time plotting
`pg.PlotCurveItem.setData()` is normally used for real-time updates to
curves and takes in a whole new array of data to graphics.
It makes sense to stick with this interface especially if
the current datum graphic will originally be drawn from tick quotes and
later filled in when bars data is available (eg. IB has this option in
TWS charts for volume). Additionally, having a data feed api where the push
process/task can write to shared memory and the UI task(s) can read from
that space is ideal. It allows for indicator and algo calculations to be
run in parallel (via actors) with initial price draw instructions
such that plotting of downstream metrics can be "pipelined" into the
chart UI's render loop. This essentially makes the chart UI async
programmable from multiple remote processes (or at least that's the
goal).

Some details:
- Only store a single ref to the source array data on the
  `LinkedSplitCharts`.  There should only be one reference since the main
  relation is **that** x-time aligned sequence.
- Add `LinkedSplitCharts.update_from_quote()` which takes in a quote
  dict and updates the OHLC array from it's contents.
- Add `ChartPlotWidget.update_from_array()` method to trigger graphics
  updates per chart with consideration for overlay curves.
2020-10-02 12:13:28 -04:00
Tyler Goodlet 5e8e48c7b7 Support updating bars graphics from array
This makes a OHLC graphics "sequence" update very similar (actually API
compatible) with `pg.PlotCurveItem.setData()`. The difference here is
that only latest OHLC datum is used to update the charts last bar.
2020-10-02 12:13:28 -04:00
Tyler Goodlet 048a13dd0e Drop disk caching of quotes 2020-10-02 12:13:28 -04:00
Tyler Goodlet 6ba0692851 Revert weird bad .time access 2020-10-02 12:13:28 -04:00