Commit Graph

2509 Commits (0000d9a3141a0b337db5d62c2568ccb2215acc53)

Author SHA1 Message Date
Tyler Goodlet 909e068121 Support multi-client order-dialog management
This patch was originally to fix a bug where new clients who
re-connected to an `emsd` that was running a paper engine were not
getting updates from new fills and/or cancels. It turns out the solution
is more general: now, any client that creates a order dialog will be
subscribing to receive updates on the order flow set mapped for that
symbol/instrument as long as the client has registered for that
particular fqsn with the EMS. This means re-connecting clients as well
as "monitoring" clients can see the same orders, alerts, fills and
clears.

Impl details:
- change all var names spelled as `dialogues` -> `dialogs` to be
  murican.
- make `Router.dialogs: dict[str, defaultdict[str, list]]` so that each
  dialog id (oid) maps to a set of potential subscribing ems clients.
- add `Router.fqsn2dialogs: dict[str, list[str]]` a map of fqsn entries to
  sets of oids.
- adjust all core task code to make appropriate lookups into these 2 new
  tables instead of being handed specific client streams as input.
- start the `translate_and_relay_brokerd_events` task as a daemon task
  that lives with the particular `TradesRelay` such that dialogs cleared
  while no client is connected are still processed.
- rename `TradesRelay.brokerd_dialogue` -> `.brokerd_stream`
- broadcast all status msgs to all subscribed clients in the relay loop.
- always de-reg each client stream from the `Router.dialogs` table on close.
2022-10-05 01:41:18 -04:00
Tyler Goodlet cf835b97ca Add some info logs around paper fills 2022-10-05 01:41:18 -04:00
Tyler Goodlet 30bce42c0b Don't spin paper clear loop on non-clearing ticks
Not sure what exactly happened but it seemed clears weren't working in
some cases without this, also there's no point in spinning the simulated
clearing loop if we're handling a non-clearing tick type.
2022-10-05 01:41:18 -04:00
Tyler Goodlet 48ff4859e6 Update to new pair schema, adds `.cost_decimals` field 2022-10-05 01:41:18 -04:00
Tyler Goodlet 887583d27f Bleh, convert fill data to `float`s in kraken broker.. 2022-10-05 01:41:18 -04:00
Tyler Goodlet 45b97bf6c3 Make fill msg `.action: str` optional for `kraken` 2022-10-05 01:41:18 -04:00
Tyler Goodlet 91397b85a4 Fix missing f-str in ems msg sender err block 2022-10-05 01:41:18 -04:00
Tyler Goodlet 47f81b31af Kraken can cause status msg key error!? 2022-10-05 01:41:18 -04:00
Tyler Goodlet 41b0c11aaa Hide existing level line markers on startup 2022-09-23 17:17:32 -04:00
Tyler Goodlet cc67d23eee Drop old marker drawing code from `LevelLine.paint()`
We haven't been using it for a while and the supposed (remembered)
latency issue on interaction doesn't seem existing after applying the
cache mode. This allows dropping some internal state-logic and generally
simplifying the show-on-hover checks.

Further add `.show_markers()` and `.hide_markers()` as explicit methods
that can be called externally by UI business logic.
2022-09-23 17:17:32 -04:00
Tyler Goodlet 4818af1445 Add better doc string on marker factory 2022-09-21 15:43:35 -04:00
Tyler Goodlet 2cf1742999 Always apply at least the pos size as the limit 2022-09-21 15:43:35 -04:00
Tyler Goodlet 25ac6e6665 Soft pop lines, handle error-cancel races 2022-09-21 15:43:35 -04:00
Tyler Goodlet 90754f979b Tick the slow chart task on a 1sec index event 2022-09-19 17:39:26 -04:00
Tyler Goodlet c0d490ed63 Only show pos nav on non-zero size 2022-09-19 16:17:05 -04:00
Tyler Goodlet 7c6d12d982 Always set marker y-pos even if we're tracking its x-pos 2022-09-19 16:17:05 -04:00
Tyler Goodlet fd8c05e024 A lines entry should always exist or it's a bug 2022-09-19 16:17:05 -04:00
Tyler Goodlet 5d65c86c84 Don't delete pp lines or markers
Bit of a face palm but obviously `LevelLine.delete()` also removes any
`._marker` from the view which makes it disappear permanently when
moving from non-zero to zero to non-zero positions.. We don't really
need to delete the line since it can be re-used so just remove that
code.

Further this patch removes marker style setting logic from within the
`pp_line()` factory and instead expects the caller to set the correct
"direction" (for long / short) afterward.
2022-09-19 16:17:05 -04:00
Tyler Goodlet cf11e8d7d8 Update navs on all slow and fast charts, only default the fast chart on switch 2022-09-19 16:17:05 -04:00
Tyler Goodlet ed868f6246 Go back to origin slow chart split proportion 2022-09-19 16:17:05 -04:00
Tyler Goodlet 6897aed6b6 Don't call show on marker in `Nav.show()` 2022-09-14 16:02:07 -04:00
Tyler Goodlet a61a11f86b Add draft but commented "scale-to-fast-chart" logic 2022-09-14 10:11:43 -04:00
Tyler Goodlet 286f620f8e Use fqsn to key pnl tasks 2022-09-13 18:59:12 -04:00
Tyler Goodlet b7e60b9653 Hide labels, show markers for lines on slow chart 2022-09-13 18:31:21 -04:00
Tyler Goodlet df42e7acc4 Add `LevelLine.get_cursor()` to get any currently hovering mouse-cursor 2022-09-13 18:26:06 -04:00
Tyler Goodlet e492e9ca0c Fix pp arrow/label placement bugs
- Every time a symbol is switched on chart we need to wait until the
  search bar sidepane has been added beside the slow chart before
  determining the offset for the pp line's arrow/labels; trigger this in
  `GodWidget.load_symbol()` -> required monkeypatching on a
  `.mode: OrderMode` to the `.rt_linked` for now..
- Drop the search pane widget removal from the current linked chart,
  seems faster?
- On the slow chart override the `LevelMarker.scene_x()` callback to
  adjust for the case where no L1 labels are shown beside the y-axis.
2022-09-13 17:58:20 -04:00
Tyler Goodlet 44c6f6dfda Add level line flag to allow tracking its marker x-position 2022-09-13 17:43:04 -04:00
Tyler Goodlet ad2100fe3f Only don't pp arrow on startup 2022-09-13 16:21:49 -04:00
Tyler Goodlet ae64ac79a6 Doc str tweaks 2022-09-13 16:13:46 -04:00
Tyler Goodlet 20663dfa1c Add (more) order mode race guards to avoid crashes on "kitty-keys" 2022-09-12 20:25:15 -04:00
Tyler Goodlet 70f2241d22 Hide pp markers on startup 2022-09-12 20:25:15 -04:00
Tyler Goodlet b3fcc25e21 Add extra row count for header, drop prints 2022-09-12 20:25:15 -04:00
Tyler Goodlet 4f15ce346b Drop splitter resizes except for once at startup
Also adds a `GodWidget.resize_all()` helper method which resizes all
sub-widgets and charts to their default ratios and/or parent-widget
dependent defaults using the detected available space on screen. This is
a "default layout" config method that eventually we'll probably want
allow users to customize.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 445849337f Always resize to slow chart height, not just on changes 2022-09-12 20:25:15 -04:00
Tyler Goodlet 3fd7107e08 Scale view to measured results row count
In other words instead of some static view size previously determined by
the accompanying (slow) chart's height, (recursively) calculate the
number of displayed rows and compute the minimal height needed. This
still caps the view at the height of the chart such that the view will
switch to scroll bar mode when too many results are shown and can't all
be fit in the vertical space.

Deats:
- add a ``CompleterView.iter_df_rows()`` which recursively iterates all
  rows in depth-first order making it simple to compute the absolute
  number of result rows in view and thus the minimal number of pixels to
  show all results.
- always pass the height in the `.on_resize()` handler to ensure
  triggering the height logic when new results are generated in the
  search loop.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 73a02d54b7 Down size the slots bar by .9 2022-09-12 20:25:15 -04:00
Tyler Goodlet b734af6dd0 Only delete lines under cursor if not `None` 2022-09-12 20:25:15 -04:00
Tyler Goodlet f7c0ee930a Offset last (live) datum from y-axis by a 16th 2022-09-12 20:25:15 -04:00
Tyler Goodlet ead426abc4 More space to fast chart(s), less to slow chart 2022-09-12 20:25:15 -04:00
Tyler Goodlet bcd6bbb7ca Increase the `brokerd` mem-chan size
Intention is to hopefully minimize (as many) context switches when
processing (near-)HFT feeds - tho not sure if it's improving things that
much XD
2022-09-12 20:25:15 -04:00
Tyler Goodlet 80929d080f Add more detailed splitter of splitters comment 2022-09-12 20:25:15 -04:00
Tyler Goodlet eed47b3733 Add splitter move handler which calls search widget resizer method 2022-09-12 20:25:15 -04:00
Tyler Goodlet d5f0c59b57 Ignore resize events with the same height (for now) 2022-09-12 20:25:15 -04:00
Tyler Goodlet d11dc787a1 First working attempt of search results view scaling
Scales the "view" instance that holds search results to the size of the
accompanying "slow chart" for which the search pane is a "sidepane".
A lot of mucking about was required due to resizing of the view
seemingly feeding back into window resizing and further implementing the
sizing logic such that the parent `QSplitter` can be resized as the
user's whim as well.

Details,
- add a `CompleterView._init: bool` which is set once (and only once)
  after startup where the first display of the current symbol/feed is
  shown allowing and a single *width* padding applied once at startup
  to ensure we don't have an awkward line to the right of the longest
  result.
- in `.resize_to_results()` only apply a minimum height to the view
  using `.setMinimumHeight()` with a down-scaled (`0.91` for now) height
  value from input.
- re-implement `CompleterView.show_matches()` to accept and optional
  width, heigh tuple and when not supplied pull the slow chart's
  dimensions and pass as input to the resize method.
- Make `SearchWidget` x dim sizing policy "fixed".
- register the `SearchWidget` for resize events with god.
- add `.show_only_cache_entries()` for easy results clearing.
- add `.space_dims()` to retrieve slow linked-charts dimensions.
- implement `SearchWidget.on_resize()` which is the caller of all the
  previously mentioned resizing routines.
- do resizing and cache entry showing on search loop startup and be sure
  to clear to cache when the user selects a symbol-feed with Enter.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 1e81feee46 Finally get chart startup view-state kinda correct
It ended up being what'd you expect, races on the accessing shm buffer
data by the UI during the whole "mega-async-startup-everything" phase XD

So we add the following list of ad-hoc startup steps:
- do `.default_view()` on the slow chart after the fast chart is mostly
  fully spawned with the intention being to capture the state where the
  historical buffer is mostly loaded before sizing the view to the
  graphical form of the data.
- resize slow chart sidepanes from the fast chart just before sleeping
  forever (and after order mode has booted).
2022-09-12 20:25:15 -04:00
Tyler Goodlet 40a9761943 Actually support resize events..
Turns out god widget resizes aren't triggered implicitly by window
resizes, so instead, hook into the window by moving what was our useless
method to that class. Further we explicitly define and declare that our
window has a `.godwidget: GodWidget` and set it up in the bootstrap
phase - in `run_qutractor()` during `trio` guest mode configuration.

Further deatz:
- retype the runtime/bootstrap routines to take a qwidget "type" not an
  instance, and drop the whole implicit `.main_widget` stuff.
- delegate into the `GodWidget.on_win_resize()` for any window resize
  which then triggers all the custom resize callbacks we already had in
  place.
- privatize `ChartnPane.sidepane` so that it can't be mutated willy
  nilly without calling `.set_sidepane()`.
- always adjust splitter sizes inside `LinkeSplits.add_plot()`.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 256bcf36d3 Drop use `tractor.trionics.gather_contexts()` in `open_handlers()` 2022-09-12 20:25:15 -04:00
Tyler Goodlet 9944277096 Handle null lines that were removed, don't error on bad $size 2022-09-12 20:25:15 -04:00
Tyler Goodlet f9dc5637fa Use rt buffer for last price on nan in ems 2022-09-12 20:25:15 -04:00
Tyler Goodlet addedc20f1 WIP search pane always shown.. 2022-09-12 20:25:15 -04:00
Tyler Goodlet 1fa6e8d9ba Only show slow chart xlabel when focussed 2022-09-12 20:25:15 -04:00
Tyler Goodlet 2a06dc997f Use pixel caching on our level lines 2022-09-12 20:25:15 -04:00
Tyler Goodlet 6b93eedcda Port to new `._position.Nav` apis in order mode 2022-09-12 20:25:15 -04:00
Tyler Goodlet a786df65de Factor pos tracker UI element mgmt into new type
More or less moves all the UI related position "nav" logic and graphics
item management into a new `._position.Nav` composite type + api for
high level mgmt of position graphics indicators across multiple charts
(fast and slow).
2022-09-12 20:25:15 -04:00
Tyler Goodlet 8f2823d5f0 Stage line only on active cursor chart 2022-09-12 20:25:15 -04:00
Tyler Goodlet 58fe220fde Use ref annotations in position mod 2022-09-12 20:25:15 -04:00
Tyler Goodlet 161448c31a Support order staging from slow chart using `.get_cursor()` 2022-09-12 20:25:15 -04:00
Tyler Goodlet 1c685189d1 Change to using real type annots 2022-09-12 20:25:15 -04:00
Tyler Goodlet ceac3f2ee4 Adjust corresponding fast/slow chart line level on edits 2022-09-12 20:25:15 -04:00
Tyler Goodlet a07367fae2 Fix div-by-zero split sizing bug 2022-09-12 20:25:15 -04:00
Tyler Goodlet 006190d227 Add fill arrow-mark support to history view 2022-09-12 20:25:15 -04:00
Tyler Goodlet 412197019e Make ArrowEditor.add()` expect a `PlotItem` as input for render 2022-09-12 20:25:15 -04:00
Tyler Goodlet 271e378ce3 Add `GodWidget.iter_linked()` interator over linked split charts 2022-09-12 20:25:15 -04:00
Tyler Goodlet 8e07fda88f Expose multi-chart-lines support through to order mode api 2022-09-12 20:25:15 -04:00
Tyler Goodlet a4935b8fa8 Make line editor multi-line aware, drop `dataclass` for `Struct` 2022-09-12 20:25:15 -04:00
Tyler Goodlet 2b76baeb10 Pass god widget to line editor and order mode instances 2022-09-12 20:25:15 -04:00
Tyler Goodlet 2dfa8976a0 Make line editor expect god as input, use new .`get_cursor()` api 2022-09-12 20:25:15 -04:00
Tyler Goodlet d3402f715b Set godwidget active cursor from xhair callback 2022-09-12 20:25:15 -04:00
Tyler Goodlet f070f9a984 Add "active cursor" api to god widget 2022-09-12 20:25:15 -04:00
Tyler Goodlet 416270ee6c Refocus view on ctl-c from search 2022-09-12 20:25:15 -04:00
Tyler Goodlet 14bee778ec Hook up kb ctrls to hist chart, order mode not working yet 2022-09-12 20:25:15 -04:00
Tyler Goodlet 10c1944de5 Proper slow chart auto y-range support
The slow (history) chart requires it's own y-range checker logic which
needs to be run in 2 cases:
- the last datum is in view and goes outside the previous mx/mn in view
- the chart is incremented a step

Since we need this duplicate logic this patch also factors the incremental
graphics update info "reading" into a new `DisplayState.incr_info()`
method that can be configured to a chart and input state and returns all
relevant "graphics update measure" in a tuple (for now).

Use this method throughout the rest of the display loop for both fast
and slow chart checks and in the `increment_history_view()` slow chart
task.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 7958d8ad4f Up sample info poll loop iters 2022-09-12 20:25:15 -04:00
Tyler Goodlet 50c5dc255c Update history view y-sticky with last clear price 2022-09-12 20:25:15 -04:00
Tyler Goodlet 31735f26d3 Poll for sampling info at startup, tolerate races
Use the new `Feed.get_ds_info()` method in a poll loop to definitively
get the inter-chart sampling info and avoid races with shm buffer
backfilling.

Also, factor the history increment closure-task into
`graphics_update_loop()` which will make it clearer how to factor
all the "should we update" logic into some `DisplayState` API.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 2ef6460853 Add `Feed.get_ds_info()` to detect/compute sample rates 2022-09-12 20:25:15 -04:00
Tyler Goodlet 5e98a30537 Add simplified history incrementer consumer task 2022-09-12 20:25:15 -04:00
Tyler Goodlet dd03ef42ac Return empty search result on connection failure
If you spawn a brokerd set and no `ib` data feed was started (via our
`.data.feed.Feed` api) then there will be no active client loaded and
thus wont' be connected. So in these cases just return nothing, and
I guess we'll figure out real connection failures later?
2022-09-12 20:25:15 -04:00
Tyler Goodlet 59884d251e Update history "last" bar, compute ampling ratio
Add an update call to the display loop to consistently update the last
datum in the history view chart. Compute the inter-chart sampling ratio
and use it to sync the linear region.
2022-09-12 20:25:15 -04:00
Tyler Goodlet e06e257a81 Another history view splitter proportion tweak 2022-09-12 20:25:15 -04:00
Tyler Goodlet 6e574835c8 Update history shm buffer in ohlc sampler loop 2022-09-12 20:25:15 -04:00
Tyler Goodlet 49ccfdd673 Pass history shm "last index" in init msg, assign on feed 2022-09-12 20:25:15 -04:00
Tyler Goodlet 3a434f312b Add sidepane like color region styling 2022-09-12 20:25:15 -04:00
Tyler Goodlet bb4dc448b3 Add history chart and "linear region" for syncing
Add a first draft of a working `pyqtgraph.LinearRegionItem` link between
a history view chart (+ data set) and the normal real-time "HFT" chart
set.

Add the history view (aka more downsampled data view) chart set to the
rt/hft set's splitter as it's "first widget". Hook up linear region
callbacks to enable syncing between charts including compenstating for
the downsampling rate ration (in this case hardcoded 60 since 1s to 1M,
but we'll actually compute it going forward obvs).

More to come dawgys..
2022-09-12 20:25:15 -04:00
Tyler Goodlet 9846396df2 Add initial history (view) to charting sys
Adds an additional `GodWidget.hist_linked: LinkedSplits` alongside the
renamed `.rt_linked` to enable 2 sets of linked charts with different
sampled data sets/flows. The history set is added without "all the
fixins" for now (i.e. no order mode sidepane or search integration) such
that it is merely a top level chart which shows a much longer term
history and can be added to the UI via embedding the entire history
linked-splits instance into the real-time linked set's splitter.

Further impl deats:
- adjust the `GodWidget._chart_cache: dict[str, tuple]]` to store both
  linked split chart sets per symbol so that symbol switching will
  continue to work with the added history chart (set).
- rework `.load_symbol()` to operate on both the real-time (HFT) chart
  set and the history set.
- rework `LinkedSplits.set_split_sizes()` to compensate for the history
  chart and do more detailed height calcs arithmetic to make it appear
  by default as a minor sub-chart.
- adjust `LinkedSplits.add_plot()` and `ChartPlotWidget` internals to allow
  adding a plot without a sidepane and/or container `ChartnPane`
  composite widget by checking for a `sidepane == False` input.
- make `.default_view()` accept a manual y-axis offset kwarg.
- adjust search mode to provide history linked splits to
  `.set_chart_symbol()` call.
2022-09-12 20:25:15 -04:00
Tyler Goodlet f0d417ce42 Drop status msg var deleting from ns 2022-09-12 20:25:15 -04:00
Tyler Goodlet 55fc4114b4 Initial draft code working with `pg.LinearRegionItem` 2022-09-12 20:25:15 -04:00
Tyler Goodlet 97b074365b Use rt buffer for close price pnl calcs 2022-09-12 20:25:15 -04:00
Tyler Goodlet f79c3617d6 Always load FSPs with the default (fast) sampling period 2022-09-12 20:25:15 -04:00
Tyler Goodlet 861fe791eb Allocate 2 shm buffers for history and real-time
As part of supporting a "history view" chart which shows downsampled
datums alongside our 1s (or higher) sampled OHLC we need a separate
buffer to store a the slower history from broker backends. This begins
that design by allocating 2 buffers:
- `rt_shm: ShmArray` which maps to a `/dev/shm/` file with `_rt` suffix
- `hist_shm: ShmArray` which maps to a file with `_hist` suffix

Deliver both of these shms back from both `manage_history()` and load
them as `Feed.rt_shm`/`.hist_shm` on the client side.

Impl deats:
- init the rt buffer with the first datum from loaded history and
  assign all OHLC values to that row's 'close' and the vlm to 0.
- pass the hist buffer to the backfiller task
- only spawn **one** global sampler array-row increment task per
  `brokerd` and pass in the 1s delay which we presume is our lowest
  OHLC sample rate for now.
- drop `open_sample_step_stream()` and just move its body contents into
  `Feed.index_stream()`
2022-09-12 20:25:15 -04:00
Tyler Goodlet 60052ff73a Presume shortest delay input to `increment_ohlc_buffer()`
Instead of worrying about the increment period per shm subscription,
just use the value passed as input and presume the caller knows that
only one task is necessary and that the wakeup (sampling) period should
be the shortest that is needed.

It's very unlikely we don't want at least a 1s sampling (both in terms
of task switching cost and general usage) which will eventually ship as
the default "real-time" feed "timeframe". Further, this "fast" increment
sampling task can handle all lower sampling periods (eg. 1m, 5m, 1H)
based on the current implementation just the same.

Also, add a global default sample period as `_defaul_delay_s` for use in
other internal modules.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 4d2708cd42 Force 1s sample step so crypto boiz can seee 2022-09-12 20:25:15 -04:00
Tyler Goodlet d1cc52dff5 Use new public lifetime-stack class attr 2022-09-12 20:24:56 -04:00
Tyler Goodlet 4fa901dbcb Port to new `tractor._runtime` mod 2022-09-12 20:24:56 -04:00
Tyler Goodlet 4a9c16d298 Fix stream type annot 2022-09-12 15:52:50 -04:00
Tyler Goodlet b9d5b904f4 Drop order entry removals on modify 2022-09-12 15:52:22 -04:00
Tyler Goodlet 0aef762d9a Bleh `kraken`, fix another ref error in fill block
Clearly, the linter didn't help us here.. but, just pass the
`brokerd` time for now in the `.broker_time` field; we can't get it from
the fill-case incremental updates in the `openOrders` sub. Add some
notes about this and how we might approach for backends with this
limitation.
2022-09-12 15:52:22 -04:00
goodboy c724117c1a
Merge pull request #398 from pikers/paper_clear_logics_fix
Oof, reverse clearing logic-routines in paper eng
2022-09-11 22:20:04 -04:00
Tyler Goodlet cc3bb85c66 Oof, reverse clearing logic-routines in paper eng 2022-09-10 16:35:31 -04:00
Tyler Goodlet 23d0b8a7ac Lul, fix name error on msg var name.. 2022-09-05 21:15:22 -04:00