Commit Graph

2616 Commits (f2df32a673cd0c0ff5d78e43a4326fc004442265)

Author SHA1 Message Date
Tyler Goodlet 286228c290 Only wait on backfill if provider supports timeframe 2022-10-28 16:17:14 -04:00
Tyler Goodlet a1a24da7b6 Make `binance` reject 1s OHLC history requests 2022-10-28 16:17:14 -04:00
Tyler Goodlet 553d0557b6 Raise `DataUnavailable` when a contract's 'earliest time' is hit 2022-10-28 16:17:14 -04:00
Tyler Goodlet 2f7b272d8c Make `ib` client's `.get_head_time()` (only) expect an fqsn 2022-10-28 16:17:14 -04:00
Tyler Goodlet dc1edeecda Do tsdb backloading to shm concurrently
Not only improves startup latency but also avoids a bug where the rt
buffer was being tsdb-history prepended *before* the backfilling of
recent data from the backend was complete resulting in our of order
frames in shm.
2022-10-28 16:17:14 -04:00
Tyler Goodlet 4ca7817735 Use feed-shm offsets in fill-arrow indexing arithmetic 2022-10-28 16:17:14 -04:00
Tyler Goodlet 5b63585398 Pack multi-chart region linking into helper
Factor the multi-sample-rate region UI connecting into a new helper
`link_views_with_region()` which reads in the shm buffer offsets from
the `Feed` and appropriately connects the fast and slow chart handlers
for the linear region graphics. Add detailed comments writeup for the
inter-sampling transform algebra.
2022-10-28 16:17:14 -04:00
Tyler Goodlet 0000d9a314 Handle backends with no 1s OHLC history
If a history manager raises a `DataUnavailable` just assume the sample
rate isn't supported and that no shm prepends will be done. Further seed
the shm array in such cases as before from the 1m history's last datum.

Also, fix tsdb -> shm back-loading, cancelling tsdb queries when either
no array-data is returned or a frame is delivered which has a start time
no lesser then the least last retrieved. Use strict timeframes for every
`Storage` API call.
2022-10-28 16:17:14 -04:00
Tyler Goodlet f7ec66362e Only get dbus user on sudo-user-present 2022-10-28 16:17:14 -04:00
Tyler Goodlet b7ef0596b9 Drop remaining timeframe scanning from `.read_ohlcv()` 2022-10-28 16:17:14 -04:00
Tyler Goodlet 143e86a80c Handle super annoying mkts query bug..
Turns out querying for a high freq timeframe (like 1sec) will still
return a lower freq timeframe (like 1Min) SMH, and no idea if it's the
server or the client's fault, so we have to explicitly check the sample
step size and discard lower freq series-results. Do this inside
`Storage.read_ohlcv()` and return an empty `dict` when the wrong time
step is detected from the query result.

Further enforcements,
- both `.load()` and `read_ohlcv()` now require an explicit `timeframe:
  int` input to guarantee the time step of the output array.
- drop all calls `.load()` with non-timeframe specific input.
2022-10-28 16:17:14 -04:00
Tyler Goodlet 956c7d3435 Add concurrent multi-time-frame history loading
Our default sample periods are 60s (1m) for the history chart and 1s for
the fast chart. This patch adds concurrent loading of both (or more)
different sample period data sets using the existing loading code but
with new support for looping through a passed "timeframe" table which
points to each shm instance.

More detailed adjustments include:
- breaking the "basic" and tsdb loading into 2 new funcs:
  `basic_backfill()` and `tsdb_backfill()` the latter of which is run
  when the tsdb daemon is discovered.
- adjust the fast shm buffer to offset with one day's worth of 1s so
  that only up to a day is backfilled as history in the fast chart.
- adjust bus task starting in `manage_history()` to deliver back the
  offset indices for both fast and slow shms and set them on the
  `Feed` object as `.izero_hist/rt: int` values:
  - allows the chart-UI linked view region handlers to use the offsets
    in the view-linking-transform math to index-align the history and
    fast chart.
2022-10-28 16:17:14 -04:00
Tyler Goodlet 330d16262e Add data-reset-task global state var
Allows keeping mutex state around data reset requests which (if more
then one are sent) can cause a throttling condition where ib's servers
will get slower and slower to conduct a reconnect. With this you can
have multiple ongoing contract requests without hitting that issue and
we can go back to having a nice 3s timeout on the history queries before
activating the hack.
2022-10-28 16:17:14 -04:00
Tyler Goodlet c7f57b940c Add back adhoc symbol lookup support, some exchs info is off 2022-10-28 16:17:14 -04:00
Tyler Goodlet 27bd3c07af Comment format tweak 2022-10-28 16:17:14 -04:00
Tyler Goodlet 55dc27a197 Subtract duration instead of passing to `.subtract()` (facepalm) 2022-10-28 16:17:14 -04:00
Tyler Goodlet a11f20fac2 Fix `piker services`; `tractor.run()` is done.. 2022-10-28 16:17:14 -04:00
Tyler Goodlet daebb78755 Re-request quote feed on data reset events
When a network outage or data feed connection is reset often the
`ib_insync` task will hang until some kind of (internal?) timeout takes
place or, in some (worst) cases it never re-establishes (the event
stream) and thus the backend needs to restart or the live feed will
never resume..

In order to avoid this issue once and for all this patch implements an
additional (extremely simple) task that is started with the  real-time
feed and simply waits for any market data reset events; when detected
restarts the `open_aio_quote_stream()` call in a loop using
a surrounding cancel scope.

Been meaning to implement this for ages and it's finally working!
2022-10-28 16:17:14 -04:00
Tyler Goodlet 90a395a069 Support no-disconnect on `open_aio_clients()` exit
Allows for easier restarts of certain `trio` side tasks without killing
the `asyncio`-side clients; support via flag.

Also fix a bug in `Client.bars()`: we need to return the duration on the
empty bars case..
2022-10-28 16:17:14 -04:00
Tyler Goodlet 23d0353934 Drop duplicate frame request
Must have gotten left in during refactor from the `trimeter` version?
Drop down to 6 years for 1m sampling.
2022-10-28 16:17:14 -04:00
Tyler Goodlet ede67ed184 Return history-frame duration from `.bars()`
This allows the history manager to know the decrement size for
`end_dt: datetime` on the next query if a no-data / gap case was
encountered; subtract this in `get_bars()` in such cases. Define the
expected `pendulum.Duration`s in the `.api._samplings` table.

Also add a bit of query latency profiling that we may use later to more
dynamically determine timeout driven data feed resets. Factor the `162`
error cases into a common exception handler block.
2022-10-28 16:17:14 -04:00
Tyler Goodlet 811d21e111 Explicit fast chart naming, auto-yrange the fast chart on increment 2022-10-28 16:17:14 -04:00
Tyler Goodlet 54567d33da More correct no-data output handling
When we get a timeout or a `NoData` condition still return a tuple of
empty sequences instead of `None` from `Client.bars()`. Move the
sampling period-duration table to module level.
2022-10-28 16:17:14 -04:00
Tyler Goodlet 61ca5f7e19 Drop `trimeter`-ized concurrent history querying
It doesn't seem to be any slower on our least throttled backend
(binance) and it removes a bunch of hard to get correct frame
re-ordering logic that i'm not sure really ever fully worked XD

Commented some issues we still need to resolve as well.
2022-10-28 16:17:13 -04:00
Tyler Goodlet 7396624be0 Rework history frame request concurrency
Manual tinker-testing demonstrated that triggering data resets
completely independent of the frame request gets more throughput and
further, that repeated requests (for the same frame after cancelling on
the `trio`-side) can yield duplicate frame responses. Re-work the
dual-task structure to instead have one task wait indefinitely on the
frame response (and thus not trigger duplicate frames) and the 2nd data
reset task poll for the first task to complete in a poll loop which
terminates when the frame arrives via an event.

Dirty deatz:
- make `get_bars()` take an optional timeout (which will eventually be
  dynamically passed from the history mgmt machinery) and move request
  logic inside a new `query()` closure meant to be spawned in a task
  which sets an event on frame arrival, add data reset poll loop in the
  main/parent task, deliver result on nursery completion.
- handle frame request cancelled event case without crash.
- on no-frame result (due to real history gap) hack in a 1 day decrement
  case which we need to eventually allow the caller to control likely
  based on measured frame rx latency.
- make `wait_on_data_reset()` a predicate without output indicating
  reset success as well as `trio.Nursery.start()` compat so that it can
  be started in a new task with the started values yielded being
  a cancel scope and completion event.
- drop the legacy `backfill_bars()`, not longer used.
2022-10-28 16:17:13 -04:00
Tyler Goodlet 25b90afbdb Add `timeframe` input to `kraken` history api 2022-10-28 16:17:13 -04:00
Tyler Goodlet 72dfeb2b4e Pass back interal cancel scope from data reset task 2022-10-28 16:17:13 -04:00
Tyler Goodlet 6b34c9e866 Temporarily disable error on pos size mismatch 2022-10-28 16:17:13 -04:00
Tyler Goodlet e7ec01b8e6 Pass in default history time of 1 min
Adjust all history query machinery to pass a `timeframe: int` in seconds
and set default of 60 (aka 1m) such that history views from here forward
will be 1m sampled OHLCV. Further when the tsdb is detected as up load
a full 10 years of data if possible on the 1m - backends will eventually
get a config section (`brokers.toml`) that allow user's to tune this.
2022-10-28 16:17:13 -04:00
Tyler Goodlet fce7055c62 Make `binance` history api accept a timeframe 2022-10-28 16:17:13 -04:00
Tyler Goodlet bf7d5e9a71 Make `marketstore` storage api timeframe aware
The `Store.load()`, `.read_ohlcv()` and `.write_ohlcv()` and
`.delete_ts()` now can take a `timeframe: Optional[float]` param which
is used to look up the appropriate sampling period table-key from
`marketstore`.
2022-10-28 16:17:13 -04:00
Tyler Goodlet 2a866dde65 Make history routines `timeframe` aware
Allow data feed sub-system to specify the timeframe (aka OHLC sample
period) to the `open_history_client()` delivered history fetching API.
Factor the data keycombo hack into a new routine to be used also from
the history backfiller code when request latency increases; there is
a first draft at trying to use the feed reset to speed up 1m frame
throttling by timing out on the history frame response, but it needs
a lot of fine tuning.
2022-10-28 16:17:13 -04:00
Tyler Goodlet 220981e718 Add 1m ohlc sample rate support to `Client.bars()`; frame query is 1 day 2022-10-28 16:17:13 -04:00
Tyler Goodlet 8537a4091b Use new `Status.cancel_called` in EMS msg loops 2022-10-28 16:16:45 -04:00
Tyler Goodlet 71a11a23bd Add `Status.cancel_called: bool`
This is a simpler (and oddly more `trio`-nic and/or SC) way to handle
the cancelled-before-acked race for order dialogs. Will allow keeping
the `.req` field as solely an `Order` msg.
2022-10-28 16:16:45 -04:00
Tyler Goodlet fa368b1263 'Just getitem access the 'action' from req msg' 2022-10-28 16:16:45 -04:00
Tyler Goodlet e6dd1458f8 `kraken`: the apiflows chain map needs a `dict` 2022-10-28 16:16:45 -04:00
Tyler Goodlet 9486d993ce Drop order mode settings change logmsgs to `.runtime` again 2022-10-28 16:16:45 -04:00
Tyler Goodlet 30994dac10 Better handle order-cancelled-but-not-yet-acked races
When the client is faster then a `brokerd` at submitting and cancelling
an order we run into the case where we need to specify that the EMS
cancels the order-flow as soon as the brokerd's ack arrives. Previously
we were stashing a `BrokerdCancel` msg as the `Status.req` msg (to be
both tested for as a "already cancelled" and sent immediately on ack arrival to
the broker), but for such
cases we can't use that msg to find the fqsn (since only the client side
msgs have it defined) which is required by the new
`Router.client_broadcast()`.

So, Since `Status.req` is supposed to be a client-side flow msg anyway,
and we need the fqsn for client broadcasting, we change this `.req`
value to the client's submitted `Cancel` msg (thus rectifying the
missing `Router.client_broadcast()` fqsn input issue) and build the
`BrokerdCancel` request from that `Cancel` inline in the relay loop
from the `.req: Cancel` status msg lookup.

Further we allow `Cancel` msgs to define an `.account` and adjust the
order mode loop to expect `Cancel` source requests in cancelled status
updates.
2022-10-28 16:16:45 -04:00
Tyler Goodlet 8a61211c8c Handle brokerd errors even when no client-side-status found 2022-10-28 16:16:45 -04:00
Tyler Goodlet c43f7eb656 Fix missing `costmin: float` field in pair msgs 2022-10-28 16:16:45 -04:00
Tyler Goodlet c437f9370a Factor out all `maybe_open_context()` guff 2022-10-07 14:13:52 -04:00
Tyler Goodlet 94f81587ab Cache EMS trade relay tasks on feed fqsn
Except for paper accounts (in which case we need a trades dialog and
paper engine per symbol to enable simulated clearing) we can rely on the
instrument feed (symbol name) to be the caching key. Utilize
`tractor.trionics.maybe_open_context()` and the new key-as-callable
support in the paper case to ensure we have separate paper clearing
loops per symbol.

Requires https://github.com/goodboy/tractor/pull/329
2022-10-07 14:13:52 -04:00
Tyler Goodlet 2bc25e3593 Repair already-open order relay, fix causality dilemma
With the refactor of the dark loop into a daemon task already-open order
relaying from a `brokerd` was broken since no subscribed clients were
registered prior to the relay loop sending status msgs for such existing
live orders. Repair that by adding one more synchronization phase to the
`Router.open_trade_relays()` task: deliver a `client_ready: trio.Event`
which is set by the client task once the client stream has been
established and don't start the `brokerd` order dialog relay loop until
this event is ready.

Further implementation deats:
- factor the `brokerd` relay caching back into it's own `@acm` method:
  `maybe_open_brokerd_dialog()` since we do want (but only this) stream
  singleton-cached per broker backend.
- spawn all relay tasks on every entry for the moment until we figure
  out what we're caching against (any client pre-existing right, which
  would mean there's an entry in the `.subscribers` table?)
- rename `_DarkBook` -> `DarkBook` and `DarkBook.orders` -> `.triggers`
2022-10-07 14:13:52 -04:00
Tyler Goodlet 1d9ab7b0de More direct import 2022-10-07 14:13:52 -04:00
Tyler Goodlet 4c96a4878e Process unknown order mode msgs 2022-10-07 14:13:52 -04:00
Tyler Goodlet 8cd56cb6d3 Flip ems-side-client (`OrderBook`) to be a struct
`@dataclass` is so 2 years ago ;)

Also rename `.update()` -> `.send_update()` to be a bit more explicit
about actually sending an update msg.
2022-10-07 14:13:52 -04:00
Tyler Goodlet c246dcef6f Drop uuid from notify func inputs 2022-10-07 14:13:52 -04:00
Tyler Goodlet 26d6e10ad7 Parameterize duration, pprint msg 2022-10-07 14:13:52 -04:00
Tyler Goodlet 3924c66bd0 Move headless notifies into `.client_broadcast()` 2022-10-07 14:13:52 -04:00
Tyler Goodlet 2fbfe583dd Drop the `Router.clients: set`, `.subscribers` is enough 2022-10-07 14:13:52 -04:00
Tyler Goodlet 525f805cdb Port order mode to new notify routine 2022-10-07 14:13:52 -04:00
Tyler Goodlet b65c02336d Don't short circuit relay loop when headless
If no clients are connected we now process as normal and try to fire
a desktop notification on linux.
2022-10-07 14:13:52 -04:00
Tyler Goodlet d3abfce540 Start notify mod, linux only 2022-10-07 14:13:52 -04:00
Tyler Goodlet 49433ea87d Run dark-clear-loop in daemon task
This enables "headless" dark order matching and clearing where an `emsd`
daemon subactor can be left running with active dark (or other
algorithmic) orders which will still trigger despite to attached-controlling
ems-client.

Impl details:
- rename/add `Router.maybe_open_trade_relays()` which now does all work
  of starting up ems-side long living clearing and relay tasks and the
  associated data feed; make is a `Nursery.start()`-able task instead of
  an `@acm`.
- drop `open_brokerd_trades_dialog()` and move/factor contents into the
  above method.
- add support for a `router.client_broadcast('all', msg)` to wholesale
  fan out a msg to all clients.
2022-10-07 14:13:52 -04:00
Tyler Goodlet 35871d0213 Support line update from `Order` msg in `.on_submit()` 2022-10-05 01:41:18 -04:00
Tyler Goodlet 4877af9bc3 Add pub-sub broadcasting
Establishes a more formalized subscription based fan out pattern to ems
clients who subscribe for order flow for a particular symbol (the fqsn
is the default subscription key for now).

Make `Router.client_broadcast()` take a `sub_key: str` value which
determines the set of clients to forward a message to and drop all such
manually defined broadcast loops from task (func) code. Also add
`.get_subs()` which (hackily) allows getting the set of clients for
a given sub key where any stream that is detected as "closed" is
discarded in the output. Further we simplify to `Router.dialogs:
defaultdict[str, set[tractor.MsgStream]]` and `.subscriptions` as maps
to sets of streams for much easier broadcast management/logic using set
operations inside `.client_broadcast()`.
2022-10-05 01:41:18 -04:00
Tyler Goodlet 909e068121 Support multi-client order-dialog management
This patch was originally to fix a bug where new clients who
re-connected to an `emsd` that was running a paper engine were not
getting updates from new fills and/or cancels. It turns out the solution
is more general: now, any client that creates a order dialog will be
subscribing to receive updates on the order flow set mapped for that
symbol/instrument as long as the client has registered for that
particular fqsn with the EMS. This means re-connecting clients as well
as "monitoring" clients can see the same orders, alerts, fills and
clears.

Impl details:
- change all var names spelled as `dialogues` -> `dialogs` to be
  murican.
- make `Router.dialogs: dict[str, defaultdict[str, list]]` so that each
  dialog id (oid) maps to a set of potential subscribing ems clients.
- add `Router.fqsn2dialogs: dict[str, list[str]]` a map of fqsn entries to
  sets of oids.
- adjust all core task code to make appropriate lookups into these 2 new
  tables instead of being handed specific client streams as input.
- start the `translate_and_relay_brokerd_events` task as a daemon task
  that lives with the particular `TradesRelay` such that dialogs cleared
  while no client is connected are still processed.
- rename `TradesRelay.brokerd_dialogue` -> `.brokerd_stream`
- broadcast all status msgs to all subscribed clients in the relay loop.
- always de-reg each client stream from the `Router.dialogs` table on close.
2022-10-05 01:41:18 -04:00
Tyler Goodlet cf835b97ca Add some info logs around paper fills 2022-10-05 01:41:18 -04:00
Tyler Goodlet 30bce42c0b Don't spin paper clear loop on non-clearing ticks
Not sure what exactly happened but it seemed clears weren't working in
some cases without this, also there's no point in spinning the simulated
clearing loop if we're handling a non-clearing tick type.
2022-10-05 01:41:18 -04:00
Tyler Goodlet 48ff4859e6 Update to new pair schema, adds `.cost_decimals` field 2022-10-05 01:41:18 -04:00
Tyler Goodlet 887583d27f Bleh, convert fill data to `float`s in kraken broker.. 2022-10-05 01:41:18 -04:00
Tyler Goodlet 45b97bf6c3 Make fill msg `.action: str` optional for `kraken` 2022-10-05 01:41:18 -04:00
Tyler Goodlet 91397b85a4 Fix missing f-str in ems msg sender err block 2022-10-05 01:41:18 -04:00
Tyler Goodlet 47f81b31af Kraken can cause status msg key error!? 2022-10-05 01:41:18 -04:00
Tyler Goodlet 41b0c11aaa Hide existing level line markers on startup 2022-09-23 17:17:32 -04:00
Tyler Goodlet cc67d23eee Drop old marker drawing code from `LevelLine.paint()`
We haven't been using it for a while and the supposed (remembered)
latency issue on interaction doesn't seem existing after applying the
cache mode. This allows dropping some internal state-logic and generally
simplifying the show-on-hover checks.

Further add `.show_markers()` and `.hide_markers()` as explicit methods
that can be called externally by UI business logic.
2022-09-23 17:17:32 -04:00
Tyler Goodlet 4818af1445 Add better doc string on marker factory 2022-09-21 15:43:35 -04:00
Tyler Goodlet 2cf1742999 Always apply at least the pos size as the limit 2022-09-21 15:43:35 -04:00
Tyler Goodlet 25ac6e6665 Soft pop lines, handle error-cancel races 2022-09-21 15:43:35 -04:00
Tyler Goodlet 90754f979b Tick the slow chart task on a 1sec index event 2022-09-19 17:39:26 -04:00
Tyler Goodlet c0d490ed63 Only show pos nav on non-zero size 2022-09-19 16:17:05 -04:00
Tyler Goodlet 7c6d12d982 Always set marker y-pos even if we're tracking its x-pos 2022-09-19 16:17:05 -04:00
Tyler Goodlet fd8c05e024 A lines entry should always exist or it's a bug 2022-09-19 16:17:05 -04:00
Tyler Goodlet 5d65c86c84 Don't delete pp lines or markers
Bit of a face palm but obviously `LevelLine.delete()` also removes any
`._marker` from the view which makes it disappear permanently when
moving from non-zero to zero to non-zero positions.. We don't really
need to delete the line since it can be re-used so just remove that
code.

Further this patch removes marker style setting logic from within the
`pp_line()` factory and instead expects the caller to set the correct
"direction" (for long / short) afterward.
2022-09-19 16:17:05 -04:00
Tyler Goodlet cf11e8d7d8 Update navs on all slow and fast charts, only default the fast chart on switch 2022-09-19 16:17:05 -04:00
Tyler Goodlet ed868f6246 Go back to origin slow chart split proportion 2022-09-19 16:17:05 -04:00
Tyler Goodlet 6897aed6b6 Don't call show on marker in `Nav.show()` 2022-09-14 16:02:07 -04:00
Tyler Goodlet a61a11f86b Add draft but commented "scale-to-fast-chart" logic 2022-09-14 10:11:43 -04:00
Tyler Goodlet 286f620f8e Use fqsn to key pnl tasks 2022-09-13 18:59:12 -04:00
Tyler Goodlet b7e60b9653 Hide labels, show markers for lines on slow chart 2022-09-13 18:31:21 -04:00
Tyler Goodlet df42e7acc4 Add `LevelLine.get_cursor()` to get any currently hovering mouse-cursor 2022-09-13 18:26:06 -04:00
Tyler Goodlet e492e9ca0c Fix pp arrow/label placement bugs
- Every time a symbol is switched on chart we need to wait until the
  search bar sidepane has been added beside the slow chart before
  determining the offset for the pp line's arrow/labels; trigger this in
  `GodWidget.load_symbol()` -> required monkeypatching on a
  `.mode: OrderMode` to the `.rt_linked` for now..
- Drop the search pane widget removal from the current linked chart,
  seems faster?
- On the slow chart override the `LevelMarker.scene_x()` callback to
  adjust for the case where no L1 labels are shown beside the y-axis.
2022-09-13 17:58:20 -04:00
Tyler Goodlet 44c6f6dfda Add level line flag to allow tracking its marker x-position 2022-09-13 17:43:04 -04:00
Tyler Goodlet ad2100fe3f Only don't pp arrow on startup 2022-09-13 16:21:49 -04:00
Tyler Goodlet ae64ac79a6 Doc str tweaks 2022-09-13 16:13:46 -04:00
Tyler Goodlet 20663dfa1c Add (more) order mode race guards to avoid crashes on "kitty-keys" 2022-09-12 20:25:15 -04:00
Tyler Goodlet 70f2241d22 Hide pp markers on startup 2022-09-12 20:25:15 -04:00
Tyler Goodlet b3fcc25e21 Add extra row count for header, drop prints 2022-09-12 20:25:15 -04:00
Tyler Goodlet 4f15ce346b Drop splitter resizes except for once at startup
Also adds a `GodWidget.resize_all()` helper method which resizes all
sub-widgets and charts to their default ratios and/or parent-widget
dependent defaults using the detected available space on screen. This is
a "default layout" config method that eventually we'll probably want
allow users to customize.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 445849337f Always resize to slow chart height, not just on changes 2022-09-12 20:25:15 -04:00
Tyler Goodlet 3fd7107e08 Scale view to measured results row count
In other words instead of some static view size previously determined by
the accompanying (slow) chart's height, (recursively) calculate the
number of displayed rows and compute the minimal height needed. This
still caps the view at the height of the chart such that the view will
switch to scroll bar mode when too many results are shown and can't all
be fit in the vertical space.

Deats:
- add a ``CompleterView.iter_df_rows()`` which recursively iterates all
  rows in depth-first order making it simple to compute the absolute
  number of result rows in view and thus the minimal number of pixels to
  show all results.
- always pass the height in the `.on_resize()` handler to ensure
  triggering the height logic when new results are generated in the
  search loop.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 73a02d54b7 Down size the slots bar by .9 2022-09-12 20:25:15 -04:00
Tyler Goodlet b734af6dd0 Only delete lines under cursor if not `None` 2022-09-12 20:25:15 -04:00
Tyler Goodlet f7c0ee930a Offset last (live) datum from y-axis by a 16th 2022-09-12 20:25:15 -04:00
Tyler Goodlet ead426abc4 More space to fast chart(s), less to slow chart 2022-09-12 20:25:15 -04:00
Tyler Goodlet bcd6bbb7ca Increase the `brokerd` mem-chan size
Intention is to hopefully minimize (as many) context switches when
processing (near-)HFT feeds - tho not sure if it's improving things that
much XD
2022-09-12 20:25:15 -04:00
Tyler Goodlet 80929d080f Add more detailed splitter of splitters comment 2022-09-12 20:25:15 -04:00
Tyler Goodlet eed47b3733 Add splitter move handler which calls search widget resizer method 2022-09-12 20:25:15 -04:00
Tyler Goodlet d5f0c59b57 Ignore resize events with the same height (for now) 2022-09-12 20:25:15 -04:00
Tyler Goodlet d11dc787a1 First working attempt of search results view scaling
Scales the "view" instance that holds search results to the size of the
accompanying "slow chart" for which the search pane is a "sidepane".
A lot of mucking about was required due to resizing of the view
seemingly feeding back into window resizing and further implementing the
sizing logic such that the parent `QSplitter` can be resized as the
user's whim as well.

Details,
- add a `CompleterView._init: bool` which is set once (and only once)
  after startup where the first display of the current symbol/feed is
  shown allowing and a single *width* padding applied once at startup
  to ensure we don't have an awkward line to the right of the longest
  result.
- in `.resize_to_results()` only apply a minimum height to the view
  using `.setMinimumHeight()` with a down-scaled (`0.91` for now) height
  value from input.
- re-implement `CompleterView.show_matches()` to accept and optional
  width, heigh tuple and when not supplied pull the slow chart's
  dimensions and pass as input to the resize method.
- Make `SearchWidget` x dim sizing policy "fixed".
- register the `SearchWidget` for resize events with god.
- add `.show_only_cache_entries()` for easy results clearing.
- add `.space_dims()` to retrieve slow linked-charts dimensions.
- implement `SearchWidget.on_resize()` which is the caller of all the
  previously mentioned resizing routines.
- do resizing and cache entry showing on search loop startup and be sure
  to clear to cache when the user selects a symbol-feed with Enter.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 1e81feee46 Finally get chart startup view-state kinda correct
It ended up being what'd you expect, races on the accessing shm buffer
data by the UI during the whole "mega-async-startup-everything" phase XD

So we add the following list of ad-hoc startup steps:
- do `.default_view()` on the slow chart after the fast chart is mostly
  fully spawned with the intention being to capture the state where the
  historical buffer is mostly loaded before sizing the view to the
  graphical form of the data.
- resize slow chart sidepanes from the fast chart just before sleeping
  forever (and after order mode has booted).
2022-09-12 20:25:15 -04:00
Tyler Goodlet 40a9761943 Actually support resize events..
Turns out god widget resizes aren't triggered implicitly by window
resizes, so instead, hook into the window by moving what was our useless
method to that class. Further we explicitly define and declare that our
window has a `.godwidget: GodWidget` and set it up in the bootstrap
phase - in `run_qutractor()` during `trio` guest mode configuration.

Further deatz:
- retype the runtime/bootstrap routines to take a qwidget "type" not an
  instance, and drop the whole implicit `.main_widget` stuff.
- delegate into the `GodWidget.on_win_resize()` for any window resize
  which then triggers all the custom resize callbacks we already had in
  place.
- privatize `ChartnPane.sidepane` so that it can't be mutated willy
  nilly without calling `.set_sidepane()`.
- always adjust splitter sizes inside `LinkeSplits.add_plot()`.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 256bcf36d3 Drop use `tractor.trionics.gather_contexts()` in `open_handlers()` 2022-09-12 20:25:15 -04:00
Tyler Goodlet 9944277096 Handle null lines that were removed, don't error on bad $size 2022-09-12 20:25:15 -04:00
Tyler Goodlet f9dc5637fa Use rt buffer for last price on nan in ems 2022-09-12 20:25:15 -04:00
Tyler Goodlet addedc20f1 WIP search pane always shown.. 2022-09-12 20:25:15 -04:00
Tyler Goodlet 1fa6e8d9ba Only show slow chart xlabel when focussed 2022-09-12 20:25:15 -04:00
Tyler Goodlet 2a06dc997f Use pixel caching on our level lines 2022-09-12 20:25:15 -04:00
Tyler Goodlet 6b93eedcda Port to new `._position.Nav` apis in order mode 2022-09-12 20:25:15 -04:00
Tyler Goodlet a786df65de Factor pos tracker UI element mgmt into new type
More or less moves all the UI related position "nav" logic and graphics
item management into a new `._position.Nav` composite type + api for
high level mgmt of position graphics indicators across multiple charts
(fast and slow).
2022-09-12 20:25:15 -04:00
Tyler Goodlet 8f2823d5f0 Stage line only on active cursor chart 2022-09-12 20:25:15 -04:00
Tyler Goodlet 58fe220fde Use ref annotations in position mod 2022-09-12 20:25:15 -04:00
Tyler Goodlet 161448c31a Support order staging from slow chart using `.get_cursor()` 2022-09-12 20:25:15 -04:00
Tyler Goodlet 1c685189d1 Change to using real type annots 2022-09-12 20:25:15 -04:00
Tyler Goodlet ceac3f2ee4 Adjust corresponding fast/slow chart line level on edits 2022-09-12 20:25:15 -04:00
Tyler Goodlet a07367fae2 Fix div-by-zero split sizing bug 2022-09-12 20:25:15 -04:00
Tyler Goodlet 006190d227 Add fill arrow-mark support to history view 2022-09-12 20:25:15 -04:00
Tyler Goodlet 412197019e Make ArrowEditor.add()` expect a `PlotItem` as input for render 2022-09-12 20:25:15 -04:00
Tyler Goodlet 271e378ce3 Add `GodWidget.iter_linked()` interator over linked split charts 2022-09-12 20:25:15 -04:00
Tyler Goodlet 8e07fda88f Expose multi-chart-lines support through to order mode api 2022-09-12 20:25:15 -04:00
Tyler Goodlet a4935b8fa8 Make line editor multi-line aware, drop `dataclass` for `Struct` 2022-09-12 20:25:15 -04:00
Tyler Goodlet 2b76baeb10 Pass god widget to line editor and order mode instances 2022-09-12 20:25:15 -04:00
Tyler Goodlet 2dfa8976a0 Make line editor expect god as input, use new .`get_cursor()` api 2022-09-12 20:25:15 -04:00
Tyler Goodlet d3402f715b Set godwidget active cursor from xhair callback 2022-09-12 20:25:15 -04:00
Tyler Goodlet f070f9a984 Add "active cursor" api to god widget 2022-09-12 20:25:15 -04:00
Tyler Goodlet 416270ee6c Refocus view on ctl-c from search 2022-09-12 20:25:15 -04:00
Tyler Goodlet 14bee778ec Hook up kb ctrls to hist chart, order mode not working yet 2022-09-12 20:25:15 -04:00
Tyler Goodlet 10c1944de5 Proper slow chart auto y-range support
The slow (history) chart requires it's own y-range checker logic which
needs to be run in 2 cases:
- the last datum is in view and goes outside the previous mx/mn in view
- the chart is incremented a step

Since we need this duplicate logic this patch also factors the incremental
graphics update info "reading" into a new `DisplayState.incr_info()`
method that can be configured to a chart and input state and returns all
relevant "graphics update measure" in a tuple (for now).

Use this method throughout the rest of the display loop for both fast
and slow chart checks and in the `increment_history_view()` slow chart
task.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 7958d8ad4f Up sample info poll loop iters 2022-09-12 20:25:15 -04:00
Tyler Goodlet 50c5dc255c Update history view y-sticky with last clear price 2022-09-12 20:25:15 -04:00
Tyler Goodlet 31735f26d3 Poll for sampling info at startup, tolerate races
Use the new `Feed.get_ds_info()` method in a poll loop to definitively
get the inter-chart sampling info and avoid races with shm buffer
backfilling.

Also, factor the history increment closure-task into
`graphics_update_loop()` which will make it clearer how to factor
all the "should we update" logic into some `DisplayState` API.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 2ef6460853 Add `Feed.get_ds_info()` to detect/compute sample rates 2022-09-12 20:25:15 -04:00
Tyler Goodlet 5e98a30537 Add simplified history incrementer consumer task 2022-09-12 20:25:15 -04:00
Tyler Goodlet dd03ef42ac Return empty search result on connection failure
If you spawn a brokerd set and no `ib` data feed was started (via our
`.data.feed.Feed` api) then there will be no active client loaded and
thus wont' be connected. So in these cases just return nothing, and
I guess we'll figure out real connection failures later?
2022-09-12 20:25:15 -04:00
Tyler Goodlet 59884d251e Update history "last" bar, compute ampling ratio
Add an update call to the display loop to consistently update the last
datum in the history view chart. Compute the inter-chart sampling ratio
and use it to sync the linear region.
2022-09-12 20:25:15 -04:00
Tyler Goodlet e06e257a81 Another history view splitter proportion tweak 2022-09-12 20:25:15 -04:00
Tyler Goodlet 6e574835c8 Update history shm buffer in ohlc sampler loop 2022-09-12 20:25:15 -04:00
Tyler Goodlet 49ccfdd673 Pass history shm "last index" in init msg, assign on feed 2022-09-12 20:25:15 -04:00
Tyler Goodlet 3a434f312b Add sidepane like color region styling 2022-09-12 20:25:15 -04:00
Tyler Goodlet bb4dc448b3 Add history chart and "linear region" for syncing
Add a first draft of a working `pyqtgraph.LinearRegionItem` link between
a history view chart (+ data set) and the normal real-time "HFT" chart
set.

Add the history view (aka more downsampled data view) chart set to the
rt/hft set's splitter as it's "first widget". Hook up linear region
callbacks to enable syncing between charts including compenstating for
the downsampling rate ration (in this case hardcoded 60 since 1s to 1M,
but we'll actually compute it going forward obvs).

More to come dawgys..
2022-09-12 20:25:15 -04:00
Tyler Goodlet 9846396df2 Add initial history (view) to charting sys
Adds an additional `GodWidget.hist_linked: LinkedSplits` alongside the
renamed `.rt_linked` to enable 2 sets of linked charts with different
sampled data sets/flows. The history set is added without "all the
fixins" for now (i.e. no order mode sidepane or search integration) such
that it is merely a top level chart which shows a much longer term
history and can be added to the UI via embedding the entire history
linked-splits instance into the real-time linked set's splitter.

Further impl deats:
- adjust the `GodWidget._chart_cache: dict[str, tuple]]` to store both
  linked split chart sets per symbol so that symbol switching will
  continue to work with the added history chart (set).
- rework `.load_symbol()` to operate on both the real-time (HFT) chart
  set and the history set.
- rework `LinkedSplits.set_split_sizes()` to compensate for the history
  chart and do more detailed height calcs arithmetic to make it appear
  by default as a minor sub-chart.
- adjust `LinkedSplits.add_plot()` and `ChartPlotWidget` internals to allow
  adding a plot without a sidepane and/or container `ChartnPane`
  composite widget by checking for a `sidepane == False` input.
- make `.default_view()` accept a manual y-axis offset kwarg.
- adjust search mode to provide history linked splits to
  `.set_chart_symbol()` call.
2022-09-12 20:25:15 -04:00
Tyler Goodlet f0d417ce42 Drop status msg var deleting from ns 2022-09-12 20:25:15 -04:00
Tyler Goodlet 55fc4114b4 Initial draft code working with `pg.LinearRegionItem` 2022-09-12 20:25:15 -04:00
Tyler Goodlet 97b074365b Use rt buffer for close price pnl calcs 2022-09-12 20:25:15 -04:00
Tyler Goodlet f79c3617d6 Always load FSPs with the default (fast) sampling period 2022-09-12 20:25:15 -04:00
Tyler Goodlet 861fe791eb Allocate 2 shm buffers for history and real-time
As part of supporting a "history view" chart which shows downsampled
datums alongside our 1s (or higher) sampled OHLC we need a separate
buffer to store a the slower history from broker backends. This begins
that design by allocating 2 buffers:
- `rt_shm: ShmArray` which maps to a `/dev/shm/` file with `_rt` suffix
- `hist_shm: ShmArray` which maps to a file with `_hist` suffix

Deliver both of these shms back from both `manage_history()` and load
them as `Feed.rt_shm`/`.hist_shm` on the client side.

Impl deats:
- init the rt buffer with the first datum from loaded history and
  assign all OHLC values to that row's 'close' and the vlm to 0.
- pass the hist buffer to the backfiller task
- only spawn **one** global sampler array-row increment task per
  `brokerd` and pass in the 1s delay which we presume is our lowest
  OHLC sample rate for now.
- drop `open_sample_step_stream()` and just move its body contents into
  `Feed.index_stream()`
2022-09-12 20:25:15 -04:00
Tyler Goodlet 60052ff73a Presume shortest delay input to `increment_ohlc_buffer()`
Instead of worrying about the increment period per shm subscription,
just use the value passed as input and presume the caller knows that
only one task is necessary and that the wakeup (sampling) period should
be the shortest that is needed.

It's very unlikely we don't want at least a 1s sampling (both in terms
of task switching cost and general usage) which will eventually ship as
the default "real-time" feed "timeframe". Further, this "fast" increment
sampling task can handle all lower sampling periods (eg. 1m, 5m, 1H)
based on the current implementation just the same.

Also, add a global default sample period as `_defaul_delay_s` for use in
other internal modules.
2022-09-12 20:25:15 -04:00
Tyler Goodlet 4d2708cd42 Force 1s sample step so crypto boiz can seee 2022-09-12 20:25:15 -04:00
Tyler Goodlet d1cc52dff5 Use new public lifetime-stack class attr 2022-09-12 20:24:56 -04:00