Commit Graph

246 Commits (bb9ca8636c76930f80b952f615a97114bc1df232)

Author SHA1 Message Date
Tyler Goodlet 30bce42c0b Don't spin paper clear loop on non-clearing ticks
Not sure what exactly happened but it seemed clears weren't working in
some cases without this, also there's no point in spinning the simulated
clearing loop if we're handling a non-clearing tick type.
2022-10-05 01:41:18 -04:00
Tyler Goodlet 45b97bf6c3 Make fill msg `.action: str` optional for `kraken` 2022-10-05 01:41:18 -04:00
Tyler Goodlet 91397b85a4 Fix missing f-str in ems msg sender err block 2022-10-05 01:41:18 -04:00
Tyler Goodlet 47f81b31af Kraken can cause status msg key error!? 2022-10-05 01:41:18 -04:00
Tyler Goodlet 2cf1742999 Always apply at least the pos size as the limit 2022-09-21 15:43:35 -04:00
Tyler Goodlet f9dc5637fa Use rt buffer for last price on nan in ems 2022-09-12 20:25:15 -04:00
Tyler Goodlet f0d417ce42 Drop status msg var deleting from ns 2022-09-12 20:25:15 -04:00
Tyler Goodlet 4a9c16d298 Fix stream type annot 2022-09-12 15:52:50 -04:00
Tyler Goodlet b9d5b904f4 Drop order entry removals on modify 2022-09-12 15:52:22 -04:00
Tyler Goodlet cc3bb85c66 Oof, reverse clearing logic-routines in paper eng 2022-09-10 16:35:31 -04:00
Tyler Goodlet d5c1cdd91d Configure allocator from pos msg on startup
This fixes a regression added after moving the msg parsing to later in
the order mode startup sequence. The `Allocator` needs to be configured
*to* the initial pos otherwise default settings will show in the UI..

Move the startup config logic from inside `mk_allocator()` to
`PositionTracker.update_from_pp()` and add a flag to allow setting the
`.startup_pp` from the current live one as is needed during initial
load.
2022-08-29 11:39:28 -04:00
Tyler Goodlet 46d3fe88ca Fix sub-slot-remains limiting for -ve sizes
In the short case (-ve size) we had a bug where the last sub-slots worth
of exit size would never be limited to zero once the allocator limit pos
size was hit (i.e. you could keep going more -ve on the pos,
exponentially per slot over the limit). It's a simple fix, just
a `max()` around the `l_sub_pp` var used in the next-step-size calc.

Resolves #392
2022-08-28 13:51:54 -04:00
Tyler Goodlet 5c8c5d8fbf Fix disti-mode paper pps relaying
Turns out we were putting too many brokername suffixes in the symbol
field and thus the order mode msg parser wasn't matching the current
asset to said msgs correctly and pps weren't being shown...

This repairs that plus simplifies the order mode initial pos msg loading
to just delegate into `process_trade_msg()` just as is done for
real-time msg updates.
2022-08-27 15:37:54 -04:00
Tyler Goodlet 02f53d0c13 Error on zero-size orders received by paper engine 2022-08-26 10:46:47 -04:00
Tyler Goodlet 4cedfedc21 Support clearing ticks ('last' & 'trade') fills
Previously we only simulated paper engine fills when the data feed
provide L1 queue-levels matched an execution. This patch add further
support for clear-level matches when there are real live clears on the
data feed that are faster/not synced with the L1 (aka usually during
periods of HFT).

The solution was to simply iterate the interleaved paper book entries on
both sides for said tick types and instead yield side-specific predicate
per entry.
2022-08-26 10:46:46 -04:00
Tyler Goodlet fe3d0c6fdd Handle too-fast-edits with `defaultdict[str, bidict[str, tuple]]`
Not entirely sure why this all of a sudden became a problem but it seems
price changes on order edits were sometimes resulting in key errors when
modifying paper book entries quickly. This changes the implementation to
not care about matching the last price when keying/popping old orders
and use `bidict`s to more easily pop cleared orders in the paper loop.
2022-08-26 10:46:46 -04:00
Tyler Goodlet 9200e8da57 Raw-dog-pop cancelled paper entries; old price dun matter 2022-08-26 10:46:46 -04:00
Tyler Goodlet 430d065da6 Handle paper-engine too-fast clearing race cases
When the paper engine is used it seems we can definitely hit races where
order ack msgs arrive close enough to status messages that `trio`
schedules the status processing before the acks. In such cases we want
to be tolerant and not crash but instead warn that we got an
unknown/out-of-order msg.
2022-08-26 10:46:46 -04:00
Tyler Goodlet ecd93cb05a Pass symbol with broker suffix to `.submit_limit()`; fix clearing 2022-08-26 10:46:46 -04:00
Tyler Goodlet 973bf87e67 Don't log aboout unknown status msg if no oid 2022-08-18 11:51:12 -04:00
Tyler Goodlet 43bdd4d022 Pass correct instrument symbol in position msgs 2022-08-18 11:51:12 -04:00
Tyler Goodlet bafd2cb44f Only relay fills if dialog still alive 2022-08-18 11:51:12 -04:00
Tyler Goodlet 317610e00a Store positions globally and deliver on ctx connects 2022-08-18 11:51:12 -04:00
Tyler Goodlet c4af706d51 Make order-book-vars globals to persist across ems-dialog connections 2022-08-18 11:51:12 -04:00
Tyler Goodlet bec32956a8 Move fill case-block earlier, log broker errors 2022-08-18 11:51:12 -04:00
Tyler Goodlet 16012f6f02 Include both symbols in error msg when a mismatch 2022-08-18 11:51:12 -04:00
Tyler Goodlet 2b61672723 Handle 'closed' vs. 'fill` race case..
`ib` is super good not being reliable with order event sequence order
and duplication of fill info. This adds some guards to try and avoid
popping the last status status too early if we end up receiving
a `'closed'` before the expected `'fill`' event(s). Further delete the
`status_msg` ref on each iteration to avoid stale reference lookups in
the relay task/loop.
2022-08-18 11:51:12 -04:00
Tyler Goodlet 176b230a46 Use modern `Union` pipe op syntax for msg fields 2022-08-18 11:51:12 -04:00
Tyler Goodlet 7fa9dbf869 Add full EMS order-dialog (re-)load support!
This includes darks, lives and alerts with all connecting clients
being broadcast all existing order-flow dialog states. Obviously
for now darks and alerts only live as long as the `emsd` actor lifetime
(though we will store these in local state eventually) and "live" orders
have lifetimes managed by their respective backend broker.

The details of this change-set is extensive, so here we go..

Messaging schema:
- change the messaging `Status` status-key set to:
  `resp: Literal['pending', 'open', 'dark_open', 'triggered',
                'closed',  'fill', 'canceled', 'error']`

  which better reflects the semantics of order lifetimes and was
  partially inspired by the status keys `kraken` provides for their
  order-entry API. The prior key set was based on `ib`'s horrible
  semantics which sound like they're right out of the 80s..
  Also, we reflect this same set in the `BrokerdStatus` msg and likely
  we'll just get rid of the separate brokerd-dialog side type
  eventually.
- use `Literal` type annots for statuses where applicable and as they
  are supported by `msgspec`.
- add additional optional `Status` fields:
  -`req: Order` to allow each status msg to optionally ref its
    commanding order-request msg allowing at least a request-response
    style implicit tracing in all response msgs.
  -`src: str` tag string to show the source of the msg.
  -`reqid: str | int` such that the ems can relay the `brokerd`
    request id both to the client side and have one spot to look
    up prior status msgs and
- draft a (unused/commented) `Dialog` type which can be eventually used
  at all EMS endpoints to track msg-flow states

EMS engine adjustments/rework:
- use the new status key set throughout and expect `BrokerdStatus` msgs
  to use the same new schema as `Status`.
- add a `_DarkBook._active: dict[str, Status]` table which is now used for
  all per-leg-dialog associations and order flow state tracking
  allowing for the both the brokerd-relay and client-request handler loops
  to read/write the same msg-table and provides for delivering
  the overall EMS-active-orders state to newly/re-connecting clients
  with minimal processing; this table replaces what the `._ems_entries`
  table from prior.
- add `Router.client_broadcast()` to send a msg to all currently
  connected peers.
- a variety of msg handler block logic tweaks including more `case:`
  blocks to be both flatter and improve explicitness:
  - for the relay loop move all `Status` msg update and sending to
    within each block instead of a fallthrough case plus hard-to-follow
    state logic.
  - add a specific case for unhandled backend status keys and just log
    them.
  - pop alerts from `._active` immediately once triggered.
  - where possible mutate status msgs fields over instantiating new
    ones.
- insert and expect `Order` instances in the dark clearing loop and
  adjust `case:` blocks accordingly.
- tag `dark_open` and `triggered` statuses as sourced from the ems.
- drop all the `ChainMap` stuff for now; we're going to make our own
  `Dialog` type for this purpose..

Order mode rework:
- always parse the `Status` msg and use match syntax cases with object
  patterns, hackily assign the `.req` in many blocks to work around not
  yet having proper on-the-wire decoding yet.
- make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed
  `.req: Order` as input.
- change `OrderDialog` -> `Dialog` in prep for a general purpose type
  of the same name.

`ib` backend order loading support:
- do "closed" status detection inside the msg-relay loop instead
  of expecting the ems to do this..
- add an attempt to cancel inactive orders by scheduling cancel
  submissions continually (no idea if this works).
- add a status map to go from the 80s keys to our new set.
- deliver `Status` msgs with an embedded `Order` for existing live order
  loading and make sure to try an get the source exchange info (instead
  of SMART).

Paper engine ported to match:
- use new status keys in `BrokerdStatus` msgs
- use `match:` syntax in request handler loop
2022-08-18 11:51:12 -04:00
Tyler Goodlet 87ed9abefa WIP playing with a `ChainMap` of messages 2022-08-18 11:51:12 -04:00
Tyler Goodlet 2548aae73d Deliver existing dialog (msgs) to every EMS client
Ideally every client that connects to the ems can know its state
(immediately) meaning relay all the order dialogs that are currently
active. This adds full (hacky WIP) support to receive those dialog
(msgs) from the `open_ems()` startup values via the `.started()` msg
from `_emsd_main()`.

Further this adds support to the order mode chart-UI to display existing
(live) orders on the chart during startup. Details include,

- add a `OrderMode.load_unknown_dialog_from_msg()` for processing and
  displaying a ``BrokerdStatus`` (for now) msg from the EMS that was not
  previously created by the current ems client and registering and
  displaying it on the chart.
- break out the ems msg processing into a new
  `order_mode.process_trade_msg()` func so that it can be called on the
  startup dialog-msg set as well as eventually used a more general low
  level auto-strat API (eg. when we get to displaying auto-strat and
  group trading automatically on an observing chart UI.
- hackyness around msg-processing for the dialogs delivery since we're
  technically delivering `BrokerdStatus` msgs when the client-side
  processing technically expects `Status` msgs.. we'll rectify this
  soon!
2022-08-18 11:51:12 -04:00
Tyler Goodlet e34ea94f9f Start brokerd relay loop after opening client stream
In order to avoid missed existing order message emissions on startup we
need to be sure the client side stream is registered with the router
first. So break out the starting of the
`translate_and_relay_brokerd_events()` task until inside the client
stream block and start the task using the dark clearing loop nursery.

Also, ensure `oid` (and thus for `ib` the equivalent re-used `reqid`)
are cast to `str` before registering the dark book. Deliver the dark
book entries as part of the `_emsd_main()` context `.started()` values.
2022-08-18 11:51:12 -04:00
Tyler Goodlet 682a0191ef First draft: relay open orders through ems and display on chart 2022-08-18 11:51:12 -04:00
Tyler Goodlet 109b35f6eb Matchify paper clearing loop 2022-08-05 21:02:15 -04:00
Tyler Goodlet e28c1748fc Comment out "unknown msg" case for now 2022-08-05 21:02:15 -04:00
Tyler Goodlet 72889b4d1f Fix reference error 2022-08-05 21:02:15 -04:00
Tyler Goodlet ae001c3dd7 Matchify the dark trigger loop 2022-08-05 21:02:15 -04:00
Tyler Goodlet 2309e7ab05 Flatten the brokerd-dialog relay loop using `match:` 2022-08-05 21:02:15 -04:00
Tyler Goodlet 46c51b55f7 Flatten the client-request handler loop with `match:` 2022-08-05 21:02:15 -04:00
Tyler Goodlet 5dc9a61ec4 Use cancel level logging for cancelled orders 2022-07-30 17:33:45 -04:00
Tyler Goodlet ab08dc582d Make ems relay loop report on brokerd `.reqid` changes 2022-07-30 17:33:45 -04:00
Tyler Goodlet 2766fad719 Fix #222 multi-symbol paper engine support 2022-07-27 12:18:59 -04:00
Tyler Goodlet ae71168216 Change name `be_price` -> `ppu` throughout codebase 2022-07-27 12:18:36 -04:00
Tyler Goodlet a0c238daa7 Adjust paper-engine to use `Transaction` for pps updates 2022-07-27 11:20:59 -04:00
Tyler Goodlet fd22f45178 Oof, paper engine msg fixes after using `msgspec.Struct`.. 2022-07-11 13:04:07 -04:00
Tyler Goodlet 2a99f7a4d7 Drop remaining `BaseModel` api usage from rest of codebase 2022-07-09 12:38:17 -04:00
Tyler Goodlet b44e2d9ed9 Support `0` value `reqid`s 🤦 2022-07-09 12:10:23 -04:00
Tyler Goodlet 795d4d76f4 Add some todo-reminders for ``msgspec`` stuff 2022-07-09 12:09:50 -04:00
Tyler Goodlet 11b6699a54 Change all clearing msgs over to `msgspec` 2022-07-09 12:09:38 -04:00
Tyler Goodlet 2baea21c7d Drop pydantic from allocator 2022-07-09 12:09:38 -04:00
Tyler Goodlet c870665be0 Remove `BaseModel` use from all dataclass-like uses 2022-07-09 12:08:41 -04:00
Tyler Goodlet 4bcb791161 Finally solve the last-price-is-`nan` issue..
Not sure why I put this off for so long but the check is in now such
that if the market isn't open or no rt quote comes in from the first
query, we just pull from the last shm history 'close' value.
Includes another fix to avoid raising when a double remove on the client
side stream from the registry sometimes happens.
2022-07-08 17:30:34 -04:00
Tyler Goodlet 7442d68ecf Drop nesting level from emsd's pp cacheing, adjust order mode 2022-07-02 16:19:58 -04:00
Tyler Goodlet 453ebdfe30 Fix field name to new `.bsuid` 2022-06-28 10:07:57 -04:00
Tyler Goodlet cc68501c7a Make pp msg `.currency` not required 2022-06-28 10:07:57 -04:00
Tyler Goodlet c617a06905 Port everything to `Position.be_price` 2022-06-28 10:07:56 -04:00
Tyler Goodlet 05a1a4e3d8 Use new `Position.bsuid` field throughout 2022-06-28 10:07:56 -04:00
Tyler Goodlet f768576060 Delegate paper engine pp tracking to new type 2022-06-28 10:07:56 -04:00
Tyler Goodlet 1eb7e109e6 Start `piker.pp` module, LIFO pp updates
Start a generic "position related" util mod and bring in the `Position`
type from the allocator , convert it to a `msgspec.Struct` and add
a `.lifo_update()` method. Implement a WIP pp parser from a trades
ledger and use the new lifo method to gather position entries.
2022-06-28 10:07:56 -04:00
Tyler Goodlet e6d03ba97f Add missing f-str prefix 2022-06-05 22:13:36 -04:00
Tyler Goodlet 47cf4aa4f7 Error log brokerd msgs that have `.reqid == None`
Relates to the bug discovered in #310, this should avoid out-of-order
msgs which do not have a `.reqid` set to be error logged to console.
Further, add `pformat()` to kraken logging of ems msging.
2022-06-05 22:13:08 -04:00
Tyler Goodlet 34e6db6d98 Allow `None` for `BrokerdError.reqid`
Found this caused breakage on `kraken` orders which triggered the
"insufficient funds" error response. Makes sense since they won't
generate an order id if the order can't ever be submitted.
2022-05-09 10:56:47 -04:00
Tyler Goodlet 2b12742992 More ems resiliency: discard broken client dialogs 2022-04-30 11:36:23 -04:00
Tyler Goodlet ed03d77e6e Make a derivs intrustment type table for alloc config checks 2022-04-30 11:36:23 -04:00
Tyler Goodlet 772f871272 Use units by default for continuous futes 2022-04-30 11:36:23 -04:00
Tyler Goodlet ebe2680355 Change `uncons_fqsn()` -> `unpack_fqsn()` 2022-04-11 01:01:36 -04:00
Tyler Goodlet e92632bd34 Remove old commented nan checking lines 2022-04-10 21:51:22 -04:00
Tyler Goodlet c9e6c81459 Expect fqsn input to paper clearing engine 2022-04-10 17:33:02 -04:00
Tyler Goodlet c7f3e59105 Expect fqsn in ems and order mode
Use fqsn as input to the client-side EMS apis but strip broker-name
stuff before generating and sending `Brokerd*` msgs to each backend for
live order requests (since it's weird for a backend to expect it's own
name, though maybe that could be a sanity check?).

Summary of fqsn use vs. broker native keys:
- client side pps, order requests and general UX for order management
  use an fqsn for tracking
- brokerd side order dialogs use the broker-specific symbol which is
  usually nearly the same key minus the broker name
- internal dark book and quote feed lookups use the fqsn where possible
2022-04-10 17:30:02 -04:00
Tyler Goodlet c034ea742f Fix comment: filled not executed is a valid status key 2022-04-09 16:46:25 -04:00
Konstantine Tsafatinos 48c7b5262c get positions working for kraken 2022-03-06 15:17:26 -05:00
Tyler Goodlet 16b9e39e11 Dis-allow an allocator limit less then the current pp size 2022-03-02 10:05:33 -05:00
Tyler Goodlet 7b13124dd4 Keep clear loop price pedantically up to date
To avoid the "trigger finger" issue (darks execing before they should
due to a stale last price state, normally when generating a trigger
predicate..) always iterate the loop and update the last known book
price even when no execs/triggered orders are registered.
2022-02-11 10:30:30 -05:00
Tyler Goodlet ca1c1cf415 Annoying doc strings 2022-02-11 10:30:30 -05:00
wattygetlood fc3c0741b8 Set isn't serializable on std msgpack 2022-02-10 10:26:52 -05:00
Tyler Goodlet 16c04e11e4 Comment out nan-price assert previously for `ib` in ems clear tasks 2022-02-07 09:49:45 -05:00
Tyler Goodlet 47d0c81a2d Only warn if trigger predicate was already popped 2022-01-25 07:57:01 -05:00
Tyler Goodlet b63ce088f2 Error out clearing task on first quote being nan 2022-01-25 07:57:01 -05:00
Tyler Goodlet 50e8b3464f Can't serialize a `set` on `msgpack` codec 2022-01-14 09:16:34 -05:00
Tyler Goodlet 6825ad4804 Add some type annots around pp msg handling 2021-11-05 10:05:11 -04:00
Tyler Goodlet de0cc6d81a Expect accounts as tuple, don't start rt pnl on no live pp 2021-11-05 10:05:11 -04:00
Tyler Goodlet c737de7c74 Rage drop the limit size unit enum 2021-11-05 10:05:11 -04:00
Tyler Goodlet 8f70398d88 Fix exit-slot-edge-case when only one discrete unit remains 2021-11-05 10:05:11 -04:00
Tyler Goodlet d706f35668 Keep slots ratio of 1 on derivs at startup 2021-11-05 10:05:11 -04:00
Tyler Goodlet f5d73edd1b Switch imports to new `tractor.trionics` subpkg 2021-11-01 13:22:23 -04:00
Tyler Goodlet fd8be33f10 Add portal getter, store throttle rate 2021-09-21 15:48:40 -04:00
Tyler Goodlet f4740da6a2 Drop `.accounts` field from allocator 2021-09-14 13:10:39 -04:00
Tyler Goodlet b04645aa47 Expect `accounts: set[str]` startup msg through clearing system 2021-09-14 10:36:13 -04:00
Tyler Goodlet c6cc592f4e Fix wrong key, use account value 2021-09-13 08:24:45 -04:00
Tyler Goodlet 6acfd6c38a Ugh, positions relay hotfix
Must have run into some confusion with data structures in `brokerd` vs.
`emsd`. This fixes the ems `relay.positions` state tracking to be
composed maps, vs. messages from `brokerd` should just be a sequence.
2021-09-12 19:30:43 -04:00
Tyler Goodlet f81d47efc4 Detail some comments 2021-09-11 13:10:20 -04:00
Tyler Goodlet c00cf12f94 Deliver ems cached pps are dict of lists 2021-09-10 18:54:34 -04:00
Tyler Goodlet 054ddf6732 Send error on non-paper account requests to paperboi 2021-09-10 18:54:04 -04:00
Tyler Goodlet e1efb0943b Track per-account pps in ems memory 2021-09-10 11:33:08 -04:00
Tyler Goodlet 15aba154f2 Return account name in next order info 2021-09-09 10:33:52 -04:00
Tyler Goodlet b01538f183 Support an account field in clearing system
Each backend broker may support multiple (types) of accounts; this patch
lets clients send order requests that pass through an `account` field in
certain `emsd` <-> `brokerd` transactions. This allows each provider to read
in and conduct logic based on what account value is passed via requests
to the `trades_dialogue()` endpoint as well as tie together positioning
updates with relevant account keys for display in UIs.

This also adds relay support for a `Status` msg with a `'broker_errored'`
status which for now will trigger the same logic as cancelled orders on
the client side and thus will remove order lines submitted on a chart.
2021-09-08 15:46:33 -04:00
Tyler Goodlet 504040eb59 Add an `account` field to EMS msging schemas 2021-09-08 14:03:18 -04:00
Tyler Goodlet 0d2cddec9a Return accounts in `bidict` 2021-09-08 14:01:54 -04:00
Tyler Goodlet 09d34f7355 Make `accounts` field public, add an account name method 2021-09-07 09:21:55 -04:00
Tyler Goodlet d38a6bf032 Create alloc instance in factory body, template out defaults loading 2021-09-07 08:38:24 -04:00
Tyler Goodlet 214c622328 Move allocator components to clearing sub-pkg 2021-09-06 22:05:42 -04:00
Tyler Goodlet 39d2cac3a6 Handle race with order-request-ack msg 2021-09-06 09:28:10 -04:00
Tyler Goodlet 87bd6046e5 Not sure how this worked before but, pass reqid from existing live order 2021-09-06 09:28:10 -04:00
Tyler Goodlet c1379001e5 Expect `Order` msg instance to ems client `.send()` 2021-09-06 09:28:10 -04:00
Tyler Goodlet 374967dc6f Support temp `Symbol` and non-copied model refs on `Order` msg 2021-09-06 09:28:10 -04:00
Tyler Goodlet a4028d3475 Actually position msgs get relayed verbatim 2021-09-06 09:28:10 -04:00
Tyler Goodlet bd754b740d Only re-calc avg pp price on pp size increases 2021-09-05 13:59:40 -04:00
Tyler Goodlet 449c4210e4 Add per session paper position tracking
Generate and maintain position messages in the paper engine for each
`pikerd` session. We no longer tear down the engine on each client
disconnect. Ensure -ve size on sells to make the math work.
2021-09-05 13:59:40 -04:00
Tyler Goodlet cae7f486e4 Revert "Lol, don't use `maybe_open_feed()` for now, it's totally borked..."
Think this was fixed by passing through `**kwargs` in
`maybe_open_feed()`, the shielding for fsp respawns wasn't being
properly passed through..

This reverts commit 2f1455d423.
2021-08-30 17:55:10 -04:00
Tyler Goodlet 2f1455d423 Lol, don't use `maybe_open_feed()` for now, it's totally borked... 2021-08-26 17:04:59 -04:00
Tyler Goodlet 71b50fdae8 Use broadcast chan for order client and avoid chan repacking 2021-08-26 17:04:59 -04:00
Tyler Goodlet 7d0f47364c Use `maybe_open_feed()` in ems and fsp daemons 2021-08-26 17:04:59 -04:00
Tyler Goodlet 66f1d91541 Let's abstractify: -> 2021-08-26 17:04:59 -04:00
Tyler Goodlet 0ce8057823 Move feed cacheing to cache mod; put entry retreival into ctx mng 2021-08-26 17:04:59 -04:00
Tyler Goodlet 146c684f21 Cache `brokerd` feeds for reuse in clearing loop 2021-08-26 17:04:59 -04:00
Tyler Goodlet eeeeb29f71 Add order cancellation and submission statuses
Generalize the methods for cancelling groups of orders (all or those
under cursor) and add new group status support such that statuses for
each cancel or order submission is displayed in the status bar. In the
"cancel-all-orders" case, use the new group status stuff.
2021-07-08 10:55:47 -04:00
Tyler Goodlet 91907bf5ef Drop old dialogues pop flag, the client does it on teardown 2021-07-08 10:17:31 -04:00
Tyler Goodlet 9f897ba75c Support multiple client trade flows over one `brokerd~
There is no reason to have more then `brokerd` trades dialogue stream
open per `emsd`. Here we minimize to managing that lone stream and
multiplexing msgs from each client such that multiple clients can be
connected to the ems, conducting trading without requiring multiple
ems-client connections to the backend broker and without the broker
being aware there are even multiple flows going on.

This patch also sets up for being able to have ems clients which
register to receive and track trade flows from other piker clients thus
enabling so called "multi-player" trading where orders for both paper
and live trades can be shared between multiple participants in the form
of a pre-broker, local clearing service and trade signals dark book.
2021-07-07 07:51:09 -04:00
Tyler Goodlet 003fa6254f Don't forget to pop the brokerd dialogue on teardown.. 2021-07-07 07:51:09 -04:00
Tyler Goodlet 8f05254c80 Better live order handling logic 2021-07-07 07:51:09 -04:00
Tyler Goodlet a7e106be96 Avoid multiple `brokerd` trades dialogue flows
This solves a bunch of issues to do with `brokerd` order status msgs
getting relayed for each order to **every** correspondingly connected
EMS client. Previously we weren't keeping track of which emsd orders
were associated with which clients so you had backend msgs getting
broadcast to all clients which not only resulted in duplicate (and
sometimes erroneous, due to state tracking) actions taking place in the
UI's order mode, but it's also just duplicate traffic (usually to the
same actor) over multiple logical streams. Instead, only keep up **one**
(cached) stream with the `trades_dialogue()` endpoint such that **all**
emsd orders route over that single connection to the particular
`brokerd` actor.
2021-07-07 07:51:09 -04:00
Tyler Goodlet 57a35a3c6c Port feed bus endpoint to a `@tractor.context` 2021-06-14 10:55:01 -04:00
Tyler Goodlet a9cc3210d8 Grr pydantic being a weirdo 2021-06-10 08:37:21 -04:00
Tyler Goodlet b06cb5bb5a Comments clean and improvments 2021-06-10 08:24:10 -04:00
Tyler Goodlet a1f605bd52 Clear out old commented code 2021-06-09 13:21:56 -04:00
Tyler Goodlet 47e7baa0c9 Ensure paperboi is shield killed on teardown 2021-06-08 15:56:34 -04:00
Tyler Goodlet 6e58f31fd8 Port EMS to typed messaging + bidir streaming
This moves the entire clearing system to use typed messages using
`pydantic.BaseModel` such that the streamed request-response order
submission protocols can be explicitly viewed in terms of message
schema, flow, and sequencing. Using the explicit message formats we can
now dig into simplifying and normalizing across broker provider apis to
get the best uniformity and simplicity.

The order submission sequence is now fully async: an order request is
expected to be explicitly acked with a new message and if cancellation
is requested by the client before the ack arrives, the cancel message is
stashed and then later sent immediately on receipt of the order
submission's ack from the backend broker. Backend brokers are now
controlled using a 2-way request-response streaming dialogue which is
fully api agnostic of the clearing system's core processing; This
leverages the new bi-directional streaming apis from `tractor`.  The
clearing core (emsd) was also simplified by moving the paper engine to
it's own sub-actor and making it api-symmetric with expected `brokerd`
endpoints.

A couple of the ems status messages were changed/added:
'dark_executed' -> 'dark_triggered'
added 'alert_triggered'

More cleaning of old code to come!
2021-06-08 14:33:30 -04:00
Tyler Goodlet 0dabc6ad26 Port paper engine to new msgs and run in sub-actor
This makes the paper engine look IPC-wise exactly like any
broker-provider backend module and uses the new ``trades_dialogue()``
2-way streaming endpoint for commanding order requests.

This serves as a first step toward truly distributed forward testing
since the paper engine can now be run out-of tree from `pikerd` if
needed thus demonstrating how real-time clearing signals can be shared
between fully distinct services.
2021-06-08 14:32:03 -04:00
Tyler Goodlet 23094d8624 Spec out brokerd 2-way trade dialogue messages 2021-06-08 14:31:43 -04:00
Tyler Goodlet f9238f3a8a Validate client message updates 2021-06-08 14:30:47 -04:00
Tyler Goodlet 9931accc52 Port clearing systems to new tractor context api
This avoids somewhat convoluted "hackery" making 2 one-way streams
between the order client and the EMS and instead uses the new
bi-directional streaming and context API from `tractor`. Add a router
type to the EMS that gets setup by the initial service tree and which
we'll eventually use to work toward multi-provider executions and
order-trigger monitoring. Move to py3.9 style where possible throughout.
2021-06-01 10:48:23 -04:00
Tyler Goodlet 7fb2c95ef1 Factor daemon spawning logic, use it to spawn emsd 2021-06-01 10:48:23 -04:00
Tyler Goodlet 0cd3cb3328 Drop old todo 2021-05-27 13:05:23 -04:00
Tyler Goodlet f51e12819a Attach to order client *after* feed connection to speed up the startup time 2021-05-24 12:27:23 -04:00
Tyler Goodlet 4d03d62641 Don't submit limits on alerts 2021-05-24 12:17:16 -04:00
Tyler Goodlet d3b50b9920 Port clearing engine to new tractor stream api 2021-04-29 09:10:36 -04:00
Tyler Goodlet 9622254cdb Hack broadcast chan for order submissions, smh 2021-04-14 11:02:14 -04:00
Tyler Goodlet 4774881812 Handle predicate not yet inserted to exec loop 2021-04-10 14:34:29 -04:00
Tyler Goodlet 47f823ab22 Always close emsd data connection on close 2021-04-10 14:18:40 -04:00
Tyler Goodlet 1281755d65 Don't use mem chan cloning, it doesn't actually broadcast 2021-04-10 14:18:40 -04:00
Tyler Goodlet 29b73b41fb Slight rename; fix predicate is None bug 2021-04-10 14:18:40 -04:00
Tyler Goodlet 7cc395b5bf Open data feed in ems main entrypoint 2021-04-10 14:18:40 -04:00
Tyler Goodlet 17d3e7a9e2 Don't shield ems feed; enact remote cancels 2021-04-10 14:18:40 -04:00
Tyler Goodlet 38471b7b34 Use early return instead of branching for remote emsd spawning 2021-04-03 12:22:14 -04:00
Tyler Goodlet 8ce37875a0 Use pikerd to spawn emsd 2021-04-03 12:22:14 -04:00
Tyler Goodlet 26ee7260d6 Clearing is a better name 2021-04-03 12:22:14 -04:00