piker/piker/clearing/_messages.py

309 lines
7.9 KiB
Python
Raw Normal View History

# piker: trading gear for hackers
# Copyright (C) Tyler Goodlet (in stewardship for pikers)
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
"""
Clearing sub-system message and protocols.
"""
# from collections import (
# ChainMap,
# deque,
# )
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
from typing import (
Optional,
Literal,
)
from ..data._source import Symbol
from ..data.types import Struct
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
# TODO: a composite for tracking msg flow on 2-legged
# dialogs.
# class Dialog(ChainMap):
# '''
# Msg collection abstraction to easily track the state changes of
# a msg flow in one high level, query-able and immutable construct.
# The main use case is to query data from a (long-running)
# msg-transaction-sequence
# '''
# def update(
# self,
# msg,
# ) -> None:
# self.maps.insert(0, msg.to_dict())
# def flatten(self) -> dict:
# return dict(self)
# TODO: ``msgspec`` stuff worth paying attention to:
# - schema evolution:
# https://jcristharif.com/msgspec/usage.html#schema-evolution
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
# - for eg. ``BrokerdStatus``, instead just have separate messages?
# - use literals for a common msg determined by diff keys?
# - https://jcristharif.com/msgspec/usage.html#literal
# --------------
# Client -> emsd
# --------------
class Order(Struct):
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
# TODO: ideally we can combine these 2 fields into
# 1 and just use the size polarity to determine a buy/sell.
# i would like to see this become more like
# https://jcristharif.com/msgspec/usage.html#literal
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
# action: Literal[
# 'live',
# 'dark',
# 'alert',
# ]
action: Literal[
'buy',
'sell',
'alert',
]
# determines whether the create execution
# will be submitted to the ems or directly to
# the backend broker
exec_mode: Literal[
'dark',
'live',
# 'paper', no right?
]
# internal ``emdsd`` unique "order id"
oid: str # uuid4
symbol: str | Symbol
account: str # should we set a default as '' ?
price: float
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
size: float # -ve is "sell", +ve is "buy"
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
brokers: Optional[list[str]] = []
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
class Cancel(Struct):
'''
Cancel msg for removing a dark (ems triggered) or
broker-submitted (live) trigger/order.
'''
action: str = 'cancel'
oid: str # uuid4
symbol: str
# --------------
# Client <- emsd
# --------------
# update msgs from ems which relay state change info
# from the active clearing engine.
class Status(Struct):
name: str = 'status'
time_ns: int
oid: str # uuid4 ems-order dialog id
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
resp: Literal[
'pending', # acked by broker but not yet open
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
'open',
'dark_open', # dark/algo triggered order is open in ems clearing loop
'triggered', # above triggered order sent to brokerd, or an alert closed
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
'closed', # fully cleared all size/units
'fill', # partial execution
'canceled',
'error',
]
# this maps normally to the ``BrokerdOrder.reqid`` below, an id
# normally allocated internally by the backend broker routing system
reqid: Optional[int | str] = None
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
# the (last) source order/request msg if provided
# (eg. the Order/Cancel which causes this msg) and
# acts as a back-reference to the corresponding
# request message which was the source of this msg.
req: Optional[Order | Cancel] = None
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
# XXX: better design/name here?
# flag that can be set to indicate a message for an order
# event that wasn't originated by piker's emsd (eg. some external
# trading system which does it's own order control but that you
# might want to "track" using piker UIs/systems).
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
src: Optional[str] = None
# for relaying a boxed brokerd-dialog-side msg data "through" the
# ems layer to clients.
brokerd_msg: dict = {}
# ---------------
# emsd -> brokerd
# ---------------
# requests *sent* from ems to respective backend broker daemon
class BrokerdCancel(Struct):
action: str = 'cancel'
oid: str # piker emsd order id
time_ns: int
account: str
# "broker request id": broker specific/internal order id if this is
# None, creates a new order otherwise if the id is valid the backend
# api must modify the existing matching order. If the broker allows
# for setting a unique order id then this value will be relayed back
# on the emsd order request stream as the ``BrokerdOrderAck.reqid``
# field
reqid: Optional[int | str] = None
class BrokerdOrder(Struct):
oid: str
account: str
time_ns: int
# TODO: if we instead rely on a +ve/-ve size to determine
# the action we more or less don't need this field right?
action: str = '' # {buy, sell}
# "broker request id": broker specific/internal order id if this is
# None, creates a new order otherwise if the id is valid the backend
# api must modify the existing matching order. If the broker allows
# for setting a unique order id then this value will be relayed back
# on the emsd order request stream as the ``BrokerdOrderAck.reqid``
# field
reqid: Optional[int | str] = None
symbol: str # fqsn
price: float
size: float
# ---------------
# emsd <- brokerd
# ---------------
# requests *received* to ems from broker backend
class BrokerdOrderAck(Struct):
2022-02-10 16:58:45 +00:00
'''
Immediate reponse to a brokerd order request providing the broker
specific unique order id so that the EMS can associate this
(presumably differently formatted broker side ID) with our own
``.oid`` (which is a uuid4).
'''
name: str = 'ack'
# defined and provided by backend
reqid: int | str
# emsd id originally sent in matching request msg
oid: str
account: str = ''
class BrokerdStatus(Struct):
name: str = 'status'
reqid: int | str
time_ns: int
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
status: Literal[
'open',
'canceled',
'fill',
'pending',
'error',
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
]
account: str
filled: float = 0.0
reason: str = ''
remaining: float = 0.0
Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str | int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop
2022-08-10 04:16:08 +00:00
# external: bool = False
# XXX: not required schema as of yet
broker_details: dict = {
'name': '',
}
class BrokerdFill(Struct):
2022-02-10 16:58:45 +00:00
'''
A single message indicating a "fill-details" event from the broker
if avaiable.
'''
name: str = 'fill'
reqid: int | str
time_ns: int
# order exeuction related
action: str
size: float
price: float
broker_details: dict = {} # meta-data (eg. commisions etc.)
# brokerd timestamp required for order mode arrow placement on x-axis
# TODO: maybe int if we force ns?
# we need to normalize this somehow since backends will use their
# own format and likely across many disparate epoch clocks...
broker_time: float
class BrokerdError(Struct):
2022-02-10 16:58:45 +00:00
'''
Optional error type that can be relayed to emsd for error handling.
This is still a TODO thing since we're not sure how to employ it yet.
2022-02-10 16:58:45 +00:00
'''
name: str = 'error'
oid: str
# if no brokerd order request was actually submitted (eg. we errored
# at the ``pikerd`` layer) then there will be ``reqid`` allocated.
reqid: Optional[int | str] = None
symbol: str
reason: str
broker_details: dict = {}
class BrokerdPosition(Struct):
'''Position update event from brokerd.
'''
name: str = 'position'
broker: str
account: str
symbol: str
size: float
avg_price: float
2022-06-21 20:43:52 +00:00
currency: str = ''