Commit Graph

2651 Commits (a0b415095a0bfd502d71736621f0598e9ed56403)

Author SHA1 Message Date
Tyler Goodlet fcd7e0f3f3 Avoid crash on trades ledger msgs
Just ignore them for now using new `match:` syntax B)
but we'll do incremental update sooon!

Resolves 
2022-07-02 16:20:22 -04:00
Tyler Goodlet 9106d13dfe Drop wacky if block logic, while loop, handle errors and prep for async batching 2022-07-02 16:20:22 -04:00
Tyler Goodlet d3caad6e11 Factor data feeds endpoints into new sub-mod 2022-07-02 16:20:22 -04:00
Tyler Goodlet f87a2a810a Make broker mod import from new api mod 2022-07-02 16:20:21 -04:00
Tyler Goodlet 208e2e9e97 Move core api code into sub-module 2022-07-02 16:20:21 -04:00
Tyler Goodlet 90cc6eb317 Factor clearing related endpoints into new `.kraken.broker` submod 2022-07-02 16:20:21 -04:00
Tyler Goodlet b118becc84 Start `kraken` sub-pkg 2022-07-02 16:20:21 -04:00
Tyler Goodlet 7442d68ecf Drop nesting level from emsd's pp cacheing, adjust order mode 2022-07-02 16:19:58 -04:00
Tyler Goodlet 076c167d6e Fix ib pkg mod doc string 2022-07-02 16:14:34 -04:00
Tyler Goodlet 64d8cd448f Right, handle brand-new pp case.. 2022-07-02 16:14:34 -04:00
Tyler Goodlet ec6a28a8b1 Drop stale comment 2022-07-02 16:14:34 -04:00
Tyler Goodlet cc15d02488 Fix `.minimize_clears()` to include clears since zero
This was just implemented totally wrong but somehow worked XD

The idea was to include all trades that contribute to ongoing position
size since the last time the position was "net zero", i.e. no position
in the asset. Adjust arithmetic to *subtract* from the current size
until a zero size condition is met and then keep all those clears as
part of the "current state" clears table.

Additionally this fixes another bug where the positions freshly loaded
from a ledger *were not* being merged with the current `pps.toml` state.
2022-07-02 16:14:34 -04:00
goodboy d5bc43e8dd
Merge pull request from pikers/lifo_pps_ib
LIFO/"breakeven" pps for `ib`
2022-06-29 10:07:56 -04:00
Tyler Goodlet 287a2c8396 Put swb2 in venue filter for now 2022-06-29 10:00:38 -04:00
Tyler Goodlet 453ebdfe30 Fix field name to new `.bsuid` 2022-06-28 10:07:57 -04:00
Tyler Goodlet 2b1fb90e03 Add tractor breaker assert.. 2022-06-28 10:07:57 -04:00
Tyler Goodlet 695ba5288d Comment-drop adhoc symbol (futes) matching in search 2022-06-28 10:07:57 -04:00
Tyler Goodlet d6c32bba86 Use new adhoc sym map for symbols without exchange tags (usually futes) 2022-06-28 10:07:57 -04:00
Tyler Goodlet fa89207583 Use sign of the new size which indicates direction of position 2022-06-28 10:07:57 -04:00
Tyler Goodlet 557562e25c Build out adhoc sym map from futes list 2022-06-28 10:07:57 -04:00
Tyler Goodlet c6efa2641b Cost part of position breakeven calc is direction dependent 2022-06-28 10:07:57 -04:00
Tyler Goodlet 8a7e391b4e Terser startup msg fields 2022-06-28 10:07:57 -04:00
Tyler Goodlet aec48a1dd5 Right, zero sized "closed out" msgs are totally fine 2022-06-28 10:07:57 -04:00
Tyler Goodlet 87f301500d Simplify updates to single-pass, fix clears minimizing
Gah, was a remaining bug where if you tried to update the pps state with
both new trades and from the ledger you'd do a double add of
transactions that were cleared during a `update_pps()` loop. Instead now
keep all clears in tact until ready to serialize to the `pps.toml` file
in which cases we call a new method `Position.minimize_clears()` which
does the work of only keep clears since the last net-zero size.

Re-implement `update_pps_conf()` update logic as a single pass loop
which does expiry and size checking for closed pps all in one pass thus
allowing us to drop `dump_active()` which was kinda redundant anyway..
2022-06-28 10:07:57 -04:00
Tyler Goodlet 566a54ffb6 Reset the clears table on zero size conditions 2022-06-28 10:07:57 -04:00
Tyler Goodlet f9c4b3cc96 Fixes for newly opened and closed pps
Before we weren't emitting pp msgs when a position went back to "net
zero" (aka the size is zero) nor when a new one was opened (wasn't
previously loaded from the `pps.toml`). This reworks a bunch of the
incremental update logic as well as ports to the changes in the
`piker.pp` module:

- rename a few of the normalizing helpers to be more explicit.
- drop calling `pp.get_pps()` in the trades dialog task and instead
  create msgs iteratively, per account, by iterating through collected
  position and API trade records and calling instead
  `pp.update_pps_conf()`.
- always from-ledger-update both positions reported from ib's pp sys and
  session api trades detected on ems-trade-dialog startup.
- `update_ledger_from_api_trades()` now does **just** that: only updates
  the trades ledger and returns the transaction set.
- `update_and_audit_msgs()` now only the input list of msgs and properly
  generates new msgs for newly created positions that weren't previously
  loaded from the `pps.toml`.
2022-06-28 10:07:57 -04:00
Tyler Goodlet a12e6800ff Support per-symbol reload from ledger pp loading
- use `tomli` package for reading since it's the fastest pure python
  reader available apparently.
- add new fields to each pp's clears table: price, size, dt
- make `load_pps_from_toml()`'s `reload_records` a dict that can be
  passed in by the caller and is verbatim used to re-read a ledger and
  filter to the specified symbol set to build out fresh pp objects.
- add a `update_from_ledger: bool` flag to `load_pps_from_toml()`
  to allow forcing a full backend ledger read.
- if a set of trades records is passed into `update_pps_conf()` parse
  out the meta data required to cause a ledger reload as per 2 bullets
  above.
- return active and closed pps in separate by-account maps from
  `update_pps_conf()`.
- drop the `key_by` kwarg.
2022-06-28 10:07:57 -04:00
Tyler Goodlet cc68501c7a Make pp msg `.currency` not required 2022-06-28 10:07:57 -04:00
Tyler Goodlet 7ebf8a8dc0 Add `tomli` as dep being fastest in the west 2022-06-28 10:07:57 -04:00
Tyler Goodlet 4475823e48 Add draft ip-mismatch skip case 2022-06-28 10:07:57 -04:00
Tyler Goodlet 3713288b48 Strip ib prefix before acctid use 2022-06-28 10:07:57 -04:00
Tyler Goodlet 4fdfb81876 Support re-processing a filtered ledger entry set
This makes it possible to refresh a single fqsn-position in one's
`pps.toml` by simply deleting the file entry, in which case, if there is
new trade records passed to `load_pps_from_toml()` via the new
`reload_records` kwarg, then the backend ledger entries matching that
symbol will be filtered and used to recompute a fresh position.

This turns out to be super handy when you have crashes that prevent
a `pps.toml` entry from being updated correctly but where the ledger
does have all the data necessary to calculate a fresh correct entry.
2022-06-28 10:07:57 -04:00
Tyler Goodlet f32b4d37cb Support pp audits with multiple accounts 2022-06-28 10:07:56 -04:00
Tyler Goodlet 2063b9d8bb Drop ledger entries that have no transaction id 2022-06-28 10:07:56 -04:00
Tyler Goodlet fe14605034 Fix null case return 2022-06-28 10:07:56 -04:00
Tyler Goodlet 68b32208de Key pps by bsuid to avoid incorrect disparate entries 2022-06-28 10:07:56 -04:00
Tyler Goodlet f1fe369bbf Write clears table as a list of tables in toml 2022-06-28 10:07:56 -04:00
Tyler Goodlet 16b2937d23 Passthrough toml lib kwargs 2022-06-28 10:07:56 -04:00
Tyler Goodlet bfad676b7c Add expiry and datetime support to ledger parsing 2022-06-28 10:07:56 -04:00
Tyler Goodlet c617a06905 Port everything to `Position.be_price` 2022-06-28 10:07:56 -04:00
Tyler Goodlet ff74f4302a Support pp expiries, datetimes on transactions
Since some positions obviously expire and thus shouldn't continually
exist inside a `pps.toml` add naive support for tracking and discarding
expired contracts:
- add `Transaction.expiry: Optional[pendulum.datetime]`.
- add `Position.expiry: Optional[pendulum.datetime]` which can be parsed
  from a transaction ledger.
- only write pps with a non-none expiry to the `pps.toml`
- change `Position.avg_price` -> `.be_price` (be is "breakeven")
  since it's a much less ambiguous name.
- change `load_pps_from_legder()` to *not* call `dump_active()` since
  for the only use case it ends up getting called later anyway.
2022-06-28 10:07:56 -04:00
Tyler Goodlet 21153a0e1e Ugh, hack our own toml encoder since it seems everything in the lib is half-baked.. 2022-06-28 10:07:56 -04:00
Tyler Goodlet b6f344f34a Only emit pps msg for trade triggering instrument
We can probably make this better (and with less file sys accesses) later
such that we keep a consistent pps state in mem and only write async
maybe from another side-task?
2022-06-28 10:07:56 -04:00
Tyler Goodlet ecdc747ced Allow packing pps by a different key set 2022-06-28 10:07:56 -04:00
Tyler Goodlet 5147cd7be0 Drop global proxies table, isn't multi-task safe.. 2022-06-28 10:07:56 -04:00
Tyler Goodlet 3dcb72d429 Only finally-write around the ledger yield up 2022-06-28 10:07:56 -04:00
Tyler Goodlet fbee33b00d Get real-time trade oriented pp updates workin
What a nightmare this was.. main holdup was that cost (commissions)
reports are fired independent from "fills" so you can't really emit
a proper full position update until they both arrive.

Deatz:
- move `push_tradesies()` and relay loop in `deliver_trade_events()` to
  the new py3.10 `match:` syntax B)
- subscribe for, and handle `CommissionReport` events from `ib_insync`
  and repack as a `cost` event type.
- handle cons with no primary/listing exchange (like futes) in
  `update_ledger_from_api_trades()` by falling back to the plain
  'exchange' field.
- drop reverse fqsn lookup from ib positions map; just use contract
  lookup for api trade logs since we're already connected..
- make validation in `update_and_audit()` optional via flag.
- pass in the accounts def, ib pp msg table and the proxies table to the
  trade event relay task-loop.
- add `emit_pp_update()` too encapsulate a full api trade entry
  incremental update which calls into the `piker.pp` apis to,
  - update the ledger
  - update the pps.toml
  - generate a new `BrokerdPosition` msg to send to the ems
- adjust trades relay loop to only emit pp updates when a cost report
  arrives for the fill/execution by maintaining a small table per exec
  id.
2022-06-28 10:07:56 -04:00
Tyler Goodlet 3991d8f911 Add `update_and_audit()` in prep for rt per-trade-event pp udpates 2022-06-28 10:07:56 -04:00
Tyler Goodlet 7b2e8f1ba5 Return object form from `update_pps_conf()` 2022-06-28 10:07:56 -04:00
Tyler Goodlet cbcbb2b243 Filter pps loading to client-active accounts set 2022-06-28 10:07:56 -04:00