Commit Graph

4105 Commits (749401e500c22e5caebac3d4881f0fc0470e0301)

Author SHA1 Message Date
Tyler Goodlet 56cd15fa51 ib: maybe incr client id; can't catch api errors..
Turns out we don't hookup our eventkit handler until after the
`load_aio_clients()` is complete, which means we can't get
`ib_insync.Client.apiError` events unless inside the asyncio side task.
So I guess try to report any such errors during API scan (note the
duplicate client id case is a special one from ibis itself) even though
we're not going to catch them trio side. The hack to work around this is
to just increment the client id value with the `connect_retries` led `i`
value even though that will break on more then 3 clients attached to an
API endpoint lul ..

Further adjustments that were to the end of trying to fix this proper:
- add `remove_handler_on_err()` cm to disconnect a handler when the trio
  side of the channel closes.
- actually connect to client api erros in our `Client.inline_errors()`
- increase connect timeout to a sec.
- change the trio-asyncio proxy response-msg loop over to `match:`
  syntax and raise on unhandled msgs from eventkit handlers.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 879657cc75 Detail `pikerd` sock bind collision in error 2023-05-09 14:49:26 -04:00
Tyler Goodlet fb13c7cbf6 `ib`: drop pp mismatch err block, we already do it in audit routine 2023-05-09 14:49:26 -04:00
Tyler Goodlet 72abe98475 Async-ify order client methods and some renaming
We previously only offered a sync API (which was recently renamed to
`.<meth>_nowait()` style) since initially all order control was from our
`OrderMode` Qt driven UI/UX. This adds the equivalent async methods for
both testing as well as eventual auto-strat driven control B)

Also includes a bunch of renaming:
- `OrderBook` -> `OrderClient`.
- better internal renaming of the client's mem chan vars and add a ref
  `._ems_stream: tractor.MsgStream`.
- drop `get_orders()` factory, just always check for the actor-global
  instance and always set the ems stream on that client (in case old one
  was closed).
2023-05-09 14:49:26 -04:00
Tyler Goodlet 48f096995f `kraken`: write ledger and pps files on startup 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2cc77c21ba Rework paper engine for "offline" pp loading
This will end up being super handy for testing our accounting subsystems
as well as providing unified and simple cli utils for managing ledgers
and position tracking. Allows loading the paper boi without starting
a data feed and instead just trigger ledger and pps loading without
starting the entire clearing engine.

Deatz:
- only init `PaperBoi` and start clearing loop (tasks) if a non-`None`
  fqme is provided, ow just `Context.started()` the existing pps msgs
  as loaded from the ledger.
- always update both the ledger and pp table on startup and pass
  a single instance of each obj to the `PaperBoi` for reuse (without
  opening and closing backing config files since we now have
  `.write_config()`).
- drop the global `_positions` dict, it's not needed any more if we use
  a `PaperBoi.ppt: PpTable` which persists with the engine actor's
  lifetime.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 1560330acd Convert `Flume.MktPair.size_tick` to float for dark clearing 2023-05-09 14:49:26 -04:00
Tyler Goodlet a74caa9f77 Add paper engine "offline loading" support to the ledger cli 2023-05-09 14:49:26 -04:00
Tyler Goodlet 61fb783c4e Formalize a ledger type + API: `TransactionLedger`
Add a new `class TransactionLedger(collections.UserDict)` for managing
ledger (files) from a `dict`-like API. The main motivations being easy
conversion between `dict` <-> `Transaction` obj forms as well as dynamic
(toml) file updates via a set of methods:

- `.write_config()` to render and write state to the local toml file.
- `.iter_trans()` to allow iterator style conversion to `Transaction`
  form for each entry.
- `.to_trans()` for the dict output from the above.

Some adjustments to `Transaction` namely making `.sym/.sys` optional for
now so that paper engine entries can be loaded (offline) without
connecting to the emulated broker backend. Move to using `pathlib.Path`
throughout for bootyful toml file mgmt B)
2023-05-09 14:49:26 -04:00
Tyler Goodlet 9f7aa3d1ff Always use the "most resolved" `Position.symbol: MktPair`
When loading a `Position` from a pps file we might not have the entire
`MktPair` field-set loaded (though going forward that shouldn't really
ever happen except in the case of a legacy `pps.toml`), in which case we
can check if the `.fqme: str` value loaded from the transaction is
longer and use that instead - presuming it must have more mkt meta-data
filled out.

Also includes some more `fqsn` -> `fqme` renames.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 50be10a9bd `ib`: keep broker name in `Transaction.fqsn` 2023-05-09 14:49:26 -04:00
Tyler Goodlet 29a5910b90 `ib`: move flex utils to new submod 2023-05-09 14:49:26 -04:00
Tyler Goodlet a336def65f `ib`: again, only *update* ledger records from API 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2cb59fe450 Flatter format for pos/ledger mngr statements 2023-05-09 14:49:26 -04:00
Tyler Goodlet 4494acbc01 Write a separate `pps.<brokername>.<accountname>.toml` file per account 2023-05-09 14:49:26 -04:00
Tyler Goodlet 7b3d724908 Rework `.config` routines to use `pathlib.Path`
Been meaning to do this port for a while and since it makes passing
around file handles (presumably alongside the in mem obj form) a lot
simpler/nicer and the implementations of all the config file handling
much more terse with less presumptions about the form of filename/dir
`str` values all over the place B)

moar technically, let's us:
- drop remaining `.config` usage of `os.path`.
- return `Path`s from most routines.
- adds a special case to `get_conf_path()` such that if the input name
  contains a `pps.` pattern, we avoid validating the name; this is going
  to be used by new `.accounting.open_pps()` code which will instead
  write a separate TOML file for each account B)
2023-05-09 14:49:26 -04:00
Tyler Goodlet bc249fbeca Move `.clearing._allocate` -> `accounting._allocate` 2023-05-09 14:49:26 -04:00
Tyler Goodlet 53c76d3680 Drop `Optional` use from daemon mod 2023-05-09 14:49:26 -04:00
Tyler Goodlet 60123066e1 Use our `@acm` alias in paper eng 2023-05-09 14:49:26 -04:00
Tyler Goodlet 29ad20bc63 `ib`: only process ledger-txs once per client
Previous we were re-processing all ledgers for every position msg
received from the API, per client.. Instead do that once in a first pass
and drop all key-miss lookups for `bs_mktid`s; it should never happen.

Better typing for in-routine vars, convert pos msg/objects to `dict`
prior to logging so it's sane to read on console. Skip processing
specifically option contracts for now.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 978c59f5f0 `ib`: break up data vs. broker enabled modules 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2c23bc166b First working `brokerd` -> `trades_dialogue()` ep loader 2023-05-09 14:49:26 -04:00
Tyler Goodlet ff285fbbda `binance`: adjust earch to expect `Pair`s 2023-05-09 14:49:26 -04:00
Tyler Goodlet ccfafeeec2 Drop `cryptofeed`, what a mess XD 2023-05-09 14:49:26 -04:00
Tyler Goodlet e0067a4e1d WIP: trying out `typer` for ledger cli 2023-05-09 14:49:26 -04:00
Tyler Goodlet 485a17af26 Drop weird extra line from license headers 2023-05-09 14:49:26 -04:00
Tyler Goodlet c5b172a7df `binance`: pre-process `Pair` filters at init
Allows us to keep the struct frozen as well avoid complexity in the pure
data type. Also changes `.price/size_tick` to plain ol' properties.
2023-05-09 14:49:26 -04:00
Tyler Goodlet b718b5634e `binance`: use `MktPair` in live feed setup
Turns out `binance` is pretty great with their schema  since they have
more or less the same data schema for their exchange info ep which we
wrap in a `Pair` struct:
https://binance-docs.github.io/apidocs/spot/en/#exchange-information

That makes it super easy to provide the most general case for filling
out a `MktPair` with both `.src/dst: Asset` to maintain maximum
meta-data B)

Deatz:
- adjust `Pair` to have `.size/price_tick: Decimal` by parsing out
  the values from the filters field; TODO: we should probably just rewrite
  the input `.filter` at init time so we can keep the frozen style.
- rename `Client.mkt_info()` (was `.symbol_info` to `.exch_info()`
  better matching the ep name and have it build, cache, and return
  a `dict[str, Pair]`; allows dropping `.cache_symbols()`
- only pass the `mkt_info: MktPair` field in the init msg!
2023-05-09 14:49:26 -04:00
Tyler Goodlet 8f79c37b99 Generalize `MktPair.from_msg()` handling
Accept a msg with any of:
- `.src: Asset` and `.dst: Asset`
- `.src: str` and `.dst: str`
- `.src: Asset` and `.dst: str`

but not the final combo tho XD
Also, fix `.key` to properly cast any `.src: Asset` to string!
2023-05-09 14:49:26 -04:00
Tyler Goodlet aa5f25231a `ib`: never override existing ledger records
If user has loaded from a flex report then we don't want the API records
from the same period to override those; instead just update with any
missing fields from the API schema.

Also, always `str`-ify the contract id (what is set for the `.bs_mktid`
*before* packing into transaction type to ensure when serialized to
`pps.toml` there are no discrepancies at the codec level.. smh
2023-05-09 14:49:26 -04:00
Tyler Goodlet f3049016d6 `ib`: drop use of `_account2clients` in `load_clients_for_trio()`
Instead adjust `load_aio_clients()` to only reload clients detected as
non-loaded or disconnected (2 birds), and avoid use of the global module
table which could result in stale disconnected clients persisting on
multiple `brokerd` client reconnects, resulting in error.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 16e11d447c Move toml table decoder to separate mod 2023-05-09 14:49:26 -04:00
Tyler Goodlet 199a5e8b38 `ib`: stick exc handler around client connection erros 2023-05-09 14:49:26 -04:00
Tyler Goodlet 59b095b2d5 `kraken`: heh, use `trio_util` for trades streamz tooo XD 2023-05-09 14:49:26 -04:00
Tyler Goodlet c59ec77d9c WIP: refactor ib pp load init 2023-05-09 14:49:26 -04:00
Tyler Goodlet 3e5da64571 Cache contract lookups from `Client.get_con()` 2023-05-09 14:49:26 -04:00
Tyler Goodlet 1c576d72d1 Dump `Position`s as pformatted dicts for now.. 2023-05-09 14:49:26 -04:00
Tyler Goodlet ea42f66b54 Use common `.brokers` logger in most backends 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2ae9576cd8 Add common logger instance for `.brokers` 2023-05-09 14:49:26 -04:00
Tyler Goodlet a462de6f2d Use a single log for entire `.service` subsys 2023-05-09 14:49:26 -04:00
Tyler Goodlet 3bf48ab597 Use a single log for entire `.clearing` subsys 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2454dda18f Use `MktPair` attr `.size_tick` in charting 2023-05-09 14:49:26 -04:00
Tyler Goodlet 7498cbb5f4 Use `Struct.copy()` with update dict for `Order` from staged 2023-05-09 14:49:25 -04:00
Tyler Goodlet 581782800d Rename `Client.send_update()` -> `.update_nowait()` 2023-05-09 14:49:25 -04:00
Tyler Goodlet 069466218e Use `str(cmd.symbol)` for fqme on cancels, add `_nowait()` method names 2023-05-09 14:49:25 -04:00
Tyler Goodlet fd9e484b55 Add `.__str__()` to mktpair and symbol types, fix `MktPair.fqme` token order 2023-05-09 14:49:25 -04:00
Tyler Goodlet 406565f74d Rename `fqsn` -> `fqme` in paper engine 2023-05-09 14:49:25 -04:00
Tyler Goodlet 6272cae8d4 Drop more `Optional` usage on our `Struct` 2023-05-09 14:49:25 -04:00
Tyler Goodlet dc2332c980 '`kraken`: finally, use new `MktPair` in `'mkt_info'` init msg field!' 2023-05-09 14:49:25 -04:00
Tyler Goodlet 7be85a882b Drop use of legacy `Symbol.broker_info` in display startup 2023-05-09 14:49:25 -04:00