Commit Graph

3657 Commits (114e7660aa44397e09ab06876fa429187bf82588)

Author SHA1 Message Date
Tyler Goodlet 926373dd33 `ib`: again, only *update* ledger records from API 2023-04-14 15:58:54 -04:00
Tyler Goodlet 0ea806c7bd Flatter format for pos/ledger mngr statements 2023-04-14 15:58:54 -04:00
Tyler Goodlet 68437fb5af Write a separate `pps.<brokername>.<accountname>.toml` file per account 2023-04-14 15:58:53 -04:00
Tyler Goodlet 14418b4c5c Rework `.config` routines to use `pathlib.Path`
Been meaning to do this port for a while and since it makes passing
around file handles (presumably alongside the in mem obj form) a lot
simpler/nicer and the implementations of all the config file handling
much more terse with less presumptions about the form of filename/dir
`str` values all over the place B)

moar technically, let's us:
- drop remaining `.config` usage of `os.path`.
- return `Path`s from most routines.
- adds a special case to `get_conf_path()` such that if the input name
  contains a `pps.` pattern, we avoid validating the name; this is going
  to be used by new `.accounting.open_pps()` code which will instead
  write a separate TOML file for each account B)
2023-04-14 15:58:53 -04:00
Tyler Goodlet 10ae9106e2 Move `.clearing._allocate` -> `accounting._allocate` 2023-04-14 15:58:53 -04:00
Tyler Goodlet d644436a3c Drop `Optional` use from daemon mod 2023-04-14 15:58:53 -04:00
Tyler Goodlet 7c3418def6 Use our `@acm` alias in paper eng 2023-04-14 15:58:53 -04:00
Tyler Goodlet c3686185c1 `ib`: only process ledger-txs once per client
Previous we were re-processing all ledgers for every position msg
received from the API, per client.. Instead do that once in a first pass
and drop all key-miss lookups for `bs_mktid`s; it should never happen.

Better typing for in-routine vars, convert pos msg/objects to `dict`
prior to logging so it's sane to read on console. Skip processing
specifically option contracts for now.
2023-04-14 15:58:53 -04:00
Tyler Goodlet f0d181e3f7 `ib`: break up data vs. broker enabled modules 2023-04-14 15:58:53 -04:00
Tyler Goodlet 312c4cdec7 First working `brokerd` -> `trades_dialogue()` ep loader 2023-04-14 15:58:53 -04:00
Tyler Goodlet f234483a1f `binance`: adjust earch to expect `Pair`s 2023-04-14 15:58:53 -04:00
Tyler Goodlet 3c32c9b3c9 Drop `cryptofeed`, what a mess XD 2023-04-14 15:58:53 -04:00
Tyler Goodlet 2675039b16 WIP: trying out `typer` for ledger cli 2023-04-14 15:58:53 -04:00
Tyler Goodlet 5919d75e85 Drop weird extra line from license headers 2023-04-14 15:58:53 -04:00
Tyler Goodlet 62a40c57a0 `binance`: pre-process `Pair` filters at init
Allows us to keep the struct frozen as well avoid complexity in the pure
data type. Also changes `.price/size_tick` to plain ol' properties.
2023-04-14 15:58:53 -04:00
Tyler Goodlet a50452dbfd `binance`: use `MktPair` in live feed setup
Turns out `binance` is pretty great with their schema  since they have
more or less the same data schema for their exchange info ep which we
wrap in a `Pair` struct:
https://binance-docs.github.io/apidocs/spot/en/#exchange-information

That makes it super easy to provide the most general case for filling
out a `MktPair` with both `.src/dst: Asset` to maintain maximum
meta-data B)

Deatz:
- adjust `Pair` to have `.size/price_tick: Decimal` by parsing out
  the values from the filters field; TODO: we should probably just rewrite
  the input `.filter` at init time so we can keep the frozen style.
- rename `Client.mkt_info()` (was `.symbol_info` to `.exch_info()`
  better matching the ep name and have it build, cache, and return
  a `dict[str, Pair]`; allows dropping `.cache_symbols()`
- only pass the `mkt_info: MktPair` field in the init msg!
2023-04-14 15:58:53 -04:00
Tyler Goodlet 2142c13228 Generalize `MktPair.from_msg()` handling
Accept a msg with any of:
- `.src: Asset` and `.dst: Asset`
- `.src: str` and `.dst: str`
- `.src: Asset` and `.dst: str`

but not the final combo tho XD
Also, fix `.key` to properly cast any `.src: Asset` to string!
2023-04-14 15:58:53 -04:00
Tyler Goodlet 4236e5c3b1 `ib`: never override existing ledger records
If user has loaded from a flex report then we don't want the API records
from the same period to override those; instead just update with any
missing fields from the API schema.

Also, always `str`-ify the contract id (what is set for the `.bs_mktid`
*before* packing into transaction type to ensure when serialized to
`pps.toml` there are no discrepancies at the codec level.. smh
2023-04-14 15:58:53 -04:00
Tyler Goodlet 366de901df `ib`: drop use of `_account2clients` in `load_clients_for_trio()`
Instead adjust `load_aio_clients()` to only reload clients detected as
non-loaded or disconnected (2 birds), and avoid use of the global module
table which could result in stale disconnected clients persisting on
multiple `brokerd` client reconnects, resulting in error.
2023-04-14 15:58:53 -04:00
Tyler Goodlet 8ee3fc4aa5 Move toml table decoder to separate mod 2023-04-14 15:58:53 -04:00
Tyler Goodlet d9344aa468 `ib`: stick exc handler around client connection erros 2023-04-14 15:58:53 -04:00
Tyler Goodlet 49958e68ea `kraken`: heh, use `trio_util` for trades streamz tooo XD 2023-04-14 15:58:53 -04:00
Tyler Goodlet 116f7fd40f WIP: refactor ib pp load init 2023-04-14 15:58:53 -04:00
Tyler Goodlet 8d7b968c44 Cache contract lookups from `Client.get_con()` 2023-04-14 15:58:53 -04:00
Tyler Goodlet 5f89ec4feb Dump `Position`s as pformatted dicts for now.. 2023-04-14 15:58:53 -04:00
Tyler Goodlet f783d3eba3 Use common `.brokers` logger in most backends 2023-04-14 15:58:53 -04:00
Tyler Goodlet a72a9e76e9 Add common logger instance for `.brokers` 2023-04-14 15:58:53 -04:00
Tyler Goodlet c412197413 Use a single log for entire `.service` subsys 2023-04-14 15:58:53 -04:00
Tyler Goodlet 0930074e76 Use a single log for entire `.clearing` subsys 2023-04-14 15:58:53 -04:00
Tyler Goodlet ca752bea8c Use `MktPair` attr `.size_tick` in charting 2023-04-14 15:58:53 -04:00
Tyler Goodlet 963523359c Use `Struct.copy()` with update dict for `Order` from staged 2023-04-14 15:58:53 -04:00
Tyler Goodlet 3db9854271 Rename `Client.send_update()` -> `.update_nowait()` 2023-04-14 15:58:53 -04:00
Tyler Goodlet b1e241781e Use `str(cmd.symbol)` for fqme on cancels, add `_nowait()` method names 2023-04-14 15:58:53 -04:00
Tyler Goodlet 71019ad54b Add `.__str__()` to mktpair and symbol types, fix `MktPair.fqme` token order 2023-04-14 15:58:53 -04:00
Tyler Goodlet 2a1d485cd1 Rename `fqsn` -> `fqme` in paper engine 2023-04-14 15:58:53 -04:00
Tyler Goodlet f7f2d1247b Drop more `Optional` usage on our `Struct` 2023-04-14 15:58:53 -04:00
Tyler Goodlet 8594a39c61 '`kraken`: finally, use new `MktPair` in `'mkt_info'` init msg field!' 2023-04-14 15:58:53 -04:00
Tyler Goodlet 08b4d69a91 Drop use of legacy `Symbol.broker_info` in display startup 2023-04-14 15:58:53 -04:00
Tyler Goodlet 0aacb835dc Typecast `OrderMode.staged.symbol: str` before `.copy()`! 2023-04-14 15:58:53 -04:00
Tyler Goodlet 4447f45430 `kraken`: parse our source asset key and set on `MktPair.src: str` 2023-04-14 15:58:53 -04:00
Tyler Goodlet 0e0338e217 Always cast `Order.symbol: str` for now
To make nested `msgspec.Struct`s work we need to tell the codec that the
`.symbol` is some struct def, since we don't really need to enforce that
(yet) we're just going to enc/dec as `str` until we further formalize
and/or need something more complex.
2023-04-14 15:58:53 -04:00
Tyler Goodlet ef2d0c46d4 Expect new `MktPair.tick_size: Decimal` attr in ems 2023-04-14 15:58:53 -04:00
Tyler Goodlet 986bb4e7c8 Use `MktPair` for `Flume.symbol` when used by backend
Initial attempt at getting the sampling and shm layer to use the new mkt
info meta-data type. Draft out a potential `BackendInitMsg:
msgspec.Struct` for validating the init msg returned from the
`stream_quotes()` start value; obvs don't actually use it yet.
2023-04-14 15:58:53 -04:00
Tyler Goodlet 76fe9018cf `.clearing`: broad rename of `fqsn` -> `fqme` 2023-04-14 15:58:53 -04:00
Tyler Goodlet d7c1e5e188 Add `MktPair.suffix: str` read from contract info
To be compat with the `Symbol` (for now) and generally allow for reading
the (derivative) contract specific part of the fqme. Adjust
`contract_info: list[str]` and make `src: str = ''` by default.
2023-04-14 15:58:53 -04:00
Tyler Goodlet 52de60c7ee Optionally load `MktPair` in `Flume`s 2023-04-14 15:58:47 -04:00
Tyler Goodlet 8b0aead72e First stage port of `.data.feed` to `MktPair`
Add `MktPair` handling block for when a backend delivers
a `mkt_info`-field containing init msg. Adjust the original
`Symbol`-style `'symbol_info'` msg processing to do `Decimal` defaults
and convert to `MktPair` including slapping in a hacky `_atype: str`
field XD

General initial name changes to `bs_mktid` and `_fqme` throughout!
2023-04-14 15:58:47 -04:00
Tyler Goodlet da7371988a Comment about `Struct.typecast()` conflict with frozen instances 2023-04-14 15:58:47 -04:00
Tyler Goodlet 0866d46484 Default `pps.toml` precision fields to `Decimal`
For `price_tick` and `size_tick` we read in `str` and decimal-ize
and now correctly fail over to default values of the same type..
Also, always treat `bs_mktid` field as a `str` in TOML form.

Drop the strange `clears: dict` var from the loading code (not sure why
that was left in smh) and better name `toml_clears_list` for the
TOML-loaded-pre-transaction sequence.
2023-04-14 15:58:47 -04:00
Tyler Goodlet 98d8b4a0e8 Implement `MktPair.from_msg()` constructor
Handle case where `'dst'` field is just a `str` (in which case delegate to
`.from_fqme()`) as well as do `Asset` loading and use our
`Struct.copy()` to enforce type-casting to (for eg. `Decimal`s) such
that we'll now capture typing errors despite IPC transport.

Change `Symbol.tick_size` and `.lot_tick_size` defaults to decimal
for proper casting and type `MktPair.atype: str` since `msgspec` can't
cast to `AssetTypeName` without special handling..
2023-04-14 15:58:47 -04:00