Commit Graph

3890 Commits (5e2107ff15d9965cd3e4bfa7391a91c94be1d179)

Author SHA1 Message Date
Tyler Goodlet 7b3d724908 Rework `.config` routines to use `pathlib.Path`
Been meaning to do this port for a while and since it makes passing
around file handles (presumably alongside the in mem obj form) a lot
simpler/nicer and the implementations of all the config file handling
much more terse with less presumptions about the form of filename/dir
`str` values all over the place B)

moar technically, let's us:
- drop remaining `.config` usage of `os.path`.
- return `Path`s from most routines.
- adds a special case to `get_conf_path()` such that if the input name
  contains a `pps.` pattern, we avoid validating the name; this is going
  to be used by new `.accounting.open_pps()` code which will instead
  write a separate TOML file for each account B)
2023-05-09 14:49:26 -04:00
Tyler Goodlet bc249fbeca Move `.clearing._allocate` -> `accounting._allocate` 2023-05-09 14:49:26 -04:00
Tyler Goodlet 53c76d3680 Drop `Optional` use from daemon mod 2023-05-09 14:49:26 -04:00
Tyler Goodlet 60123066e1 Use our `@acm` alias in paper eng 2023-05-09 14:49:26 -04:00
Tyler Goodlet 29ad20bc63 `ib`: only process ledger-txs once per client
Previous we were re-processing all ledgers for every position msg
received from the API, per client.. Instead do that once in a first pass
and drop all key-miss lookups for `bs_mktid`s; it should never happen.

Better typing for in-routine vars, convert pos msg/objects to `dict`
prior to logging so it's sane to read on console. Skip processing
specifically option contracts for now.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 978c59f5f0 `ib`: break up data vs. broker enabled modules 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2c23bc166b First working `brokerd` -> `trades_dialogue()` ep loader 2023-05-09 14:49:26 -04:00
Tyler Goodlet ff285fbbda `binance`: adjust earch to expect `Pair`s 2023-05-09 14:49:26 -04:00
Tyler Goodlet ccfafeeec2 Drop `cryptofeed`, what a mess XD 2023-05-09 14:49:26 -04:00
Tyler Goodlet e0067a4e1d WIP: trying out `typer` for ledger cli 2023-05-09 14:49:26 -04:00
Tyler Goodlet 485a17af26 Drop weird extra line from license headers 2023-05-09 14:49:26 -04:00
Tyler Goodlet c5b172a7df `binance`: pre-process `Pair` filters at init
Allows us to keep the struct frozen as well avoid complexity in the pure
data type. Also changes `.price/size_tick` to plain ol' properties.
2023-05-09 14:49:26 -04:00
Tyler Goodlet b718b5634e `binance`: use `MktPair` in live feed setup
Turns out `binance` is pretty great with their schema  since they have
more or less the same data schema for their exchange info ep which we
wrap in a `Pair` struct:
https://binance-docs.github.io/apidocs/spot/en/#exchange-information

That makes it super easy to provide the most general case for filling
out a `MktPair` with both `.src/dst: Asset` to maintain maximum
meta-data B)

Deatz:
- adjust `Pair` to have `.size/price_tick: Decimal` by parsing out
  the values from the filters field; TODO: we should probably just rewrite
  the input `.filter` at init time so we can keep the frozen style.
- rename `Client.mkt_info()` (was `.symbol_info` to `.exch_info()`
  better matching the ep name and have it build, cache, and return
  a `dict[str, Pair]`; allows dropping `.cache_symbols()`
- only pass the `mkt_info: MktPair` field in the init msg!
2023-05-09 14:49:26 -04:00
Tyler Goodlet 8f79c37b99 Generalize `MktPair.from_msg()` handling
Accept a msg with any of:
- `.src: Asset` and `.dst: Asset`
- `.src: str` and `.dst: str`
- `.src: Asset` and `.dst: str`

but not the final combo tho XD
Also, fix `.key` to properly cast any `.src: Asset` to string!
2023-05-09 14:49:26 -04:00
Tyler Goodlet aa5f25231a `ib`: never override existing ledger records
If user has loaded from a flex report then we don't want the API records
from the same period to override those; instead just update with any
missing fields from the API schema.

Also, always `str`-ify the contract id (what is set for the `.bs_mktid`
*before* packing into transaction type to ensure when serialized to
`pps.toml` there are no discrepancies at the codec level.. smh
2023-05-09 14:49:26 -04:00
Tyler Goodlet f3049016d6 `ib`: drop use of `_account2clients` in `load_clients_for_trio()`
Instead adjust `load_aio_clients()` to only reload clients detected as
non-loaded or disconnected (2 birds), and avoid use of the global module
table which could result in stale disconnected clients persisting on
multiple `brokerd` client reconnects, resulting in error.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 16e11d447c Move toml table decoder to separate mod 2023-05-09 14:49:26 -04:00
Tyler Goodlet 199a5e8b38 `ib`: stick exc handler around client connection erros 2023-05-09 14:49:26 -04:00
Tyler Goodlet 59b095b2d5 `kraken`: heh, use `trio_util` for trades streamz tooo XD 2023-05-09 14:49:26 -04:00
Tyler Goodlet c59ec77d9c WIP: refactor ib pp load init 2023-05-09 14:49:26 -04:00
Tyler Goodlet 3e5da64571 Cache contract lookups from `Client.get_con()` 2023-05-09 14:49:26 -04:00
Tyler Goodlet 1c576d72d1 Dump `Position`s as pformatted dicts for now.. 2023-05-09 14:49:26 -04:00
Tyler Goodlet ea42f66b54 Use common `.brokers` logger in most backends 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2ae9576cd8 Add common logger instance for `.brokers` 2023-05-09 14:49:26 -04:00
Tyler Goodlet a462de6f2d Use a single log for entire `.service` subsys 2023-05-09 14:49:26 -04:00
Tyler Goodlet 3bf48ab597 Use a single log for entire `.clearing` subsys 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2454dda18f Use `MktPair` attr `.size_tick` in charting 2023-05-09 14:49:26 -04:00
Tyler Goodlet 7498cbb5f4 Use `Struct.copy()` with update dict for `Order` from staged 2023-05-09 14:49:25 -04:00
Tyler Goodlet 581782800d Rename `Client.send_update()` -> `.update_nowait()` 2023-05-09 14:49:25 -04:00
Tyler Goodlet 069466218e Use `str(cmd.symbol)` for fqme on cancels, add `_nowait()` method names 2023-05-09 14:49:25 -04:00
Tyler Goodlet fd9e484b55 Add `.__str__()` to mktpair and symbol types, fix `MktPair.fqme` token order 2023-05-09 14:49:25 -04:00
Tyler Goodlet 406565f74d Rename `fqsn` -> `fqme` in paper engine 2023-05-09 14:49:25 -04:00
Tyler Goodlet 6272cae8d4 Drop more `Optional` usage on our `Struct` 2023-05-09 14:49:25 -04:00
Tyler Goodlet dc2332c980 '`kraken`: finally, use new `MktPair` in `'mkt_info'` init msg field!' 2023-05-09 14:49:25 -04:00
Tyler Goodlet 7be85a882b Drop use of legacy `Symbol.broker_info` in display startup 2023-05-09 14:49:25 -04:00
Tyler Goodlet b6df83a0e9 Typecast `OrderMode.staged.symbol: str` before `.copy()`! 2023-05-09 14:49:25 -04:00
Tyler Goodlet d62fb655eb `kraken`: parse our source asset key and set on `MktPair.src: str` 2023-05-09 14:49:25 -04:00
Tyler Goodlet a9778e4001 Always cast `Order.symbol: str` for now
To make nested `msgspec.Struct`s work we need to tell the codec that the
`.symbol` is some struct def, since we don't really need to enforce that
(yet) we're just going to enc/dec as `str` until we further formalize
and/or need something more complex.
2023-05-09 14:49:25 -04:00
Tyler Goodlet 580165f2f4 Expect new `MktPair.tick_size: Decimal` attr in ems 2023-05-09 14:49:25 -04:00
Tyler Goodlet 0f3041724b Use `MktPair` for `Flume.symbol` when used by backend
Initial attempt at getting the sampling and shm layer to use the new mkt
info meta-data type. Draft out a potential `BackendInitMsg:
msgspec.Struct` for validating the init msg returned from the
`stream_quotes()` start value; obvs don't actually use it yet.
2023-05-09 14:49:25 -04:00
Tyler Goodlet 1d08ee6d01 `.clearing`: broad rename of `fqsn` -> `fqme` 2023-05-09 14:49:25 -04:00
Tyler Goodlet d4a5a3057c Add `MktPair.suffix: str` read from contract info
To be compat with the `Symbol` (for now) and generally allow for reading
the (derivative) contract specific part of the fqme. Adjust
`contract_info: list[str]` and make `src: str = ''` by default.
2023-05-09 14:49:25 -04:00
Tyler Goodlet 452cd7db8a Optionally load `MktPair` in `Flume`s 2023-05-09 14:49:25 -04:00
Tyler Goodlet 2cc80d53ca First stage port of `.data.feed` to `MktPair`
Add `MktPair` handling block for when a backend delivers
a `mkt_info`-field containing init msg. Adjust the original
`Symbol`-style `'symbol_info'` msg processing to do `Decimal` defaults
and convert to `MktPair` including slapping in a hacky `_atype: str`
field XD

General initial name changes to `bs_mktid` and `_fqme` throughout!
2023-05-09 14:49:25 -04:00
Tyler Goodlet 7eb0b1d249 Comment about `Struct.typecast()` conflict with frozen instances 2023-05-09 14:49:25 -04:00
Tyler Goodlet 589b3f4201 Default `pps.toml` precision fields to `Decimal`
For `price_tick` and `size_tick` we read in `str` and decimal-ize
and now correctly fail over to default values of the same type..
Also, always treat `bs_mktid` field as a `str` in TOML form.

Drop the strange `clears: dict` var from the loading code (not sure why
that was left in smh) and better name `toml_clears_list` for the
TOML-loaded-pre-transaction sequence.
2023-05-09 14:49:25 -04:00
Tyler Goodlet 6d5d9731ed Implement `MktPair.from_msg()` constructor
Handle case where `'dst'` field is just a `str` (in which case delegate to
`.from_fqme()`) as well as do `Asset` loading and use our
`Struct.copy()` to enforce type-casting to (for eg. `Decimal`s) such
that we'll now capture typing errors despite IPC transport.

Change `Symbol.tick_size` and `.lot_tick_size` defaults to decimal
for proper casting and type `MktPair.atype: str` since `msgspec` can't
cast to `AssetTypeName` without special handling..
2023-05-09 14:49:25 -04:00
Tyler Goodlet 25363ebd2e `ib`: deliver mkt precision info as `Decimal` 2023-05-09 14:49:25 -04:00
Tyler Goodlet b9c7e1b0c7 `binance`: deliver mkt precision info as `Decimal` 2023-05-09 14:49:25 -04:00
Tyler Goodlet ea9ea4a6d7 Rename `float_digits()` -> `dec_digits()`, since decimal. 2023-05-09 14:49:25 -04:00