Commit Graph

3613 Commits (f46a5337d592fd632640f22f9bc0b579e62ea441)

Author SHA1 Message Date
Tyler Goodlet f46a5337d5 Convert `Flume.MktPair.size_tick` to float for dark clearing 2023-04-14 15:58:54 -04:00
Tyler Goodlet 5c61055411 Add paper engine "offline loading" support to the ledger cli 2023-04-14 15:58:54 -04:00
Tyler Goodlet 2f31e40d3b Formalize a ledger type + API: `TransactionLedger`
Add a new `class TransactionLedger(collections.UserDict)` for managing
ledger (files) from a `dict`-like API. The main motivations being easy
conversion between `dict` <-> `Transaction` obj forms as well as dynamic
(toml) file updates via a set of methods:

- `.write_config()` to render and write state to the local toml file.
- `.iter_trans()` to allow iterator style conversion to `Transaction`
  form for each entry.
- `.to_trans()` for the dict output from the above.

Some adjustments to `Transaction` namely making `.sym/.sys` optional for
now so that paper engine entries can be loaded (offline) without
connecting to the emulated broker backend. Move to using `pathlib.Path`
throughout for bootyful toml file mgmt B)
2023-04-14 15:58:54 -04:00
Tyler Goodlet 055025c64c Always use the "most resolved" `Position.symbol: MktPair`
When loading a `Position` from a pps file we might not have the entire
`MktPair` field-set loaded (though going forward that shouldn't really
ever happen except in the case of a legacy `pps.toml`), in which case we
can check if the `.fqme: str` value loaded from the transaction is
longer and use that instead - presuming it must have more mkt meta-data
filled out.

Also includes some more `fqsn` -> `fqme` renames.
2023-04-14 15:58:54 -04:00
Tyler Goodlet 4631af60e9 `ib`: keep broker name in `Transaction.fqsn` 2023-04-14 15:58:54 -04:00
Tyler Goodlet ba31f2dfb3 `ib`: move flex utils to new submod 2023-04-14 15:58:54 -04:00
Tyler Goodlet 926373dd33 `ib`: again, only *update* ledger records from API 2023-04-14 15:58:54 -04:00
Tyler Goodlet 0ea806c7bd Flatter format for pos/ledger mngr statements 2023-04-14 15:58:54 -04:00
Tyler Goodlet 68437fb5af Write a separate `pps.<brokername>.<accountname>.toml` file per account 2023-04-14 15:58:53 -04:00
Tyler Goodlet 14418b4c5c Rework `.config` routines to use `pathlib.Path`
Been meaning to do this port for a while and since it makes passing
around file handles (presumably alongside the in mem obj form) a lot
simpler/nicer and the implementations of all the config file handling
much more terse with less presumptions about the form of filename/dir
`str` values all over the place B)

moar technically, let's us:
- drop remaining `.config` usage of `os.path`.
- return `Path`s from most routines.
- adds a special case to `get_conf_path()` such that if the input name
  contains a `pps.` pattern, we avoid validating the name; this is going
  to be used by new `.accounting.open_pps()` code which will instead
  write a separate TOML file for each account B)
2023-04-14 15:58:53 -04:00
Tyler Goodlet 10ae9106e2 Move `.clearing._allocate` -> `accounting._allocate` 2023-04-14 15:58:53 -04:00
Tyler Goodlet d644436a3c Drop `Optional` use from daemon mod 2023-04-14 15:58:53 -04:00
Tyler Goodlet 7c3418def6 Use our `@acm` alias in paper eng 2023-04-14 15:58:53 -04:00
Tyler Goodlet c3686185c1 `ib`: only process ledger-txs once per client
Previous we were re-processing all ledgers for every position msg
received from the API, per client.. Instead do that once in a first pass
and drop all key-miss lookups for `bs_mktid`s; it should never happen.

Better typing for in-routine vars, convert pos msg/objects to `dict`
prior to logging so it's sane to read on console. Skip processing
specifically option contracts for now.
2023-04-14 15:58:53 -04:00
Tyler Goodlet f0d181e3f7 `ib`: break up data vs. broker enabled modules 2023-04-14 15:58:53 -04:00
Tyler Goodlet 312c4cdec7 First working `brokerd` -> `trades_dialogue()` ep loader 2023-04-14 15:58:53 -04:00
Tyler Goodlet f234483a1f `binance`: adjust earch to expect `Pair`s 2023-04-14 15:58:53 -04:00
Tyler Goodlet 3c32c9b3c9 Drop `cryptofeed`, what a mess XD 2023-04-14 15:58:53 -04:00
Tyler Goodlet 2675039b16 WIP: trying out `typer` for ledger cli 2023-04-14 15:58:53 -04:00
Tyler Goodlet 5919d75e85 Drop weird extra line from license headers 2023-04-14 15:58:53 -04:00
Tyler Goodlet 62a40c57a0 `binance`: pre-process `Pair` filters at init
Allows us to keep the struct frozen as well avoid complexity in the pure
data type. Also changes `.price/size_tick` to plain ol' properties.
2023-04-14 15:58:53 -04:00
Tyler Goodlet a50452dbfd `binance`: use `MktPair` in live feed setup
Turns out `binance` is pretty great with their schema  since they have
more or less the same data schema for their exchange info ep which we
wrap in a `Pair` struct:
https://binance-docs.github.io/apidocs/spot/en/#exchange-information

That makes it super easy to provide the most general case for filling
out a `MktPair` with both `.src/dst: Asset` to maintain maximum
meta-data B)

Deatz:
- adjust `Pair` to have `.size/price_tick: Decimal` by parsing out
  the values from the filters field; TODO: we should probably just rewrite
  the input `.filter` at init time so we can keep the frozen style.
- rename `Client.mkt_info()` (was `.symbol_info` to `.exch_info()`
  better matching the ep name and have it build, cache, and return
  a `dict[str, Pair]`; allows dropping `.cache_symbols()`
- only pass the `mkt_info: MktPair` field in the init msg!
2023-04-14 15:58:53 -04:00
Tyler Goodlet 2142c13228 Generalize `MktPair.from_msg()` handling
Accept a msg with any of:
- `.src: Asset` and `.dst: Asset`
- `.src: str` and `.dst: str`
- `.src: Asset` and `.dst: str`

but not the final combo tho XD
Also, fix `.key` to properly cast any `.src: Asset` to string!
2023-04-14 15:58:53 -04:00
Tyler Goodlet 4236e5c3b1 `ib`: never override existing ledger records
If user has loaded from a flex report then we don't want the API records
from the same period to override those; instead just update with any
missing fields from the API schema.

Also, always `str`-ify the contract id (what is set for the `.bs_mktid`
*before* packing into transaction type to ensure when serialized to
`pps.toml` there are no discrepancies at the codec level.. smh
2023-04-14 15:58:53 -04:00
Tyler Goodlet 366de901df `ib`: drop use of `_account2clients` in `load_clients_for_trio()`
Instead adjust `load_aio_clients()` to only reload clients detected as
non-loaded or disconnected (2 birds), and avoid use of the global module
table which could result in stale disconnected clients persisting on
multiple `brokerd` client reconnects, resulting in error.
2023-04-14 15:58:53 -04:00
Tyler Goodlet 8ee3fc4aa5 Move toml table decoder to separate mod 2023-04-14 15:58:53 -04:00
Tyler Goodlet d9344aa468 `ib`: stick exc handler around client connection erros 2023-04-14 15:58:53 -04:00
Tyler Goodlet 49958e68ea `kraken`: heh, use `trio_util` for trades streamz tooo XD 2023-04-14 15:58:53 -04:00
Tyler Goodlet 116f7fd40f WIP: refactor ib pp load init 2023-04-14 15:58:53 -04:00
Tyler Goodlet 8d7b968c44 Cache contract lookups from `Client.get_con()` 2023-04-14 15:58:53 -04:00
Tyler Goodlet 5f89ec4feb Dump `Position`s as pformatted dicts for now.. 2023-04-14 15:58:53 -04:00
Tyler Goodlet f783d3eba3 Use common `.brokers` logger in most backends 2023-04-14 15:58:53 -04:00
Tyler Goodlet a72a9e76e9 Add common logger instance for `.brokers` 2023-04-14 15:58:53 -04:00
Tyler Goodlet c412197413 Use a single log for entire `.service` subsys 2023-04-14 15:58:53 -04:00
Tyler Goodlet 0930074e76 Use a single log for entire `.clearing` subsys 2023-04-14 15:58:53 -04:00
Tyler Goodlet ca752bea8c Use `MktPair` attr `.size_tick` in charting 2023-04-14 15:58:53 -04:00
Tyler Goodlet 963523359c Use `Struct.copy()` with update dict for `Order` from staged 2023-04-14 15:58:53 -04:00
Tyler Goodlet 3db9854271 Rename `Client.send_update()` -> `.update_nowait()` 2023-04-14 15:58:53 -04:00
Tyler Goodlet b1e241781e Use `str(cmd.symbol)` for fqme on cancels, add `_nowait()` method names 2023-04-14 15:58:53 -04:00
Tyler Goodlet 71019ad54b Add `.__str__()` to mktpair and symbol types, fix `MktPair.fqme` token order 2023-04-14 15:58:53 -04:00
Tyler Goodlet 2a1d485cd1 Rename `fqsn` -> `fqme` in paper engine 2023-04-14 15:58:53 -04:00
Tyler Goodlet f7f2d1247b Drop more `Optional` usage on our `Struct` 2023-04-14 15:58:53 -04:00
Tyler Goodlet 8594a39c61 '`kraken`: finally, use new `MktPair` in `'mkt_info'` init msg field!' 2023-04-14 15:58:53 -04:00
Tyler Goodlet 08b4d69a91 Drop use of legacy `Symbol.broker_info` in display startup 2023-04-14 15:58:53 -04:00
Tyler Goodlet 0aacb835dc Typecast `OrderMode.staged.symbol: str` before `.copy()`! 2023-04-14 15:58:53 -04:00
Tyler Goodlet 4447f45430 `kraken`: parse our source asset key and set on `MktPair.src: str` 2023-04-14 15:58:53 -04:00
Tyler Goodlet 0e0338e217 Always cast `Order.symbol: str` for now
To make nested `msgspec.Struct`s work we need to tell the codec that the
`.symbol` is some struct def, since we don't really need to enforce that
(yet) we're just going to enc/dec as `str` until we further formalize
and/or need something more complex.
2023-04-14 15:58:53 -04:00
Tyler Goodlet ef2d0c46d4 Expect new `MktPair.tick_size: Decimal` attr in ems 2023-04-14 15:58:53 -04:00
Tyler Goodlet 986bb4e7c8 Use `MktPair` for `Flume.symbol` when used by backend
Initial attempt at getting the sampling and shm layer to use the new mkt
info meta-data type. Draft out a potential `BackendInitMsg:
msgspec.Struct` for validating the init msg returned from the
`stream_quotes()` start value; obvs don't actually use it yet.
2023-04-14 15:58:53 -04:00
Tyler Goodlet 76fe9018cf `.clearing`: broad rename of `fqsn` -> `fqme` 2023-04-14 15:58:53 -04:00