Commit Graph

3998 Commits (f8af13d0101ad3bd766296bcce1aa5ed19c0201d)

Author SHA1 Message Date
Tyler Goodlet a74caa9f77 Add paper engine "offline loading" support to the ledger cli 2023-05-09 14:49:26 -04:00
Tyler Goodlet 61fb783c4e Formalize a ledger type + API: `TransactionLedger`
Add a new `class TransactionLedger(collections.UserDict)` for managing
ledger (files) from a `dict`-like API. The main motivations being easy
conversion between `dict` <-> `Transaction` obj forms as well as dynamic
(toml) file updates via a set of methods:

- `.write_config()` to render and write state to the local toml file.
- `.iter_trans()` to allow iterator style conversion to `Transaction`
  form for each entry.
- `.to_trans()` for the dict output from the above.

Some adjustments to `Transaction` namely making `.sym/.sys` optional for
now so that paper engine entries can be loaded (offline) without
connecting to the emulated broker backend. Move to using `pathlib.Path`
throughout for bootyful toml file mgmt B)
2023-05-09 14:49:26 -04:00
Tyler Goodlet 9f7aa3d1ff Always use the "most resolved" `Position.symbol: MktPair`
When loading a `Position` from a pps file we might not have the entire
`MktPair` field-set loaded (though going forward that shouldn't really
ever happen except in the case of a legacy `pps.toml`), in which case we
can check if the `.fqme: str` value loaded from the transaction is
longer and use that instead - presuming it must have more mkt meta-data
filled out.

Also includes some more `fqsn` -> `fqme` renames.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 50be10a9bd `ib`: keep broker name in `Transaction.fqsn` 2023-05-09 14:49:26 -04:00
Tyler Goodlet 29a5910b90 `ib`: move flex utils to new submod 2023-05-09 14:49:26 -04:00
Tyler Goodlet a336def65f `ib`: again, only *update* ledger records from API 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2cb59fe450 Flatter format for pos/ledger mngr statements 2023-05-09 14:49:26 -04:00
Tyler Goodlet 4494acbc01 Write a separate `pps.<brokername>.<accountname>.toml` file per account 2023-05-09 14:49:26 -04:00
Tyler Goodlet 7b3d724908 Rework `.config` routines to use `pathlib.Path`
Been meaning to do this port for a while and since it makes passing
around file handles (presumably alongside the in mem obj form) a lot
simpler/nicer and the implementations of all the config file handling
much more terse with less presumptions about the form of filename/dir
`str` values all over the place B)

moar technically, let's us:
- drop remaining `.config` usage of `os.path`.
- return `Path`s from most routines.
- adds a special case to `get_conf_path()` such that if the input name
  contains a `pps.` pattern, we avoid validating the name; this is going
  to be used by new `.accounting.open_pps()` code which will instead
  write a separate TOML file for each account B)
2023-05-09 14:49:26 -04:00
Tyler Goodlet bc249fbeca Move `.clearing._allocate` -> `accounting._allocate` 2023-05-09 14:49:26 -04:00
Tyler Goodlet 53c76d3680 Drop `Optional` use from daemon mod 2023-05-09 14:49:26 -04:00
Tyler Goodlet 60123066e1 Use our `@acm` alias in paper eng 2023-05-09 14:49:26 -04:00
Tyler Goodlet 29ad20bc63 `ib`: only process ledger-txs once per client
Previous we were re-processing all ledgers for every position msg
received from the API, per client.. Instead do that once in a first pass
and drop all key-miss lookups for `bs_mktid`s; it should never happen.

Better typing for in-routine vars, convert pos msg/objects to `dict`
prior to logging so it's sane to read on console. Skip processing
specifically option contracts for now.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 978c59f5f0 `ib`: break up data vs. broker enabled modules 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2c23bc166b First working `brokerd` -> `trades_dialogue()` ep loader 2023-05-09 14:49:26 -04:00
Tyler Goodlet ff285fbbda `binance`: adjust earch to expect `Pair`s 2023-05-09 14:49:26 -04:00
Tyler Goodlet ccfafeeec2 Drop `cryptofeed`, what a mess XD 2023-05-09 14:49:26 -04:00
Tyler Goodlet e0067a4e1d WIP: trying out `typer` for ledger cli 2023-05-09 14:49:26 -04:00
Tyler Goodlet 485a17af26 Drop weird extra line from license headers 2023-05-09 14:49:26 -04:00
Tyler Goodlet c5b172a7df `binance`: pre-process `Pair` filters at init
Allows us to keep the struct frozen as well avoid complexity in the pure
data type. Also changes `.price/size_tick` to plain ol' properties.
2023-05-09 14:49:26 -04:00
Tyler Goodlet b718b5634e `binance`: use `MktPair` in live feed setup
Turns out `binance` is pretty great with their schema  since they have
more or less the same data schema for their exchange info ep which we
wrap in a `Pair` struct:
https://binance-docs.github.io/apidocs/spot/en/#exchange-information

That makes it super easy to provide the most general case for filling
out a `MktPair` with both `.src/dst: Asset` to maintain maximum
meta-data B)

Deatz:
- adjust `Pair` to have `.size/price_tick: Decimal` by parsing out
  the values from the filters field; TODO: we should probably just rewrite
  the input `.filter` at init time so we can keep the frozen style.
- rename `Client.mkt_info()` (was `.symbol_info` to `.exch_info()`
  better matching the ep name and have it build, cache, and return
  a `dict[str, Pair]`; allows dropping `.cache_symbols()`
- only pass the `mkt_info: MktPair` field in the init msg!
2023-05-09 14:49:26 -04:00
Tyler Goodlet 8f79c37b99 Generalize `MktPair.from_msg()` handling
Accept a msg with any of:
- `.src: Asset` and `.dst: Asset`
- `.src: str` and `.dst: str`
- `.src: Asset` and `.dst: str`

but not the final combo tho XD
Also, fix `.key` to properly cast any `.src: Asset` to string!
2023-05-09 14:49:26 -04:00
Tyler Goodlet aa5f25231a `ib`: never override existing ledger records
If user has loaded from a flex report then we don't want the API records
from the same period to override those; instead just update with any
missing fields from the API schema.

Also, always `str`-ify the contract id (what is set for the `.bs_mktid`
*before* packing into transaction type to ensure when serialized to
`pps.toml` there are no discrepancies at the codec level.. smh
2023-05-09 14:49:26 -04:00
Tyler Goodlet f3049016d6 `ib`: drop use of `_account2clients` in `load_clients_for_trio()`
Instead adjust `load_aio_clients()` to only reload clients detected as
non-loaded or disconnected (2 birds), and avoid use of the global module
table which could result in stale disconnected clients persisting on
multiple `brokerd` client reconnects, resulting in error.
2023-05-09 14:49:26 -04:00
Tyler Goodlet 16e11d447c Move toml table decoder to separate mod 2023-05-09 14:49:26 -04:00
Tyler Goodlet 199a5e8b38 `ib`: stick exc handler around client connection erros 2023-05-09 14:49:26 -04:00
Tyler Goodlet 59b095b2d5 `kraken`: heh, use `trio_util` for trades streamz tooo XD 2023-05-09 14:49:26 -04:00
Tyler Goodlet c59ec77d9c WIP: refactor ib pp load init 2023-05-09 14:49:26 -04:00
Tyler Goodlet 3e5da64571 Cache contract lookups from `Client.get_con()` 2023-05-09 14:49:26 -04:00
Tyler Goodlet 1c576d72d1 Dump `Position`s as pformatted dicts for now.. 2023-05-09 14:49:26 -04:00
Tyler Goodlet ea42f66b54 Use common `.brokers` logger in most backends 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2ae9576cd8 Add common logger instance for `.brokers` 2023-05-09 14:49:26 -04:00
Tyler Goodlet a462de6f2d Use a single log for entire `.service` subsys 2023-05-09 14:49:26 -04:00
Tyler Goodlet 3bf48ab597 Use a single log for entire `.clearing` subsys 2023-05-09 14:49:26 -04:00
Tyler Goodlet 2454dda18f Use `MktPair` attr `.size_tick` in charting 2023-05-09 14:49:26 -04:00
Tyler Goodlet 7498cbb5f4 Use `Struct.copy()` with update dict for `Order` from staged 2023-05-09 14:49:25 -04:00
Tyler Goodlet 581782800d Rename `Client.send_update()` -> `.update_nowait()` 2023-05-09 14:49:25 -04:00
Tyler Goodlet 069466218e Use `str(cmd.symbol)` for fqme on cancels, add `_nowait()` method names 2023-05-09 14:49:25 -04:00
Tyler Goodlet fd9e484b55 Add `.__str__()` to mktpair and symbol types, fix `MktPair.fqme` token order 2023-05-09 14:49:25 -04:00
Tyler Goodlet 406565f74d Rename `fqsn` -> `fqme` in paper engine 2023-05-09 14:49:25 -04:00
Tyler Goodlet 6272cae8d4 Drop more `Optional` usage on our `Struct` 2023-05-09 14:49:25 -04:00
Tyler Goodlet dc2332c980 '`kraken`: finally, use new `MktPair` in `'mkt_info'` init msg field!' 2023-05-09 14:49:25 -04:00
Tyler Goodlet 7be85a882b Drop use of legacy `Symbol.broker_info` in display startup 2023-05-09 14:49:25 -04:00
Tyler Goodlet b6df83a0e9 Typecast `OrderMode.staged.symbol: str` before `.copy()`! 2023-05-09 14:49:25 -04:00
Tyler Goodlet d62fb655eb `kraken`: parse our source asset key and set on `MktPair.src: str` 2023-05-09 14:49:25 -04:00
Tyler Goodlet a9778e4001 Always cast `Order.symbol: str` for now
To make nested `msgspec.Struct`s work we need to tell the codec that the
`.symbol` is some struct def, since we don't really need to enforce that
(yet) we're just going to enc/dec as `str` until we further formalize
and/or need something more complex.
2023-05-09 14:49:25 -04:00
Tyler Goodlet 580165f2f4 Expect new `MktPair.tick_size: Decimal` attr in ems 2023-05-09 14:49:25 -04:00
Tyler Goodlet 0f3041724b Use `MktPair` for `Flume.symbol` when used by backend
Initial attempt at getting the sampling and shm layer to use the new mkt
info meta-data type. Draft out a potential `BackendInitMsg:
msgspec.Struct` for validating the init msg returned from the
`stream_quotes()` start value; obvs don't actually use it yet.
2023-05-09 14:49:25 -04:00
Tyler Goodlet 1d08ee6d01 `.clearing`: broad rename of `fqsn` -> `fqme` 2023-05-09 14:49:25 -04:00
Tyler Goodlet d4a5a3057c Add `MktPair.suffix: str` read from contract info
To be compat with the `Symbol` (for now) and generally allow for reading
the (derivative) contract specific part of the fqme. Adjust
`contract_info: list[str]` and make `src: str = ''` by default.
2023-05-09 14:49:25 -04:00