All the max pain math now is in this two functions:
- get_total_intrinsic_values(): calculate the total value for all strike_prices and stores then in a dict[str, Decimal]
- `get_intrinsic_value_and_max_pain()` given the `intrinsic_values` dict, returns the `max_pain` strike price and the `total_intrinsic_value` for that `strike_price`
- To calculate the `max_pain` first we need an expiration date,
get_expiration_dates()` retrieves them and the user then enters one of
the shown, then using the select expiry_date on `get_instruments()` we
are good to build the `oi_by_strikes` important!
- Add `update_oi_by_strikes()`.
- Add `check_if_complete()`.
- `get_max_pain()`: here's where all the action takes place, the
`oi_by_strikes` must be complete to start the calculations,
- Use `maybe_open_oi_feed` for open a oi_feed.
- Add `max_pain_readme.rst`
- `get_timestamp_int`: added this is the hack, so we can aboid use the custom deribit date format.
- `get_currencies`: added so we could get all deribit's available currencies.
- `get_instruments`: for a especific expiration date, it return a list of criptofeed.Symbol.
- `get_expiration_dates`: expirations dates available for btc's option contracts .
- `get_strikes_dict`: all the strike prices for an especific expiration date.
- `aio_open_interest_feed_relay` `open_oi_feed` `maybe_open_oi_feed`: this three handles all the portal stuff and the cryptofeed callbacks for the open interest and trades, for some reason it need both to work, i need to check that out at some point.
- Also a couple of format fixes.
- `get_timestamp_int`: added this is the hack, so we can aboid use the custom deribit date format.
- `get_currencies`: added so we could get all deribit's available currencies.
- Also a couple of format fixes.
The main change needed to make `piker.data.feed._FeedsBus` work was
to correctly format the `'trade'` msgs with the (new schema) expected
`'ticks': list[dict]` field which,
- we compute the `piker` quote-msg-`dict` from the (now directly proxied through)
`cryptofeed.types.Trade`'s fields inside the body of `stream_quotes()`.
- similarly, move the `'l1'` msg processing, **out of** the `asyncio`-side
`_l1()` callback (defined as a closure in `.api.aio_price_feed_relay()`
and passed to the `cryptofeed.FeedHandler`) and instead mod the
callback to simply pass through the `.types.L1Book` ref directly to
the `piker`/`trio` side task for conversion.
In support of all that,
- mask-to-drop the alt-branch to wait on a first rt event when the
`cryptofeed.LastTradesResult.trades: list[Trade]` is empty; doesn't
seem like this ever even happens?
- add a buncha typing, comments and doc-strs to the routines in
`.deribit.api` including notes on where we can choose to mod the
`.bs_fqme` for our eventually preferred `piker` style format.
- simplify some nested `@acm` enters to the new single `async with
<tuple>)` style.
- be particularly pedantic about typing
`tractor.to_asyncio.LinkedTaskChannel`
- bit of pep8 line-spacing fixes in `.venues`.
The quote-msg `'topic'` field was being set and sent as the
`OptionPair.symbol: str` value instead of as the `MktPair.bs_fqme: str`
as is required for matching on the `piker.data.feed` side. So change to
that and simplify the actual `.bs_fqme: str` value to NOT include the
ISO-format time (for now) since it's a big ugly and longer term we need
a `piker`-fqme friendly-on-ze-eyes format/style anyway..
Such that the `get_hist()` query func raises `DataUnavailable` with an
explicit message regarding the start of the (option) contract's
lifetime.
Other,
- mask some unused imports (for now?)
- drop a duplicate `tractor.get_console_log()` call which was causing
duplicate console emits (it's already setup by brokerd init now).
- comment various unused code bits i found.
- add a info log around live quotes so we can see for the moment when
they actually occur.. XD
There were some imports missing or unused as well as a variety of spots
that had grokability issues due to missing type hints.
Other tweaks as part some more thorough manual testing:
- always raise when not `brokers.toml` section since the API can never
work (no free data without keys).
- inline the `Asset.atype='crypto_currency` field despite it maybe not
being the best value for `OptionPair` instruments..
- tossed in a now-masked pause block for debugging history queries in
`Client.bars()`.
- commented out all the live order ctl (internal) endpoints for now
since they're unused.
- `FeedInit` for init_msgs in `stream_quotes`.
- new cache is `client_pairs` so is replacing the old `client.cache_symbols`.
- `get_mkt_info` added
- `get_ohlc` fixed to comply the new ways of the feed.
key changes:
- Resolved the issue with the expiration dates from deribits, now we int instead of the crazy custom deribits format.
- The client now has a new `_json_rpc_auth_wrapper` that adquires a first access token and then will refresh the access token when this expires.
- `get_assets` fixed, now we use the public endpoint to check the availables assets, in the future probably this will change, but for now is working just fine.
- `get_mkt_pairs` added.
- `exch_info` added.
- `cache_symbols` fixed.
- Also a lot of reformat made in api.
Seems to be some kinda super weird env bug since we moved to using
`uv`? When it triggers it also seems to cause a pretty fundamental crash
that not only breaks `tractor.devx._debug` stuff but also seems to get
us in a perma-hang state where no SIGINT or other sys sig will be able
to kill the root proc!?!?
TODO, a `gitea` issue to track so we can fix the fundamental problem as
well as transitive fault in `tractor`'s core which seems to be due to
the error taking place during a sub-actor's module import phase which
prevents the runtime from booting fully and then the proc getting stuck
in a real gnarly SIG-state..
This must have broke at some point during the new `MktPair` and thus
`.fqme: str` updates; mas-o-menos the symbol key in the quote-msg-`dict`
was NOT set to the `MktPair.bs_fqme: str` value and thus wasn't being
processed by the downstream sampling and feed subsys.
So fix that as well as a few other refinements,
- set the `topic: mkt.bs_fqme` in quote msgs obvi.
- drop the "wait for first clearing vlm" quote poll loop; going to fix
the sampler to handle a `first_quote` without a `'last'` key.
- add some typing around calls to `get_mkt_info()`.
- rename `stream_messages()` -> `iter_normed_quotes()`.
In line with our move to `uv` and recent `nix` support update a bunch of
the summary content and factor out the order-control section to a new
`.piker.clearing` readme file with embedded todos therein.
Like other backends use the `AsyncClient` for all venue specific
client-sessions but change to allocating them inside `get_client()`
using an `AsyncExitStack` and inserting directly in the
`Client.venue_sesh: dict` table during init.
Supporting impl tweaks:
- remove most of the API client session building logic and instead make
`Client.__init__()` take in a `venue_sessions: dict` (set it to
`.venue_sesh`) and `conf: dict`, instead opting to do the http client
configuration inside `get_client()` since all that code only needs to
be run once.
|_load config inside `get_client()` once.
|_move session token creation into a new util func `init_api_keys()` and
also call it from `get_client()` factory; toss in an ex. toml section
config to the doc string.
- define `_venue_urls: dict[str, str]` (content taken from old static
`.venue_sesh` dict) at module level and feed them as `base_url: str`
inputs to the client create loop.
- adjust all call sigs in httpx-sesh-using methods, namely just
`._api()`.
- do a `.exch_info()` call in `get_client()` to cache the symbology
set.
Unrelated changes for various other outstanding buggers:
- to get futures feeds correctly loading when selected
from search (like 'XMRUSDT.USDTM.PERP'), expect a `MktPair` input to
`Client.bars()` such that the exact venue-key can be looked up (via
a new `.pair2venuekey()` meth) and then passed to `._api()`.
- adjust `.broker.open_trade_dialog()` to failover to paper engine when
there's no `api_key` key set for the `subconf` venue-key.
Like we do with other history backends to indicate lack of a data set.
This avoids any raise that will will bring down the backloader task with
some downstream error.
Raise a `ValueError` on no time index for now.
About time we tidy'd a buncha status logging in this backend..
particularly for boot-up where there's lots of client-try-connect poll
looping with account detection from the user config.
`.api.Client` pprint and logging fmt improvements:
- add `Client.__repr__()` which shows the minimally useful set of info
from the underlying `.ib: IB` as well as a new `.acnts: list[str]`
of the account aliases defined in the user's `brokers.toml`.
- mk `.bars()` define a comprehensive `query_info: str` with all the
request deats but only display if there's a problem with the response
data.
- mk `.get_config()` report both the config file path and the acnt
aliases (NOT the actual account #s).
- move all `.load_aio_clients()` client poll loop requests do
`log.runtime()` statuses, only falling through to a `.warning()` when
the loop fails to connect the client to the spec-ed API-gw addr, and
|_ don't allow loading accounts for which the user has not defined an
alias in `brokers.toml::[ib]`; raise a value-error in such cases
with a message indicating how to mod the config.
|_ only `log.info()` about acnts if some were loaded..
Other mod logging de-noising:
- better status fmting in `.symbols.open_symbol_search()` with
`repr(Client)`.
- for `.feed.stream_quotes()` first quote reporting use `.runtime()`.
- timestamps came as `'date'`-keyed from 2022 and before but now are
`'datetime'`..
- some symbols seem to have no commission field, so handle that..
- when no `'price'` field found return `None` from `norm_trade()`.
- add a warn log on mid-fill commission updates.
Code base is already ported for `Qt6` so this removes the pyqt5 dep,
adds latest pyqt6 as well as buncha other updates:
- add `xonsh` and ptk as dev deps for those of us using wacky shells ;P
- bump compiled deps as needed for python 3.12 (`numpy`, `numba`)
- add `httpx` and drop `asks` since the latter is zombied and not compat
with other libs on 3.12.
- add `ruff` linting ignore rules for the new `.ui.qt` shim mod layer.
- few other deps updates to latest versions.
- add in the `keywords` and `classifiers` sections from the old
`setup.py`.
This also officially moves the code base to using `PyQt6` including all
necessary reference changes and enum namespace path moves.
Also includes a small `.ui.order_mode` fix to cancel any
`Order.price <= 0` and a name error fix with logging using `msg`,
which is already used for the input order msg..
For the future, like if we ever get a `PyQt7` (or wtv else..), add
a module which allows changing Qt binding lib imports from one spot for
all other `.ui` submodules. In some sense this is like a shoddier, less
dynamic version of how `pyqtgraph.Qt.__init__.py` supports multiple
libs; it might actually make sense eventually to instead import from
their shim layer instead?
Included is a draft attempt at exposing a bunch of enums which under
custom names:
- while the specific grouping of values seem to always stay consistent,
the root enum's seem to almost always get moved around in the `PyQtX`
module namespace.
- changing groupings and/or each top level enum's ns location can more
simply be changed/re-orged from one spot.
- allows `.ui` consumer code to use a name more relevant to `piker`'s
usage of wtv UI component is being configured.
Nelson Torres
Tyler Goodlet
goodboy