Commit Graph

3880 Commits (d8b5109e4742e20637327e814cdd1cabb50904ea)

Author SHA1 Message Date
Tyler Goodlet d8b5109e47 .clearing._ems: Don't require `first_quote['last']`
Instead just check for the field (which i'm not huge on the key-name for
anyway) and if not found get the "last price" from the real-time shm
buffer's latest 'close' sample.

Unrelatedly, use a `subs.copy()` in the `Router.client_broadcast()` loop
such that if a `client_stream` is popped on connection failure, we don't
RTE for the "size changed on iteration".
2025-01-29 00:56:18 -03:00
Tyler Goodlet 781810b4e1 `kucoin`: repair live quotes streaming..
This must have broke at some point during the new `MktPair` and thus
`.fqme: str` updates; mas-o-menos the symbol key in the quote-msg-`dict`
was NOT set to the `MktPair.bs_fqme: str` value and thus wasn't being
processed by the downstream sampling and feed subsys.

So fix that as well as a few other refinements,
- set the `topic: mkt.bs_fqme` in quote msgs obvi.
- drop the "wait for first clearing vlm" quote poll loop; going to fix
  the sampler to handle a `first_quote` without a `'last'` key.
- add some typing around calls to `get_mkt_info()`.
- rename `stream_messages()` -> `iter_normed_quotes()`.
2025-01-29 00:56:18 -03:00
Nelson Torres 1459ed9235 Deleted settlePlan field from binance FutesPair. 2025-01-29 00:56:18 -03:00
Nelson Torres ac45e212aa Added missing fields for kucoin.
feeCategory, makerFeeCoefficient, takerFeeCoefficient and st.
2025-01-29 00:56:18 -03:00
Nelson Torres c915c270d5 get_assets now uses public endpoint
It's better if the data is available through a public endpoint.
2025-01-29 00:56:18 -03:00
Nelson Torres 063af21180 now using exch_info in search_symbols 2025-01-29 00:56:18 -03:00
Nelson Torres 98e6d2e436 Fix bs_fqme using venue and expiry 2025-01-29 00:56:18 -03:00
Nelson Torres 4e18b43346 Added expiry property for OptionPair 2025-01-29 00:56:18 -03:00
Nelson Torres a55bbf429d No longer needed 2025-01-29 00:56:18 -03:00
Nelson Torres 3a57d803cd bs_mktid instead bs_fqme for deribits options 2025-01-29 00:56:18 -03:00
Nelson Torres 90681e86e5 Fixed pair instrument name in search_symbols endpoint.
Fixed instrument in bars endpoint, for options in deribits bs_mktid instead bs_fqme.
Fixed the id is in msg.
2025-01-29 00:56:18 -03:00
Tyler Goodlet 501f269656 data._web_bs: try to raise jsonrpc errors in parent task 2025-01-29 00:56:18 -03:00
Nelson Torres b7622d7ffe Add necessary classes in init file for deribit 2025-01-29 00:56:18 -03:00
Nelson Torres 857628e43c Minor refactor in open_symbol_search 2025-01-29 00:56:18 -03:00
Nelson Torres 5336c79546 stream_quotes now using FeedInit 2025-01-29 00:56:18 -03:00
Nelson Torres 4c992e7569 symbol_info refactor 2025-01-29 00:56:18 -03:00
Nelson Torres 02a3fe046d search_symbols output type fix 2025-01-29 00:56:18 -03:00
Nelson Torres ea919e7b25 add get_mkt_pairs method 2025-01-29 00:56:18 -03:00
Nelson Torres d2c7aa5643 get_assets refactor 2025-01-29 00:56:18 -03:00
Nelson Torres 67c9b0fb40 formatting 2025-01-29 00:56:18 -03:00
Nelson Torres 86896f1b10 created exch_info in api class 2025-01-29 00:56:18 -03:00
Nelson Torres 599316cf09 modify self_pairs type to ChainMap 2025-01-29 00:56:18 -03:00
Nelson Torres 4a89569f86 Necessary imports 2025-01-29 00:56:18 -03:00
Nelson Torres 88d5cd1d38 add get_market_info 2025-01-29 00:56:18 -03:00
Nelson Torres 6613f37156 Necessary imports 2025-01-29 00:56:18 -03:00
Nelson Torres cb125b793f minor fixes in venues 2025-01-29 00:56:18 -03:00
Nelson Torres f6fef07df8 add class Pair in venues, PAIRTYPES for future uses 2025-01-29 00:56:18 -03:00
Nelson Torres 6c19cc5d55 fix syms for venues.
little refactor in get_config, and created get_fh_config for cryptofeed.
2025-01-29 00:56:18 -03:00
Nelson Torres 1ed40ebbf7 venues for deribit 2025-01-29 00:56:18 -03:00
Nelson Torres fce1ded121 Added cryptofeed and pyarrow necessary for the feed, enable deribit
in the brokers init file, at this point the feed is working, to check
the tables use vd tool.
2025-01-29 00:56:18 -03:00
Tyler Goodlet b577180773 ib: `.api` mod and log-fmt cleaning
About time we tidy'd a buncha status logging in this backend..
particularly for boot-up where there's lots of client-try-connect poll
looping with account detection from the user config.

`.api.Client` pprint and logging fmt improvements:
- add `Client.__repr__()` which shows the minimally useful set of info
  from the underlying `.ib: IB` as well as a new `.acnts: list[str]`
  of the account aliases defined in the user's `brokers.toml`.
- mk `.bars()` define a comprehensive `query_info: str` with all the
  request deats but only display if there's a problem with the response
  data.
- mk `.get_config()` report both the config file path and the acnt
  aliases (NOT the actual account #s).
- move all `.load_aio_clients()` client poll loop requests do
  `log.runtime()` statuses, only falling through to a `.warning()` when
  the loop fails to connect the client to the spec-ed API-gw addr, and
 |_ don't allow loading accounts for which the user has not defined an
    alias in `brokers.toml::[ib]`; raise a value-error in such cases
    with a message indicating how to mod the config.
 |_ only `log.info()` about acnts if some were loaded..

Other mod logging de-noising:
- better status fmting in `.symbols.open_symbol_search()` with
  `repr(Client)`.
- for `.feed.stream_quotes()` first quote reporting use `.runtime()`.
2024-06-20 14:40:21 -04:00
Tyler Goodlet f12c452d96 ib: warn about mkt precision cuckups that `Contract`s clearly deliver wrong.. 2024-06-18 12:42:21 -04:00
Tyler Goodlet 3531c2edc1 ib: mask out trade and vlm rates for now 2024-06-18 10:03:34 -04:00
Tyler Goodlet 97dd7e766a ib: more trade record edge case handling
- timestamps came as `'date'`-keyed from 2022 and before but now are
  `'datetime'`..
- some symbols seem to have no commission field, so handle that..
- when no `'price'` field found return `None` from `norm_trade()`.
- add a warn log on mid-fill commission updates.
2024-06-18 10:00:18 -04:00
Tyler Goodlet ab1463d942 Port binance to `httpx`
Like other backends use the `AsyncClient` for all venue specific
client-sessions but change to allocating them inside `get_client()`
using an `AsyncExitStack` and inserting directly in the
`Client.venue_sesh: dict` table during init.

Supporting impl tweaks:
- remove most of the API client session building logic and instead make
  `Client.__init__()` take in a `venue_sessions: dict` (set it to
  `.venue_sesh`) and `conf: dict`, instead opting to do the http client
  configuration inside `get_client()` since all that code only needs to
  be run once.
 |_load config inside `get_client()` once.
 |_move session token creation into a new util func `init_api_keys()` and
  also call it from `get_client()` factory; toss in an ex. toml section
  config to the doc string.
- define `_venue_urls: dict[str, str]` (content taken from old static
  `.venue_sesh` dict) at module level and feed them as `base_url: str`
  inputs to the client create loop.
- adjust all call sigs in httpx-sesh-using methods, namely just
  `._api()`.
- do a `.exch_info()` call in `get_client()` to cache the symbology
  set.

Unrelated changes for various other outstanding buggers:
- to get futures feeds correctly loading when selected
  from search (like 'XMRUSDT.USDTM.PERP'), expect a `MktPair` input to
  `Client.bars()` such that the exact venue-key can be looked up (via
  a new `.pair2venuekey()` meth) and then passed to `._api()`.
- adjust `.broker.open_trade_dialog()` to failover to paper engine when
  there's no `api_key` key set for the `subconf` venue-key.
2024-06-12 09:41:23 -04:00
Nelson Torres 5314cb79d4 Added note to exception when missing field in SpotPair class 2024-06-11 16:57:59 -04:00
Nelson Torres 0c0b7116e3 Added new fields to SpotPair class in venues 2024-06-11 16:57:59 -04:00
Tyler Goodlet 19c343e8b2 binance: raise `NoData` on null hist arrays
Like we do with other history backends to indicate lack of a data set.
This avoids any raise that will will bring down the backloader task with
some downstream error.

Raise a `ValueError` on no time index for now.
2024-06-11 10:28:56 -04:00
Tyler Goodlet b7883325a9 Woops, `data` can be an empty list XD 2024-05-28 16:19:28 -04:00
Tyler Goodlet 37ca081555 Woops, fix missing `api_url` ref in error log 2024-05-24 12:24:25 -04:00
Tyler Goodlet 44b8c70521 Change type-annots to use `httpx.Response` 2024-05-20 12:55:45 -04:00
Tyler Goodlet e6af97c596 Port `kucoin` backend to `httpx` 2024-05-20 11:09:30 -04:00
Tyler Goodlet 95ace5acb8 Port `kraken` backend to `httpx` 2024-05-20 11:09:10 -04:00
Tyler Goodlet 1fd8654ca5 Port all `.ui*` submods to new `.ui.qt` imports
This also officially moves the code base to using `PyQt6` including all
necessary reference changes and enum namespace path moves.

Also includes a small `.ui.order_mode` fix to cancel any
`Order.price <= 0` and a name error fix with logging using `msg`,
which is already used for the input order msg..
2024-05-01 14:33:10 -04:00
Tyler Goodlet d0170982bf Add `piker.ui.qt` as a `PyQt6` shim module
For the future, like if we ever get a `PyQt7` (or wtv else..), add
a module which allows changing Qt binding lib imports from one spot for
all other `.ui` submodules. In some sense this is like a shoddier, less
dynamic version of how `pyqtgraph.Qt.__init__.py` supports multiple
libs; it might actually make sense eventually to instead import from
their shim layer instead?

Included is a draft attempt at exposing a bunch of enums which under
custom names:
- while the specific grouping of values seem to always stay consistent,
  the root enum's seem to almost always get moved around in the `PyQtX`
  module namespace.
- changing groupings and/or each top level enum's ns location can more
  simply be changed/re-orged from one spot.
- allows `.ui` consumer code to use a name more relevant to `piker`'s
  usage of wtv UI component is being configured.
2024-05-01 14:30:18 -04:00
Tyler Goodlet 821e73a409 Use a `unit_prefix: str` (like u or $) on health bar 2024-05-01 14:09:39 -04:00
Tyler Goodlet 3d03781810 Impl a sane (with nesting) `.types.Struct.pformat()`
Such that our internal structs can be pretty printed with indented and
type-hinted fields, AND for nested `Struct`-fields call `.pformat()` but
avoiding any recursion errors using `pprint.saferepr()`. Add
a `._sin_props()` iterator over the non-property fields; use it for
`dict` casting when called with `.to_dict(include_non_members=False)`.

Actually, we should also probably figure out how to only pprint like
when required by the user in a REPL or log msg by context-selectively
`pprint.PrettyPrinter` right? Also, if we can generalize decently enough
it'd be cool to maybe patch this in as a util to upstream `msgspec`?
2024-01-17 15:50:27 -05:00
Tyler Goodlet 83d1f117a8 Always cancel (loaded) zero-priced orders
Ran into this trading a peenee where a dark got (mistakenly) submitted
at a price of 0 and then that consecutively broke upstream allocator/pp
code due to a divide-by-zero.. So instead always check for a zero-price
(since that should never ever be valid in any market) and instead cancel
any such order in the EMS and return `None` so that upstream callers can
ignore it without crash handling.
2024-01-17 10:29:43 -05:00
Tyler Goodlet e4ce79f720 Delegate `.toolz.open_crash_handler()` to `tractor.devx`
Means we can drop `.toolz.debug` module outright.
2024-01-16 10:26:38 -05:00
Tyler Goodlet 264246d89b Fix `brokers.toml` load for `kraken` backend 2024-01-10 17:53:15 -05:00