Commit Graph

4405 Commits (11a883892d56d126e2fb1b83fe44f7673ae76277)

Author SHA1 Message Date
Nelson Torres 11a883892d config refactor
only one get_config method for api class and cryptofeed feed handler
2025-01-29 00:56:18 -03:00
Nelson Torres d25864af2d move constants to venue 2025-01-29 00:56:18 -03:00
Nelson Torres a2e2d4706a refactor redundant code 2025-01-29 00:56:18 -03:00
Nelson Torres ae5bd1f74f name formatting fixes 2025-01-29 00:56:18 -03:00
Nelson Torres 45d24c85bf get_mkt_info cleanup 2025-01-29 00:56:18 -03:00
Nelson Torres f19fc01f6b cache_symbols refactor 2025-01-29 00:56:18 -03:00
Nelson Torres f99d5f5a57 json_rpc_auth_wrapper 2025-01-29 00:56:18 -03:00
Nelson Torres 52aee44838 move object classes to venue 2025-01-29 00:56:18 -03:00
Nelson Torres 104afa80cf Added options symbols to get_assets 2025-01-29 00:56:18 -03:00
Tyler Goodlet d8b5109e47 .clearing._ems: Don't require `first_quote['last']`
Instead just check for the field (which i'm not huge on the key-name for
anyway) and if not found get the "last price" from the real-time shm
buffer's latest 'close' sample.

Unrelatedly, use a `subs.copy()` in the `Router.client_broadcast()` loop
such that if a `client_stream` is popped on connection failure, we don't
RTE for the "size changed on iteration".
2025-01-29 00:56:18 -03:00
Tyler Goodlet 781810b4e1 `kucoin`: repair live quotes streaming..
This must have broke at some point during the new `MktPair` and thus
`.fqme: str` updates; mas-o-menos the symbol key in the quote-msg-`dict`
was NOT set to the `MktPair.bs_fqme: str` value and thus wasn't being
processed by the downstream sampling and feed subsys.

So fix that as well as a few other refinements,
- set the `topic: mkt.bs_fqme` in quote msgs obvi.
- drop the "wait for first clearing vlm" quote poll loop; going to fix
  the sampler to handle a `first_quote` without a `'last'` key.
- add some typing around calls to `get_mkt_info()`.
- rename `stream_messages()` -> `iter_normed_quotes()`.
2025-01-29 00:56:18 -03:00
Nelson Torres 1459ed9235 Deleted settlePlan field from binance FutesPair. 2025-01-29 00:56:18 -03:00
Nelson Torres ac45e212aa Added missing fields for kucoin.
feeCategory, makerFeeCoefficient, takerFeeCoefficient and st.
2025-01-29 00:56:18 -03:00
Nelson Torres c915c270d5 get_assets now uses public endpoint
It's better if the data is available through a public endpoint.
2025-01-29 00:56:18 -03:00
Nelson Torres 063af21180 now using exch_info in search_symbols 2025-01-29 00:56:18 -03:00
Nelson Torres 98e6d2e436 Fix bs_fqme using venue and expiry 2025-01-29 00:56:18 -03:00
Nelson Torres 4e18b43346 Added expiry property for OptionPair 2025-01-29 00:56:18 -03:00
Nelson Torres a55bbf429d No longer needed 2025-01-29 00:56:18 -03:00
Nelson Torres 3a57d803cd bs_mktid instead bs_fqme for deribits options 2025-01-29 00:56:18 -03:00
Nelson Torres 90681e86e5 Fixed pair instrument name in search_symbols endpoint.
Fixed instrument in bars endpoint, for options in deribits bs_mktid instead bs_fqme.
Fixed the id is in msg.
2025-01-29 00:56:18 -03:00
Tyler Goodlet 501f269656 data._web_bs: try to raise jsonrpc errors in parent task 2025-01-29 00:56:18 -03:00
Nelson Torres b7622d7ffe Add necessary classes in init file for deribit 2025-01-29 00:56:18 -03:00
Nelson Torres 857628e43c Minor refactor in open_symbol_search 2025-01-29 00:56:18 -03:00
Nelson Torres 5336c79546 stream_quotes now using FeedInit 2025-01-29 00:56:18 -03:00
Nelson Torres 4c992e7569 symbol_info refactor 2025-01-29 00:56:18 -03:00
Nelson Torres 02a3fe046d search_symbols output type fix 2025-01-29 00:56:18 -03:00
Nelson Torres ea919e7b25 add get_mkt_pairs method 2025-01-29 00:56:18 -03:00
Nelson Torres d2c7aa5643 get_assets refactor 2025-01-29 00:56:18 -03:00
Nelson Torres 67c9b0fb40 formatting 2025-01-29 00:56:18 -03:00
Nelson Torres 86896f1b10 created exch_info in api class 2025-01-29 00:56:18 -03:00
Nelson Torres 599316cf09 modify self_pairs type to ChainMap 2025-01-29 00:56:18 -03:00
Nelson Torres 4a89569f86 Necessary imports 2025-01-29 00:56:18 -03:00
Nelson Torres 88d5cd1d38 add get_market_info 2025-01-29 00:56:18 -03:00
Nelson Torres 6613f37156 Necessary imports 2025-01-29 00:56:18 -03:00
Nelson Torres cb125b793f minor fixes in venues 2025-01-29 00:56:18 -03:00
Nelson Torres f6fef07df8 add class Pair in venues, PAIRTYPES for future uses 2025-01-29 00:56:18 -03:00
Nelson Torres 6c19cc5d55 fix syms for venues.
little refactor in get_config, and created get_fh_config for cryptofeed.
2025-01-29 00:56:18 -03:00
Nelson Torres 1ed40ebbf7 venues for deribit 2025-01-29 00:56:18 -03:00
Nelson Torres fce1ded121 Added cryptofeed and pyarrow necessary for the feed, enable deribit
in the brokers init file, at this point the feed is working, to check
the tables use vd tool.
2025-01-29 00:56:18 -03:00
Tyler Goodlet b577180773 ib: `.api` mod and log-fmt cleaning
About time we tidy'd a buncha status logging in this backend..
particularly for boot-up where there's lots of client-try-connect poll
looping with account detection from the user config.

`.api.Client` pprint and logging fmt improvements:
- add `Client.__repr__()` which shows the minimally useful set of info
  from the underlying `.ib: IB` as well as a new `.acnts: list[str]`
  of the account aliases defined in the user's `brokers.toml`.
- mk `.bars()` define a comprehensive `query_info: str` with all the
  request deats but only display if there's a problem with the response
  data.
- mk `.get_config()` report both the config file path and the acnt
  aliases (NOT the actual account #s).
- move all `.load_aio_clients()` client poll loop requests do
  `log.runtime()` statuses, only falling through to a `.warning()` when
  the loop fails to connect the client to the spec-ed API-gw addr, and
 |_ don't allow loading accounts for which the user has not defined an
    alias in `brokers.toml::[ib]`; raise a value-error in such cases
    with a message indicating how to mod the config.
 |_ only `log.info()` about acnts if some were loaded..

Other mod logging de-noising:
- better status fmting in `.symbols.open_symbol_search()` with
  `repr(Client)`.
- for `.feed.stream_quotes()` first quote reporting use `.runtime()`.
2024-06-20 14:40:21 -04:00
Tyler Goodlet f12c452d96 ib: warn about mkt precision cuckups that `Contract`s clearly deliver wrong.. 2024-06-18 12:42:21 -04:00
Tyler Goodlet 3531c2edc1 ib: mask out trade and vlm rates for now 2024-06-18 10:03:34 -04:00
Tyler Goodlet 97dd7e766a ib: more trade record edge case handling
- timestamps came as `'date'`-keyed from 2022 and before but now are
  `'datetime'`..
- some symbols seem to have no commission field, so handle that..
- when no `'price'` field found return `None` from `norm_trade()`.
- add a warn log on mid-fill commission updates.
2024-06-18 10:00:18 -04:00
Tyler Goodlet ab1463d942 Port binance to `httpx`
Like other backends use the `AsyncClient` for all venue specific
client-sessions but change to allocating them inside `get_client()`
using an `AsyncExitStack` and inserting directly in the
`Client.venue_sesh: dict` table during init.

Supporting impl tweaks:
- remove most of the API client session building logic and instead make
  `Client.__init__()` take in a `venue_sessions: dict` (set it to
  `.venue_sesh`) and `conf: dict`, instead opting to do the http client
  configuration inside `get_client()` since all that code only needs to
  be run once.
 |_load config inside `get_client()` once.
 |_move session token creation into a new util func `init_api_keys()` and
  also call it from `get_client()` factory; toss in an ex. toml section
  config to the doc string.
- define `_venue_urls: dict[str, str]` (content taken from old static
  `.venue_sesh` dict) at module level and feed them as `base_url: str`
  inputs to the client create loop.
- adjust all call sigs in httpx-sesh-using methods, namely just
  `._api()`.
- do a `.exch_info()` call in `get_client()` to cache the symbology
  set.

Unrelated changes for various other outstanding buggers:
- to get futures feeds correctly loading when selected
  from search (like 'XMRUSDT.USDTM.PERP'), expect a `MktPair` input to
  `Client.bars()` such that the exact venue-key can be looked up (via
  a new `.pair2venuekey()` meth) and then passed to `._api()`.
- adjust `.broker.open_trade_dialog()` to failover to paper engine when
  there's no `api_key` key set for the `subconf` venue-key.
2024-06-12 09:41:23 -04:00
Nelson Torres 5314cb79d4 Added note to exception when missing field in SpotPair class 2024-06-11 16:57:59 -04:00
Nelson Torres 0c0b7116e3 Added new fields to SpotPair class in venues 2024-06-11 16:57:59 -04:00
Tyler Goodlet 19c343e8b2 binance: raise `NoData` on null hist arrays
Like we do with other history backends to indicate lack of a data set.
This avoids any raise that will will bring down the backloader task with
some downstream error.

Raise a `ValueError` on no time index for now.
2024-06-11 10:28:56 -04:00
Tyler Goodlet b7883325a9 Woops, `data` can be an empty list XD 2024-05-28 16:19:28 -04:00
Tyler Goodlet 37ca081555 Woops, fix missing `api_url` ref in error log 2024-05-24 12:24:25 -04:00
Tyler Goodlet 44b8c70521 Change type-annots to use `httpx.Response` 2024-05-20 12:55:45 -04:00