For example in the paper-eng, if you have a backend that doesn't fully
support a symcache (yet) it's handy to be able to ignore processing
other paper-eng txns when all you care about at the moment is the
simulated symbol.
NOTE, that currently this will still result in a key-error when you load
more then one mkt with the paper engine (for which the backend does not
have the symcache implemented) since no fqme ad-hoc query was made for
the 2nd symbol (and i'm not sure we should support that kinda hackery
over just encouraging the sym-cache being added?). Def needs a little
more thought depending on how many backends are never going to be able
to (easily) support caching..
Instead just check for the field (which i'm not huge on the key-name for
anyway) and if not found get the "last price" from the real-time shm
buffer's latest 'close' sample.
Unrelatedly, use a `subs.copy()` in the `Router.client_broadcast()` loop
such that if a `client_stream` is popped on connection failure, we don't
RTE for the "size changed on iteration".
This must have broke at some point during the new `MktPair` and thus
`.fqme: str` updates; mas-o-menos the symbol key in the quote-msg-`dict`
was NOT set to the `MktPair.bs_fqme: str` value and thus wasn't being
processed by the downstream sampling and feed subsys.
So fix that as well as a few other refinements,
- set the `topic: mkt.bs_fqme` in quote msgs obvi.
- drop the "wait for first clearing vlm" quote poll loop; going to fix
the sampler to handle a `first_quote` without a `'last'` key.
- add some typing around calls to `get_mkt_info()`.
- rename `stream_messages()` -> `iter_normed_quotes()`.
Since `tractor`'s new and improved inter-actor cancellation semantics
are much more pedantic, AND bc we use the `ServiceMngr` for spawning
service actors on-demand, the caller of `maybe_spawn_daemon()` should
NEVER conduct a so called "out of band" `Actor`-runtime cancel request
since this is precisely the job of our `ServiceMngr` XD
Add a super in depth note explaining the underlying issue and adding
a todo list of how we should prolly augment `tractor` to make such cases
easier to grok and fix in the future!
Given it's a fairly simple yet useful abstraction, it makes sense to
offer this sub-sys alongside the core `tractor` runtime lib.
Without going into extreme detail on the impl changes (it'll come in
the commit that moves to the other repo) here is the high level summary:
------ - ------
- rename `Services` -> `ServiceMngr` and use an factory `@acm`
to guarantee a single-instance-per-actor using a niche approach for a
singleton object using a default keyword-arg B)
- the mod level `open_service_mngr()` and `get_service_mngr()` are the
new allocation/access API.
- add a `ServiceMngr.start_service()` method which does the work of both
spawning a new subactor (for the daemon) and uses its portal to start
the mngr side supervision task.
- open actor/task nurseries inside the `@acm` allocator.
Adjust other dependent subsystems to match:
------ - ------
- use `open_service_mngr()` when first allocated in `open_pikerd()`.
- use `get_service_mngr()` instead of importing the class ref inside
`.service.maybe_spawn_daemon()`, `.brokers._daemon.spawn_brokerd()`
and `.data._sampling.spawn_samplerd()` using a `partial` to pack in
the endpoint ctx kwargs (unpacked inside `.start_service()` XD).
Was orig for debugging an issue with `kucoin` i think but definitely
shouldn't be left in XD
Also add `'perpetual_future'` to `start_backfill()` input literal set.
About time we tidy'd a buncha status logging in this backend..
particularly for boot-up where there's lots of client-try-connect poll
looping with account detection from the user config.
`.api.Client` pprint and logging fmt improvements:
- add `Client.__repr__()` which shows the minimally useful set of info
from the underlying `.ib: IB` as well as a new `.acnts: list[str]`
of the account aliases defined in the user's `brokers.toml`.
- mk `.bars()` define a comprehensive `query_info: str` with all the
request deats but only display if there's a problem with the response
data.
- mk `.get_config()` report both the config file path and the acnt
aliases (NOT the actual account #s).
- move all `.load_aio_clients()` client poll loop requests do
`log.runtime()` statuses, only falling through to a `.warning()` when
the loop fails to connect the client to the spec-ed API-gw addr, and
|_ don't allow loading accounts for which the user has not defined an
alias in `brokers.toml::[ib]`; raise a value-error in such cases
with a message indicating how to mod the config.
|_ only `log.info()` about acnts if some were loaded..
Other mod logging de-noising:
- better status fmting in `.symbols.open_symbol_search()` with
`repr(Client)`.
- for `.feed.stream_quotes()` first quote reporting use `.runtime()`.
- timestamps came as `'date'`-keyed from 2022 and before but now are
`'datetime'`..
- some symbols seem to have no commission field, so handle that..
- when no `'price'` field found return `None` from `norm_trade()`.
- add a warn log on mid-fill commission updates.
Like other backends use the `AsyncClient` for all venue specific
client-sessions but change to allocating them inside `get_client()`
using an `AsyncExitStack` and inserting directly in the
`Client.venue_sesh: dict` table during init.
Supporting impl tweaks:
- remove most of the API client session building logic and instead make
`Client.__init__()` take in a `venue_sessions: dict` (set it to
`.venue_sesh`) and `conf: dict`, instead opting to do the http client
configuration inside `get_client()` since all that code only needs to
be run once.
|_load config inside `get_client()` once.
|_move session token creation into a new util func `init_api_keys()` and
also call it from `get_client()` factory; toss in an ex. toml section
config to the doc string.
- define `_venue_urls: dict[str, str]` (content taken from old static
`.venue_sesh` dict) at module level and feed them as `base_url: str`
inputs to the client create loop.
- adjust all call sigs in httpx-sesh-using methods, namely just
`._api()`.
- do a `.exch_info()` call in `get_client()` to cache the symbology
set.
Unrelated changes for various other outstanding buggers:
- to get futures feeds correctly loading when selected
from search (like 'XMRUSDT.USDTM.PERP'), expect a `MktPair` input to
`Client.bars()` such that the exact venue-key can be looked up (via
a new `.pair2venuekey()` meth) and then passed to `._api()`.
- adjust `.broker.open_trade_dialog()` to failover to paper engine when
there's no `api_key` key set for the `subconf` venue-key.
Like we do with other history backends to indicate lack of a data set.
This avoids any raise that will will bring down the backloader task with
some downstream error.
Raise a `ValueError` on no time index for now.
Tyler Goodlet
Nelson Torres