Compare commits

..

20 Commits

Author SHA1 Message Date
Tyler Goodlet 179a42ac78 TOCHERRY service_mng_to_tractor: type fix to `.hilevel.ServicecMngr` 2025-02-13 12:31:33 -05:00
Tyler Goodlet b101cc19ee Enable `greenback` for `.pause_from_sync()` by default? 2025-02-13 12:30:41 -05:00
Tyler Goodlet d4f1769d64 `.brokers.cli`: module type and todo for `--pdb` flag to NOT src from sub-cmd 2025-02-13 12:29:43 -05:00
Tyler Goodlet f8e05effbc Catch using `Sampler.bcast_errors` where possible
In all other possible IPC disconnect handling blocks. Also more
comprehensive typing throughout `uniform_rate_send()`.
2025-02-13 12:05:11 -05:00
Tyler Goodlet d31bf9395b Group bcast errors as `Sampler.bcast_errors`
A new class var `tuple[Exception]` such that the err set can be reffed
externally as needed for catching other similar pub-sub/IPC failures in
other (related) real-time sub-systems.

Also added some now-masked logging for debugging live-feed stream reading
issues that should ONLY be used for debugging since they'll greatly
degrade HFT perf. Used the new `log.mk_repr()` stuff (that one day we
should prolly pull from `modden` as a dep) for pretty console emissions.
2025-02-13 11:44:50 -05:00
Tyler Goodlet 02b7e21a07 Suppress `trio.EndOfChannel`s raised by remote peer
Since now `tractor` will raise this native `trio`-exc translated from
a `Stop` msg when the peer gracefully terminates a `tractor.MsgStream`.
Just `info()` log in such cases versus continuing to warn for the
others.
2025-02-13 11:38:29 -05:00
Tyler Goodlet fdad4c46f0 `.tsp._anal`: add (unused) `detect_vlm_gaps()` 2025-02-13 11:36:59 -05:00
Tyler Goodlet 6df6d6f946 `.storage.cli`: collect gap-markup-aids into `tf2aids: dict` prior to pause for introspection 2025-02-13 11:35:49 -05:00
Tyler Goodlet fe815f46a1 Delegate to `tractor.msg.pretty_struct` since it was factored from here! 2025-02-13 11:21:29 -05:00
Tyler Goodlet f5e0b47aaf Teensie `piker.data` styling tweaks
- use more compact optional value style with `|`-union
- fix `.flows` typing-only import since we need `MktPair` to be
  immediately defined for use on a `msgspec.Struct` field.
- more "tree-like" warning msg in `.validate()` reporting.
2025-02-13 11:04:59 -05:00
Tyler Goodlet 89abbd592d Invert `getattr()` check for `get_mkt_pairs()` ep
Such that we `return` early when not defined by the provider backend to
reduce an indent level in `SymbologyCache.load()`.
2025-02-13 10:59:25 -05:00
Tyler Goodlet 8cd154f31e Various `.clearing` todos/notes on potential issues with loglevel settings.. 2025-02-13 10:46:44 -05:00
Tyler Goodlet 75e326f117 Type loaded backend modules 2025-02-13 10:44:23 -05:00
Tyler Goodlet 165b3432ed Bump various `.brokers.core` doc string content/style 2025-02-13 10:43:50 -05:00
Tyler Goodlet 91d4ab0a23 Ignore any non-`.parquet` files under `.config/piker/nativedb/` subdir 2025-02-12 18:20:55 -05:00
Tyler Goodlet de11ed1e0c Doc-n-clean `.data._web_bs.open_jsonrpc_session()`
Add a doc-string reflecting recent refinements, drop all the old hook
params, rename `n: trio.Nursery` -> `tn` for "task nursery" fitting with
code base's naming style.
2025-02-12 18:20:55 -05:00
Tyler Goodlet 08c33d1d9e Allow ledger passes to ignore (symcache) unknown fqmes
For example in the paper-eng, if you have a backend that doesn't fully
support a symcache (yet) it's handy to be able to ignore processing
other paper-eng txns when all you care about at the moment is the
simulated symbol.

NOTE, that currently this will still result in a key-error when you load
more then one mkt with the paper engine (for which the backend does not
have the symcache implemented) since no fqme ad-hoc query was made for
the 2nd symbol (and i'm not sure we should support that kinda hackery
over just encouraging the sym-cache being added?). Def needs a little
more thought depending on how many backends are never going to be able
to (easily) support caching..
2025-02-12 18:20:55 -05:00
Tyler Goodlet c35fcce9f4 .clearing._ems: Don't require `first_quote['last']`
Instead just check for the field (which i'm not huge on the key-name for
anyway) and if not found get the "last price" from the real-time shm
buffer's latest 'close' sample.

Unrelatedly, use a `subs.copy()` in the `Router.client_broadcast()` loop
such that if a `client_stream` is popped on connection failure, we don't
RTE for the "size changed on iteration".
2025-02-12 18:20:55 -05:00
Tyler Goodlet ba0321c07e data._web_bs: try to raise jsonrpc errors in parent task 2025-02-12 18:20:55 -05:00
Tyler Goodlet 5599a6b7de Mask no-data pause, add perps to no-`/src`-in-fqme asset set
Was orig for debugging an issue with `kucoin` i think but definitely
shouldn't be left in XD

Also add `'perpetual_future'` to the `.start_backfill()` input literal
set since we don't expect the 'btc/usd.perp.binance' for now.
2025-02-12 18:20:55 -05:00
2 changed files with 11 additions and 15 deletions

View File

@ -42,6 +42,7 @@ from trio_typing import TaskStatus
from pendulum import (
from_timestamp,
)
from rapidfuzz import process as fuzzy
import numpy as np
import tractor
@ -110,7 +111,6 @@ class AggTrade(Struct, frozen=True):
async def stream_messages(
ws: NoBsWs,
) -> AsyncGenerator[NoBsWs, dict]:
# TODO: match syntax here!
@ -221,8 +221,6 @@ def make_sub(pairs: list[str], sub_name: str, uid: int) -> dict[str, str]:
}
# TODO, why aren't frame resp `log.info()`s showing in upstream
# code?!
@acm
async def open_history_client(
mkt: MktPair,
@ -465,8 +463,6 @@ async def stream_quotes(
):
init_msgs: list[FeedInit] = []
for sym in symbols:
mkt: MktPair
pair: Pair
mkt, pair = await get_mkt_info(sym)
# build out init msgs according to latest spec
@ -515,6 +511,7 @@ async def stream_quotes(
# start streaming
async for typ, quote in msg_gen:
# period = time.time() - last
# hz = 1/period if period else float('inf')
# if hz > 60:
@ -550,7 +547,7 @@ async def open_symbol_search(
)
# repack in fqme-keyed table
byfqme: dict[str, Pair] = {}
byfqme: dict[start, Pair] = {}
for pair in pairs.values():
byfqme[pair.bs_fqme] = pair

View File

@ -104,15 +104,14 @@ def get_app_dir(
# `tractor`) with the testing dir and check for it whenever we
# detect `pytest` is being used (which it isn't under normal
# operation).
# if "pytest" in sys.modules:
# import tractor
# actor = tractor.current_actor(err_on_no_runtime=False)
# if actor: # runtime is up
# rvs = tractor._state._runtime_vars
# import pdbp; pdbp.set_trace()
# testdirpath = Path(rvs['piker_vars']['piker_test_dir'])
# assert testdirpath.exists(), 'piker test harness might be borked!?'
# app_name = str(testdirpath)
if "pytest" in sys.modules:
import tractor
actor = tractor.current_actor(err_on_no_runtime=False)
if actor: # runtime is up
rvs = tractor._state._runtime_vars
testdirpath = Path(rvs['piker_vars']['piker_test_dir'])
assert testdirpath.exists(), 'piker test harness might be borked!?'
app_name = str(testdirpath)
if platform.system() == 'Windows':
key = "APPDATA" if roaming else "LOCALAPPDATA"