pikers
/
piker
3
0
Fork 0
Code Issues 4 Pull Requests 6 Packages Projects Releases Wiki Activity

Only use `frame_types` if delivered during enter 

The `open_history_client()` provider endpoint can *optionally*
deliver a `frame_types: dict[int, pendulum.Duration]` subsection in its
`config: dict[str, dict]` (as was implemented with the `ib` backend).
This allows the `tsp` backfilling machinery to use this "recommended
frame duration" to subtract from the `last_start_dt` any time a `NoData`
gap is signalled by the `get_hist()` call allowing gaps to be ignored
safely without missing history by knowing the next earliest dt we can
query from using the `end_dt`. However, currently all crypto$ providers
haven't implemented this feat yet..

As such only try to use the `frame_types` feature if provided when
handling `NoData` conditions inside `tsp.start_backfill()` and otherwise
raise as normal.
fix_deribit_hist_queries
Tyler Goodlet 2024-11-18 09:27:44 -05:00
parent 6555ccfbba
commit f96bd51442
1 changed files with 18 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
piker/tsp/__init__.py
View File

@ -434,21 +434,32 @@ async def start_backfill(
# - some other unknown error (ib blocking the # - some other unknown error (ib blocking the
# history bc they don't want you seeing how they # history bc they don't want you seeing how they
# cucked all the tinas..) # cucked all the tinas..)
if dur := frame_types.get(timeframe): if (
# decrement by a frame's worth of duration and frame_types
# retry a few times. and
last_start_dt.subtract( (dur := frame_types.get(timeframe))
):
# decrement by a duration's (frame) worth of time
# as maybe indicated by the backend to see if we
# can get older data before this possible
# "history gap".
orig_last_start_dt = last_start_dt
last_start_dt = last_start_dt.subtract(
seconds=dur.total_seconds() seconds=dur.total_seconds()
) )
log.warning( log.warning(
f'{mod.name} -> EMPTY FRAME for end_dt?\n' f'{mod.name} -> EMPTY FRAME for end_dt?\n'
f'tf@fqme: {timeframe}@{mkt.fqme}\n' f'tf@fqme: {timeframe}@{mkt.fqme}\n'
'bf_until <- last_start_dt:\n' f'Decrementing `end_dt` by {dur} and retry..\n\n'
f'{backfill_until_dt} <- {last_start_dt}\n'
f'Decrementing `end_dt` by {dur} and retry..\n' f'orig_last_start_dt: {orig_last_start_dt}\n'
f'dur subtracted last_start_dt: {last_start_dt}\n'
f'bf_until: {backfill_until_dt}\n'
) )
continue continue
raise
# broker says there never was or is no more history to pull # broker says there never was or is no more history to pull
except DataUnavailable: except DataUnavailable:
log.warning( log.warning(