Only use `frame_types` if delivered during enter
The `open_history_client()` provider endpoint can *optionally* deliver a `frame_types: dict[int, pendulum.Duration]` subsection in its `config: dict[str, dict]` (as was implemented with the `ib` backend). This allows the `tsp` backfilling machinery to use this "recommended frame duration" to subtract from the `last_start_dt` any time a `NoData` gap is signalled by the `get_hist()` call allowing gaps to be ignored safely without missing history by knowing the next earliest dt we can query from using the `end_dt`. However, currently all crypto$ providers haven't implemented this feat yet.. As such only try to use the `frame_types` feature if provided when handling `NoData` conditions inside `tsp.start_backfill()` and otherwise raise as normal.fix_deribit_hist_queries_NEW
							parent
							
								
									8165c684e5
								
							
						
					
					
						commit
						d30cfeab2c
					
				|  | @ -434,21 +434,32 @@ async def start_backfill( | |||
|                 # - some other unknown error (ib blocking the | ||||
|                 #   history bc they don't want you seeing how they | ||||
|                 #   cucked all the tinas..) | ||||
|                 if dur := frame_types.get(timeframe): | ||||
|                     # decrement by a frame's worth of duration and | ||||
|                     # retry a few times. | ||||
|                     last_start_dt.subtract( | ||||
|                 if ( | ||||
|                     frame_types | ||||
|                     and | ||||
|                     (dur := frame_types.get(timeframe)) | ||||
|                 ): | ||||
|                     # decrement by a duration's (frame) worth of time | ||||
|                     # as maybe indicated by the backend to see if we | ||||
|                     # can get older data before this possible | ||||
|                     # "history gap". | ||||
|                     orig_last_start_dt = last_start_dt | ||||
|                     last_start_dt = last_start_dt.subtract( | ||||
|                         seconds=dur.total_seconds() | ||||
|                     ) | ||||
|                     log.warning( | ||||
|                         f'{mod.name} -> EMPTY FRAME for end_dt?\n' | ||||
|                         f'tf@fqme: {timeframe}@{mkt.fqme}\n' | ||||
|                         'bf_until <- last_start_dt:\n' | ||||
|                         f'{backfill_until_dt} <- {last_start_dt}\n' | ||||
|                         f'Decrementing `end_dt` by {dur} and retry..\n' | ||||
|                         f'Decrementing `end_dt` by {dur} and retry..\n\n' | ||||
| 
 | ||||
|                         f'orig_last_start_dt: {orig_last_start_dt}\n' | ||||
|                         f'dur subtracted last_start_dt: {last_start_dt}\n' | ||||
|                         f'bf_until: {backfill_until_dt}\n' | ||||
|                     ) | ||||
|                     continue | ||||
| 
 | ||||
|                 raise | ||||
| 
 | ||||
|             # broker says there never was or is no more history to pull | ||||
|             except DataUnavailable: | ||||
|                 log.warning( | ||||
|  |  | |||
		Loading…
	
		Reference in New Issue