3.4 KiB
Prompt
The human requested the IB request-boundary repair in a new worktree based on ib_connectivity_harden and a test exercising real persistence. The existing MNQ parquet gap is outside this repair scope.
Code output
git diff efa72980 -- piker/brokers/ib/feed.py
git diff efa72980 -- piker/tsp/_history.py
git diff efa72980 -- tests/test_ib_history.py
git diff efa72980 -- tests/test_history_backfill.py
get_bars() now completes HMDS no-data responses without changing the requested date. open_history_client() retains its NoData and earliest history classification. start_backfill() tracks the published boundary separately from the request cursor after empty intervals.
Unedited progress output
The new test failed on the original code as intended: the temporary parquet contained only 4 bars instead of 2,004. The day-skip caused 2,000 available bars to be omitted.
The test also includes a simulated venue closure that must remain a gap. It checks that we fetch available bars without inventing data for an empty interval.
The IB fix now preserves all 2,004 bars in parquet. The SHM assertion exposed a second bug: after stepping over an empty interval, the backfiller drops the last bar of the next response as though it were already published.
I’ll separate the request date from the published-data boundary so the same test can verify that both memory and disk retain every available bar.
Validation
Using the existing fsp_backfill_sync/py313 interpreter, with cwd in ib_history_request_bounds and the Piker import checked against that cwd:
- tests/test_history_backfill.py: 11 passed in 1.92s.
- tests/test_ib_history.py: 17 passed in 0.36s.
- Ruff on all four changed Python files: all checks passed.
- git diff –check: passed.
The integration test uses simulated IB bars responses, real IB history adapter and backfill logic, real SHM, and temporary NativeDB parquet. It checks a fresh disk load. It does not test a live gateway, actor RPC, or Qt rendering, and does not prove the origin of the user’s live gap.
Unedited review response
None is internal to the IB adapter. The backfiller receives NoData, or DataUnavailable when the requested date has reached the known beginning of history. I’ll make that distinction explicit.
The two dates answer different questions: “where should the next history query end?” and “what is the oldest bar already in SHM?” An empty reply changes only the first. I’ll rename them to request_end_dt and oldest_shm_dt and explain the filtering with a concrete example.
git diff efa72980 -- piker/brokers/ib/feed.py piker/tsp/_history.py
History-filter review follow-up
The human flagged the unused append_until_dt argument and confusing append/prepend terminology in diff_history(). The helper had one caller, using only prepend_until_dt. Removed the helper and inlined its inclusive lower-time cutoff beside the SHM duplicate filter. The human’s nearby TODO comment is resolved by this removal; the replacement comment describes the retained timestamps directly. The storage comparison remains >= backfill_until_dt; the SHM comparison remains < oldest_shm_dt.