88 lines
3.4 KiB
Markdown
88 lines
3.4 KiB
Markdown
---
|
||
model: gpt-6 (exact variant not exposed)
|
||
service: codex
|
||
timestamp: 2026-09-23T16:46:56.830897+00:00
|
||
git_ref: efa72980
|
||
diff_cmd: git diff efa72980 -- piker/brokers/ib/feed.py piker/tsp/_history.py tests/test_ib_history.py tests/test_history_backfill.py
|
||
---
|
||
|
||
## Prompt
|
||
|
||
The human requested the IB request-boundary repair in a new worktree
|
||
based on ib_connectivity_harden and a test exercising real persistence.
|
||
The existing MNQ parquet gap is outside this repair scope.
|
||
|
||
## Code output
|
||
|
||
> `git diff efa72980 -- piker/brokers/ib/feed.py`
|
||
|
||
> `git diff efa72980 -- piker/tsp/_history.py`
|
||
|
||
> `git diff efa72980 -- tests/test_ib_history.py`
|
||
|
||
> `git diff efa72980 -- tests/test_history_backfill.py`
|
||
|
||
get_bars() now completes HMDS no-data responses without changing the
|
||
requested date. open_history_client() retains its NoData and earliest
|
||
history classification. start_backfill() tracks the published boundary
|
||
separately from the request cursor after empty intervals.
|
||
|
||
## Unedited progress output
|
||
|
||
The new test failed on the original code as intended: the
|
||
temporary parquet contained only 4 bars instead of 2,004.
|
||
The day-skip caused 2,000 available bars to be omitted.
|
||
|
||
The test also includes a simulated venue closure that must remain
|
||
a gap. It checks that we fetch available bars without inventing
|
||
data for an empty interval.
|
||
|
||
The IB fix now preserves all 2,004 bars in parquet. The SHM
|
||
assertion exposed a second bug: after stepping over an empty
|
||
interval, the backfiller drops the last bar of the next response
|
||
as though it were already published.
|
||
|
||
I’ll separate the request date from the published-data boundary
|
||
so the same test can verify that both memory and disk retain
|
||
every available bar.
|
||
|
||
## Validation
|
||
|
||
Using the existing fsp_backfill_sync/py313 interpreter, with cwd in
|
||
ib_history_request_bounds and the Piker import checked against that cwd:
|
||
|
||
- tests/test_history_backfill.py: 11 passed in 1.92s.
|
||
- tests/test_ib_history.py: 17 passed in 0.36s.
|
||
- Ruff on all four changed Python files: all checks passed.
|
||
- git diff --check: passed.
|
||
|
||
The integration test uses simulated IB bars responses, real IB history
|
||
adapter and backfill logic, real SHM, and temporary NativeDB parquet.
|
||
It checks a fresh disk load. It does not test a live gateway, actor RPC,
|
||
or Qt rendering, and does not prove the origin of the user's live gap.
|
||
|
||
## Unedited review response
|
||
|
||
`None` is internal to the IB adapter. The backfiller receives
|
||
`NoData`, or `DataUnavailable` when the requested date has reached
|
||
the known beginning of history. I’ll make that distinction explicit.
|
||
|
||
The two dates answer different questions: “where should the next
|
||
history query end?” and “what is the oldest bar already in SHM?”
|
||
An empty reply changes only the first. I’ll rename them to
|
||
`request_end_dt` and `oldest_shm_dt` and explain the filtering with
|
||
a concrete example.
|
||
|
||
> `git diff efa72980 -- piker/brokers/ib/feed.py piker/tsp/_history.py`
|
||
|
||
## History-filter review follow-up
|
||
|
||
The human flagged the unused append_until_dt argument and confusing
|
||
append/prepend terminology in diff_history(). The helper had one
|
||
caller, using only prepend_until_dt. Removed the helper and inlined
|
||
its inclusive lower-time cutoff beside the SHM duplicate filter.
|
||
The human's nearby TODO comment is resolved by this removal; the
|
||
replacement comment describes the retained timestamps directly.
|
||
The storage comparison remains >= backfill_until_dt; the SHM
|
||
comparison remains < oldest_shm_dt.
|