.agents/skills/piker-clearing-expert/SKILL.md

@@ -1,174 +0,0 @@
----
-name: piker-clearing-expert
-description: >
-  Piker clearing, EMS/order-control, accounting, position, and
-  trade-ledger expertise. Apply when changing or debugging Order,
-  Status, Brokerd* protocols, live or paper fills, dark orders,
-  order IDs, TransactionLedger persistence, Account/Position/PPU,
-  FQME/bs_mktid identity, SymbologyCache lookups, or broker trade
-  normalization.
-compatibility: >
-  Requires a piker checkout and familiarity with Python, trio,
-  tractor, msgspec Structs, and backend-specific trading APIs.
-metadata:
-  author: goodboy
-  version: "1.0"
----
-
-# Piker Clearing Expert
-
-Use this mental model before touching order control or accounting:
-
-```text
-OrderClient / UI
-  -> emsd routes intent and owns order-dialog state
-  -> live brokerd OR paperboi executes and reports events
-  -> executing backend owns fill-to-accounting integration
-  -> emsd translates and broadcasts broker events
-  -> UI consumes order state and position summaries
-```
-
-The EMS is not the trade ledger. It correlates IDs, runs dark
-predicates, and relays lifecycle events. Paper and accounting-aware
-live backends may normalize fills into transactions, update local
-ledger/account state, and emit `BrokerdPosition` summaries. Inspect
-each live backend; this integration is not uniform.
-
-## Runtime ownership
-
-- `OrderClient` owns client intent and local sent-order tracking.
-- `emsd` owns `Router`, `DarkBook`, active `Status` dialogs,
-  subscribers, and `oid <-> reqid` correlation.
-- `brokerd.<broker>` owns credentialed live order control.
-- `paperboi.<broker>` owns simulated execution and paper accounting.
-- `TransactionLedger` owns trade-record persistence, not order state.
-- `Account` owns positions keyed by backend-system market identity.
-- UI `Position.update_from_msg()` is a summary reset, not durable
-  transaction accounting.
-
-For daemon placement, context streams, cancellation, and actor-local
-state, also load `piker-conc-expert`.
-
-## Event lifecycle
-
-1. A client sends `Order` or `Cancel` through `OrderClient`.
-2. `process_client_order_cmds()` creates or updates EMS `Status`.
-3. EMS routes a `BrokerdOrder` or stores a dark trigger predicate.
-4. The executing backend acknowledges with `BrokerdOrderAck`.
-5. EMS records the current `oid <-> reqid` relation.
-6. Accepted, ack-correlated status/fill events become client updates.
-7. The backend integrates fills into accounting when supported.
-8. A resulting `BrokerdPosition` is relayed through EMS to clients.
-
-Cancellation may arrive before acknowledgement. Preserve
-`Status.cancel_called`; once the ack supplies `reqid`, EMS can issue
-the deferred `BrokerdCancel`.
-
-## Identifier namespaces
-
-Never substitute one namespace merely because two values happen to
-match in a backend:
-
-| ID | Owner and invariant |
-|---|---|
-| `oid` | Stable client/EMS order-dialog ID across submit, modify, cancel |
-| `reqid` | Backend order-control ID established by an ack; it may change |
-| venue order ID | Native API identity; may differ from both IDs above |
-| `tid` | Stable clear ID used to deduplicate position events and TOML keys |
-| `fqme` | Piker-normalized market address and symcache lookup key |
-| `bs_fqme` | FQME without its final broker suffix |
-| `bs_mktid` | Backend-system market key and `Account.pps` key |
-
-Normalize every durably persisted `tid` to a stable string. Although
-`Transaction.tid` permits integers, TOML mapping keys do not.
-
-Paper transactions commonly set `Transaction.bs_mktid` to the full
-FQME. The embedded `Position.mkt.bs_mktid` can still retain the
-provider-native market ID. Check the concrete object and table owner
-before asserting key equality.
-
-## Accounting invariants
-
-- `Transaction.size` is signed; preserve side semantics during
-  broker normalization.
-- `Transaction` is the normalized in-memory interchange. Live ledger
-  records may remain backend-native and normalize through
-  `mod.norm_trade()` when read.
-- `Position._events` is keyed by `tid`; `add_clear()` is the
-  idempotence boundary.
-- `Position.ppu` and `Position.cumsize` are derived from transaction
-  events, not independent mutable truth.
-- Normalized accounting transactions need a stable, non-null
-  `bs_mktid`; `Account.pps` uses it as the position key.
-- `Account.update_from_ledger()` resolves market metadata from
-  `SymbologyCache.mktmaps` or its explicit fallback table.
-- UI position messages are projections. Do not feed their synthetic
-  reset event back into durable broker accounting.
-
-## Persistence contracts
-
-Trade ledgers live under the accounting ledger directory as
-`trades_<broker>_<account>.toml`. Top-level keys are transaction IDs;
-record schemas are backend-specific unless the paper engine wrote a
-normalized transaction dictionary.
-
-`open_trade_ledger()` writes on context exit when its data comparison
-detects a change or `rewrite=True`. Its snapshot is shallow, so nested
-in-place record mutation can evade dirty detection. Explicit writes can
-also fail inside fill handlers. Treat all persistence points as part of
-the live execution path.
-
-Accounts persist active positions. `dump_active()` separates open and
-net-zero positions. Do not assume `minimized_clears()` safely preserves
-size/PPU: current sign-transition and longer net-zero slicing lacks
-round-trip coverage and can retain the wrong event subset.
-
-## Market identity and symcache
-
-`SymbologyCache` has three distinct key-addressed mappings:
-
-- `assets`: provider asset ID to `Asset`;
-- `pairs`: native `bs_mktid` to backend pair `Struct`;
-- `mktmaps`: searchable FQME or explicit alias to normalized
-  `MktPair`.
-
-Search mapping keys, then resolve values. RapidFuzz treats mapping
-values as choices; passing `mktmaps` directly makes it operate on
-`MktPair` objects. Preserve in-memory aliases because IB can store both
-native IDs and FQMEs for one `MktPair`. Do not persist such aliases
-without fixing reload: `from_dict()` currently requires every key to
-equal `mkt.fqme`.
-
-## Change workflow
-
-Before editing:
-
-1. Identify the owning layer: client, EMS, executor, accounting, or UI.
-2. Write down every ID/key transition across that boundary.
-3. Trace both normal and cancel-before-ack event ordering.
-4. Determine whether records are native, normalized, or summaries.
-5. Check actor-local caches and context-exit writes.
-6. Add a deterministic regression at the narrowest broken contract.
-7. Run live/backend tests only when their credentials and side effects
-   are explicitly authorized.
-
-Do not fix an accounting defect by adding state to EMS, or fix an EMS
-correlation defect by rewriting durable transaction history.
-
-## Canonical source map
-
-- `piker/clearing/_messages.py`: client and brokerd protocol types.
-- `piker/clearing/_client.py`: `OrderClient` and `open_ems()`.
-- `piker/clearing/_ems.py`: router, dark book, ID mapping, translation.
-- `piker/clearing/_paper_engine.py`: simulated execution/accounting.
-- `piker/accounting/_ledger.py`: transactions and ledger persistence.
-- `piker/accounting/_pos.py`: positions, accounts, PPU event state.
-- `piker/accounting/_mktinfo.py`: `Asset`, `MktPair`, FQME schema.
-- `piker/data/_symcache.py`: provider symbology mappings and search.
-- `piker/accounting/calc.py`: transaction ordering and PPU/PnL math.
-- `piker/brokers/*/broker.py`: backend order-control integration.
-- `piker/brokers/*/ledger.py`: backend trade normalization.
-
-See [architecture.md](architecture.md) for complete flows,
-[gotchas.md](gotchas.md) for symptom-driven diagnosis, and
-[test-map.md](test-map.md) for verification choices.

.agents/skills/piker-clearing-expert/architecture.md

@@ -1,210 +0,0 @@
-# Clearing And Accounting Architecture
-
-## Protocol direction
-
-Client-facing messages and broker-facing messages intentionally differ.
-
-```text
-client -> EMS
-  Order, Cancel
-
-EMS -> executor
-  BrokerdOrder, BrokerdCancel
-
-executor -> EMS
-  BrokerdOrderAck, BrokerdStatus, BrokerdFill,
-  BrokerdError, BrokerdPosition
-
-EMS -> subscribed clients
-  Status, BrokerdPosition
-```
-
-`Order` carries the client `oid`, account, market, price, size, action,
-and execution mode. `BrokerdOrderAck` establishes the executor's
-`reqid`; subsequent broker messages should carry that ID so EMS can
-resolve the client dialog.
-
-`Status` is mutable current dialog state, not a history envelope. Exact
-client states include `pending`, `open`, `dark_open`, `triggered`,
-`fill`, `closed`, `canceled`, and `error`. Audit existing translation
-before adding a state; backend `BrokerdStatus` vocabulary is narrower
-and current producers are not fully conformant.
-
-## EMS routing
-
-`_emsd_main()` serves each client/market context. The actor-local
-`Router` owns books, subscribers, order dialogs, and cached live trades
-relays. Paper mode opens per-symbol relay contexts under the same broker
-key, a current overwrite/pop hazard for concurrent paper symbols.
-
-`open_brokerd_dialog()` is the session live-versus-paper selection
-point:
-
-- live execution talks to `brokerd.<broker>`;
-- paper execution talks to `paperboi.<broker>`;
-- chart-only and paper sessions should not boot credentialed brokerd.
-
-Do not infer paper execution from each order's `exec_mode`. Session
-mode selects the executor; per-order mode primarily distinguishes dark
-from immediately routed orders, despite incomplete literal annotations.
-
-`process_client_order_cmds()` handles submit, modify, cancel, dark,
-and alert intent. It updates EMS dialog state before routing broker
-commands. `translate_and_relay_brokerd_events()` applies acks, statuses,
-fills, errors, and positions to dialogs and subscribers.
-
-The `DarkBook` stores local predicates and watches quote ticks. When a
-predicate triggers, `clear_dark_triggers()` emits the live broker order
-and updates the existing client dialog rather than inventing a second
-order identity.
-
-## Ack and cancel ordering
-
-Required actionable ordering is submit, ack, open, zero or more fills,
-then terminal status. EMS handles cancel-before-ack, but it does not
-buffer arbitrary status-before-ack events; those currently fall through
-as unhandled. Backends must ack before actionable statuses.
-
-The important race is:
-
-```text
-client Cancel(oid)
-  -> EMS has no reqid yet
-  -> mark Status.cancel_called
-BrokerdOrderAck(oid, reqid)
-  -> install mapping
-  -> send deferred BrokerdCancel(reqid)
-```
-
-Do not drop a pre-ack cancel and do not send a broker cancel with an
-`oid` unless the backend explicitly uses that value as its `reqid`.
-
-## Paper execution
-
-`PaperBoi` maintains simulated orders, opens the market feed, and fills
-orders when L1 ask/bid quotes cross their prices. `trade`/`last` ticks
-zip buy and sell iterators, so a lone resting side currently may not
-fill from those tick types. `fake_fill()` is the core fill-to-account
-path:
-
-1. emit `BrokerdFill`;
-2. emit terminal `BrokerdStatus` for the simulated order;
-3. construct a signed normalized `Transaction`;
-4. `TransactionLedger.update_from_t()`;
-5. `Account.update_from_ledger()`;
-6. persist the ledger and account;
-7. emit `BrokerdPosition` with current size and PPU.
-
-The protocol notifications and persistence occur in one task but are
-not a database transaction. If persistence fails after fill/status
-publication, restart/reconciliation behavior matters. Do not reorder
-the sequence casually.
-
-Partial fills currently reuse `oid` as `tid`. A later partial therefore
-overwrites the paper ledger record and is rejected by position-event
-dedupe. Fix unique fill IDs before relying on paper partial-fill math.
-
-`open_trade_dialog()` opens symcache, paper ledger, account, and feed
-resources under structured concurrency. An exception from any nested
-resource, including ledger context exit, cancels the dialog and its feed
-stream.
-
-## Live backend variation
-
-There is no universal live-broker accounting implementation:
-
-- IB normalizes execution/commission events, updates ledger/account
-  state, and emits position updates.
-- Kraken has backend trade normalization and position reconstruction,
-  but its live path writes account state without inserting every new
-  trade into `TransactionLedger`.
-- Binance currently relays venue order/position events and does not
-  share every local `TransactionLedger`/`Account` path.
-
-Inspect the backend's `broker.py`, `ledger.py`, and `norm_trade()` before
-moving generic logic. A paper behavior is not automatically a live
-backend contract.
-
-## Ledger normalization
-
-`TransactionLedger.data` is the durable record mapping. For paper it
-contains transaction-like dictionaries. For live brokers it can contain
-raw provider records.
-
-`iter_txns()` sorts records and chooses normalization:
-
-- paper uses `_paper_engine.norm_trade()`;
-- live accounts use `brokermod.norm_trade()`.
-
-Normalizer output should carry datetime, signed size, price, cost,
-FQME, backend market ID, and transaction ID. Some normalizers return
-`None` for unusable records; `iter_txns()` does not filter it, while
-`to_txns()` does. Audit direct iterator consumers.
-
-`write_config()` sorts records for stable persistence. Paper rewrite
-logic migrates old symbol fields and qualifies legacy FQMEs through the
-symcache. Search can return no matches; callers which select the first
-result need an explicit fallback or a descriptive failure policy.
-
-## Position event model
-
-`Position` holds a market and a transaction-event mapping. New clears
-are deduplicated by `tid`. Current size and PPU are recomputed from the
-event sequence.
-
-`Account.update_from_ledger()`:
-
-1. receives normalized transactions;
-2. resolves each market through `mktmaps` or a fallback table;
-3. looks up or creates the position by `Transaction.bs_mktid`;
-4. adds unseen clears;
-5. returns visited positions, including duplicates whose clear was
-   rejected by `add_clear()`.
-
-`minimized_clears()` intends to discard irrelevant history after
-net-zero or sign transitions, but current slicing is not proven safe
-and can reconstruct the wrong active size/PPU. Treat it as a known bug
-surface until sign-transition round trips are covered.
-
-`Position.update_from_msg()` serves UI summaries by replacing local
-event state with a synthetic clear representing reported size and PPU.
-That projection is intentionally lossy.
-
-## Symbology contracts
-
-`MktPair.fqme` is a normalized endpoint address assembled from pair,
-venue, expiry/contract information, and broker. `MktPair.bs_fqme`
-drops the broker suffix. `MktPair.bs_mktid` is the provider-native
-market identity.
-
-`SymbologyCache.load()` attempts to obtain provider assets and pair
-structs, calls `get_mkt_info()`, and populates conventional mappings.
-Assets are optional; missing/empty pair support returns early.
-
-```text
-assets[provider_asset_id] -> Asset
-pairs[pair.bs_mktid]      -> backend Pair Struct
-mktmaps[mkt.fqme]         -> MktPair
-```
-
-Some backends add aliases such as `mktmaps[bs_mktid]`. Search must
-return the matched mapping key rather than rebuilding a key from the
-value, otherwise aliases silently collapse. Current serialization then
-writes aliases which reload rejects because `from_dict()` asserts each
-key equals `mkt.fqme`.
-
-## Persistence boundaries
-
-`open_trade_ledger()` loads account TOML and compares a shallow snapshot
-in `finally`. Top-level additions/replacements trigger writes; nested
-in-place mutation may not. `open_account()` similarly mediates account
-state. These contexts run inside actor tasks, so serialization errors
-are runtime failures, not merely offline maintenance failures.
-
-When changing schemas:
-
-- preserve backend-native records unless migration is explicit;
-- keep old paper field migration deterministic;
-- avoid fuzzy qualification when an exact `bs_mktid`/FQME exists;
-- test context exit, not only in-memory updates;
-- inspect emitted positions after disk round trips.

.agents/skills/piker-clearing-expert/gotchas.md

@@ -1,146 +0,0 @@
-# Clearing And Ledger Gotchas
-
-## Trade dialog dies immediately after a fill
-
-Likely cause: an explicit ledger/account write in the fill handler, or
-a later context-exit write, raised after event publication.
-
-Inspect, in order:
-
-1. the first non-cancellation exception in the actor traceback;
-2. `PaperBoi.fake_fill()` or the live backend fill handler;
-3. explicit and context-exit `write_config()` calls;
-4. FQME/`bs_mktid` qualification and TOML-compatible values;
-5. the persisted ledger/account files for a partial update.
-
-Tractor cancellation logs are often the consequence. Root-cause the
-inner serialization, lookup, or normalization exception first.
-
-## `MktPair` has no `len()` in RapidFuzz
-
-Cause: a mapping of string keys to `MktPair` values was passed directly
-to `rapidfuzz.process.extract()`. Mapping values become fuzzy choices.
-
-Fix: search a list/view of string keys and resolve matched keys back
-through the original mapping. Do not add sequence methods to `MktPair`
-and do not use `processor=str` when aliases must be preserved.
-
-## Fuzzy qualification raises `IndexError`
-
-Cause: the caller selected the first match from an empty result after
-the score cutoff rejected every key.
-
-Decide explicitly whether to retain the input identity, reject the
-record with a descriptive exception, or obtain missing market metadata.
-Do not lower the cutoff blindly; the wrong market corrupts accounting.
-
-## Cancel is ignored or sent with the wrong ID
-
-Cause: confusion between client `oid` and backend `reqid`, commonly
-during cancel-before-ack ordering.
-
-Verify the ack installed both mapping directions and that
-`Status.cancel_called` survives until `reqid` exists. Log both IDs and
-the actor/backend account at every transition.
-
-## Fills appear but positions do not move
-
-Check these boundaries:
-
-- the backend emitted `BrokerdFill` but never normalized a transaction;
-- transaction size lost its buy/sell sign;
-- `tid` collided with an existing position event;
-- `Transaction.bs_mktid` selected a different `Account.pps` key;
-- `mktmaps` lacked the exact transaction identity;
-- position publication happened before accounting update;
-- the backend intentionally reports venue positions instead of local
-  ledger-derived positions.
-
-## Duplicate or inflated position size
-
-Do not deduplicate by price/time alone. Confirm stable transaction IDs,
-chronological sorting, and `Position.add_clear()` idempotence. Broker
-execution IDs and order IDs are not interchangeable.
-
-For partial fills, several `tid`s may correctly share one order
-`reqid`. Collapsing them loses clears; replaying one `tid` twice
-inflates nothing only if the dedupe boundary is preserved.
-
-Current paper fills use `tid=oid`, so later partials overwrite the
-ledger record and are deduplicated out of the position. Treat paper
-partial fills as broken until each clear receives a stable unique ID.
-
-## Paper and live state disagree
-
-Paper uses normalized local transactions and can use the full FQME as
-its account position key. A live backend may use native IDs, raw ledger
-records, or venue-reported positions.
-
-Compare normalized `Transaction`s and emitted `BrokerdPosition`s, not
-raw TOML dictionaries across backends.
-
-## Account appears under the wrong name
-
-Separate UI/EMS account aliases from backend account IDs and ledger file
-names. Backends may prefix account names for routing while persistence
-uses an unprefixed native account key.
-
-Trace account identity through `open_ems()`, broker dialog startup,
-ledger opening, and `BrokerdPosition.account` before changing naming.
-
-## A module-global cache is unexpectedly empty
-
-Symcache, clients, contracts, and order tables are actor-local. Data
-loaded in `datad` is not automatically available in `brokerd` or
-`paperboi`. Also, `_symcache._caches` is currently read but never
-populated, so do not assume repeated `open_symcache()` calls hit it.
-
-Load `piker-conc-expert` and audit actor-local globals, async caches,
-and dialog startup warming. Never rely on an import side effect from a
-sibling actor.
-
-## Teardown emits a cancellation storm
-
-Find the first application exception. Structured cancellation then
-closes feed streams, broker streams, and nested contexts. Avoid masking
-the originating exception with broad cancellation handling.
-
-For stream ownership and context cancellation semantics, use
-`piker-conc-expert`.
-
-## Broker status is logged as unhandled
-
-EMS currently accepts only ack-correlated `open`, `closed`, and
-`canceled` broker statuses. Declared `pending` and backend-produced
-`fill`/`filled` variants fall through, as do statuses arriving before
-ack. Verify producer vocabulary and ack ordering before adding another
-consumer branch.
-
-## Another client sees order or position events
-
-Current translation broadcasts order updates by FQME and positions to
-all attached clients; `Router.dialogs` is not used as an event privacy
-filter. Treat subscriber isolation as incomplete, not guaranteed.
-
-## Debug breadcrumb
-
-Capture one row per event with:
-
-```text
-actor, backend, account, fqme, bs_mktid,
-oid, reqid, venue_order_id, tid,
-message_type, status, action, signed_size, price, timestamp
-```
-
-Then compare:
-
-1. client command order;
-2. EMS status history;
-3. broker event order;
-4. normalized transactions;
-5. position event IDs and computed size/PPU;
-6. ledger/account disk state;
-7. emitted position summaries.
-
-This separates protocol loss, identity skew, accounting math, and
-persistence failure without guessing from UI state.

.agents/skills/piker-clearing-expert/test-map.md

@@ -1,84 +0,0 @@
-# Clearing And Accounting Test Map
-
-## Deterministic first-pass tests
-
-Use the narrowest test that owns the broken contract:
-
-| Area | First target |
-|---|---|
-| Ledger/account persistence | targeted node in `tests/test_accounting.py` |
-| EMS bad-backend handling | `tests/test_ems.py::test_ems_err_on_bad_broker` |
-| IB request/response routing | `tests/test_ib_method_proxy.py` |
-| IB history normalization | `tests/test_ib_history.py` |
-| Service/actor startup | targeted node in `tests/test_services.py` |
-
-Known missing high-value regressions include paper partial-fill IDs,
-status-before-ack handling, `minimized_clears()` sign-transition round
-trips, nested ledger dirty detection, and symcache alias reload.
-
-For `SymbologyCache.search()` and paper ledger qualification, cover:
-
-- real `Asset` and `MktPair` values;
-- a native-ID alias and canonical FQME key;
-- the actual `TransactionLedger.write_config()` path;
-- persisted TOML fields after the round trip;
-- no use of live APIs or user configuration.
-
-## Side-effectful tests
-
-Do not run these without explicit authorization and the repository test
-harness guidance:
-
-- paper EMS tests which open live market/symbology feeds;
-- live Binance/Kraken feed suites;
-- IB account tests using configured accounts;
-- tests which write tracked ledger/account fixtures;
-- broker dialogs requiring credentials or venue connectivity.
-
-After any accounting test using `tests/_inputs`, inspect its diff. Some
-context managers can rewrite fixture ledgers on exit.
-
-## Regression design
-
-Prefer a direct contract regression over a full actor test when the
-failure is synchronous and deterministic. Use an actor-level test when
-the bug depends on:
-
-- ack/cancel or fill/status interleaving;
-- context cancellation or stream ownership;
-- actor-local state/cache separation;
-- daemon selection or startup;
-- publication ordering across EMS and broker streams.
-
-Every regression should document the original failure, triggering
-state, violated invariant, arrangement, and proof of the fix.
-
-## Verification layers
-
-1. `git diff --cached --check` for the exact staged boundary.
-2. Ruff with the repository's intended rule profile.
-3. A deterministic unit/filesystem regression.
-4. Adjacent accounting or EMS tests which do not require live state.
-5. Explicitly authorized backend/actor integration tests.
-
-Do not interpret a passing ellipsis-body or hard-skipped test as
-behavioral coverage. Read the selected node before using it as a gate.
-
-## Manual reconciliation check
-
-For production-only failures, compare one fill end to end:
-
-```text
-BrokerdFill
-  -> normalized Transaction
-  -> ledger record
-  -> Position._events[tid]
-  -> Position.cumsize / ppu
-  -> account TOML
-  -> BrokerdPosition
-  -> UI summary
-```
-
-All identities, sign, price, and account values should remain explainable
-at each arrow. A mismatch identifies the owning boundary for the next
-focused test.

.agents/skills/piker-fsp-expert/SKILL.md

@@ -1,78 +0,0 @@
----
-name: piker-fsp-expert
-description: Piker financial signal processing expertise. Apply when changing or debugging FSP definitions, Cascade execution, derived shared-memory arrays, history/realtime handoff, backfill synchronization, FSP graph composition, online financial algorithms, or FSP chart lifecycle and scaling.
----
-
-# Piker FSP Expertise
-
-Treat financial signal processing as causal, revision-aware processing of
-event-time market data, not as generic batch array manipulation.
-
-## Start Here
-
-1. Identify the source `Flume`, timeframe, event clock, and writer actor.
-2. State whether the operation is historical bootstrap, backfill revision,
-   current-sample mutation, or sample-step append.
-3. Record source and destination absolute SHM bounds before reasoning from
-   lengths or `ShmArray.index`.
-4. Separate compute, SHM publication, IPC wakeup, and graphics costs before
-   optimizing.
-5. Preserve causality, warm-up behavior, missing-data semantics, and output
-   schema in every implementation and test.
-
-Read [architecture.md](architecture.md) before modifying runtime code. Read
-[signal-processing.md](signal-processing.md) before adding an algorithm. Use
-[roadmap-and-research.md](roadmap-and-research.md) for design and dependency
-work, and [test-map.md](test-map.md) for verification.
-
-## Core Contracts
-
-- `piker.data` owns provider ingestion, source `Flume`s, history publication,
-  quote fan-out, OHLC mutation, and `samplerd` step events.
-- `piker.fsp` owns operator declarations, historical computation, realtime
-  state updates, and destination SHM publication.
-- `piker.ui._fsp.FspAdmin` currently owns FSP worker selection, destination
-  allocation, cascade lifetime, and chart attachment.
-- One logical writer owns each destination FSP array.
-- Output event time and absolute indices derive from the source series, not
-  task scheduling or quote arrival order.
-- A source prepend changes historical provenance even when the current sample
-  index is unchanged. Recompute or patch the affected destination range.
-- One-step destination lag is normal between source sampling and destination
-  append. A destination lead or unexplained bound shift is not.
-- Backfill notifications are source and timeframe specific even though the
-  current sampler transport broadcasts them globally.
-- Only graphics backed by a rebuilt destination should refresh. Never use an
-  FSP update as an unconditional whole-chart redraw request.
-
-## Editing Workflow
-
-1. Trace the operator from `@fsp` declaration through `cascade()`,
-   `connect_streams()`, destination SHM, `FspAdmin`, and every consuming
-   `Viz`.
-2. Write an event-order table covering source writes, bound publication,
-   notifications, cancellation, recomputation, and destination publication.
-3. Add deterministic tests for history and realtime phases independently.
-4. Add a controlled interleaving test for every race fix.
-5. Profile representative visible ranges and feed rates before adding a
-   dependency or native boundary.
-
-## Companion Skills
-
-- Use `piker-conc-expert` for Tractor contexts, actor topology, cancellation,
-  service ownership, and cross-actor state.
-- Use `timeseries-optimization` for NumPy/Polars lookup and vectorization.
-- Use `piker-profiling` for distributed performance measurements.
-- Use `pyqtgraph-optimization` for renderer and graphics batching work.
-- Use `piker-clearing-expert` when a derived signal reaches EMS or accounting.
-
-## Do Not
-
-- Do not hide a historical mismatch merely to stop a resync log.
-- Do not infer a complete synchronization state from array length alone.
-- Do not run Python loops over historical numeric arrays without evidence.
-- Do not apply regular-grid DSP blindly to irregular tick arrival times.
-- Do not introduce an external stream runtime that duplicates Tractor before
-  measuring an isolated protocol or kernel prototype.
-- Do not route strategy signals directly around EMS order-state, risk, or
-  accounting contracts.

.agents/skills/piker-fsp-expert/architecture.md

@@ -1,117 +0,0 @@
-# FSP Runtime Architecture
-
-## Current Topology
-
-```text
-provider backend
-  -> datad feed bus
-  -> source Flume
-     -> rt ShmArray (typically 1 second)
-     -> hist ShmArray (typically 60 seconds)
-  -> samplerd sample and backfill events
-
-chart actor
-  -> FspAdmin
-  -> chart.fsp_N worker actor
-     -> cascade()
-     -> connect_streams()
-     -> destination Flume / ShmArray
-  -> targeted Viz updates
-```
-
-The source path is implemented mainly by `piker.data.feed`,
-`piker.data.flows`, `piker.data._sampling`, and `piker.tsp._history`. The FSP
-path is implemented by `piker.fsp._api`, `piker.fsp._engine`, and operator
-modules such as `_volume` and `_momo`. Worker and graphics lifecycle currently
-live in `piker.ui._fsp` and `piker.ui._display`.
-
-## Operator Contract
-
-`@fsp` wraps an async generator. Its first yield is historical output computed
-from the source array. Later yields are `(field, value)` realtime mutations.
-`connect_streams()` converts the first yield into the destination structured
-array, aligns time and absolute bounds, then updates the current destination
-row from realtime quote events.
-
-`samplerd` owns row advancement. At a new source sample, the cascade copies or
-zeros the previous destination row and aligns destination timestamps to the
-source. Quote-time mutations and sample-time appends are distinct operations.
-
-## SHM Synchronization
-
-Treat each array as an absolute half-open range:
-
-```text
-[first, last)
-```
-
-For a normal source history prepend:
-
-```text
-before: src=[10000, 12000) dst=[10000, 12000)
-after:  src=[ 8000, 12000) dst=[10000, 12000)
-```
-
-The current step is unchanged, but destination history is missing 2,000 rows.
-The destination must be recomputed or incrementally repaired.
-
-For a normal one-step source lead:
-
-```text
-src=[8000, 12001) dst=[8000, 12000)
-```
-
-The cascade should append exactly one destination row, not recompute history.
-
-Use `_first.value` and `_last.value` for synchronization. `ShmArray.index` is
-the last bound modulo capacity and is not a complete absolute position.
-Snapshot bounds once per check; shared first and last counters do not form a
-transactional pair.
-
-## Backfill Event Ordering
-
-1. `start_backfill()` receives an older provider frame.
-2. `shm_push_in_between()` writes the frame and publishes the earlier first
-   bound.
-3. `notify_backfill()` asks `samplerd` to broadcast the market and timeframe.
-4. Relevant cascades treat the event as a history revision. A prepend changes
-   bounds; an in-place gap repair may not.
-5. Each cascade cancels its current compute task and waits for completion.
-6. The historical phase recomputes against the newest readable source range.
-7. Destination bounds are published and the UI receives an `fsp_update`.
-8. Only `Viz` objects backed by that destination token redraw.
-
-Multiple provider frames may arrive while a recomputation runs. The cascade
-must reject mixed-bound bootstrap results, replay in-place revisions, recheck
-bounds after restart, and converge on the newest source range.
-
-## Actor-Local State
-
-`Fsp._flow_registry`, attached SHM handles, caches, and nurseries are local to
-each actor. Never assume worker siblings share warmed registries or Python
-objects. Graph dependencies currently rely on startup order, as demonstrated
-by `flow_rates` starting only after `dolla_vlm`.
-
-## Cross-Subsystem Boundaries
-
-- Data ingest should normalize identity and event-time semantics before an FSP
-  consumes a stream.
-- UI renderers should consume revision/range information without controlling
-  computation correctness.
-- Clearing may consume a derived signal, but EMS remains responsible for order
-  intent, status, routing, fills, positions, and accounting.
-- FSP output exposed as a feed needs the same fan-out, identity, lifecycle,
-  and backpressure contracts as provider feeds.
-
-## Architectural Gaps
-
-The current protocol infers history revisions from shared bounds. A future
-revision message should carry at least:
-
-```text
-(fqme, timeframe, revision, first, last, start_ts, end_ts)
-```
-
-Destination publication should identify the source revision used. This allows
-coalescing, range repair, stale-result rejection, and deterministic graph
-dependency startup without turning every prepend into a global wakeup.

.agents/skills/piker-fsp-expert/roadmap-and-research.md

@@ -1,104 +0,0 @@
-# Roadmap and Research
-
-Issue states change. Re-read each tracker before planning work and preserve
-human ownership of issue and checklist state.
-
-## Guiding Roadmap
-
-- [piker #270](https://github.com/pikers/piker/issues/270): typed,
-  stream-composable FSP graph and DSL direction.
-- [piker #107](https://github.com/pikers/piker/issues/107): cross-actor shared
-  memory tick/ring-buffer direction.
-- [piker #216](https://github.com/pikers/piker/issues/216): exposing FSP output
-  through feed-like interfaces.
-- [piker #325](https://github.com/pikers/piker/issues/325): faster streaming
-  y-range computation and overlap with FSP reductions.
-- [piker #536](https://github.com/pikers/piker/issues/536): Arrow/Parquet data
-  ingest and columnar boundaries.
-- [piker #98](https://github.com/pikers/piker/issues/98): older realtime feed
-  architecture discussion. Treat it as likely superseded; compare every idea
-  against current `Flume`, feed-bus, history, and sampler code.
-- [Tractor #339](https://github.com/goodboy/tractor/issues/339): shared-memory
-  array and localhost IPC evolution.
-- [hotbaud issues](https://github.com/guilledk/hotbaud/issues): experimental
-  high-throughput local transport and ring-buffer work.
-
-## Near-Term Protocol Research
-
-Prototype an explicit source revision envelope and measure it with two FSPs
-and two overlaid feeds. The prototype should prove source-specific wakeups,
-coalescing of consecutive prepends, stale-result rejection, and targeted UI
-range updates. Keep the protocol independent of the final IPC transport.
-
-Characterize historical repair strategies:
-
-1. Full replay from the new first bound.
-2. Prefix-only calculation for pointwise operators.
-3. Replay from an operator checkpoint for stateful recurrences.
-4. Bounded overlap repair for finite-window operators.
-
-An FSP declaration eventually needs repair metadata describing locality,
-state checkpointability, warm-up/overlap, and whether old inputs can alter all
-later outputs.
-
-## Candidate Libraries
-
-Candidates are experiments, not dependency recommendations.
-
-### Bounded Near-Term Prototypes
-
-- [Bottleneck](https://github.com/pydata/bottleneck): compare rolling min/max,
-  rank, and moving-window kernels against NumPy/Numba for chart y-ranges and
-  finite-window FSPs.
-- [River](https://github.com/online-ml/river): study stateful online estimator
-  APIs, drift detectors, and reference semantics. Do not assume its object
-  model belongs in the hottest tick path.
-- [nanoarrow](https://github.com/apache/arrow-nanoarrow): evaluate lightweight
-  Arrow C data interfaces at native/process boundaries already motivated by
-  `pyarrow` and piker #536.
-- [DuckDB](https://github.com/duckdb/duckdb): use for historical Parquet query,
-  replay slicing, and validation, outside realtime tick execution.
-
-### Watchlist
-
-- [hotbaud](https://github.com/guilledk/hotbaud): track API and correctness
-  maturity for localhost SHM/ring transport.
-- [iceoryx2](https://github.com/eclipse-iceoryx/iceoryx2): study zero-copy
-  pub/sub ownership and lifecycle, but avoid competing with Tractor without a
-  narrow adapter benchmark.
-- [DataFusion Python](https://github.com/apache/datafusion-python): Arrow-native
-  historical query and expression execution if existing Polars paths cannot
-  meet a demonstrated need.
-- [Feldera](https://github.com/feldera/feldera): study DBSP/incremental-view
-  semantics for revision-aware graphs; its service/SQL runtime is not a direct
-  piker integration target.
-- [Bytewax](https://github.com/bytewax/bytewax) and
-  [Arroyo](https://github.com/ArroyoSystems/arroyo): study dataflow semantics,
-  watermarking, and recovery, not as replacements for the actor tree.
-
-GPU stacks such as JAX and CuPy are poor defaults for low-batch per-tick work
-because transfer, compilation, and scheduling can dominate. Reconsider only
-for measured large-batch research or many-market matrix workloads.
-
-## Benchmark Design
-
-Report separate distributions for:
-
-- provider decode and normalization;
-- source SHM write and bound publication;
-- historical kernel and realtime recurrence;
-- actor notification and stream fan-out;
-- destination publication;
-- visible-range formatting and Qt paint.
-
-Use p50, p95, p99, maximum, allocation rate, and resident-memory growth. Test
-steady state, burst load, backfill overlap, cancellation, and slow consumers.
-Throughput without event-time correctness and bounded teardown is not a win.
-
-## Medium-Term Sequence
-
-First establish revisioned range messages and operator repair semantics. Then
-decouple FSP orchestration from chart ownership and expose destination flows
-through feed-like subscriptions. Next benchmark localhost transport adapters
-behind the same protocol. Only after those contracts stabilize should graph
-placement, fusion, worker scaling, and remote partitioning be automated.

.agents/skills/piker-fsp-expert/signal-processing.md

@@ -1,103 +0,0 @@
-# Financial Signal Processing
-
-## Model the Clock First
-
-Market inputs can be quote events, trades, book changes, fixed-time bars, or
-revised historical bars. Name both event time and processing time. For
-irregular events, either use time-aware recurrences or define the resampling
-and interpolation policy explicitly.
-
-Never let backfilled information leak into a value presented as historically
-tradable. Distinguish corrected analysis history from the signal values that a
-live strategy could have observed.
-
-## Useful Technique Families
-
-### Online Moments and Robust Statistics
-
-Use Welford/Chan recurrences for stable online moments, compensated summation
-for long accumulations, and mergeable summaries when partitioning actors.
-Rolling median, quantiles, MAD, and winsorized measures are useful under heavy
-tails, but exact rolling order statistics need more state than moments.
-
-### Time-Aware Exponential Filters
-
-For irregular event intervals, derive decay from elapsed time:
-
-```text
-alpha(dt) = 1 - exp(-dt / tau)
-y_t = y_prev + alpha(dt) * (x_t - y_prev)
-```
-
-Record initialization, session reset, gap, and backfill behavior. A fixed
-sample alpha is valid only after an explicit regularization policy.
-
-### State-Space Models
-
-Kalman and robust state-space filters fit latent price, spread, volatility,
-and lead/lag estimates. Prefer square-root or Joseph-form covariance updates
-when conditioning is poor. State revisions after backfill require replay from
-a checkpoint or a bounded smoother; silently prepending observations without
-replaying state is incorrect.
-
-### Point Processes and Microstructure
-
-Trade and quote arrivals are events, not merely sampled amplitudes. Consider
-intensity, duration, signed order flow, imbalance, spread, queue change, and
-self-excitation models. Validate against provider-specific aggregation and
-duplicate/out-of-order behavior before interpreting a statistic.
-
-### Volatility and Covariance
-
-Realized variance, bipower variation, EW covariance, and range estimators can
-be updated online. Asynchronous cross-market covariance needs synchronization
-or estimators designed for non-synchronous observations; naive row alignment
-creates lead/lag and Epps-effect artifacts.
-
-### Spectral and Multiscale Methods
-
-FFT methods assume a regular grid and a window. Declare detrending, tapering,
-overlap, normalization, and latency. For tick data, resample deliberately or
-use irregular-time methods. Wavelets and multiresolution filters can separate
-horizons, but boundary handling and causal delay must be visible to strategy
-code.
-
-### Change and Anomaly Detection
-
-CUSUM, Page-Hinkley, sequential likelihood ratios, and robust z-scores are
-cheap online tools. Calibrate false alarms under dependence and regime shifts;
-do not treat IID thresholds as market guarantees.
-
-## Numerical Implementation Order
-
-1. Establish a scalar reference with explicit state and semantics.
-2. Vectorize historical bootstrap with NumPy.
-3. Keep realtime updates as O(1) recurrences where possible.
-4. Use Numba for measured numeric kernels with stable dtypes and no Python
-   object traffic.
-5. Use Polars or Arrow for columnar historical transforms and interchange,
-   not automatically for each tick.
-6. Partition by independent market/operator state before adding threads or a
-   new distributed runtime.
-
-## Existing Stack
-
-- NumPy 2.x: canonical dense and structured-array kernels.
-- Numba: compiled CPU loops and recurrences that do not vectorize cleanly.
-- Polars: parallel/lazy columnar history preparation and validation.
-- PyArrow: columnar interchange, storage, and ingest boundaries.
-- Tractor/Trio: structured distributed execution and lifecycle.
-- PyQtGraph: visible-range rendering, not a compute scheduler.
-
-## Algorithm Acceptance Criteria
-
-Every new FSP should define:
-
-- input fields, clock, ordering, duplicate, gap, and revision policy;
-- output dtype, units, identity, and parameterization;
-- warm-up length and initialization bias;
-- causal latency and strategy-visible publication point;
-- historical/realtime equivalence tolerance;
-- reset and session-boundary behavior;
-- complexity, state size, and representative throughput;
-- replay behavior when historical source data changes.

.agents/skills/piker-fsp-expert/test-map.md

@@ -1,52 +0,0 @@
-# FSP Test Map
-
-## Operator Tests
-
-- Historical output schema, dtype, units, and source timestamp alignment.
-- Realtime update equivalence with the final row of a batch recomputation.
-- Warm-up, NaN, zero, gap, duplicate, and out-of-order behavior.
-- Session reset and elapsed-time behavior.
-- Parameter and source identity isolation.
-
-## Engine Tests
-
-- Equal absolute source/destination bounds.
-- Normal one-step destination lag.
-- Destination lead and lag greater than one step.
-- Source prepend by one, two, and a full provider frame.
-- In-place gap repair with unchanged source bounds.
-- Equal lengths with shifted absolute bounds.
-- Source change during historical recomputation and convergence afterward.
-- Cancellation while the quote stream, sample stream, or compute generator is
-  blocked.
-- Foreign FQME and timeframe backfill broadcasts.
-
-## Graph and Actor Tests
-
-- Dependency startup before downstream operator lookup.
-- Actor-local flow registry isolation.
-- One destination writer and multiple readers.
-- Consumer cancellation without orphaned cascade or worker tasks.
-- Worker failure propagation and SHM cleanup.
-- Feed-like fan-out with a slow or disconnected consumer.
-
-## UI Tests
-
-- An `fsp_update` redraws every field visualization sharing its destination
-  token and no unrelated market, OHLC, or FSP visualization.
-- A prepend outside the visible range does not move or re-range the live view.
-- A revised value inside the visible range invalidates only required formatter
-  and graphics caches.
-- Hidden charts defer rendering without stopping compute correctness.
-
-## Performance Tests
-
-- Historical bootstrap across realistic SHM capacities.
-- O(1) realtime recurrence cost under bursty quotes.
-- Consecutive provider-frame prepends with two or more cascades.
-- Multi-market sampler broadcasts and targeted wakeups.
-- Visible-range redraw cost at several units-per-pixel levels.
-
-Prefer deterministic local arrays and controlled Trio task interleavings.
-Use live providers only as a separate integration layer because credentials,
-rate limits, sessions, and changing history make poor regression oracles.

.claude/skills/piker-clearing-expert

@@ -1 +0,0 @@
-../../.agents/skills/piker-clearing-expert

.claude/skills/piker-fsp-expert

@@ -1 +0,0 @@
-../../.agents/skills/piker-fsp-expert

.gitignore

@@ -211,7 +211,6 @@ docs/conversations/
 # BEGIN ai.skillz: runtime:open-wkt
 .claude/wkts/
 claude_wkts
-/wkts/
 # END ai.skillz: runtime:open-wkt
 
 # BEGIN ai.skillz: direct:symlink:claude:open-wkt

ai/README.md

@@ -21,14 +21,11 @@ track new integration ideas and proposals in
 
 ## Shared Skills
 
-Provider-neutral skill bundles should use the portable Agent Skills
-subset and live under `.agents/skills/` as their single source.
-OpenCode discovers that project directory natively. Relative links in
-provider directories such as `.claude/skills/` may bridge clients which
-do not yet scan `.agents/skills/`; never copy the skill body.
-
-Existing repo skills under `.claude/skills/` predate this convention
-and can migrate separately without coupling their history to new skills.
+Repo-specific skills use the portable Agent Skills
+frontmatter subset and live under `.claude/skills/` as a
+single source. Claude Code discovers that directory
+natively, and OpenCode discovers the same project skills
+without copies or generated wrappers.
 
 Harness-specific command files should only delegate to a
 shared skill. They must not duplicate the skill body.

ai/claude-code/README.md

@@ -8,7 +8,6 @@ integration for piker's shared coding-harness skills.
 | Skill | Invocable | Description |
 |-------|-----------|-------------|
 | [`commit-msg`](#commit-msg) | `/commit-msg` | Generate piker-style commit messages |
-| `piker-clearing-expert` | auto | Clearing, order-control, and accounting invariants |
 | `piker-conc-expert` | auto | Actor-tree and structured-concurrency invariants |
 | `piker-profiling` | auto | `Profiler` API patterns for perf work |
 | `piker-slang` | auto | Communication style + slang guide |
@@ -20,12 +19,10 @@ Skills marked **auto** are background knowledge
 applied automatically when Claude detects relevance.
 Only `commit-msg` is user-invoked via slash command.
 
-New portable skill source files live under
-`.agents/skills/<skill-name>/SKILL.md`. Relative links under
-`.claude/skills/` expose them to Claude Code without copying bodies.
-Existing pre-migration skills can remain under `.claude/skills/`.
-Claude-specific behavior belongs in command or settings files, not
-shared skill bodies.
+Portable skill source files live under
+`.claude/skills/<skill-name>/SKILL.md` and are shared with
+OpenCode. Claude-specific behavior belongs in command or
+settings files, not the shared skill bodies.
 
 ---
 

ai/opencode/README.md

@@ -1,16 +1,14 @@
 # OpenCode Integration
 
-[OpenCode](https://opencode.ai/) discovers provider-neutral skills
-directly from `.agents/skills/` and pre-migration piker skills from
-`.claude/skills/`. There is no second copy of each skill to keep in
-sync.
+[OpenCode](https://opencode.ai/) uses piker's shared
+repo-specific skills directly from `.claude/skills/`.
+There is no second copy of each skill to keep in sync.
 
 ## Available Skills
 
 | Skill | Activation | Specialization |
 |-------|------------|----------------|
 | `commit-msg` | `/commit-msg` or explicit request | Piker commit-message style and artifacts |
-| `piker-clearing-expert` | automatic | Clearing, order-control, and accounting invariants |
 | `piker-conc-expert` | automatic | `tractor` actor topology, RPC, and cancellation |
 | `piker-profiling` | automatic | Cross-actor `Profiler` instrumentation |
 | `piker-slang` | automatic | Project communication style |
@@ -51,5 +49,3 @@ hot-reloaded into an active session.
   hardcode Claude Code attribution in shared workflows.
 - Keep supporting references beside each shared skill so
   relative links resolve in every harness.
-- Put new provider-neutral bundles under `.agents/skills/`; use
-  relative provider links only when a harness requires one.

ai/prompt-io/codex/20260915T155229Z_fadab3d2_prompt_io.md

@@ -1,36 +0,0 @@
----
-model: gpt-6 (exact variant not exposed)
-service: codex
-session: fsp-session-recovery-takeover-20260915
-timestamp: 2026-09-15T15:52:29Z
-git_ref: fadab3d2
-scope: code
-substantive: true
-raw_file: 20260915T155229Z_fadab3d2_prompt_io.raw.md
----
-
-## Prompt
-
-Recover the interrupted OpenCode FSP session and take over its ongoing
-work. The recovered child instruction records human authorization for
-the minimal EMA omitted-default fix followed by a complete commit plan.
-
-## Response summary
-
-Removed the eager Numba signature that rejected Python's omitted
-default arguments. Preserved nopython/nogil compilation and numerical
-behavior. Added nine numerical regressions and verified all 38 focused
-FSP cases. Rebuilt planning artifacts separately from production code.
-
-## Files changed
-
-- `piker/fsp/_momo.py`: drop the eager signature and unused type imports.
-- `tests/test_fsp_momo.py`: omitted/explicit argument and seed coverage.
-
-## Human edits
-
-The human identified the omitted-default limitation in the earlier
-session, authorized the minimal lazy-specialization approach, and
-requested this takeover after OpenCode failed. Existing source changes
-and older OpenCode provenance were retained. The new fix remains
-unstaged and awaits human review; no acceptance is inferred.

ai/prompt-io/codex/20260915T155229Z_fadab3d2_prompt_io.raw.md

@@ -1,50 +0,0 @@
----
-model: gpt-6 (exact variant not exposed)
-service: codex
-timestamp: 2026-09-15T15:52:29Z
-git_ref: fadab3d2
-diff_cmd: git diff HEAD~1..HEAD
----
-
-## Prompt
-
-> i'm thinking maybe you can just take over that work and we can
-> abandon that oc dialog for now?
-
-Recovered child-session instructions describe the earlier human
-authorization to remove EMA's eager Numba signature while retaining
-`jit(nopython=True, nogil=True)`, verify defaults, and finish the commit
-plan without staging or committing. This is recovered context, not a
-verbatim quote of the earlier human authorization.
-
-## Response
-
-> `git diff HEAD~1..HEAD -- piker/fsp/_momo.py`
-
-Removed EMA's eager Numba signature and now-unused Numba type imports.
-Retained nopython/nogil compilation, the numerical kernel, historical
-comments, and pre-existing staged and unstaged edits.
-
-> `git diff HEAD~1..HEAD -- tests/test_fsp_momo.py`
-
-Added production-dispatcher regressions for omitted positional and
-keyword arguments, explicit None and numeric parameters, and the
-previously corrected one-sample continuation. Expected outputs are
-hand-computed recurrences, rather than the Python implementation.
-
-Before the fix, the first case failed with:
-
-```text
-TypeError: No matching definition for argument type(s)
-array(float64, 1d, C), omitted(default=None), omitted(default=None)
-```
-
-After the fix, the focused run outside the socket-restricted sandbox
-reported:
-
-```text
-38 passed in 0.66s
-```
-
-Ruff passed for the modified momentum module and new regression file.
-Combined-worktree results do not attest to isolated commit boundaries.

ai/prompt-io/codex/README.md

@@ -1,10 +0,0 @@
-# AI Prompt I/O Log — codex
-
-This directory tracks prompts and outputs for AI-assisted development
-using Codex. Substantive contributions include model identification,
-prompt context, output records, and human contribution accounting.
-
-Code output uses Git diff references; non-code output is retained in
-the paired `.raw.md` file. Human contributors remain responsible for
-review and acceptance. Entries follow the
-[NLNet generative AI policy](https://nlnet.nl/foundation/policies/generativeAI/).

ai/prompt-io/opencode/20260828T013057Z_6ca6ab2a_prompt_io.md

@@ -1,35 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: opencode-pid-469264
-timestamp: 2026-08-28T01:30:57Z
-git_ref: 6ca6ab2a
-scope: code
-substantive: true
-raw_file: 20260828T013057Z_6ca6ab2a_prompt_io.raw.md
----
-
-## Prompt
-
-Investigate Piker's clearing, accounting, order-control, and
-trade-ledger flows in depth while fixing a paper-engine crash where
-RapidFuzz called `len()` on a `MktPair` during ledger persistence.
-
-## Response summary
-
-Changed `SymbologyCache.search()` to fuzzy-match table keys instead of
-struct values, preserving matched aliases and preventing the paper
-trade dialog from crashing when qualifying legacy ledger FQMEs. Added
-an end-to-end regression for the implicated ledger-write path.
-
-## Files changed
-
-- `piker/data/_symcache.py` - searches string keys and resolves values.
-- `tests/test_accounting.py` - covers aliases and paper-ledger writes.
-
-## Human edits
-
-The human supplied the production traceback, identified the affected
-subsystems, and directed an implementation-level fix with deep system
-familiarization. No follow-up correction or direct manual source edit
-was identified during this interaction.

ai/prompt-io/opencode/20260828T013057Z_6ca6ab2a_prompt_io.raw.md

@@ -1,44 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-28T01:30:57Z
-git_ref: 6ca6ab2a
-diff_cmd: git diff HEAD~1..HEAD
----
-
-## Prompt
-
-Become familiar with Piker's clearing, accounting, order-control,
-and trade-ledger tracking subsystems in deep detail while fixing a
-paper-engine crash. The supplied traceback showed
-`TransactionLedger.write_config()` calling
-`SymbologyCache.search()`, followed by RapidFuzz raising
-`TypeError: object of type 'MktPair' has no len()` after a paper fill.
-
-## Response
-
-Fixed the crash at the symbology-cache contract boundary.
-`SymbologyCache.search()` now fuzzy-matches the selected table's
-string keys and resolves each matched key back to its struct value.
-RapidFuzz therefore never receives `MktPair` values as sequences,
-and native market-ID aliases remain intact in search results.
-
-> `git diff HEAD~1..HEAD -- piker/data/_symcache.py`
-
-Added a regression that recreates the failing paper-ledger write with
-an unqualified Binance perpetual FQME. It also checks a native market-ID
-alias, preventing a weaker `processor=str` workaround that would lose
-the actual matched key.
-
-> `git diff HEAD~1..HEAD -- tests/test_accounting.py`
-
-Verification:
-
-`./py313/bin/python -m pytest -p no:xonsh -q -x --tb=short
---no-header tests/test_accounting.py::test_account_file_default_empty
-tests/test_accounting.py::test_paper_ledger_fuzzy_qualifies_mktmap_keys`
-
-Result: `2 passed in 0.02s`.
-
-`git diff --check` also passed. Ruff was not available in the existing
-`py313` environment, so no environment provisioning was performed.

ai/prompt-io/opencode/20260828T021411Z_6ca6ab2a_prompt_io.md

@@ -1,46 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: opencode-pid-469264
-timestamp: 2026-08-28T02:14:11Z
-git_ref: 6ca6ab2a
-scope: config
-substantive: true
-raw_file: 20260828T021411Z_6ca6ab2a_prompt_io.raw.md
----
-
-## Prompt
-
-Move the fix out of the root checkout into an isolated worktree and
-include a Piker-specific clearing, accounting, order-control, and
-trade-ledger expert skill in the worktree's commit plan.
-Use a cross-harness, AI-provider-neutral standard location for its
-canonical files instead of making one provider directory authoritative.
-
-## Response summary
-
-Created a portable `piker-clearing-expert` skill bundle under the
-provider-neutral `.agents/skills/` convention, with a relative Claude
-compatibility link. It covers subsystem ownership, protocols,
-identities, persistence, backend variation, debugging hazards, and
-test strategy. Added the canonical `wkts/` runtime ignore.
-
-## Files changed
-
-- `.gitignore` - ignores the canonical root-level worktree directory.
-- `.agents/skills/piker-clearing-expert/SKILL.md` - core expertise.
-- `.agents/skills/piker-clearing-expert/architecture.md` - deep flows.
-- `.agents/skills/piker-clearing-expert/gotchas.md` - diagnosis.
-- `.agents/skills/piker-clearing-expert/test-map.md` - test map.
-- `.claude/skills/piker-clearing-expert` - relative Claude bridge.
-- `ai/README.md` - defines the provider-neutral source convention.
-- `ai/claude-code/README.md` - advertises the shared skill to Claude.
-- `ai/opencode/README.md` - advertises the shared skill to OpenCode.
-
-## Human edits
-
-The human rejected planning in the dirty root checkout, required a
-dedicated worktree, expanded the work to include a durable expert
-skill, and then directed its canonical source into a cross-provider
-standard location. The agent applied those directed scope and layout
-changes; no direct manual source edit was identified.

ai/prompt-io/opencode/20260828T021411Z_6ca6ab2a_prompt_io.raw.md

@@ -1,62 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-28T02:14:11Z
-git_ref: 6ca6ab2a
-diff_cmd: git diff HEAD~1..HEAD
----
-
-## Prompt
-
-Move the symcache/accounting fix into an isolated worktree and, as
-part of its commit plan, distill a Piker-specific skill for deep
-clearing, accounting, order-control, and trade-ledger expertise.
-A follow-up required the skill's canonical files to begin moving into
-a cross-harness, AI-provider-neutral standard location.
-
-## Response
-
-Created the shared repository skill bundle under the provider-neutral
-`.agents/skills/piker-clearing-expert/` location. Its main skill defines
-activation cues,
-runtime ownership, event and identifier contracts, accounting and
-persistence invariants, market identity rules, an editing workflow,
-and a canonical source map.
-
-> `git diff HEAD~1..HEAD -- .agents/skills/piker-clearing-expert/SKILL.md`
-
-Added a detailed architecture reference for EMS routing, ack/cancel
-ordering, paper and live execution, ledger normalization, position
-events, symbology, and context-managed persistence.
-
-> `git diff HEAD~1..HEAD -- .agents/skills/piker-clearing-expert/architecture.md`
-
-Added symptom-driven gotchas and debugging breadcrumbs for actor
-teardown, ID skew, position reconciliation, duplicate fills,
-paper/live differences, and RapidFuzz mapping misuse.
-
-> `git diff HEAD~1..HEAD -- .agents/skills/piker-clearing-expert/gotchas.md`
-
-Added a deterministic-to-live test map and regression-design guidance.
-
-> `git diff HEAD~1..HEAD -- .agents/skills/piker-clearing-expert/test-map.md`
-
-Updated the worktree runtime ignore block to include the canonical
-root-level `wkts/` directory required by the worktree lifecycle.
-
-> `git diff HEAD~1..HEAD -- .gitignore`
-
-Added a relative Claude compatibility link and documented the
-provider-neutral source convention in the shared and harness-specific
-inventories.
-
-> `git diff HEAD~1..HEAD -- .claude/skills/piker-clearing-expert`
-
-> `git diff HEAD~1..HEAD -- ai/README.md`
-
-> `git diff HEAD~1..HEAD -- ai/claude-code/README.md`
-
-> `git diff HEAD~1..HEAD -- ai/opencode/README.md`
-
-The skill cross-links `piker-conc-expert` for actor-runtime mechanics
-instead of duplicating structured-concurrency guidance.

ai/prompt-io/opencode/20260828T063126Z_1606122b_prompt_io.md

@@ -1,35 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: pid-469264
-timestamp: 2026-08-28T06:31:26Z
-git_ref: 1606122b
-scope: code
-substantive: true
-raw_file: 20260828T063126Z_1606122b_prompt_io.raw.md
----
-
-## Prompt
-
-Review the staged IB broker patch before committing it. After the review
-identified a deterministic error-parser crash, continue with the available
-next steps while leaving the typed EMS worktree and Ruff change untouched.
-
-## Response summary
-
-Repaired bracketed IB API error parsing by splitting on the matched error
-prefix. Added a deterministic regression that routes bracketed and plain
-strings through the real trade-event relay and verifies their EMS error
-messages.
-
-## Files changed
-
-- `piker/brokers/ib/broker.py` - safely extracts bracketed API error codes.
-- `tests/test_ib_broker.py` - covers bracketed and plain error relay inputs.
-
-## Human edits
-
-The human required review before the root commit, accepted the identified
-parser defect as the next work item, and explicitly authorized continuation.
-The pre-existing staged IB refactor is human-authored; the agent applied only
-the parser repair and focused regression described above.

ai/prompt-io/opencode/20260828T063126Z_1606122b_prompt_io.raw.md

@@ -1,30 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-28T06:31:26Z
-git_ref: fix_symcache_search
-diff_cmd: git diff HEAD~1..HEAD
----
-
-## Prompt
-
-The human asked for a review of the root checkout's staged IB patch before
-committing it. The review found that bracketed API error strings call
-`str.partition()` without its required separator. The human then instructed
-the agent to continue with available next steps, or ask only if unsure.
-
-## Response
-
-> `git diff HEAD~1..HEAD -- piker/brokers/ib/broker.py`
-
-The generated parser repair uses the recognized `[code ` or `[Errno ` prefix
-to extract the numeric code without changing the surrounding event relay.
-
-> `git diff HEAD~1..HEAD -- tests/test_ib_broker.py`
-
-The generated regression drives bracketed and plain strings through
-`deliver_trade_events()` and verifies that one `BrokerdError` reaches EMS
-with the expected reason and unknown request ID.
-
-The typed EMS worktree and the unrelated Ruff policy change were left
-untouched. No commit was created.

ai/prompt-io/opencode/20260828T195032Z_9115d24d_prompt_io.md

@@ -1,67 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: opencode-fsp-backfill-sync-20260828
-timestamp: 2026-08-28T19:50:32+00:00
-git_ref: 9115d24d
-scope: code
-substantive: true
-raw_file: 20260828T195032Z_9115d24d_prompt_io.raw.md
----
-
-## Prompt
-
-The human requested a repo-specialist role for realtime financial signal
-processing, a new FSP expert skill, and a fix for repeated FSP desynchronizing
-and whole-chart refreshes during near-term backfill with multiple overlaid
-market feeds. Work was required in a new worktree based on the root checkout.
-
-Follow-up direction required explicit typing throughout the patch, exact
-Qt/pyqtgraph widget types, project-style union spacing, and a final
-per-behaviour multi-commit plan.
-
-## Response summary
-
-The response traced the reported `step_diff=0`, `len_diff=2000` state to
-legitimate source-history prepends. It retained required historical
-recomputation while preventing backfill wakeups from advancing destination
-rows or refreshing unrelated charts. It introduced immutable bootstrap
-snapshots, source/timeframe event filtering, absolute SHM-bound checks,
-targeted graphics invalidation, and deterministic tests.
-
-The typing follow-up established a canonical FSP generator protocol, removed
-an engine-side cast, normalized the `wma` and `rsi` built-ins, validated
-historical output schemas, and made generator cleanup deterministic. A later
-review replaced arbitrary redraw kwargs with an explicit control, typed the
-`Viz`/`PlotItem`/`Axis`/`YAxisLabel` path, added normalized quote and tick
-shapes, and corrected false iterator and NumPy/Numba annotations.
-
-The new provider-neutral `piker-fsp-expert` skill captures architecture,
-mathematical methods, optimization guidance, roadmap context, dependency
-research, and a test matrix.
-
-## Files changed
-
-- `piker/fsp/_api.py` - typed FSP decorator and generator contract.
-- `piker/fsp/_engine.py` - backfill synchronization, snapshot, publication,
-  event filtering, and cleanup behavior.
-- `piker/fsp/_momo.py` - protocol-conforming WMA and RSI streams.
-- `piker/fsp/_volume.py` - typed volume FSP stream declarations.
-- `piker/data/ticktools.py` - concrete quote/tick and iterator contracts.
-- `piker/data/_sharedmem.py` - serializable SHM token message shape.
-- `piker/ui/_dataviz.py` - piker `PlotItem` ownership on `Viz`.
-- `piker/ui/_pg_overrides.py` - piker view and axis return types.
-- `piker/ui/_fsp.py` - destination-token-targeted redraw and y-range refresh.
-- `tests/test_fsp_sync.py` - deterministic engine, UI, and operator tests.
-- `.agents/skills/piker-fsp-expert/` - reusable FSP specialist knowledge.
-- `.claude/skills/piker-fsp-expert` - Claude-compatible skill link.
-
-## Human edits
-
-The human identified insufficient typing in the initial patch and required
-the implementation to expose and repair the actual FSP protocol instead of
-using broad or misleading casts. In later review, the human specifically
-rejected `**kwargs: Any`, an optional `Viz.shm`, a broad sticky-label type,
-and spaced union syntax, then directed the agent to infer reusable types from
-the actual graphics widgets. The human also required a per-behaviour commit
-plan for final review. No direct manual source edits were observed.

ai/prompt-io/opencode/20260828T195032Z_9115d24d_prompt_io.raw.md

@@ -1,54 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-28T19:50:32+00:00
-git_ref: 9115d24d
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The response implemented and analyzed the requested FSP backfill fix.
-
-> `git diff HEAD~1..HEAD -- piker/fsp/_api.py`
->
-> `git diff HEAD~1..HEAD -- piker/fsp/_engine.py`
->
-> `git diff HEAD~1..HEAD -- piker/fsp/_momo.py`
->
-> `git diff HEAD~1..HEAD -- piker/fsp/_volume.py`
->
-> `git diff HEAD~1..HEAD -- piker/data/ticktools.py`
->
-> `git diff HEAD~1..HEAD -- piker/data/_sharedmem.py`
->
-> `git diff HEAD~1..HEAD -- piker/ui/_dataviz.py`
->
-> `git diff HEAD~1..HEAD -- piker/ui/_pg_overrides.py`
->
-> `git diff HEAD~1..HEAD -- piker/ui/_fsp.py`
->
-> `git diff HEAD~1..HEAD -- tests/test_fsp_sync.py`
->
-> `git diff HEAD~1..HEAD -- .agents/skills/piker-fsp-expert/`
-
-The generated implementation distinguishes history revisions from sample
-advancement, filters foreign backfill events, computes historical output from
-an immutable source snapshot, and targets FSP-backed visualizations for cache
-invalidation, redraw, and y-range updates. It also establishes a typed FSP
-stream protocol, normalizes the dormant scalar built-ins to that protocol,
-and adds deterministic regression coverage. The typing review replaces
-arbitrary redraw kwargs with an explicit boolean control, records the
-concrete piker graphics object chain, and introduces reusable normalized
-quote, tick, and SHM-token message shapes.
-
-The generated skill bundle records the FSP runtime architecture, financial
-signal-processing principles, testing matrix, roadmap issues, and candidate
-numerics and IPC research projects.
-
-Verification reported by the response:
-
-- focused FSP and deterministic history tests passed;
-- Ruff passed for all changed Python files;
-- focused `ty` checks passed for tick and scalar FSP contracts;
-- compileall passed for all changed Python files;
-- `git diff --check` passed;
-- repeated read-only code review found no remaining actionable findings.

ai/prompt-io/opencode/20260903T232525Z_fadab3d2_prompt_io.md

@@ -1,41 +0,0 @@
----
-model: gpt-5.6-sol
-service: opencode
-session: fsp-backfill-sync-live-volume
-timestamp: 2026-09-03T23:25:25Z
-git_ref: fadab3d2
-scope: code
-substantive: true
-raw_file: 20260903T232525Z_fadab3d2_prompt_io.raw.md
----
-
-## Prompt
-
-Diagnose and fix MNQ's same-index volume repaint, missing post-closure
-volume curve, stale zoomed-out downsample path, and repeated
-`No yrange provided for dolla_vlm` warnings using live SHM evidence.
-
-## Response summary
-
-Declared source sample periods on `Flume` so FSP cascades cannot mistake
-a market-closure timestamp gap for their regular cadence. Clamped
-negative provider volume at sampling and volume-FSP boundaries, honored
-M4 renderer cache invalidation, and skipped unchanged realtime FSP
-redraws while retaining forced history repairs.
-
-## Files changed
-
-- `piker/data/flows.py` - declare RT/history sample periods.
-- `piker/data/_sampling.py` - prevent negative sampled volume.
-- `piker/fsp/_engine.py` - subscribe using the declared RT period.
-- `piker/fsp/_volume.py` - clamp volume history and trade sizes.
-- `piker/ui/_dataviz.py` - honor renderer cache resets.
-- `piker/ui/_fsp.py` - skip identical realtime FSP redraws.
-- `tests/test_fsp_sync.py` - cover the observed regressions.
-
-## Human edits
-
-The human identified each visual failure, supplied the exact console
-warning, confirmed that the later-started `flow_rates` cascade behaved
-correctly, requested declared cadence instead of timestamp inference,
-performed chart restarts, and live-validated the final behavior.

ai/prompt-io/opencode/20260903T232525Z_fadab3d2_prompt_io.raw.md

@@ -1,71 +0,0 @@
----
-model: gpt-5.6-sol
-service: opencode
-timestamp: 2026-09-03T23:25:25Z
-git_ref: fadab3d2
-diff_cmd: git diff HEAD~4..HEAD
----
-
-## Prompt
-
-The human reported that the MNQ volume subchart repeatedly rendered
-its last bar at one index around the equity-futures closure, then
-reported that the volume curve disappeared after reopening. During
-live testing, the human confirmed that `flow_rates`, which started
-later, advanced correctly while `dolla_vlm` remained pinned. The human
-then reported a stale downsampled curve on zoom-out and supplied the
-repeated console warning:
-
-`No yrange provided for dolla_vlm!?`
-
-The human requested a live diagnosis and fix, validated each restart,
-and confirmed the final chart behavior was fixed.
-
-## Response
-
-Read-only SHM snapshots showed that `dolla_vlm` remained at exclusive
-bound `172801` while the source advanced hundreds of rows. Its one
-current row kept receiving current quote values, explaining the
-same-index repaint. `flow_rates` remained aligned because it started
-after regular one-second timestamps resumed.
-
-The cascade had inferred its sampler period from the last two distinct
-timestamps. A market closure therefore looked like the source cadence.
-Explicit `Flume` RT/history periods now drive sampling and downsampling
-metadata instead.
-
-> `git diff HEAD~4..HEAD -- piker/data/flows.py`
-
-> `git diff HEAD~4..HEAD -- piker/fsp/_engine.py`
-
-Further SHM inspection found an IB source row with volume `-2319096`.
-`dolla_vlm` multiplied that value into a `-68277471626` outlier, causing
-`Viz.maxmin()` to reject the zoomed-out range. Sample ingestion and
-volume FSP history/realtime calculations now clamp negative volume.
-
-> `git diff HEAD~4..HEAD -- piker/data/_sampling.py`
-
-> `git diff HEAD~4..HEAD -- piker/fsp/_volume.py`
-
-The M4 renderer already returned a cache-reset signal after rebuilding
-a downsampled path, but `Viz.update_graphics()` overwrote that signal
-with `False`. The override was removed so Qt's device-coordinate cache
-is invalidated with the new path.
-
-> `git diff HEAD~4..HEAD -- piker/ui/_dataviz.py`
-
-Normal quote cycles also redrew unchanged FSP state. A per-`Viz`
-bounds/timestamp/value signature now suppresses only identical
-realtime refreshes; forced history redraws still bypass it.
-
-> `git diff HEAD~4..HEAD -- piker/ui/_fsp.py`
-
-Regression coverage models closure-gap cadence, negative provider
-volume, unchanged realtime redraws, and the broader FSP synchronization
-behavior.
-
-> `git diff HEAD~4..HEAD -- tests/test_fsp_sync.py`
-
-Live verification after restart showed source and `dolla_vlm` advancing
-at the same bound, zero negative destination rows, and correct zoomed-out
-rendering. The focused suite completed with 38 passing tests.

ai/prompt-io/opencode/20260908T020619Z_fadab3d2_prompt_io.md

@@ -1,40 +0,0 @@
----
-model: gpt-5.6-sol
-service: opencode
-session: tuicr-fsp-api-review
-timestamp: 2026-09-08T02:06:19Z
-git_ref: fadab3d2
-scope: code
-substantive: true
-raw_file: 20260908T020619Z_fadab3d2_prompt_io.raw.md
----
-
-## Prompt
-
-Address all 14 comments from the local staged FSP Tuicr review, account
-for relevant unstaged behavior changes, verify the fixes, and add local
-responses without staging or committing repository changes.
-
-## Response summary
-
-Clarified typed wire-schema and generic-decorator intent, adopted more
-specific FSP and feed type names, made momentum array typing explicit,
-documented the WAP tuple fields, and verified both FSP regressions and
-the production Numba call shape.
-
-## Files changed
-
-- `piker/data/__init__.py` - export the renamed feed quote type.
-- `piker/data/ticktools.py` - clarify wire dicts and future structs.
-- `piker/fsp/_api.py` - clarify history and decorator types.
-- `piker/fsp/_engine.py` - consume the renamed public types.
-- `piker/fsp/_momo.py` - make array and stream types explicit.
-- `piker/fsp/_volume.py` - document WAP return fields.
-- `tests/test_fsp_sync.py` - consume the renamed feed quote type.
-
-## Human edits
-
-The human reviewed the staged API boundary in Tuicr, requested the type
-renames and documentation, rejected an opaque NumPy alias, identified
-missing local annotations, required per-field tuple comments, and asked
-that existing unstaged fixes be considered when responding.

ai/prompt-io/opencode/20260908T020619Z_fadab3d2_prompt_io.raw.md

@@ -1,47 +0,0 @@
----
-model: gpt-5.6-sol
-service: opencode
-timestamp: 2026-09-08T02:06:19Z
-git_ref: fadab3d2
-diff_cmd: git diff HEAD~1..HEAD
----
-
-## Prompt
-
-The human supplied 14 comments from the local Tuicr review session
-`fsp_backfill_sync@wkt-fsp_backfill_sync/staged/fadab3d` and invoked
-`/code-review-changes -r`. They asked that comments already addressed
-by unstaged behavioral work be recognized when choosing each response.
-
-## Response
-
-The review was correlated one-to-one with its persisted Tuicr records.
-The resulting adjustments clarify the existing dict wire schema and a
-future paired `msgspec.Struct` migration, and rename the broad `Quote`
-and `HistoryOutput` types to `FeedQuote` and `FspHistory`.
-
-> `git diff HEAD~1..HEAD -- piker/data/__init__.py`
-
-> `git diff HEAD~1..HEAD -- piker/data/ticktools.py`
-
-> `git diff HEAD~1..HEAD -- piker/fsp/_api.py`
-
-> `git diff HEAD~1..HEAD -- piker/fsp/_engine.py`
-
-The FSP decorator now documents its PEP 695 parameter specification and
-its bare and configured return forms. Momentum types use explicit
-`NDArray[np.float64]` annotations, and WMA locals and the ignored tick
-target are named and typed explicitly. The WAP tuple return is split
-across documented fields.
-
-> `git diff HEAD~1..HEAD -- piker/fsp/_momo.py`
-
-> `git diff HEAD~1..HEAD -- piker/fsp/_volume.py`
-
-> `git diff HEAD~1..HEAD -- tests/test_fsp_sync.py`
-
-The focused FSP regression file passed with 29 tests. Ruff, Ruff's E501
-line-length selection, compileall, and diff whitespace checks passed.
-The production Numba call shape was also executed successfully with all
-three arguments; its pre-existing explicit signature does not accept
-omitted defaults.

piker/brokers/ib/broker.py

@@ -52,6 +52,7 @@ from ib_async.objects import (
 from piker import config
 from piker.log import (
     get_logger,
+    get_console_log,
 )
 from piker.types import Struct
 from piker.accounting import (
@@ -1428,35 +1429,27 @@ async def deliver_trade_events(
                 # NOTE: see impl deats in
                 # `Client.inline_errors()::push_err()`
                 err: dict|str = item
-                is_msg: bool = False
 
                 # std case, never relay errors for non-order-control
                 # related issues.
                 # https://interactivebrokers.github.io/tws-api/message_codes.html
-                match err:
-                    case dict():
-                    # if isinstance(err, dict):
+                if isinstance(err, dict):
                     code: int = err['error_code']
                     reason: str = err['reason']
                     reqid: str = str(err['reqid'])
-                        is_msg = True
 
-                    case str():
                 # XXX, sometimes you'll get just a `str` of the form,
                 # '[code 104] connection failed' or something..
-                    # elif isinstance(err, str):
-                        code: int|str = '<NA-err-code>'
+                elif isinstance(err, str):
                     code_part, _, reason = err.rpartition(']')
                     if code_part:
                         for prefix_patt in [
                             '[Errno ',
                             '[code ',
                         ]:
-                                _, prefix, code_str = (
-                                    code_part.partition(prefix_patt)
-                                )
-                                if prefix:
-                                    code = int(code_str)
+                            code_part, _, code = code_part.partition()
+                            if code:
+                                code = int(code)
                                 break
 
                     reqid: str = '<unknown>'
@@ -1504,17 +1497,16 @@ async def deliver_trade_events(
                     )
                     continue
 
-                if (
-                    is_msg
-                    and
-                    err['reqid'] == -1
-                ):
+                if err['reqid'] == -1:
                     log.error(
                         f'TWS external order error ??\n'
                         f'{pformat(err)}\n'
                     )
 
-                flow: dict = dict(flows.get(reqid) or {})
+                flow: dict = dict(
+                    flows.get(reqid)
+                    or {}
+                )
 
                 # TODO: we don't want to relay data feed / lookup errors
                 # so we need some further filtering logic here..

piker/data/__init__.py

@@ -22,11 +22,7 @@ and storing data from your brokers as well as
 sharing live streams over a network.
 
 """
-from .ticktools import (
-    FeedQuote,
-    iterticks,
-    Tick,
-)
+from .ticktools import iterticks
 from tractor.ipc._shm import (
     ShmArray,
     get_shm_token,
@@ -58,9 +54,7 @@ __all__: list[str] = [
     'Feed',
     'open_feed',
     'ShmArray',
-    'FeedQuote',
     'iterticks',
-    'Tick',
     'maybe_open_shm_array',
     'match_from_pairs',
     'attach_shm_array',

piker/data/_sampling.py

@@ -670,10 +670,7 @@ async def sample_and_broadcast(
                             ['open', 'high', 'low', 'volume']
                         ]
 
-                        new_v: float = max(
-                            float(tick.get('size', 0)),
-                            0.,
-                        )
+                        new_v: float = tick.get('size', 0)
 
                         if v == 0 and new_v:
                             # no trades for this bar yet so the open
@@ -681,14 +678,11 @@ async def sample_and_broadcast(
                             o = last
 
                         if sum_tick_vlm:
-                            volume = max(float(v), 0.) + new_v
+                            volume = v + new_v
                         else:
                             # presume backend takes care of summing
                             # it's own vlm
-                            volume = max(
-                                float(quote['volume']),
-                                0.,
-                            )
+                            volume = quote['volume']
 
                         shm.array[[
                             'open',

piker/data/_sharedmem.py

@@ -26,8 +26,6 @@ are now imported directly from tractor throughout
 the codebase.
 
 '''
-from typing import TypedDict
-
 import numpy as np
 
 from tractor.ipc._shm import (
@@ -41,20 +39,6 @@ from tractor.ipc._shm import (
 
 from ._util import log
 
-type DTypeField = (
-    tuple[str, str]
-    |tuple[str, str, tuple[int, ...]]
-)
-
-
-class NDTokenMsg(TypedDict):
-    shm_name: str
-    shm_first_index_name: str
-    shm_last_index_name: str
-    dtype_descr: tuple[DTypeField, ...]
-    size: int
-    key: str|None
-
 
 def _make_token(
     key: str,
@@ -178,7 +162,7 @@ def maybe_open_shm_array(
 
 def try_read(
     array: np.ndarray,
-) -> np.void|None:
+) -> np.ndarray|None:
     '''
     Try to read the last row from a shared mem
     array or ``None`` if the array read returns

piker/data/_symcache.py

@@ -334,16 +334,15 @@ class SymbologyCache(Struct):
         matches in a `dict` including the `MktPair` values.
 
         '''
-        entries: dict[str, Struct] = getattr(self, table)
         matches = fuzzy.extract(
             pattern,
-            list(entries),
+            getattr(self, table),
             score_cutoff=50,
         )
 
         # repack in dict[fqme, MktPair] form
         return {
-            item[0]: entries[item[0]]
+            item[0].fqme: item[0]
             for item in matches
         }
 

piker/data/flows.py

@@ -24,6 +24,7 @@ real-time data processing data-structures.
 from __future__ import annotations
 
 import tractor
+import pendulum
 import numpy as np
 
 from piker.types import Struct
@@ -72,8 +73,6 @@ class Flume(Struct):
     izero_hist: int = 0
     izero_rt: int = 0
     throttle_rate: int | None = None
-    rt_sample_period_s: float = 1.
-    hist_sample_period_s: float = 60.
 
     @property
     def rt_shm(self) -> ShmArray:
@@ -116,9 +115,17 @@ class Flume(Struct):
         period and ratio between them.
 
         '''
-        rt_step_size_s: float = self.rt_sample_period_s
-        hist_step_size_s: float = self.hist_sample_period_s
-        ratio: float = hist_step_size_s / rt_step_size_s
+        times: np.ndarray = self.hist_shm.array['time']
+        end: float | int = pendulum.from_timestamp(times[-1])
+        start: float | int = pendulum.from_timestamp(times[times != times[-1]][-1])
+        hist_step_size_s: float = (end - start).seconds
+
+        times = self.rt_shm.array['time']
+        end = pendulum.from_timestamp(times[-1])
+        start = pendulum.from_timestamp(times[times != times[-1]][-1])
+        rt_step_size_s = (end - start).seconds
+
+        ratio = hist_step_size_s / rt_step_size_s
         return (
             rt_step_size_s,
             hist_step_size_s,

piker/data/ticktools.py

@@ -20,31 +20,10 @@ Tick event stream processing, filter-by-types, format-normalization.
 '''
 from itertools import chain
 from typing import (
-    Iterable,
-    Iterator,
-    Sequence,
-    TypedDict,
+    Any,
+    AsyncIterator,
 )
 
-
-# TODO: migrate feed emitters and consumers to ``msgspec.Struct``
-# together. These ``TypedDict`` types only describe the builtin dicts
-# currently transported over IPC; changing only the receiver-side
-# annotations would not change serialization.
-class Tick(TypedDict, total=False):
-    type: str
-    price: float
-    size: float
-    time: float
-
-
-class FeedQuote(TypedDict, total=False):
-    ticks: list[Tick]
-    tradeRate: float
-    volumeRate: float
-    broker_ts: float
-    brokerd_ts: float
-
 # tick-type-classes template for all possible "lowest level" events
 # that can can be emitted by the "top of book" L1 queues and
 # price-matching (with eventual clearing) in a double auction
@@ -62,12 +41,15 @@ _auction_ticks: set[str] = set.union(*_tick_groups.values())
 
 
 def frame_ticks(
-    quote: FeedQuote,
+    quote: dict[str, Any],
 
-    ticks_by_type: dict[str, list[Tick]]|None = None,
-    ticks_in_order: list[Tick]|None = None,
+    ticks_by_type: dict | None = None,
+    ticks_in_order: list[dict[str, Any]] | None = None
 
-) -> dict[str, list[Tick]]:
+) -> dict[
+    str,
+    list[dict[str, Any]]
+]:
     '''
     XXX: build a tick-by-type table of lists
     of tick messages. This allows for less
@@ -100,7 +82,7 @@ def frame_ticks(
 
     # append in reverse FIFO order for in-order iteration on
     # receiver side.
-    tick: Tick
+    tick: dict[str, Any]
     for tick in ticks:
         tbt.setdefault(
             tick['type'],
@@ -122,8 +104,8 @@ def frame_ticks(
 
 
 def iterticks(
-    quote: FeedQuote,
-    types: Sequence[str] = (
+    quote: dict,
+    types: tuple[str] = (
         'trade',
         'dark_trade',
     ),
@@ -134,7 +116,7 @@ def iterticks(
     # with this?
     frame_by_type: bool = False,
 
-) -> Iterator[Tick]:
+) -> AsyncIterator:
     '''
     Iterate through ticks delivered per quote cycle, filter and
     yield any declared in `types`.
@@ -181,13 +163,10 @@ def iterticks(
             ticks.extend(list(chain(trades.values(), darks.values())))
 
         # most-recent-first
-        ticks_iter: Iterable[Tick] = (
-            reversed(ticks)
-            if reverse
-            else ticks
-        )
+        if reverse:
+            ticks = reversed(ticks)
 
-        for tick in ticks_iter:
+        for tick in ticks:
             # print(f"{quote['symbol']}: {tick}")
             ttype = tick.get('type')
             if ttype in types:

piker/fsp/_api.py

@@ -25,11 +25,12 @@ FSP (financial signal processing) apis.
 # - composition of fsps / implicit chaining syntax (we need an issue)
 
 from __future__ import annotations
+from functools import partial
 from typing import (
-    AsyncGenerator,
+    Any,
     Callable,
-    overload,
-    Protocol,
+    Awaitable,
+    Optional,
 )
 
 import numpy as np
@@ -46,32 +47,11 @@ from ..log import get_logger
 
 log = get_logger(__name__)
 
-type FspHistory = dict[str, np.ndarray|None]|np.ndarray
-type RealtimeValue = np.ndarray|np.number|int|float
-type FspYield = FspHistory|tuple[str, RealtimeValue]
-type FspStream = AsyncGenerator[FspYield, None]
-type FspConfigValue = str|bool|int|float
-
-
-# ``**P`` declares a PEP 695 ``ParamSpec`` so wrappers retain each
-# decorated FSP's positional and keyword parameter types:
-# https://docs.python.org/3/reference/compound_stmts.html#type-params
-class FspFunc[**P](Protocol):
-    @property
-    def __name__(self) -> str: ...
-
-    def __call__(
-        self,
-        *args: P.args,
-        **kwargs: P.kwargs,
-
-    ) -> FspStream: ...
-
 # global fsp registry filled out by @fsp decorator below
-_fsp_registry: dict[NamespacePath, Fsp] = {}
+_fsp_registry = {}
 
 
-def _load_builtins() -> dict[NamespacePath, Fsp]:
+def _load_builtins() -> dict[tuple, Callable]:
 
     # import to implicity trigger registration via ``@fsp``
     from . import _momo  # noqa
@@ -80,7 +60,7 @@ def _load_builtins() -> dict[NamespacePath, Fsp]:
     return _fsp_registry
 
 
-class Fsp[**P]:
+class Fsp:
     '''
     "Financial signal processor" decorator wrapped async function.
 
@@ -99,28 +79,28 @@ class Fsp[**P]:
     # shm flow.
     _flow_registry: dict[
         tuple[NDToken, str],
-        tuple[NDToken, ShmArray|None],
+        tuple[NDToken, Optional[ShmArray]],
     ] = {}
 
     def __init__(
         self,
-        func: FspFunc[P],
+        func: Callable[..., Awaitable],
         *,
-        outputs: tuple[str, ...] = (),
-        display_name: str|None = None,
-        **config: FspConfigValue,
+        outputs: tuple[str] = (),
+        display_name: Optional[str] = None,
+        **config,
 
     ) -> None:
 
         # TODO (maybe):
         # - type introspection?
         # - should we make this a wrapt object proxy?
-        self.func: FspFunc[P] = func
+        self.func = func
         self.__name__ = func.__name__  # XXX: must have func-object name
 
-        self.ns_path: NamespacePath = NamespacePath.from_ref(func)
+        self.ns_path: tuple[str, str] = NamespacePath.from_ref(func)
         self.outputs = outputs
-        self.config: dict[str, FspConfigValue] = config
+        self.config: dict[str, Any] = config
 
         # register with declared set.
         _fsp_registry[self.ns_path] = self
@@ -136,10 +116,9 @@ class Fsp[**P]:
         # type annots from pep 612:
         # https://www.python.org/dev/peps/pep-0612/
         # instance,
-        *args: P.args,
-        **kwargs: P.kwargs,
-
-    ) -> FspStream:
+        *args,
+        **kwargs
+    ):
         return self.func(*args, **kwargs)
 
     def get_shm(
@@ -175,58 +154,23 @@ class Fsp[**P]:
         return maybe_array
 
 
-@overload
-def fsp[**P](
-    wrapped: FspFunc[P],
+def fsp(
+    wrapped=None,
     *,
-    outputs: tuple[str, ...] = (),
-    display_name: str|None = None,
-    **config: FspConfigValue,
+    outputs: tuple[str] = (),
+    display_name: Optional[str] = None,
+    **config,
 
-) -> Fsp[P]:
-    ...
-
-
-@overload
-def fsp[**P](
-    wrapped: None = None,
-    *,
-    outputs: tuple[str, ...] = (),
-    display_name: str|None = None,
-    **config: FspConfigValue,
-
-) -> Callable[[FspFunc[P]], Fsp[P]]:
-    ...
-
-
-def fsp[**P](
-    wrapped: FspFunc[P]|None = None,
-    *,
-    outputs: tuple[str, ...] = (),
-    display_name: str|None = None,
-    **config: FspConfigValue,
-
-) -> Fsp[P]|Callable[[FspFunc[P]], Fsp[P]]:
-    '''
-    Wrap an FSP function directly or return its configured decorator.
-
-    Bare ``@fsp`` calls this function with ``wrapped`` and returns an
-    ``Fsp``. Parameterized ``@fsp(...)`` returns ``decorate`` first;
-    Python then passes the decorated function to that closure.
-
-    '''
+) -> Fsp:
 
     if wrapped is None:
-        def decorate(func: FspFunc[P]) -> Fsp[P]:
-            return Fsp(
-                func,
+        return partial(
+            Fsp,
             outputs=outputs,
             display_name=display_name,
             **config,
         )
 
-        return decorate
-
     return Fsp(wrapped, outputs=(wrapped.__name__,))
 
 
@@ -236,7 +180,7 @@ def maybe_mk_fsp_shm(
     size: int,
     readonly: bool = True,
 
-) -> tuple[str, ShmArray, bool]:
+) -> (str, ShmArray, bool):
     '''
     Allocate a single row shm array for an symbol-fsp pair if none
     exists, otherwise load the shm already existing for that token.

piker/fsp/_engine.py

@@ -22,10 +22,7 @@ from __future__ import annotations
 from contextlib import asynccontextmanager as acm
 from functools import partial
 from typing import (
-    Any,
     AsyncIterator,
-    Awaitable,
-    cast,
     Callable,
     TYPE_CHECKING,
 )
@@ -42,20 +39,15 @@ from ..log import (
     get_console_log,
 )
 from .. import data
-from ..data.ticktools import FeedQuote
 from ..data.flows import Flume
-from ..data._sharedmem import NDTokenMsg
 from tractor.ipc._shm import ShmArray
 from ..data._sampling import (
+    _default_delay_s,
     open_sample_stream,
 )
 from ..accounting import MktPair
 from ._api import (
     Fsp,
-    FspFunc,
-    FspStream,
-    FspYield,
-    FspHistory,
     _load_builtins,
     NDToken,
 )
@@ -66,70 +58,13 @@ if TYPE_CHECKING:
 
 log = get_logger(__name__)
 
-type QuoteFrame = dict[str, FeedQuote]
-type BackfillNotice = tuple[str, float]|list[str|float]
-type SampleValue = int|float|BackfillNotice
-type SampleMsg = dict[str, SampleValue]
-type EdgeFunc = FspFunc
-
-
-def _is_relevant_sample_msg(
-    msg: SampleMsg,
-    fqme: str,
-    period_s: float,
-
-) -> bool:
-    backfill: SampleValue|None = msg.get('backfilling')
-    if backfill is None:
-        return True
-
-    if (
-        not isinstance(backfill, (list, tuple))
-        or
-        len(backfill) != 2
-    ):
-        return False
-
-    backfill_fqme: str | float
-    timeframe: str | float
-    backfill_fqme, timeframe = backfill
-    return (
-        backfill_fqme == fqme
-        and
-        timeframe == period_s
-    )
-
-
-def _should_advance_dst(
-    msg: SampleMsg,
-    step_diff: int,
-
-) -> bool:
-    return (
-        'backfilling' not in msg
-        and
-        step_diff == 1
-    )
-
-
-def _needs_history_resync(
-    msg: SampleMsg,
-    synced: bool,
-
-) -> bool:
-    return (
-        'backfilling' in msg
-        or
-        not synced
-    )
-
 
 async def filter_quotes_by_sym(
 
     sym: str,
-    quote_stream: AsyncIterator[QuoteFrame],
+    quote_stream: tractor.MsgStream,
 
-) -> AsyncIterator[FeedQuote]:
+) -> AsyncIterator[dict]:
     '''
     Filter quote stream by target symbol.
 
@@ -140,91 +75,10 @@ async def filter_quotes_by_sym(
     yield {}
 
     async for quotes in quote_stream:
-        quote: FeedQuote|None = quotes.get(sym)
+        quote = quotes.get(sym)
         if quote:
             yield quote
 
-
-class _HistoryShm:
-    '''
-    Present an immutable history view before switching to live SHM.
-
-    '''
-    def __init__(self, shm: ShmArray) -> None:
-        self._shm: ShmArray = shm
-        self.first: int = shm._first.value
-        self.last_index: int = shm._last.value
-        self.history: np.ndarray = shm._array[
-            self.first:self.last_index
-        ].copy()
-        self.live: bool = False
-
-    @property
-    def array(self) -> np.ndarray:
-        if self.live:
-            return self._shm.array
-        return self.history
-
-    @property
-    def index(self) -> int:
-        if self.live:
-            return self._shm.index
-        return self.last_index % self._shm._len
-
-    def last(self, length: int = 1) -> np.ndarray:
-        return self.array[-length:]
-
-    def __getattr__(self, name: str) -> Any:
-        return getattr(self._shm, name)
-
-
-async def _open_history_snapshot(
-    edge_func: EdgeFunc,
-    quote_stream: AsyncIterator[QuoteFrame],
-    fqme: str,
-    src_shm: ShmArray,
-
-) -> tuple[
-    FspStream,
-    FspHistory,
-    tuple[int, int],
-    np.ndarray,
-]:
-    '''
-    Start an FSP against an immutable source-history snapshot.
-
-    '''
-    history_shm: _HistoryShm = _HistoryShm(src_shm)
-    out_stream: FspStream = edge_func(
-        filter_quotes_by_sym(fqme, quote_stream),
-        cast(ShmArray, history_shm),
-    )
-    try:
-        first_yield: FspYield = await anext(out_stream)
-    except BaseException:
-        with trio.CancelScope(shield=True):
-            await out_stream.aclose()
-        raise
-
-    if isinstance(first_yield, tuple):
-        await out_stream.aclose()
-        raise TypeError(
-            f'FSP `{edge_func.__name__}` yielded a realtime update '
-            f'before its historical output'
-        )
-
-    history_output: FspHistory = first_yield
-    history_shm.live = True
-    return (
-        out_stream,
-        history_output,
-        (
-            history_shm.first,
-            history_shm.last_index,
-        ),
-        history_shm.history,
-    )
-
 # TODO: unifying the abstractions in this FSP subsys/layer:
 # -[ ] move the `.data.flows.Flume` type into this
 #   module/subsys/pkg?
@@ -270,7 +124,7 @@ class Cascade(Struct):
     fsp: Fsp  # UI-side middleware ctl API
 
     # filled during cascade/.bind_func() (fsp_compute) init phases
-    bind_func: Callable[..., Awaitable[None]]|None = None
+    bind_func: Callable | None = None
     complete: trio.Event | None = None
     cs: trio.CancelScope | None = None
     client_stream: tractor.MsgStream | None = None
@@ -279,24 +133,15 @@ class Cascade(Struct):
         # TODO: adopt an incremental update engine/approach
         # where possible here eventually!
         log.info(f're-syncing fsp {self.fsp.name} to source')
-        cs: trio.CancelScope|None = self.cs
-        complete: trio.Event|None = self.complete
-        bind_func = self.bind_func
-        client_stream = self.client_stream
-        assert cs is not None
-        assert complete is not None
-        assert bind_func is not None
-        assert client_stream is not None
-
-        cs.cancel()
-        await complete.wait()
-        index: int = await self.tn.start(bind_func)
+        self.cs.cancel()
+        await self.complete.wait()
+        index: int = await self.tn.start(self.bind_func)
 
         # always trigger UI refresh after history update,
         # see ``piker.ui._fsp.FspAdmin.open_chain()`` and
         # ``piker.ui._display.trigger_update()``.
         dst_shm: ShmArray = self.dst.rt_shm
-        await client_stream.send({
+        await self.client_stream.send({
             'fsp_update': {
                 'key': dst_shm.token,
                 'first': dst_shm._first.value,
@@ -313,48 +158,27 @@ class Cascade(Struct):
         '''
         src_shm: ShmArray = self.src.rt_shm
         dst_shm: ShmArray = self.dst.rt_shm
-        src_first = src_shm._first.value
-        src_last = src_shm._last.value
-        dst_first = dst_shm._first.value
-        dst_last = dst_shm._last.value
+        step_diff = src_shm.index - dst_shm.index
+        len_diff = abs(len(src_shm.array) - len(dst_shm.array))
+        synced: bool = not (
+            # the source is likely backfilling and we must
+            # sync history calculations
+            len_diff > 2
 
-        # Compare absolute bounds instead of ``ShmArray.index``, which
-        # wraps at the allocation size. A one-step source lead is the
-        # normal state before the destination sample is appended.
-        step_diff = src_last - dst_last
-        len_diff = abs(
-            (src_last - src_first)
-            - (dst_last - dst_first)
-        )
-        synced: bool = (
-            src_first == dst_first
-            and
-            step_diff in (0, 1)
+            # we aren't step synced to the source and may be
+            # leading/lagging by a step
+            or step_diff > 1
+            or step_diff < 0
         )
         if not synced:
             fsp: Fsp = self.fsp
-            report = (
-                f'ns-path: {fsp.ns_path!r}\n'
-                f'shm-token: {src_shm.token}\n'
-                f'src-bounds: {(src_first, src_last)}\n'
-                f'dst-bounds: {(dst_first, dst_last)}\n'
-                f'step_diff: {step_diff}\n'
-                f'len_diff: {len_diff}\n'
-            )
-            if (
-                src_first < dst_first
-                and
-                src_last == dst_last
-            ):
-                log.info(
-                    f'FSP source history grew:\n'
-                    f'{report}'
-                )
-            else:
             log.warning(
                 f'***DESYNCED fsp***\n'
                 f'------------------\n'
-                    f'{report}'
+                f'ns-path: {fsp.ns_path!r}\n'
+                f'shm-token: {src_shm.token}\n'
+                f'step_diff: {step_diff}\n'
+                f'len_diff: {len_diff}\n'
             )
         return (
             synced,
@@ -363,20 +187,17 @@ class Cascade(Struct):
         )
 
     async def poll_and_sync_to_step(self) -> int:
-        while True:
+        synced, step_diff, _ = self.is_synced()
+        while not synced:
             await self.resync()
-            (
-                synced,
-                step_diff,
-                _,
-            ) = self.is_synced()
-            if synced:
+            synced, step_diff, _ = self.is_synced()
+
         return step_diff
 
     @acm
     async def open_edge(
         self,
-        bind_func: Callable[..., Awaitable[None]],
+        bind_func: Callable,
     ) -> int:
         self.bind_func = bind_func
         index = await self.tn.start(bind_func)
@@ -385,123 +206,14 @@ class Cascade(Struct):
         # -[ ] dynamic reconnection after update?
 
 
-async def _close_fsp_stream(out_stream: FspStream) -> None:
-    with trio.CancelScope(shield=True):
-        await out_stream.aclose()
-
-
-def _write_history_output(
-    dst_shm: ShmArray,
-    history_output: FspHistory,
-    src_bounds: tuple[int, int],
-    src_history: np.ndarray,
-    func_name: str,
-    profiler: Profiler,
-
-) -> int:
-    src_first, src_last = src_bounds
-    fields = dst_shm.array.dtype.fields
-    if fields is None:
-        raise TypeError('FSP destination SHM must be a structured array')
-    fields = fields.copy()
-    fields.pop('index')
-    fields.pop('time')
-    history_by_field: np.ndarray|None = None
-
-    if len(fields) > 1:
-        if not isinstance(history_output, dict):
-            raise ValueError(
-                f'`{func_name}` is a multi-output FSP and should '
-                f'yield a `dict[str, np.ndarray]` for history'
-            )
-
-        output_fields: list[str] = [
-            key
-            for key in fields
-            if key in history_output
-        ]
-        if not output_fields:
-            raise ValueError(
-                f'`{func_name}` produced no declared history fields'
-            )
-
-        for key in output_fields:
-            output: np.ndarray|None = history_output[key]
-            if history_by_field is None:
-                length: int = (
-                    len(src_history)
-                    if output is None
-                    else len(output)
-                )
-
-                # using the first output, determine the length of the
-                # struct-array that will be pushed to shm.
-                history_by_field = np.zeros(
-                    length,
-                    dtype=dst_shm.array.dtype,
-                )
-
-            if output is not None:
-                history_by_field[key] = output
-
-    else:
-        if not isinstance(history_output, np.ndarray):
-            raise ValueError(
-                f'`{func_name}` is a single output FSP and should '
-                f'yield an `np.ndarray` for history'
-            )
-        history_by_field = np.zeros(
-            len(history_output),
-            dtype=dst_shm.array.dtype,
-        )
-        history_by_field[func_name] = history_output
-
-    if history_by_field is None:
-        raise ValueError(f'`{func_name}` produced no history fields')
-    if len(history_by_field) != len(src_history):
-        raise ValueError(
-            f'`{func_name}` history length '
-            f'{len(history_by_field)} does not match source length '
-            f'{len(src_history)}'
-        )
-
-    history_by_field['time'] = src_history['time']
-
-    # TODO: XXX:
-    # THERE'S A BIG BUG HERE WITH THE `index` field since we're
-    # prepending a copy of the first value a few times to make
-    # sub-curves align with the parent bar chart.
-    # This likely needs to be fixed either by,
-    # - manually assigning the index and historical data
-    #   seperately to the shm array (i.e. not using .push())
-    # - developing some system on top of the shared mem array that
-    #   is `index` aware such that historical data can be indexed
-    #   relative to the true first datum? Not sure if this is sane
-    #   for incremental compuations.
-    first: int = src_first
-    dst_shm._first.value = first
-
-    # TODO: can we use this `start` flag instead of the manual
-    # setting above?
-    index: int = dst_shm.push(
-        history_by_field,
-        start=first,
-    )
-
-    profiler(f'{func_name} pushed history')
-    profiler.finish()
-    assert index == src_last
-    return index
-
-
 async def connect_streams(
     casc: Cascade,
     mkt: MktPair,
-    quote_stream: AsyncIterator[QuoteFrame],
+    quote_stream: trio.abc.ReceiveChannel,
     src: Flume,
     dst: Flume,
 
-    edge_func: EdgeFunc,
+    edge_func: Callable,
 
     # attach_stream: bool = False,
     task_status: TaskStatus[None] = trio.TASK_STATUS_IGNORED,
@@ -527,38 +239,119 @@ async def connect_streams(
     # fqme: str = mkt.fqme
     fqme: str = src.mkt.fqme
 
+    # TODO: dynamic introspection of what the underlying (vertex)
+    # function actually requires from input node (flumes) then
+    # deliver those inputs as part of a graph "compilation" step?
+    out_stream = edge_func(
+
+        # TODO: do we even need this if we do the feed api right?
+        # shouldn't a local stream do this before we get a handle
+        # to the async iterable? it's that or we do some kinda
+        # async itertools style?
+        filter_quotes_by_sym(fqme, quote_stream),
+
+        # XXX: currently the ``ohlcv`` arg, but we should allow
+        # (dynamic) requests for src flume (node) streams?
+        src.rt_shm,
+    )
+
     # HISTORY COMPUTE PHASE
     # conduct a single iteration of fsp with historical bars input
     # and get historical output.
-    history_output: FspHistory
-    src_shm: ShmArray = src.rt_shm
-    (
-        out_stream,
-        history_output,
-        src_bounds,
-        src_history,
-    ) = await _open_history_snapshot(
-        edge_func,
-        quote_stream,
-        fqme,
-        src_shm,
+    history_output: (
+        dict[str, np.ndarray]  # multi-output case
+        | np.ndarray,  # single output case
     )
-    func_name: str = edge_func.__name__
+    history_output = await anext(out_stream)
+
+    func_name = edge_func.__name__
     profiler(f'{func_name} generated history')
 
+    # build struct array with an 'index' field to push as history
+
+    # TODO: push using a[['f0', 'f1', .., 'fn']] = .. syntax no?
+    # if the output array is multi-field then push
+    # each respective field.
     dst_shm: ShmArray = dst.rt_shm
-    try:
-        index: int = _write_history_output(
-            dst_shm,
-            history_output,
-            src_bounds,
-            src_history,
-            func_name,
-            profiler,
+    fields = getattr(dst_shm.array.dtype, 'fields', None).copy()
+    fields.pop('index')
+    history_by_field: np.ndarray | None = None
+    src_shm: ShmArray = src.rt_shm
+    src_time = src_shm.array['time']
+
+    if (
+        fields and
+        len(fields) > 1
+    ):
+        if not isinstance(history_output, dict):
+            raise ValueError(
+                f'`{func_name}` is a multi-output FSP and should yield a '
+                '`dict[str, np.ndarray]` for history'
             )
-    except BaseException:
-        await _close_fsp_stream(out_stream)
-        raise
+
+        for key in fields.keys():
+            if key in history_output:
+                output = history_output[key]
+
+                if history_by_field is None:
+
+                    if output is None:
+                        length = len(src_shm.array)
+                    else:
+                        length = len(output)
+
+                    # using the first output, determine
+                    # the length of the struct-array that
+                    # will be pushed to shm.
+                    history_by_field = np.zeros(
+                        length,
+                        dtype=dst_shm.array.dtype
+                    )
+
+                if output is None:
+                    continue
+
+                history_by_field[key] = output
+
+    # single-key output stream
+    else:
+        if not isinstance(history_output, np.ndarray):
+            raise ValueError(
+                f'`{func_name}` is a single output FSP and should yield an '
+                '`np.ndarray` for history'
+            )
+        history_by_field = np.zeros(
+            len(history_output),
+            dtype=dst_shm.array.dtype
+        )
+        history_by_field[func_name] = history_output
+
+    history_by_field['time'] = src_time[-len(history_by_field):]
+
+    history_output['time'] = src_shm.array['time']
+
+    # TODO: XXX:
+    # THERE'S A BIG BUG HERE WITH THE `index` field since we're
+    # prepending a copy of the first value a few times to make
+    # sub-curves align with the parent bar chart.
+    # This likely needs to be fixed either by,
+    # - manually assigning the index and historical data
+    #   seperately to the shm array (i.e. not using .push())
+    # - developing some system on top of the shared mem array that
+    #   is `index` aware such that historical data can be indexed
+    #   relative to the true first datum? Not sure if this is sane
+    #   for incremental compuations.
+    first = dst_shm._first.value = src_shm._first.value
+
+    # TODO: can we use this `start` flag instead of the manual
+    # setting above?
+    index = dst_shm.push(
+        history_by_field,
+        start=first,
+    )
+
+    profiler(f'{func_name} pushed history')
+    profiler.finish()
 
     # setup a respawn handle
     with trio.CancelScope() as cs:
@@ -605,9 +398,6 @@ async def connect_streams(
                 # if hz > 60:
                 #     log.info(f'FSP quote too fast: {hz}')
                 # last = time.time()
-        finally:
-            try:
-                await _close_fsp_stream(out_stream)
         finally:
             casc.complete.set()
 
@@ -620,13 +410,11 @@ async def cascade(
     fqme: str,
 
     # flume pair cascaded using an "edge function"
-    src_flume_addr: dict[str, Any],
-    dst_flume_addr: dict[str, Any],
+    src_flume_addr: dict,
+    dst_flume_addr: dict,
     ns_path: NamespacePath,
 
-    shm_registry: list[
-        tuple[NDTokenMsg, str, NDTokenMsg]
-    ],
+    shm_registry: dict[str, NDToken],
 
     zero_on_step: bool = False,
     loglevel: str|None = None,
@@ -664,7 +452,7 @@ async def cascade(
     # src: ShmArray = attach_shm_array(token=src_shm_token)
     # dst: ShmArray = attach_shm_array(readonly=False, token=dst_shm_token)
 
-    reg: dict[NamespacePath, Fsp] = _load_builtins()
+    reg = _load_builtins()
     lines = '\n'.join([f'{key.rpartition(":")[2]} => {key}' for key in reg])
     log.info(
         f'Registered FSP set:\n{lines}'
@@ -677,21 +465,18 @@ async def cascade(
     # not sure how else to do it.
     for (token, fsp_name, dst_token) in shm_registry:
         Fsp._flow_registry[(
-            NDToken.from_msg(dict(token)),
+            NDToken.from_msg(token),
             fsp_name,
-        )] = NDToken.from_msg(dict(dst_token)), None
+        )] = NDToken.from_msg(dst_token), None
 
-    fsp: Fsp|None = reg.get(
+    fsp: Fsp = reg.get(
         NamespacePath(ns_path)
     )
-    if (
-        fsp is None
-        or
-        not fsp.func
-    ):
+    func: Callable = fsp.func
+
+    if not func:
         # TODO: assume it's a func target path
         raise ValueError(f'Unknown fsp target: {ns_path}')
-    func: FspFunc = fsp.func
 
     _fqme: str = src.mkt.fqme
     assert _fqme == fqme
@@ -736,7 +521,7 @@ async def cascade(
             # the target task is spawned implicitly and then the event is
             # set via some higher level api? At that poing we might as well
             # be writing a one-cancels-one nursery though right?
-            casc: Cascade = Cascade(
+            casc = Cascade(
                 src,
                 dst,
                 tn,
@@ -744,7 +529,7 @@ async def cascade(
             )
 
             # TODO: this seems like it should be wrapped somewhere?
-            fsp_target: Callable[..., Awaitable[None]] = partial(
+            fsp_target = partial(
                 connect_streams,
                 casc=casc,
                 mkt=mkt,
@@ -758,20 +543,9 @@ async def cascade(
                 # and renders dst flume output(s)
                 edge_func=func
             )
-
-            # Subscribe before history bootstrap so in-place repairs
-            # with unchanged SHM bounds can not lose invalidations.
-            period_s: float = src.rt_sample_period_s
-
-            async with (
-                open_sample_stream(
-                    period_s=period_s,
-                    loglevel=loglevel,
-                ) as istream,
-                casc.open_edge(
+            async with casc.open_edge(
                 bind_func=fsp_target,
-                ) as index,
-            ):
+            ) as index:
                 # casc.bind_func = fsp_target
                 # index = await tn.start(fsp_target)
                 dst_shm: ShmArray = dst.rt_shm
@@ -793,51 +567,45 @@ async def cascade(
                 async with ctx.open_stream() as client_stream:
                     casc.client_stream: tractor.MsgStream = client_stream
 
+                    s, step, ld = casc.is_synced()
+
+                    # detect sample period step for subscription to increment
+                    # signal
+                    times = src.rt_shm.array['time']
+                    if len(times) > 1:
+                        last_ts = times[-1]
+                        delay_s: float = float(last_ts - times[times != last_ts][-1])
+                    else:
+                        # our default "HFT" sample rate.
+                        delay_s: float = _default_delay_s
+
+                    # sub and increment the underlying shared memory buffer
+                    # on every step msg received from the global `samplerd`
+                    # service.
+                    async with open_sample_stream(
+                        period_s=float(delay_s),
+                        loglevel=loglevel,
+                    ) as istream:
+
                         profiler(f'{func_name}: sample stream up')
                         profiler.finish()
 
-                    async for sample_msg in istream:
-                        # print(f'FSP incrementing {sample_msg}')
-                        msg: SampleMsg = cast(
-                            SampleMsg,
-                            sample_msg,
-                        )
+                        async for i in istream:
+                            # print(f'FSP incrementing {i}')
 
-                        # ``samplerd`` broadcasts history events for
-                        # every feed and period. Only this cascade's
-                        # source can invalidate its destination.
-                        if not _is_relevant_sample_msg(
-                            msg,
-                            fqme,
-                            period_s,
-                        ):
-                            continue
-
-                        # Respawn the compute task after source history
-                        # revisions or an actual bounds desync.
-                        (
-                            synced,
-                            step_diff,
-                            _,
-                        ) = casc.is_synced()
-                        if _needs_history_resync(
-                            msg,
-                            synced,
-                        ):
-                            step_diff = (
-                                await casc.poll_and_sync_to_step()
-                            )
-
-                        # Backfill broadcasts only announce source
-                        # history revisions. They never represent a
-                        # new sample row. Duplicate sample wakeups are
-                        # likewise non-advancing when already aligned.
-                        if not _should_advance_dst(
-                            msg,
-                            step_diff,
-                        ):
+                            # respawn the compute task if the source
+                            # array has been updated such that we compute
+                            # new history from the (prepended) source.
+                            synced, step_diff, _ = casc.is_synced()
+                            if not synced:
+                                step_diff: int = await casc.poll_and_sync_to_step()
+
+                                # skip adding a last bar since we should already
+                                # be step alinged
+                                if step_diff == 0:
                                     continue
 
+                            # read out last shm row, copy and write new row
                             array = dst_shm.array
 
                             # some metrics like vlm should be reset
@@ -851,12 +619,8 @@ async def cascade(
 
                             # sync with source buffer's time step
                             src_l2 = src_shm.array[-2:]
-                        src_li, src_lt = src_l2[-1][
-                            ['index', 'time']
-                        ]
-                        src_2li, src_2lt = src_l2[-2][
-                            ['index', 'time']
-                        ]
+                            src_li, src_lt = src_l2[-1][['index', 'time']]
+                            src_2li, src_2lt = src_l2[-2][['index', 'time']]
                             dst_shm._array['time'][src_li] = src_lt
                             dst_shm._array['time'][src_2li] = src_2lt
 

piker/fsp/_momo.py

@@ -18,37 +18,32 @@
 Momentum bby.
 
 """
-from typing import (
-    AsyncIterator,
-)
+from typing import AsyncIterator, Optional
 
 import numpy as np
-from numpy.typing import NDArray
-from numba import jit
+from numba import jit, float64, optional, int64
 
-from ._api import (
-    fsp,
-    FspStream,
-)
-from ..data import (
-    FeedQuote,
-    iterticks,
-    Tick,
-)
+from ._api import fsp
+from ..data import iterticks
 from tractor.ipc._shm import ShmArray
 
 
 @jit(
+    float64[:](
+        float64[:],
+        optional(float64),
+        optional(float64)
+    ),
     nopython=True,
     nogil=True
 )
 def ema(
 
-    y: NDArray[np.float64],
-    alpha: float|None = None,
-    ylast: float|None = None,
+    y: 'np.ndarray[float64]',
+    alpha: optional(float64) = None,
+    ylast: optional(float64) = None,
 
-) -> NDArray[np.float64]:
+) -> 'np.ndarray[float64]':
     r'''
     Exponential weighted moving average owka 'Exponential smoothing'.
 
@@ -85,14 +80,10 @@ def ema(
         # directly to the com of a SMA or WMA:
         alpha = 2 / float(n + 1)
 
-    s = np.empty(n, dtype=np.float64)
+    s = np.empty(n, dtype=float64)
 
     if n == 1:
-        s[0] = (
-            y[0]
-            if ylast is None
-            else y[0] * alpha + ylast * (1 - alpha)
-        )
+        s[0] = y[0] * alpha + ylast * (1 - alpha)
 
     else:
         if ylast is None:
@@ -118,12 +109,13 @@ def ema(
 # )
 def _rsi(
 
-    signal: NDArray[np.float64],
-    period: int = 14,
-    up_ema_last: float|None = None,
-    down_ema_last: float|None = None,
+    # TODO: use https://github.com/ramonhagenaars/nptyping
+    signal: 'np.ndarray[float64]',
+    period: int64 = 14,
+    up_ema_last: float64 = None,
+    down_ema_last: float64 = None,
 
-) -> tuple[NDArray[np.float64], float, float]:
+) -> 'np.ndarray[float64]':
     '''
     relative strengggth.
 
@@ -133,18 +125,10 @@ def _rsi(
     df = np.diff(signal, prepend=0)
 
     up = np.where(df > 0, df, 0)
-    up_ema = ema(
-        up,
-        alpha,
-        up_ema_last,
-    )
+    up_ema = ema(up, alpha, up_ema_last)
 
     down = np.where(df < 0, -df, 0)
-    down_ema = ema(
-        down,
-        alpha,
-        down_ema_last,
-    )
+    down_ema = ema(down, alpha, down_ema_last)
 
     # avoid dbz errors, this leaves the first
     # index == 0 right?
@@ -167,7 +151,7 @@ def _wma(
 
     signal: np.ndarray,
     length: int,
-    weights: np.ndarray|None = None,
+    weights: Optional[np.ndarray] = None,
 
 ) -> np.ndarray:
     '''
@@ -190,11 +174,11 @@ def _wma(
 @fsp
 async def wma(
 
-    source: AsyncIterator[FeedQuote],
-    ohlcv: ShmArray,
-    length: int = 14,
+    source,  #: AsyncStream[np.ndarray],
+    length: int,
+    ohlcv: np.ndarray,  # price time-frame "aware"
 
-) -> FspStream:
+) -> AsyncIterator[np.ndarray]:  # maybe something like like FspStream?
     '''
     Streaming weighted moving average.
 
@@ -204,33 +188,23 @@ async def wma(
 
     '''
     # deliver historical output as "first yield"
-    close: NDArray[np.float64] = ohlcv.array['close']
-    history: NDArray[np.float64] = np.full(len(close), np.nan)
-    if len(close) >= length:
-        history[length - 1:] = _wma(close, length)
-    yield history
+    yield _wma(ohlcv.array['close'], length)
 
     # begin real-time section
 
     async for quote in source:
-        _tick: Tick
-        for _tick in iterticks(
-            quote,
-            types=['trade'],
-        ):
-            closes: np.ndarray = ohlcv.last(length)['close']
-            if len(closes) == length:
-                yield 'wma', _wma(closes, length)[-1]
+        for tick in iterticks(quote, type='trade'):
+            yield _wma(ohlcv.last(length))
 
 
 @fsp
 async def rsi(
 
-    source: AsyncIterator[FeedQuote],
+    source: 'QuoteStream[Dict[str, Any]]',  # noqa
     ohlcv: ShmArray,
     period: int = 14,
 
-) -> FspStream:
+) -> AsyncIterator[np.ndarray]:
     '''
     Multi-timeframe streaming RSI.
 
@@ -276,4 +250,4 @@ async def rsi(
                 up_ema_last=last_up_ema_close,
                 down_ema_last=last_down_ema_close,
             )
-            yield 'rsi', rsi_out[-1]
+            yield rsi_out[-1:]

piker/fsp/_volume.py

@@ -14,17 +14,13 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
+from typing import AsyncIterator, Optional, Union
+
 import numpy as np
 from tractor.trionics._broadcast import AsyncReceiver
 
-from ._api import (
-    fsp,
-    FspStream,
-)
-from ..data import (
-    FeedQuote,
-    iterticks,
-)
+from ._api import fsp
+from ..data import iterticks
 from tractor.ipc._shm import ShmArray
 from ._momo import _wma
 from ..log import get_logger
@@ -40,11 +36,7 @@ def wap(
     signal: np.ndarray,
     weights: np.ndarray,
 
-) -> tuple[
-    np.ndarray,  # weighted average price
-    np.ndarray,  # cumulative weighted input
-    np.ndarray,  # cumulative weights
-]:
+) -> np.ndarray:
     '''
     Weighted average price from signal and weights.
 
@@ -70,13 +62,16 @@ def wap(
 @fsp
 async def tina_vwap(
 
-    source: AsyncReceiver[FeedQuote],
+    source: AsyncReceiver[dict],
     ohlcv: ShmArray,  # OHLC sampled history
 
     # TODO: anchor logic (eg. to session start)
-    anchors: np.ndarray|None = None,
+    anchors: Optional[np.ndarray] = None,
 
-) -> FspStream:
+) -> Union[
+    AsyncIterator[np.ndarray],
+    float
+]:
     '''
     Streaming volume weighted moving average.
 
@@ -90,7 +85,7 @@ async def tina_vwap(
 
     a = ohlcv.array
     chl3 = (a['close'] + a['high'] + a['low']) / 3
-    v = np.maximum(a['volume'], 0)
+    v = a['volume']
 
     h_vwap, cum_wp, cum_v = wap(chl3, v)
 
@@ -112,7 +107,7 @@ async def tina_vwap(
             # ]
 
             # this computes tick-by-tick weightings from here forward
-            size = max(float(tick['size']), 0.)
+            size = tick['size']
             price = tick['price']
 
             v_tot += size
@@ -132,10 +127,12 @@ async def tina_vwap(
     curve_style='step',
 )
 async def dolla_vlm(
-    source: AsyncReceiver[FeedQuote],
+    source: AsyncReceiver[dict],
     ohlcv: ShmArray,  # OHLC sampled history
 
-) -> FspStream:
+) -> AsyncIterator[
+    tuple[str, Union[np.ndarray, float]],
+]:
     '''
     "Dollar Volume", aka the volume in asset-currency-units (usually
     a fiat) computed from some price function for the sample step
@@ -147,7 +144,7 @@ async def dolla_vlm(
     '''
     a = ohlcv.array
     chl3 = (a['close'] + a['high'] + a['low']) / 3
-    v = np.maximum(a['volume'], 0)
+    v = a['volume']
 
     # on first iteration yield history
     yield {
@@ -170,7 +167,7 @@ async def dolla_vlm(
         ):
 
             # this computes tick-by-tick weightings from here forward
-            size = max(float(tick['size']), 0.)
+            size = tick['size']
             price = tick['price']
 
             li = ohlcv.index
@@ -230,14 +227,14 @@ async def dolla_vlm(
     curve_style='line',
 )
 async def flow_rates(
-    source: AsyncReceiver[FeedQuote],
+    source: AsyncReceiver[dict],
     ohlcv: ShmArray,  # OHLC sampled history
 
     # TODO (idea): a dynamic generic / boxing type that can be updated by other
     # FSPs, user input, and possibly any general event stream in
     # real-time. Hint: ideally implemented with caching until mutated
     # ;)
-    period: int = 1,
+    period: 'Param[int]' = 1,  # noqa
 
     # TODO: support other means by providing a map
     # to weights `partial()`-ed with `wma()`?
@@ -255,7 +252,9 @@ async def flow_rates(
     # lazy copy in that case?
     # dvlm: 'Fsp[dolla_vlm]'
 
-) -> FspStream:
+) -> AsyncIterator[
+    tuple[str, Union[np.ndarray, float]],
+]:
     # generally no history available prior to real-time calcs
     yield {
         # from ib

piker/ui/_dataviz.py

@@ -71,7 +71,6 @@ from ..toolz.profile import (
 
 
 if TYPE_CHECKING:
-    from . import _pg_overrides as pgo
     from ._interaction import ChartView
     from ._chart import ChartPlotWidget
     from ._display import DisplayState
@@ -283,7 +282,7 @@ class Viz(Struct):
 
     '''
     name: str
-    plot: pgo.PlotItem
+    plot: pg.PlotItem
     _shm: ShmArray
     flume: Flume
     graphics: Curve | BarItems
@@ -319,7 +318,6 @@ class Viz(Struct):
     _in_ds: bool = False
     _index_step: float | None = None
     _time_step: float | None = None
-    _last_fsp_update_sig: tuple[int, int, bytes]|None = None
 
     # map from uppx -> (downsampled data, incremental graphics)
     _src_r: Renderer | None = None
@@ -985,10 +983,10 @@ class Viz(Struct):
         # XXX: SUPER UGGGHHH... without this we get stale cache
         # graphics that "smear" across the view horizontally
         # when panning and the first datum is out of view..
+        reset_cache = False
         if (
             reset_cache
-            or
-            should_redraw
+            or should_redraw
         ):
             # assign output paths to graphicis obj but
             # after a coords-cache reset.

piker/ui/_fsp.py

@@ -20,8 +20,6 @@ FSP UI and graphics components.
 Financial signal processing cluster and real-time graphics management.
 
 '''
-from __future__ import annotations
-
 from contextlib import asynccontextmanager as acm
 from functools import partial
 import inspect
@@ -29,11 +27,9 @@ from itertools import cycle
 from typing import (
     AsyncGenerator,
     Any,
-    TYPE_CHECKING,
 )
 
 import msgspec
-import numpy as np
 import pyqtgraph as pg
 import tractor
 from tractor.trionics import maybe_open_context
@@ -57,11 +53,7 @@ from piker.data._sharedmem import try_read
 from piker.log import get_logger
 from piker.toolz import Profiler
 from piker.types import Struct
-from ._axes import (
-    Axis,
-    PriceAxis,
-    YAxisLabel,
-)
+from ._axes import PriceAxis
 from ..calc import humanize
 from ._chart import (
     ChartPlotWidget,
@@ -73,27 +65,26 @@ from ._forms import (
     open_form_input_handling,
 )
 
-if TYPE_CHECKING:
-    from ._dataviz import Viz
-    from ._interaction import ChartView
-
 log = get_logger(__name__)
 
 
 def update_fsp_chart(
-    viz: Viz,
+    viz,
     graphics_name: str,
-    array_key: str,
-    force_redraw: bool = False,
+    array_key: str|None,
+    **kwargs,
 
 ) -> None:
 
-    shm: ShmArray = viz.shm
-    array: np.ndarray = shm.array
-    last_row: np.void|None = try_read(array)
+    shm = viz.shm
+    if not shm:
+        return
+
+    array = shm.array
+    last_row = try_read(array)
 
     # guard against unreadable case
-    if last_row is None:
+    if not last_row:
         log.warning(
             f'Read-race on shm array,\n'
             f'graphics_name: {graphics_name!r}\n'
@@ -101,100 +92,23 @@ def update_fsp_chart(
         )
         return
 
-    # `update_fsp_chart()` runs for every source quote, while an FSP can
-    # publish more slowly. `Viz._last_fsp_update_sig` combines the
-    # destination's absolute half-open `ShmArray` bounds with bitwise
-    # `last_row['time']` and `last_row[array_key]` scalars. Comparing
-    # bytes keeps repeated NaNs stable; scalar equality treats every NaN
-    # as changed. A changed `ShmArray._first.value` catches history
-    # prepends, a changed `ShmArray._last.value` catches sample appends,
-    # and changed row bytes catch mutation of the current sample.
-    # `force_redraw` bypasses this gate because an equal-bound gap repair
-    # can rewrite earlier rows without changing the current row. This is
-    # only a repaint key, not synchronization proof; a mixed SHM read
-    # merely causes one extra later redraw.
-    update_sig: tuple[int, int, bytes] = (
-        shm._first.value,
-        shm._last.value,
-        np.asarray(last_row['time']).tobytes()
-        + np.asarray(last_row[array_key]).tobytes(),
-    )
-    if (
-        not force_redraw
-        and
-        viz._last_fsp_update_sig == update_sig
-    ):
-        return
-
     # update graphics
     # NOTE: this does a length check internally which allows it
     # staying above the last row check below..
-    viz.update_graphics(force_redraw=force_redraw)
-    viz._last_fsp_update_sig = update_sig
+    viz.update_graphics()
 
     # XXX: re: ``array_key``: fsp func names must be unique meaning we
     # can't have duplicates of the underlying data even if multiple
     # sub-charts reference it under different 'named charts'.
 
     # read from last calculated value and update any label
-    axis: Axis = viz.plot.getAxis('right')
-    last_val_sticky: YAxisLabel|None = axis._stickies.get(
-        graphics_name,
-    )
-    if last_val_sticky is not None:
-        last: float = float(last_row[array_key])
+    last_val_sticky = viz.plot.getAxis(
+        'right')._stickies.get(graphics_name)
+    if last_val_sticky:
+        last = last_row[array_key]
         last_val_sticky.update_from_data(-1, last)
 
 
-def update_fsp_vizs(
-    linked: LinkedSplits,
-    shm_token: NDToken,
-
-) -> int:
-    '''
-    Redraw each `Viz` whose `Viz.shm` matches `shm_token`.
-
-    The FSP engine sends its destination token after rewriting
-    history. Every chart or subplot reading that SHM needs its
-    graphics caches updated and its view rescaled.
-
-    '''
-    charts: tuple[ChartPlotWidget, ...] = (
-        linked.chart,
-        *linked.subplots.values(),
-    )
-    updated: int = 0
-    views: list[ChartView] = []
-    for chart in charts:
-        for viz in chart._vizs.values():
-            viz: Viz
-            shm: ShmArray = viz.shm
-            if shm._token != shm_token:
-                continue
-
-            viz._mxmns.clear()
-            # Backfills can rewrite visible values without changing
-            # bounds, so bypass `Viz.update_graphics()` cache checks.
-            update_fsp_chart(
-                viz,
-                graphics_name=viz.name,
-                array_key=viz.name,
-                force_redraw=True,
-            )
-            updated += 1
-            view: ChartView = viz.plot.vb
-            if view not in views:
-                views.append(view)
-
-    for view in views:
-        view.interact_graphics_cycle(
-            do_linked_charts=False,
-            do_overlay_scaling=True,
-        )
-
-    return updated
-
-
 @acm
 async def open_fsp_sidepane(
     linked: LinkedSplits,
@@ -371,6 +285,7 @@ async def run_fsp_ui(
         # first UI update, usually from shm pushed history
         viz = chart.get_viz(array_key)
         update_fsp_chart(
+            chart,
             viz,
             name,
             array_key=array_key,
@@ -546,11 +461,9 @@ class FspAdmin:
                             log.debug(
                                 f'Re-syncing graphics for fsp: {ns_path}'
                             )
-                            update_fsp_vizs(
-                                self.linked,
-                                shm_token=NDToken.from_msg(
-                                    dict(info['key']),
-                                ),
+                            self.linked.graphics_cycle(
+                                trigger_all=True,
+                                prepend_update_index=info['first'],
                             )
                     else:
                         log.info(f'recved unexpected fsp engine msg: {msg}')

piker/ui/_pg_overrides.py

@@ -22,20 +22,8 @@ Generally, our does not require "scentific precision" for pixel perfect
 view transforms.
 
 """
-from __future__ import annotations
-
-from typing import (
-    Callable,
-    TYPE_CHECKING,
-)
-
 import pyqtgraph as pg
 
-if TYPE_CHECKING:
-    from ._axes import Axis
-    from ._chart import ChartPlotWidget
-    from ._interaction import ChartView
-
 
 def invertQTransform(tr):
     """Return a QTransform that is the inverse of *tr*.
@@ -95,11 +83,6 @@ class PlotItem(pg.PlotItem):
       options
 
     '''
-    vb: ChartView
-    chart_widget: ChartPlotWidget|None
-    if TYPE_CHECKING:
-        getAxis: Callable[[str], Axis]
-
     def __init__(
         self,
         parent=None,

tests/test_accounting.py

@@ -5,26 +5,20 @@
 - config file IO
 
 '''
-from decimal import Decimal
 from pathlib import Path
-from types import ModuleType
-import tomllib
 
 import pytest
 from piker import config
 from piker.accounting import (
     Account,
-    Asset,
     calc,
     open_account,
     load_account,
     load_account_from_ledger,
-    MktPair,
     open_trade_ledger,
     Position,
     TransactionLedger,
 )
-from piker.data._symcache import SymbologyCache
 import tractor
 
 
@@ -53,82 +47,6 @@ def test_account_file_default_empty(
     assert path.parent.name == 'accounting'
 
 
-def test_paper_ledger_fuzzy_qualifies_mktmap_keys(
-    tmp_path: Path,
-):
-    '''
-    Qualify legacy paper-ledger FQMEs via symcache string keys.
-
-    ``SymbologyCache.search()`` previously passed its ``mktmaps``
-    mapping directly to RapidFuzz. RapidFuzz searches mapping values,
-    so a paper fill with an unqualified FQME made RapidFuzz call
-    ``len()`` on a ``MktPair`` while
-    ``TransactionLedger.write_config()`` exited.
-    This test installs both native-ID and FQME keys for one market,
-    proves search preserves the matched alias, then writes the same
-    unqualified paper transaction shape. The persisted fully
-    qualified keys prove the ledger exit path resolves the market
-    without treating ``MktPair`` values as fuzzy-search sequences.
-
-    '''
-    usdt = Asset(
-        name='usdt',
-        atype='crypto',
-        tx_tick=Decimal('0.00000001'),
-    )
-    nvda = Asset(
-        name='nvda',
-        atype='stock',
-        tx_tick=Decimal('0.00000001'),
-    )
-    mkt = MktPair(
-        dst=nvda,
-        src=usdt,
-        price_tick=Decimal('0.01'),
-        size_tick=Decimal('0.001'),
-        bs_mktid='NVDAUSDT',
-        broker='binance',
-        venue='usdtm',
-        expiry='perp',
-    )
-    symcache = SymbologyCache(
-        mod=ModuleType('binance'),
-        fp=tmp_path / 'binance.symcache.toml',
-        mktmaps={
-            mkt.bs_mktid: mkt,
-            mkt.fqme: mkt,
-        },
-    )
-
-    matches = symcache.search(mkt.bs_mktid)
-    assert next(iter(matches)) == mkt.bs_mktid
-    assert matches[mkt.bs_mktid] is mkt
-
-    ledger_path = tmp_path / 'trades_binance_paper.toml'
-    unqualified_fqme = 'nvdausdt.usdtm.perp'
-    ledger = TransactionLedger(
-        ledger_dict={
-            'fill-id': {
-                'fqme': unqualified_fqme,
-                'bs_mktid': unqualified_fqme,
-            },
-        },
-        file_path=ledger_path,
-        account='paper',
-        mod=symcache.mod,
-        tx_sort=lambda txns: txns.items(),
-        symcache=symcache,
-    )
-
-    ledger.write_config()
-
-    with ledger_path.open('rb') as ledger_file:
-        saved = tomllib.load(ledger_file)['fill-id']
-
-    assert saved['fqme'] == mkt.fqme
-    assert saved['bs_mktid'] == mkt.fqme
-
-
 @pytest.mark.parametrize(
     'fq_acnt',
     [

tests/test_fsp_momo.py

@@ -1,64 +0,0 @@
-'''
-Momentum FSP numerical regressions.
-
-'''
-import numpy as np
-import pytest
-
-from piker.fsp._momo import ema
-
-
-@pytest.mark.parametrize(
-    'args,kwargs,expected',
-    [
-        ((), {}, [1., 1.5, 2.25]),
-        ((None,), {}, [1., 1.5, 2.25]),
-        ((None, None), {}, [1., 1.5, 2.25]),
-        ((0.25,), {}, [1., 1.25, 1.6875]),
-        ((), {'alpha': 0.25}, [1., 1.25, 1.6875]),
-        ((), {'ylast': 4.}, [4., 3., 3.]),
-        ((None, 4.), {}, [4., 3., 3.]),
-        ((0.5, 4.), {}, [4., 3., 3.]),
-    ],
-)
-def test_ema_optional_arguments(
-    args: tuple[float|None, ...],
-    kwargs: dict[str, float],
-    expected: list[float],
-) -> None:
-    '''
-    Accept omitted EMA defaults through the real Numba dispatcher.
-
-    `ema()` exposed optional smoothing and seed arguments, but its
-    eager signature accepted only explicit values or `None`.
-    Numba's omitted-argument types therefore raised `TypeError`
-    before the numerical kernel ran. Exercise positional and keyword
-    omissions as well as explicit arguments, checking the resulting
-    recurrence against hand-computed values. Import the production
-    dispatcher so this catches signature regressions that a call to
-    `ema.py_func` would miss.
-
-    '''
-    signal: np.ndarray = np.array([1., 2., 3.])
-    result: np.ndarray = ema(signal, *args, **kwargs)
-
-    np.testing.assert_allclose(result, expected)
-    assert result.dtype == np.float64
-
-
-def test_ema_single_sample_continuation() -> None:
-    '''
-    Preserve the previous EMA when advancing one realtime sample.
-
-    The old one-sample path multiplied an absent seed by a float,
-    failing instead of initializing from the sample. Removing the
-    eager Numba signature must retain the existing initialization
-    fix and the previous-value update used by realtime RSI. Use a
-    distinct seed and smoothing factor so copying either the seed
-    or sample fails; verify an omitted seed uses the sole sample.
-
-    '''
-    signal: np.ndarray = np.array([3.])
-
-    np.testing.assert_allclose(ema(signal), [3.])
-    np.testing.assert_allclose(ema(signal, 0.25, 7.), [6.])

tests/test_fsp_sync.py

@@ -1,610 +0,0 @@
-'''
-FSP history synchronization regressions.
-
-'''
-from collections.abc import AsyncIterator
-from types import SimpleNamespace
-from typing import cast
-
-import numpy as np
-import pytest
-import trio
-
-from piker.fsp._engine import (
-    _is_relevant_sample_msg,
-    _needs_history_resync,
-    _open_history_snapshot,
-    _should_advance_dst,
-    Cascade,
-    EdgeFunc,
-    QuoteFrame,
-)
-from piker.fsp._momo import (
-    rsi,
-    wma,
-)
-from piker.fsp._volume import (
-    dolla_vlm,
-    tina_vwap,
-)
-from piker.accounting import MktPair
-from piker.data.flows import Flume
-from piker.data._sharedmem import NDTokenMsg
-from piker.data.ticktools import FeedQuote
-from piker.fsp._api import Fsp
-from piker.ui._chart import LinkedSplits
-from piker.ui._fsp import (
-    update_fsp_chart,
-    update_fsp_vizs,
-)
-from tractor.ipc._shm import (
-    NDToken,
-    ShmArray,
-)
-
-type SampleMsg = dict[
-    str,
-    int|float|tuple[str, float],
-]
-
-
-def test_sample_period_ignores_market_closure_gap() -> None:
-    '''
-    Keep FSP cascades subscribed to their regular sampler period.
-
-    Starting a cascade immediately after a market closure left the
-    source SHM tail with a large gap between its last two distinct
-    timestamps. Using only that pair subscribed the cascade to the gap
-    duration instead of the regular one-second stream. Realtime FSP
-    writes then repeatedly replaced one row without advancing its SHM
-    bound. Give both source SHMs closure-gap tail deltas and prove
-    `Flume.get_ds_info()` returns its declared one- and 60-second
-    periods instead of deriving cadence from those rows.
-
-    '''
-    rt_shm = OhlcvShm(length=4)
-    hist_shm = OhlcvShm(length=4)
-    rt_shm._array['time'][:4] = [100, 101, 102, 3600]
-    hist_shm._array['time'][:4] = [100, 160, 220, 3600]
-    flume = Flume(
-        mkt=cast(MktPair, SimpleNamespace()),
-        first_quote={},
-        _rt_shm_token=rt_shm._token,
-        _hist_shm_token=hist_shm._token,
-    )
-    flume._rt_shm = cast(ShmArray, rt_shm)
-    flume._hist_shm = cast(ShmArray, hist_shm)
-
-    assert np.diff(rt_shm.array['time'])[-1] == 3498
-    assert np.diff(hist_shm.array['time'])[-1] == 3380
-    assert flume.get_ds_info() == (1, 60, 60)
-
-
-class Value:
-    def __init__(self, value: int) -> None:
-        self.value: int = value
-
-
-class Shm:
-    def __init__(
-        self,
-        first: int,
-        last: int,
-        token: str = 'fsp',
-
-    ) -> None:
-        self._first: Value = Value(first)
-        self._last: Value = Value(last)
-        self._array: np.ndarray = np.ones(4096)
-        self._len: int = len(self._array)
-        self._token: NDToken = NDToken(
-            shm_name=token,
-            shm_first_index_name=f'{token}_first',
-            shm_last_index_name=f'{token}_last',
-            dtype_descr=(('value', '<f8'),),
-            size=len(self._array),
-        )
-
-    @property
-    def array(self) -> np.ndarray:
-        return self._array[
-            self._first.value:self._last.value
-        ]
-
-    @property
-    def token(self) -> NDTokenMsg:
-        return cast(NDTokenMsg, self._token.as_msg())
-
-    @property
-    def index(self) -> int:
-        return self._last.value % len(self._array)
-
-    def last(self, length: int = 1) -> np.ndarray:
-        return self.array[-length:]
-
-
-class OhlcvShm(Shm):
-    def __init__(self, length: int = 32) -> None:
-        dtype = np.dtype([
-            ('index', '<i8'),
-            ('time', '<i8'),
-            ('open', '<f8'),
-            ('high', '<f8'),
-            ('low', '<f8'),
-            ('close', '<f8'),
-            ('volume', '<f8'),
-        ])
-        self._first = Value(0)
-        self._last = Value(length)
-        self._array = np.ones(length + 8, dtype=dtype)
-        self._array['index'] = np.arange(length + 8)
-        self._array['time'] = np.arange(length + 8)
-        self._array['close'] = np.arange(length + 8) + 1
-        self._array['high'] = self._array['close'] + 1
-        self._array['low'] = self._array['close'] - 1
-        self._len = len(self._array)
-        self._token = NDToken(
-            shm_name='ohlcv',
-            shm_first_index_name='ohlcv_first',
-            shm_last_index_name='ohlcv_last',
-            dtype_descr=tuple(dtype.descr),
-            size=self._len,
-        )
-
-
-class FspShm(Shm):
-    def __init__(
-        self,
-        first: int = 0,
-        last: int = 4,
-        token: str = 'fsp',
-
-    ) -> None:
-        dtype = np.dtype([
-            ('index', '<i8'),
-            ('time', '<f8'),
-            ('flow', '<f8'),
-            ('dark_flow', '<f8'),
-        ])
-        size: int = max(last + 2, 4096)
-        self._first = Value(first)
-        self._last = Value(last)
-        self._array = np.zeros(size, dtype=dtype)
-        self._array['index'] = np.arange(size)
-        self._array['time'] = np.arange(size)
-        self._array['flow'] = np.arange(size)
-        self._len = len(self._array)
-        self._token = NDToken(
-            shm_name=token,
-            shm_first_index_name=f'{token}_first',
-            shm_last_index_name=f'{token}_last',
-            dtype_descr=tuple(dtype.descr),
-            size=self._len,
-        )
-
-
-def mk_cascade(
-    src_bounds: tuple[int, int],
-    dst_bounds: tuple[int, int],
-
-) -> Cascade:
-    fsp: Fsp = cast(
-        Fsp,
-        SimpleNamespace(
-            name='test_fsp',
-            ns_path='tests:test_fsp',
-        ),
-    )
-    src: Flume = cast(
-        Flume,
-        SimpleNamespace(
-            rt_shm=Shm(*src_bounds, token='src'),
-        ),
-    )
-    dst: Flume = cast(
-        Flume,
-        SimpleNamespace(
-            rt_shm=Shm(*dst_bounds, token='dst'),
-        ),
-    )
-    nursery: trio.Nursery = cast(trio.Nursery, None)
-    return Cascade(src, dst, nursery, fsp)
-
-
-@pytest.mark.parametrize(
-    'src_bounds,dst_bounds,expected',
-    [
-        ((10, 20), (10, 20), (True, 0, 0)),
-        ((10, 21), (10, 20), (True, 1, 1)),
-        ((0, 3000), (2000, 3000), (False, 0, 2000)),
-        ((9, 20), (10, 20), (False, 0, 1)),
-        ((8, 20), (10, 20), (False, 0, 2)),
-        ((10, 20), (9, 20), (False, 0, 1)),
-        ((11, 21), (10, 20), (False, 1, 0)),
-        ((10, 22), (10, 20), (False, 2, 2)),
-        ((10, 19), (10, 20), (False, -1, 1)),
-    ],
-)
-def test_cascade_sync_uses_absolute_bounds(
-    src_bounds: tuple[int, int],
-    dst_bounds: tuple[int, int],
-    expected: tuple[bool, int, int],
-
-) -> None:
-    '''
-    Detect every history-bound mismatch without modulo-index skew.
-
-    Source prepends move only its first bound, while a normal realtime
-    step moves only its last bound. The former requires a historical
-    recompute even for one or two rows; the latter permits exactly one
-    destination append. This matrix also shifts equal-length bounds and
-    puts the destination ahead to prove neither state is accepted.
-
-    '''
-    cascade = mk_cascade(src_bounds, dst_bounds)
-
-    assert cascade.is_synced() == expected
-
-
-@pytest.mark.parametrize(
-    'msg,expected',
-    [
-        ({'index': 1}, True),
-        ({'backfilling': ('tsla.nasdaq.ib', 1)}, True),
-        ({'backfilling': ('btcusdt.binance', 1)}, False),
-        ({'backfilling': ('tsla.nasdaq.ib', 60)}, False),
-    ],
-)
-def test_cascade_filters_foreign_backfill_events(
-    msg: SampleMsg,
-    expected: bool,
-
-) -> None:
-    '''
-    Ignore backfill wakeups from overlay markets and other periods.
-
-    Samplerd broadcasts every backfill marker to every period and FSP
-    subscriber. With multiple markets overlaid, an unrelated provider
-    frame used to wake each primary-market cascade and race its own SHM
-    inspection. Feed regular sample steps and matching history events
-    through, but reject both a foreign FQME and the 60-second period.
-
-    '''
-    assert _is_relevant_sample_msg(
-        msg,
-        fqme='tsla.nasdaq.ib',
-        period_s=1,
-    ) is expected
-
-
-@pytest.mark.parametrize(
-    'msg,step_diff,expected',
-    [
-        ({'index': 1}, 1, True),
-        ({'index': 1}, 0, False),
-        ({'backfilling': ('tsla.nasdaq.ib', 1)}, 1, False),
-        ({'backfilling': ('tsla.nasdaq.ib', 1)}, 0, False),
-    ],
-)
-def test_only_new_sample_steps_advance_destination(
-    msg: SampleMsg,
-    step_diff: int,
-    expected: bool,
-
-) -> None:
-    '''
-    Keep queued history and duplicate wakeups from adding FSP rows.
-
-    A recomputation can consume several prepends before their sampler
-    markers are delivered. Those stale markers then observe aligned SHM
-    bounds and previously appended duplicate destination rows. Model the
-    queued marker and regular duplicate cases, and prove only a genuine
-    one-step source lead authorizes destination advancement.
-
-    '''
-    assert _should_advance_dst(msg, step_diff) is expected
-
-
-@pytest.mark.parametrize(
-    'msg,synced,expected',
-    [
-        ({'index': 1}, True, False),
-        ({'index': 1}, False, True),
-        ({'backfilling': ('tsla.nasdaq.ib', 1)}, True, True),
-        ({'backfilling': ('tsla.nasdaq.ib', 1)}, False, True),
-    ],
-)
-def test_backfill_always_invalidates_fsp_history(
-    msg: SampleMsg,
-    synced: bool,
-    expected: bool,
-
-) -> None:
-    '''
-    Recompute in-place history repairs with unchanged SHM bounds.
-
-    Gap repair can replace null source rows without moving either SHM
-    bound. A bounds-only predicate therefore reports synchronization
-    even though derived values are stale. Prove every relevant backfill
-    marker forces history replay while an aligned sample event does not.
-
-    '''
-    assert _needs_history_resync(msg, synced) is expected
-
-
-def test_history_compute_uses_immutable_source_snapshot() -> None:
-    '''
-    Do not publish output and timestamps from different source ranges.
-
-    A source append or prepend can occur while the FSP's first yield is
-    computing. The old path read timestamps only afterward and could pair
-    an N-row result with a shifted N-row timestamp tail. Mutate the live
-    last bound during the first yield, then prove the accepted output and
-    timestamps retain the original range while later reads switch to live
-    SHM.
-
-    '''
-    shm = Shm(10, 20, token='source')
-    live_lengths: list[int] = []
-
-    async def edge(
-        _source: AsyncIterator[FeedQuote],
-        src_shm: ShmArray,
-
-    ) -> AsyncIterator[np.ndarray]:
-        shm._last.value += 1
-        yield np.ones(len(src_shm.array))
-        live_lengths.append(len(src_shm.array))
-
-    async def main() -> None:
-        (
-            out_stream,
-            history,
-            bounds,
-            src_history,
-        ) = await _open_history_snapshot(
-            cast(EdgeFunc, edge),
-            quote_stream=cast(
-                AsyncIterator[QuoteFrame],
-                object(),
-            ),
-            fqme='tsla.nasdaq.ib',
-            src_shm=cast(ShmArray, shm),
-        )
-        assert bounds == (10, 20)
-        assert len(history) == 10
-        assert len(src_history) == 10
-        with pytest.raises(StopAsyncIteration):
-            await anext(out_stream)
-
-    trio.run(main)
-
-    assert live_lengths == [11]
-
-
-@pytest.mark.parametrize('target', [wma, rsi, tina_vwap])
-def test_builtin_fsp_stream_contract(target: Fsp) -> None:
-    '''
-    Keep every registered scalar FSP on the engine's yield protocol.
-
-    The momentum operators previously had incompatible call signatures,
-    short historical arrays, and bare realtime yields, all hidden by an
-    engine-side cast. Run each against one OHLCV snapshot and one trade,
-    proving the first yield is a source-aligned array and the next yield
-    is a named realtime field/value pair.
-
-    '''
-    shm = cast(ShmArray, OhlcvShm())
-
-    async def source() -> AsyncIterator[FeedQuote]:
-        yield {
-            'ticks': [{
-                'type': 'trade',
-                'price': 42.0,
-                'size': 1.0,
-            }],
-        }
-
-    async def main() -> None:
-        stream = target.func(source(), shm)
-        history = await anext(stream)
-        assert isinstance(history, np.ndarray)
-        assert len(history) == len(shm.array)
-
-        realtime = await anext(stream)
-        assert isinstance(realtime, tuple)
-        assert realtime[0] == target.name
-        await stream.aclose()
-
-    trio.run(main)
-
-
-def test_dolla_vlm_rejects_negative_trade_volume() -> None:
-    '''
-    Keep a provider volume reset from hiding the zoomed-out curve.
-
-    IB can emit a large negative trade size around a session closure.
-    The sampler previously stored that size as bar volume and the
-    dollar-volume history multiplied it by price. The resulting outlier
-    made `Viz.maxmin()` reject the entire downsampled range, leaving a
-    stale cached path or no visible curve. Seed one historical negative
-    volume and stream one negative trade tick, then prove both historical
-    and realtime dollar volume remain nonnegative.
-
-    '''
-    shm = OhlcvShm(length=4)
-    shm._array['volume'][:4] = [1, 2, -2_319_096, 3]
-
-    async def source() -> AsyncIterator[FeedQuote]:
-        yield {
-            'ticks': [{
-                'type': 'trade',
-                'price': 29_495.75,
-                'size': -2_319_096.,
-            }],
-        }
-
-    async def main() -> None:
-        stream = dolla_vlm.func(
-            source(),
-            cast(ShmArray, shm),
-        )
-        history = await anext(stream)
-        assert isinstance(history, dict)
-        values = history['dolla_vlm']
-        assert isinstance(values, np.ndarray)
-        assert values[2] == 0
-        assert np.all(values >= 0)
-
-        realtime = await anext(stream)
-        assert realtime == ('dolla_vlm', 0)
-        await stream.aclose()
-
-    trio.run(main)
-
-
-class Viz:
-    def __init__(self, name: str, shm: Shm) -> None:
-        self.name: str = name
-        self.shm: Shm = shm
-        self.updates: int = 0
-        self.force_redraws: list[bool] = []
-        self._last_fsp_update_sig: (
-            tuple[int, int, bytes]|None
-        ) = None
-        self._mxmns: dict[
-            tuple[int, int],
-            tuple[int, int],
-        ] = {(10, 20): (1, 2)}
-        self.view = SimpleNamespace(
-            rescale_count=0,
-        )
-
-        def rescale(
-            *,
-            do_linked_charts: bool,
-            do_overlay_scaling: bool,
-
-        ) -> None:
-            assert not do_linked_charts
-            assert do_overlay_scaling
-            self.view.rescale_count += 1
-
-        self.view.interact_graphics_cycle = rescale
-        self.plot: SimpleNamespace = SimpleNamespace(
-            getAxis=lambda name: SimpleNamespace(_stickies={}),
-            vb=self.view,
-        )
-
-    def update_graphics(
-        self,
-        force_redraw: bool = False,
-
-    ) -> None:
-        self.updates += 1
-        self.force_redraws.append(force_redraw)
-
-
-def test_fsp_chart_skips_unchanged_realtime_state() -> None:
-    '''
-    Avoid repainting an unchanged current FSP datum on every quote.
-
-    The display loop calls `update_fsp_chart()` for every realtime
-    quote, even when a given FSP destination has not advanced or
-    changed its current value. That rebuilt and painted the same last
-    step rectangle repeatedly. Exercise identical state, an in-place
-    value update, a new sample with the same value, and a forced
-    history repair. The update counts prove unchanged requests are
-    skipped while each legitimate invalidation still redraws. A final
-    stable-NaN pair proves scalar NaN inequality can not defeat the
-    guard.
-
-    '''
-    shm = FspShm()
-    viz = Viz('flow', shm)
-
-    update_fsp_chart(viz, 'flow', 'flow')
-    update_fsp_chart(viz, 'flow', 'flow')
-    assert viz.updates == 1
-
-    shm._array['flow'][shm._last.value - 1] = 42
-    update_fsp_chart(viz, 'flow', 'flow')
-    assert viz.updates == 2
-
-    shm._array['flow'][shm._last.value] = 42
-    shm._last.value += 1
-    update_fsp_chart(viz, 'flow', 'flow')
-    assert viz.updates == 3
-
-    update_fsp_chart(
-        viz,
-        'flow',
-        'flow',
-        force_redraw=True,
-    )
-    assert viz.updates == 4
-
-    shm._array['flow'][shm._last.value - 1] = np.nan
-    update_fsp_chart(viz, 'flow', 'flow')
-    update_fsp_chart(viz, 'flow', 'flow')
-    assert viz.updates == 5
-    assert viz.force_redraws == [
-        False,
-        False,
-        False,
-        True,
-        False,
-    ]
-
-
-def test_fsp_history_update_redraws_only_derived_vizs() -> None:
-    '''
-    Keep a source-history prepend from refreshing the whole chart.
-
-    FSP cascades recompute after each near-term backfill frame. The old
-    notification called the linked chart's full graphics cycle, which
-    redrew the primary market and every overlay and disturbed the live
-    view repeatedly. Arrange two visualizations sharing the rebuilt FSP
-    SHM plus unrelated source and overlay SHMs, then prove only the two
-    derived curves receive an update.
-
-    '''
-    fsp_shm = FspShm(10, 20, token='derived')
-    source_viz = Viz('source', Shm(10, 20, token='source'))
-    first_fsp_viz = Viz('flow', fsp_shm)
-    second_fsp_viz = Viz('dark_flow', fsp_shm)
-    overlay_viz = Viz('overlay', Shm(10, 20, token='overlay'))
-
-    linked: LinkedSplits = cast(
-        LinkedSplits,
-        SimpleNamespace(
-            chart=SimpleNamespace(
-                _vizs={
-                    'source': source_viz,
-                    'overlay': overlay_viz,
-                },
-            ),
-            subplots={
-                'volume': SimpleNamespace(
-                    _vizs={
-                        'flow': first_fsp_viz,
-                        'dark_flow': second_fsp_viz,
-                    },
-                ),
-            },
-        ),
-    )
-
-    updated = update_fsp_vizs(linked, fsp_shm._token)
-
-    assert updated == 2
-    assert first_fsp_viz.updates == 1
-    assert second_fsp_viz.updates == 1
-    assert first_fsp_viz.force_redraws == [True]
-    assert second_fsp_viz.force_redraws == [True]
-    assert first_fsp_viz._mxmns == {}
-    assert second_fsp_viz._mxmns == {}
-    assert first_fsp_viz.view.rescale_count == 1
-    assert second_fsp_viz.view.rescale_count == 1
-    assert source_viz.updates == 0
-    assert overlay_viz.updates == 0

tests/test_ib_broker.py

@@ -1,99 +0,0 @@
-'''
-IB broker event-relay regressions.
-
-'''
-from collections.abc import AsyncIterator
-from typing import Any
-
-import pytest
-import trio
-
-from piker.brokers.ib.broker import deliver_trade_events
-from piker.clearing import OrderDialogs
-from piker.clearing._messages import BrokerdError
-
-
-@pytest.mark.parametrize(
-    ('error', 'expected_reason'),
-    [
-        (
-            '[code 104] connection failed',
-            ' connection failed',
-        ),
-        (
-            '[Errno 111] Connect call failed',
-            ' Connect call failed',
-        ),
-        (
-            'plain API error',
-            'plain API error',
-        ),
-    ],
-)
-def test_trade_error_strings_are_relayed(
-    error: str,
-    expected_reason: str,
-) -> None:
-    '''
-    Raw IB API error strings must not kill the trade-event relay.
-
-    ``Client.inline_errors()`` can emit bracketed ``[code N]`` and
-    ``[Errno N]`` strings instead of structured error dictionaries.
-    The string branch called ``str.partition()`` without a separator,
-    raising ``TypeError`` before EMS received the error and
-    terminating all later trade-event delivery. Feed both documented
-    bracket forms and an unprefixed control through
-    ``deliver_trade_events()``. The exact emitted reason and unknown
-    request ID prove each event crossed the real relay path without
-    crashing or being misclassified.
-
-    '''
-    class FakeEmsStream:
-        '''
-        Collect messages sent to the EMS stream.
-
-        '''
-        def __init__(self) -> None:
-            '''
-            Initialize the captured message sequence.
-
-            '''
-            self.messages: list[Any] = []
-
-        async def send(self, msg: Any) -> None:
-            '''
-            Capture one relayed broker message.
-
-            '''
-            self.messages.append(msg)
-
-    async def events() -> AsyncIterator[tuple[str, str]]:
-        '''
-        Emit the raw IB error under test.
-
-        '''
-        yield 'error', error
-
-    async def main() -> list[Any]:
-        '''
-        Drive the broker relay to input exhaustion.
-
-        '''
-        ems_stream = FakeEmsStream()
-        await deliver_trade_events(
-            events(),
-            ems_stream,
-            accounts_def={},
-            proxies={},
-            ledgers={},
-            tables={},
-            flows=OrderDialogs(_flows={}),
-        )
-        return ems_stream.messages
-
-    messages = trio.run(main)
-    assert len(messages) == 1
-    msg = messages[0]
-    assert isinstance(msg, BrokerdError)
-    assert msg.reqid == '<unknown>'
-    assert msg.reason == expected_reason