Add `piker-clearing-expert` skill
Capture EMS/order-control ownership, identifier boundaries, ledger and position invariants, backend variation, and current known hazards. Store the canonical bundle under `.agents/skills/` and bridge Claude discovery through a portable relative link. Also, - add architecture, debugging, and test-map references; - document the provider-neutral source convention; - ignore the canonical `wkts/` runtime root. Prompt-IO: ai/prompt-io/opencode/20260828T021411Z_6ca6ab2a_prompt_io.md (this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))wkt/fsp_backfill_sync
parent
7b51927f6b
commit
1606122b45
|
|
@ -0,0 +1,174 @@
|
|||
---
|
||||
name: piker-clearing-expert
|
||||
description: >
|
||||
Piker clearing, EMS/order-control, accounting, position, and
|
||||
trade-ledger expertise. Apply when changing or debugging Order,
|
||||
Status, Brokerd* protocols, live or paper fills, dark orders,
|
||||
order IDs, TransactionLedger persistence, Account/Position/PPU,
|
||||
FQME/bs_mktid identity, SymbologyCache lookups, or broker trade
|
||||
normalization.
|
||||
compatibility: >
|
||||
Requires a piker checkout and familiarity with Python, trio,
|
||||
tractor, msgspec Structs, and backend-specific trading APIs.
|
||||
metadata:
|
||||
author: goodboy
|
||||
version: "1.0"
|
||||
---
|
||||
|
||||
# Piker Clearing Expert
|
||||
|
||||
Use this mental model before touching order control or accounting:
|
||||
|
||||
```text
|
||||
OrderClient / UI
|
||||
-> emsd routes intent and owns order-dialog state
|
||||
-> live brokerd OR paperboi executes and reports events
|
||||
-> executing backend owns fill-to-accounting integration
|
||||
-> emsd translates and broadcasts broker events
|
||||
-> UI consumes order state and position summaries
|
||||
```
|
||||
|
||||
The EMS is not the trade ledger. It correlates IDs, runs dark
|
||||
predicates, and relays lifecycle events. Paper and accounting-aware
|
||||
live backends may normalize fills into transactions, update local
|
||||
ledger/account state, and emit `BrokerdPosition` summaries. Inspect
|
||||
each live backend; this integration is not uniform.
|
||||
|
||||
## Runtime ownership
|
||||
|
||||
- `OrderClient` owns client intent and local sent-order tracking.
|
||||
- `emsd` owns `Router`, `DarkBook`, active `Status` dialogs,
|
||||
subscribers, and `oid <-> reqid` correlation.
|
||||
- `brokerd.<broker>` owns credentialed live order control.
|
||||
- `paperboi.<broker>` owns simulated execution and paper accounting.
|
||||
- `TransactionLedger` owns trade-record persistence, not order state.
|
||||
- `Account` owns positions keyed by backend-system market identity.
|
||||
- UI `Position.update_from_msg()` is a summary reset, not durable
|
||||
transaction accounting.
|
||||
|
||||
For daemon placement, context streams, cancellation, and actor-local
|
||||
state, also load `piker-conc-expert`.
|
||||
|
||||
## Event lifecycle
|
||||
|
||||
1. A client sends `Order` or `Cancel` through `OrderClient`.
|
||||
2. `process_client_order_cmds()` creates or updates EMS `Status`.
|
||||
3. EMS routes a `BrokerdOrder` or stores a dark trigger predicate.
|
||||
4. The executing backend acknowledges with `BrokerdOrderAck`.
|
||||
5. EMS records the current `oid <-> reqid` relation.
|
||||
6. Accepted, ack-correlated status/fill events become client updates.
|
||||
7. The backend integrates fills into accounting when supported.
|
||||
8. A resulting `BrokerdPosition` is relayed through EMS to clients.
|
||||
|
||||
Cancellation may arrive before acknowledgement. Preserve
|
||||
`Status.cancel_called`; once the ack supplies `reqid`, EMS can issue
|
||||
the deferred `BrokerdCancel`.
|
||||
|
||||
## Identifier namespaces
|
||||
|
||||
Never substitute one namespace merely because two values happen to
|
||||
match in a backend:
|
||||
|
||||
| ID | Owner and invariant |
|
||||
|---|---|
|
||||
| `oid` | Stable client/EMS order-dialog ID across submit, modify, cancel |
|
||||
| `reqid` | Backend order-control ID established by an ack; it may change |
|
||||
| venue order ID | Native API identity; may differ from both IDs above |
|
||||
| `tid` | Stable clear ID used to deduplicate position events and TOML keys |
|
||||
| `fqme` | Piker-normalized market address and symcache lookup key |
|
||||
| `bs_fqme` | FQME without its final broker suffix |
|
||||
| `bs_mktid` | Backend-system market key and `Account.pps` key |
|
||||
|
||||
Normalize every durably persisted `tid` to a stable string. Although
|
||||
`Transaction.tid` permits integers, TOML mapping keys do not.
|
||||
|
||||
Paper transactions commonly set `Transaction.bs_mktid` to the full
|
||||
FQME. The embedded `Position.mkt.bs_mktid` can still retain the
|
||||
provider-native market ID. Check the concrete object and table owner
|
||||
before asserting key equality.
|
||||
|
||||
## Accounting invariants
|
||||
|
||||
- `Transaction.size` is signed; preserve side semantics during
|
||||
broker normalization.
|
||||
- `Transaction` is the normalized in-memory interchange. Live ledger
|
||||
records may remain backend-native and normalize through
|
||||
`mod.norm_trade()` when read.
|
||||
- `Position._events` is keyed by `tid`; `add_clear()` is the
|
||||
idempotence boundary.
|
||||
- `Position.ppu` and `Position.cumsize` are derived from transaction
|
||||
events, not independent mutable truth.
|
||||
- Normalized accounting transactions need a stable, non-null
|
||||
`bs_mktid`; `Account.pps` uses it as the position key.
|
||||
- `Account.update_from_ledger()` resolves market metadata from
|
||||
`SymbologyCache.mktmaps` or its explicit fallback table.
|
||||
- UI position messages are projections. Do not feed their synthetic
|
||||
reset event back into durable broker accounting.
|
||||
|
||||
## Persistence contracts
|
||||
|
||||
Trade ledgers live under the accounting ledger directory as
|
||||
`trades_<broker>_<account>.toml`. Top-level keys are transaction IDs;
|
||||
record schemas are backend-specific unless the paper engine wrote a
|
||||
normalized transaction dictionary.
|
||||
|
||||
`open_trade_ledger()` writes on context exit when its data comparison
|
||||
detects a change or `rewrite=True`. Its snapshot is shallow, so nested
|
||||
in-place record mutation can evade dirty detection. Explicit writes can
|
||||
also fail inside fill handlers. Treat all persistence points as part of
|
||||
the live execution path.
|
||||
|
||||
Accounts persist active positions. `dump_active()` separates open and
|
||||
net-zero positions. Do not assume `minimized_clears()` safely preserves
|
||||
size/PPU: current sign-transition and longer net-zero slicing lacks
|
||||
round-trip coverage and can retain the wrong event subset.
|
||||
|
||||
## Market identity and symcache
|
||||
|
||||
`SymbologyCache` has three distinct key-addressed mappings:
|
||||
|
||||
- `assets`: provider asset ID to `Asset`;
|
||||
- `pairs`: native `bs_mktid` to backend pair `Struct`;
|
||||
- `mktmaps`: searchable FQME or explicit alias to normalized
|
||||
`MktPair`.
|
||||
|
||||
Search mapping keys, then resolve values. RapidFuzz treats mapping
|
||||
values as choices; passing `mktmaps` directly makes it operate on
|
||||
`MktPair` objects. Preserve in-memory aliases because IB can store both
|
||||
native IDs and FQMEs for one `MktPair`. Do not persist such aliases
|
||||
without fixing reload: `from_dict()` currently requires every key to
|
||||
equal `mkt.fqme`.
|
||||
|
||||
## Change workflow
|
||||
|
||||
Before editing:
|
||||
|
||||
1. Identify the owning layer: client, EMS, executor, accounting, or UI.
|
||||
2. Write down every ID/key transition across that boundary.
|
||||
3. Trace both normal and cancel-before-ack event ordering.
|
||||
4. Determine whether records are native, normalized, or summaries.
|
||||
5. Check actor-local caches and context-exit writes.
|
||||
6. Add a deterministic regression at the narrowest broken contract.
|
||||
7. Run live/backend tests only when their credentials and side effects
|
||||
are explicitly authorized.
|
||||
|
||||
Do not fix an accounting defect by adding state to EMS, or fix an EMS
|
||||
correlation defect by rewriting durable transaction history.
|
||||
|
||||
## Canonical source map
|
||||
|
||||
- `piker/clearing/_messages.py`: client and brokerd protocol types.
|
||||
- `piker/clearing/_client.py`: `OrderClient` and `open_ems()`.
|
||||
- `piker/clearing/_ems.py`: router, dark book, ID mapping, translation.
|
||||
- `piker/clearing/_paper_engine.py`: simulated execution/accounting.
|
||||
- `piker/accounting/_ledger.py`: transactions and ledger persistence.
|
||||
- `piker/accounting/_pos.py`: positions, accounts, PPU event state.
|
||||
- `piker/accounting/_mktinfo.py`: `Asset`, `MktPair`, FQME schema.
|
||||
- `piker/data/_symcache.py`: provider symbology mappings and search.
|
||||
- `piker/accounting/calc.py`: transaction ordering and PPU/PnL math.
|
||||
- `piker/brokers/*/broker.py`: backend order-control integration.
|
||||
- `piker/brokers/*/ledger.py`: backend trade normalization.
|
||||
|
||||
See [architecture.md](architecture.md) for complete flows,
|
||||
[gotchas.md](gotchas.md) for symptom-driven diagnosis, and
|
||||
[test-map.md](test-map.md) for verification choices.
|
||||
|
|
@ -0,0 +1,210 @@
|
|||
# Clearing And Accounting Architecture
|
||||
|
||||
## Protocol direction
|
||||
|
||||
Client-facing messages and broker-facing messages intentionally differ.
|
||||
|
||||
```text
|
||||
client -> EMS
|
||||
Order, Cancel
|
||||
|
||||
EMS -> executor
|
||||
BrokerdOrder, BrokerdCancel
|
||||
|
||||
executor -> EMS
|
||||
BrokerdOrderAck, BrokerdStatus, BrokerdFill,
|
||||
BrokerdError, BrokerdPosition
|
||||
|
||||
EMS -> subscribed clients
|
||||
Status, BrokerdPosition
|
||||
```
|
||||
|
||||
`Order` carries the client `oid`, account, market, price, size, action,
|
||||
and execution mode. `BrokerdOrderAck` establishes the executor's
|
||||
`reqid`; subsequent broker messages should carry that ID so EMS can
|
||||
resolve the client dialog.
|
||||
|
||||
`Status` is mutable current dialog state, not a history envelope. Exact
|
||||
client states include `pending`, `open`, `dark_open`, `triggered`,
|
||||
`fill`, `closed`, `canceled`, and `error`. Audit existing translation
|
||||
before adding a state; backend `BrokerdStatus` vocabulary is narrower
|
||||
and current producers are not fully conformant.
|
||||
|
||||
## EMS routing
|
||||
|
||||
`_emsd_main()` serves each client/market context. The actor-local
|
||||
`Router` owns books, subscribers, order dialogs, and cached live trades
|
||||
relays. Paper mode opens per-symbol relay contexts under the same broker
|
||||
key, a current overwrite/pop hazard for concurrent paper symbols.
|
||||
|
||||
`open_brokerd_dialog()` is the session live-versus-paper selection
|
||||
point:
|
||||
|
||||
- live execution talks to `brokerd.<broker>`;
|
||||
- paper execution talks to `paperboi.<broker>`;
|
||||
- chart-only and paper sessions should not boot credentialed brokerd.
|
||||
|
||||
Do not infer paper execution from each order's `exec_mode`. Session
|
||||
mode selects the executor; per-order mode primarily distinguishes dark
|
||||
from immediately routed orders, despite incomplete literal annotations.
|
||||
|
||||
`process_client_order_cmds()` handles submit, modify, cancel, dark,
|
||||
and alert intent. It updates EMS dialog state before routing broker
|
||||
commands. `translate_and_relay_brokerd_events()` applies acks, statuses,
|
||||
fills, errors, and positions to dialogs and subscribers.
|
||||
|
||||
The `DarkBook` stores local predicates and watches quote ticks. When a
|
||||
predicate triggers, `clear_dark_triggers()` emits the live broker order
|
||||
and updates the existing client dialog rather than inventing a second
|
||||
order identity.
|
||||
|
||||
## Ack and cancel ordering
|
||||
|
||||
Required actionable ordering is submit, ack, open, zero or more fills,
|
||||
then terminal status. EMS handles cancel-before-ack, but it does not
|
||||
buffer arbitrary status-before-ack events; those currently fall through
|
||||
as unhandled. Backends must ack before actionable statuses.
|
||||
|
||||
The important race is:
|
||||
|
||||
```text
|
||||
client Cancel(oid)
|
||||
-> EMS has no reqid yet
|
||||
-> mark Status.cancel_called
|
||||
BrokerdOrderAck(oid, reqid)
|
||||
-> install mapping
|
||||
-> send deferred BrokerdCancel(reqid)
|
||||
```
|
||||
|
||||
Do not drop a pre-ack cancel and do not send a broker cancel with an
|
||||
`oid` unless the backend explicitly uses that value as its `reqid`.
|
||||
|
||||
## Paper execution
|
||||
|
||||
`PaperBoi` maintains simulated orders, opens the market feed, and fills
|
||||
orders when L1 ask/bid quotes cross their prices. `trade`/`last` ticks
|
||||
zip buy and sell iterators, so a lone resting side currently may not
|
||||
fill from those tick types. `fake_fill()` is the core fill-to-account
|
||||
path:
|
||||
|
||||
1. emit `BrokerdFill`;
|
||||
2. emit terminal `BrokerdStatus` for the simulated order;
|
||||
3. construct a signed normalized `Transaction`;
|
||||
4. `TransactionLedger.update_from_t()`;
|
||||
5. `Account.update_from_ledger()`;
|
||||
6. persist the ledger and account;
|
||||
7. emit `BrokerdPosition` with current size and PPU.
|
||||
|
||||
The protocol notifications and persistence occur in one task but are
|
||||
not a database transaction. If persistence fails after fill/status
|
||||
publication, restart/reconciliation behavior matters. Do not reorder
|
||||
the sequence casually.
|
||||
|
||||
Partial fills currently reuse `oid` as `tid`. A later partial therefore
|
||||
overwrites the paper ledger record and is rejected by position-event
|
||||
dedupe. Fix unique fill IDs before relying on paper partial-fill math.
|
||||
|
||||
`open_trade_dialog()` opens symcache, paper ledger, account, and feed
|
||||
resources under structured concurrency. An exception from any nested
|
||||
resource, including ledger context exit, cancels the dialog and its feed
|
||||
stream.
|
||||
|
||||
## Live backend variation
|
||||
|
||||
There is no universal live-broker accounting implementation:
|
||||
|
||||
- IB normalizes execution/commission events, updates ledger/account
|
||||
state, and emits position updates.
|
||||
- Kraken has backend trade normalization and position reconstruction,
|
||||
but its live path writes account state without inserting every new
|
||||
trade into `TransactionLedger`.
|
||||
- Binance currently relays venue order/position events and does not
|
||||
share every local `TransactionLedger`/`Account` path.
|
||||
|
||||
Inspect the backend's `broker.py`, `ledger.py`, and `norm_trade()` before
|
||||
moving generic logic. A paper behavior is not automatically a live
|
||||
backend contract.
|
||||
|
||||
## Ledger normalization
|
||||
|
||||
`TransactionLedger.data` is the durable record mapping. For paper it
|
||||
contains transaction-like dictionaries. For live brokers it can contain
|
||||
raw provider records.
|
||||
|
||||
`iter_txns()` sorts records and chooses normalization:
|
||||
|
||||
- paper uses `_paper_engine.norm_trade()`;
|
||||
- live accounts use `brokermod.norm_trade()`.
|
||||
|
||||
Normalizer output should carry datetime, signed size, price, cost,
|
||||
FQME, backend market ID, and transaction ID. Some normalizers return
|
||||
`None` for unusable records; `iter_txns()` does not filter it, while
|
||||
`to_txns()` does. Audit direct iterator consumers.
|
||||
|
||||
`write_config()` sorts records for stable persistence. Paper rewrite
|
||||
logic migrates old symbol fields and qualifies legacy FQMEs through the
|
||||
symcache. Search can return no matches; callers which select the first
|
||||
result need an explicit fallback or a descriptive failure policy.
|
||||
|
||||
## Position event model
|
||||
|
||||
`Position` holds a market and a transaction-event mapping. New clears
|
||||
are deduplicated by `tid`. Current size and PPU are recomputed from the
|
||||
event sequence.
|
||||
|
||||
`Account.update_from_ledger()`:
|
||||
|
||||
1. receives normalized transactions;
|
||||
2. resolves each market through `mktmaps` or a fallback table;
|
||||
3. looks up or creates the position by `Transaction.bs_mktid`;
|
||||
4. adds unseen clears;
|
||||
5. returns visited positions, including duplicates whose clear was
|
||||
rejected by `add_clear()`.
|
||||
|
||||
`minimized_clears()` intends to discard irrelevant history after
|
||||
net-zero or sign transitions, but current slicing is not proven safe
|
||||
and can reconstruct the wrong active size/PPU. Treat it as a known bug
|
||||
surface until sign-transition round trips are covered.
|
||||
|
||||
`Position.update_from_msg()` serves UI summaries by replacing local
|
||||
event state with a synthetic clear representing reported size and PPU.
|
||||
That projection is intentionally lossy.
|
||||
|
||||
## Symbology contracts
|
||||
|
||||
`MktPair.fqme` is a normalized endpoint address assembled from pair,
|
||||
venue, expiry/contract information, and broker. `MktPair.bs_fqme`
|
||||
drops the broker suffix. `MktPair.bs_mktid` is the provider-native
|
||||
market identity.
|
||||
|
||||
`SymbologyCache.load()` attempts to obtain provider assets and pair
|
||||
structs, calls `get_mkt_info()`, and populates conventional mappings.
|
||||
Assets are optional; missing/empty pair support returns early.
|
||||
|
||||
```text
|
||||
assets[provider_asset_id] -> Asset
|
||||
pairs[pair.bs_mktid] -> backend Pair Struct
|
||||
mktmaps[mkt.fqme] -> MktPair
|
||||
```
|
||||
|
||||
Some backends add aliases such as `mktmaps[bs_mktid]`. Search must
|
||||
return the matched mapping key rather than rebuilding a key from the
|
||||
value, otherwise aliases silently collapse. Current serialization then
|
||||
writes aliases which reload rejects because `from_dict()` asserts each
|
||||
key equals `mkt.fqme`.
|
||||
|
||||
## Persistence boundaries
|
||||
|
||||
`open_trade_ledger()` loads account TOML and compares a shallow snapshot
|
||||
in `finally`. Top-level additions/replacements trigger writes; nested
|
||||
in-place mutation may not. `open_account()` similarly mediates account
|
||||
state. These contexts run inside actor tasks, so serialization errors
|
||||
are runtime failures, not merely offline maintenance failures.
|
||||
|
||||
When changing schemas:
|
||||
|
||||
- preserve backend-native records unless migration is explicit;
|
||||
- keep old paper field migration deterministic;
|
||||
- avoid fuzzy qualification when an exact `bs_mktid`/FQME exists;
|
||||
- test context exit, not only in-memory updates;
|
||||
- inspect emitted positions after disk round trips.
|
||||
|
|
@ -0,0 +1,146 @@
|
|||
# Clearing And Ledger Gotchas
|
||||
|
||||
## Trade dialog dies immediately after a fill
|
||||
|
||||
Likely cause: an explicit ledger/account write in the fill handler, or
|
||||
a later context-exit write, raised after event publication.
|
||||
|
||||
Inspect, in order:
|
||||
|
||||
1. the first non-cancellation exception in the actor traceback;
|
||||
2. `PaperBoi.fake_fill()` or the live backend fill handler;
|
||||
3. explicit and context-exit `write_config()` calls;
|
||||
4. FQME/`bs_mktid` qualification and TOML-compatible values;
|
||||
5. the persisted ledger/account files for a partial update.
|
||||
|
||||
Tractor cancellation logs are often the consequence. Root-cause the
|
||||
inner serialization, lookup, or normalization exception first.
|
||||
|
||||
## `MktPair` has no `len()` in RapidFuzz
|
||||
|
||||
Cause: a mapping of string keys to `MktPair` values was passed directly
|
||||
to `rapidfuzz.process.extract()`. Mapping values become fuzzy choices.
|
||||
|
||||
Fix: search a list/view of string keys and resolve matched keys back
|
||||
through the original mapping. Do not add sequence methods to `MktPair`
|
||||
and do not use `processor=str` when aliases must be preserved.
|
||||
|
||||
## Fuzzy qualification raises `IndexError`
|
||||
|
||||
Cause: the caller selected the first match from an empty result after
|
||||
the score cutoff rejected every key.
|
||||
|
||||
Decide explicitly whether to retain the input identity, reject the
|
||||
record with a descriptive exception, or obtain missing market metadata.
|
||||
Do not lower the cutoff blindly; the wrong market corrupts accounting.
|
||||
|
||||
## Cancel is ignored or sent with the wrong ID
|
||||
|
||||
Cause: confusion between client `oid` and backend `reqid`, commonly
|
||||
during cancel-before-ack ordering.
|
||||
|
||||
Verify the ack installed both mapping directions and that
|
||||
`Status.cancel_called` survives until `reqid` exists. Log both IDs and
|
||||
the actor/backend account at every transition.
|
||||
|
||||
## Fills appear but positions do not move
|
||||
|
||||
Check these boundaries:
|
||||
|
||||
- the backend emitted `BrokerdFill` but never normalized a transaction;
|
||||
- transaction size lost its buy/sell sign;
|
||||
- `tid` collided with an existing position event;
|
||||
- `Transaction.bs_mktid` selected a different `Account.pps` key;
|
||||
- `mktmaps` lacked the exact transaction identity;
|
||||
- position publication happened before accounting update;
|
||||
- the backend intentionally reports venue positions instead of local
|
||||
ledger-derived positions.
|
||||
|
||||
## Duplicate or inflated position size
|
||||
|
||||
Do not deduplicate by price/time alone. Confirm stable transaction IDs,
|
||||
chronological sorting, and `Position.add_clear()` idempotence. Broker
|
||||
execution IDs and order IDs are not interchangeable.
|
||||
|
||||
For partial fills, several `tid`s may correctly share one order
|
||||
`reqid`. Collapsing them loses clears; replaying one `tid` twice
|
||||
inflates nothing only if the dedupe boundary is preserved.
|
||||
|
||||
Current paper fills use `tid=oid`, so later partials overwrite the
|
||||
ledger record and are deduplicated out of the position. Treat paper
|
||||
partial fills as broken until each clear receives a stable unique ID.
|
||||
|
||||
## Paper and live state disagree
|
||||
|
||||
Paper uses normalized local transactions and can use the full FQME as
|
||||
its account position key. A live backend may use native IDs, raw ledger
|
||||
records, or venue-reported positions.
|
||||
|
||||
Compare normalized `Transaction`s and emitted `BrokerdPosition`s, not
|
||||
raw TOML dictionaries across backends.
|
||||
|
||||
## Account appears under the wrong name
|
||||
|
||||
Separate UI/EMS account aliases from backend account IDs and ledger file
|
||||
names. Backends may prefix account names for routing while persistence
|
||||
uses an unprefixed native account key.
|
||||
|
||||
Trace account identity through `open_ems()`, broker dialog startup,
|
||||
ledger opening, and `BrokerdPosition.account` before changing naming.
|
||||
|
||||
## A module-global cache is unexpectedly empty
|
||||
|
||||
Symcache, clients, contracts, and order tables are actor-local. Data
|
||||
loaded in `datad` is not automatically available in `brokerd` or
|
||||
`paperboi`. Also, `_symcache._caches` is currently read but never
|
||||
populated, so do not assume repeated `open_symcache()` calls hit it.
|
||||
|
||||
Load `piker-conc-expert` and audit actor-local globals, async caches,
|
||||
and dialog startup warming. Never rely on an import side effect from a
|
||||
sibling actor.
|
||||
|
||||
## Teardown emits a cancellation storm
|
||||
|
||||
Find the first application exception. Structured cancellation then
|
||||
closes feed streams, broker streams, and nested contexts. Avoid masking
|
||||
the originating exception with broad cancellation handling.
|
||||
|
||||
For stream ownership and context cancellation semantics, use
|
||||
`piker-conc-expert`.
|
||||
|
||||
## Broker status is logged as unhandled
|
||||
|
||||
EMS currently accepts only ack-correlated `open`, `closed`, and
|
||||
`canceled` broker statuses. Declared `pending` and backend-produced
|
||||
`fill`/`filled` variants fall through, as do statuses arriving before
|
||||
ack. Verify producer vocabulary and ack ordering before adding another
|
||||
consumer branch.
|
||||
|
||||
## Another client sees order or position events
|
||||
|
||||
Current translation broadcasts order updates by FQME and positions to
|
||||
all attached clients; `Router.dialogs` is not used as an event privacy
|
||||
filter. Treat subscriber isolation as incomplete, not guaranteed.
|
||||
|
||||
## Debug breadcrumb
|
||||
|
||||
Capture one row per event with:
|
||||
|
||||
```text
|
||||
actor, backend, account, fqme, bs_mktid,
|
||||
oid, reqid, venue_order_id, tid,
|
||||
message_type, status, action, signed_size, price, timestamp
|
||||
```
|
||||
|
||||
Then compare:
|
||||
|
||||
1. client command order;
|
||||
2. EMS status history;
|
||||
3. broker event order;
|
||||
4. normalized transactions;
|
||||
5. position event IDs and computed size/PPU;
|
||||
6. ledger/account disk state;
|
||||
7. emitted position summaries.
|
||||
|
||||
This separates protocol loss, identity skew, accounting math, and
|
||||
persistence failure without guessing from UI state.
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
# Clearing And Accounting Test Map
|
||||
|
||||
## Deterministic first-pass tests
|
||||
|
||||
Use the narrowest test that owns the broken contract:
|
||||
|
||||
| Area | First target |
|
||||
|---|---|
|
||||
| Ledger/account persistence | targeted node in `tests/test_accounting.py` |
|
||||
| EMS bad-backend handling | `tests/test_ems.py::test_ems_err_on_bad_broker` |
|
||||
| IB request/response routing | `tests/test_ib_method_proxy.py` |
|
||||
| IB history normalization | `tests/test_ib_history.py` |
|
||||
| Service/actor startup | targeted node in `tests/test_services.py` |
|
||||
|
||||
Known missing high-value regressions include paper partial-fill IDs,
|
||||
status-before-ack handling, `minimized_clears()` sign-transition round
|
||||
trips, nested ledger dirty detection, and symcache alias reload.
|
||||
|
||||
For `SymbologyCache.search()` and paper ledger qualification, cover:
|
||||
|
||||
- real `Asset` and `MktPair` values;
|
||||
- a native-ID alias and canonical FQME key;
|
||||
- the actual `TransactionLedger.write_config()` path;
|
||||
- persisted TOML fields after the round trip;
|
||||
- no use of live APIs or user configuration.
|
||||
|
||||
## Side-effectful tests
|
||||
|
||||
Do not run these without explicit authorization and the repository test
|
||||
harness guidance:
|
||||
|
||||
- paper EMS tests which open live market/symbology feeds;
|
||||
- live Binance/Kraken feed suites;
|
||||
- IB account tests using configured accounts;
|
||||
- tests which write tracked ledger/account fixtures;
|
||||
- broker dialogs requiring credentials or venue connectivity.
|
||||
|
||||
After any accounting test using `tests/_inputs`, inspect its diff. Some
|
||||
context managers can rewrite fixture ledgers on exit.
|
||||
|
||||
## Regression design
|
||||
|
||||
Prefer a direct contract regression over a full actor test when the
|
||||
failure is synchronous and deterministic. Use an actor-level test when
|
||||
the bug depends on:
|
||||
|
||||
- ack/cancel or fill/status interleaving;
|
||||
- context cancellation or stream ownership;
|
||||
- actor-local state/cache separation;
|
||||
- daemon selection or startup;
|
||||
- publication ordering across EMS and broker streams.
|
||||
|
||||
Every regression should document the original failure, triggering
|
||||
state, violated invariant, arrangement, and proof of the fix.
|
||||
|
||||
## Verification layers
|
||||
|
||||
1. `git diff --cached --check` for the exact staged boundary.
|
||||
2. Ruff with the repository's intended rule profile.
|
||||
3. A deterministic unit/filesystem regression.
|
||||
4. Adjacent accounting or EMS tests which do not require live state.
|
||||
5. Explicitly authorized backend/actor integration tests.
|
||||
|
||||
Do not interpret a passing ellipsis-body or hard-skipped test as
|
||||
behavioral coverage. Read the selected node before using it as a gate.
|
||||
|
||||
## Manual reconciliation check
|
||||
|
||||
For production-only failures, compare one fill end to end:
|
||||
|
||||
```text
|
||||
BrokerdFill
|
||||
-> normalized Transaction
|
||||
-> ledger record
|
||||
-> Position._events[tid]
|
||||
-> Position.cumsize / ppu
|
||||
-> account TOML
|
||||
-> BrokerdPosition
|
||||
-> UI summary
|
||||
```
|
||||
|
||||
All identities, sign, price, and account values should remain explainable
|
||||
at each arrow. A mismatch identifies the owning boundary for the next
|
||||
focused test.
|
||||
|
|
@ -0,0 +1 @@
|
|||
../../.agents/skills/piker-clearing-expert
|
||||
|
|
@ -211,6 +211,7 @@ docs/conversations/
|
|||
# BEGIN ai.skillz: runtime:open-wkt
|
||||
.claude/wkts/
|
||||
claude_wkts
|
||||
/wkts/
|
||||
# END ai.skillz: runtime:open-wkt
|
||||
|
||||
# BEGIN ai.skillz: direct:symlink:claude:open-wkt
|
||||
|
|
|
|||
13
ai/README.md
13
ai/README.md
|
|
@ -21,11 +21,14 @@ track new integration ideas and proposals in
|
|||
|
||||
## Shared Skills
|
||||
|
||||
Repo-specific skills use the portable Agent Skills
|
||||
frontmatter subset and live under `.claude/skills/` as a
|
||||
single source. Claude Code discovers that directory
|
||||
natively, and OpenCode discovers the same project skills
|
||||
without copies or generated wrappers.
|
||||
Provider-neutral skill bundles should use the portable Agent Skills
|
||||
subset and live under `.agents/skills/` as their single source.
|
||||
OpenCode discovers that project directory natively. Relative links in
|
||||
provider directories such as `.claude/skills/` may bridge clients which
|
||||
do not yet scan `.agents/skills/`; never copy the skill body.
|
||||
|
||||
Existing repo skills under `.claude/skills/` predate this convention
|
||||
and can migrate separately without coupling their history to new skills.
|
||||
|
||||
Harness-specific command files should only delegate to a
|
||||
shared skill. They must not duplicate the skill body.
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ integration for piker's shared coding-harness skills.
|
|||
| Skill | Invocable | Description |
|
||||
|-------|-----------|-------------|
|
||||
| [`commit-msg`](#commit-msg) | `/commit-msg` | Generate piker-style commit messages |
|
||||
| `piker-clearing-expert` | auto | Clearing, order-control, and accounting invariants |
|
||||
| `piker-conc-expert` | auto | Actor-tree and structured-concurrency invariants |
|
||||
| `piker-profiling` | auto | `Profiler` API patterns for perf work |
|
||||
| `piker-slang` | auto | Communication style + slang guide |
|
||||
|
|
@ -19,10 +20,12 @@ Skills marked **auto** are background knowledge
|
|||
applied automatically when Claude detects relevance.
|
||||
Only `commit-msg` is user-invoked via slash command.
|
||||
|
||||
Portable skill source files live under
|
||||
`.claude/skills/<skill-name>/SKILL.md` and are shared with
|
||||
OpenCode. Claude-specific behavior belongs in command or
|
||||
settings files, not the shared skill bodies.
|
||||
New portable skill source files live under
|
||||
`.agents/skills/<skill-name>/SKILL.md`. Relative links under
|
||||
`.claude/skills/` expose them to Claude Code without copying bodies.
|
||||
Existing pre-migration skills can remain under `.claude/skills/`.
|
||||
Claude-specific behavior belongs in command or settings files, not
|
||||
shared skill bodies.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,14 +1,16 @@
|
|||
# OpenCode Integration
|
||||
|
||||
[OpenCode](https://opencode.ai/) uses piker's shared
|
||||
repo-specific skills directly from `.claude/skills/`.
|
||||
There is no second copy of each skill to keep in sync.
|
||||
[OpenCode](https://opencode.ai/) discovers provider-neutral skills
|
||||
directly from `.agents/skills/` and pre-migration piker skills from
|
||||
`.claude/skills/`. There is no second copy of each skill to keep in
|
||||
sync.
|
||||
|
||||
## Available Skills
|
||||
|
||||
| Skill | Activation | Specialization |
|
||||
|-------|------------|----------------|
|
||||
| `commit-msg` | `/commit-msg` or explicit request | Piker commit-message style and artifacts |
|
||||
| `piker-clearing-expert` | automatic | Clearing, order-control, and accounting invariants |
|
||||
| `piker-conc-expert` | automatic | `tractor` actor topology, RPC, and cancellation |
|
||||
| `piker-profiling` | automatic | Cross-actor `Profiler` instrumentation |
|
||||
| `piker-slang` | automatic | Project communication style |
|
||||
|
|
@ -49,3 +51,5 @@ hot-reloaded into an active session.
|
|||
hardcode Claude Code attribution in shared workflows.
|
||||
- Keep supporting references beside each shared skill so
|
||||
relative links resolve in every harness.
|
||||
- Put new provider-neutral bundles under `.agents/skills/`; use
|
||||
relative provider links only when a harness requires one.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: opencode-pid-469264
|
||||
timestamp: 2026-08-28T02:14:11Z
|
||||
git_ref: 6ca6ab2a
|
||||
scope: config
|
||||
substantive: true
|
||||
raw_file: 20260828T021411Z_6ca6ab2a_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Move the fix out of the root checkout into an isolated worktree and
|
||||
include a Piker-specific clearing, accounting, order-control, and
|
||||
trade-ledger expert skill in the worktree's commit plan.
|
||||
Use a cross-harness, AI-provider-neutral standard location for its
|
||||
canonical files instead of making one provider directory authoritative.
|
||||
|
||||
## Response summary
|
||||
|
||||
Created a portable `piker-clearing-expert` skill bundle under the
|
||||
provider-neutral `.agents/skills/` convention, with a relative Claude
|
||||
compatibility link. It covers subsystem ownership, protocols,
|
||||
identities, persistence, backend variation, debugging hazards, and
|
||||
test strategy. Added the canonical `wkts/` runtime ignore.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `.gitignore` - ignores the canonical root-level worktree directory.
|
||||
- `.agents/skills/piker-clearing-expert/SKILL.md` - core expertise.
|
||||
- `.agents/skills/piker-clearing-expert/architecture.md` - deep flows.
|
||||
- `.agents/skills/piker-clearing-expert/gotchas.md` - diagnosis.
|
||||
- `.agents/skills/piker-clearing-expert/test-map.md` - test map.
|
||||
- `.claude/skills/piker-clearing-expert` - relative Claude bridge.
|
||||
- `ai/README.md` - defines the provider-neutral source convention.
|
||||
- `ai/claude-code/README.md` - advertises the shared skill to Claude.
|
||||
- `ai/opencode/README.md` - advertises the shared skill to OpenCode.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human rejected planning in the dirty root checkout, required a
|
||||
dedicated worktree, expanded the work to include a durable expert
|
||||
skill, and then directed its canonical source into a cross-provider
|
||||
standard location. The agent applied those directed scope and layout
|
||||
changes; no direct manual source edit was identified.
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-28T02:14:11Z
|
||||
git_ref: 6ca6ab2a
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Move the symcache/accounting fix into an isolated worktree and, as
|
||||
part of its commit plan, distill a Piker-specific skill for deep
|
||||
clearing, accounting, order-control, and trade-ledger expertise.
|
||||
A follow-up required the skill's canonical files to begin moving into
|
||||
a cross-harness, AI-provider-neutral standard location.
|
||||
|
||||
## Response
|
||||
|
||||
Created the shared repository skill bundle under the provider-neutral
|
||||
`.agents/skills/piker-clearing-expert/` location. Its main skill defines
|
||||
activation cues,
|
||||
runtime ownership, event and identifier contracts, accounting and
|
||||
persistence invariants, market identity rules, an editing workflow,
|
||||
and a canonical source map.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .agents/skills/piker-clearing-expert/SKILL.md`
|
||||
|
||||
Added a detailed architecture reference for EMS routing, ack/cancel
|
||||
ordering, paper and live execution, ledger normalization, position
|
||||
events, symbology, and context-managed persistence.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .agents/skills/piker-clearing-expert/architecture.md`
|
||||
|
||||
Added symptom-driven gotchas and debugging breadcrumbs for actor
|
||||
teardown, ID skew, position reconciliation, duplicate fills,
|
||||
paper/live differences, and RapidFuzz mapping misuse.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .agents/skills/piker-clearing-expert/gotchas.md`
|
||||
|
||||
Added a deterministic-to-live test map and regression-design guidance.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .agents/skills/piker-clearing-expert/test-map.md`
|
||||
|
||||
Updated the worktree runtime ignore block to include the canonical
|
||||
root-level `wkts/` directory required by the worktree lifecycle.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .gitignore`
|
||||
|
||||
Added a relative Claude compatibility link and documented the
|
||||
provider-neutral source convention in the shared and harness-specific
|
||||
inventories.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .claude/skills/piker-clearing-expert`
|
||||
|
||||
> `git diff HEAD~1..HEAD -- ai/README.md`
|
||||
|
||||
> `git diff HEAD~1..HEAD -- ai/claude-code/README.md`
|
||||
|
||||
> `git diff HEAD~1..HEAD -- ai/opencode/README.md`
|
||||
|
||||
The skill cross-links `piker-conc-expert` for actor-runtime mechanics
|
||||
instead of duplicating structured-concurrency guidance.
|
||||
Loading…
Reference in New Issue