piker/.agents/skills/piker-clearing-expert/SKILL.md

Piker Clearing Expert

Use this mental model before touching order control or accounting:

OrderClient / UI
  -> emsd routes intent and owns order-dialog state
  -> live brokerd OR paperboi executes and reports events
  -> executing backend owns fill-to-accounting integration
  -> emsd translates and broadcasts broker events
  -> UI consumes order state and position summaries

The EMS is not the trade ledger. It correlates IDs, runs dark predicates, and relays lifecycle events. Paper and accounting-aware live backends may normalize fills into transactions, update local ledger/account state, and emit BrokerdPosition summaries. Inspect each live backend; this integration is not uniform.

Runtime ownership

• OrderClient owns client intent and local sent-order tracking.
• emsd owns Router, DarkBook, active Status dialogs, subscribers, and oid <-> reqid correlation.
• brokerd.<broker> owns credentialed live order control.
• paperboi.<broker> owns simulated execution and paper accounting.
• TransactionLedger owns trade-record persistence, not order state.
• Account owns positions keyed by backend-system market identity.
• UI Position.update_from_msg() is a summary reset, not durable transaction accounting.

For daemon placement, context streams, cancellation, and actor-local state, also load piker-conc-expert.

Event lifecycle

1. A client sends Order or Cancel through OrderClient.
2. process_client_order_cmds() creates or updates EMS Status.
3. EMS routes a BrokerdOrder or stores a dark trigger predicate.
4. The executing backend acknowledges with BrokerdOrderAck.
5. EMS records the current oid <-> reqid relation.
6. Accepted, ack-correlated status/fill events become client updates.
7. The backend integrates fills into accounting when supported.
8. A resulting BrokerdPosition is relayed through EMS to clients.

Cancellation may arrive before acknowledgement. Preserve Status.cancel_called; once the ack supplies reqid, EMS can issue the deferred BrokerdCancel.

Identifier namespaces

Never substitute one namespace merely because two values happen to match in a backend:

ID	Owner and invariant
oid	Stable client/EMS order-dialog ID across submit, modify, cancel
reqid	Backend order-control ID established by an ack; it may change
venue order ID	Native API identity; may differ from both IDs above
tid	Stable clear ID used to deduplicate position events and TOML keys
fqme	Piker-normalized market address and symcache lookup key
bs_fqme	FQME without its final broker suffix
bs_mktid	Backend-system market key and Account.pps key

Normalize every durably persisted tid to a stable string. Although Transaction.tid permits integers, TOML mapping keys do not.

Paper transactions commonly set Transaction.bs_mktid to the full FQME. The embedded Position.mkt.bs_mktid can still retain the provider-native market ID. Check the concrete object and table owner before asserting key equality.

Accounting invariants

• Transaction.size is signed; preserve side semantics during broker normalization.
• Transaction is the normalized in-memory interchange. Live ledger records may remain backend-native and normalize through mod.norm_trade() when read.
• Position._events is keyed by tid; add_clear() is the idempotence boundary.
• Position.ppu and Position.cumsize are derived from transaction events, not independent mutable truth.
• Normalized accounting transactions need a stable, non-null bs_mktid; Account.pps uses it as the position key.
• Account.update_from_ledger() resolves market metadata from SymbologyCache.mktmaps or its explicit fallback table.
• UI position messages are projections. Do not feed their synthetic reset event back into durable broker accounting.

Persistence contracts

Trade ledgers live under the accounting ledger directory as trades_<broker>_<account>.toml. Top-level keys are transaction IDs; record schemas are backend-specific unless the paper engine wrote a normalized transaction dictionary.

open_trade_ledger() writes on context exit when its data comparison detects a change or rewrite=True. Its snapshot is shallow, so nested in-place record mutation can evade dirty detection. Explicit writes can also fail inside fill handlers. Treat all persistence points as part of the live execution path.

Accounts persist active positions. dump_active() separates open and net-zero positions. Do not assume minimized_clears() safely preserves size/PPU: current sign-transition and longer net-zero slicing lacks round-trip coverage and can retain the wrong event subset.

Market identity and symcache

SymbologyCache has three distinct key-addressed mappings:

• assets: provider asset ID to Asset;
• pairs: native bs_mktid to backend pair Struct;
• mktmaps: searchable FQME or explicit alias to normalized MktPair.

Search mapping keys, then resolve values. RapidFuzz treats mapping values as choices; passing mktmaps directly makes it operate on MktPair objects. Preserve in-memory aliases because IB can store both native IDs and FQMEs for one MktPair. Do not persist such aliases without fixing reload: from_dict() currently requires every key to equal mkt.fqme.

Change workflow

Before editing:

1. Identify the owning layer: client, EMS, executor, accounting, or UI.
2. Write down every ID/key transition across that boundary.
3. Trace both normal and cancel-before-ack event ordering.
4. Determine whether records are native, normalized, or summaries.
5. Check actor-local caches and context-exit writes.
6. Add a deterministic regression at the narrowest broken contract.
7. Run live/backend tests only when their credentials and side effects are explicitly authorized.

Do not fix an accounting defect by adding state to EMS, or fix an EMS correlation defect by rewriting durable transaction history.

Canonical source map

• piker/clearing/_messages.py: client and brokerd protocol types.
• piker/clearing/_client.py: OrderClient and open_ems().
• piker/clearing/_ems.py: router, dark book, ID mapping, translation.
• piker/clearing/_paper_engine.py: simulated execution/accounting.
• piker/accounting/_ledger.py: transactions and ledger persistence.
• piker/accounting/_pos.py: positions, accounts, PPU event state.
• piker/accounting/_mktinfo.py: Asset, MktPair, FQME schema.
• piker/data/_symcache.py: provider symbology mappings and search.
• piker/accounting/calc.py: transaction ordering and PPU/PnL math.
• piker/brokers/*/broker.py: backend order-control integration.
• piker/brokers/*/ledger.py: backend trade normalization.

See architecture.md for complete flows, gotchas.md for symptom-driven diagnosis, and test-map.md for verification choices.