Added note to exception when missing field in SpotPair class

README.rst

@@ -1,161 +1,162 @@
 piker
 -----
-trading gear for hackers
+trading gear for hackers.
 
 |gh_actions|
 
 .. |gh_actions| image:: https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fpikers%2Fpiker%2Fbadge&style=popout-square
     :target: https://actions-badge.atrox.dev/piker/pikers/goto
 
-``piker`` is a broker agnostic, next-gen FOSS toolset and runtime for
+``piker`` is a broker agnostic, next-gen FOSS toolset for real-time
-real-time computational trading targeted at `hardcore Linux users
+computational trading targeted at `hardcore Linux users <comp_trader>`_ .
-<comp_trader>`_ .
 
-we use much bleeding edge tech including (but not limited to):
+we use as much bleeding edge tech as possible including (but not limited to):
 
 - latest python for glue_
-- uv_ for packaging and distribution
+- trio_ & tractor_ for our distributed, multi-core, real-time streaming
-- trio_ & tractor_ for our distributed `structured concurrency`_ runtime
+  `structured concurrency`_ runtime B)
-- Qt_ for pristine low latency UIs
+- Qt_ for pristine high performance UIs
-- pyqtgraph_ (which we've extended) for real-time charting and graphics
+- pyqtgraph_ for real-time charting
-- ``polars`` ``numpy`` and ``numba`` for redic `fast numerics`_
+- ``polars`` ``numpy`` and ``numba`` for `fast numerics`_
-- `apache arrow and parquet`_ for time-series storage
+- `apache arrow and parquet`_ for time series history management
+  persistence and sharing
+- (prototyped) techtonicdb_ for L2 book storage
 
-potential projects we might integrate with soon,
+.. |travis| image:: https://img.shields.io/travis/pikers/piker/master.svg
-
+    :target: https://travis-ci.org/pikers/piker
-- (already prototyped in ) techtonicdb_ for L2 book storage
-
-.. _comp_trader: https://jfaleiro.wordpress.com/2019/10/09/computational-trader/
-.. _glue: https://numpy.org/doc/stable/user/c-info.python-as-glue.html#using-python-as-glue
-.. _uv: https://docs.astral.sh/uv/
 .. _trio: https://github.com/python-trio/trio
 .. _tractor: https://github.com/goodboy/tractor
 .. _structured concurrency: https://trio.discourse.group/
+.. _marketstore: https://github.com/alpacahq/marketstore
+.. _techtonicdb: https://github.com/0b01/tectonicdb
 .. _Qt: https://www.qt.io/
 .. _pyqtgraph: https://github.com/pyqtgraph/pyqtgraph
+.. _glue: https://numpy.org/doc/stable/user/c-info.python-as-glue.html#using-python-as-glue
 .. _apache arrow and parquet: https://arrow.apache.org/faq/
 .. _fast numerics: https://zerowithdot.com/python-numpy-and-pandas-performance/
-.. _techtonicdb: https://github.com/0b01/tectonicdb
+.. _comp_trader: https://jfaleiro.wordpress.com/2019/10/09/computational-trader/
 
 
-focus and feats:
+focus and features:
-****************
+*******************
-fitting with these tenets, we're always open to new
+- 100% federated: your code, your hardware, your data feeds, your broker fills.
-framework/lib/service interop suggestions and ideas!
+- zero web: low latency, native software that doesn't try to re-invent the OS
+- maximal **privacy**: prevent brokers and mms from knowing your
+  planz; smack their spreads with dark volume.
+- zero clutter: modal, context oriented UIs that echew minimalism, reduce
+  thought noise and encourage un-emotion.
+- first class parallelism: built from the ground up on next-gen structured concurrency
+  primitives.
+- traders first: broker/exchange/asset-class agnostic
+- systems grounded: real-time financial signal processing that will
+  make any queuing or DSP eng juice their shorts.
+- non-tina UX: sleek, powerful keyboard driven interaction with expected use in tiling wms
+- data collaboration: every process and protocol is multi-host scalable.
+- fight club ready: zero interest in adoption by suits; no corporate friendly license, ever.
 
-- **100% federated**:
+fitting with these tenets, we're always open to new framework suggestions and ideas.
-  your code, your hardware, your data feeds, your broker fills.
 
-- **zero web**:
+building the best looking, most reliable, keyboard friendly trading
-  low latency as a prime objective, native UIs and modern IPC
+platform is the dream; join the cause.
-  protocols without trying to re-invent the "OS-as-an-app"..
-
-- **maximal privacy**:
-  prevent brokers and mms from knowing your planz; smack their
-  spreads with dark volume from a VPN tunnel.
-
-- **zero clutter**:
-  modal, context oriented UIs that echew minimalism, reduce thought
-  noise and encourage un-emotion.
-
-- **first class parallelism**:
-  built from the ground up on a next-gen structured concurrency
-  supervision sys.
-
-- **traders first**:
-  broker/exchange/venue/asset-class/money-sys agnostic
-
-- **systems grounded**:
-  real-time financial signal processing (fsp) that will make any
-  queuing or DSP eng juice their shorts.
-
-- **non-tina UX**:
-  sleek, powerful keyboard driven interaction with expected use in
-  tiling wms (or maybe even a DDE).
-
-- **data collab at scale**:
-  every actor-process and protocol is multi-host aware.
-
-- **fight club ready**:
-  zero interest in adoption by suits; no corporate friendly license,
-  ever.
-
-building the hottest looking, fastest, most reliable, keyboard
-friendly FOSS trading platform is the dream; join the cause.
 
 
-a sane install with `uv`
+sane install with `poetry`
-************************
+**************************
-bc why install with `python` when you can faster with `rust` ::
+TODO!
 
-    uv lock
+
+rigorous install on ``nixos`` using ``poetry2nix``
+**************************************************
+TODO!
 
 
 hacky install on nixos
 **********************
-``NixOS`` is our core devs' distro of choice for which we offer
+`NixOS` is our core devs' distro of choice for which we offer
 a stringently defined development shell envoirment that can be loaded with::
 
-    nix-shell default.nix
+    nix-shell develop.nix
+
+this will setup the required python environment to run piker, make sure to
+run::
+
+    pip install -r requirements.txt -e .
+
+once after loading the shell
 
 
-start a chart
+install wild-west style via `pip`
-*************
+*********************************
-run a realtime OHLCV chart stand-alone::
+``piker`` is currently under heavy pre-alpha development and as such
+should be cloned from this repo and hacked on directly.
 
-    piker -l info chart btcusdt.spot.binance xmrusdt.spot.kraken
+for a development install::
 
-this runs a chart UI (with 1m sampled OHLCV) and shows 2 spot markets from 2 diff cexes
+    git clone git@github.com:pikers/piker.git
-overlayed on the same graph. Use of `piker` without first starting
+    cd piker
-a daemon (`pikerd` - see below) means there is an implicit spawning of the
+    virtualenv env
-multi-actor-runtime (implemented as a `tractor` app).
+    source ./env/bin/activate
-
+    pip install -r requirements.txt -e .
-For additional subsystem feats available through our chart UI see the
-various sub-readmes:
-
-- order control using a mouse-n-keyboard UX B)
-- cross venue market-pair (what most call "symbol") search, select, overlay Bo
-- financial-signal-processing (`piker.fsp`) write-n-reload to sub-chart BO
-- src-asset derivatives scan for anal, like the infamous "max pain" XO
 
 
-spawn a daemon standalone
+check out our charts
-*************************
+********************
-we call the root actor-process the ``pikerd``. it can be (and is
+bet you weren't expecting this from the foss::
-recommended normally to be) started separately from the ``piker
+
-chart`` program::
+    piker -l info -b kraken -b binance chart btcusdt.binance --pdb
+
+
+this runs the main chart (currently with 1m sampled OHLC) in in debug
+mode and you can practice paper trading using the following
+micro-manual:
+
+``order_mode`` (
+    edge triggered activation by any of the following keys,
+    ``mouse-click`` on y-level to submit at that price
+    ):
+
+    - ``f``/ ``ctl-f`` to stage buy
+    - ``d``/ ``ctl-d`` to stage sell
+    - ``a`` to stage alert
+
+
+``search_mode`` (
+    ``ctl-l`` or ``ctl-space`` to open,
+    ``ctl-c`` or ``ctl-space`` to close
+    ) :
+
+    - begin typing to have symbol search automatically lookup
+      symbols from all loaded backend (broker) providers
+    - arrow keys and mouse click to navigate selection
+    - vi-like ``ctl-[hjkl]`` for navigation
+
+
+you can also configure your position allocation limits from the
+sidepane.
+
+
+run in distributed mode
+***********************
+start the service manager and data feed daemon in the background and
+connect to it::
 
     pikerd -l info --pdb
 
-the daemon does nothing until a ``piker``-client (like ``piker
-chart``) connects and requests some particular sub-system. for
-a connecting chart ``pikerd`` will spawn and manage at least,
 
-- a data-feed daemon: ``datad`` which does all the work of comms with
+connect your chart::
-  the backend provider (in this case the ``binance`` cex).
-- a paper-trading engine instance, ``paperboi.binance``, (if no live
-  account has been configured) which allows for auto/manual order
-  control against the live quote stream.
 
-*using* an actor-service (aka micro-daemon) manager which dynamically
+    piker -l info -b kraken -b binance chart xmrusdt.binance --pdb
-supervises various sub-subsystems-as-services throughout the ``piker``
-runtime-stack.
 
-now you can (implicitly) connect your chart::
 
-    piker chart btcusdt.spot.binance
+enjoy persistent real-time data feeds tied to daemon lifetime. the next
-
+time you spawn a chart it will load much faster since the data feed has
-since ``pikerd`` was started separately you can now enjoy a persistent
+been cached and is now always running live in the background until you
-real-time data stream tied to the daemon-tree's lifetime. i.e. the next
+kill ``pikerd``.
-time you spawn a chart it will obviously not only load much faster
-(since the underlying ``datad.binance`` is left running with its
-in-memory IPC data structures) but also the data-feed and any order
-mgmt states should be persistent until you finally cancel ``pikerd``.
 
 
 if anyone asks you what this project is about
 *********************************************
-you don't talk about it; just use it.
+you don't talk about it.
 
 
 how do i get involved?
@@ -165,15 +166,6 @@ enter the matrix.
 
 how come there ain't that many docs
 ***********************************
-i mean we want/need them but building the core right has been higher
+suck it up, learn the code; no one is trying to sell you on anything.
-prio then marketting (and likely will stay that way Bp).
+also, we need lotsa help so if you want to start somewhere and can't
-
+necessarily write serious code, this might be the place for you!
-soo, suck it up bc,
-
-- no one is trying to sell you on anything
-- learning the code base is prolly way more valuable
-- the UI/UXs are intended to be "intuitive" for any hacker..
-
-we obviously need tonz help so if you want to start somewhere and
-can't necessarily write "advanced" concurrent python/rust code, this
-helping document literally anything might be the place for you!

default.nix

@@ -1,134 +1,74 @@
 with (import <nixpkgs> {});
+with python311Packages;
 let
-  glibStorePath = lib.getLib glib;
+  rapidfuzzStorePath = lib.getLib rapidfuzz;
-  zlibStorePath = lib.getLib zlib;
+  qdarkstyleStorePath = lib.getLib qdarkstyle;
-  zstdStorePath = lib.getLib zstd;
+  qtpyStorePath = lib.getLib qtpy;
-  dbusStorePath = lib.getLib dbus;
+  pyqt5StorePath = lib.getLib pyqt5;
-  libGLStorePath = lib.getLib libGL;
+  pyqt5SipStorePath = lib.getLib pyqt5_sip;
-  freetypeStorePath = lib.getLib freetype;
-  qt6baseStorePath = lib.getLib qt6.qtbase;
-  fontconfigStorePath = lib.getLib fontconfig;
-  libxkbcommonStorePath = lib.getLib libxkbcommon;
-  xcbutilcursorStorePath = lib.getLib xcb-util-cursor;
-
-  qtpyStorePath = lib.getLib python312Packages.qtpy;
-  pyqt6StorePath = lib.getLib python312Packages.pyqt6;
-  pyqt6SipStorePath = lib.getLib python312Packages.pyqt6-sip;
-  rapidfuzzStorePath = lib.getLib python312Packages.rapidfuzz;
-  qdarkstyleStorePath = lib.getLib python312Packages.qdarkstyle;
-
-  xorgLibX11StorePath = lib.getLib xorg.libX11;
-  xorgLibxcbStorePath = lib.getLib xorg.libxcb;
-  xorgxcbutilwmStorePath = lib.getLib xorg.xcbutilwm;
-  xorgxcbutilimageStorePath = lib.getLib xorg.xcbutilimage;
-  xorgxcbutilerrorsStorePath = lib.getLib xorg.xcbutilerrors;
-  xorgxcbutilkeysymsStorePath = lib.getLib xorg.xcbutilkeysyms;
-  xorgxcbutilrenderutilStorePath = lib.getLib xorg.xcbutilrenderutil;
 in
 stdenv.mkDerivation {
-  name = "piker-qt6-uv";
+  name = "piker-poetry-shell-with-qt-fix";
   buildInputs = [
     # System requirements.
-    glib
+    libsForQt5.qt5.qtbase
-    zlib
-    dbus
-    zstd
-    libGL
-    freetype
-    qt6.qtbase
-    libgcc.lib
-    fontconfig
-    libxkbcommon
-
-    # Xorg requirements
-    xcb-util-cursor
-    xorg.libxcb
-    xorg.libX11
-    xorg.xcbutilwm
-    xorg.xcbutilimage
-    xorg.xcbutilerrors
-    xorg.xcbutilkeysyms
-    xorg.xcbutilrenderutil
 
     # Python requirements.
-    python312Full
+    python311Full
-    python312Packages.uv
+    poetry-core
-    python312Packages.qdarkstyle
+    rapidfuzz
-    python312Packages.rapidfuzz
+    qdarkstyle
-    python312Packages.pyqt6
+    qtpy
-    python312Packages.qtpy
+    pyqt5
   ];
   src = null;
   shellHook = ''
     set -e
 
+    QTBASE_PATH="${qt5.qtbase.bin}/lib/qt-${qt5.qtbase.version}"
+
     # Set the Qt plugin path
     # export QT_DEBUG_PLUGINS=1
+    export QT_PLUGIN_PATH="$QTBASE_PATH/plugins"
+    export QT_QPA_PLATFORM_PLUGIN_PATH="$QT_PLUGIN_PATH/platforms"
 
-    QTBASE_PATH="${qt6baseStorePath}/lib"
+    # Maybe create venv & install deps
-    QT_PLUGIN_PATH="$QTBASE_PATH/qt-6/plugins"
+    poetry install --with=nix-shell
-    QT_QPA_PLATFORM_PLUGIN_PATH="$QT_PLUGIN_PATH/platforms"
 
-    LIB_GCC_PATH="${libgcc.lib}/lib"
+    # Use pyqt5 from System, patch activate script
-    GLIB_PATH="${glibStorePath}/lib"
+    ACTIVATE_SCRIPT_PATH="$(poetry env info --path)/bin/activate"
-    ZSTD_PATH="${zstdStorePath}/lib"
+    export RPDFUZZ_PATH="${rapidfuzzStorePath}/lib/python3.11/site-packages"
-    ZLIB_PATH="${zlibStorePath}/lib"
+    export QDRKSTYLE_PATH="${qdarkstyleStorePath}/lib/python3.11/site-packages"
-    DBUS_PATH="${dbusStorePath}/lib"
+    export QTPY_PATH="${qtpyStorePath}/lib/python3.11/site-packages"
-    LIBGL_PATH="${libGLStorePath}/lib"
+    export PYQT5_PATH="${pyqt5StorePath}/lib/python3.11/site-packages"
-    FREETYPE_PATH="${freetypeStorePath}/lib"
+    export PYQT5_SIP_PATH="${pyqt5SipStorePath}/lib/python3.11/site-packages"
-    FONTCONFIG_PATH="${fontconfigStorePath}/lib"
+    echo "rapidfuzz at:   $RPDFUZZ_PATH"
-    LIB_XKB_COMMON_PATH="${libxkbcommonStorePath}/lib"
+    echo "qdarkstyle at:  $QDRKSTYLE_PATH"
+    echo "qtpy at:        $QTPY_PATH"
+    echo "pyqt5 at:       $PYQT5_PATH"
+    echo "pyqt5-sip at:   $PYQT5_SIP_PATH"
+    echo ""
 
-    XCB_UTIL_CURSOR_PATH="${xcbutilcursorStorePath}/lib"
+    PATCH="export PYTHONPATH=\""
-    XORG_LIB_X11_PATH="${xorgLibX11StorePath}/lib"
-    XORG_LIB_XCB_PATH="${xorgLibxcbStorePath}/lib"
-    XORG_XCB_UTIL_IMAGE_PATH="${xorgxcbutilimageStorePath}/lib"
-    XORG_XCB_UTIL_WM_PATH="${xorgxcbutilwmStorePath}/lib"
-    XORG_XCB_UTIL_RENDER_UTIL_PATH="${xorgxcbutilrenderutilStorePath}/lib"
-    XORG_XCB_UTIL_KEYSYMS_PATH="${xorgxcbutilkeysymsStorePath}/lib"
-    XORG_XCB_UTIL_ERRORS_PATH="${xorgxcbutilerrorsStorePath}/lib"
 
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$QTBASE_PATH"
+    PATCH="$PATCH\$RPDFUZZ_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$QT_PLUGIN_PATH"
+    PATCH="$PATCH:\$QDRKSTYLE_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$QT_QPA_PLATFORM_PLUGIN_PATH"
+    PATCH="$PATCH:\$QTPY_PATH"
+    PATCH="$PATCH:\$PYQT5_PATH"
+    PATCH="$PATCH:\$PYQT5_SIP_PATH"
 
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$LIB_GCC_PATH"
+    PATCH="$PATCH\""
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$DBUS_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$GLIB_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$ZLIB_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$ZSTD_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$LIBGL_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$FONTCONFIG_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$FREETYPE_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$LIB_XKB_COMMON_PATH"
 
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$XCB_UTIL_CURSOR_PATH"
+    if grep -q "$PATCH" "$ACTIVATE_SCRIPT_PATH"; then
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$XORG_LIB_X11_PATH"
+        echo "venv is already patched."
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$XORG_LIB_XCB_PATH"
+    else
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$XORG_XCB_UTIL_IMAGE_PATH"
+        echo "patching $ACTIVATE_SCRIPT_PATH to use pyqt5 from nixos..."
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$XORG_XCB_UTIL_WM_PATH"
+        sed -i "\$i$PATCH" $ACTIVATE_SCRIPT_PATH
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$XORG_XCB_UTIL_RENDER_UTIL_PATH"
+    fi
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$XORG_XCB_UTIL_KEYSYMS_PATH"
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$XORG_XCB_UTIL_ERRORS_PATH"
 
-    export LD_LIBRARY_PATH
+    echo "qt plguin path: $QT_PLUGIN_PATH"
-
+    echo ""
-    RPDFUZZ_PATH="${rapidfuzzStorePath}/lib/python3.12/site-packages"
-    QDRKSTYLE_PATH="${qdarkstyleStorePath}/lib/python3.12/site-packages"
-    QTPY_PATH="${qtpyStorePath}/lib/python3.12/site-packages"
-    PYQT6_PATH="${pyqt6StorePath}/lib/python3.12/site-packages"
-    PYQT6_SIP_PATH="${pyqt6SipStorePath}/lib/python3.12/site-packages"
-
-    PATCH="$PATCH:$RPDFUZZ_PATH"
-    PATCH="$PATCH:$QDRKSTYLE_PATH"
-    PATCH="$PATCH:$QTPY_PATH"
-    PATCH="$PATCH:$PYQT6_PATH"
-    PATCH="$PATCH:$PYQT6_SIP_PATH"
-
-    export PATCH
-
-    # Install deps
-    uv lock
 
+    poetry shell
   '';
 }

develop.nix

@@ -1,47 +0,0 @@
-with (import <nixpkgs> {});
-
-stdenv.mkDerivation {
-  name = "poetry-env";
-  buildInputs = [
-    # System requirements.
-    readline
-
-    # TODO: hacky non-poetry install stuff we need to get rid of!!
-    poetry
-    # virtualenv
-    # setuptools
-    # pip
-
-    # Python requirements (enough to get a virtualenv going).
-    python311Full
-
-    # obviously, and see below for hacked linking
-    python311Packages.pyqt5
-    python311Packages.pyqt5_sip
-    # python311Packages.qtpy
-
-    # numerics deps
-    python311Packages.levenshtein
-    python311Packages.fastparquet
-    python311Packages.polars
-
-  ];
-  # environment.sessionVariables = {
-  #   LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
-  # };
-  src = null;
-  shellHook = ''
-    # Allow the use of wheels.
-    SOURCE_DATE_EPOCH=$(date +%s)
-
-    # Augment the dynamic linker path
-    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${R}/lib/R/lib:${readline}/lib
-    export QT_QPA_PLATFORM_PLUGIN_PATH="${qt5.qtbase.bin}/lib/qt-${qt5.qtbase.version}/plugins";
-
-    if [ ! -d ".venv" ]; then
-        poetry install --with uis
-    fi
-
-    poetry shell
-  '';
-}

piker/brokers/__init__.py

@@ -50,7 +50,7 @@ __brokers__: list[str] = [
     'binance',
     'ib',
     'kraken',
-    'kucoin',
+    'kucoin'
 
     # broken but used to work
     # 'questrade',
@@ -71,7 +71,7 @@ def get_brokermod(brokername: str) -> ModuleType:
     Return the imported broker module by name.
 
     '''
-    module: ModuleType = import_module('.' + brokername, 'piker.brokers')
+    module = import_module('.' + brokername, 'piker.brokers')
     # we only allow monkeying because it's for internal keying
     module.name = module.__name__.split('.')[-1]
     return module

piker/brokers/_util.py

@@ -18,11 +18,10 @@
 Handy cross-broker utils.
 
 """
-from __future__ import annotations
 from functools import partial
 
 import json
-import httpx
+import asks
 import logging
 
 from ..log import (
@@ -61,11 +60,11 @@ class NoData(BrokerError):
     def __init__(
         self,
         *args,
-        info: dict|None = None,
+        info: dict,
 
     ) -> None:
         super().__init__(*args)
-        self.info: dict|None = info
+        self.info: dict = info
 
         # when raised, machinery can check if the backend
         # set a "frame size" for doing datetime calcs.
@@ -91,18 +90,16 @@ class DataThrottle(BrokerError):
 
 
 def resproc(
-    resp: httpx.Response,
+    resp: asks.response_objects.Response,
     log: logging.Logger,
     return_json: bool = True,
     log_resp: bool = False,
 
-) -> httpx.Response:
+) -> asks.response_objects.Response:
-    '''
+    """Process response and return its json content.
-    Process response and return its json content.
 
     Raise the appropriate error on non-200 OK responses.
-
+    """
-    '''
     if not resp.status_code == 200:
         raise BrokerError(resp.body)
     try:

piker/brokers/binance/api.py

@@ -25,13 +25,14 @@ from __future__ import annotations
 from collections import ChainMap
 from contextlib import (
     asynccontextmanager as acm,
-    AsyncExitStack,
 )
 from datetime import datetime
 from pprint import pformat
 from typing import (
     Any,
     Callable,
+    Hashable,
+    Sequence,
     Type,
 )
 import hmac
@@ -42,7 +43,8 @@ import trio
 from pendulum import (
     now,
 )
-import httpx
+import asks
+from rapidfuzz import process as fuzzy
 import numpy as np
 
 from piker import config
@@ -52,7 +54,6 @@ from piker.clearing._messages import (
 from piker.accounting import (
     Asset,
     digits_to_dec,
-    MktPair,
 )
 from piker.types import Struct
 from piker.data import (
@@ -68,6 +69,7 @@ from .venues import (
     PAIRTYPES,
     Pair,
     MarketType,
+
     _spot_url,
     _futes_url,
     _testnet_futes_url,
@@ -77,18 +79,19 @@ from .venues import (
 log = get_logger('piker.brokers.binance')
 
 
-def get_config() -> dict[str, Any]:
+def get_config() -> dict:
+
     conf: dict
     path: Path
     conf, path = config.load(
         conf_name='brokers',
         touch_if_dne=True,
     )
-    section: dict = conf.get('binance')
+
+    section = conf.get('binance')
+
     if not section:
-        log.warning(
+        log.warning(f'No config section found for binance in {path}')
-            f'No config section found for binance in {path}'
-        )
         return {}
 
     return section
@@ -144,7 +147,7 @@ def binance_timestamp(
 
 class Client:
     '''
-    Async ReST API client using `trio` + `httpx` B)
+    Async ReST API client using ``trio`` + ``asks`` B)
 
     Supports all of the spot, margin and futures endpoints depending
     on method.
@@ -153,17 +156,10 @@ class Client:
     def __init__(
         self,
 
-        venue_sessions: dict[
-            str,  # venue key
-            tuple[httpx.AsyncClient, str]  # session, eps path
-        ],
-        conf: dict[str, Any],
         # TODO: change this to `Client.[mkt_]venue: MarketType`?
         mkt_mode: MarketType = 'spot',
 
     ) -> None:
-        self.conf = conf
-
         # build out pair info tables for each market type
         # and wrap in a chain-map view for search / query.
         self._spot_pairs: dict[str, Pair] = {}  # spot info table
@@ -190,13 +186,44 @@ class Client:
         # market symbols for use by search. See `.exch_info()`.
         self._pairs: ChainMap[str, Pair] = ChainMap()
 
+        # spot EPs sesh
+        self._sesh = asks.Session(connections=4)
+        self._sesh.base_location: str = _spot_url
+        # spot testnet
+        self._test_sesh: asks.Session = asks.Session(connections=4)
+        self._test_sesh.base_location: str = _testnet_spot_url
+
+        # margin and extended spot endpoints session.
+        self._sapi_sesh = asks.Session(connections=4)
+        self._sapi_sesh.base_location: str = _spot_url
+
+        # futes EPs sesh
+        self._fapi_sesh = asks.Session(connections=4)
+        self._fapi_sesh.base_location: str = _futes_url
+        # futes testnet
+        self._test_fapi_sesh: asks.Session = asks.Session(connections=4)
+        self._test_fapi_sesh.base_location: str = _testnet_futes_url
+
         # global client "venue selection" mode.
         # set this when you want to switch venues and not have to
         # specify the venue for the next request.
         self.mkt_mode: MarketType = mkt_mode
 
-        # per-mkt-venue API client table
+        # per 8
-        self.venue_sesh = venue_sessions
+        self.venue_sesh: dict[
+            str,  # venue key
+            tuple[asks.Session, str]  # session, eps path
+        ] = {
+            'spot': (self._sesh, '/api/v3/'),
+            'spot_testnet': (self._test_sesh, '/fapi/v1/'),
+
+            'margin': (self._sapi_sesh, '/sapi/v1/'),
+
+            'usdtm_futes': (self._fapi_sesh, '/fapi/v1/'),
+            'usdtm_futes_testnet': (self._test_fapi_sesh, '/fapi/v1/'),
+
+            # 'futes_coin': self._dapi,  # TODO
+        }
 
         # lookup for going from `.mkt_mode: str` to the config
         # subsection `key: str`
@@ -211,6 +238,40 @@ class Client:
             'futes': ['usdtm_futes'],
         }
 
+        # for creating API keys see,
+        # https://www.binance.com/en/support/faq/how-to-create-api-keys-on-binance-360002502072
+        self.conf: dict = get_config()
+
+        for key, subconf in self.conf.items():
+            if api_key := subconf.get('api_key', ''):
+                venue_keys: list[str] = self.confkey2venuekeys[key]
+
+                venue_key: str
+                sesh: asks.Session
+                for venue_key in venue_keys:
+                    sesh, _ = self.venue_sesh[venue_key]
+
+                    api_key_header: dict = {
+                        # taken from official:
+                        # https://github.com/binance/binance-futures-connector-python/blob/main/binance/api.py#L47
+                        "Content-Type": "application/json;charset=utf-8",
+
+                        # TODO: prolly should just always query and copy
+                        # in the real latest ver?
+                        "User-Agent": "binance-connector/6.1.6smbz6",
+                        "X-MBX-APIKEY": api_key,
+                    }
+                    sesh.headers.update(api_key_header)
+
+                    # if `.use_tesnet = true` in the config then
+                    # also add headers for the testnet session which
+                    # will be used for all order control
+                    if subconf.get('use_testnet', False):
+                        testnet_sesh, _ = self.venue_sesh[
+                            venue_key + '_testnet'
+                        ]
+                        testnet_sesh.headers.update(api_key_header)
+
     def _mk_sig(
         self,
         data: dict,
@@ -229,6 +290,7 @@ class Client:
                 'to define the creds for auth-ed endpoints!?'
             )
 
+
         # XXX: Info on security and authentification
         # https://binance-docs.github.io/apidocs/#endpoint-security-type
         if not (api_secret := subconf.get('api_secret')):
@@ -268,9 +330,8 @@ class Client:
         - /fapi/v3/ USD-M FUTURES, or
         - /api/v3/ SPOT/MARGIN
 
-        account/market endpoint request depending on either passed in
+        account/market endpoint request depending on either passed in `venue: str`
-        `venue: str` or the current setting `.mkt_mode: str` setting,
+        or the current setting `.mkt_mode: str` setting, default `'spot'`.
-        default `'spot'`.
 
 
         Docs per venue API:
@@ -299,6 +360,9 @@ class Client:
                 venue=venue_key,
             )
 
+        sesh: asks.Session
+        path: str
+
         # Check if we're configured to route order requests to the
         # venue equivalent's testnet.
         use_testnet: bool = False
@@ -323,12 +387,11 @@ class Client:
             # ctl machinery B)
             venue_key += '_testnet'
 
-        client: httpx.AsyncClient
+        sesh, path = self.venue_sesh[venue_key]
-        path: str
+
-        client, path = self.venue_sesh[venue_key]
+        meth: Callable = getattr(sesh, method)
-        meth: Callable = getattr(client, method)
         resp = await meth(
-            url=path + endpoint,
+            path=path + endpoint,
             params=params,
             timeout=float('inf'),
         )
@@ -373,11 +436,7 @@ class Client:
             try:
                 pair: Pair = pair_type(**item)
             except Exception as e:
-                e.add_note(
+                e.add_note(f'\nDon\'t panic, check out this https://binance-docs.github.io/apidocs/spot/en/#exchange-information')
-                    "\nDon't panic, prolly stupid binance changed their symbology schema again..\n"
-                    'Check out their API docs here:\n\n'
-                    'https://binance-docs.github.io/apidocs/spot/en/#exchange-information'
-                )
                 raise
             pair_table[pair.symbol.upper()] = pair
 
@@ -473,9 +532,7 @@ class Client:
 
         '''
         pair_table: dict[str, Pair] = self._venue2pairs[
-            venue
+            venue or self.mkt_mode
-            or
-            self.mkt_mode
         ]
         if (
             expiry
@@ -494,9 +551,9 @@ class Client:
             venues: list[str] = [venue]
 
         # batch per-venue download of all exchange infos
-        async with trio.open_nursery() as tn:
+        async with trio.open_nursery() as rn:
             for ven in venues:
-                tn.start_soon(
+                rn.start_soon(
                     self._cache_pairs,
                     ven,
                 )
@@ -549,11 +606,11 @@ class Client:
 
     ) -> dict[str, Any]:
 
-        fq_pairs: dict[str, Pair] = await self.exch_info()
+        fq_pairs: dict = await self.exch_info()
 
         # TODO: cache this list like we were in
         # `open_symbol_search()`?
-        # keys: list[str] = list(fq_pairs)
+        keys: list[str] = list(fq_pairs)
 
         return match_from_pairs(
             pairs=fq_pairs,
@@ -561,20 +618,9 @@ class Client:
             score_cutoff=50,
         )
 
-    def pair2venuekey(
-        self,
-        pair: Pair,
-    ) -> str:
-        return {
-            'USDTM': 'usdtm_futes',
-            'SPOT': 'spot',
-            # 'COINM': 'coin_futes',
-            # ^-TODO-^ bc someone might want it..?
-        }[pair.venue]
-
     async def bars(
         self,
-        mkt: MktPair,
+        symbol: str,
 
         start_dt: datetime | None = None,
         end_dt: datetime | None = None,
@@ -604,20 +650,16 @@ class Client:
         start_time = binance_timestamp(start_dt)
         end_time = binance_timestamp(end_dt)
 
-        bs_pair: Pair = self._pairs[mkt.bs_fqme.upper()]
-
         # https://binance-docs.github.io/apidocs/spot/en/#kline-candlestick-data
         bars = await self._api(
             'klines',
             params={
-                # NOTE: always query using their native symbology!
+                'symbol': symbol.upper(),
-                'symbol': mkt.bs_mktid.upper(),
                 'interval': '1m',
                 'startTime': start_time,
                 'endTime': end_time,
                 'limit': limit
             },
-            venue=self.pair2venuekey(bs_pair),
             allow_testnet=False,
         )
         new_bars: list[tuple] = []
@@ -934,148 +976,17 @@ class Client:
         await self.close_listen_key(key)
 
 
-_venue_urls: dict[str, str] = {
-    'spot': (
-        _spot_url,
-        '/api/v3/',
-    ),
-    'spot_testnet': (
-        _testnet_spot_url,
-        '/fapi/v1/'
-    ),
-    # margin and extended spot endpoints session.
-    # TODO: did this ever get implemented fully?
-    # 'margin': (
-    #     _spot_url,
-    #     '/sapi/v1/'
-    # ),
-
-    'usdtm_futes': (
-        _futes_url,
-        '/fapi/v1/',
-    ),
-
-    'usdtm_futes_testnet': (
-        _testnet_futes_url,
-        '/fapi/v1/',
-    ),
-
-    # TODO: for anyone who actually needs it ;P
-    # 'coin_futes': ()
-}
-
-
-def init_api_keys(
-    client: Client,
-    conf: dict[str, Any],
-) -> None:
-    '''
-    Set up per-venue API keys each http client according to the user's
-    `brokers.conf`.
-
-    For ex, to use spot-testnet and live usdt futures APIs:
-
-    ```toml
-        [binance]
-        # spot test net
-        spot.use_testnet = true
-        spot.api_key = '<spot_api_key_from_binance_account>'
-        spot.api_secret = '<spot_api_key_password>'
-
-        # futes live
-        futes.use_testnet = false
-        accounts.usdtm = 'futes'
-        futes.api_key = '<futes_api_key_from_binance>'
-        futes.api_secret = '<futes_api_key_password>''
-
-        # if uncommented will use the built-in paper engine and not
-        # connect to `binance` API servers for order ctl.
-        # accounts.paper = 'paper'
-    ```
-
-    '''
-    for key, subconf in conf.items():
-        if api_key := subconf.get('api_key', ''):
-            venue_keys: list[str] = client.confkey2venuekeys[key]
-
-            venue_key: str
-            client: httpx.AsyncClient
-            for venue_key in venue_keys:
-                client, _ = client.venue_sesh[venue_key]
-
-                api_key_header: dict = {
-                    # taken from official:
-                    # https://github.com/binance/binance-futures-connector-python/blob/main/binance/api.py#L47
-                    "Content-Type": "application/json;charset=utf-8",
-
-                    # TODO: prolly should just always query and copy
-                    # in the real latest ver?
-                    "User-Agent": "binance-connector/6.1.6smbz6",
-                    "X-MBX-APIKEY": api_key,
-                }
-                client.headers.update(api_key_header)
-
-                # if `.use_tesnet = true` in the config then
-                # also add headers for the testnet session which
-                # will be used for all order control
-                if subconf.get('use_testnet', False):
-                    testnet_sesh, _ = client.venue_sesh[
-                        venue_key + '_testnet'
-                    ]
-                    testnet_sesh.headers.update(api_key_header)
-
-
 @acm
-async def get_client(
+async def get_client() -> Client:
-    mkt_mode: MarketType = 'spot',
-) -> Client:
-    '''
-    Construct an single `piker` client which composes multiple underlying venue
-    specific API clients both for live and test networks.
 
-    '''
+    client = Client()
-    venue_sessions: dict[
+    await client.exch_info()
-        str,  # venue key
-        tuple[httpx.AsyncClient, str]  # session, eps path
-    ] = {}
-    async with AsyncExitStack() as client_stack:
-        for name, (base_url, path) in _venue_urls.items():
-            api: httpx.AsyncClient = await client_stack.enter_async_context(
-                httpx.AsyncClient(
-                    base_url=base_url,
-                    # headers={},
-
-                    # TODO: is there a way to numerate this?
-                    # https://www.python-httpx.org/advanced/clients/#why-use-a-client
-                    # connections=4
-                )
-            )
-            venue_sessions[name] = (
-                api,
-                path,
-            )
-
-        conf: dict[str, Any] = get_config()
-        # for creating API keys see,
-        # https://www.binance.com/en/support/faq/how-to-create-api-keys-on-binance-360002502072
-        client = Client(
-            venue_sessions=venue_sessions,
-            conf=conf,
-            mkt_mode=mkt_mode,
-        )
-        init_api_keys(
-            client=client,
-            conf=conf,
-        )
-        fq_pairs: dict[str, Pair] = await client.exch_info()
-        assert fq_pairs
     log.info(
-            f'Loaded multi-venue `Client` in mkt_mode={client.mkt_mode!r}\n\n'
+        f'{client} in {client.mkt_mode} mode: caching exchange infos..\n'
-            f'Symbology Summary:\n'
+        'Cached multi-market pairs:\n'
-            f'------ - ------\n'
         f'spot: {len(client._spot_pairs)}\n'
         f'usdtm_futes: {len(client._ufutes_pairs)}\n'
-            '------ - ------\n'
+        f'Total: {len(client._pairs)}\n'
-            f'total: {len(client._pairs)}\n'
     )
+
     yield client

piker/brokers/binance/broker.py

@@ -264,20 +264,15 @@ async def open_trade_dialog(
     # do a open_symcache() call.. though maybe we can hide
     # this in a new async version of open_account()?
     async with open_cached_client('binance') as client:
-        subconf: dict|None = client.conf.get(venue_name)
+        subconf: dict = client.conf[venue_name]
+        use_testnet = subconf.get('use_testnet', False)
 
         # XXX: if no futes.api_key or spot.api_key has been set we
         # always fall back to the paper engine!
-        if (
+        if not subconf.get('api_key'):
-            not subconf
-            or
-            not subconf.get('api_key')
-        ):
             await ctx.started('paper')
             return
 
-        use_testnet: bool = subconf.get('use_testnet', False)
-
     async with (
         open_cached_client('binance') as client,
     ):

piker/brokers/binance/feed.py

@@ -42,12 +42,12 @@ from trio_typing import TaskStatus
 from pendulum import (
     from_timestamp,
 )
+from rapidfuzz import process as fuzzy
 import numpy as np
 import tractor
 
 from piker.brokers import (
     open_cached_client,
-    NoData,
 )
 from piker._cacheables import (
     async_lifo_cache,
@@ -110,7 +110,6 @@ class AggTrade(Struct, frozen=True):
 
 async def stream_messages(
     ws: NoBsWs,
-
 ) -> AsyncGenerator[NoBsWs, dict]:
 
     # TODO: match syntax here!
@@ -221,8 +220,6 @@ def make_sub(pairs: list[str], sub_name: str, uid: int) -> dict[str, str]:
     }
 
 
-# TODO, why aren't frame resp `log.info()`s showing in upstream
-# code?!
 @acm
 async def open_history_client(
     mkt: MktPair,
@@ -255,30 +252,24 @@ async def open_history_client(
             else:
                 client.mkt_mode = 'spot'
 
-            array: np.ndarray = await client.bars(
+            # NOTE: always query using their native symbology!
-                mkt=mkt,
+            mktid: str = mkt.bs_mktid
+            array = await client.bars(
+                mktid,
                 start_dt=start_dt,
                 end_dt=end_dt,
             )
-            if array.size == 0:
-                raise NoData(
-                    f'No frame for {start_dt} -> {end_dt}\n'
-                )
-
             times = array['time']
-            if not times.any():
+            if (
-                raise ValueError(
+                end_dt is None
-                    'Bad frame with null-times?\n\n'
+            ):
-                    f'{times}'
+                inow = round(time.time())
-                )
-
-            if end_dt is None:
-                inow: int = round(time.time())
                 if (inow - times[-1]) > 60:
                     await tractor.pause()
 
             start_dt = from_timestamp(times[0])
             end_dt = from_timestamp(times[-1])
+
             return array, start_dt, end_dt
 
         yield get_ohlc, {'erlangs': 3, 'rate': 3}
@@ -465,8 +456,6 @@ async def stream_quotes(
     ):
         init_msgs: list[FeedInit] = []
         for sym in symbols:
-            mkt: MktPair
-            pair: Pair
             mkt, pair = await get_mkt_info(sym)
 
             # build out init msgs according to latest spec
@@ -515,6 +504,7 @@ async def stream_quotes(
 
             # start streaming
             async for typ, quote in msg_gen:
+
                 # period = time.time() - last
                 # hz = 1/period if period else float('inf')
                 # if hz > 60:
@@ -550,7 +540,7 @@ async def open_symbol_search(
                 )
 
                 # repack in fqme-keyed table
-                byfqme: dict[str, Pair] = {}
+                byfqme: dict[start, Pair] = {}
                 for pair in pairs.values():
                     byfqme[pair.bs_fqme] = pair
 

piker/brokers/binance/venues.py

@@ -181,6 +181,7 @@ class FutesPair(Pair):
     quoteAsset: str  # 'USDT',
     quotePrecision: int  # 8,
     requiredMarginPercent: float  # '5.0000',
+    settlePlan: int  # 0,
     timeInForce: list[str]  # ['GTC', 'IOC', 'FOK', 'GTX'],
     triggerProtect: float  # '0.0500',
     underlyingSubType: list[str]  # ['PoW'],

piker/brokers/ib/_util.py

@@ -100,7 +100,7 @@ async def data_reset_hack(
         log.warning(
             no_setup_msg
             +
-            'REQUIRES A `vnc_addrs: array` ENTRY'
+            f'REQUIRES A `vnc_addrs: array` ENTRY'
         )
 
     vnc_host, vnc_port = vnc_sockaddr.get(

piker/brokers/ib/api.py

@@ -287,31 +287,9 @@ class Client:
         self.conf = config
 
         # NOTE: the ib.client here is "throttled" to 45 rps by default
-        self.ib: IB = ib
+        self.ib = ib
         self.ib.RaiseRequestErrors: bool = True
 
-        # self._acnt_names: set[str] = {}
-        self._acnt_names: list[str] = []
-
-    @property
-    def acnts(self) -> list[str]:
-        # return list(self._acnt_names)
-        return self._acnt_names
-
-    def __repr__(self) -> str:
-        return (
-            f'<{type(self).__name__}('
-            f'ib={self.ib} '
-            f'acnts={self.acnts}'
-
-            # TODO: we need to mask out acnt-#s and other private
-            # infos if we're going to console this!
-            # f' |_.conf:\n'
-            # f'    {pformat(self.conf)}\n'
-
-            ')>'
-        )
-
     async def get_fills(self) -> list[Fill]:
         '''
         Return list of rents `Fills` from trading session.
@@ -398,21 +376,19 @@ class Client:
             # whatToShow='MIDPOINT',
             # whatToShow='TRADES',
         )
+        log.info(
+            f'REQUESTING {ib_duration_str} worth {bar_size} BARS\n'
+            f'fqme: {fqme}\n'
+            f'global _enters: {_enters}\n'
+            f'kwargs: {pformat(kwargs)}\n'
+        )
+
         bars = await self.ib.reqHistoricalDataAsync(
             **kwargs,
         )
 
-        query_info: str = (
-            f'REQUESTING IB history BARS\n'
-            f'    ------ - ------\n'
-            f'dt_duration: {dt_duration}\n'
-            f'ib_duration_str: {ib_duration_str}\n'
-            f'bar_size: {bar_size}\n'
-            f'fqme: {fqme}\n'
-            f'actor-global _enters: {_enters}\n'
-            f'kwargs: {pformat(kwargs)}\n'
-        )
         # tail case if no history for range or none prior.
+        if not bars:
             # NOTE: there's actually 3 cases here to handle (and
             # this should be read alongside the implementation of
             # `.reqHistoricalDataAsync()`):
@@ -422,39 +398,33 @@ class Client:
             #   a weekend, holiday or other non-trading period prior to
             #   ``end_dt`` which exceeds the ``duration``,
             # - LITERALLY this is the start of the mkt's history!
-        if not bars:
-            # TODO: figure out wut's going on here.
 
-            # TODO: is this handy, a sync requester for tinkering
-            # with empty frame cases?
-            # def get_hist():
-            #     return self.ib.reqHistoricalData(**kwargs)
-            # import pdbp
-            # pdbp.set_trace()
 
-            log.critical(
+            # sync requester for debugging empty frame cases
-                'STUPID IB SAYS NO HISTORY\n\n'
+            def get_hist():
-                + query_info
+                return self.ib.reqHistoricalData(**kwargs)
-            )
+
+            assert get_hist
+            import pdbp
+            pdbp.set_trace()
 
-            # TODO: we could maybe raise ``NoData`` instead if we
-            # rewrite the method in the first case?
-            # right now there's no way to detect a timeout..
             return [], np.empty(0), dt_duration
+            # TODO: we could maybe raise ``NoData`` instead if we
+            # rewrite the method in the first case? right now there's no
+            # way to detect a timeout.
 
-        log.info(query_info)
+        # NOTE XXX: ensure minimum duration in bars B)
-        # NOTE XXX: ensure minimum duration in bars?
+        # => we recursively call this method until we get at least
-        # => recursively call this method until we get at least as
+        # as many bars such that they sum in aggregate to the the
-        #   many bars such that they sum in aggregate to the the
         # desired total time (duration) at most.
+        # XXX XXX XXX
+        # WHY DID WE EVEN NEED THIS ORIGINALLY!?
+        # XXX XXX XXX
         # - if you query over a gap and get no data
         #   that may short circuit the history 
         if (
-            # XXX XXX XXX
+            end_dt
-            # => WHY DID WE EVEN NEED THIS ORIGINALLY!? <=
+            and False
-            # XXX XXX XXX
-            False
-            and end_dt
         ):
             nparr: np.ndarray = bars_to_np(bars)
             times: np.ndarray = nparr['time']
@@ -957,10 +927,7 @@ class Client:
                         warnset = True
 
             else:
-                log.info(
+                log.info(f'Got first quote for {contract}')
-                    'Got first quote for contract\n'
-                    f'{contract}\n'
-                )
                 break
         else:
             if timeouterr and raise_on_timeout:
@@ -1024,12 +991,8 @@ class Client:
                     outsideRth=True,
 
                     optOutSmartRouting=True,
-                    # TODO: need to understand this setting better as
-                    # it pertains to shit ass mms..
                     routeMarketableToBbo=True,
-
                     designatedLocation='SMART',
-
                     # TODO: make all orders GTC?
                     # https://interactivebrokers.github.io/tws-api/classIBApi_1_1Order.html#a95539081751afb9980f4c6bd1655a6ba
                     # goodTillDate=f"yyyyMMdd-HH:mm:ss",
@@ -1157,8 +1120,8 @@ def get_config() -> dict[str, Any]:
     names = list(accounts.keys())
     accts = section['accounts'] = bidict(accounts)
     log.info(
-        f'{path} defines {len(accts)} account aliases:\n'
+        f'brokers.toml defines {len(accts)} accounts: '
-        f'{pformat(names)}\n'
+        f'{pformat(names)}'
     )
 
     if section is None:
@@ -1225,7 +1188,7 @@ async def load_aio_clients(
         try_ports = list(try_ports.values())
 
     _err = None
-    accounts_def: dict[str, str] = config.load_accounts(['ib'])
+    accounts_def = config.load_accounts(['ib'])
     ports = try_ports if port is None else [port]
     combos = list(itertools.product(hosts, ports))
     accounts_found: dict[str, Client] = {}
@@ -1250,12 +1213,6 @@ async def load_aio_clients(
 
         for i in range(connect_retries):
             try:
-                log.info(
-                    'Trying `ib_async` connect\n'
-                    f'{host}: {port}\n'
-                    f'clientId: {client_id}\n'
-                    f'timeout: {connect_timeout}\n'
-                )
                 await ib.connectAsync(
                     host,
                     port,
@@ -1270,9 +1227,7 @@ async def load_aio_clients(
                 client = Client(ib=ib, config=conf)
 
                 # update all actor-global caches
-                log.runtime(
+                log.info(f"Caching client for {sockaddr}")
-                    f'Connected and caching `Client` @ {sockaddr!r}'
-                )
                 _client_cache[sockaddr] = client
                 break
 
@@ -1287,54 +1242,32 @@ async def load_aio_clients(
                 OSError,
             ) as ce:
                 _err = ce
-                message: str = (
+                log.warning(
-                    f'Failed to connect on {host}:{port} after {i} tries with\n'
+                    f'Failed to connect on {host}:{port} for {i} time with,\n'
-                    f'{ib.client.apiError.value()!r}\n\n'
+                    f'{ib.client.apiError.value()}\n'
-                    'Retrying with a new client id..\n'
+                    'retrying with a new client id..')
-                )
-                log.runtime(message)
-        else:
-            # XXX report loudly if we never established after all
-            # re-tries
-            log.warning(message)
 
         # Pre-collect all accounts available for this
         # connection and map account names to this client
         # instance.
         for value in ib.accountValues():
-            acct_number: str = value.account
+            acct_number = value.account
 
-            acnt_alias: str = accounts_def.inverse.get(acct_number)
+            entry = accounts_def.inverse.get(acct_number)
-            if not acnt_alias:
+            if not entry:
-
-                # TODO: should we constuct the below reco-ex from
-                # the existing config content?
-                _, path = config.load(
-                    conf_name='brokers',
-                )
                 raise ValueError(
-                    'No alias in account section for account!\n'
+                    'No section in brokers.toml for account:'
-                    f'Please add an acnt alias entry to your {path}\n'
+                    f' {acct_number}\n'
-                    'For example,\n\n'
+                    f'Please add entry to continue using this API client'
-
-                    '[ib.accounts]\n'
-                    'margin = {accnt_number!r}\n'
-                    '^^^^^^ <- you need this part!\n\n'
-
-                    'This ensures `piker` will not leak private acnt info '
-                    'to console output by default!\n'
                 )
 
             # surjection of account names to operating clients.
-            if acnt_alias not in accounts_found:
+            if acct_number not in accounts_found:
-                accounts_found[acnt_alias] = client
+                accounts_found[entry] = client
-                # client._acnt_names.add(acnt_alias)
-                client._acnt_names.append(acnt_alias)
 
-        if accounts_found:
         log.info(
-                f'Loaded accounts for api client\n\n'
+            f'Loaded accounts for client @ {host}:{port}\n'
-                f'{pformat(accounts_found)}\n'
+            f'{pformat(accounts_found)}'
         )
 
         # XXX: why aren't we just updating this directy above
@@ -1373,9 +1306,7 @@ async def load_clients_for_trio(
     a ``tractor.to_asyncio.open_channel_from()``.
 
     '''
-    async with load_aio_clients(
+    async with load_aio_clients() as accts2clients:
-        disconnect_on_exit=False,
-    ) as accts2clients:
 
         to_trio.send_nowait(accts2clients)
 
@@ -1541,7 +1472,7 @@ async def open_aio_client_method_relay(
         msg: tuple[str, dict] | dict | None = await from_trio.get()
         match msg:
             case None:  # termination sentinel
-                log.info('asyncio `Client` method-proxy SHUTDOWN!')
+                print('asyncio PROXY-RELAY SHUTDOWN')
                 break
 
             case (meth_name, kwargs):

piker/brokers/ib/broker.py

@@ -1183,14 +1183,7 @@ async def deliver_trade_events(
                         pos
                         and fill
                     ):
-                        now_cr: CommissionReport = fill.commissionReport
+                        assert fill.commissionReport == cr
-                        if (now_cr != cr):
-                            log.warning(
-                                'UhhHh ib updated the commission report mid-fill..?\n'
-                                f'was: {pformat(cr)}\n'
-                                f'now: {pformat(now_cr)}\n'
-                            )
-
                         await emit_pp_update(
                             ems_stream,
                             accounts_def,

piker/brokers/ib/feed.py

@@ -671,8 +671,8 @@ async def _setup_quote_stream(
         # making them mostly useless and explains why the scanner
         # is always slow XD
         # '293',  # Trade count for day
-        # '294',  # Trade rate / minute
+        '294',  # Trade rate / minute
-        # '295',  # Vlm rate / minute
+        '295',  # Vlm rate / minute
     ),
     contract: Contract | None = None,
 
@@ -915,13 +915,9 @@ async def stream_quotes(
 
         if first_ticker:
             first_quote: dict = normalize(first_ticker)
-
+            log.info(
-            # TODO: we need a stack-oriented log levels filters for
+                'Rxed init quote:\n'
-            # this!
+                f'{pformat(first_quote)}'
-            # log.info(message, filter={'stack': 'live_feed'}) ?
-            log.runtime(
-                'Rxed init quote:\n\n'
-                f'{pformat(first_quote)}\n'
             )
 
         # NOTE: it might be outside regular trading hours for
@@ -973,11 +969,7 @@ async def stream_quotes(
             raise_on_timeout=True,
         )
         first_quote: dict = normalize(first_ticker)
-
+        log.info(
-        # TODO: we need a stack-oriented log levels filters for
-        # this!
-        # log.info(message, filter={'stack': 'live_feed'}) ?
-        log.runtime(
             'Rxed init quote:\n'
             f'{pformat(first_quote)}'
         )

piker/brokers/ib/ledger.py

@@ -31,11 +31,7 @@ from typing import (
 )
 
 from bidict import bidict
-from pendulum import (
+import pendulum
-    DateTime,
-    parse,
-    from_timestamp,
-)
 from ib_insync import (
     Contract,
     Commodity,
@@ -70,11 +66,10 @@ tx_sort: Callable = partial(
     iter_by_dt,
     parsers={
         'dateTime': parse_flex_dt,
-        'datetime': parse,
+        'datetime': pendulum.parse,
-
+        # for some some fucking 2022 and
-        # XXX: for some some fucking 2022 and
+        # back options records...fuck me.
-        # back options records.. f@#$ me..
+        'date': pendulum.parse,
-        'date': parse,
     }
 )
 
@@ -94,38 +89,15 @@ def norm_trade(
 
     conid: int = str(record.get('conId') or record['conid'])
     bs_mktid: str = str(conid)
+    comms = record.get('commission')
+    if comms is None:
+        comms = -1*record['ibCommission']
 
-    # NOTE: sometimes weird records (like BTTX?)
+    price = record.get('price') or record['tradePrice']
-    # have no field for this?
-    comms: float = -1 * (
-        record.get('commission')
-        or record.get('ibCommission')
-        or 0
-    )
-    if not comms:
-        log.warning(
-            'No commissions found for record?\n'
-            f'{pformat(record)}\n'
-        )
-
-    price: float = (
-        record.get('price')
-        or record.get('tradePrice')
-    )
-    if price is None:
-        log.warning(
-            'No `price` field found in record?\n'
-            'Skipping normalization..\n'
-            f'{pformat(record)}\n'
-        )
-        return None
 
     # the api doesn't do the -/+ on the quantity for you but flex
     # records do.. are you fucking serious ib...!?
-    size: float|int = (
+    size = record.get('quantity') or record['shares'] * {
-        record.get('quantity')
-        or record['shares']
-    ) * {
         'BOT': 1,
         'SLD': -1,
     }[record['side']]
@@ -156,31 +128,26 @@ def norm_trade(
         # otype = tail[6]
         # strike = tail[7:]
 
-        log.warning(
+        print(f'skipping opts contract {symbol}')
-            f'Skipping option contract -> NO SUPPORT YET!\n'
-            f'{symbol}\n'
-        )
         return None
 
     # timestamping is way different in API records
-    dtstr: str = record.get('datetime')
+    dtstr = record.get('datetime')
-    date: str = record.get('date')
+    date = record.get('date')
-    flex_dtstr: str = record.get('dateTime')
+    flex_dtstr = record.get('dateTime')
 
     if dtstr or date:
-        dt: DateTime = parse(dtstr or date)
+        dt = pendulum.parse(dtstr or date)
 
     elif flex_dtstr:
         # probably a flex record with a wonky non-std timestamp..
-        dt: DateTime = parse_flex_dt(record['dateTime'])
+        dt = parse_flex_dt(record['dateTime'])
 
     # special handling of symbol extraction from
     # flex records using some ad-hoc schema parsing.
-    asset_type: str = (
+    asset_type: str = record.get(
-        record.get('assetCategory')
+        'assetCategory'
-        or record.get('secType')
+    ) or record.get('secType', 'STK')
-        or 'STK'
-    )
 
     if (expiry := (
             record.get('lastTradeDateOrContractMonth')
@@ -390,7 +357,6 @@ def norm_trade_records(
         if txn is None:
             continue
 
-        # inject txns sorted by datetime
         insort(
             records,
             txn,
@@ -439,7 +405,7 @@ def api_trades_to_ledger_entries(
                     txn_dict[attr_name] = val
 
         tid = str(txn_dict['execId'])
-        dt = from_timestamp(txn_dict['time'])
+        dt = pendulum.from_timestamp(txn_dict['time'])
         txn_dict['datetime'] = str(dt)
         acctid = accounts[txn_dict['acctNumber']]
 

piker/brokers/ib/symbols.py

@@ -209,10 +209,7 @@ async def open_symbol_search(ctx: tractor.Context) -> None:
                 break
 
             ib_client = proxy._aio_ns.ib
-            log.info(
+            log.info(f'Using {ib_client} for symbol search')
-                f'Using API client for symbol-search\n'
-                f'{ib_client}\n'
-            )
 
             last = time.time()
             async for pattern in stream:
@@ -297,7 +294,7 @@ async def open_symbol_search(ctx: tractor.Context) -> None:
                     elif stock_results:
                         break
                     # else:
-                    # await tractor.pause()
+                    await tractor.pause()
 
                     # # match against our ad-hoc set immediately
                     # adhoc_matches = fuzzy.extract(
@@ -525,21 +522,7 @@ async def get_mkt_info(
         venue = con.primaryExchange or con.exchange
 
     price_tick: Decimal = Decimal(str(details.minTick))
-    ib_min_tick_gt_2: Decimal = Decimal('0.01')
+    # price_tick: Decimal = Decimal('0.01')
-    if (
-        price_tick < ib_min_tick_gt_2
-    ):
-        # TODO: we need to add some kinda dynamic rounding sys
-        # to our MktPair i guess?
-        # not sure where the logic should sit, but likely inside
-        # the `.clearing._ems` i suppose...
-        log.warning(
-            'IB seems to disallow a min price tick < 0.01 '
-            'when the price is > 2.0..?\n'
-            f'Decreasing min tick precision for {fqme} to 0.01'
-        )
-        # price_tick = ib_min_tick
-        # await tractor.pause()
 
     if atype == 'stock':
         # XXX: GRRRR they don't support fractional share sizes for

piker/brokers/kraken/api.py

@@ -27,8 +27,8 @@ from typing import (
 )
 import time
 
-import httpx
 import pendulum
+import asks
 import numpy as np
 import urllib.parse
 import hashlib
@@ -60,11 +60,6 @@ log = get_logger('piker.brokers.kraken')
 
 # <uri>/<version>/
 _url = 'https://api.kraken.com/0'
-
-_headers: dict[str, str] = {
-    'User-Agent': 'krakenex/2.1.0 (+https://github.com/veox/python3-krakenex)'
-}
-
 # TODO: this is the only backend providing this right?
 # in which case we should drop it from the defaults and
 # instead make a custom fields descr in this module!
@@ -140,15 +135,16 @@ class Client:
     def __init__(
         self,
         config: dict[str, str],
-        httpx_client: httpx.AsyncClient,
-
         name: str = '',
         api_key: str = '',
         secret: str = ''
     ) -> None:
-
+        self._sesh = asks.Session(connections=4)
-        self._sesh: httpx.AsyncClient = httpx_client
+        self._sesh.base_location = _url
-
+        self._sesh.headers.update({
+            'User-Agent':
+                'krakenex/2.1.0 (+https://github.com/veox/python3-krakenex)'
+        })
         self._name = name
         self._api_key = api_key
         self._secret = secret
@@ -170,9 +166,10 @@ class Client:
         method: str,
         data: dict,
     ) -> dict[str, Any]:
-        resp: httpx.Response = await self._sesh.post(
+        resp = await self._sesh.post(
-            url=f'/public/{method}',
+            path=f'/public/{method}',
             json=data,
+            timeout=float('inf')
         )
         return resproc(resp, log)
 
@@ -183,18 +180,18 @@ class Client:
         uri_path: str
     ) -> dict[str, Any]:
         headers = {
-            'Content-Type': 'application/x-www-form-urlencoded',
+            'Content-Type':
-            'API-Key': self._api_key,
+                'application/x-www-form-urlencoded',
-            'API-Sign': get_kraken_signature(
+            'API-Key':
-                uri_path,
+                self._api_key,
-                data,
+            'API-Sign':
-                self._secret,
+                get_kraken_signature(uri_path, data, self._secret)
-            ),
         }
-        resp: httpx.Response = await self._sesh.post(
+        resp = await self._sesh.post(
-            url=f'/private/{method}',
+            path=f'/private/{method}',
             data=data,
             headers=headers,
+            timeout=float('inf')
         )
         return resproc(resp, log)
 
@@ -668,19 +665,10 @@ class Client:
 @acm
 async def get_client() -> Client:
 
-    conf: dict[str, Any] = get_config()
+    conf = get_config()
-    async with httpx.AsyncClient(
-        base_url=_url,
-        headers=_headers,
-
-        # TODO: is there a way to numerate this?
-        # https://www.python-httpx.org/advanced/clients/#why-use-a-client
-        # connections=4
-    ) as trio_client:
     if conf:
         client = Client(
             conf,
-                httpx_client=trio_client,
 
             # TODO: don't break these up and just do internal
             # conf lookups instead..
@@ -689,10 +677,7 @@ async def get_client() -> Client:
             secret=conf['secret']
         )
     else:
-            client = Client(
+        client = Client({})
-                conf={},
-                httpx_client=trio_client,
-            )
 
     # at startup, load all symbols, and asset info in
     # batch requests.

piker/brokers/kraken/broker.py

@@ -612,18 +612,18 @@ async def open_trade_dialog(
 
                 # enter relay loop
                 await handle_order_updates(
-                    client=client,
+                    client,
-                    ws=ws,
+                    ws,
-                    ws_stream=stream,
+                    stream,
-                    ems_stream=ems_stream,
+                    ems_stream,
-                    apiflows=apiflows,
+                    apiflows,
-                    ids=ids,
+                    ids,
-                    reqids2txids=reqids2txids,
+                    reqids2txids,
-                    acnt=acnt,
+                    acnt,
-                    ledger=ledger,
+                    api_trans,
-                    acctid=acctid,
+                    acctid,
-                    acc_name=acc_name,
+                    acc_name,
-                    token=token,
+                    token,
                 )
 
 
@@ -639,8 +639,7 @@ async def handle_order_updates(
 
     # transaction records which will be updated
     # on new trade clearing events (aka order "fills")
-    ledger: TransactionLedger,
+    ledger_trans: dict[str, Transaction],
-    # ledger_trans: dict[str, Transaction],
     acctid: str,
     acc_name: str,
     token: str,
@@ -700,8 +699,7 @@ async def handle_order_updates(
                     # if tid not in ledger_trans
                 }
                 for tid, trade in trades.items():
-                    # assert tid not in ledger_trans
+                    assert tid not in ledger_trans
-                    assert tid not in ledger
                     txid = trade['ordertxid']
                     reqid = trade.get('userref')
 
@@ -749,17 +747,11 @@ async def handle_order_updates(
                     client,
                     api_name_set='wsname',
                 )
-                ppmsgs: list[BrokerdPosition] = trades2pps(
+                ppmsgs = trades2pps(
-                    acnt=acnt,
+                    acnt,
-                    ledger=ledger,
+                    acctid,
-                    acctid=acctid,
+                    new_trans,
-                    new_trans=new_trans,
                 )
-                # ppmsgs = trades2pps(
-                #     acnt,
-                #     acctid,
-                #     new_trans,
-                # )
                 for pp_msg in ppmsgs:
                     await ems_stream.send(pp_msg)
 

piker/brokers/kucoin.py

@@ -16,9 +16,10 @@
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 '''
-Kucoin cex API backend.
+Kucoin broker backend
 
 '''
+
 from contextlib import (
     asynccontextmanager as acm,
     aclosing,
@@ -41,7 +42,7 @@ import wsproto
 from uuid import uuid4
 
 from trio_typing import TaskStatus
-import httpx
+import asks
 from bidict import bidict
 import numpy as np
 import pendulum
@@ -62,7 +63,7 @@ from piker._cacheables import (
 )
 from piker.log import get_logger
 from piker.data.validate import FeedInit
-from piker.types import Struct  # NOTE, this is already a `tractor.msg.Struct`
+from piker.types import Struct
 from piker.data import (
     def_iohlcv_fields,
     match_from_pairs,
@@ -98,18 +99,9 @@ class KucoinMktPair(Struct, frozen=True):
     def size_tick(self) -> Decimal:
         return Decimal(str(self.quoteMinSize))
 
-    callauctionFirstStageStartTime: None|float
-    callauctionIsEnabled: bool
-    callauctionPriceCeiling: float|None
-    callauctionPriceFloor: float|None
-    callauctionSecondStageStartTime: float|None
-    callauctionThirdStageStartTime: float|None
-
     enableTrading: bool
-    feeCategory: int
     feeCurrency: str
     isMarginEnabled: bool
-    makerFeeCoefficient: float
     market: str
     minFunds: float
     name: str
@@ -119,10 +111,7 @@ class KucoinMktPair(Struct, frozen=True):
     quoteIncrement: float
     quoteMaxSize: float
     quoteMinSize: float
-    st: bool
     symbol: str  # our bs_mktid, kucoin's internal id
-    takerFeeCoefficient: float
-    tradingStartTime: float|None
 
 
 class AccountTrade(Struct, frozen=True):
@@ -223,11 +212,7 @@ def get_config() -> BrokerConfig | None:
 
 class Client:
 
-    def __init__(
+    def __init__(self) -> None:
-        self,
-        httpx_client: httpx.AsyncClient,
-    ) -> None:
-        self._http: httpx.AsyncClient = httpx_client
         self._config: BrokerConfig | None = get_config()
         self._pairs: dict[str, KucoinMktPair] = {}
         self._fqmes2mktids: bidict[str, str] = bidict()
@@ -242,24 +227,18 @@ class Client:
 
     ) -> dict[str, str | bytes]:
         '''
-        Generate authenticated request headers:
+        Generate authenticated request headers
-
         https://docs.kucoin.com/#authentication
-        https://www.kucoin.com/docs/basic-info/connection-method/authentication/creating-a-request
-        https://www.kucoin.com/docs/basic-info/connection-method/authentication/signing-a-message
 
         '''
+
         if not self._config:
             raise ValueError(
-                'No config found when trying to send authenticated request'
+                'No config found when trying to send authenticated request')
-            )
 
         str_to_sign = (
             str(int(time.time() * 1000))
-            +
+            + action + f'/api/{api}/{endpoint.lstrip("/")}'
-            action
-            +
-            f'/api/{api}/{endpoint.lstrip("/")}'
         )
 
         signature = base64.b64encode(
@@ -270,7 +249,6 @@ class Client:
             ).digest()
         )
 
-        # TODO: can we cache this between calls?
         passphrase = base64.b64encode(
             hmac.new(
                 self._config.key_secret.encode('utf-8'),
@@ -292,10 +270,8 @@ class Client:
         self,
         action: Literal['POST', 'GET'],
         endpoint: str,
-
         api: str = 'v2',
         headers: dict = {},
-
     ) -> Any:
         '''
         Generic request wrapper for Kucoin API
@@ -308,19 +284,14 @@ class Client:
                 api,
             )
 
-        req_meth: Callable = getattr(
+        api_url = f'https://api.kucoin.com/api/{api}/{endpoint}'
-            self._http,
+
-            action.lower(),
+        res = await asks.request(action, api_url, headers=headers)
-        )
+
-        res = await req_meth(
+        json = res.json()
-            url=f'/{api}/{endpoint}',
+        if 'data' in json:
-            headers=headers,
+            return json['data']
-        )
-        json: dict = res.json()
-        if (data := json.get('data')) is not None:
-            return data
         else:
-            api_url: str = self._http.base_url
             log.error(
                 f'Error making request to {api_url} ->\n'
                 f'{pformat(res)}'
@@ -378,8 +349,8 @@ class Client:
             currencies: dict[str, Currency] = {}
             entries: list[dict] = await self._request(
                 'GET',
-                endpoint='currencies',
                 api='v1',
+                endpoint='currencies',
             )
             for entry in entries:
                 curr = Currency(**entry).copy()
@@ -395,22 +366,13 @@ class Client:
         dict[str, KucoinMktPair],
         bidict[str, KucoinMktPair],
     ]:
-        entries = await self._request(
+        entries = await self._request('GET', 'symbols')
-            'GET',
-            endpoint='symbols',
-        )
         log.info(f' {len(entries)} Kucoin market pairs fetched')
 
         pairs: dict[str, KucoinMktPair] = {}
         fqmes2mktids: bidict[str, str] = bidict()
         for item in entries:
-            try:
             pair = pairs[item['name']] = KucoinMktPair(**item)
-            except TypeError as te:
-                raise TypeError(
-                    '`KucoinMktPair` and reponse fields do not match ??\n'
-                    f'{KucoinMktPair.fields_diff(item)}\n'
-                ) from te
             fqmes2mktids[
                 item['name'].lower().replace('-', '')
             ] = pair.name
@@ -605,18 +567,10 @@ def fqme_to_kucoin_sym(
 
 @acm
 async def get_client() -> AsyncGenerator[Client, None]:
-    '''
+    client = Client()
-    Load an API `Client` preconfigured from user settings
 
-    '''
+    async with trio.open_nursery() as n:
-    async with (
+        n.start_soon(client.get_mkt_pairs)
-        httpx.AsyncClient(
-            base_url='https://api.kucoin.com/api',
-        ) as trio_client,
-    ):
-        client = Client(httpx_client=trio_client)
-        async with trio.open_nursery() as tn:
-            tn.start_soon(client.get_mkt_pairs)
         await client.get_currencies()
 
     yield client
@@ -655,7 +609,7 @@ async def open_ping_task(
                 await trio.sleep((ping_interval - 1000) / 1000)
                 await ws.send_msg({'id': connect_id, 'type': 'ping'})
 
-        log.warning('Starting ping task for kucoin ws connection')
+        log.info('Starting ping task for kucoin ws connection')
         n.start_soon(ping_server)
 
         yield
@@ -667,14 +621,9 @@ async def open_ping_task(
 async def get_mkt_info(
     fqme: str,
 
-) -> tuple[
+) -> tuple[MktPair, KucoinMktPair]:
-    MktPair,
-    KucoinMktPair,
-]:
     '''
-    Query for and return both a `piker.accounting.MktPair` and
+    Query for and return a `MktPair` and `KucoinMktPair`.
-    `KucoinMktPair` from provided `fqme: str`
-    (fully-qualified-market-endpoint).
 
     '''
     async with open_cached_client('kucoin') as client:
@@ -749,8 +698,6 @@ async def stream_quotes(
 
         log.info(f'Starting up quote stream(s) for {symbols}')
         for sym_str in symbols:
-            mkt: MktPair
-            pair: KucoinMktPair
             mkt, pair = await get_mkt_info(sym_str)
             init_msgs.append(
                 FeedInit(mkt_info=mkt)
@@ -758,11 +705,7 @@ async def stream_quotes(
 
         ws: NoBsWs
         token, ping_interval = await client._get_ws_token()
-        log.info('API reported ping_interval: {ping_interval}\n')
+        connect_id = str(uuid4())
-
-        connect_id: str = str(uuid4())
-        typ: str
-        quote: dict
         async with (
             open_autorecon_ws(
                 (
@@ -776,37 +719,20 @@ async def stream_quotes(
                 ),
             ) as ws,
             open_ping_task(ws, ping_interval, connect_id),
-            aclosing(
+            aclosing(stream_messages(ws, sym_str)) as msg_gen,
-                iter_normed_quotes(
-                    ws, sym_str
-                )
-            ) as iter_quotes,
         ):
-            typ, quote = await anext(iter_quotes)
+            typ, quote = await anext(msg_gen)
 
+            while typ != 'trade':
                 # take care to not unblock here until we get a real
-            # trade quote?
+                # trade quote
-            # ^TODO, remove this right?
+                typ, quote = await anext(msg_gen)
-            # -[ ] what often blocks chart boot/new-feed switching
-            #   since we'ere waiting for a live quote instead of just
-            #   loading history afap..
-            #  |_ XXX, not sure if we require a bit of rework to core
-            #    feed init logic or if backends justg gotta be
-            #    changed up.. feel like there was some causality
-            #    dilema prolly only seen with IB too..
-            # while typ != 'trade':
-            #     typ, quote = await anext(iter_quotes)
 
             task_status.started((init_msgs, quote))
             feed_is_live.set()
 
-            # XXX NOTE, DO NOT include the `.<backend>` suffix!
+            async for typ, msg in msg_gen:
-            # OW the sampling loop will not broadcast correctly..
+                await send_chan.send({sym_str: msg})
-            # since `bus._subscribers.setdefault(bs_fqme, set())`
-            # is used inside `.data.open_feed_bus()` !!!
-            topic: str = mkt.bs_fqme
-            async for typ, quote in iter_quotes:
-                await send_chan.send({topic: quote})
 
 
 @acm
@@ -861,7 +787,7 @@ async def subscribe(
             )
 
 
-async def iter_normed_quotes(
+async def stream_messages(
     ws: NoBsWs,
     sym: str,
 
@@ -892,9 +818,6 @@ async def iter_normed_quotes(
 
                 yield 'trade', {
                     'symbol': sym,
-                    # TODO, is 'last' even used elsewhere/a-good
-                    # semantic? can't we just read the ticks with our
-                    # .data.ticktools.frame_ticks()`/
                     'last': trade_data.price,
                     'brokerd_ts': last_trade_ts,
                     'ticks': [
@@ -987,7 +910,7 @@ async def open_history_client(
             if end_dt is None:
                 inow = round(time.time())
 
-                log.debug(
+                print(
                     f'difference in time between load and processing'
                     f'{inow - times[-1]}'
                 )

piker/clearing/README.rst

@@ -1,49 +0,0 @@
-piker.clearing
-______________
-trade execution-n-control subsys for both live and paper trading as
-well as algo-trading manual override/interaction across any backend
-broker and data provider.
-
-avail UIs
-*********
-
-order ctl
----------
-the `piker.clearing` subsys is exposed mainly though
-the `piker chart` GUI as a "chart trader" style UX and
-is automatically enabled whenever a chart is opened.
-
-.. ^TODO, more prose here!
-
-the "manual" order control features are exposed via the
-`piker.ui.order_mode` API and can pretty much always be
-used (at least) in simulated-trading mode, aka "paper"-mode, and
-the micro-manual is as follows:
-
-``order_mode`` (
-    edge triggered activation by any of the following keys,
-    ``mouse-click`` on y-level to submit at that price
-    ):
-
-    - ``f``/ ``ctl-f`` to stage buy
-    - ``d``/ ``ctl-d`` to stage sell
-    - ``a`` to stage alert
-
-
-``search_mode`` (
-    ``ctl-l`` or ``ctl-space`` to open,
-    ``ctl-c`` or ``ctl-space`` to close
-    ) :
-
-    - begin typing to have symbol search automatically lookup
-      symbols from all loaded backend (broker) providers
-    - arrow keys and mouse click to navigate selection
-    - vi-like ``ctl-[hjkl]`` for navigation
-
-
-position (pp) mgmt
-------------------
-you can also configure your position allocation limits from the
-sidepane.
-
-.. ^TODO, explain and provide tut once more refined!

piker/config.py

@@ -104,15 +104,14 @@ def get_app_dir(
     # `tractor`) with the testing dir and check for it whenever we
     # detect `pytest` is being used (which it isn't under normal
     # operation).
-    # if "pytest" in sys.modules:
+    if "pytest" in sys.modules:
-    #     import tractor
+        import tractor
-    #     actor = tractor.current_actor(err_on_no_runtime=False)
+        actor = tractor.current_actor(err_on_no_runtime=False)
-    #     if actor:  # runtime is up
+        if actor:  # runtime is up
-    #         rvs = tractor._state._runtime_vars
+            rvs = tractor._state._runtime_vars
-    #         import pdbp; pdbp.set_trace()
+            testdirpath = Path(rvs['piker_vars']['piker_test_dir'])
-    #         testdirpath = Path(rvs['piker_vars']['piker_test_dir'])
+            assert testdirpath.exists(), 'piker test harness might be borked!?'
-    #         assert testdirpath.exists(), 'piker test harness might be borked!?'
+            app_name = str(testdirpath)
-    #         app_name = str(testdirpath)
 
     if platform.system() == 'Windows':
         key = "APPDATA" if roaming else "LOCALAPPDATA"

piker/data/_web_bs.py

@@ -273,7 +273,7 @@ async def _reconnect_forever(
                 nobsws._connected.set()
                 await trio.sleep_forever()
         except HandshakeError:
-            log.exception('Retrying connection')
+            log.exception(f'Retrying connection')
 
         # ws & nursery block ends
 
@@ -359,8 +359,8 @@ async def open_autorecon_ws(
 
 
 '''
-JSONRPC response-request style machinery for transparent multiplexing
+JSONRPC response-request style machinery for transparent multiplexing of msgs
-of msgs over a `NoBsWs`.
+over a NoBsWs.
 
 '''
 
@@ -377,82 +377,43 @@ async def open_jsonrpc_session(
     url: str,
     start_id: int = 0,
     response_type: type = JSONRPCResult,
-    msg_recv_timeout: float = float('inf'),
+    request_type: Optional[type] = None,
-    # ^NOTE, since only `deribit` is using this jsonrpc stuff atm
+    request_hook: Optional[Callable] = None,
-    # and options mkts are generally "slow moving"..
+    error_hook: Optional[Callable] = None,
-    #
-    # FURTHER if we break the underlying ws connection then since we
-    # don't pass a `fixture` to the task that manages `NoBsWs`, i.e.
-    # `_reconnect_forever()`, the jsonrpc "transport pipe" get's
-    # broken and never restored with wtv init sequence is required to
-    # re-establish a working req-resp session.
-
 ) -> Callable[[str, dict], dict]:
-    '''
-    Init a json-RPC-over-websocket connection to the provided `url`.
-
-    A `json_rpc: Callable[[str, dict], dict` is delivered to the
-    caller for sending requests and a bg-`trio.Task` handles
-    processing of response msgs including error reporting/raising in
-    the parent/caller task.
-
-    '''
-    # NOTE, store all request msgs so we can raise errors on the
-    # caller side!
-    req_msgs: dict[int, dict] = {}
 
     async with (
-        trio.open_nursery() as tn,
+        trio.open_nursery() as n,
-        open_autorecon_ws(
+        open_autorecon_ws(url) as ws
-            url=url,
-            msg_recv_timeout=msg_recv_timeout,
-        ) as ws
     ):
-        rpc_id: Iterable[int] = count(start_id)
+        rpc_id: Iterable = count(start_id)
         rpc_results: dict[int, dict] = {}
 
-        async def json_rpc(
+        async def json_rpc(method: str, params: dict) -> dict:
-            method: str,
-            params: dict,
-        ) -> dict:
             '''
             perform a json rpc call and wait for the result, raise exception in
             case of error field present on response
             '''
-            nonlocal req_msgs
-
-            req_id: int = next(rpc_id)
             msg = {
                 'jsonrpc': '2.0',
-                'id': req_id,
+                'id': next(rpc_id),
                 'method': method,
                 'params': params
             }
             _id = msg['id']
 
-            result = rpc_results[_id] = {
+            rpc_results[_id] = {
                 'result': None,
-                'error': None,
+                'event': trio.Event()
-                'event': trio.Event(),  # signal caller resp arrived
             }
-            req_msgs[_id] = msg
 
             await ws.send_msg(msg)
 
-            # wait for reponse before unblocking requester code
             await rpc_results[_id]['event'].wait()
 
-            if (maybe_result := result['result']):
+            ret = rpc_results[_id]['result']
-                ret = maybe_result
-                del rpc_results[_id]
 
-            else:
+            del rpc_results[_id]
-                err = result['error']
-                raise Exception(
-                    f'JSONRPC request failed\n'
-                    f'req: {msg}\n'
-                    f'resp: {err}\n'
-                )
 
             if ret.error is not None:
                 raise Exception(json.dumps(ret.error, indent=4))
@@ -467,7 +428,6 @@ async def open_jsonrpc_session(
             the server side.
 
             '''
-            nonlocal req_msgs
             async for msg in ws:
                 match msg:
                     case {
@@ -491,28 +451,19 @@ async def open_jsonrpc_session(
                         'params': _,
                     }:
                         log.debug(f'Recieved\n{msg}')
+                        if request_hook:
+                            await request_hook(request_type(**msg))
 
                     case {
                         'error': error
                     }:
-                        # retreive orig request msg, set error
+                        log.warning(f'Recieved\n{error}')
-                        # response in original "result" msg,
+                        if error_hook:
-                        # THEN FINALLY set the event to signal caller
+                            await error_hook(response_type(**msg))
-                        # to raise the error in the parent task.
-                        req_id: int = error['id']
-                        req_msg: dict = req_msgs[req_id]
-                        result: dict = rpc_results[req_id]
-                        result['error'] = error
-                        result['event'].set()
-                        log.error(
-                            f'JSONRPC request failed\n'
-                            f'req: {req_msg}\n'
-                            f'resp: {error}\n'
-                        )
 
                     case _:
                         log.warning(f'Unhandled JSON-RPC msg!?\n{msg}')
 
-        tn.start_soon(recv_task)
+        n.start_soon(recv_task)
         yield json_rpc
-        tn.cancel_scope.cancel()
+        n.cancel_scope.cancel()

piker/storage/cli.py

@@ -386,8 +386,6 @@ def ldshm(
             open_annot_ctl() as actl,
         ):
             shm_df: pl.DataFrame | None = None
-            tf2aids: dict[float, dict] = {}
-
             for (
                 shmfile,
                 shm,
@@ -528,17 +526,16 @@ def ldshm(
                             new_df,
                             step_gaps,
                         )
+
                         # last chance manual overwrites in REPL
-                        # await tractor.pause()
+                        await tractor.pause()
                         assert aids
-                        tf2aids[period_s] = aids
 
                 else:
                     # allow interaction even when no ts problems.
-                    assert not diff
-
                     await tractor.pause()
-            log.info('Exiting TSP shm anal-izer!')
+                    # assert not diff
+
 
             if shm_df is None:
                 log.error(

piker/storage/nativedb.py

@@ -161,13 +161,7 @@ class NativeStorageClient:
 
     def index_files(self):
         for path in self._datadir.iterdir():
-            if (
+            if path.name in {'borked', 'expired',}:
-                path.is_dir()
-                or
-                '.parquet' not in str(path)
-                # or
-                # path.name in {'borked', 'expired',}
-            ):
                 continue
 
             key: str = path.name.rstrip('.parquet')

piker/tsp/__init__.py

@@ -44,10 +44,8 @@ import trio
 from trio_typing import TaskStatus
 import tractor
 from pendulum import (
-    Interval,
     DateTime,
     Duration,
-    duration as mk_duration,
     from_timestamp,
 )
 import numpy as np
@@ -216,8 +214,7 @@ async def maybe_fill_null_segments(
         # pair, immediately stop backfilling?
         if (
             start_dt
-            and
+            and end_dt < start_dt
-            end_dt < start_dt
         ):
             await tractor.pause()
             break
@@ -265,7 +262,6 @@ async def maybe_fill_null_segments(
         except tractor.ContextCancelled:
             # log.exception
             await tractor.pause()
-            raise
 
     null_segs_detected.set()
     # RECHECK for more null-gaps
@@ -353,7 +349,7 @@ async def maybe_fill_null_segments(
 
 async def start_backfill(
     get_hist,
-    def_frame_duration: Duration,
+    frame_types: dict[str, Duration] | None,
     mod: ModuleType,
     mkt: MktPair,
     shm: ShmArray,
@@ -383,23 +379,22 @@ async def start_backfill(
         update_start_on_prepend: bool = False
         if backfill_until_dt is None:
 
-            # TODO: per-provider default history-durations?
+            # TODO: drop this right and just expose the backfill
-            # -[ ] inside the `open_history_client()` config allow
+            # limits inside a [storage] section in conf.toml?
-            #    declaring the history duration limits instead of
+            # when no tsdb "last datum" is provided, we just load
-            #    guessing and/or applying the same limits to all?
+            # some near-term history.
-            #
+            # periods = {
-            # -[ ] allow declaring (default) per-provider backfill
+            #     1: {'days': 1},
-            #     limits inside a [storage] sub-section in conf.toml?
+            #     60: {'days': 14},
-            #
+            # }
-            # NOTE, when no tsdb "last datum" is provided, we just
+
-            # load some near-term history by presuming a "decently
+            # do a decently sized backfill and load it into storage.
-            # large" 60s duration limit and a much shorter 1s range.
             periods = {
                 1: {'days': 2},
                 60: {'years': 6},
             }
             period_duration: int = periods[timeframe]
-            update_start_on_prepend: bool = True
+            update_start_on_prepend = True
 
             # NOTE: manually set the "latest" datetime which we intend to
             # backfill history "until" so as to adhere to the history
@@ -421,6 +416,7 @@ async def start_backfill(
                 f'backfill_until_dt: {backfill_until_dt}\n'
                 f'last_start_dt: {last_start_dt}\n'
             )
+
             try:
                 (
                     array,
@@ -430,114 +426,71 @@ async def start_backfill(
                     timeframe,
                     end_dt=last_start_dt,
                 )
+
             except NoData as _daterr:
-                orig_last_start_dt: datetime = last_start_dt
+                # 3 cases:
-                gap_report: str = (
+                # - frame in the middle of a legit venue gap
-                    f'EMPTY FRAME for `end_dt: {last_start_dt}`?\n'
+                # - history actually began at the `last_start_dt`
-                    f'{mod.name} -> tf@fqme: {timeframe}@{mkt.fqme}\n'
+                # - some other unknown error (ib blocking the
-                    f'last_start_dt: {orig_last_start_dt}\n\n'
+                #   history bc they don't want you seeing how they
-                    f'bf_until: {backfill_until_dt}\n'
+                #   cucked all the tinas..)
+                if dur := frame_types.get(timeframe):
+                    # decrement by a frame's worth of duration and
+                    # retry a few times.
+                    last_start_dt.subtract(
+                        seconds=dur.total_seconds()
                     )
-                # EMPTY FRAME signal with 3 (likely) causes:
+                    log.warning(
-                #
+                        f'{mod.name} -> EMPTY FRAME for end_dt?\n'
-                # 1. range contains legit gap in venue history
+                        f'tf@fqme: {timeframe}@{mkt.fqme}\n'
-                # 2. history actually (edge case) **began** at the
+                        'bf_until <- last_start_dt:\n'
-                #    value `last_start_dt`
+                        f'{backfill_until_dt} <- {last_start_dt}\n'
-                # 3. some other unknown error (ib blocking the
+                        f'Decrementing `end_dt` by {dur} and retry..\n'
-                #    history-query bc they don't want you seeing how
-                #    they cucked all the tinas.. like with options
-                #    hist)
-                #
-                if def_frame_duration:
-                    # decrement by a duration's (frame) worth of time
-                    # as maybe indicated by the backend to see if we
-                    # can get older data before this possible
-                    # "history gap".
-                    last_start_dt: datetime = last_start_dt.subtract(
-                        seconds=def_frame_duration.total_seconds()
                     )
-                    gap_report += (
-                        f'Decrementing `end_dt` and retrying with,\n'
-                        f'def_frame_duration: {def_frame_duration}\n'
-                        f'(new) last_start_dt: {last_start_dt}\n'
-                    )
-                    log.warning(gap_report)
-                    # skip writing to shm/tsdb and try the next
-                    # duration's worth of prior history.
                     continue
 
-                else:
-                    # await tractor.pause()
-                    raise DataUnavailable(gap_report)
-
             # broker says there never was or is no more history to pull
-            except DataUnavailable as due:
+            except DataUnavailable:
-                message: str = due.args[0]
                 log.warning(
-                    f'Provider {mod.name!r} halted backfill due to,\n\n'
+                    f'NO-MORE-DATA in range?\n'
-
+                    f'`{mod.name}` halted history:\n'
-                    f'{message}\n'
+                    f'tf@fqme: {timeframe}@{mkt.fqme}\n'
-
+                    'bf_until <- last_start_dt:\n'
-                    f'fqme: {mkt.fqme}\n'
+                    f'{backfill_until_dt} <- {last_start_dt}\n'
-                    f'timeframe: {timeframe}\n'
-                    f'last_start_dt: {last_start_dt}\n'
-                    f'bf_until: {backfill_until_dt}\n'
                 )
-                # UGH: what's a better way?
+
-                # TODO: backends are responsible for being correct on
+                # ugh, what's a better way?
-                # this right!?
+                # TODO: fwiw, we probably want a way to signal a throttle
-                # -[ ] in the `ib` case we could maybe offer some way
+                # condition (eg. with ib) so that we can halt the
-                #     to halt the request loop until the condition is
+                # request loop until the condition is resolved?
-                #     resolved or should the backend be entirely in
+                if timeframe > 1:
-                #     charge of solving such faults? yes, right?
+                    await tractor.pause()
                 return
 
-            time: np.ndarray = array['time']
             assert (
-                time[0]
+                array['time'][0]
                 ==
                 next_start_dt.timestamp()
             )
 
-            assert time[-1] == next_end_dt.timestamp()
+            diff = last_start_dt - next_start_dt
-
+            frame_time_diff_s = diff.seconds
-            expected_dur: Interval = last_start_dt - next_start_dt
 
             # frame's worth of sample-period-steps, in seconds
             frame_size_s: float = len(array) * timeframe
-            recv_frame_dur: Duration = (
+            expected_frame_size_s: float = frame_size_s + timeframe
-                from_timestamp(array[-1]['time'])
+            if frame_time_diff_s > expected_frame_size_s:
-                -
+
-                from_timestamp(array[0]['time'])
-            )
-            if (
-                (lt_frame := (recv_frame_dur < expected_dur))
-                or
-                (null_frame := (frame_size_s == 0))
-                # ^XXX, should NEVER hit now!
-            ):
                 # XXX: query result includes a start point prior to our
                 # expected "frame size" and thus is likely some kind of
                 # history gap (eg. market closed period, outage, etc.)
                 # so just report it to console for now.
-                if lt_frame:
-                    reason = 'Possible GAP (or first-datum)'
-                else:
-                    assert null_frame
-                    reason = 'NULL-FRAME'
-
-                missing_dur: Interval = expected_dur.end - recv_frame_dur.end
                 log.warning(
-                    f'{timeframe}s-series {reason} detected!\n'
+                    'GAP DETECTED:\n'
-                    f'fqme: {mkt.fqme}\n'
+                    f'last_start_dt: {last_start_dt}\n'
-                    f'last_start_dt: {last_start_dt}\n\n'
+                    f'diff: {diff}\n'
-                    f'recv interval: {recv_frame_dur}\n'
+                    f'frame_time_diff_s: {frame_time_diff_s}\n'
-                    f'expected interval: {expected_dur}\n\n'
-
-                    f'Missing duration of history of {missing_dur.in_words()!r}\n'
-                    f'{missing_dur}\n'
                 )
-                # await tractor.pause()
 
             to_push = diff_history(
                 array,
@@ -612,27 +565,22 @@ async def start_backfill(
             # long-term storage.
             if (
                 storage is not None
-                and
+                and write_tsdb
-                write_tsdb
             ):
                 log.info(
                     f'Writing {ln} frame to storage:\n'
                     f'{next_start_dt} -> {last_start_dt}'
                 )
 
-                # NOTE, always drop the src asset token for
+                # always drop the src asset token for
                 # non-currency-pair like market types (for now)
-                #
-                # THAT IS, for now our table key schema is NOT
-                # including the dst[/src] source asset token. SO,
-                # 'tsla.nasdaq.ib' over 'tsla/usd.nasdaq.ib' for
-                # historical reasons ONLY.
                 if mkt.dst.atype not in {
                     'crypto',
                     'crypto_currency',
                     'fiat',  # a "forex pair"
-                    'perpetual_future',  # stupid "perps" from cex land
                 }:
+                    # for now, our table key schema is not including
+                    # the dst[/src] source asset token.
                     col_sym_key: str = mkt.get_fqme(
                         delim_char='',
                         without_src=True,
@@ -737,7 +685,7 @@ async def back_load_from_tsdb(
         last_tsdb_dt
         and latest_start_dt
     ):
-        backfilled_size_s: Duration = (
+        backfilled_size_s = (
             latest_start_dt - last_tsdb_dt
         ).seconds
         # if the shm buffer len is not large enough to contain
@@ -960,8 +908,6 @@ async def tsdb_backfill(
             f'{pformat(config)}\n'
         )
 
-        # concurrently load the provider's most-recent-frame AND any
-        # pre-existing tsdb history already saved in `piker` storage.
         dt_eps: list[DateTime, DateTime] = []
         async with trio.open_nursery() as tn:
             tn.start_soon(
@@ -972,6 +918,7 @@ async def tsdb_backfill(
                 timeframe,
                 config,
             )
+
             tsdb_entry: tuple = await load_tsdb_hist(
                 storage,
                 mkt,
@@ -1000,25 +947,6 @@ async def tsdb_backfill(
                 mr_end_dt,
             ) = dt_eps
 
-            first_frame_dur_s: Duration = (mr_end_dt - mr_start_dt).seconds
-            calced_frame_size: Duration = mk_duration(
-                seconds=first_frame_dur_s,
-            )
-            # NOTE, attempt to use the backend declared default frame
-            # sizing (as allowed by their time-series query APIs) and
-            # if not provided try to construct a default from the
-            # first frame received above.
-            def_frame_durs: dict[
-                int,
-                Duration,
-            ]|None = config.get('frame_types', None)
-            if def_frame_durs:
-                def_frame_size: Duration = def_frame_durs[timeframe]
-                assert def_frame_size == calced_frame_size
-            else:
-                # use what we calced from first frame above.
-                def_frame_size = calced_frame_size
-
             # NOTE: when there's no offline data, there's 2 cases:
             # - data backend doesn't support timeframe/sample
             #   period (in which case `dt_eps` should be `None` and
@@ -1049,7 +977,7 @@ async def tsdb_backfill(
                     partial(
                         start_backfill,
                         get_hist=get_hist,
-                        def_frame_duration=def_frame_size,
+                        frame_types=config.get('frame_types', None),
                         mod=mod,
                         mkt=mkt,
                         shm=shm,

piker/tsp/_anal.py

@@ -616,18 +616,6 @@ def detect_price_gaps(
     # ])
     ...
 
-# TODO: probably just use the null_segs impl above?
-def detect_vlm_gaps(
-    df: pl.DataFrame,
-    col: str = 'volume',
-
-) -> pl.DataFrame:
-
-    vnull: pl.DataFrame = w_dts.filter(
-        pl.col(col) == 0
-    )
-    return vnull
-
 
 def dedupe(
     src_df: pl.DataFrame,
@@ -638,6 +626,7 @@ def dedupe(
 
 ) -> tuple[
     pl.DataFrame,  # with dts
+    pl.DataFrame,  # gaps
     pl.DataFrame,  # with deduplicated dts (aka gap/repeat removal)
     int,  # len diff between input and deduped
 ]:
@@ -650,22 +639,19 @@ def dedupe(
     '''
     wdts: pl.DataFrame = with_dts(src_df)
 
-    deduped = wdts
-
-    # remove duplicated datetime samples/sections
-    deduped: pl.DataFrame = wdts.unique(
-        # subset=['dt'],
-        subset=['time'],
-        maintain_order=True,
-    )
-
     # maybe sort on any time field
     if sort:
-        deduped = deduped.sort(by='time')
+        wdts = wdts.sort(by='time')
         # TODO: detect out-of-order segments which were corrected!
         # -[ ] report in log msg
         # -[ ] possibly return segment sections which were moved?
 
+    # remove duplicated datetime samples/sections
+    deduped: pl.DataFrame = wdts.unique(
+        subset=['dt'],
+        maintain_order=True,
+    )
+
     diff: int = (
         wdts.height
         -

piker/ui/__init__.py

@@ -14,8 +14,9 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
-'''
+"""
-UI components built using `Qt` with major versions swapped in via
+Stuff for your eyes, aka super hawt Qt UI components.
-the import indirection in the `.qt` sub-mod.
 
-'''
+Currently we only support PyQt5 due to this issue in Pyside2:
+https://bugreports.qt.io/projects/PYSIDE/issues/PYSIDE-1313
+"""

piker/ui/_anchors.py

@@ -21,10 +21,8 @@ Anchor funtions for UI placement of annotions.
 from __future__ import annotations
 from typing import Callable, TYPE_CHECKING
 
-from piker.ui.qt import (
+from PyQt5.QtCore import QPointF
-    QPointF,
+from PyQt5.QtWidgets import QGraphicsPathItem
-    QGraphicsPathItem,
-)
 
 if TYPE_CHECKING:
     from ._chart import ChartPlotWidget

piker/ui/_annotate.py

@@ -20,22 +20,12 @@ Annotations for ur faces.
 """
 from typing import Callable
 
-from pyqtgraph import (
+from PyQt5 import QtCore, QtGui, QtWidgets
-    Point,
+from PyQt5.QtCore import QPointF, QRectF
-    functions as fn,
+from PyQt5.QtWidgets import QGraphicsPathItem
-    Color,
+from pyqtgraph import Point, functions as fn, Color
-)
 import numpy as np
 
-from piker.ui.qt import (
-    QtCore,
-    QtGui,
-    QtWidgets,
-    QPointF,
-    QRectF,
-    QGraphicsPathItem,
-)
-
 
 def mk_marker_path(
 

piker/ui/_app.py

@@ -21,11 +21,9 @@ Main app startup and run.
 from functools import partial
 from types import ModuleType
 
+from PyQt5.QtCore import QEvent
 import trio
 
-from piker.ui.qt import (
-    QEvent,
-)
 from ..service import maybe_spawn_brokerd
 from . import _event
 from ._exec import run_qtractor

piker/ui/_axes.py

@@ -25,16 +25,9 @@ from math import floor
 
 import polars as pl
 import pyqtgraph as pg
+from PyQt5 import QtCore, QtGui, QtWidgets
+from PyQt5.QtCore import QPointF
 
-from piker.ui.qt import (
-    QtCore,
-    QtGui,
-    QtWidgets,
-    QPointF,
-    txt_flag,
-    align_flag,
-    px_cache_mode,
-)
 from . import _pg_overrides as pgo
 from ..accounting._mktinfo import float_digits
 from ._label import Label
@@ -421,15 +414,11 @@ class AxisLabel(pg.GraphicsObject):
         super().__init__()
         self.setParentItem(parent)
 
-        self.setFlag(
+        self.setFlag(self.ItemIgnoresTransformations)
-            self.GraphicsItemFlag.ItemIgnoresTransformations
-        )
         self.setZValue(100)
 
         # XXX: pretty sure this is faster
-        self.setCacheMode(
+        self.setCacheMode(QtWidgets.QGraphicsItem.DeviceCoordinateCache)
-            px_cache_mode.DeviceCoordinateCache
-        )
 
         self._parent = parent
 
@@ -566,14 +555,21 @@ class AxisLabel(pg.GraphicsObject):
 
         return (self.rect.width(), self.rect.height())
 
+# _common_text_flags = (
+#     QtCore.Qt.TextDontClip |
+#     QtCore.Qt.AlignCenter |
+#     QtCore.Qt.AlignTop |
+#     QtCore.Qt.AlignHCenter |
+#     QtCore.Qt.AlignVCenter
+# )
 
 
 class XAxisLabel(AxisLabel):
     _x_margin = 8
 
     text_flags = (
-        align_flag.AlignCenter
+        QtCore.Qt.TextDontClip
-        | txt_flag.TextDontClip
+        | QtCore.Qt.AlignCenter
     )
 
     def size_hint(self) -> tuple[float, float]:
@@ -630,10 +626,10 @@ class YAxisLabel(AxisLabel):
     _y_margin: int = 4
 
     text_flags = (
-        align_flag.AlignLeft
+        QtCore.Qt.AlignLeft
-        | align_flag.AlignVCenter
+        # QtCore.Qt.AlignHCenter
-        # | align_flag.AlignHCenter
+        | QtCore.Qt.AlignVCenter
-        | txt_flag.TextDontClip
+        | QtCore.Qt.TextDontClip
     )
 
     def __init__(

piker/ui/_chart.py

@@ -28,20 +28,22 @@ from typing import (
     TYPE_CHECKING,
 )
 
-import pyqtgraph as pg
+from PyQt5 import QtCore, QtWidgets
-import trio
+from PyQt5.QtCore import (
-
-from piker.ui.qt import (
-    QtCore,
-    QtWidgets,
     Qt,
     QLineF,
+    # QPointF,
+)
+from PyQt5.QtWidgets import (
     QFrame,
     QWidget,
     QHBoxLayout,
     QVBoxLayout,
     QSplitter,
 )
+import pyqtgraph as pg
+import trio
+
 from ._axes import (
     DynamicDateAxis,
     PriceAxis,
@@ -568,8 +570,8 @@ class LinkedSplits(QWidget):
 
         # style?
         self.chart.setFrameStyle(
-            QFrame.Shape.StyledPanel |
+            QFrame.StyledPanel |
-            QFrame.Shadow.Plain
+            QFrame.Plain
         )
 
         return self.chart
@@ -687,8 +689,8 @@ class LinkedSplits(QWidget):
 
         cpw.plotItem.vb.linked = self
         cpw.setFrameStyle(
-            QFrame.Shape.StyledPanel
+            QtWidgets.QFrame.StyledPanel
-            # | QFrame.Shadow.Plain
+            # | QtWidgets.QFrame.Plain
         )
 
         # don't show the little "autoscale" A label.

piker/ui/_cursor.py

@@ -28,14 +28,9 @@ from typing import (
 import inspect
 import numpy as np
 import pyqtgraph as pg
+from PyQt5 import QtCore, QtWidgets
+from PyQt5.QtCore import QPointF, QRectF
 
-from piker.ui.qt import (
-    QPointF,
-    QRectF,
-    QtCore,
-    QtWidgets,
-    px_cache_mode,
-)
 from ._style import (
     _xaxis_at,
     hcolor,
@@ -109,9 +104,7 @@ class LineDot(pg.CurvePoint):
         dot.setParentItem(self)
 
         # keep a static size
-        self.setFlag(
+        self.setFlag(self.ItemIgnoresTransformations)
-            self.GraphicsItemFlag.ItemIgnoresTransformations
-        )
 
     def event(
         self,
@@ -431,10 +424,10 @@ class Cursor(pg.GraphicsObject):
         # vertical and horizonal lines and a y-axis label
 
         vl = plot.addLine(x=0, pen=self.lines_pen, movable=False)
-        vl.setCacheMode(px_cache_mode.DeviceCoordinateCache)
+        vl.setCacheMode(QtWidgets.QGraphicsItem.DeviceCoordinateCache)
 
         hl = plot.addLine(y=0, pen=self.lines_pen, movable=False)
-        hl.setCacheMode(px_cache_mode.DeviceCoordinateCache)
+        hl.setCacheMode(QtWidgets.QGraphicsItem.DeviceCoordinateCache)
         hl.hide()
 
         yl = YAxisLabel(
@@ -518,10 +511,7 @@ class Cursor(pg.GraphicsObject):
             plot=chart
         )
         chart.addItem(cursor)
-        self.graphics[chart].setdefault(
+        self.graphics[chart].setdefault('cursors', []).append(cursor)
-            'cursors',
-            [],
-        ).append(cursor)
         return cursor
 
     def mouseAction(

piker/ui/_curve.py

@@ -19,21 +19,20 @@ Fast, smooth, sexy curves.
 
 """
 from contextlib import contextmanager as cm
-from enum import EnumType
 from typing import Callable
 
 import numpy as np
 import pyqtgraph as pg
-
+from PyQt5 import QtWidgets
-from piker.ui.qt import (
+from PyQt5.QtWidgets import QGraphicsItem
-    QtWidgets,
+from PyQt5.QtCore import (
-    QGraphicsItem,
     Qt,
     QLineF,
     QRectF,
+)
+from PyQt5.QtGui import (
     QPainter,
     QPainterPath,
-    px_cache_mode,
 )
 from ._style import hcolor
 from ..log import get_logger
@@ -43,16 +42,15 @@ from ..toolz.profile import (
     ms_slower_then,
 )
 
+
 log = get_logger(__name__)
 
 
-pen_style: EnumType = Qt.PenStyle
-
 _line_styles: dict[str, int] = {
-    'solid': pen_style.SolidLine,
+    'solid': Qt.PenStyle.SolidLine,
-    'dash': pen_style.DashLine,
+    'dash': Qt.PenStyle.DashLine,
-    'dot': pen_style.DotLine,
+    'dot': Qt.PenStyle.DotLine,
-    'dashdot': pen_style.DashDotLine,
+    'dashdot': Qt.PenStyle.DashDotLine,
 }
 
 
@@ -71,12 +69,12 @@ class FlowGraphic(pg.GraphicsObject):
     # XXX-NOTE-XXX: graphics caching B)
     # see explanation for different caching modes:
     # https://stackoverflow.com/a/39410081
-    cache_mode: int = px_cache_mode.DeviceCoordinateCache
+    cache_mode: int = QGraphicsItem.DeviceCoordinateCache
     # XXX: WARNING item caching seems to only be useful
     # if we don't re-generate the entire QPainterPath every time
     # don't ever use this - it's a colossal nightmare of artefacts
     # and is disastrous for performance.
-    # cache_mode.ItemCoordinateCache
+    # QGraphicsItem.ItemCoordinateCache
     # TODO: still questions todo with coord-cacheing that we should
     # probably talk to a core dev about:
     # - if this makes trasform interactions slower (such as zooming)
@@ -178,7 +176,7 @@ class FlowGraphic(pg.GraphicsObject):
     @cm
     def reset_cache(self) -> None:
         try:
-            none = px_cache_mode.NoCache
+            none = QGraphicsItem.NoCache
             log.debug(
                 f'{self._name} -> CACHE DISABLE: {none}'
             )

piker/ui/_dataviz.py

@@ -40,8 +40,8 @@ from numpy import (
     ndarray,
 )
 import pyqtgraph as pg
+from PyQt5.QtCore import QLineF
 
-from piker.ui.qt import QLineF
 from ..data._sharedmem import (
     ShmArray,
 )

piker/ui/_display.py

@@ -57,7 +57,6 @@ from piker.toolz import (
     Profiler,
 )
 from piker.log import get_logger
-from piker import config
 # from ..data._source import tf_in_1s
 from ._axes import YAxisLabel
 from ._chart import (
@@ -1232,8 +1231,6 @@ async def link_views_with_region(
     # region.sigRegionChangeFinished.connect(update_pi_from_region)
 
 
-# NOTE: default is set to 60 FPS until the runtime delivers the
-# discoverd hw value below.
 _quote_throttle_rate: int = 60 - 6
 
 
@@ -1275,54 +1272,26 @@ async def display_symbol_data(
     # TODO: ctl over update loop's maximum frequency.
     # - load this from a config.toml!
     # - allow dyanmic configuration from chart UI?
-    (
-        conf,
-        path,
-    ) = config.load()
-    ui_conf: dict = conf['ui']
-
     global _quote_throttle_rate
     from ._window import main_window
-
+    display_rate = main_window().current_screen().refreshRate()
-    display_rate: int = floor(
+    _quote_throttle_rate = floor(display_rate) - 6
-        main_window().current_screen().refreshRate()
-    ) - 6
-
-    mx_redraw_rate: int = ui_conf.get(
-        'max_redraw_rate',
-        _quote_throttle_rate,
-    )
-
-    if mx_redraw_rate < display_rate:
-        log.info(
-            'Down-throttling redraw rate to config setting\n'
-            f'display FPS: {display_rate}\n'
-            'max_redraw_rate: {max_redraw_rate}\n'
-        )
-    else:
-        _quote_throttle_rate = display_rate
 
     # TODO: we should be able to increase this if we use some
     # `mypyc` speedups elsewhere? 22ish seems to be the sweet
     # spot for single-feed chart.
     num_of_feeds = len(fqmes)
-    # if num_of_feeds > 1:
+    mx: int = 22
-
+    if num_of_feeds > 1:
         # there will be more ctx switches with more than 1 feed so we
         # max throttle down a bit more.
-    mx_per_feed: int = (
+        mx = 16
-        ui_conf.get(
-        'per_feed_redraw_rate',
-        mx_redraw_rate,
-        )
-        or 16
-    )
 
     # limit to at least display's FPS
     # avoiding needless Qt-in-guest-mode context switches
     cycles_per_feed = min(
         round(_quote_throttle_rate/num_of_feeds),
-        mx_per_feed,
+        mx,
     )
 
     feed: Feed

piker/ui/_editors.py

@@ -32,21 +32,24 @@ from pyqtgraph import (
     QtCore,
     QtWidgets,
 )
+from PyQt5.QtCore import (
+    QPointF,
+    QRectF,
+)
+from PyQt5.QtGui import (
+    QColor,
+    QTransform,
+)
+from PyQt5.QtWidgets import (
+    QGraphicsProxyWidget,
+    QGraphicsScene,
+    QLabel,
+)
 
 from pyqtgraph import functions as fn
 import numpy as np
 
 from piker.types import Struct
-from piker.ui.qt import (
-    Qt,
-    QPointF,
-    QRectF,
-    QGraphicsProxyWidget,
-    QGraphicsScene,
-    QLabel,
-    QColor,
-    QTransform,
-)
 from ._style import (
     hcolor,
     _font,
@@ -313,9 +316,7 @@ class SelectRect(QtWidgets.QGraphicsRectItem):
         self.setZValue(1e9)
 
         label = self._label = QLabel()
-        label.setTextFormat(
+        label.setTextFormat(0)  # markdown
-            Qt.TextFormat.MarkdownText
-        )
         label.setFont(_font.font)
         label.setMargin(0)
         label.setAlignment(

piker/ui/_event.py

@@ -23,29 +23,28 @@ from typing import Callable
 
 import trio
 from tractor.trionics import gather_contexts
-
+from PyQt5 import QtCore
-from piker.ui.qt import (
+from PyQt5.QtCore import QEvent, pyqtBoundSignal
-    QtCore,
+from PyQt5.QtWidgets import QWidget
-    QWidget,
+from PyQt5.QtWidgets import (
-    QEvent,
+    QGraphicsSceneMouseEvent as gs_mouse,
-    keys,
-    gs_keys,
-    pyqtBoundSignal,
 )
+
 from piker.types import Struct
 
 
 MOUSE_EVENTS = {
-    gs_keys.GraphicsSceneMousePress,
+    gs_mouse.GraphicsSceneMousePress,
-    gs_keys.GraphicsSceneMouseRelease,
+    gs_mouse.GraphicsSceneMouseRelease,
-    keys.MouseButtonPress,
+    QEvent.MouseButtonPress,
-    keys.MouseButtonRelease,
+    QEvent.MouseButtonRelease,
     # QtGui.QMouseEvent,
 }
 
 
 # TODO: maybe consider some constrained ints down the road?
 # https://pydantic-docs.helpmanual.io/usage/types/#constrained-types
+
 class KeyboardMsg(Struct):
     '''Unpacked Qt keyboard event data.
 
@@ -115,10 +114,7 @@ class EventRelay(QtCore.QObject):
         # something to do with Qt internals and calling the
         # parent handler?
 
-        if etype in {
+        if etype in {QEvent.KeyPress, QEvent.KeyRelease}:
-            QEvent.Type.KeyPress,
-            QEvent.Type.KeyRelease,
-        }:
 
             msg = KeyboardMsg(
                 event=ev,
@@ -164,9 +160,7 @@ class EventRelay(QtCore.QObject):
 async def open_event_stream(
 
     source_widget: QWidget,
-    event_types: set[QEvent] = {
+    event_types: set[QEvent] = {QEvent.KeyPress},
-        QEvent.Type.KeyPress,
-    },
     filter_auto_repeats: bool = True,
 
 ) -> trio.abc.ReceiveChannel:

piker/ui/_exec.py

@@ -30,22 +30,25 @@ from typing import (
 import platform
 import traceback
 
+# Qt specific
+import PyQt5  # noqa
+from PyQt5.QtWidgets import (
+    QWidget,
+    QMainWindow,
+    QApplication,
+)
+from PyQt5 import QtCore
+from PyQt5.QtCore import (
+    pyqtRemoveInputHook,
+    Qt,
+    QCoreApplication,
+)
 import qdarkstyle
 from qdarkstyle import DarkPalette
 # import qdarkgraystyle  # TODO: play with it
 import trio
 from outcome import Error
 
-# Qt version-agnostic
-from .qt import (
-    QWidget,
-    QMainWindow,
-    QApplication,
-    QtCore,
-    pyqtRemoveInputHook,
-    Qt,
-    QCoreApplication,
-)
 from ..service import (
     maybe_open_pikerd,
     get_runtime_vars,
@@ -147,7 +150,7 @@ def run_qtractor(
 
     # load dark theme
     stylesheet = qdarkstyle.load_stylesheet(
-        qt_api='pyqt6',
+        qt_api='pyqt5',
         palette=DarkPalette,
     )
     app.setStyleSheet(stylesheet)

piker/ui/_forms.py

@@ -28,15 +28,9 @@ from typing import (
 )
 
 import trio
-
+from PyQt5 import QtGui
-from piker.ui.qt import (
+from PyQt5.QtCore import QSize, QModelIndex, Qt, QEvent
-    keys,
+from PyQt5.QtWidgets import (
-    size_policy,
-    QtGui,
-    QSize,
-    QModelIndex,
-    Qt,
-    QEvent,
     QWidget,
     QLabel,
     QComboBox,
@@ -45,6 +39,7 @@ from piker.ui.qt import (
     QVBoxLayout,
     QFormLayout,
     QProgressBar,
+    QSizePolicy,
     QStyledItemDelegate,
     QStyleOptionViewItem,
 )
@@ -76,14 +71,14 @@ class Edit(QLineEdit):
 
         if width_in_chars:
             self._chars = int(width_in_chars)
-            x_size_policy = size_policy.Fixed
+            x_size_policy = QSizePolicy.Fixed
 
         else:
             # chart count which will be used to calculate
             # width of input field.
             self._chars: int = 6
             # fit to surroundingn frame width
-            x_size_policy = size_policy.Expanding
+            x_size_policy = QSizePolicy.Expanding
 
         super().__init__(parent)
 
@@ -91,7 +86,7 @@ class Edit(QLineEdit):
         # https://doc.qt.io/qt-5/qsizepolicy.html#Policy-enum
         self.setSizePolicy(
             x_size_policy,
-            size_policy.Fixed,
+            QSizePolicy.Fixed,
         )
         self.setFont(font.font)
 
@@ -185,13 +180,11 @@ class Selection(QComboBox):
 
         self._items: dict[str, int] = {}
         super().__init__(parent=parent)
-        self.setSizeAdjustPolicy(
+        self.setSizeAdjustPolicy(QComboBox.AdjustToContents)
-            QComboBox.SizeAdjustPolicy.AdjustToContents,
-        )
         # make line edit expand to surrounding frame
         self.setSizePolicy(
-            size_policy.Expanding,
+            QSizePolicy.Expanding,
-            size_policy.Fixed,
+            QSizePolicy.Fixed,
         )
         view = self.view()
         view.setUniformItemSizes(True)
@@ -315,8 +308,8 @@ class FieldsForm(QWidget):
 
         # size it as we specify
         self.setSizePolicy(
-            size_policy.Expanding,
+            QSizePolicy.Expanding,
-            size_policy.Expanding,
+            QSizePolicy.Expanding,
         )
 
         # XXX: not sure why we have to create this here exactly
@@ -423,8 +416,8 @@ class FieldsForm(QWidget):
         select.set_items(values)
 
         self.setSizePolicy(
-            size_policy.Fixed,
+            QSizePolicy.Fixed,
-            size_policy.Fixed,
+            QSizePolicy.Fixed,
         )
         select.show()
         self.form.addRow(label, select)
@@ -444,10 +437,7 @@ async def handle_field_input(
 
     async for kbmsg in recv_chan:
 
-        if kbmsg.etype in {
+        if kbmsg.etype in {QEvent.KeyPress, QEvent.KeyRelease}:
-            keys.KeyPress,
-            keys.KeyRelease,
-        }:
             event, etype, key, mods, txt = kbmsg.to_tuple()
             print(f'key: {kbmsg.key}, mods: {kbmsg.mods}, txt: {kbmsg.txt}')
 
@@ -713,8 +703,7 @@ def mk_fill_status_bar(
     )
 
     bottom_label = form.add_field_label(
-        # 'x: {step_size}',
+        'x: {step_size}',
-       '{unit_prefix}: {step_size}',
         font_size=bar_label_font_size,
         font_color='gunmetal',
     )

piker/ui/_icons.py

@@ -15,18 +15,15 @@
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 '''
-`QIcon` hackery.
+``QIcon`` hackery.
-
-Mostly dynamically loading pixmaps for use with `QGraphicsScene`.
 
 '''
-from piker.ui.qt import (
+from PyQt5.QtWidgets import QStyle
-    QSize,
+from PyQt5.QtGui import (
-    QStyle,
+    QIcon, QPixmap, QColor
-    QIcon,
-    QPixmap,
-    QColor,
 )
+from PyQt5.QtCore import QSize
+
 from ._style import hcolor
 
 # https://www.pythonguis.com/faq/built-in-qicons-pyqt/
@@ -47,8 +44,7 @@ def mk_icons(
     size: QSize,
 
 ) -> dict[str, QIcon]:
-    '''
+    '''This helper is indempotent.
-    This helper is indempotent.
 
     '''
     global _icons, _icon_names
@@ -60,11 +56,7 @@ def mk_icons(
     # load account selection using current style
     for name, icon_name in _icon_names.items():
 
-        stdpixmap = getattr(
+        stdpixmap = getattr(QStyle, icon_name)
-            # https://www.pythonguis.com/faq/built-in-qicons-pyqt/
-            QStyle.StandardPixmap,  # pyqt/pyside6
-            icon_name,
-        )
         stdicon = style.standardIcon(stdpixmap)
         pixmap = stdicon.pixmap(size)
 

piker/ui/_interaction.py

@@ -36,21 +36,23 @@ import pyqtgraph as pg
 # this down the road.. Bo
 from pyqtgraph.GraphicsScene import mouseEvents as mevs
 # from pyqtgraph.GraphicsScene.mouseEvents import MouseDragEvent
+from PyQt5.QtWidgets import QGraphicsSceneMouseEvent as gs_mouse
+from PyQt5.QtGui import (
+    QWheelEvent,
+)
+from PyQt5.QtCore import (
+    Qt,
+    QEvent,
+)
 from pyqtgraph import (
     ViewBox,
     Point,
     QtCore,
-    functions as fn,
 )
+from pyqtgraph import functions as fn
 import numpy as np
 import trio
 
-from piker.ui.qt import (
-    QWheelEvent,
-    QGraphicsSceneMouseEvent as gs_mouse,
-    Qt,
-    QEvent,
-)
 from ..log import get_logger
 from ..toolz import (
     Profiler,
@@ -79,22 +81,22 @@ if TYPE_CHECKING:
 log = get_logger(__name__)
 
 NUMBER_LINE = {
-    Qt.Key.Key_1,
+    Qt.Key_1,
-    Qt.Key.Key_2,
+    Qt.Key_2,
-    Qt.Key.Key_3,
+    Qt.Key_3,
-    Qt.Key.Key_4,
+    Qt.Key_4,
-    Qt.Key.Key_5,
+    Qt.Key_5,
-    Qt.Key.Key_6,
+    Qt.Key_6,
-    Qt.Key.Key_7,
+    Qt.Key_7,
-    Qt.Key.Key_8,
+    Qt.Key_8,
-    Qt.Key.Key_9,
+    Qt.Key_9,
-    Qt.Key.Key_0,
+    Qt.Key_0,
 }
 
 ORDER_MODE = {
-    Qt.Key.Key_A,
+    Qt.Key_A,
-    Qt.Key.Key_F,
+    Qt.Key_F,
-    Qt.Key.Key_D,
+    Qt.Key_D,
 }
 
 

piker/ui/_l1.py

@@ -21,12 +21,9 @@ Double auction top-of-book (L1) graphics.
 from typing import Tuple
 
 import pyqtgraph as pg
+from PyQt5 import QtCore, QtGui
+from PyQt5.QtCore import QPointF
 
-from piker.ui.qt import (
-    QPointF,
-    QtCore,
-    QtGui,
-)
 from ._axes import YAxisLabel
 from ._style import hcolor
 from ._pg_overrides import PlotItem

piker/ui/_label.py

@@ -25,17 +25,10 @@ from typing import (
 )
 
 import pyqtgraph as pg
+from PyQt5 import QtGui, QtWidgets
+from PyQt5.QtWidgets import QLabel, QSizePolicy
+from PyQt5.QtCore import QPointF, QRectF, Qt
 
-from piker.ui.qt import (
-    px_cache_mode,
-    QtGui,
-    QtWidgets,
-    QLabel,
-    size_policy,
-    QPointF,
-    QRectF,
-    Qt,
-)
 from ._style import (
     DpiAwareFont,
     hcolor,
@@ -85,7 +78,7 @@ class Label:
         self._x_offset = x_offset
 
         txt = self.txt = QtWidgets.QGraphicsTextItem(parent=parent)
-        txt.setCacheMode(px_cache_mode.DeviceCoordinateCache)
+        txt.setCacheMode(QtWidgets.QGraphicsItem.DeviceCoordinateCache)
 
         vb.scene().addItem(txt)
 
@@ -110,7 +103,7 @@ class Label:
         self._anchor_func = self.txt.pos().x
 
         # not sure if this makes a diff
-        self.txt.setCacheMode(px_cache_mode.DeviceCoordinateCache)
+        self.txt.setCacheMode(QtWidgets.QGraphicsItem.DeviceCoordinateCache)
 
         # TODO: edit and selection support
         # https://doc.qt.io/qt-5/qt.html#TextInteractionFlag-enum
@@ -306,14 +299,12 @@ class FormatLabel(QLabel):
             """
         )
         self.setFont(_font.font)
-        self.setTextFormat(
+        self.setTextFormat(Qt.MarkdownText)  # markdown
-            Qt.TextFormat.MarkdownText
-        )
         self.setMargin(0)
 
         self.setSizePolicy(
-            size_policy.Expanding,
+            QSizePolicy.Expanding,
-            size_policy.Expanding,
+            QSizePolicy.Expanding,
         )
         self.setAlignment(
             Qt.AlignVCenter | Qt.AlignLeft

piker/ui/_lines.py

@@ -27,22 +27,20 @@ from typing import (
 )
 
 import pyqtgraph as pg
-from pyqtgraph import (
+from pyqtgraph import Point, functions as fn
-    Point,
+from PyQt5 import (
-    functions as fn,
-)
-
-from piker.ui.qt import (
-    px_cache_mode,
     QtCore,
     QtGui,
+)
+from PyQt5.QtWidgets import (
     QGraphicsPathItem,
     QStyleOptionGraphicsItem,
     QGraphicsItem,
     QGraphicsScene,
     QWidget,
-    QPointF,
 )
+from PyQt5.QtCore import QPointF
+
 from ._annotate import LevelMarker
 from ._anchors import (
     vbr_left,
@@ -142,9 +140,7 @@ class LevelLine(pg.InfiniteLine):
         self._right_end_sc: float = 0
 
         # use px caching
-        self.setCacheMode(
+        self.setCacheMode(QGraphicsItem.DeviceCoordinateCache)
-            px_cache_mode.DeviceCoordinateCache
-        )
 
     def txt_offsets(self) -> tuple[int, int]:
         return 0, 0
@@ -215,7 +211,7 @@ class LevelLine(pg.InfiniteLine):
     ) -> None:
 
         if not called_from_on_pos_change:
-            last: float = self.value()
+            last = self.value()
 
             # if the position hasn't changed then ``.update_labels()``
             # will not be called by a non-triggered `.on_pos_change()`,

piker/ui/_ohlc.py

@@ -20,14 +20,16 @@ Super fast OHLC sampling graphics types.
 from __future__ import annotations
 
 import numpy as np
-
+from PyQt5 import (
-from piker.ui.qt import (
     QtGui,
     QtWidgets,
-    QPainterPath,
+)
+from PyQt5.QtCore import (
     QLineF,
     QRectF,
 )
+from PyQt5.QtGui import QPainterPath
+
 from ._curve import FlowGraphic
 from ..toolz import (
     Profiler,

piker/ui/_position.py

@@ -344,10 +344,7 @@ class SettingsPane:
         dsize = tracker.live_pp.dsize
 
         # READ out settings and update the status UI / settings widgets
-        unit_char: str = {
+        suffix = {'currency': ' $', 'units': ' u'}[alloc.size_unit]
-            'currency': '$',
-            'units': 'u',
-        }[alloc.size_unit]
         size_unit, limit = alloc.limit_info()
 
         step_size, currency_per_slot = alloc.step_sizes()
@@ -361,11 +358,10 @@ class SettingsPane:
             self.apply_setting('limit', limit)
 
         self.step_label.format(
-            unit_prefix=unit_char,
+            step_size=str(humanize(step_size)) + suffix
-            step_size=str(humanize(step_size))
         )
         self.limit_label.format(
-            limit=f'{unit_char}: {str(humanize(limit))}'
+            limit=str(humanize(limit)) + suffix
         )
 
         # update size unit in UI

piker/ui/_remote_ctl.py

@@ -38,14 +38,14 @@ from tractor import (
     Context,
     MsgStream,
 )
+from PyQt5.QtWidgets import (
+    QGraphicsItem,
+)
 
 from piker.log import get_logger
 from piker.types import Struct
 from piker.service import find_service
 from piker.brokers import SymbolNotFound
-from piker.ui.qt import (
-    QGraphicsItem,
-)
 from ._display import DisplayState
 from ._interaction import ChartView
 from ._editors import SelectRect

piker/ui/_render.py

@@ -30,8 +30,8 @@ from typing import (
 import msgspec
 import numpy as np
 import pyqtgraph as pg
+from PyQt5.QtGui import QPainterPath
 
-from piker.ui.qt import QPainterPath
 from ..data._formatters import (
     IncrementalFormatter,
 )

piker/ui/_search.py

@@ -48,24 +48,27 @@ from pprint import pformat
 from rapidfuzz import process as fuzzy
 import trio
 from trio_typing import TaskStatus
-
+from PyQt5 import QtCore
-from piker.ui.qt import (
+from PyQt5 import QtWidgets
-    size_policy,
+from PyQt5.QtCore import (
-    align_flag,
     Qt,
-    QtCore,
-    QtWidgets,
     QModelIndex,
     QItemSelectionModel,
+)
+from PyQt5.QtGui import (
     # QLayout,
     QStandardItem,
     QStandardItemModel,
+)
+from PyQt5.QtWidgets import (
     QWidget,
     QTreeView,
     # QListWidgetItem,
     # QAbstractScrollArea,
     # QStyledItemDelegate,
 )
+
+
 from ..log import get_logger
 from ._style import (
     _font,
@@ -126,8 +129,8 @@ class CompleterView(QTreeView):
 
         # ux settings
         self.setSizePolicy(
-            size_policy.Expanding,
+            QtWidgets.QSizePolicy.Expanding,
-            size_policy.Expanding,
+            QtWidgets.QSizePolicy.Expanding,
         )
         self.setItemsExpandable(True)
         self.setExpandsOnDoubleClick(False)
@@ -564,8 +567,8 @@ class SearchWidget(QtWidgets.QWidget):
 
         # size it as we specify
         self.setSizePolicy(
-            size_policy.Fixed,
+            QtWidgets.QSizePolicy.Fixed,
-            size_policy.Fixed,
+            QtWidgets.QSizePolicy.Fixed,
         )
 
         self.godwidget = godwidget
@@ -589,16 +592,14 @@ class SearchWidget(QtWidgets.QWidget):
             }}
             """
         )
-        label.setTextFormat(
+        label.setTextFormat(3)  # markdown
-            Qt.TextFormat.MarkdownText
-        )
         label.setFont(_font.font)
         label.setMargin(4)
         label.setText("search:")
         label.show()
         label.setAlignment(
-            align_flag.AlignVCenter
+            QtCore.Qt.AlignVCenter
-            | align_flag.AlignLeft
+            | QtCore.Qt.AlignLeft
         )
 
         self.bar_hbox.addWidget(label)
@@ -616,17 +617,9 @@ class SearchWidget(QtWidgets.QWidget):
 
         self.vbox.addLayout(self.bar_hbox)
 
-        self.vbox.setAlignment(
+        self.vbox.setAlignment(self.bar, Qt.AlignTop | Qt.AlignRight)
-            self.bar,
-            align_flag.AlignTop
-            | align_flag.AlignRight,
-        )
         self.vbox.addWidget(self.bar.view)
-        self.vbox.setAlignment(
+        self.vbox.setAlignment(self.view, Qt.AlignTop | Qt.AlignLeft)
-            self.view,
-            align_flag.AlignTop
-            | align_flag.AlignLeft,
-        )
 
     def focus(self) -> None:
         self.show()

piker/ui/_style.py

@@ -22,14 +22,10 @@ from typing import Dict
 import math
 
 import pyqtgraph as pg
+from PyQt5 import QtCore, QtGui
+from PyQt5.QtCore import Qt, QCoreApplication
 from qdarkstyle import DarkPalette
 
-from .qt import (
-    QtCore,
-    QtGui,
-    Qt,
-    QCoreApplication,
-)
 from ..log import get_logger
 
 from .. import config

piker/ui/_window.py

@@ -27,14 +27,16 @@ from typing import (
 )
 import uuid
 
-from piker.ui.qt import (
+from PyQt5 import QtCore
-    Qt,
+from PyQt5.QtWidgets import (
-    QtCore,
     QWidget,
     QMainWindow,
     QApplication,
     QLabel,
     QStatusBar,
+)
+
+from PyQt5.QtGui import (
     QScreen,
     QCloseEvent,
 )
@@ -195,9 +197,7 @@ class MainWindow(QMainWindow):
                 """
                 # font-size : {font_size}px;
             )
-            label.setTextFormat(
+            label.setTextFormat(3)  # markdown
-                Qt.TextFormat.MarkdownText
-            )
             label.setFont(_font_small.font)
             label.setMargin(2)
             label.setAlignment(

piker/ui/order_mode.py

@@ -34,6 +34,7 @@ import uuid
 from bidict import bidict
 import tractor
 import trio
+from PyQt5.QtCore import Qt
 
 from piker import config
 from piker.accounting import (
@@ -58,7 +59,6 @@ from piker.data import (
 )
 from piker.types import Struct
 from piker.log import get_logger
-from piker.ui.qt import Qt
 from ._editors import LineEditor, ArrowEditor
 from ._lines import order_line, LevelLine
 from ._position import (
@@ -494,7 +494,7 @@ class OrderMode:
         uuid: str,
         order: Order | None = None,
 
-    ) -> Dialog | None:
+    ) -> Dialog:
         '''
         Order submitted status event handler.
 
@@ -515,11 +515,6 @@ class OrderMode:
             # if an order msg is provided update the line
             # **from** that msg.
             if order:
-                if order.price <= 0:
-                    log.error(f'Order has 0 price, cancelling..\n{order}')
-                    self.cancel_orders([order.oid])
-                    return None
-
                 line.set_level(order.price)
                 self.on_level_change_update_next_order_info(
                     level=order.price,
@@ -1018,13 +1013,8 @@ async def process_trade_msg(
 
 ) -> tuple[Dialog, Status]:
 
-    # TODO: obvi once we're parsing to native struct instances we can
+    fmsg = pformat(msg)
-    # drop the `pformat()` call Bo
+    log.debug(f'Received order msg:\n{fmsg}')
-    fmtmsg: Struct | dict = msg
-    if not isinstance(msg, Struct):
-        fmtmsg: str = pformat(msg)
-
-    log.debug(f'Received order msg:\n{fmtmsg}')
     name = msg['name']
 
     if name in (
@@ -1040,7 +1030,7 @@ async def process_trade_msg(
         ):
             log.info(
                 f'Loading position for `{fqme}`:\n'
-                f'{fmtmsg}'
+                f'{fmsg}'
             )
             tracker = mode.trackers[msg['account']]
             tracker.live_pp.update_from_msg(msg)
@@ -1082,7 +1072,7 @@ async def process_trade_msg(
 
             elif order.action != 'cancel':
                 log.warning(
-                    f'received msg for untracked dialog:\n{fmtmsg}'
+                    f'received msg for untracked dialog:\n{fmsg}'
                 )
                 assert msg.resp in ('open', 'dark_open'), f'Unknown msg: {msg}'
 
@@ -1149,7 +1139,7 @@ async def process_trade_msg(
             req={'exec_mode': 'dark'},
         ):
             # TODO: UX for a "pending" clear/live order
-            log.info(f'Dark order triggered for {fmtmsg}')
+            log.info(f'Dark order triggered for {fmsg}')
 
         case Status(
             resp='triggered',

piker/ui/qt.py

@@ -1,104 +0,0 @@
-# piker: trading gear for hackers
-# Copyright (C) Tyler Goodlet (in stewardship for pikers)
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-Qt UI framework version shimming.
-
-Allow importing sub-pkgs from this module instead of worrying about
-major version specifics, any enum moves or component renames.
-
-Code in `piker.ui.*` should always explicitlyimport directly from
-this module like `from piker.ui.qt import ( ..`
-
-'''
-from enum import EnumType
-
-from PyQt6 import (
-    QtCore,
-    QtGui,
-    QtWidgets,
-)
-from PyQt6.QtCore import (
-    Qt,
-    QCoreApplication,
-    QLineF,
-    QRectF,
-    # NOTE: for enums use the `.Type`  subattr-space
-    QEvent,
-    QPointF,
-    QSize,
-    QModelIndex,
-    QItemSelectionModel,
-    pyqtBoundSignal,
-    pyqtRemoveInputHook,
-)
-
-align_flag: EnumType = Qt.AlignmentFlag
-txt_flag: EnumType = Qt.TextFlag
-keys: EnumType = QEvent.Type
-scrollbar_policy: EnumType = Qt.ScrollBarPolicy
-
-# ^-NOTE-^: handy snippet to discover enums:
-# import enum
-# [attr for attr_name in dir(QFrame)
-#  if (attr := getattr(QFrame, attr_name))
-#  and isinstance(attr, enum.EnumType)]
-
-from PyQt6.QtGui import (
-    QPainter,
-    QPainterPath,
-    QIcon,
-    QPixmap,
-    QColor,
-    QTransform,
-    QStandardItem,
-    QStandardItemModel,
-    QWheelEvent,
-    QScreen,
-    QCloseEvent,
-)
-
-from PyQt6.QtWidgets import (
-    QMainWindow,
-    QApplication,
-    QLabel,
-    QStatusBar,
-    QLineEdit,
-    QHBoxLayout,
-    QVBoxLayout,
-    QFormLayout,
-    QProgressBar,
-    QSizePolicy,
-    QStyledItemDelegate,
-    QStyleOptionViewItem,
-    QComboBox,
-    QWidget,
-    QFrame,
-    QSplitter,
-    QTreeView,
-    QStyle,
-    QGraphicsItem,
-    QGraphicsPathItem,
-    # QGraphicsView,
-    QStyleOptionGraphicsItem,
-    QGraphicsScene,
-    QGraphicsSceneMouseEvent,
-    QGraphicsProxyWidget,
-)
-
-gs_keys: EnumType = QGraphicsSceneMouseEvent.Type
-size_policy: EnumType = QtWidgets.QSizePolicy.Policy
-px_cache_mode: EnumType = QGraphicsItem.CacheMode

pyproject.toml

@@ -15,119 +15,140 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 [build-system]
-requires = ["hatchling"]
+requires = ["poetry-core"]
-build-backend = "hatchling.build"
+build-backend = "poetry.core.masonry.api"
 
-[project]
+# ------ - ------
+
+[tool.poetry]
 name = "piker"
-version = "0.1.0a0dev0"
+version = "0.1.0.alpha0.dev0"
 description = "trading gear for hackers"
-authors = [{ name = "Tyler Goodlet", email = "goodboy_foss@protonmail.com" }]
+authors = ["Tyler Goodlet <jgbt@protonmail.com>"]
-requires-python = ">=3.12, <3.13"
+license = "AGPLv3"
-license = "AGPL-3.0-or-later"
 readme = "README.rst"
-keywords = [
-    "async",
-    "trading",
-    "finance",
-    "quant",
-    "charting",
-]
-classifiers = [
-    "Development Status :: 3 - Alpha",
-    "License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
-    "Operating System :: POSIX :: Linux",
-    "Programming Language :: Python :: Implementation :: CPython",
-    "Programming Language :: Python :: 3 :: Only",
-    "Programming Language :: Python :: 3.11",
-    "Programming Language :: Python :: 3.12",
-    "Intended Audience :: Financial and Insurance Industry",
-    "Intended Audience :: Science/Research",
-    "Intended Audience :: Developers",
-    "Intended Audience :: Education",
-]
-dependencies = [
-    "async-generator >=1.10, <2.0.0",
-    "attrs >=23.1.0, <24.0.0",
-    "bidict >=0.22.1, <0.23.0",
-    "colorama >=0.4.6, <0.5.0",
-    "colorlog >=6.7.0, <7.0.0",
-    "ib-insync >=0.9.86, <0.10.0",
-    "numba >=0.59.0, <0.60.0",
-    "numpy >=1.25, <2.0",
-    "polars >=0.18.13, <0.19.0",
-    "pygments >=2.16.1, <3.0.0",
-    "rich >=13.5.2, <14.0.0",
-    "tomli >=2.0.1, <3.0.0",
-    "tomli-w >=1.0.0, <2.0.0",
-    "trio-util >=0.7.0, <0.8.0",
-    "trio-websocket >=0.10.3, <0.11.0",
-    "typer >=0.9.0, <1.0.0",
-    "rapidfuzz >=3.5.2, <4.0.0",
-    "pdbp >=1.5.0, <2.0.0",
-    "trio >=0.24, <0.25",
-    "pendulum >=3.0.0, <4.0.0",
-    "httpx >=0.27.0, <0.28.0",
-    "cryptofeed >=2.4.0, <3.0.0",
-    "pyarrow >=17.0.0, <18.0.0",
-    "websockets ==12.0",
-    "msgspec",
-    "tractor",
-    "asyncvnc",
-    "tomlkit",
-]
 
-[project.optional-dependencies]
+# TODO: add meta-data from setup.py
-uis = [
+# keywords=[
-    # https://docs.astral.sh/uv/concepts/projects/dependencies/#optional-dependencies
+#     "async",
+#     "trading",
+#     "finance",
+#     "quant",
+#     "charting",
+# ],
+# classifiers=[
+#     'Development Status :: 3 - Alpha',
+#     'License :: OSI Approved :: ',
+#     'Operating System :: POSIX :: Linux',
+#     "Programming Language :: Python :: Implementation :: CPython",
+#     "Programming Language :: Python :: 3 :: Only",
+#     "Programming Language :: Python :: 3.10",
+#     "Programming Language :: Python :: 3.11",
+#     'Intended Audience :: Financial and Insurance Industry',
+#     'Intended Audience :: Science/Research',
+#     'Intended Audience :: Developers',
+#     'Intended Audience :: Education',
+# ],
+
+# ------ - ------
+
+[tool.poetry.dependencies]
+asks = "^3.0.0"
+async-generator = "^1.10"
+attrs = "^23.1.0"
+bidict = "^0.22.1"
+colorama = "^0.4.6"
+colorlog = "^6.7.0"
+cython = "^3.0.0"
+greenback = "^1.1.1"
+ib-insync = "^0.9.86"
+msgspec = "^0.18.0"
+numba = "^0.57.1"
+numpy = "1.24"
+pendulum = "^2.1.2"
+polars = "^0.18.13"
+pygments = "^2.16.1"
+python = "^3.10"
+rich = "^13.5.2"
+# setuptools = "^68.0.0"
+tomli = "^2.0.1"
+tomli-w = "^1.0.0"
+trio = "^0.22.2"
+trio-util = "^0.7.0"
+trio-websocket = "^0.10.3"
+typer = "^0.9.0"
+
+
+[tool.poetry.dependencies.asyncvnc]
+git = 'https://github.com/pikers/asyncvnc.git'
+branch = 'main'
+
+[tool.poetry.dependencies.tomlkit]
+git = 'https://github.com/pikers/tomlkit.git'
+branch = 'piker_pin'
+develop = true
+# path = "../tomlkit/"
+
+[tool.poetry.dependencies.tractor]
+git = 'https://github.com/goodboy/tractor.git'
+branch = 'multihomed'
+# branch = 'piker_pin'
+develop = true
+# path = '../tractor/'
+
+# ------ - ------
+
+[tool.poetry.group.uis]
+optional = true
+[tool.poetry.group.uis.dependencies]
+# https://python-poetry.org/docs/managing-dependencies/#dependency-groups
 # TODO: make sure the levenshtein shit compiles on nix..
 # rapidfuzz = {extras = ["speedup"], version = "^0.18.0"}
-    "rapidfuzz >=3.2.0, <4.0.0",
+rapidfuzz = "^3.2.0"
-    "qdarkstyle >=3.0.2, <4.0.0",
+qdarkstyle = ">=3.0.2"
-    "pyqt6 >=6.7.0, <7.0.0",
+pyqt5 = "^5.15.9"
-    "pyqtgraph",
+pyqtgraph = { git = 'https://github.com/pikers/pyqtgraph.git' }
+# pyqt6 = "^6.5.2"
 
-    # for consideration,
+# ------ - ------
-    # - 'visidata'
 
-    # TODO: add an `--only daemon` group for running non-ui / pikerd
+# only diference with uis group is on nix we use some sys py packages
-    # service tree in distributed mode B)
+[tool.poetry.group.nix-shell]
-    # https://docs.astral.sh/uv/concepts/projects/dependencies/#optional-dependencies
+optional = true
-]
+[tool.poetry.group.nix-shell.dependencies]
+# rapidfuzz = "^3.2.0"
+# qdarkstyle = ">=3.0.2"
+# pyqt5 = "^5.15.9"
+pyqtgraph = { git = 'https://github.com/pikers/pyqtgraph.git' }
+# pyqt6 = "^6.5.2"
 
-[dependency-groups]
+# ------ - ------
-# TODO: a toolset that makes debugging a `pikerd` service (tree) easy
+
-# to hack on directly using more or less the local env:
+[tool.poetry.group.dev]
+optional = true
+[tool.poetry.group.dev.dependencies]
+# testing / CI
+pytest = "^6.0.0"
+elasticsearch = "^8.9.0"
+
+# console ehancements and eventually remote debugging
+# extras/helpers.
+# TODO: add a toolset that makes debugging a `pikerd` service
+# (tree) easy to hack on directly using more or less the local env:
 # - xonsh + xxh
 # - rsyscall + pdbp
 # - actor runtime control console like BEAM/OTP
-#
+xonsh = "^0.14.0"  # XXX: explicit env install for shell use w nix
-# console ehancements and eventually remote debugging extras/helpers.
+prompt-toolkit = "^3.0.39"
-# use `uv --dev` to enable
-dev = [
-    "pytest >=6.0.0, <7.0.0",
-    "elasticsearch >=8.9.0, <9.0.0",
-    "xonsh >=0.14.2, <0.15.0",
-    "prompt-toolkit ==3.0.40",
-    "cython >=3.0.0, <4.0.0",
-    "greenback >=1.1.1, <2.0.0",
-    "ruff>=0.9.6",
-]
 
-[project.scripts]
+# ------ - ------
-piker = "piker.cli:cli"
-pikerd = "piker.cli:pikerd"
-ledger = "piker.accounting.cli:ledger"
 
-[tool.hatch.build.targets.sdist]
+# TODO: add an `--only daemon` group for running non-ui / pikerd
-include = ["piker"]
+# service tree in distributed mode B)
+# https://python-poetry.org/docs/managing-dependencies/#installing-group-dependencies
+# [tool.poetry.group.daemon.dependencies]
 
-[tool.hatch.build.targets.wheel]
+[tool.poetry.scripts]
-include = ["piker"]
+piker = 'piker.cli:cli'
-
+pikerd = 'piker.cli:pikerd'
-[tool.uv.sources]
+ledger = 'piker.accounting.cli:ledger'
-pyqtgraph = { git = "https://github.com/pikers/pyqtgraph.git" }
-asyncvnc = { git = "https://github.com/pikers/asyncvnc.git", branch = "main" }
-tomlkit = { git = "https://github.com/pikers/tomlkit.git", branch ="piker_pin" }
-msgspec = { git = "https://github.com/jcrist/msgspec.git" }
-tractor = { path = "../tractor", editable = true }

ruff.toml

@@ -1,93 +0,0 @@
-# from default `ruff.toml` @
-# https://docs.astral.sh/ruff/configuration/
-
-# Exclude a variety of commonly ignored directories.
-exclude = [
-    ".bzr",
-    ".direnv",
-    ".eggs",
-    ".git",
-    ".git-rewrite",
-    ".hg",
-    ".ipynb_checkpoints",
-    ".mypy_cache",
-    ".nox",
-    ".pants.d",
-    ".pyenv",
-    ".pytest_cache",
-    ".pytype",
-    ".ruff_cache",
-    ".svn",
-    ".tox",
-    ".venv",
-    ".vscode",
-    "__pypackages__",
-    "_build",
-    "buck-out",
-    "build",
-    "dist",
-    "node_modules",
-    "site-packages",
-    "venv",
-]
-
-# Same as Black.
-line-length = 88
-indent-width = 4
-
-# Assume Python 3.9
-target-version = "py312"
-
-# ------ - ------
-# TODO, stop warnings around `anext()` builtin use?
-# tool.ruff.target-version = "py310"
-
-
-[lint]
-# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`)  codes by default.
-# Unlike Flake8, Ruff doesn't enable pycodestyle warnings (`W`) or
-# McCabe complexity (`C901`) by default.
-select = ["E4", "E7", "E9", "F"]
-ignore = []
-ignore-init-module-imports = false
-
-[lint.per-file-ignores]
-"piker/ui/qt.py" = [
-  "E402",
-  'F401',  # unused imports (without __all__ or blah as blah)
-  # "F841", # unused variable rules
-]
-
-# Allow fix for all enabled rules (when `--fix`) is provided.
-fixable = ["ALL"]
-unfixable = []
-
-# Allow unused variables when underscore-prefixed.
-dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
-
-[format]
-# Use single quotes in `ruff format`.
-quote-style = "single"
-
-# Like Black, indent with spaces, rather than tabs.
-indent-style = "space"
-
-# Like Black, respect magic trailing commas.
-skip-magic-trailing-comma = false
-
-# Like Black, automatically detect the appropriate line ending.
-line-ending = "auto"
-
-# Enable auto-formatting of code examples in docstrings. Markdown,
-# reStructuredText code/literal blocks and doctests are all supported.
-#
-# This is currently disabled by default, but it is planned for this
-# to be opt-out in the future.
-docstring-code-format = false
-
-# Set the line length limit used when formatting code snippets in
-# docstrings.
-#
-# This only has an effect when the `docstring-code-format` setting is
-# enabled.
-docstring-code-line-length = "dynamic"