the FOSS computational trading platform (for non-tinas..) https://github.com/pikers/piker

Go to file

Tyler Goodlet 7fe3e3f482 Add full EMS order-dialog (re-)load support! This includes darks, lives and alerts with all connecting clients being broadcast all existing order-flow dialog states. Obviously for now darks and alerts only live as long as the `emsd` actor lifetime (though we will store these in local state eventually) and "live" orders have lifetimes managed by their respective backend broker. The details of this change-set is extensive, so here we go.. Messaging schema: - change the messaging `Status` status-key set to: `resp: Literal['pending', 'open', 'dark_open', 'triggered', 'closed', 'fill', 'canceled', 'error']` which better reflects the semantics of order lifetimes and was partially inspired by the status keys `kraken` provides for their order-entry API. The prior key set was based on `ib`'s horrible semantics which sound like they're right out of the 80s.. Also, we reflect this same set in the `BrokerdStatus` msg and likely we'll just get rid of the separate brokerd-dialog side type eventually. - use `Literal` type annots for statuses where applicable and as they are supported by `msgspec`. - add additional optional `Status` fields: -`req: Order` to allow each status msg to optionally ref its commanding order-request msg allowing at least a request-response style implicit tracing in all response msgs. -`src: str` tag string to show the source of the msg. -`reqid: str \| int` such that the ems can relay the `brokerd` request id both to the client side and have one spot to look up prior status msgs and - draft a (unused/commented) `Dialog` type which can be eventually used at all EMS endpoints to track msg-flow states EMS engine adjustments/rework: - use the new status key set throughout and expect `BrokerdStatus` msgs to use the same new schema as `Status`. - add a `_DarkBook._active: dict[str, Status]` table which is now used for all per-leg-dialog associations and order flow state tracking allowing for the both the brokerd-relay and client-request handler loops to read/write the same msg-table and provides for delivering the overall EMS-active-orders state to newly/re-connecting clients with minimal processing; this table replaces what the `._ems_entries` table from prior. - add `Router.client_broadcast()` to send a msg to all currently connected peers. - a variety of msg handler block logic tweaks including more `case:` blocks to be both flatter and improve explicitness: - for the relay loop move all `Status` msg update and sending to within each block instead of a fallthrough case plus hard-to-follow state logic. - add a specific case for unhandled backend status keys and just log them. - pop alerts from `._active` immediately once triggered. - where possible mutate status msgs fields over instantiating new ones. - insert and expect `Order` instances in the dark clearing loop and adjust `case:` blocks accordingly. - tag `dark_open` and `triggered` statuses as sourced from the ems. - drop all the `ChainMap` stuff for now; we're going to make our own `Dialog` type for this purpose.. Order mode rework: - always parse the `Status` msg and use match syntax cases with object patterns, hackily assign the `.req` in many blocks to work around not yet having proper on-the-wire decoding yet. - make `.load_unknown_dialog_from_msg()` expect a `Status` with boxed `.req: Order` as input. - change `OrderDialog` -> `Dialog` in prep for a general purpose type of the same name. `ib` backend order loading support: - do "closed" status detection inside the msg-relay loop instead of expecting the ems to do this.. - add an attempt to cancel inactive orders by scheduling cancel submissions continually (no idea if this works). - add a status map to go from the 80s keys to our new set. - deliver `Status` msgs with an embedded `Order` for existing live order loading and make sure to try an get the source exchange info (instead of SMART). Paper engine ported to match: - use new status keys in `BrokerdStatus` msgs - use `match:` syntax in request handler loop		2022-08-10 13:38:23 -04:00
.github/workflows	Use checkout and setup-python v3 actions and drop dev install	2022-04-12 22:14:22 -04:00
config	Add example flex variables to brokers template	2022-06-06 19:26:14 -04:00
dockering/ib	Only allow vnc client connections from localhost	2022-07-27 14:40:32 -04:00
piker	Add full EMS order-dialog (re-)load support!	2022-08-10 13:38:23 -04:00
scripts	Disable re-connect for now in ib script	2022-05-09 11:15:13 -04:00
snippets	Move ib data reset script into a new `scripts/` dir	2022-05-09 11:15:13 -04:00
tests	Fix conftest config mod import	2021-09-11 18:15:42 -04:00
.gitignore	Ignore snippets dir	2022-03-01 12:36:32 -05:00
LICENSE	Relicense to AGPLv3	2020-06-16 00:12:19 -04:00
MANIFEST.in	Slap in brokers.toml template if none exists	2021-05-22 15:11:39 -04:00
README.rst	Link to container readme	2022-06-04 20:40:36 -04:00
notes_to_self.rst	Start nts	2022-01-25 07:57:01 -05:00
requirements-test.txt	Add seperate test job installing test deps	2021-05-20 16:23:56 -04:00
requirements.txt	Implement reset hacks via our patched `asyncvnc` client	2022-06-04 20:44:03 -04:00
setup.py	Remove `pydantic` from deps	2022-07-09 13:10:09 -04:00

README.rst

piker

trading gear for hackers.

piker is a broker agnostic, next-gen FOSS toolset for real-time computational trading targeted at hardcore Linux users .

we use as much bleeding edge tech as possible including (but not limited to):

latest python for glue
trio for structured concurrency
tractor for distributed, multi-core, real-time streaming
marketstore for historical and real-time tick data persistence and sharing
techtonicdb for L2 book storage
Qt for pristine high performance UIs
pyqtgraph for real-time charting
numpy and numba for fast numerics

focus and features:

100% federated: your code, your hardware, your data feeds, your broker fills.
zero web: low latency, native software that doesn't try to re-invent the OS
maximal privacy: prevent brokers and mms from knowing your planz; smack their spreads with dark volume.
zero clutter: modal, context oriented UIs that echew minimalism, reduce thought noise and encourage un-emotion.
first class parallelism: built from the ground up on next-gen structured concurrency primitives.
traders first: broker/exchange/asset-class agnostic
systems grounded: real-time financial signal processing that will make any queuing or DSP eng juice their shorts.
non-tina UX: sleek, powerful keyboard driven interaction with expected use in tiling wms
data collaboration: every process and protocol is multi-host scalable.
fight club ready: zero interest in adoption by suits; no corporate friendly license, ever.

fitting with these tenets, we're always open to new framework suggestions and ideas.

building the best looking, most reliable, keyboard friendly trading platform is the dream; join the cause.

install

piker is currently under heavy pre-alpha development and as such should be cloned from this repo and hacked on directly.

for a development install:

git clone git@github.com:pikers/piker.git
cd piker
virtualenv env
source ./env/bin/activate
pip install -r requirements.txt -e .

install for tinas

for windows peeps you can start by installing all the prerequisite software:

install git with all default settings - https://git-scm.com/download/win
install anaconda all default settings - https://www.anaconda.com/products/individual
install microsoft build tools (check the box for Desktop development for C++, you might be able to uncheck some optional downloads) - https://visualstudio.microsoft.com/visual-cpp-build-tools/
install visual studio code default settings - https://code.visualstudio.com/download

then, crack a conda shell and run the following commands:

mkdir code # create code directory
cd code # change directory to code
git clone https://github.com/pikers/piker.git # downloads piker installation package from github
cd piker # change directory to piker

conda create -n pikonda # creates conda environment named pikonda
conda activate pikonda # activates pikonda

conda install -c conda-forge python-levenshtein # in case it is not already installed
conda install pip # may already be installed
pip # will show if pip is installed

pip install -e . -r requirements.txt # install piker in editable mode

test Piker to see if it is working:

piker -b binance chart btcusdt.binance # formatting for loading a chart
piker -b kraken -b binance chart xbtusdt.kraken
piker -b kraken -b binance -b ib chart qqq.nasdaq.ib
piker -b ib chart tsla.nasdaq.ib

potential error:

FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\user\\AppData\\Roaming\\piker\\brokers.toml'

solution:

navigate to file directory above (may be different on your machine, location should be listed in the error code)
copy and paste file from 'C:\Users\user\code\data/brokers.toml' or create a blank file using notepad at the location above

Visual Studio Code setup:

now that piker is installed we can set up vscode as the default terminal for running piker and editing the code
open Visual Studio Code
file --> Add Folder to Workspace --> C:Usersusercodepiker (adds piker directory where all piker files are located)
file --> Save Workspace As --> save it wherever you want and call it whatever you want, this is going to be your default workspace for running and editing piker code
ctrl + shift + p --> start typing Python: Select Interpetter --> when the option comes up select it --> Select at the workspace level --> select the one that shows ('pikonda')
change the default terminal to cmd.exe instead of powershell (default)
now when you create a new terminal VScode should automatically activate you conda env so that piker can be run as the first command after a new terminal is created

also, try out fancyzones as part of powertoyz for a decent tiling windows manager to manage all the cool new software you are going to be running.

provider support

for live data feeds the in-progress set of supported brokers is:

IB via ib_insync, also see our container docs
binance and kraken for crypto over their public websocket API
questrade (ish) which comes with effectively free L1

coming soon...

webull via the reverse engineered public API
yahoo via yliveticker

if you want your broker supported and they have an API let us know.

check out our charts

bet you weren't expecting this from the foss:

piker -l info -b kraken -b binance chart btcusdt.binance --pdb

this runs the main chart (currently with 1m sampled OHLC) in in debug mode and you can practice paper trading using the following micro-manual:

order_mode (

edge triggered activation by any of the following keys, mouse-click on y-level to submit at that price ):

f/ ctl-f to stage buy
d/ ctl-d to stage sell
a to stage alert

search_mode (

ctl-l or ctl-space to open, ctl-c or ctl-space to close ) :

begin typing to have symbol search automatically lookup symbols from all loaded backend (broker) providers
arrow keys and mouse click to navigate selection
vi-like ctl-[hjkl] for navigation

you can also configure your position allocation limits from the sidepane.

run in distributed mode

start the service manager and data feed daemon in the background and connect to it:

pikerd -l info --pdb

connect your chart:

piker -l info -b kraken -b binance chart xmrusdt.binance --pdb

enjoy persistent real-time data feeds tied to daemon lifetime. the next time you spawn a chart it will load much faster since the data feed has been cached and is now always running live in the background until you kill pikerd.

if anyone asks you what this project is about

you don't talk about it.

how do i get involved?

enter the matrix.

how come there ain't that many docs

suck it up, learn the code; no one is trying to sell you on anything. also, we need lotsa help so if you want to start somewhere and can't necessarily write serious code, this might be the place for you!