Commit Graph

3522 Commits (a0a19a952f868f0779cd7e9b931e86561d12a1c0)

Author SHA1 Message Date
Tyler Goodlet d0af280a59 Port view downsampling handler to new update apis 2022-06-05 22:13:08 -04:00
Tyler Goodlet 599c77ff84 Port ui components to use flows, drop all late assignments of shm 2022-06-05 22:13:08 -04:00
Tyler Goodlet c097016fd2 Add new `ui._flows` module
This begins the removal of data processing / analysis methods from the
chart widget and instead moving them to our new `Flow` API (in the new
module introduce here) and delegating the old chart methods to the
respective internal flow. Most importantly is no longer storing the
"last read" of an array from shm in an internal chart table (was
`._arrays`) and instead the `ShmArray` instance is passed as input and
stored in the `Flow` instance. This greatly simplifies lookup logic such
that the display loop now doesn't have to worry about reading shm, it
can be done by internal graphics logic as desired. Generally speaking,
all previous `._arrays`/`._graphics` lookups are now delegated to the
entries in the chart's `._flows` table.

The new `Flow` methods are generally better factored and provide more
detailed output regarding data-stream <-> graphics inter-relations for
the future purpose of allowing much more efficient update calls in the
display loop as well as supporting low latency interaction UX.

The concept here is that we're introducing an intermediary layer that
ties together graphics and real-time data flows such that widget code is
oriented around plot layout and the flow apis are oriented around
real-time low latency updates and providing an efficient high level
metric layer for the UX.

The summary api transition is something like:
- `update_graphics_from_array()` -> `.update_graphics_from_flow()`
- `.bars_range()` -> `Flow.datums_range()`
- `.bars_range()` -> `Flow.datums_range()`
2022-06-05 22:13:08 -04:00
goodboy be7c4e70f0
Merge pull request #321 from pikers/ib_dedicated_data_client
Ib dedicated data client
2022-06-05 22:12:46 -04:00
Tyler Goodlet 051680e259 Fill data client sockaddr in feed status as `data_ep` field 2022-06-05 22:08:03 -04:00
Tyler Goodlet 55a453a710 Update `ib` section in brokers config template 2022-06-05 22:08:03 -04:00
Tyler Goodlet 88eccc1e15 Fill in label with pairs from `status` value of backend init msg 2022-06-05 22:08:00 -04:00
Tyler Goodlet 488506d8b8 Move feed status label generation into a new module 2022-06-05 22:07:13 -04:00
Tyler Goodlet 78b9333bcd Expect `list` of ports in `ib.ports` section
Given that naming the port map is mostly pointless, since accounts can
be detected once the client connects, just expect a `brokers.toml` to
define a simple sequence of port numbers. Toss in a warning for using
the old map/`dict` style.
2022-06-05 22:07:13 -04:00
Tyler Goodlet 7229a39f47 Drop data reset tries to 2 before connection reset 2022-06-04 20:44:43 -04:00
Tyler Goodlet d870a09a4b Increase timeouts, always connection reset after 3 tries 2022-06-04 20:44:03 -04:00
Tyler Goodlet 5d53ecb433 Switch vnc server to port 3003 2022-06-04 20:44:03 -04:00
Tyler Goodlet 06832b94d4 Add vnc password auth, connection reset logic
Now that we have working client auth thanks to:
https://github.com/barneygale/asyncvnc/pull/4 and related issue,
we can use a pw for the vnc server, though we should eventually
auto-generate a random one from a docker super obviously.

Add logic to the data reset hack loop to do a connection reset after
2 failed/timeout attempts at the regular data reset. We need to also add
this logic around reconnectionn events that are due to the host
network connection: aka roaming that's faster then timing logic
builtin to the gateway.
2022-06-04 20:44:03 -04:00
Tyler Goodlet 8d6c5b214e Add 6, 6s retries on feed resets 2022-06-04 20:44:03 -04:00
Tyler Goodlet a5389beccd Rejig scan loop for flaky TCP connects, better caching
`ib-gw` seems particularly fragile to connections from clients with the
same id (can result in weird connect hangs and even crashes) and
`ib_insync` doesn't handle intermittent tcp disconnects that
well..(especially on dockerized IBC setups). This adds a bunch of
changes to our client caching and scan loop as well a proper
task-locking-to-cache-proxies so that,

- `asyncio`-side clients aren't double-loaded/connected even when
  explicitly trying to reconnect repeatedly with a given client to work
  around the unreliability of the `asyncio.Transport` design in
  `ib_insync`.
- we can use `tractor.trionics.maybe_open_context()` to lock the `trio`
  side from loading more then one `Client` on the `asyncio` side and
  instead on cache hits only making a new `MethodProxy` around the
  reused `asyncio`-side client (since each `trio` task needs its own
  inter-task msg channel).
- a `finally:` block teardown on all clients loaded in the scan loop
  avoids stale connections.
- the connect params are now exposed as named args to
  `load_aio_clients()` can be easily controlled from caller code.

Oh, and we properly hooked up the internal `ib_insync` logging to our
own internal schema - makes it a lot easier to debug wtf is going on XD
2022-06-04 20:44:03 -04:00
Tyler Goodlet 26f47227d2 Fix `.ib` pattern match 2022-06-04 20:44:03 -04:00
Tyler Goodlet b357a120b9 Fix output unpack 2022-06-04 20:44:03 -04:00
Tyler Goodlet aba8b05a33 Fix null match 2022-06-04 20:44:03 -04:00
Tyler Goodlet c3142aec81 Drop `i3ipc + `xdotool` approach for feed hacks 2022-06-04 20:44:03 -04:00
Tyler Goodlet bff625725e Implement reset hacks via our patched `asyncvnc` client 2022-06-04 20:44:03 -04:00
Tyler Goodlet 6f172479eb Drop task-per-method `trio`-`asyncio` proxying
Use method proxies through the remaining endpoints and drop the old
spawn-a-task-per-method-call style helpers from module.
2022-06-04 20:44:03 -04:00
Tyler Goodlet a96f1dec3a Proxy heaven, choose one "preferred data client"
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
2022-06-04 20:44:03 -04:00
goodboy 86caf5f6a3
Merge pull request #322 from pikers/dockerize_ib_gw
Dockerize `ib-gw` 🏄🏼
2022-06-04 20:42:32 -04:00
Tyler Goodlet 72b4273ddc Link to container readme 2022-06-04 20:40:36 -04:00
Tyler Goodlet 4281936ff4 Add readme for `ib-gw` container usage 2022-06-04 20:18:29 -04:00
Guillermo Rodriguez 4ddf04f68b
Merge pull request #328 from pikers/windows_tiling_fix
fix windows snap problem by removing maximum window size
2022-06-04 20:58:37 -03:00
dinkus 339fcda727 fix windows snap problem by removing maximum window size 2022-06-04 17:53:27 -04:00
Tyler Goodlet 4b7d7d688e Bind to port 3003 2022-06-03 10:22:50 -04:00
Tyler Goodlet 7ae7b2f864 Lol, bind vnc server to localhost only 2022-06-03 10:21:56 -04:00
Tyler Goodlet fa9f8c78c3 Only bind IBC command server to localhost 2022-06-03 10:21:37 -04:00
Tyler Goodlet 3bbbc21d2b Comment unneeded port map for now 2022-06-03 10:21:29 -04:00
Tyler Goodlet b03603a6b4 Drop password auth from vnc server
Currently we're held back by an `asyncvnc` issue,
https://github.com/barneygale/asyncvnc/issues/1 but even still, given
we're running the container to be only accessible by localhost i'm not
sure we need this for the moment (or at all) anyway.
2022-05-24 09:29:35 -04:00
Tyler Goodlet 81b77df544 Flip tz to NY, add note about .env file 2022-05-24 09:29:35 -04:00
Tyler Goodlet a79a99fc71 Add working, template docker setup for `ib-gw`
Based on the now defunct project @
https://github.com/waytrade/ib-gateway-docker

Adds a `docker-compose.yml` and necessary gateway and `IBC` config
files to make it possible to spin up a local gateway on localhost:4002
and connect to it without issue using `ib_insync`.

Next up, we'll want to,
- automated the equivalent docker-compose steps using our
  `.data._ahab` supervisor system
- probably simplify and roll our own container (likely alpine or nixos
  based) which drops uneeded deps (`socat`, vnc) and adds `xdotool`.
- allow for API socket mapping to just be pulled direct from
  a user's `brokers.toml` and we'll just pass that direct to `IBC`'s
  config.
2022-05-21 14:22:11 -04:00
goodboy 9f47515f59
Merge pull request #320 from pikers/drop_pandas
Drop `pandas`
2022-05-15 13:57:06 -04:00
Tyler Goodlet 09f2f32d5b Drop `pandas` timestamp for qt 2022-05-15 13:49:54 -04:00
Tyler Goodlet e718120cc7 Drop `pandas` as dep 2022-05-15 13:49:54 -04:00
Tyler Goodlet fb5df5ab5e Drop `pandas` usage throughout brokers cli 2022-05-15 13:49:50 -04:00
Tyler Goodlet 6e2e2fc03f Use `pendulum` for timestamp parsing 2022-05-15 13:45:44 -04:00
Tyler Goodlet a3b2ba9ae9 Use `numpy.datetime64` for x-axis tick strings 2022-05-15 13:45:37 -04:00
goodboy 7083c5a0bd
Merge pull request #319 from pikers/no_ib_pps
No ib pps? Account names should still load.
2022-05-13 16:26:46 -04:00
Tyler Goodlet de55565f60 We already collect account values/names in `load_io_clients()` 2022-05-12 14:21:31 -04:00
Tyler Goodlet d0530c4e26 Deliver accounts from query instead of just pps with `ib` 2022-05-12 13:22:51 -04:00
goodboy 21b16b4a9e
Merge pull request #318 from pikers/trimeter_dep
Add `trimeter` dep.. that we forgot
2022-05-11 16:28:29 -04:00
Tyler Goodlet ed85079d0f Add `trimeter` dep.. that we forgot 2022-05-11 13:19:47 -04:00
goodboy 5b540a53e9
Merge pull request #317 from pikers/l1_precision_fix
Convert `binance` tick/lot step sizes to `float`
2022-05-11 11:17:43 -04:00
Tyler Goodlet fb91e27651 Well that was easy, convert tick/lot step sizes to `float` 2022-05-11 10:41:03 -04:00
goodboy 482fc1da10
Merge pull request #308 from pikers/storage_layer
Storage layer: initial `marketstore` tsdb support with async OHLCV history loading.
2022-05-11 10:18:33 -04:00
Tyler Goodlet b3f9c4f93d Only assert if input array actually has a size 2022-05-10 17:59:24 -04:00
Tyler Goodlet 09431aad85 Add support for no `._first.value` update shm prepends 2022-05-10 17:59:16 -04:00