Commit Graph

2197 Commits (incr_update_backup)

Author SHA1 Message Date
Tyler Goodlet 39b7c9340d Add (ostensibly) working first attempt at M4 algo
All the refs are in the comments and original sample code from infinite
has been reworked to expect the input x/y arrays to already be sliced
(though we can later support passing in the start-end indexes if
desired).

The new routines are `ds_m4()` the python top level API and `_m4()` the
fast `numba` implementation.
2022-04-30 11:36:23 -04:00
Tyler Goodlet e7481b1634 Array diff lengths must be int 2022-04-30 11:36:23 -04:00
Tyler Goodlet 09d95157dc Limit real-time chart updates in "big data" cases
- the chart's uppx (units-per-pixel) is > 4 (i.e. zoomed out a lot)
- don't shift the chart (to keep the most recent step in view) if the
  last datum isn't in view (aka the user is probably looking at history)
2022-04-30 11:36:23 -04:00
Tyler Goodlet ea5b8f1dd0 Only trigger downsampling on manual changes, add a uppx method 2022-04-30 11:36:23 -04:00
Tyler Goodlet 7e49b7c033 Add for a `BarItems` to display a line on high uppx
When a bars graphic is zoomed out enough you get a high uppx, datum
units-per-pixel, and there is no point in drawing the 6-lines in each
bar element-graphic if you can't see them on the screen/display device.

Instead here we offer converting to a `FastAppendCurve` which traces
the high-low outline and instead display that when it's impossible to see the
details of bars - approximately when the uppx >= 2.

There is also some draft-commented code in here for downsampling the
outlines as zoom level increases but it's not fully working and should
likely be factored out into a higher level api anyway.
2022-04-30 11:36:23 -04:00
Tyler Goodlet e7dc1a036b Original index offset was right 2022-04-30 11:36:23 -04:00
Tyler Goodlet ab8ea41b93 Add an ohlcv high/low tracer with optional downsampling 2022-04-30 11:36:23 -04:00
Tyler Goodlet dbe55ad4d2 Pass linked charts into `BarItems` so that graphics can be cycled on downsample 2022-04-30 11:36:23 -04:00
Tyler Goodlet d7a9928293 Move graphics compression routines to new module 2022-04-30 11:36:23 -04:00
Tyler Goodlet 02300efb59 Use 12Hz as default fps throttle 2022-04-30 11:36:23 -04:00
Tyler Goodlet 7c4e55ed2c Add comment on how to enable `pyqtgraph` profiling 2022-04-30 11:36:23 -04:00
Tyler Goodlet 7811508307 Add basic optional polyline support, draft out downsampling routine 2022-04-30 11:36:23 -04:00
Tyler Goodlet 7e853fe345 Add a downsampled line-curve support to `BarItems`
In effort to start getting some graphics speedups as detailed in #109,
this adds a `FastAppendCurve`to every `BarItems` as a `._ds_line` which
is only displayed (instead of the normal mult-line bars curve) when the
"width" of a bar is indistinguishable on screen from a line -> so once
the view coordinates map to > 2 pixels on the display device.
`BarItems.maybe_paint_line()` takes care of this scaling detection logic and is
called by the associated view's `.sigXRangeChanged` signal handler.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 11f8c4f350 Add detailed `.addItem()`` comment 2022-04-30 11:36:23 -04:00
Tyler Goodlet 7577443f95 Add guard for real-time-not-active last line is `None` case 2022-04-30 11:36:23 -04:00
goodboy 01cc8f347e
Merge pull request #286 from pikers/wattygetlood-patch-1
update windows install instructions
2022-04-18 11:40:17 -04:00
wattygetlood 9eefc3a521
Update README.rst
added instructions for setting up visual studio code
2022-04-17 15:57:53 -04:00
goodboy 67cec4bc54
Merge pull request #304 from pikers/offline_history_loading
Offline history loading
2022-04-16 15:57:14 -04:00
Zoltan 34df818ed9
Merge pull request #300 from pikers/kraken_editorder
fix kraken bug, allow for live order edits
2022-04-16 15:04:55 -04:00
Konstantine Tsafatinos 773ed5e7ad update to merge syntax in submit_limit, fix non_master push mistake 2022-04-16 15:01:31 -04:00
Konstantine Tsafatinos 59434b9a8a refactor submit
_limit and expore the 'paper' like feature
2022-04-16 14:54:25 -04:00
Konstantine Tsafatinos 250d9cbc03 fix kraken bug, allow for live order edits 2022-04-16 14:38:03 -04:00
goodboy 3ac9c55535
Merge pull request #303 from pikers/drop_arrow_add_predulum
Drop `arrow` add `pendulum`
2022-04-16 14:00:03 -04:00
Tyler Goodlet bcb4fe8c50 Indefinitely wait on feed hack for windows? 2022-04-16 13:25:14 -04:00
Tyler Goodlet d8db9233c9 Establish stream before `fsp_compute` so that backfill updates work again.. 2022-04-16 13:25:14 -04:00
Tyler Goodlet 82f2fa2d37 Pass in fqsn from chart UI components 2022-04-16 13:25:14 -04:00
Tyler Goodlet 8195fae289 Add a `trigger_all` arg to update cycle func; allows hard history updates 2022-04-16 13:25:14 -04:00
Tyler Goodlet 30656eda39 Use a `DisplayState` in the graphics update loop
The graphics update loop is much easier to grok when all the UI
components which potentially need to be updated on a cycle are arranged
together in a high-level composite namespace, thus this new
`DisplayState` addition. Create and set this state on each
`LinkedSplits` chart set and add a new method `.graphics_cycle()` which
let's a caller trigger a graphics loop update manually. Use this method
in the fsp graphics manager such that a chain can update new history
output even if there is no real-time feed driving the display loop (eg.
when a market is "closed").
2022-04-16 13:25:14 -04:00
Tyler Goodlet 2564acea1b Facepalm**2: only update on special "update" msg 2022-04-16 13:25:14 -04:00
Tyler Goodlet b3efa2874b Facepalm: display state must be linked charts specific 2022-04-16 13:25:14 -04:00
Tyler Goodlet ad1bbe74ad Manually trigger graphics loops updates on msgs from the fsp chain 2022-04-16 13:25:14 -04:00
Tyler Goodlet 761b823939 Always fire a "step/update message" on every fsp history update 2022-04-16 13:25:14 -04:00
Tyler Goodlet b75a3310fe Factor sync part of graphics update into func, add `trigger_update()`` 2022-04-16 13:25:14 -04:00
Tyler Goodlet ed8cfcf66d Drop `arrow` from install deps 2022-04-16 13:23:42 -04:00
Tyler Goodlet 72ec34ffd2 Port to `pendulum` equivalent apis throughout 2022-04-16 13:23:42 -04:00
Tyler Goodlet d334e61b1f Drop 22s timeout on reset hack 2022-04-16 13:23:38 -04:00
goodboy fbabfb78e0
Merge pull request #294 from pikers/broker_bumpz
Broker bumpz
2022-04-13 08:10:44 -04:00
Tyler Goodlet 4d23f6e4d7 Drop need for `ib_insync.IB.qualifyContractsAsync()' mod
As per https://github.com/erdewit/ib_insync/pull/454 the more correct
way to do this is with `.reqContractDetailsAsync()` which we wrap with
`Client.con_deats()` and which works just as well. Further drop all the
`dict`-ifying that was being done in that method and instead always
return `ContractDetails` object in an fqsn-like explicitly keyed `dict`.
2022-04-13 00:39:15 -04:00
Tyler Goodlet 8b1c521ae9 Ignore symbol-not-found errors 2022-04-13 00:39:15 -04:00
Tyler Goodlet 7586e20ab4 Use new unpacker helper name 2022-04-13 00:39:15 -04:00
Tyler Goodlet 80d70216f7 Drop back down ohlc bars request count to not trigger feed hack 2022-04-13 00:39:15 -04:00
Tyler Goodlet d1f45b0883 Add `ShmArray.last()` docstr 2022-04-13 00:39:15 -04:00
Tyler Goodlet 00a7f20292 Up the shm size to 10d of 1s ohlc 2022-04-13 00:39:15 -04:00
Tyler Goodlet 0178fcd26f Increase shm size to days of 1s steps 2022-04-13 00:39:15 -04:00
Tyler Goodlet 24fa1b8ff7 Support an array field map to `ShmArray.push()`, start index 3days in 2022-04-13 00:39:15 -04:00
Tyler Goodlet 66ea74c6d5 Put back more bars iters in loop to handle no-data in range cases 2022-04-13 00:39:15 -04:00
Tyler Goodlet b579d4b1f5 Get ib data feed hackzorz workin
ib has a throttle limit for "hft" bars but contained in here is some
hackery using ``xdotool`` to reset data farms auto-magically B)

This copies the working script into the ib backend mod as a routine and
now uses `trio.run_process()` and calls into it from the `get_bars()`
history retriever and then waits for "data re-established" events to be
received from the client before making more history queries.

TL;DR summary of changes:
- relay ib's "system status" events (like for data farm statuses)
  as a new "event" msg that can be processed by registers of
  `Client.inline_errors()` (though we should probably make a new
  method for this).
- add `MethodProxy.status_event()` which allows a proxy user to register
  for a particular "system event" (as mentioned above), which puts
  a `trio.Event` entry in a small table can be set by an relay task if
  there are any detected waiters.
- start a "msg relay task" when opening the method proxy which does
  the event setting mentioned above in the background.
- drop the request error handling around the proxy creation, doesn't
  seem necessary any more now that we have better error propagation from
  `asyncio`.
- add event waiting logic around the data feed reset hackzorin.
- change the order relay task to only log system events for now (though
  we need to do some better parsing/logic to get tws-external order
  updates to work again..
2022-04-13 00:39:15 -04:00
Tyler Goodlet 874374af06 Drop `pandas` use in ib backend for history
Found an issue (that was predictably brushed aside XD) where the
`ib_insync.util.df()` helper was changing the timestamps on bars data to
be way off (probably a `pandas.Timestamp` timezone thing?).

Anyway, dropped all that (which will hopefully let us drop `pandas` as
a hard dep) and added a buncha timestamp checking as well as start/end
datetime return values using `pendulum` so that consumer code can know
which "slice" is output.

Also added some WIP code to work around "no history found" request
errors where instead now we try to increment backward another 200
seconds - not sure if this actually correct yet.
2022-04-13 00:39:15 -04:00
Tyler Goodlet 62d073dc18 More IB repairs..
Make the throttle error propagate through to `trio` again by adding
`dict`-msg support between the two loops such that errors can be
re-raised on the `trio` side. This is all integrated into the
`MethoProxy` and accompanying result relay task.

Further fix a longer standing issue where sometimes the `ib_insync`
order entry method will raise a weird assertion error because it detects
some internal order-id state issue.. Just ignore those and make relay
back an error to the ems in such cases.

Add a bunch of notes for todos surrounding data feed reset hackery.
2022-04-13 00:39:15 -04:00
Tyler Goodlet 3e125625b1 Attempt to better handle history throttles using flag 2022-04-13 00:39:15 -04:00