Commit Graph

130 Commits (bd73ec4ea4e095ec44c88e088329b6b0f7bb2a69)

Author SHA1 Message Date
algorandpa 84cd1e0059 initial commit on copy 2023-02-25 18:59:39 -05:00
algorandpa 86b4386522 minor changes, prepare for rebase of overlays branch 2023-02-25 18:59:39 -05:00
Guillermo Rodriguez 47bf45f30e
Merge pull request #464 from pikers/elasticsearch_integration
Elasticsearch integration
2023-02-24 16:38:37 -03:00
Esmeralda Gallardo b96e2c314a
Minor style changes and removed unnecesary comments 2023-02-24 15:11:15 -03:00
Esmeralda Gallardo f96d6a04b6
Fixed UnboundLocalError on _ahab. Added test for marketstore's initialization 2023-02-22 13:28:07 -03:00
Guillermo Rodriguez acc6249d88
Remove unnesesary arguments to some pikerd functions, fix container init error
by switching from log reading to quering es health endpoint, fix install on ci
and add more logging.
2023-02-21 20:45:10 -03:00
Tyler Goodlet 0b678c97f4 Pass `loglevel: str` cli value through to service tests 2023-02-21 12:02:26 -05:00
Esmeralda Gallardo 4122c482ba
Added new tests for elasticsearch's and marketstore's initialization and stop 2023-02-21 13:34:29 -03:00
Esmeralda Gallardo b5cdf14036
Modified elasticsearch file name to 'elastic' to avoid name errors. Applied changes suggested in the pr. 2023-02-21 13:34:29 -03:00
Tyler Goodlet b3400f0d9c Add `loglevel: str` fixture, passthrough to `open_test_pikerd()` 2023-02-21 10:54:18 -05:00
Tyler Goodlet cd3e9b1b2a Move quest fixtures to test mod, clean out old travis fixture 2023-02-21 10:54:18 -05:00
Tyler Goodlet 69074f4fa5 Bump up service tree spawn timeout a couple secs 2023-01-26 17:59:25 -05:00
Tyler Goodlet fe4fb37b58 Add service tree tests for data-feeds and the EMS 2023-01-24 15:15:27 -05:00
Tyler Goodlet 7cfd431a2b Yield `Services` in `open_test_pikerd()` fixture 2023-01-24 15:15:27 -05:00
Tyler Goodlet 37e0ec7b7d Assert fixture caller is `pikerd` 2023-01-13 13:21:49 -05:00
Tyler Goodlet 6a1bb13feb Add base `pikerd` service tree custom check test 2023-01-13 13:21:49 -05:00
Tyler Goodlet 593db0ed0d Only run `kraken` feed tests in CI, use `open_test_pikerd()` 2023-01-10 11:09:19 -05:00
Tyler Goodlet 06622105cd Add a `open_test_pikerd()` acm fixture for easy booting of the service stack 2023-01-10 11:09:19 -05:00
Tyler Goodlet f232d6d4ee Add `ci_env` detector fixture 2023-01-10 11:09:19 -05:00
Tyler Goodlet b7e1443618 Use ETH on kraken to ensure enough quotes 2023-01-10 11:09:19 -05:00
Tyler Goodlet 5d021ffb85 Bump up timeout on multi-feed test for CI 2023-01-10 11:09:19 -05:00
Tyler Goodlet e348968113 Add multi-broker streaming test using both `binance` and `kraken` 2023-01-10 11:09:19 -05:00
Tyler Goodlet 36868bb86e Add `kraken` test, ensure single broker-provider for now 2023-01-10 11:09:19 -05:00
Tyler Goodlet bb6452b969 Further feed syncing fixes wrt to `Flumes`
Sync per-symbol sampler loop start to subscription registers such that
the loop can't start until the consumer's stream subscription is added;
the task-sync uses a `trio.Event`. This patch also drops a ton of
commented cruft.

Further adjustments needed to get parity with prior functionality:
- pass init msg 'symbol_info' field to the `Symbol.broker_info: dict`.
- ensure the `_FeedsBus._subscriptions` table uses the broker specific
  (without brokername suffix) as keys for lookup so that the sampler
  loop doesn't have to append in the brokername as a suffix.
- ensure the `open_feed_bus()` flumes-table-msg returned sent by
  `tractor.Context.started()` uses the `.to_msg()` form of all flume
  structs.
- ensure `maybe_open_feed()` uses `tractor.MsgStream.subscribe()` on all
  `Flume.stream`s on cache hits using the
  `tractor.trionics.gather_contexts()` helper.
2023-01-10 11:09:19 -05:00
Tyler Goodlet 32b36aa042 Expect init startup quotes from each symbol 2023-01-10 11:09:19 -05:00
Tyler Goodlet 18dc8b08e4 First draft aggregate feedz support
Orient shm-flow-arrays around the new idea of a `Flume` which provides
access, mgmt and basic measure of real-time data flow sets (see water
flow management semantics).

- We discard the previous idea of a "init message" which contained all
  the shm attachment info and instead send a startup message full of
  `Flume.to_msg()`s which are symmetrically loaded on the caller actor
  side.

- Create data-flows "entries" for every passed in fqsn such that the consumer gets back
  streams and shm for each, now all wrapped in `Flume` types. For now we
  allocate `brokermod.stream_quotes()` tasks 1-to-1 for each fqsn
  (instead of expecting each backend to do multi-plexing, though we
  might want that eventually) as well a `_FeedsBus._subscriber` entry
  for each. The pause/resume management loop is adjusted to match.
  Previously `Feed`s were  allocated 1-to-1 with each fqsn.

- Make `Feed` a `Struct` subtype instead of a `@dataclass` and move all
  flow specific attrs to the new `Flume`:
  - move `.index_stream()`, `.get_ds_info()` to `Flume`.
  - drop `.receive()`: each fqsn entry will now require knowledge of
    separate streams by feed users.
  - add multi-fqsn tables: `.flumes`, `.streams` which point to the
    appropriate per-symbol entries.

- Async load all `Flume`s from all contexts and all quote streams using
  `tractor.trionics.gather_contexts()` on the client `open_feed()` side.

- Update feeds test to include streaming 2 symbols on the same (binance)
  backend.
2023-01-10 11:09:18 -05:00
Tyler Goodlet c7d5db5f90 Start data feed layer test suite
Initial test that starts a `binance` feed and reads the quote messages
alongside shm buffers for 1s and 1m OHLC; just prints to console for
now.

Template out parametrization for multi-symbol quote-multiplexed feeds
which coming soon B)
2023-01-10 11:09:18 -05:00
Tyler Goodlet 1bf1965a8b Drop `tractor.log` level override fixture 2023-01-10 11:09:18 -05:00
Tyler Goodlet 63e9af002d Drop `tractor.testing` import in qt tests 2022-10-28 16:09:55 -04:00
Tyler Goodlet 2312b6aeb2 Fix conftest config mod import 2021-09-11 18:15:42 -04:00
Tyler Goodlet 088ea8c3ec Skip CLI tests due to questrade dependency for now 2021-05-22 15:12:12 -04:00
Tyler Goodlet 0228db7811 Skip questrade tests for now
They need to be run with a local private API key and haven't been ported
to latest data apis anyway. It's also a broker most peeps aren't going
to be using any time soon.
2021-05-20 16:11:49 -04:00
Tyler Goodlet 44eb9aa9ff Create a blank config if none exists 2019-03-18 00:04:34 -04:00
Tyler Goodlet 88bb5b9f05 Handle file not found 2019-03-17 23:22:00 -04:00
Tyler Goodlet b99c5d2abe Use confdir if provided even when in CI 2019-03-17 23:04:04 -04:00
Tyler Goodlet d0b1d0954f Catch multi-errors too? 2019-03-03 11:01:34 -05:00
Tyler Goodlet 6cbea10b83 Forgot an error 2019-02-26 19:15:53 -05:00
Tyler Goodlet 2306799110 Speed bump quoter a slight bit 2019-02-25 22:40:19 -05:00
Tyler Goodlet 1fd033d351 Token should always be from a practice account 2019-02-25 22:12:33 -05:00
Tyler Goodlet 414734f803 Add travisCI specific test suite integration
Questrade is the default broker backend (for now) so the CI
can run using a practice account token handed down through an
env variable. If we add a cached directory to the build then the token
should remain persistent in the brokers config and will only need to be
updated if something goes wrong.

Also, add a `--confdir` flag for pytest much in the same way as for
the `piker` cli.
2019-02-25 20:23:20 -05:00
Tyler Goodlet 1a0427db08 Use `DataFeed` api in streaming tests 2019-02-21 23:10:24 -05:00
Tyler Goodlet 57bef52438 Capture the right logger 2019-02-10 17:29:08 -05:00
Tyler Goodlet 308ceb1772 Use search method for `piker api` test 2019-02-10 17:28:43 -05:00
Tyler Goodlet ea289540b3 Add a test to verify auth endpoint "locking" 2019-02-09 21:58:49 -05:00
Tyler Goodlet 5339f754a1 Add a token refresh test that exhibits an API race issue 2019-02-04 00:16:16 -05:00
Tyler Goodlet 6cc8b4cc2f Test duplicate feed type quoting 2018-12-23 21:27:47 -05:00
Tyler Goodlet 1d1be9dd77 Include option stream subscription change in test 2018-12-11 17:10:36 -05:00
Tyler Goodlet 9c7ca84fef Include strike and expiry in option quotes 2018-12-09 13:40:26 -05:00
Tyler Goodlet 12d5627860 Aggregate streaming tests and test stocks + options together 2018-12-01 16:14:33 -05:00
Tyler Goodlet 2df5c76828 Adjust cli tests for new quotes list output 2018-12-01 16:13:15 -05:00
Tyler Goodlet 15dec65ba1 Add an options streaming test 2018-11-30 08:18:54 -05:00
Tyler Goodlet 48a9c389c5 Add loglevel support to tests 2018-11-30 08:18:13 -05:00
Tyler Goodlet e1d6edb3ee Skip qt tests on missing brokers.ini entry 2018-11-22 19:12:14 -05:00
Tyler Goodlet eaa2a9b05d Port streaming test to new `data` module 2018-11-22 16:31:53 -05:00
Tyler Goodlet 7b2ab504f9 Adjust tests to match 2018-11-22 09:44:47 -05:00
Tyler Goodlet 5961e458cf Add a option quote latency test 2018-11-13 14:59:49 -05:00
Tyler Goodlet 36cf68dc0f Update tests to match 2018-11-13 12:57:46 -05:00
Tyler Goodlet ab8008ad61 Repair quote streaming test 2018-11-11 21:59:41 -05:00
Tyler Goodlet 773457ac91 Drop stale import 2018-11-11 21:45:51 -05:00
Tyler Goodlet c8cb5a2fdc Remove duplicate fixture (now in conftest) 2018-11-11 21:07:34 -05:00
Tyler Goodlet 21eb68148c Add option contract and chain quote test 2018-11-11 21:06:46 -05:00
Tyler Goodlet 6bef365fd4 Add conftest 2018-11-11 21:05:44 -05:00
Tyler Goodlet 666228d82e Add initial QT stock quoting tests 2018-11-11 18:53:45 -05:00
Tyler Goodlet cbc1052bfe Update tests for new tractor api 2018-08-09 22:43:50 -04:00
Tyler Goodlet da0495816b Drop tractor unit tests 2018-07-06 17:27:42 -04:00
Tyler Goodlet 94e7083bfc `tractor.run()` is required for testing now 2018-07-05 15:33:02 -04:00
Tyler Goodlet f32ecb3d65 Arbiter now supports non-empty statespace 2018-06-27 11:45:21 -04:00
Tyler Goodlet c062b01b7b Add an actor spawning test
Test that the actor nursery API and ``tractor.run`` entrypoint work when
the sub-actor's main task is all that is run (i.e. no rpc requests).
2018-06-23 14:59:31 -04:00
Tyler Goodlet acb5dc398e Add a basic `tractor.run()` test 2018-06-12 15:23:58 -04:00
Tyler Goodlet 5a997d67d4 Add an initial `tractor` price streaming test 2018-06-07 09:18:40 -04:00
Tyler Goodlet c8539b7373 Test adding multiple tickers on cli 2018-04-25 09:11:21 -04:00
Tyler Goodlet 2070f292b1 Update tests 2018-04-10 21:57:28 -04:00
Tyler Goodlet 6b72d04427 Change watchlist write function name 2018-04-06 15:07:47 -04:00
K0nstantine 918133f265 Finalize WL Management and fix merge func 2018-04-04 00:35:12 -04:00
K0nstantine 49b760673e Remove write to file from API and move to CLI 2018-04-04 00:35:12 -04:00
K0nstantine cd69c30143 CLI tests for watchlist commands 2018-04-04 00:35:12 -04:00
K0nstantine 11f25958ba Move watchlist api tests to seperate module 2018-04-04 00:34:42 -04:00
K0nstantine e859222df4 Add initial API test, need relocation 2018-04-04 00:25:34 -04:00
K0nstantine d28a3dc461 Add initial config dir test 2018-04-04 00:25:34 -04:00
Tyler Goodlet 4c1932dd6d Add basic set of CLI tests 2018-03-27 16:28:44 -04:00