Guillermo Rodriguez
3327da8847
Refactor generate_sample_messages to be a generator and use numpy
2025-04-06 21:59:14 -03:00
Guillermo Rodriguez
a15b852b18
Add trio resource semantics to ring pubsub
2025-04-06 21:19:39 -03:00
Guillermo Rodriguez
7b668c2f33
RingBufferReceiveChannel fixes for the non clean eof case, add comments
2025-04-06 21:16:55 -03:00
Guillermo Rodriguez
2bf8ce84cf
Add trio resource semantics to eventfd
2025-04-06 21:04:18 -03:00
Guillermo Rodriguez
0de70172fc
Adhere to trio semantics on channels for closed and busy resource cases
2025-04-06 17:02:15 -03:00
Guillermo Rodriguez
5fbd6aced7
Fix test docstring
2025-04-04 02:59:50 -03:00
Guillermo Rodriguez
1c5bebfca6
Enable ordering assertion & simplify some parts of test
2025-04-04 02:57:38 -03:00
Guillermo Rodriguez
53c98958b1
Add header to generic chan orderers
2025-04-04 02:47:27 -03:00
Guillermo Rodriguez
3c1873c68a
Fully test and fix bugs on _ringbuf._pubsub
...
Add generic channel orderer
2025-04-04 02:44:45 -03:00
Guillermo Rodriguez
0b9c2de3ad
Improve ringd ringbuf lifecycle
...
Unlink sock after use in fdshare
2025-04-04 02:41:50 -03:00
Guillermo Rodriguez
eb20e5ea8d
Rename RingBuff -> RingBuffer
...
Combine RingBuffer stream and channel apis
Implement RingBufferReceiveChannel.receive_nowait
Make msg generator calculate hash
2025-04-04 02:36:59 -03:00
Guillermo Rodriguez
70d72fd173
Woops fix old typing Self stuff
2025-04-03 13:11:00 -03:00
Guillermo Rodriguez
26efa3b25f
Add header and fix white lines
2025-04-03 12:35:09 -03:00
Guillermo Rodriguez
20f7ba4c2c
Switch to using typing.Protocl instead of abc.ABC on ChannelManager, improve abstraction and add comments
2025-04-03 12:34:40 -03:00
Guillermo Rodriguez
57d48a3af7
Improve error handling in fdshare functions, add comments
2025-04-03 11:48:07 -03:00
Guillermo Rodriguez
eef98bc826
Dont use relative import on ringd
2025-04-02 15:17:56 -03:00
Guillermo Rodriguez
dfc0254995
recv_fds doesnt need to be an acm
2025-04-02 15:00:40 -03:00
Guillermo Rodriguez
b8d1fd6978
Add LICENSE headers and module docstring on new files
2025-04-02 14:56:48 -03:00
Guillermo Rodriguez
739e728065
Add ringd test, which also tests fd share
2025-04-02 14:45:21 -03:00
Guillermo Rodriguez
d01ad12b3b
Linux top-level submodule and ipc._ringbuf submodule
...
Added ringd actor to broker ring buf resources dynamically
Added ring pubsub based on ringd
Created tractor.linux submodule and moved eventfd stuff there
Implemented linux file descriptor ipc share async helpers
2025-04-02 14:39:02 -03:00
Guillermo Rodriguez
bab265b2d8
Important RingBuffBytesSender fix on non batched mode! & downgrade nix-shell python to lowest supported
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
010874bed5
Catch trio cancellation on RingBuffReceiver bg eof listener task, add batched mode to RingBuffBytesSender
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
ea010ab46a
Add direct read method on EventFD
...
Type hint all ctx managers in _ringbuf.py
Remove unnecesary send lock on ring chan sender
Handle EOF on ring chan receiver
Rename ringbuf tests to make it less redundant
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
be7fc89ae9
Add direct ctx managers for RB channels
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
2a9a78651b
Improve test_ringbuf test, drop MsgTransport ring buf impl for now in favour of a trio.abc.Channel[bytes] impl, add docstrings
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
be818a720a
Switch `tractor.ipc.MsgTransport.stream` type to `trio.abc.Stream`
...
Add EOF signaling mechanism
Support proper `receive_some` end of stream semantics
Add StapledStream non-ipc test
Create MsgpackRBStream similar to MsgpackTCPStream for buffered whole-msg reads
Add EventFD.read cancellation on EventFD.close mechanism using cancel scope
Add test for eventfd cancellation
Improve and add docstrings
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
ba353bf46f
Better encapsulate RingBuff ctx managment methods and support non ipc usage
...
Add trio.StrictFIFOLock on sender.send_all
Support max_bytes argument on receive_some, keep track of write_ptr on receiver
Add max_bytes receive test test_ringbuf_max_bytes
Add docstrings to all ringbuf tests
Remove EFD_NONBLOCK support, not necesary anymore since we can use abandon_on_cancel=True on trio.to_thread.run_sync
Close eventfd's after usage on open_ringbuf
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
9b2161506f
Break out transport protocol and tcp specifics into their own submodules under tractor.ipc
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
6b155849b7
Add buf_size to RBToken and add sender cancel test, move disable_mantracker to its own _mp_bs module
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
59c8c7bfe3
Make ring buf api use pickle-able RBToken
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
6ac6fd56c0
Address some of fomo\'s comments
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
f799e9ac51
Handle cancelation on EventFD.read
2025-03-27 20:36:46 -03:00
Guillermo Rodriguez
9980bb2bd0
Add module headers and fix spacing on tractor._ipc._linux
2025-03-27 20:36:45 -03:00
Guillermo Rodriguez
8de9ab291e
Move RingBuffSender|Receiver to its own tractor.ipc._ringbuf module
2025-03-27 20:36:45 -03:00
Guillermo Rodriguez
1a83626f26
Move linux specifics from tractor.ipc._shm into tractor.ipc._linux
2025-03-27 20:36:45 -03:00
Guillermo Rodriguez
6b4d08d030
Move tractor._shm to tractor.ipc._shm
2025-03-27 20:36:45 -03:00
Guillermo Rodriguez
7b8b9d6805
move tractor._ipc.py into tractor.ipc._chan.py
2025-03-27 20:36:45 -03:00
Guillermo Rodriguez
5afe0a0264
General improvements
...
EventFD class now expects the fd to already be init with open_eventfd
RingBuff Sender and Receiver fully manage SharedMemory and EventFD lifecycles, no aditional ctx mngrs needed
Separate ring buf tests into its own test bed
Add parametrization to test and cancellation
Add docstrings
Add simple testing data gen module .samples
2025-03-27 20:36:45 -03:00
Guillermo Rodriguez
eeb9a7d61b
IPC ring bug impl with async read
2025-03-27 20:36:45 -03:00
5cee222353
Updates from latest `piker.data._sharedmem` changes
2025-03-27 17:54:04 -04:00
8ebb1f09de
Pass `str` dtype for `use_str` case
2025-03-27 17:54:04 -04:00
2683a7f33a
Allocate size-specced "empty" sequence from default values by type
2025-03-27 17:54:04 -04:00
255209f881
Mod define `_USE_POSIX`, add a of of todos
2025-03-27 17:54:04 -04:00
9a0d529b18
Parametrize rw test with variable frame sizes
...
Demonstrates fixed size frame-oriented reads by the child where the
parent only transmits a "read" stream msg on "frame fill events" such
that the child incrementally reads the shm list data (much like in
a real-time-buffered streaming system).
2025-03-27 17:54:04 -04:00
1c441b0986
Add `ShmList` slice support in `.__getitem__()`
2025-03-27 17:54:04 -04:00
afbdb50a30
Rename token type to `NDToken` in the style of `nptyping`
2025-03-27 17:54:04 -04:00
e46033cbe7
Don't require runtime (for now), type annot fixing
2025-03-27 17:54:04 -04:00
c932bb5911
Add repetitive attach to existing segment test
2025-03-27 17:54:04 -04:00
33482d8f41
Add initial readers-writer shm list tests
2025-03-27 17:54:04 -04:00
7ae194baed
Add `ShmList` wrapping the stdlib's `ShareableList`
...
First attempt at getting `multiprocessing.shared_memory.ShareableList`
working; we wrap the stdlib type with a readonly attr and a `.key` for
cross-actor lookup. Also, rename all `numpy` specific routines to have
a `ndarray` suffix in the func names.
2025-03-27 17:54:04 -04:00
Guillermo Rodriguez
Tyler Goodlet