Expose stream subscriber lag policy

`MsgStream.subscribe()` and `LinkedTaskChannel.subscribe()` omitted
`BroadcastReceiver.raise_on_lag`, forcing downstream consumers to
mutate a private receiver attribute when overruns were acceptable.

Add `raise_on_lag` to both public wrappers. The first subscription
sets the irreversible root broadcaster's policy, while every child
selects its own strict or warn/drop/resume behavior independently.

Document both fan-out APIs. Cover policy forwarding plus real IPC
and infected-asyncio paths.

Prompt-IO: ai/prompt-io/opencode/20260812T213117Z_51185487_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
wkt/fix_broadcast_lag_count
Gud Boi 2026-08-13 00:37:07 -04:00
parent 511854870b
commit a2e0df4ba1
7 changed files with 191 additions and 2 deletions

View File

@ -0,0 +1,34 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: ses_0799212ebffe42arY96czXn89F
timestamp: 2026-08-12T21:31:17Z
git_ref: 51185487
scope: code
substantive: true
raw_file: 20260812T213117Z_51185487_prompt_io.raw.md
---
## Prompt
For Tractor PR 490 item four, expose `raise_on_lag` through the public IPC and
asyncio linked-channel subscription wrappers. Review, verify and return a
complete commit plan before applying the API downstream in piker.
## Response summary
Added public per-subscription lag policy to both wrappers, preserved first-call
root policy, documented the semantics, and covered forwarding plus real fan-out
paths.
## Files changed
- `tractor/_streaming.py` - `MsgStream` lag policy forwarding.
- `tractor/to_asyncio.py` - linked-channel lag policy forwarding.
- `docs/guide/streaming.rst` - IPC fan-out policy docs.
- `docs/guide/asyncio.rst` - linked-channel fan-out policy docs.
- `tests/test_task_broadcasting.py` - wrapper policy regression.
## Human edits
None - generated output follows the requested fourth iterative item.

View File

@ -0,0 +1,53 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-12T21:31:17Z
git_ref: 51185487
diff_cmd: git diff HEAD~1..HEAD
---
The user requested the fourth iterative refinement for Tractor PR 490: expose
subscriber lag policy through the public `MsgStream.subscribe()` and
`LinkedTaskChannel.subscribe()` wrappers, then stop for a complete commit
plan. This enables piker to replace private receiver mutation.
> `git diff HEAD~1..HEAD -- tractor/_streaming.py`
Added `raise_on_lag: bool = True` to `MsgStream.subscribe()`. The first call
passes the policy to the irreversibly allocated root broadcaster and its
child; later calls configure each child independently while retaining the
root's first-call policy.
> `git diff HEAD~1..HEAD -- tractor/to_asyncio.py`
Added equivalent lag-policy forwarding to `LinkedTaskChannel.subscribe()`.
> `git diff HEAD~1..HEAD -- docs/guide/streaming.rst`
> `git diff HEAD~1..HEAD -- docs/guide/asyncio.rst`
Documented strict versus warn/drop/resume behavior, independent child policy,
and first-call root policy for both wrapper types.
> `git diff HEAD~1..HEAD -- tests/test_task_broadcasting.py`
Added a parameterized wrapper-level regression using minimal receive-compatible
handles. It verifies a first non-raising subscription configures root and
child, then a later strict child does not mutate the sticky root policy.
Verification output:
```text
................... [100%]
19 passed in 5.71s
.................... [100%]
20 passed in 6.88s
. [100%]
1 passed in 0.86s
```
The second and third runs cover actual `MsgStream` and infected-asyncio
`LinkedTaskChannel` fan-out respectively. Python compilation and
`git diff --check` passed. Adversarial review found no actionable issues.

View File

@ -209,6 +209,11 @@ The underlying broadcast machinery is lazily allocated on first
use and is *not* reversible for the channel's remaining lifetime, use and is *not* reversible for the channel's remaining lifetime,
so only reach for it when you actually want the fan-out. so only reach for it when you actually want the fan-out.
As with ``MsgStream``, pass ``raise_on_lag=False`` for a consumer
which may warn, drop old values and resume from the retained window.
Each child chooses independently; the first subscription also fixes
the linked channel's root receive policy.
One-shot calls with ``run_task()`` One-shot calls with ``run_task()``
---------------------------------- ----------------------------------
When you just want a single ``asyncio`` result and no streaming When you just want a single ``asyncio`` result and no streaming

View File

@ -171,6 +171,12 @@ keeps pace with the *fastest* subscriber; a task falling more
than the buffered window behind has its next receive raise than the buffered window behind has its next receive raise
``tractor.trionics.Lagged`` to say it lost data. ``tractor.trionics.Lagged`` to say it lost data.
Pass ``raise_on_lag=False`` when a consumer may drop old values and
resume from the oldest retained item instead. The receiver logs the
overrun rather than raising. Each child subscription chooses its own
policy; the first call also fixes the policy of the stream's root
receive handle because broadcaster allocation is irreversible.
The broadcast handle stays duplex btw: it proxies ``send()`` The broadcast handle stays duplex btw: it proxies ``send()``
through to the underlying stream, so each subscriber task can through to the underlying stream, so each subscriber task can
keep talking upstream while consuming its fan-out copy. keep talking upstream while consuming its fan-out copy.

View File

@ -8,6 +8,7 @@ from contextlib import (
from functools import partial from functools import partial
from itertools import cycle from itertools import cycle
import time import time
from types import SimpleNamespace
from typing import Optional from typing import Optional
import warnings import warnings
@ -15,6 +16,7 @@ import pytest
import trio import trio
from trio.lowlevel import current_task from trio.lowlevel import current_task
import tractor import tractor
from tractor.to_asyncio import LinkedTaskChannel
from tractor.trionics import ( from tractor.trionics import (
broadcast_receiver, broadcast_receiver,
BroadcastReceiveError, BroadcastReceiveError,
@ -952,3 +954,74 @@ def test_no_raise_on_lag():
with pytest.raises(KeyboardInterrupt): with pytest.raises(KeyboardInterrupt):
trio.run(main) trio.run(main)
@pytest.mark.parametrize(
('subscribe', 'chan_attr'),
[
(tractor.MsgStream.subscribe, '_rx_chan'),
(LinkedTaskChannel.subscribe, '_from_aio'),
],
ids=['msg-stream', 'linked-task-channel'],
)
def test_stream_subscribe_forwards_lag_policy(
subscribe,
chan_attr: str,
) -> None:
'''
Stream wrappers must expose per-subscriber lag policy.
`MsgStream.subscribe()` and `LinkedTaskChannel.subscribe()`
previously omitted `BroadcastReceiver.raise_on_lag`, forcing
downstream users to mutate a private receiver attribute. Invoke
each public wrapper against a minimal receive-compatible handle.
Prove the first non-raising subscription configures both the
irreversible root broadcaster and its child, while a later strict
child selects its own policy without changing that root.
'''
class StreamHandle:
'''
Provide the wrapper fields needed for local fan-out.
'''
def __init__(self) -> None:
self._broadcaster = None
setattr(
self,
chan_attr,
SimpleNamespace(
_state=SimpleNamespace(max_buffer_size=1),
),
)
async def receive(self):
'''
Block if a regression unexpectedly enters source receive.
'''
await trio.sleep_forever()
async def send(self, value) -> None:
'''
Satisfy `MsgStream` duplex-handle patching.
'''
async def main() -> None:
stream = StreamHandle()
async with subscribe(
stream,
raise_on_lag=False,
) as first:
assert not stream._broadcaster._raise_on_lag
assert not first._raise_on_lag
async with subscribe(
stream,
raise_on_lag=True,
) as second:
assert not stream._broadcaster._raise_on_lag
assert second._raise_on_lag
trio.run(main)

View File

@ -512,6 +512,7 @@ class MsgStream(trio.abc.Channel):
@acm @acm
async def subscribe( async def subscribe(
self, self,
raise_on_lag: bool = True,
) -> AsyncIterator[BroadcastReceiver]: ) -> AsyncIterator[BroadcastReceiver]:
''' '''
@ -526,6 +527,11 @@ class MsgStream(trio.abc.Channel):
value from the far end via the internally created broudcast value from the far end via the internally created broudcast
receiver wrapper. receiver wrapper.
``raise_on_lag=False`` makes this subscription warn and resume
at the oldest retained value after an overrun. The first call
also sets that policy for this stream's root receive handle;
later child subscriptions choose their policy independently.
''' '''
# NOTE: This operation is indempotent and non-reversible, so be # NOTE: This operation is indempotent and non-reversible, so be
# sure you can deal with any (theoretical) overhead of the the # sure you can deal with any (theoretical) overhead of the the
@ -541,6 +547,7 @@ class MsgStream(trio.abc.Channel):
# TODO: can remove this kwarg right since # TODO: can remove this kwarg right since
# by default behaviour is to do this anyway? # by default behaviour is to do this anyway?
receive_afunc=self.receive, receive_afunc=self.receive,
raise_on_lag=raise_on_lag,
) )
# NOTE: we override the original stream instance's receive # NOTE: we override the original stream instance's receive
@ -552,7 +559,9 @@ class MsgStream(trio.abc.Channel):
# seems there's no graceful way to type this with ``mypy``? # seems there's no graceful way to type this with ``mypy``?
# https://github.com/python/mypy/issues/708 # https://github.com/python/mypy/issues/708
async with self._broadcaster.subscribe() as bstream: async with self._broadcaster.subscribe(
raise_on_lag=raise_on_lag,
) as bstream:
assert bstream.key != self._broadcaster.key assert bstream.key != self._broadcaster.key
assert bstream._recv == self._broadcaster._recv assert bstream._recv == self._broadcaster._recv

View File

@ -324,6 +324,7 @@ class LinkedTaskChannel(
@acm @acm
async def subscribe( async def subscribe(
self, self,
raise_on_lag: bool = True,
) -> AsyncIterator[BroadcastReceiver]: ) -> AsyncIterator[BroadcastReceiver]:
''' '''
@ -335,6 +336,11 @@ class LinkedTaskChannel(
See ``tractor._streaming.MsgStream.subscribe()`` for further See ``tractor._streaming.MsgStream.subscribe()`` for further
similar details. similar details.
``raise_on_lag=False`` makes this subscription warn and resume
at the oldest retained value after an overrun. The first call
also sets that policy for this channel's root receive handle;
later child subscriptions choose their policy independently.
''' '''
if self._broadcaster is None: if self._broadcaster is None:
@ -343,11 +349,14 @@ class LinkedTaskChannel(
# use memory channel size by default # use memory channel size by default
self._from_aio._state.max_buffer_size, # type: ignore self._from_aio._state.max_buffer_size, # type: ignore
receive_afunc=self.receive, receive_afunc=self.receive,
raise_on_lag=raise_on_lag,
) )
self.receive = bcast.receive # type: ignore self.receive = bcast.receive # type: ignore
async with self._broadcaster.subscribe() as bstream: async with self._broadcaster.subscribe(
raise_on_lag=raise_on_lag,
) as bstream:
assert bstream.key != self._broadcaster.key assert bstream.key != self._broadcaster.key
assert bstream._recv == self._broadcaster._recv assert bstream._recv == self._broadcaster._recv
yield bstream yield bstream