Compare commits
14 Commits
9703cbe16a
...
62e285bcd7
| Author | SHA1 | Date |
|---|---|---|
|
|
62e285bcd7 | |
|
|
162d5f76d3 | |
|
|
6a0b8567c4 | |
|
|
46ebceae02 | |
|
|
edf0af51c2 | |
|
|
fcc253f6ce | |
|
|
d704f99223 | |
|
|
a120f378d0 | |
|
|
36307c5917 | |
|
|
30c2c3cb30 | |
|
|
0f6a0676eb | |
|
|
2616f4b976 | |
|
|
b5fd2a40b1 | |
|
|
277ddc1625 |
|
|
@ -1,15 +0,0 @@
|
||||||
{
|
|
||||||
"permissions": {
|
|
||||||
"allow": [
|
|
||||||
"Write(.claude/*commit_msg*)",
|
|
||||||
"Write(.claude/git_commit_msg_LATEST.md)",
|
|
||||||
"Bash(date *)",
|
|
||||||
"Bash(cp .claude/*)",
|
|
||||||
"Bash(git diff *)",
|
|
||||||
"Bash(git log *)",
|
|
||||||
"Bash(git status)"
|
|
||||||
],
|
|
||||||
"deny": [],
|
|
||||||
"ask": []
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,86 +0,0 @@
|
||||||
---
|
|
||||||
name: commit-msg
|
|
||||||
description: >
|
|
||||||
Generate git commit messages following project style. Use when user
|
|
||||||
wants to create a commit or asks for a commit message.
|
|
||||||
argument-hint: "[optional-scope-or-description]"
|
|
||||||
disable-model-invocation: true
|
|
||||||
allowed-tools:
|
|
||||||
- Bash(git *)
|
|
||||||
- Bash(date *)
|
|
||||||
- Bash(cp *)
|
|
||||||
- Read
|
|
||||||
- Grep
|
|
||||||
- Glob
|
|
||||||
- Write
|
|
||||||
---
|
|
||||||
|
|
||||||
When generating commit messages, always follow this process:
|
|
||||||
|
|
||||||
0. **Check for staged changes**: if `git diff --staged` is
|
|
||||||
empty, STOP and tell the user "nothing is staged!" with
|
|
||||||
a reminder to `git add` before invoking this skill.
|
|
||||||
|
|
||||||
1. **Gather context** from the staged diff and recent
|
|
||||||
history:
|
|
||||||
|
|
||||||
- Staged changes: !`git diff --staged --stat`
|
|
||||||
- Recent commits: !`git log --oneline -5`
|
|
||||||
|
|
||||||
2. **Analyze the diff**: understand what files changed and
|
|
||||||
the nature of the changes (new feature, bug fix, refactor,
|
|
||||||
etc.)
|
|
||||||
|
|
||||||
3. **Write the commit message** following these rules:
|
|
||||||
|
|
||||||
**Use the accompanying style guide:**
|
|
||||||
- See [style-guide-reference.md](style-guide-reference.md)
|
|
||||||
for detailed analysis of past commits in this repo.
|
|
||||||
|
|
||||||
**Subject Line Format:**
|
|
||||||
- Present tense verbs: Add, Drop, Fix, Use, Move, Adjust, etc.
|
|
||||||
- Target 50 chars (hard max: 67)
|
|
||||||
- Backticks around ALL code elements (classes, functions, modules, vars)
|
|
||||||
- Specific about what changed
|
|
||||||
|
|
||||||
**Body Format (optional - keep simple if warranted):**
|
|
||||||
- Max 67 char line length
|
|
||||||
- Use `-` bullets for lists
|
|
||||||
- Section markers: `Also,` `Deats,` `Other,` `Further,`
|
|
||||||
- Abbreviations: msg, bg, ctx, impl, mod, obvi, tn, fn, bc, var, prolly
|
|
||||||
- Casual yet technically precise tone
|
|
||||||
- Never write lines with only whitespace
|
|
||||||
|
|
||||||
**Common Opening Patterns:**
|
|
||||||
- New features: "Add `feature` to `module`"
|
|
||||||
- Removals: "Drop `attr` from `class`"
|
|
||||||
- Bug fixes: "Fix `issue` in `function`"
|
|
||||||
- Code moves: "Move `thing` to `new_location`"
|
|
||||||
- Adoption: "Use `new_tool` for `task`"
|
|
||||||
- Minor tweaks: "Adjust `behavior` in `component`"
|
|
||||||
|
|
||||||
4. **Write to TWO files**:
|
|
||||||
- `.claude/<timestamp>_<hash>_commit_msg.md`
|
|
||||||
* with `<timestamp>` from `date -u +%Y%m%dT%H%M%SZ` or similar
|
|
||||||
filesystem-safe format.
|
|
||||||
* and `<hash>` from `git log -1 --format=%h` first 7 chars.
|
|
||||||
- `.claude/git_commit_msg_LATEST.md` (overwrite)
|
|
||||||
|
|
||||||
5. **Always include credit footer**:
|
|
||||||
|
|
||||||
When no part of the patch was written by `claude`,
|
|
||||||
```
|
|
||||||
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
|
|
||||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
|
||||||
```
|
|
||||||
|
|
||||||
when some or all of the patch was written by `claude`
|
|
||||||
instead use,
|
|
||||||
|
|
||||||
```
|
|
||||||
(this patch was generated in some part by [`claude-code`][claude-code-gh])
|
|
||||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
|
||||||
```
|
|
||||||
|
|
||||||
Keep it concise. Match the tone of recent commits. For simple
|
|
||||||
changes, use subject line only.
|
|
||||||
|
|
@ -1,225 +0,0 @@
|
||||||
# Commit Message Style Guide for `tractor`
|
|
||||||
|
|
||||||
Analysis based on 500 recent commits from the `tractor` repository.
|
|
||||||
|
|
||||||
## Core Principles
|
|
||||||
|
|
||||||
Write commit messages that are technically precise yet casual in
|
|
||||||
tone. Use abbreviations and informal language while maintaining
|
|
||||||
clarity about what changed and why.
|
|
||||||
|
|
||||||
## Subject Line Format
|
|
||||||
|
|
||||||
### Length and Structure
|
|
||||||
- Target: ~50 chars with a hard-max of 67.
|
|
||||||
- Use backticks around code elements (72.2% of commits)
|
|
||||||
- Rarely use colons (5.2%), except for file prefixes
|
|
||||||
- End with '?' for uncertain changes (rare: 0.8%)
|
|
||||||
- End with '!' for important changes (rare: 2.0%)
|
|
||||||
|
|
||||||
### Opening Verbs (Present Tense)
|
|
||||||
|
|
||||||
Most common verbs from analysis:
|
|
||||||
- `Add` (14.4%) - wholly new features/functionality
|
|
||||||
- `Use` (4.4%) - adopt new approach/tool
|
|
||||||
- `Drop` (3.6%) - remove code/feature
|
|
||||||
- `Fix` (2.4%) - bug fixes
|
|
||||||
- `Move`/`Mv` (3.6%) - relocate code
|
|
||||||
- `Adjust` (2.0%) - minor tweaks
|
|
||||||
- `Update` (1.6%) - enhance existing feature
|
|
||||||
- `Bump` (1.2%) - dependency updates
|
|
||||||
- `Rename` (1.2%) - identifier changes
|
|
||||||
- `Set` (1.2%) - configuration changes
|
|
||||||
- `Handle` (1.0%) - add handling logic
|
|
||||||
- `Raise` (1.0%) - add error raising
|
|
||||||
- `Pass` (0.8%) - pass parameters/values
|
|
||||||
- `Support` (0.8%) - add support for something
|
|
||||||
- `Hide` (1.4%) - make private/internal
|
|
||||||
- `Always` (1.4%) - enforce consistent behavior
|
|
||||||
- `Mk` (1.4%) - make/create (abbreviated)
|
|
||||||
- `Start` (1.0%) - begin implementation
|
|
||||||
|
|
||||||
Other frequent verbs: `More`, `Change`, `Extend`, `Disable`, `Log`,
|
|
||||||
`Enable`, `Ensure`, `Expose`, `Allow`
|
|
||||||
|
|
||||||
### Backtick Usage
|
|
||||||
|
|
||||||
Always use backticks for:
|
|
||||||
- Module names: `trio`, `asyncio`, `msgspec`, `greenback`, `stackscope`
|
|
||||||
- Class names: `Context`, `Actor`, `Address`, `PldRx`, `SpawnSpec`
|
|
||||||
- Method names: `.pause_from_sync()`, `._pause()`, `.cancel()`
|
|
||||||
- Function names: `breakpoint()`, `collapse_eg()`, `open_root_actor()`
|
|
||||||
- Decorators: `@acm`, `@context`
|
|
||||||
- Exceptions: `Cancelled`, `TransportClosed`, `MsgTypeError`
|
|
||||||
- Keywords: `finally`, `None`, `False`
|
|
||||||
- Variable names: `tn`, `debug_mode`
|
|
||||||
- Complex expressions: `trio.Cancelled`, `asyncio.Task`
|
|
||||||
|
|
||||||
Most backticked terms in tractor:
|
|
||||||
`trio`, `asyncio`, `Context`, `.pause_from_sync()`, `tn`,
|
|
||||||
`._pause()`, `breakpoint()`, `collapse_eg()`, `Actor`, `@acm`,
|
|
||||||
`.cancel()`, `Cancelled`, `open_root_actor()`, `greenback`
|
|
||||||
|
|
||||||
### Examples
|
|
||||||
|
|
||||||
Good subject lines:
|
|
||||||
```
|
|
||||||
Add `uds` to `._multiaddr`, tweak typing
|
|
||||||
Drop `DebugStatus.shield` attr, add `.req_finished`
|
|
||||||
Use `stackscope` for all actor-tree rendered "views"
|
|
||||||
Fix `.to_asyncio` inter-task-cancellation!
|
|
||||||
Bump `ruff.toml` to target py313
|
|
||||||
Mv `load_module_from_path()` to new `._code_load` submod
|
|
||||||
Always use `tuple`-cast for singleton parent addrs
|
|
||||||
```
|
|
||||||
|
|
||||||
## Body Format
|
|
||||||
|
|
||||||
### General Structure
|
|
||||||
- 43.2% of commits have no body (simple changes)
|
|
||||||
- Use blank line after subject
|
|
||||||
- Max line length: 67 chars
|
|
||||||
- Use `-` bullets for lists (28.0% of commits)
|
|
||||||
- Rarely use `*` bullets (2.4%)
|
|
||||||
|
|
||||||
### Section Markers
|
|
||||||
|
|
||||||
Use these markers to organize longer commit bodies:
|
|
||||||
- `Also,` (most common: 26 occurrences)
|
|
||||||
- `Other,` (13 occurrences)
|
|
||||||
- `Deats,` (11 occurrences) - for implementation details
|
|
||||||
- `Further,` (7 occurrences)
|
|
||||||
- `TODO,` (3 occurrences)
|
|
||||||
- `Impl details,` (2 occurrences)
|
|
||||||
- `Notes,` (1 occurrence)
|
|
||||||
|
|
||||||
### Common Abbreviations
|
|
||||||
|
|
||||||
Use these freely (sorted by frequency):
|
|
||||||
- `msg` (63) - message
|
|
||||||
- `bg` (37) - background
|
|
||||||
- `ctx` (30) - context
|
|
||||||
- `impl` (27) - implementation
|
|
||||||
- `mod` (26) - module
|
|
||||||
- `obvi` (17) - obviously
|
|
||||||
- `tn` (16) - task name
|
|
||||||
- `fn` (15) - function
|
|
||||||
- `vs` (15) - versus
|
|
||||||
- `bc` (14) - because
|
|
||||||
- `var` (14) - variable
|
|
||||||
- `prolly` (9) - probably
|
|
||||||
- `ep` (6) - entry point
|
|
||||||
- `OW` (5) - otherwise
|
|
||||||
- `rn` (4) - right now
|
|
||||||
- `sig` (4) - signal/signature
|
|
||||||
- `deps` (3) - dependencies
|
|
||||||
- `iface` (2) - interface
|
|
||||||
- `subproc` (2) - subprocess
|
|
||||||
- `tho` (2) - though
|
|
||||||
- `ofc` (2) - of course
|
|
||||||
|
|
||||||
### Tone and Style
|
|
||||||
|
|
||||||
- Casual but technical (use `XD` for humor: 23 times)
|
|
||||||
- Use `..` for trailing thoughts (108 occurrences)
|
|
||||||
- Use `Woops,` to acknowledge mistakes (4 subject lines)
|
|
||||||
- Don't be afraid to show personality while being precise
|
|
||||||
|
|
||||||
### Example Bodies
|
|
||||||
|
|
||||||
Simple with bullets:
|
|
||||||
```
|
|
||||||
Add `multiaddr` and bump up some deps
|
|
||||||
|
|
||||||
Since we're planning to use it for (discovery)
|
|
||||||
addressing, allowing replacement of the hacky (pretend)
|
|
||||||
attempt in `tractor._multiaddr` Bp
|
|
||||||
|
|
||||||
Also pin some deps,
|
|
||||||
- make us py312+
|
|
||||||
- use `pdbp` with my frame indexing fix.
|
|
||||||
- mv to latest `xonsh` for fancy cmd/suggestion injections.
|
|
||||||
|
|
||||||
Bump lock file to match obvi!
|
|
||||||
```
|
|
||||||
|
|
||||||
With section markers:
|
|
||||||
```
|
|
||||||
Use `stackscope` for all actor-tree rendered "views"
|
|
||||||
|
|
||||||
Instead of the (much more) limited and hacky `.devx._code`
|
|
||||||
impls, move to using the new `.devx._stackscope` API which
|
|
||||||
wraps the `stackscope` project.
|
|
||||||
|
|
||||||
Deats,
|
|
||||||
- make new `stackscope.extract_stack()` wrapper
|
|
||||||
- port over frame-descing to `_stackscope.pformat_stack()`
|
|
||||||
- move `PdbREPL` to use `stackscope` render approach
|
|
||||||
- update tests for new stack output format
|
|
||||||
|
|
||||||
Also,
|
|
||||||
- tweak log formatting for consistency
|
|
||||||
- add typing hints throughout
|
|
||||||
```
|
|
||||||
|
|
||||||
## Special Patterns
|
|
||||||
|
|
||||||
### WIP Commits
|
|
||||||
Rare (0.2%) - avoid committing WIP if possible
|
|
||||||
|
|
||||||
### Merge Commits
|
|
||||||
Auto-generated (4.4%), don't worry about style
|
|
||||||
|
|
||||||
### File References
|
|
||||||
- Use `module.py` or `.submodule` style
|
|
||||||
- Rarely use `file.py:line` references (0 in analysis)
|
|
||||||
|
|
||||||
### Links
|
|
||||||
- GitHub links used sparingly (3 total)
|
|
||||||
- Prefer code references over external links
|
|
||||||
|
|
||||||
## Footer
|
|
||||||
|
|
||||||
The default footer should credit `claude` (you) for helping generate
|
|
||||||
the commit msg content:
|
|
||||||
|
|
||||||
```
|
|
||||||
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
|
|
||||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
|
||||||
```
|
|
||||||
|
|
||||||
Further, if the patch was solely or in part written
|
|
||||||
by `claude`, instead add:
|
|
||||||
|
|
||||||
```
|
|
||||||
(this patch was generated in some part by [`claude-code`][claude-code-gh])
|
|
||||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
|
||||||
```
|
|
||||||
|
|
||||||
## Summary Checklist
|
|
||||||
|
|
||||||
Before committing, verify:
|
|
||||||
- [ ] Subject line uses present tense verb
|
|
||||||
- [ ] Subject line ~50 chars (hard max 67)
|
|
||||||
- [ ] Code elements wrapped in backticks
|
|
||||||
- [ ] Body lines ≤67 chars
|
|
||||||
- [ ] Abbreviations used where natural
|
|
||||||
- [ ] Casual yet precise tone
|
|
||||||
- [ ] Section markers if body >3 paragraphs
|
|
||||||
- [ ] Technical accuracy maintained
|
|
||||||
|
|
||||||
## Analysis Metadata
|
|
||||||
|
|
||||||
```
|
|
||||||
Source: tractor repository
|
|
||||||
Commits analyzed: 500
|
|
||||||
Date range: 2019-2025
|
|
||||||
Analysis date: 2026-02-08
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
(this style guide was generated by [`claude-code`][claude-code-gh]
|
|
||||||
analyzing commit history)
|
|
||||||
|
|
||||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
|
||||||
|
|
@ -74,44 +74,24 @@ jobs:
|
||||||
# run: mypy tractor/ --ignore-missing-imports --show-traceback
|
# run: mypy tractor/ --ignore-missing-imports --show-traceback
|
||||||
|
|
||||||
|
|
||||||
testing:
|
testing-linux:
|
||||||
name: '${{ matrix.os }} Python${{ matrix.python-version }} spawn_backend=${{ matrix.spawn_backend }} tpt_proto=${{ matrix.tpt_proto }}'
|
name: '${{ matrix.os }} Python ${{ matrix.python }} - ${{ matrix.spawn_backend }}'
|
||||||
timeout-minutes: 16
|
timeout-minutes: 10
|
||||||
runs-on: ${{ matrix.os }}
|
runs-on: ${{ matrix.os }}
|
||||||
|
|
||||||
strategy:
|
strategy:
|
||||||
fail-fast: false
|
fail-fast: false
|
||||||
matrix:
|
matrix:
|
||||||
os: [
|
os: [ubuntu-latest]
|
||||||
ubuntu-latest,
|
python-version: ['3.13']
|
||||||
macos-latest,
|
|
||||||
]
|
|
||||||
python-version: [
|
|
||||||
'3.13',
|
|
||||||
# '3.14',
|
|
||||||
]
|
|
||||||
spawn_backend: [
|
spawn_backend: [
|
||||||
'trio',
|
'trio',
|
||||||
# 'mp_spawn',
|
# 'mp_spawn',
|
||||||
# 'mp_forkserver',
|
# 'mp_forkserver',
|
||||||
# ?TODO^ is it worth it to get these running again?
|
|
||||||
#
|
|
||||||
# - [ ] next-gen backends, on 3.13+
|
|
||||||
# https://github.com/goodboy/tractor/issues/379
|
|
||||||
# 'subinterpreter',
|
|
||||||
# 'subint',
|
|
||||||
]
|
]
|
||||||
tpt_proto: [
|
|
||||||
'tcp',
|
|
||||||
'uds',
|
|
||||||
]
|
|
||||||
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
|
|
||||||
exclude:
|
|
||||||
# don't do UDS run on macOS (for now)
|
|
||||||
- os: macos-latest
|
|
||||||
tpt_proto: 'uds'
|
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
|
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
- name: 'Install uv + py-${{ matrix.python-version }}'
|
- name: 'Install uv + py-${{ matrix.python-version }}'
|
||||||
|
|
@ -138,11 +118,7 @@ jobs:
|
||||||
run: uv tree
|
run: uv tree
|
||||||
|
|
||||||
- name: Run tests
|
- name: Run tests
|
||||||
run: >
|
run: uv run pytest tests/ --spawn-backend=${{ matrix.spawn_backend }} -rsx
|
||||||
uv run
|
|
||||||
pytest tests/ -rsx
|
|
||||||
--spawn-backend=${{ matrix.spawn_backend }}
|
|
||||||
--tpt-proto=${{ matrix.tpt_proto }}
|
|
||||||
|
|
||||||
# XXX legacy NOTE XXX
|
# XXX legacy NOTE XXX
|
||||||
#
|
#
|
||||||
|
|
|
||||||
|
|
@ -102,30 +102,3 @@ venv.bak/
|
||||||
|
|
||||||
# mypy
|
# mypy
|
||||||
.mypy_cache/
|
.mypy_cache/
|
||||||
|
|
||||||
# all files under
|
|
||||||
.git/
|
|
||||||
|
|
||||||
# any commit-msg gen tmp files
|
|
||||||
.claude/*_commit_*.md
|
|
||||||
.claude/*_commit*.toml
|
|
||||||
.claude/*_commit*.txt
|
|
||||||
|
|
||||||
# nix develop --profile .nixdev
|
|
||||||
.nixdev*
|
|
||||||
|
|
||||||
# :Obsession .
|
|
||||||
Session.vim
|
|
||||||
# `gish` local `.md`-files
|
|
||||||
# TODO? better all around automation!
|
|
||||||
# -[ ] it'd be handy to also commit and sync with wtv git service?
|
|
||||||
# -[ ] everything should be put under a `.gish/` no?
|
|
||||||
gitea/
|
|
||||||
gh/
|
|
||||||
|
|
||||||
# ------ macOS ------
|
|
||||||
# Finder metadata
|
|
||||||
**/.DS_Store
|
|
||||||
|
|
||||||
# LLM conversations that should remain private
|
|
||||||
docs/conversations/
|
|
||||||
|
|
|
||||||
|
|
@ -420,17 +420,20 @@ Check out our experimental system for `guest`_-mode controlled
|
||||||
|
|
||||||
|
|
||||||
async def aio_echo_server(
|
async def aio_echo_server(
|
||||||
chan: tractor.to_asyncio.LinkedTaskChannel,
|
to_trio: trio.MemorySendChannel,
|
||||||
|
from_trio: asyncio.Queue,
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|
||||||
# a first message must be sent **from** this ``asyncio``
|
# a first message must be sent **from** this ``asyncio``
|
||||||
# task or the ``trio`` side will never unblock from
|
# task or the ``trio`` side will never unblock from
|
||||||
# ``tractor.to_asyncio.open_channel_from():``
|
# ``tractor.to_asyncio.open_channel_from():``
|
||||||
chan.started_nowait('start')
|
to_trio.send_nowait('start')
|
||||||
|
|
||||||
|
# XXX: this uses an ``from_trio: asyncio.Queue`` currently but we
|
||||||
|
# should probably offer something better.
|
||||||
while True:
|
while True:
|
||||||
# echo the msg back
|
# echo the msg back
|
||||||
chan.send_nowait(await chan.get())
|
to_trio.send_nowait(await from_trio.get())
|
||||||
await asyncio.sleep(0)
|
await asyncio.sleep(0)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -442,7 +445,7 @@ Check out our experimental system for `guest`_-mode controlled
|
||||||
# message.
|
# message.
|
||||||
async with tractor.to_asyncio.open_channel_from(
|
async with tractor.to_asyncio.open_channel_from(
|
||||||
aio_echo_server,
|
aio_echo_server,
|
||||||
) as (chan, first):
|
) as (first, chan):
|
||||||
|
|
||||||
assert first == 'start'
|
assert first == 'start'
|
||||||
await ctx.started(first)
|
await ctx.started(first)
|
||||||
|
|
@ -501,10 +504,8 @@ Yes, we spawn a python process, run ``asyncio``, start ``trio`` on the
|
||||||
``asyncio`` loop, then send commands to the ``trio`` scheduled tasks to
|
``asyncio`` loop, then send commands to the ``trio`` scheduled tasks to
|
||||||
tell ``asyncio`` tasks what to do XD
|
tell ``asyncio`` tasks what to do XD
|
||||||
|
|
||||||
The ``asyncio``-side task receives a single
|
We need help refining the `asyncio`-side channel API to be more
|
||||||
``chan: LinkedTaskChannel`` handle providing a ``trio``-like
|
`trio`-like. Feel free to sling your opinion in `#273`_!
|
||||||
API: ``.started_nowait()``, ``.send_nowait()``, ``.get()``
|
|
||||||
and more. Feel free to sling your opinion in `#273`_!
|
|
||||||
|
|
||||||
|
|
||||||
.. _#273: https://github.com/goodboy/tractor/issues/273
|
.. _#273: https://github.com/goodboy/tractor/issues/273
|
||||||
|
|
@ -640,15 +641,13 @@ Help us push toward the future of distributed `Python`.
|
||||||
- Typed capability-based (dialog) protocols ( see `#196
|
- Typed capability-based (dialog) protocols ( see `#196
|
||||||
<https://github.com/goodboy/tractor/issues/196>`_ with draft work
|
<https://github.com/goodboy/tractor/issues/196>`_ with draft work
|
||||||
started in `#311 <https://github.com/goodboy/tractor/pull/311>`_)
|
started in `#311 <https://github.com/goodboy/tractor/pull/311>`_)
|
||||||
- **macOS is now officially supported** and tested in CI
|
- We **recently disabled CI-testing on windows** and need help getting
|
||||||
alongside Linux!
|
it running again! (see `#327
|
||||||
- We **recently disabled CI-testing on windows** and need
|
<https://github.com/goodboy/tractor/pull/327>`_). **We do have windows
|
||||||
help getting it running again! (see `#327
|
support** (and have for quite a while) but since no active hacker
|
||||||
<https://github.com/goodboy/tractor/pull/327>`_). **We do
|
exists in the user-base to help test on that OS, for now we're not
|
||||||
have windows support** (and have for quite a while) but
|
actively maintaining testing due to the added hassle and general
|
||||||
since no active hacker exists in the user-base to help
|
latency..
|
||||||
test on that OS, for now we're not actively maintaining
|
|
||||||
testing due to the added hassle and general latency..
|
|
||||||
|
|
||||||
|
|
||||||
Feel like saying hi?
|
Feel like saying hi?
|
||||||
|
|
|
||||||
|
|
@ -65,7 +65,7 @@ async def trio_ctx(
|
||||||
to_asyncio.open_channel_from(
|
to_asyncio.open_channel_from(
|
||||||
bp_then_error,
|
bp_then_error,
|
||||||
# raise_after_bp=not bp_before_started,
|
# raise_after_bp=not bp_before_started,
|
||||||
) as (chan, first),
|
) as (first, chan),
|
||||||
|
|
||||||
trio.open_nursery() as tn,
|
trio.open_nursery() as tn,
|
||||||
):
|
):
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,6 @@ Verify we can dump a `stackscope` tree on a hang.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
import os
|
import os
|
||||||
import platform
|
|
||||||
import signal
|
import signal
|
||||||
|
|
||||||
import trio
|
import trio
|
||||||
|
|
@ -32,26 +31,13 @@ async def main(
|
||||||
from_test: bool = False,
|
from_test: bool = False,
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|
||||||
if platform.system() != 'Darwin':
|
|
||||||
tpt = 'uds'
|
|
||||||
else:
|
|
||||||
# XXX, precisely we can't use pytest's tmp-path generation
|
|
||||||
# for tests.. apparently because:
|
|
||||||
#
|
|
||||||
# > The OSError: AF_UNIX path too long in macOS Python occurs
|
|
||||||
# > because the path to the Unix domain socket exceeds the
|
|
||||||
# > operating system's maximum path length limit (around 104
|
|
||||||
#
|
|
||||||
# WHICH IS just, wtf hillarious XD
|
|
||||||
tpt = 'tcp'
|
|
||||||
|
|
||||||
async with (
|
async with (
|
||||||
tractor.open_nursery(
|
tractor.open_nursery(
|
||||||
debug_mode=True,
|
debug_mode=True,
|
||||||
enable_stack_on_sig=True,
|
enable_stack_on_sig=True,
|
||||||
# maybe_enable_greenback=False,
|
# maybe_enable_greenback=False,
|
||||||
loglevel='devx',
|
loglevel='devx',
|
||||||
enable_transports=[tpt],
|
enable_transports=['uds'],
|
||||||
) as an,
|
) as an,
|
||||||
):
|
):
|
||||||
ptl: tractor.Portal = await an.start_actor(
|
ptl: tractor.Portal = await an.start_actor(
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,3 @@
|
||||||
import platform
|
|
||||||
|
|
||||||
import tractor
|
import tractor
|
||||||
import trio
|
import trio
|
||||||
|
|
||||||
|
|
@ -36,22 +34,9 @@ async def just_bp(
|
||||||
|
|
||||||
async def main():
|
async def main():
|
||||||
|
|
||||||
if platform.system() != 'Darwin':
|
|
||||||
tpt = 'uds'
|
|
||||||
else:
|
|
||||||
# XXX, precisely we can't use pytest's tmp-path generation
|
|
||||||
# for tests.. apparently because:
|
|
||||||
#
|
|
||||||
# > The OSError: AF_UNIX path too long in macOS Python occurs
|
|
||||||
# > because the path to the Unix domain socket exceeds the
|
|
||||||
# > operating system's maximum path length limit (around 104
|
|
||||||
#
|
|
||||||
# WHICH IS just, wtf hillarious XD
|
|
||||||
tpt = 'tcp'
|
|
||||||
|
|
||||||
async with tractor.open_nursery(
|
async with tractor.open_nursery(
|
||||||
debug_mode=True,
|
debug_mode=True,
|
||||||
enable_transports=[tpt],
|
enable_transports=['uds'],
|
||||||
loglevel='devx',
|
loglevel='devx',
|
||||||
) as n:
|
) as n:
|
||||||
p = await n.start_actor(
|
p = await n.start_actor(
|
||||||
|
|
|
||||||
|
|
@ -90,7 +90,7 @@ async def main() -> list[int]:
|
||||||
# yes, a nursery which spawns `trio`-"actors" B)
|
# yes, a nursery which spawns `trio`-"actors" B)
|
||||||
an: ActorNursery
|
an: ActorNursery
|
||||||
async with tractor.open_nursery(
|
async with tractor.open_nursery(
|
||||||
loglevel='error',
|
loglevel='cancel',
|
||||||
# debug_mode=True,
|
# debug_mode=True,
|
||||||
) as an:
|
) as an:
|
||||||
|
|
||||||
|
|
@ -118,10 +118,8 @@ async def main() -> list[int]:
|
||||||
cancelled: bool = await portal.cancel_actor()
|
cancelled: bool = await portal.cancel_actor()
|
||||||
assert cancelled
|
assert cancelled
|
||||||
|
|
||||||
print(
|
print(f"STREAM TIME = {time.time() - start}")
|
||||||
f"STREAM TIME = {time.time() - start}\n"
|
print(f"STREAM + SPAWN TIME = {time.time() - pre_start}")
|
||||||
f"STREAM + SPAWN TIME = {time.time() - pre_start}\n"
|
|
||||||
)
|
|
||||||
assert result_stream == list(range(seed))
|
assert result_stream == list(range(seed))
|
||||||
return result_stream
|
return result_stream
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -11,17 +11,21 @@ import tractor
|
||||||
|
|
||||||
|
|
||||||
async def aio_echo_server(
|
async def aio_echo_server(
|
||||||
chan: tractor.to_asyncio.LinkedTaskChannel,
|
to_trio: trio.MemorySendChannel,
|
||||||
|
from_trio: asyncio.Queue,
|
||||||
|
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|
||||||
# a first message must be sent **from** this ``asyncio``
|
# a first message must be sent **from** this ``asyncio``
|
||||||
# task or the ``trio`` side will never unblock from
|
# task or the ``trio`` side will never unblock from
|
||||||
# ``tractor.to_asyncio.open_channel_from():``
|
# ``tractor.to_asyncio.open_channel_from():``
|
||||||
chan.started_nowait('start')
|
to_trio.send_nowait('start')
|
||||||
|
|
||||||
|
# XXX: this uses an ``from_trio: asyncio.Queue`` currently but we
|
||||||
|
# should probably offer something better.
|
||||||
while True:
|
while True:
|
||||||
# echo the msg back
|
# echo the msg back
|
||||||
chan.send_nowait(await chan.get())
|
to_trio.send_nowait(await from_trio.get())
|
||||||
await asyncio.sleep(0)
|
await asyncio.sleep(0)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -33,7 +37,7 @@ async def trio_to_aio_echo_server(
|
||||||
# message.
|
# message.
|
||||||
async with tractor.to_asyncio.open_channel_from(
|
async with tractor.to_asyncio.open_channel_from(
|
||||||
aio_echo_server,
|
aio_echo_server,
|
||||||
) as (chan, first):
|
) as (first, chan):
|
||||||
|
|
||||||
assert first == 'start'
|
assert first == 'start'
|
||||||
await ctx.started(first)
|
await ctx.started(first)
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,7 @@ name = "tractor"
|
||||||
version = "0.1.0a6dev0"
|
version = "0.1.0a6dev0"
|
||||||
description = 'structured concurrent `trio`-"actors"'
|
description = 'structured concurrent `trio`-"actors"'
|
||||||
authors = [{ name = "Tyler Goodlet", email = "goodboy_foss@protonmail.com" }]
|
authors = [{ name = "Tyler Goodlet", email = "goodboy_foss@protonmail.com" }]
|
||||||
requires-python = ">=3.12, <3.14"
|
requires-python = ">= 3.12"
|
||||||
readme = "docs/README.rst"
|
readme = "docs/README.rst"
|
||||||
license = "AGPL-3.0-or-later"
|
license = "AGPL-3.0-or-later"
|
||||||
keywords = [
|
keywords = [
|
||||||
|
|
@ -24,13 +24,11 @@ keywords = [
|
||||||
classifiers = [
|
classifiers = [
|
||||||
"Development Status :: 3 - Alpha",
|
"Development Status :: 3 - Alpha",
|
||||||
"Operating System :: POSIX :: Linux",
|
"Operating System :: POSIX :: Linux",
|
||||||
"Operating System :: MacOS",
|
|
||||||
"Framework :: Trio",
|
"Framework :: Trio",
|
||||||
"License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
|
"License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
|
||||||
"Programming Language :: Python :: Implementation :: CPython",
|
"Programming Language :: Python :: Implementation :: CPython",
|
||||||
"Programming Language :: Python :: 3 :: Only",
|
"Programming Language :: Python :: 3 :: Only",
|
||||||
"Programming Language :: Python :: 3.12",
|
"Programming Language :: Python :: 3.11",
|
||||||
"Programming Language :: Python :: 3.13",
|
|
||||||
"Topic :: System :: Distributed Computing",
|
"Topic :: System :: Distributed Computing",
|
||||||
]
|
]
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
|
@ -164,7 +162,6 @@ all_bullets = true
|
||||||
|
|
||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
minversion = '6.0'
|
minversion = '6.0'
|
||||||
# https://docs.pytest.org/en/stable/reference/reference.html#configuration-options
|
|
||||||
testpaths = [
|
testpaths = [
|
||||||
'tests'
|
'tests'
|
||||||
]
|
]
|
||||||
|
|
@ -175,17 +172,10 @@ addopts = [
|
||||||
'--import-mode=importlib',
|
'--import-mode=importlib',
|
||||||
# don't show frickin captured logs AGAIN in the report..
|
# don't show frickin captured logs AGAIN in the report..
|
||||||
'--show-capture=no',
|
'--show-capture=no',
|
||||||
|
|
||||||
# disable `xonsh` plugin
|
|
||||||
# https://docs.pytest.org/en/stable/how-to/plugins.html#disabling-plugins-from-autoloading
|
|
||||||
# https://docs.pytest.org/en/stable/how-to/plugins.html#deactivating-unregistering-a-plugin-by-name
|
|
||||||
'-p no:xonsh'
|
|
||||||
]
|
]
|
||||||
log_cli = false
|
log_cli = false
|
||||||
# TODO: maybe some of these layout choices?
|
# TODO: maybe some of these layout choices?
|
||||||
# https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
|
# https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
|
||||||
# pythonpath = "src"
|
# pythonpath = "src"
|
||||||
|
|
||||||
# https://docs.pytest.org/en/stable/reference/reference.html#confval-console_output_style
|
|
||||||
console_output_style = 'progress'
|
|
||||||
# ------ tool.pytest ------
|
# ------ tool.pytest ------
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,8 @@
|
||||||
|
# vim: ft=ini
|
||||||
|
# pytest.ini for tractor
|
||||||
|
|
||||||
|
[pytest]
|
||||||
|
# don't show frickin captured logs AGAIN in the report..
|
||||||
|
addopts = --show-capture='no'
|
||||||
|
log_cli = false
|
||||||
|
; minversion = 6.0
|
||||||
|
|
@ -11,7 +11,6 @@ import platform
|
||||||
import time
|
import time
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
import tractor
|
|
||||||
from tractor._testing import (
|
from tractor._testing import (
|
||||||
examples_dir as examples_dir,
|
examples_dir as examples_dir,
|
||||||
tractor_test as tractor_test,
|
tractor_test as tractor_test,
|
||||||
|
|
@ -23,8 +22,6 @@ pytest_plugins: list[str] = [
|
||||||
'tractor._testing.pytest',
|
'tractor._testing.pytest',
|
||||||
]
|
]
|
||||||
|
|
||||||
_ci_env: bool = os.environ.get('CI', False)
|
|
||||||
_non_linux: bool = platform.system() != 'Linux'
|
|
||||||
|
|
||||||
# Sending signal.SIGINT on subprocess fails on windows. Use CTRL_* alternatives
|
# Sending signal.SIGINT on subprocess fails on windows. Use CTRL_* alternatives
|
||||||
if platform.system() == 'Windows':
|
if platform.system() == 'Windows':
|
||||||
|
|
@ -37,8 +34,9 @@ else:
|
||||||
_INT_SIGNAL = signal.SIGINT
|
_INT_SIGNAL = signal.SIGINT
|
||||||
_INT_RETURN_CODE = 1 if sys.version_info < (3, 8) else -signal.SIGINT.value
|
_INT_RETURN_CODE = 1 if sys.version_info < (3, 8) else -signal.SIGINT.value
|
||||||
_PROC_SPAWN_WAIT = (
|
_PROC_SPAWN_WAIT = (
|
||||||
2 if _ci_env
|
0.6
|
||||||
else 1
|
if sys.version_info < (3, 7)
|
||||||
|
else 0.4
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -46,10 +44,6 @@ no_windows = pytest.mark.skipif(
|
||||||
platform.system() == "Windows",
|
platform.system() == "Windows",
|
||||||
reason="Test is unsupported on windows",
|
reason="Test is unsupported on windows",
|
||||||
)
|
)
|
||||||
no_macos = pytest.mark.skipif(
|
|
||||||
platform.system() == "Darwin",
|
|
||||||
reason="Test is unsupported on MacOS",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def pytest_addoption(
|
def pytest_addoption(
|
||||||
|
|
@ -67,7 +61,7 @@ def pytest_addoption(
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope='session', autouse=True)
|
@pytest.fixture(scope='session', autouse=True)
|
||||||
def loglevel(request) -> str:
|
def loglevel(request):
|
||||||
import tractor
|
import tractor
|
||||||
orig = tractor.log._default_loglevel
|
orig = tractor.log._default_loglevel
|
||||||
level = tractor.log._default_loglevel = request.config.option.loglevel
|
level = tractor.log._default_loglevel = request.config.option.loglevel
|
||||||
|
|
@ -75,44 +69,12 @@ def loglevel(request) -> str:
|
||||||
level=level,
|
level=level,
|
||||||
name='tractor', # <- enable root logger
|
name='tractor', # <- enable root logger
|
||||||
)
|
)
|
||||||
log.info(
|
log.info(f'Test-harness logging level: {level}\n')
|
||||||
f'Test-harness set runtime loglevel: {level!r}\n'
|
|
||||||
)
|
|
||||||
yield level
|
yield level
|
||||||
tractor.log._default_loglevel = orig
|
tractor.log._default_loglevel = orig
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope='function')
|
_ci_env: bool = os.environ.get('CI', False)
|
||||||
def test_log(
|
|
||||||
request,
|
|
||||||
loglevel: str,
|
|
||||||
) -> tractor.log.StackLevelAdapter:
|
|
||||||
'''
|
|
||||||
Deliver a per test-module-fn logger instance for reporting from
|
|
||||||
within actual test bodies/fixtures.
|
|
||||||
|
|
||||||
For example this can be handy to report certain error cases from
|
|
||||||
exception handlers using `test_log.exception()`.
|
|
||||||
|
|
||||||
'''
|
|
||||||
modname: str = request.function.__module__
|
|
||||||
log = tractor.log.get_logger(
|
|
||||||
name=modname, # <- enable root logger
|
|
||||||
# pkg_name='tests',
|
|
||||||
)
|
|
||||||
_log = tractor.log.get_console_log(
|
|
||||||
level=loglevel,
|
|
||||||
logger=log,
|
|
||||||
name=modname,
|
|
||||||
# pkg_name='tests',
|
|
||||||
)
|
|
||||||
_log.debug(
|
|
||||||
f'In-test-logging requested\n'
|
|
||||||
f'test_log.name: {log.name!r}\n'
|
|
||||||
f'level: {loglevel!r}\n'
|
|
||||||
|
|
||||||
)
|
|
||||||
yield _log
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope='session')
|
@pytest.fixture(scope='session')
|
||||||
|
|
@ -148,8 +110,6 @@ def daemon(
|
||||||
testdir: pytest.Pytester,
|
testdir: pytest.Pytester,
|
||||||
reg_addr: tuple[str, int],
|
reg_addr: tuple[str, int],
|
||||||
tpt_proto: str,
|
tpt_proto: str,
|
||||||
ci_env: bool,
|
|
||||||
test_log: tractor.log.StackLevelAdapter,
|
|
||||||
|
|
||||||
) -> subprocess.Popen:
|
) -> subprocess.Popen:
|
||||||
'''
|
'''
|
||||||
|
|
@ -165,12 +125,10 @@ def daemon(
|
||||||
"import tractor; "
|
"import tractor; "
|
||||||
"tractor.run_daemon([], "
|
"tractor.run_daemon([], "
|
||||||
"registry_addrs={reg_addrs}, "
|
"registry_addrs={reg_addrs}, "
|
||||||
"enable_transports={enable_tpts}, "
|
|
||||||
"debug_mode={debug_mode}, "
|
"debug_mode={debug_mode}, "
|
||||||
"loglevel={ll})"
|
"loglevel={ll})"
|
||||||
).format(
|
).format(
|
||||||
reg_addrs=str([reg_addr]),
|
reg_addrs=str([reg_addr]),
|
||||||
enable_tpts=str([tpt_proto]),
|
|
||||||
ll="'{}'".format(loglevel) if loglevel else None,
|
ll="'{}'".format(loglevel) if loglevel else None,
|
||||||
debug_mode=debug_mode,
|
debug_mode=debug_mode,
|
||||||
)
|
)
|
||||||
|
|
@ -189,25 +147,13 @@ def daemon(
|
||||||
**kwargs,
|
**kwargs,
|
||||||
)
|
)
|
||||||
|
|
||||||
# TODO! we should poll for the registry socket-bind to take place
|
|
||||||
# and only once that's done yield to the requester!
|
|
||||||
# -[ ] TCP: use the `._root.open_root_actor()`::`ping_tpt_socket()`
|
|
||||||
# closure!
|
|
||||||
# -[ ] UDS: can we do something similar for 'pinging" the
|
|
||||||
# file-socket?
|
|
||||||
#
|
|
||||||
global _PROC_SPAWN_WAIT
|
|
||||||
# UDS sockets are **really** fast to bind()/listen()/connect()
|
# UDS sockets are **really** fast to bind()/listen()/connect()
|
||||||
# so it's often required that we delay a bit more starting
|
# so it's often required that we delay a bit more starting
|
||||||
# the first actor-tree..
|
# the first actor-tree..
|
||||||
if tpt_proto == 'uds':
|
if tpt_proto == 'uds':
|
||||||
_PROC_SPAWN_WAIT += 1.6
|
global _PROC_SPAWN_WAIT
|
||||||
|
_PROC_SPAWN_WAIT = 0.6
|
||||||
|
|
||||||
if _non_linux and ci_env:
|
|
||||||
_PROC_SPAWN_WAIT += 1
|
|
||||||
|
|
||||||
# XXX, allow time for the sub-py-proc to boot up.
|
|
||||||
# !TODO, see ping-polling ideas above!
|
|
||||||
time.sleep(_PROC_SPAWN_WAIT)
|
time.sleep(_PROC_SPAWN_WAIT)
|
||||||
|
|
||||||
assert not proc.returncode
|
assert not proc.returncode
|
||||||
|
|
@ -217,30 +163,18 @@ def daemon(
|
||||||
# XXX! yeah.. just be reaaal careful with this bc sometimes it
|
# XXX! yeah.. just be reaaal careful with this bc sometimes it
|
||||||
# can lock up on the `_io.BufferedReader` and hang..
|
# can lock up on the `_io.BufferedReader` and hang..
|
||||||
stderr: str = proc.stderr.read().decode()
|
stderr: str = proc.stderr.read().decode()
|
||||||
stdout: str = proc.stdout.read().decode()
|
if stderr:
|
||||||
if (
|
|
||||||
stderr
|
|
||||||
or
|
|
||||||
stdout
|
|
||||||
):
|
|
||||||
print(
|
print(
|
||||||
f'Daemon actor tree produced output:\n'
|
f'Daemon actor tree produced STDERR:\n'
|
||||||
f'{proc.args}\n'
|
f'{proc.args}\n'
|
||||||
f'\n'
|
f'\n'
|
||||||
f'stderr: {stderr!r}\n'
|
f'{stderr}\n'
|
||||||
f'stdout: {stdout!r}\n'
|
|
||||||
)
|
)
|
||||||
|
if proc.returncode != -2:
|
||||||
if (rc := proc.returncode) != -2:
|
raise RuntimeError(
|
||||||
msg: str = (
|
'Daemon actor tree failed !?\n'
|
||||||
f'Daemon actor tree was not cancelled !?\n'
|
f'{proc.args}\n'
|
||||||
f'proc.args: {proc.args!r}\n'
|
|
||||||
f'proc.returncode: {rc!r}\n'
|
|
||||||
)
|
)
|
||||||
if rc < 0:
|
|
||||||
raise RuntimeError(msg)
|
|
||||||
|
|
||||||
test_log.error(msg)
|
|
||||||
|
|
||||||
|
|
||||||
# @pytest.fixture(autouse=True)
|
# @pytest.fixture(autouse=True)
|
||||||
|
|
|
||||||
|
|
@ -3,9 +3,8 @@
|
||||||
|
|
||||||
'''
|
'''
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
import platform
|
|
||||||
import signal
|
|
||||||
import time
|
import time
|
||||||
|
import signal
|
||||||
from typing import (
|
from typing import (
|
||||||
Callable,
|
Callable,
|
||||||
TYPE_CHECKING,
|
TYPE_CHECKING,
|
||||||
|
|
@ -34,17 +33,6 @@ if TYPE_CHECKING:
|
||||||
from pexpect import pty_spawn
|
from pexpect import pty_spawn
|
||||||
|
|
||||||
|
|
||||||
_non_linux: bool = platform.system() != 'Linux'
|
|
||||||
|
|
||||||
|
|
||||||
def pytest_configure(config):
|
|
||||||
# register custom marks to avoid warnings see,
|
|
||||||
# https://docs.pytest.org/en/stable/how-to/writing_plugins.html#registering-custom-markers
|
|
||||||
config.addinivalue_line(
|
|
||||||
'markers',
|
|
||||||
'ctlcs_bish: test will (likely) not behave under SIGINT..'
|
|
||||||
)
|
|
||||||
|
|
||||||
# a fn that sub-instantiates a `pexpect.spawn()`
|
# a fn that sub-instantiates a `pexpect.spawn()`
|
||||||
# and returns it.
|
# and returns it.
|
||||||
type PexpectSpawner = Callable[
|
type PexpectSpawner = Callable[
|
||||||
|
|
@ -80,16 +68,12 @@ def spawn(
|
||||||
|
|
||||||
'''
|
'''
|
||||||
import os
|
import os
|
||||||
# disable colored tbs
|
|
||||||
os.environ['PYTHON_COLORS'] = '0'
|
os.environ['PYTHON_COLORS'] = '0'
|
||||||
# disable all ANSI color output
|
|
||||||
# os.environ['NO_COLOR'] = '1'
|
|
||||||
|
|
||||||
spawned: PexpectSpawner|None = None
|
spawned: PexpectSpawner|None = None
|
||||||
|
|
||||||
def _spawn(
|
def _spawn(
|
||||||
cmd: str,
|
cmd: str,
|
||||||
expect_timeout: float = 4,
|
|
||||||
**mkcmd_kwargs,
|
**mkcmd_kwargs,
|
||||||
) -> pty_spawn.spawn:
|
) -> pty_spawn.spawn:
|
||||||
nonlocal spawned
|
nonlocal spawned
|
||||||
|
|
@ -99,17 +83,11 @@ def spawn(
|
||||||
cmd,
|
cmd,
|
||||||
**mkcmd_kwargs,
|
**mkcmd_kwargs,
|
||||||
),
|
),
|
||||||
expect_timeout=(timeout:=(
|
expect_timeout=3,
|
||||||
expect_timeout + 6
|
|
||||||
if _non_linux and _ci_env
|
|
||||||
else expect_timeout
|
|
||||||
)),
|
|
||||||
# preexec_fn=unset_colors,
|
# preexec_fn=unset_colors,
|
||||||
# ^TODO? get `pytest` core to expose underlying
|
# ^TODO? get `pytest` core to expose underlying
|
||||||
# `pexpect.spawn()` stuff?
|
# `pexpect.spawn()` stuff?
|
||||||
)
|
)
|
||||||
# sanity
|
|
||||||
assert spawned.timeout == timeout
|
|
||||||
return spawned
|
return spawned
|
||||||
|
|
||||||
# such that test-dep can pass input script name.
|
# such that test-dep can pass input script name.
|
||||||
|
|
@ -168,8 +146,6 @@ def ctlc(
|
||||||
mark.name == 'ctlcs_bish'
|
mark.name == 'ctlcs_bish'
|
||||||
and
|
and
|
||||||
use_ctlc
|
use_ctlc
|
||||||
and
|
|
||||||
all(mark.args)
|
|
||||||
):
|
):
|
||||||
pytest.skip(
|
pytest.skip(
|
||||||
f'Test {node} prolly uses something from the stdlib (namely `asyncio`..)\n'
|
f'Test {node} prolly uses something from the stdlib (namely `asyncio`..)\n'
|
||||||
|
|
@ -275,13 +251,12 @@ def assert_before(
|
||||||
err_on_false=True,
|
err_on_false=True,
|
||||||
**kwargs
|
**kwargs
|
||||||
)
|
)
|
||||||
return str(child.before.decode())
|
|
||||||
|
|
||||||
|
|
||||||
def do_ctlc(
|
def do_ctlc(
|
||||||
child,
|
child,
|
||||||
count: int = 3,
|
count: int = 3,
|
||||||
delay: float|None = None,
|
delay: float = 0.1,
|
||||||
patt: str|None = None,
|
patt: str|None = None,
|
||||||
|
|
||||||
# expect repl UX to reprint the prompt after every
|
# expect repl UX to reprint the prompt after every
|
||||||
|
|
@ -293,7 +268,6 @@ def do_ctlc(
|
||||||
) -> str|None:
|
) -> str|None:
|
||||||
|
|
||||||
before: str|None = None
|
before: str|None = None
|
||||||
delay = delay or 0.1
|
|
||||||
|
|
||||||
# make sure ctl-c sends don't do anything but repeat output
|
# make sure ctl-c sends don't do anything but repeat output
|
||||||
for _ in range(count):
|
for _ in range(count):
|
||||||
|
|
@ -304,10 +278,7 @@ def do_ctlc(
|
||||||
# if you run this test manually it works just fine..
|
# if you run this test manually it works just fine..
|
||||||
if expect_prompt:
|
if expect_prompt:
|
||||||
time.sleep(delay)
|
time.sleep(delay)
|
||||||
child.expect(
|
child.expect(PROMPT)
|
||||||
PROMPT,
|
|
||||||
timeout=(child.timeout * 2) if _ci_env else child.timeout,
|
|
||||||
)
|
|
||||||
before = str(child.before.decode())
|
before = str(child.before.decode())
|
||||||
time.sleep(delay)
|
time.sleep(delay)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -37,9 +37,6 @@ from .conftest import (
|
||||||
in_prompt_msg,
|
in_prompt_msg,
|
||||||
assert_before,
|
assert_before,
|
||||||
)
|
)
|
||||||
from ..conftest import (
|
|
||||||
_ci_env,
|
|
||||||
)
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from ..conftest import PexpectSpawner
|
from ..conftest import PexpectSpawner
|
||||||
|
|
@ -54,14 +51,13 @@ if TYPE_CHECKING:
|
||||||
# - recurrent root errors
|
# - recurrent root errors
|
||||||
|
|
||||||
|
|
||||||
_non_linux: bool = platform.system() != 'Linux'
|
|
||||||
|
|
||||||
if platform.system() == 'Windows':
|
if platform.system() == 'Windows':
|
||||||
pytest.skip(
|
pytest.skip(
|
||||||
'Debugger tests have no windows support (yet)',
|
'Debugger tests have no windows support (yet)',
|
||||||
allow_module_level=True,
|
allow_module_level=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# TODO: was trying to this xfail style but some weird bug i see in CI
|
# TODO: was trying to this xfail style but some weird bug i see in CI
|
||||||
# that's happening at collect time.. pretty soon gonna dump actions i'm
|
# that's happening at collect time.. pretty soon gonna dump actions i'm
|
||||||
# thinkin...
|
# thinkin...
|
||||||
|
|
@ -197,11 +193,6 @@ def test_root_actor_bp_forever(
|
||||||
child.expect(EOF)
|
child.expect(EOF)
|
||||||
|
|
||||||
|
|
||||||
# skip on non-Linux CI
|
|
||||||
@pytest.mark.ctlcs_bish(
|
|
||||||
_non_linux,
|
|
||||||
_ci_env,
|
|
||||||
)
|
|
||||||
@pytest.mark.parametrize(
|
@pytest.mark.parametrize(
|
||||||
'do_next',
|
'do_next',
|
||||||
(True, False),
|
(True, False),
|
||||||
|
|
@ -267,11 +258,6 @@ def test_subactor_error(
|
||||||
child.expect(EOF)
|
child.expect(EOF)
|
||||||
|
|
||||||
|
|
||||||
# skip on non-Linux CI
|
|
||||||
@pytest.mark.ctlcs_bish(
|
|
||||||
_non_linux,
|
|
||||||
_ci_env,
|
|
||||||
)
|
|
||||||
def test_subactor_breakpoint(
|
def test_subactor_breakpoint(
|
||||||
spawn,
|
spawn,
|
||||||
ctlc: bool,
|
ctlc: bool,
|
||||||
|
|
@ -494,24 +480,8 @@ def test_multi_daemon_subactors(
|
||||||
stream.
|
stream.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
non_linux = _non_linux
|
|
||||||
if non_linux and ctlc:
|
|
||||||
pytest.skip(
|
|
||||||
'Ctl-c + MacOS is too unreliable/racy for this test..\n'
|
|
||||||
)
|
|
||||||
# !TODO, if someone with more patience then i wants to muck
|
|
||||||
# with the timings on this please feel free to see all the
|
|
||||||
# `non_linux` branching logic i added on my first attempt
|
|
||||||
# below!
|
|
||||||
#
|
|
||||||
# my conclusion was that if i were to run the script
|
|
||||||
# manually, and thus as slowly as a human would, the test
|
|
||||||
# would and should pass as described in this test fn, however
|
|
||||||
# after fighting with it for >= 1hr. i decided more then
|
|
||||||
# likely the more extensive `linux` testing should cover most
|
|
||||||
# regressions.
|
|
||||||
|
|
||||||
child = spawn('multi_daemon_subactors')
|
child = spawn('multi_daemon_subactors')
|
||||||
|
|
||||||
child.expect(PROMPT)
|
child.expect(PROMPT)
|
||||||
|
|
||||||
# there can be a race for which subactor will acquire
|
# there can be a race for which subactor will acquire
|
||||||
|
|
@ -541,19 +511,8 @@ def test_multi_daemon_subactors(
|
||||||
else:
|
else:
|
||||||
raise ValueError('Neither log msg was found !?')
|
raise ValueError('Neither log msg was found !?')
|
||||||
|
|
||||||
non_linux_delay: float = 0.3
|
|
||||||
if ctlc:
|
if ctlc:
|
||||||
do_ctlc(
|
do_ctlc(child)
|
||||||
child,
|
|
||||||
delay=(
|
|
||||||
non_linux_delay
|
|
||||||
if non_linux
|
|
||||||
else None
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
if non_linux:
|
|
||||||
time.sleep(1)
|
|
||||||
|
|
||||||
# NOTE: previously since we did not have clobber prevention
|
# NOTE: previously since we did not have clobber prevention
|
||||||
# in the root actor this final resume could result in the debugger
|
# in the root actor this final resume could result in the debugger
|
||||||
|
|
@ -584,66 +543,33 @@ def test_multi_daemon_subactors(
|
||||||
# assert "in use by child ('bp_forever'," in before
|
# assert "in use by child ('bp_forever'," in before
|
||||||
|
|
||||||
if ctlc:
|
if ctlc:
|
||||||
do_ctlc(
|
do_ctlc(child)
|
||||||
child,
|
|
||||||
delay=(
|
|
||||||
non_linux_delay
|
|
||||||
if non_linux
|
|
||||||
else None
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
if non_linux:
|
|
||||||
time.sleep(1)
|
|
||||||
|
|
||||||
# expect another breakpoint actor entry
|
# expect another breakpoint actor entry
|
||||||
child.sendline('c')
|
child.sendline('c')
|
||||||
child.expect(PROMPT)
|
child.expect(PROMPT)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
before: str = assert_before(
|
assert_before(
|
||||||
child,
|
child,
|
||||||
bp_forev_parts,
|
bp_forev_parts,
|
||||||
)
|
)
|
||||||
except AssertionError:
|
except AssertionError:
|
||||||
before: str = assert_before(
|
assert_before(
|
||||||
child,
|
child,
|
||||||
name_error_parts,
|
name_error_parts,
|
||||||
)
|
)
|
||||||
|
|
||||||
else:
|
else:
|
||||||
if ctlc:
|
if ctlc:
|
||||||
before: str = do_ctlc(
|
do_ctlc(child)
|
||||||
child,
|
|
||||||
delay=(
|
|
||||||
non_linux_delay
|
|
||||||
if non_linux
|
|
||||||
else None
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
if non_linux:
|
|
||||||
time.sleep(1)
|
|
||||||
|
|
||||||
# should crash with the 2nd name error (simulates
|
# should crash with the 2nd name error (simulates
|
||||||
# a retry) and then the root eventually (boxed) errors
|
# a retry) and then the root eventually (boxed) errors
|
||||||
# after 1 or more further bp actor entries.
|
# after 1 or more further bp actor entries.
|
||||||
|
|
||||||
child.sendline('c')
|
child.sendline('c')
|
||||||
try:
|
child.expect(PROMPT)
|
||||||
child.expect(
|
|
||||||
PROMPT,
|
|
||||||
timeout=3,
|
|
||||||
)
|
|
||||||
except EOF:
|
|
||||||
before: str = child.before.decode()
|
|
||||||
print(
|
|
||||||
f'\n'
|
|
||||||
f'??? NEVER RXED `pdb` PROMPT ???\n'
|
|
||||||
f'\n'
|
|
||||||
f'{before}\n'
|
|
||||||
)
|
|
||||||
raise
|
|
||||||
|
|
||||||
assert_before(
|
assert_before(
|
||||||
child,
|
child,
|
||||||
name_error_parts,
|
name_error_parts,
|
||||||
|
|
@ -763,8 +689,7 @@ def test_multi_subactors_root_errors(
|
||||||
|
|
||||||
@has_nested_actors
|
@has_nested_actors
|
||||||
def test_multi_nested_subactors_error_through_nurseries(
|
def test_multi_nested_subactors_error_through_nurseries(
|
||||||
ci_env: bool,
|
spawn,
|
||||||
spawn: PexpectSpawner,
|
|
||||||
|
|
||||||
# TODO: address debugger issue for nested tree:
|
# TODO: address debugger issue for nested tree:
|
||||||
# https://github.com/goodboy/tractor/issues/320
|
# https://github.com/goodboy/tractor/issues/320
|
||||||
|
|
@ -787,16 +712,7 @@ def test_multi_nested_subactors_error_through_nurseries(
|
||||||
|
|
||||||
for send_char in itertools.cycle(['c', 'q']):
|
for send_char in itertools.cycle(['c', 'q']):
|
||||||
try:
|
try:
|
||||||
child.expect(
|
child.expect(PROMPT)
|
||||||
PROMPT,
|
|
||||||
timeout=(
|
|
||||||
6 if (
|
|
||||||
_non_linux
|
|
||||||
and
|
|
||||||
ci_env
|
|
||||||
) else -1
|
|
||||||
),
|
|
||||||
)
|
|
||||||
child.sendline(send_char)
|
child.sendline(send_char)
|
||||||
time.sleep(0.01)
|
time.sleep(0.01)
|
||||||
|
|
||||||
|
|
@ -973,11 +889,6 @@ def test_different_debug_mode_per_actor(
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# skip on non-Linux CI
|
|
||||||
@pytest.mark.ctlcs_bish(
|
|
||||||
_non_linux,
|
|
||||||
_ci_env,
|
|
||||||
)
|
|
||||||
def test_post_mortem_api(
|
def test_post_mortem_api(
|
||||||
spawn,
|
spawn,
|
||||||
ctlc: bool,
|
ctlc: bool,
|
||||||
|
|
@ -1222,20 +1133,14 @@ def test_ctxep_pauses_n_maybe_ipc_breaks(
|
||||||
# closed so verify we see error reporting as well as
|
# closed so verify we see error reporting as well as
|
||||||
# a failed crash-REPL request msg and can CTL-c our way
|
# a failed crash-REPL request msg and can CTL-c our way
|
||||||
# out.
|
# out.
|
||||||
|
|
||||||
# ?TODO, match depending on `tpt_proto(s)`?
|
|
||||||
# - [ ] how can we pass it into the script tho?
|
|
||||||
tpt: str = 'UDS'
|
|
||||||
if _non_linux:
|
|
||||||
tpt: str = 'TCP'
|
|
||||||
|
|
||||||
assert_before(
|
assert_before(
|
||||||
child,
|
child,
|
||||||
['peer IPC channel closed abruptly?',
|
['peer IPC channel closed abruptly?',
|
||||||
'another task closed this fd',
|
'another task closed this fd',
|
||||||
'Debug lock request was CANCELLED?',
|
'Debug lock request was CANCELLED?',
|
||||||
f"'Msgpack{tpt}Stream' was already closed locally?",
|
"'MsgpackUDSStream' was already closed locally?",
|
||||||
f"TransportClosed: 'Msgpack{tpt}Stream' was already closed 'by peer'?",
|
"TransportClosed: 'MsgpackUDSStream' was already closed 'by peer'?",
|
||||||
|
# ?TODO^? match depending on `tpt_proto(s)`?
|
||||||
]
|
]
|
||||||
|
|
||||||
# XXX races on whether these show/hit?
|
# XXX races on whether these show/hit?
|
||||||
|
|
|
||||||
|
|
@ -31,9 +31,6 @@ from .conftest import (
|
||||||
PROMPT,
|
PROMPT,
|
||||||
_pause_msg,
|
_pause_msg,
|
||||||
)
|
)
|
||||||
from ..conftest import (
|
|
||||||
no_macos,
|
|
||||||
)
|
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
from pexpect.exceptions import (
|
from pexpect.exceptions import (
|
||||||
|
|
@ -45,7 +42,6 @@ if TYPE_CHECKING:
|
||||||
from ..conftest import PexpectSpawner
|
from ..conftest import PexpectSpawner
|
||||||
|
|
||||||
|
|
||||||
@no_macos
|
|
||||||
def test_shield_pause(
|
def test_shield_pause(
|
||||||
spawn: PexpectSpawner,
|
spawn: PexpectSpawner,
|
||||||
):
|
):
|
||||||
|
|
@ -61,7 +57,6 @@ def test_shield_pause(
|
||||||
expect(
|
expect(
|
||||||
child,
|
child,
|
||||||
'Yo my child hanging..?',
|
'Yo my child hanging..?',
|
||||||
timeout=3,
|
|
||||||
)
|
)
|
||||||
assert_before(
|
assert_before(
|
||||||
child,
|
child,
|
||||||
|
|
|
||||||
|
|
@ -62,13 +62,6 @@ def test_root_passes_tpt_to_sub(
|
||||||
reg_addr: tuple,
|
reg_addr: tuple,
|
||||||
debug_mode: bool,
|
debug_mode: bool,
|
||||||
):
|
):
|
||||||
# XXX NOTE, the `reg_addr` addr won't be the same type as the
|
|
||||||
# `tpt_proto_key` would deliver here unless you pass `--tpt-proto
|
|
||||||
# <tpt_proto_key>` on the CLI.
|
|
||||||
#
|
|
||||||
# if tpt_proto_key == 'uds':
|
|
||||||
# breakpoint()
|
|
||||||
|
|
||||||
async def main():
|
async def main():
|
||||||
async with tractor.open_nursery(
|
async with tractor.open_nursery(
|
||||||
enable_transports=[tpt_proto_key],
|
enable_transports=[tpt_proto_key],
|
||||||
|
|
|
||||||
|
|
@ -1,4 +0,0 @@
|
||||||
'''
|
|
||||||
`tractor.msg.*` sub-sys test suite.
|
|
||||||
|
|
||||||
'''
|
|
||||||
|
|
@ -1,4 +0,0 @@
|
||||||
'''
|
|
||||||
`tractor.msg.*` test sub-pkg conf.
|
|
||||||
|
|
||||||
'''
|
|
||||||
|
|
@ -1,240 +0,0 @@
|
||||||
'''
|
|
||||||
Unit tests for `tractor.msg.pretty_struct`
|
|
||||||
private-field filtering in `pformat()`.
|
|
||||||
|
|
||||||
'''
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from tractor.msg.pretty_struct import (
|
|
||||||
Struct,
|
|
||||||
pformat,
|
|
||||||
iter_struct_ppfmt_lines,
|
|
||||||
)
|
|
||||||
from tractor.msg._codec import (
|
|
||||||
MsgDec,
|
|
||||||
mk_dec,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ------ test struct definitions ------ #
|
|
||||||
|
|
||||||
class PublicOnly(Struct):
|
|
||||||
'''
|
|
||||||
All-public fields for baseline testing.
|
|
||||||
|
|
||||||
'''
|
|
||||||
name: str = 'alice'
|
|
||||||
age: int = 30
|
|
||||||
|
|
||||||
|
|
||||||
class PrivateOnly(Struct):
|
|
||||||
'''
|
|
||||||
Only underscore-prefixed (private) fields.
|
|
||||||
|
|
||||||
'''
|
|
||||||
_secret: str = 'hidden'
|
|
||||||
_internal: int = 99
|
|
||||||
|
|
||||||
|
|
||||||
class MixedFields(Struct):
|
|
||||||
'''
|
|
||||||
Mix of public and private fields.
|
|
||||||
|
|
||||||
'''
|
|
||||||
name: str = 'bob'
|
|
||||||
_hidden: int = 42
|
|
||||||
value: float = 3.14
|
|
||||||
_meta: str = 'internal'
|
|
||||||
|
|
||||||
|
|
||||||
class Inner(
|
|
||||||
Struct,
|
|
||||||
frozen=True,
|
|
||||||
):
|
|
||||||
'''
|
|
||||||
Frozen inner struct with a private field,
|
|
||||||
for nesting tests.
|
|
||||||
|
|
||||||
'''
|
|
||||||
x: int = 1
|
|
||||||
_secret: str = 'nope'
|
|
||||||
|
|
||||||
|
|
||||||
class Outer(Struct):
|
|
||||||
'''
|
|
||||||
Outer struct nesting an `Inner`.
|
|
||||||
|
|
||||||
'''
|
|
||||||
label: str = 'outer'
|
|
||||||
inner: Inner = Inner()
|
|
||||||
|
|
||||||
|
|
||||||
class EmptyStruct(Struct):
|
|
||||||
'''
|
|
||||||
Struct with zero fields.
|
|
||||||
|
|
||||||
'''
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
# ------ tests ------ #
|
|
||||||
|
|
||||||
@pytest.mark.parametrize(
|
|
||||||
'struct_and_expected',
|
|
||||||
[
|
|
||||||
(
|
|
||||||
PublicOnly(),
|
|
||||||
{
|
|
||||||
'shown': ['name', 'age'],
|
|
||||||
'hidden': [],
|
|
||||||
},
|
|
||||||
),
|
|
||||||
(
|
|
||||||
MixedFields(),
|
|
||||||
{
|
|
||||||
'shown': ['name', 'value'],
|
|
||||||
'hidden': ['_hidden', '_meta'],
|
|
||||||
},
|
|
||||||
),
|
|
||||||
(
|
|
||||||
PrivateOnly(),
|
|
||||||
{
|
|
||||||
'shown': [],
|
|
||||||
'hidden': ['_secret', '_internal'],
|
|
||||||
},
|
|
||||||
),
|
|
||||||
],
|
|
||||||
ids=[
|
|
||||||
'all-public',
|
|
||||||
'mixed-pub-priv',
|
|
||||||
'all-private',
|
|
||||||
],
|
|
||||||
)
|
|
||||||
def test_field_visibility_in_pformat(
|
|
||||||
struct_and_expected: tuple[
|
|
||||||
Struct,
|
|
||||||
dict[str, list[str]],
|
|
||||||
],
|
|
||||||
):
|
|
||||||
'''
|
|
||||||
Verify `pformat()` shows public fields
|
|
||||||
and hides `_`-prefixed private fields.
|
|
||||||
|
|
||||||
'''
|
|
||||||
(
|
|
||||||
struct,
|
|
||||||
expected,
|
|
||||||
) = struct_and_expected
|
|
||||||
output: str = pformat(struct)
|
|
||||||
|
|
||||||
for field_name in expected['shown']:
|
|
||||||
assert field_name in output, (
|
|
||||||
f'{field_name!r} should appear in:\n'
|
|
||||||
f'{output}'
|
|
||||||
)
|
|
||||||
|
|
||||||
for field_name in expected['hidden']:
|
|
||||||
assert field_name not in output, (
|
|
||||||
f'{field_name!r} should NOT appear in:\n'
|
|
||||||
f'{output}'
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_iter_ppfmt_lines_skips_private():
|
|
||||||
'''
|
|
||||||
Directly verify `iter_struct_ppfmt_lines()`
|
|
||||||
never yields tuples with `_`-prefixed field
|
|
||||||
names.
|
|
||||||
|
|
||||||
'''
|
|
||||||
struct = MixedFields()
|
|
||||||
lines: list[tuple[str, str]] = list(
|
|
||||||
iter_struct_ppfmt_lines(
|
|
||||||
struct,
|
|
||||||
field_indent=2,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
# should have lines for public fields only
|
|
||||||
assert len(lines) == 2
|
|
||||||
|
|
||||||
for _prefix, line_content in lines:
|
|
||||||
field_name: str = (
|
|
||||||
line_content.split(':')[0].strip()
|
|
||||||
)
|
|
||||||
assert not field_name.startswith('_'), (
|
|
||||||
f'private field leaked: {field_name!r}'
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_nested_struct_filters_inner_private():
|
|
||||||
'''
|
|
||||||
Verify that nested struct's private fields
|
|
||||||
are also filtered out during recursion.
|
|
||||||
|
|
||||||
'''
|
|
||||||
outer = Outer()
|
|
||||||
output: str = pformat(outer)
|
|
||||||
|
|
||||||
# outer's public field
|
|
||||||
assert 'label' in output
|
|
||||||
|
|
||||||
# inner's public field (recursed into)
|
|
||||||
assert 'x' in output
|
|
||||||
|
|
||||||
# inner's private field must be hidden
|
|
||||||
assert '_secret' not in output
|
|
||||||
|
|
||||||
|
|
||||||
def test_empty_struct_pformat():
|
|
||||||
'''
|
|
||||||
An empty struct should produce a valid
|
|
||||||
`pformat()` result with no field lines.
|
|
||||||
|
|
||||||
'''
|
|
||||||
output: str = pformat(EmptyStruct())
|
|
||||||
assert 'EmptyStruct(' in output
|
|
||||||
assert output.rstrip().endswith(')')
|
|
||||||
|
|
||||||
# no field lines => only struct header+footer
|
|
||||||
lines: list[tuple[str, str]] = list(
|
|
||||||
iter_struct_ppfmt_lines(
|
|
||||||
EmptyStruct(),
|
|
||||||
field_indent=2,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
assert lines == []
|
|
||||||
|
|
||||||
|
|
||||||
def test_real_msgdec_pformat_hides_private():
|
|
||||||
'''
|
|
||||||
Verify `pformat()` on a real `MsgDec`
|
|
||||||
hides the `_dec` internal field.
|
|
||||||
|
|
||||||
NOTE: `MsgDec.__repr__` is custom and does
|
|
||||||
NOT call `pformat()`, so we call it directly.
|
|
||||||
|
|
||||||
'''
|
|
||||||
dec: MsgDec = mk_dec(spec=int)
|
|
||||||
output: str = pformat(dec)
|
|
||||||
|
|
||||||
# the private `_dec` field should be filtered
|
|
||||||
assert '_dec' not in output
|
|
||||||
|
|
||||||
# but the struct type name should be present
|
|
||||||
assert 'MsgDec(' in output
|
|
||||||
|
|
||||||
|
|
||||||
def test_pformat_repr_integration():
|
|
||||||
'''
|
|
||||||
Verify that `Struct.__repr__()` (which calls
|
|
||||||
`pformat()`) also hides private fields for
|
|
||||||
custom structs that do NOT override `__repr__`.
|
|
||||||
|
|
||||||
'''
|
|
||||||
mixed = MixedFields()
|
|
||||||
output: str = repr(mixed)
|
|
||||||
|
|
||||||
assert 'name' in output
|
|
||||||
assert 'value' in output
|
|
||||||
assert '_hidden' not in output
|
|
||||||
assert '_meta' not in output
|
|
||||||
|
|
@ -1,12 +1,7 @@
|
||||||
'''
|
"""
|
||||||
Audit the simplest inter-actor bidirectional (streaming)
|
Bidirectional streaming.
|
||||||
msg patterns.
|
|
||||||
|
|
||||||
'''
|
"""
|
||||||
from __future__ import annotations
|
|
||||||
from typing import (
|
|
||||||
Callable,
|
|
||||||
)
|
|
||||||
import pytest
|
import pytest
|
||||||
import trio
|
import trio
|
||||||
import tractor
|
import tractor
|
||||||
|
|
@ -14,8 +9,10 @@ import tractor
|
||||||
|
|
||||||
@tractor.context
|
@tractor.context
|
||||||
async def simple_rpc(
|
async def simple_rpc(
|
||||||
|
|
||||||
ctx: tractor.Context,
|
ctx: tractor.Context,
|
||||||
data: int,
|
data: int,
|
||||||
|
|
||||||
) -> None:
|
) -> None:
|
||||||
'''
|
'''
|
||||||
Test a small ping-pong server.
|
Test a small ping-pong server.
|
||||||
|
|
@ -42,13 +39,15 @@ async def simple_rpc(
|
||||||
|
|
||||||
@tractor.context
|
@tractor.context
|
||||||
async def simple_rpc_with_forloop(
|
async def simple_rpc_with_forloop(
|
||||||
|
|
||||||
ctx: tractor.Context,
|
ctx: tractor.Context,
|
||||||
data: int,
|
data: int,
|
||||||
) -> None:
|
|
||||||
'''
|
|
||||||
Same as previous test but using `async for` syntax/api.
|
|
||||||
|
|
||||||
'''
|
) -> None:
|
||||||
|
"""Same as previous test but using ``async for`` syntax/api.
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
# signal to parent that we're up
|
# signal to parent that we're up
|
||||||
await ctx.started(data + 1)
|
await ctx.started(data + 1)
|
||||||
|
|
||||||
|
|
@ -69,78 +68,62 @@ async def simple_rpc_with_forloop(
|
||||||
|
|
||||||
@pytest.mark.parametrize(
|
@pytest.mark.parametrize(
|
||||||
'use_async_for',
|
'use_async_for',
|
||||||
[
|
[True, False],
|
||||||
True,
|
|
||||||
False,
|
|
||||||
],
|
|
||||||
ids='use_async_for={}'.format,
|
|
||||||
)
|
)
|
||||||
@pytest.mark.parametrize(
|
@pytest.mark.parametrize(
|
||||||
'server_func',
|
'server_func',
|
||||||
[
|
[simple_rpc, simple_rpc_with_forloop],
|
||||||
simple_rpc,
|
|
||||||
simple_rpc_with_forloop,
|
|
||||||
],
|
|
||||||
ids='server_func={}'.format,
|
|
||||||
)
|
)
|
||||||
def test_simple_rpc(
|
def test_simple_rpc(server_func, use_async_for):
|
||||||
server_func: Callable,
|
|
||||||
use_async_for: bool,
|
|
||||||
loglevel: str,
|
|
||||||
debug_mode: bool,
|
|
||||||
):
|
|
||||||
'''
|
'''
|
||||||
The simplest request response pattern.
|
The simplest request response pattern.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
async def main():
|
async def main():
|
||||||
with trio.fail_after(6):
|
async with tractor.open_nursery() as n:
|
||||||
async with tractor.open_nursery(
|
|
||||||
loglevel=loglevel,
|
|
||||||
debug_mode=debug_mode,
|
|
||||||
) as an:
|
|
||||||
portal: tractor.Portal = await an.start_actor(
|
|
||||||
'rpc_server',
|
|
||||||
enable_modules=[__name__],
|
|
||||||
)
|
|
||||||
|
|
||||||
async with portal.open_context(
|
portal = await n.start_actor(
|
||||||
server_func, # taken from pytest parameterization
|
'rpc_server',
|
||||||
data=10,
|
enable_modules=[__name__],
|
||||||
) as (ctx, sent):
|
)
|
||||||
|
|
||||||
assert sent == 11
|
async with portal.open_context(
|
||||||
|
server_func, # taken from pytest parameterization
|
||||||
|
data=10,
|
||||||
|
) as (ctx, sent):
|
||||||
|
|
||||||
async with ctx.open_stream() as stream:
|
assert sent == 11
|
||||||
|
|
||||||
if use_async_for:
|
async with ctx.open_stream() as stream:
|
||||||
|
|
||||||
count = 0
|
if use_async_for:
|
||||||
# receive msgs using async for style
|
|
||||||
|
count = 0
|
||||||
|
# receive msgs using async for style
|
||||||
|
print('ping')
|
||||||
|
await stream.send('ping')
|
||||||
|
|
||||||
|
async for msg in stream:
|
||||||
|
assert msg == 'pong'
|
||||||
print('ping')
|
print('ping')
|
||||||
await stream.send('ping')
|
await stream.send('ping')
|
||||||
|
count += 1
|
||||||
|
|
||||||
async for msg in stream:
|
if count >= 9:
|
||||||
assert msg == 'pong'
|
break
|
||||||
print('ping')
|
|
||||||
await stream.send('ping')
|
|
||||||
count += 1
|
|
||||||
|
|
||||||
if count >= 9:
|
else:
|
||||||
break
|
# classic send/receive style
|
||||||
|
for _ in range(10):
|
||||||
|
|
||||||
else:
|
print('ping')
|
||||||
# classic send/receive style
|
await stream.send('ping')
|
||||||
for _ in range(10):
|
assert await stream.receive() == 'pong'
|
||||||
|
|
||||||
print('ping')
|
# stream should terminate here
|
||||||
await stream.send('ping')
|
|
||||||
assert await stream.receive() == 'pong'
|
|
||||||
|
|
||||||
# stream should terminate here
|
# final context result(s) should be consumed here in __aexit__()
|
||||||
|
|
||||||
# final context result(s) should be consumed here in __aexit__()
|
await portal.cancel_actor()
|
||||||
|
|
||||||
await portal.cancel_actor()
|
|
||||||
|
|
||||||
trio.run(main)
|
trio.run(main)
|
||||||
|
|
|
||||||
|
|
@ -146,6 +146,9 @@ def test_ipc_channel_break_during_stream(
|
||||||
# a user sending ctl-c by raising a KBI.
|
# a user sending ctl-c by raising a KBI.
|
||||||
if pre_aclose_msgstream:
|
if pre_aclose_msgstream:
|
||||||
expect_final_exc = KeyboardInterrupt
|
expect_final_exc = KeyboardInterrupt
|
||||||
|
if tpt_proto == 'uds':
|
||||||
|
expect_final_exc = TransportClosed
|
||||||
|
expect_final_cause = trio.BrokenResourceError
|
||||||
|
|
||||||
# XXX OLD XXX
|
# XXX OLD XXX
|
||||||
# if child calls `MsgStream.aclose()` then expect EoC.
|
# if child calls `MsgStream.aclose()` then expect EoC.
|
||||||
|
|
@ -166,6 +169,10 @@ def test_ipc_channel_break_during_stream(
|
||||||
if pre_aclose_msgstream:
|
if pre_aclose_msgstream:
|
||||||
expect_final_exc = KeyboardInterrupt
|
expect_final_exc = KeyboardInterrupt
|
||||||
|
|
||||||
|
if tpt_proto == 'uds':
|
||||||
|
expect_final_exc = TransportClosed
|
||||||
|
expect_final_cause = trio.BrokenResourceError
|
||||||
|
|
||||||
# NOTE when the parent IPC side dies (even if the child does as well
|
# NOTE when the parent IPC side dies (even if the child does as well
|
||||||
# but the child fails BEFORE the parent) we always expect the
|
# but the child fails BEFORE the parent) we always expect the
|
||||||
# IPC layer to raise a closed-resource, NEVER do we expect
|
# IPC layer to raise a closed-resource, NEVER do we expect
|
||||||
|
|
|
||||||
|
|
@ -17,8 +17,8 @@ from tractor._testing import (
|
||||||
from .conftest import no_windows
|
from .conftest import no_windows
|
||||||
|
|
||||||
|
|
||||||
_non_linux: bool = platform.system() != 'Linux'
|
def is_win():
|
||||||
_friggin_windows: bool = platform.system() == 'Windows'
|
return platform.system() == 'Windows'
|
||||||
|
|
||||||
|
|
||||||
async def assert_err(delay=0):
|
async def assert_err(delay=0):
|
||||||
|
|
@ -431,7 +431,7 @@ async def test_nested_multierrors(loglevel, start_method):
|
||||||
for subexc in err.exceptions:
|
for subexc in err.exceptions:
|
||||||
|
|
||||||
# verify first level actor errors are wrapped as remote
|
# verify first level actor errors are wrapped as remote
|
||||||
if _friggin_windows:
|
if is_win():
|
||||||
|
|
||||||
# windows is often too slow and cancellation seems
|
# windows is often too slow and cancellation seems
|
||||||
# to happen before an actor is spawned
|
# to happen before an actor is spawned
|
||||||
|
|
@ -464,7 +464,7 @@ async def test_nested_multierrors(loglevel, start_method):
|
||||||
# XXX not sure what's up with this..
|
# XXX not sure what's up with this..
|
||||||
# on windows sometimes spawning is just too slow and
|
# on windows sometimes spawning is just too slow and
|
||||||
# we get back the (sent) cancel signal instead
|
# we get back the (sent) cancel signal instead
|
||||||
if _friggin_windows:
|
if is_win():
|
||||||
if isinstance(subexc, tractor.RemoteActorError):
|
if isinstance(subexc, tractor.RemoteActorError):
|
||||||
assert subexc.boxed_type in (
|
assert subexc.boxed_type in (
|
||||||
BaseExceptionGroup,
|
BaseExceptionGroup,
|
||||||
|
|
@ -507,22 +507,17 @@ def test_cancel_via_SIGINT(
|
||||||
|
|
||||||
@no_windows
|
@no_windows
|
||||||
def test_cancel_via_SIGINT_other_task(
|
def test_cancel_via_SIGINT_other_task(
|
||||||
loglevel: str,
|
loglevel,
|
||||||
start_method: str,
|
start_method,
|
||||||
spawn_backend: str,
|
spawn_backend,
|
||||||
):
|
):
|
||||||
'''
|
"""Ensure that a control-C (SIGINT) signal cancels both the parent
|
||||||
Ensure that a control-C (SIGINT) signal cancels both the parent
|
and child processes in trionic fashion even a subprocess is started
|
||||||
and child processes in trionic fashion even a subprocess is
|
from a seperate ``trio`` child task.
|
||||||
started from a seperate ``trio`` child task.
|
"""
|
||||||
|
pid = os.getpid()
|
||||||
'''
|
timeout: float = 2
|
||||||
pid: int = os.getpid()
|
if is_win(): # smh
|
||||||
timeout: float = (
|
|
||||||
4 if _non_linux
|
|
||||||
else 2
|
|
||||||
)
|
|
||||||
if _friggin_windows: # smh
|
|
||||||
timeout += 1
|
timeout += 1
|
||||||
|
|
||||||
async def spawn_and_sleep_forever(
|
async def spawn_and_sleep_forever(
|
||||||
|
|
@ -649,11 +644,7 @@ def test_cancel_while_childs_child_in_sync_sleep(
|
||||||
#
|
#
|
||||||
# delay = 1 # no AssertionError in eg, TooSlowError raised.
|
# delay = 1 # no AssertionError in eg, TooSlowError raised.
|
||||||
# delay = 2 # is AssertionError in eg AND no TooSlowError !?
|
# delay = 2 # is AssertionError in eg AND no TooSlowError !?
|
||||||
# is AssertionError in eg AND no _cs cancellation.
|
delay = 4 # is AssertionError in eg AND no _cs cancellation.
|
||||||
delay = (
|
|
||||||
6 if _non_linux
|
|
||||||
else 4
|
|
||||||
)
|
|
||||||
|
|
||||||
with trio.fail_after(delay) as _cs:
|
with trio.fail_after(delay) as _cs:
|
||||||
# with trio.CancelScope() as cs:
|
# with trio.CancelScope() as cs:
|
||||||
|
|
@ -705,7 +696,7 @@ def test_fast_graceful_cancel_when_spawn_task_in_soft_proc_wait_for_daemon(
|
||||||
kbi_delay = 0.5
|
kbi_delay = 0.5
|
||||||
timeout: float = 2.9
|
timeout: float = 2.9
|
||||||
|
|
||||||
if _friggin_windows: # smh
|
if is_win(): # smh
|
||||||
timeout += 1
|
timeout += 1
|
||||||
|
|
||||||
async def main():
|
async def main():
|
||||||
|
|
|
||||||
|
|
@ -18,15 +18,16 @@ from tractor import RemoteActorError
|
||||||
|
|
||||||
|
|
||||||
async def aio_streamer(
|
async def aio_streamer(
|
||||||
chan: tractor.to_asyncio.LinkedTaskChannel,
|
from_trio: asyncio.Queue,
|
||||||
|
to_trio: trio.abc.SendChannel,
|
||||||
) -> trio.abc.ReceiveChannel:
|
) -> trio.abc.ReceiveChannel:
|
||||||
|
|
||||||
# required first msg to sync caller
|
# required first msg to sync caller
|
||||||
chan.started_nowait(None)
|
to_trio.send_nowait(None)
|
||||||
|
|
||||||
from itertools import cycle
|
from itertools import cycle
|
||||||
for i in cycle(range(10)):
|
for i in cycle(range(10)):
|
||||||
chan.send_nowait(i)
|
to_trio.send_nowait(i)
|
||||||
await asyncio.sleep(0.01)
|
await asyncio.sleep(0.01)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -68,7 +69,7 @@ async def wrapper_mngr(
|
||||||
else:
|
else:
|
||||||
async with tractor.to_asyncio.open_channel_from(
|
async with tractor.to_asyncio.open_channel_from(
|
||||||
aio_streamer,
|
aio_streamer,
|
||||||
) as (from_aio, first):
|
) as (first, from_aio):
|
||||||
assert not first
|
assert not first
|
||||||
|
|
||||||
# cache it so next task uses broadcast receiver
|
# cache it so next task uses broadcast receiver
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,7 @@ MESSAGE = 'tractoring at full speed'
|
||||||
|
|
||||||
|
|
||||||
def test_empty_mngrs_input_raises() -> None:
|
def test_empty_mngrs_input_raises() -> None:
|
||||||
|
|
||||||
async def main():
|
async def main():
|
||||||
with trio.fail_after(3):
|
with trio.fail_after(3):
|
||||||
async with (
|
async with (
|
||||||
|
|
@ -55,39 +56,25 @@ async def worker(
|
||||||
print(msg)
|
print(msg)
|
||||||
assert msg == MESSAGE
|
assert msg == MESSAGE
|
||||||
|
|
||||||
# ?TODO, does this ever cause a hang?
|
# TODO: does this ever cause a hang
|
||||||
# assert 0
|
# assert 0
|
||||||
|
|
||||||
|
|
||||||
# ?TODO, but needs a fn-scoped tpt_proto fixture..
|
|
||||||
# @pytest.mark.no_tpt('uds')
|
|
||||||
@tractor_test
|
@tractor_test
|
||||||
async def test_streaming_to_actor_cluster(
|
async def test_streaming_to_actor_cluster() -> None:
|
||||||
tpt_proto: str,
|
|
||||||
):
|
|
||||||
'''
|
|
||||||
Open an actor "cluster" using the (experimental) `._clustering`
|
|
||||||
API and conduct standard inter-task-ctx streaming.
|
|
||||||
|
|
||||||
'''
|
async with (
|
||||||
if tpt_proto == 'uds':
|
open_actor_cluster(modules=[__name__]) as portals,
|
||||||
pytest.skip(
|
|
||||||
f'Test currently fails with tpt-proto={tpt_proto!r}\n'
|
|
||||||
)
|
|
||||||
|
|
||||||
with trio.fail_after(6):
|
gather_contexts(
|
||||||
async with (
|
mngrs=[p.open_context(worker) for p in portals.values()],
|
||||||
open_actor_cluster(modules=[__name__]) as portals,
|
) as contexts,
|
||||||
|
|
||||||
gather_contexts(
|
gather_contexts(
|
||||||
mngrs=[p.open_context(worker) for p in portals.values()],
|
mngrs=[ctx[0].open_stream() for ctx in contexts],
|
||||||
) as contexts,
|
) as streams,
|
||||||
|
|
||||||
gather_contexts(
|
):
|
||||||
mngrs=[ctx[0].open_stream() for ctx in contexts],
|
with trio.move_on_after(1):
|
||||||
) as streams,
|
for stream in itertools.cycle(streams):
|
||||||
|
await stream.send(MESSAGE)
|
||||||
):
|
|
||||||
with trio.move_on_after(1):
|
|
||||||
for stream in itertools.cycle(streams):
|
|
||||||
await stream.send(MESSAGE)
|
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,6 @@ from itertools import count
|
||||||
import math
|
import math
|
||||||
import platform
|
import platform
|
||||||
from pprint import pformat
|
from pprint import pformat
|
||||||
import sys
|
|
||||||
from typing import (
|
from typing import (
|
||||||
Callable,
|
Callable,
|
||||||
)
|
)
|
||||||
|
|
@ -942,11 +941,6 @@ def test_one_end_stream_not_opened(
|
||||||
from tractor._runtime import Actor
|
from tractor._runtime import Actor
|
||||||
buf_size = buf_size_increase + Actor.msg_buffer_size
|
buf_size = buf_size_increase + Actor.msg_buffer_size
|
||||||
|
|
||||||
timeout: float = (
|
|
||||||
1 if sys.platform == 'linux'
|
|
||||||
else 3
|
|
||||||
)
|
|
||||||
|
|
||||||
async def main():
|
async def main():
|
||||||
async with tractor.open_nursery(
|
async with tractor.open_nursery(
|
||||||
debug_mode=debug_mode,
|
debug_mode=debug_mode,
|
||||||
|
|
@ -956,7 +950,7 @@ def test_one_end_stream_not_opened(
|
||||||
enable_modules=[__name__],
|
enable_modules=[__name__],
|
||||||
)
|
)
|
||||||
|
|
||||||
with trio.fail_after(timeout):
|
with trio.fail_after(1):
|
||||||
async with portal.open_context(
|
async with portal.open_context(
|
||||||
entrypoint,
|
entrypoint,
|
||||||
) as (ctx, sent):
|
) as (ctx, sent):
|
||||||
|
|
|
||||||
|
|
@ -1,14 +1,11 @@
|
||||||
"""
|
"""
|
||||||
Discovery subsys.
|
Actor "discovery" testing
|
||||||
|
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
import signal
|
import signal
|
||||||
import platform
|
import platform
|
||||||
from functools import partial
|
from functools import partial
|
||||||
import itertools
|
import itertools
|
||||||
import time
|
|
||||||
from typing import Callable
|
|
||||||
|
|
||||||
import psutil
|
import psutil
|
||||||
import pytest
|
import pytest
|
||||||
|
|
@ -20,9 +17,7 @@ import trio
|
||||||
|
|
||||||
|
|
||||||
@tractor_test
|
@tractor_test
|
||||||
async def test_reg_then_unreg(
|
async def test_reg_then_unreg(reg_addr):
|
||||||
reg_addr: tuple,
|
|
||||||
):
|
|
||||||
actor = tractor.current_actor()
|
actor = tractor.current_actor()
|
||||||
assert actor.is_arbiter
|
assert actor.is_arbiter
|
||||||
assert len(actor._registry) == 1 # only self is registered
|
assert len(actor._registry) == 1 # only self is registered
|
||||||
|
|
@ -32,7 +27,7 @@ async def test_reg_then_unreg(
|
||||||
) as n:
|
) as n:
|
||||||
|
|
||||||
portal = await n.start_actor('actor', enable_modules=[__name__])
|
portal = await n.start_actor('actor', enable_modules=[__name__])
|
||||||
uid = portal.channel.aid.uid
|
uid = portal.channel.uid
|
||||||
|
|
||||||
async with tractor.get_registry(reg_addr) as aportal:
|
async with tractor.get_registry(reg_addr) as aportal:
|
||||||
# this local actor should be the arbiter
|
# this local actor should be the arbiter
|
||||||
|
|
@ -87,15 +82,11 @@ async def say_hello_use_wait(
|
||||||
|
|
||||||
|
|
||||||
@tractor_test
|
@tractor_test
|
||||||
@pytest.mark.parametrize(
|
@pytest.mark.parametrize('func', [say_hello, say_hello_use_wait])
|
||||||
'func',
|
|
||||||
[say_hello,
|
|
||||||
say_hello_use_wait]
|
|
||||||
)
|
|
||||||
async def test_trynamic_trio(
|
async def test_trynamic_trio(
|
||||||
func: Callable,
|
func,
|
||||||
start_method: str,
|
start_method,
|
||||||
reg_addr: tuple,
|
reg_addr,
|
||||||
):
|
):
|
||||||
'''
|
'''
|
||||||
Root actor acting as the "director" and running one-shot-task-actors
|
Root actor acting as the "director" and running one-shot-task-actors
|
||||||
|
|
@ -128,10 +119,7 @@ async def stream_forever():
|
||||||
await trio.sleep(0.01)
|
await trio.sleep(0.01)
|
||||||
|
|
||||||
|
|
||||||
async def cancel(
|
async def cancel(use_signal, delay=0):
|
||||||
use_signal: bool,
|
|
||||||
delay: float = 0,
|
|
||||||
):
|
|
||||||
# hold on there sally
|
# hold on there sally
|
||||||
await trio.sleep(delay)
|
await trio.sleep(delay)
|
||||||
|
|
||||||
|
|
@ -144,15 +132,13 @@ async def cancel(
|
||||||
raise KeyboardInterrupt
|
raise KeyboardInterrupt
|
||||||
|
|
||||||
|
|
||||||
async def stream_from(portal: tractor.Portal):
|
async def stream_from(portal):
|
||||||
async with portal.open_stream_from(stream_forever) as stream:
|
async with portal.open_stream_from(stream_forever) as stream:
|
||||||
async for value in stream:
|
async for value in stream:
|
||||||
print(value)
|
print(value)
|
||||||
|
|
||||||
|
|
||||||
async def unpack_reg(
|
async def unpack_reg(actor_or_portal):
|
||||||
actor_or_portal: tractor.Portal|tractor.Actor,
|
|
||||||
):
|
|
||||||
'''
|
'''
|
||||||
Get and unpack a "registry" RPC request from the "arbiter" registry
|
Get and unpack a "registry" RPC request from the "arbiter" registry
|
||||||
system.
|
system.
|
||||||
|
|
@ -187,9 +173,7 @@ async def spawn_and_check_registry(
|
||||||
registry_addrs=[reg_addr],
|
registry_addrs=[reg_addr],
|
||||||
debug_mode=debug_mode,
|
debug_mode=debug_mode,
|
||||||
):
|
):
|
||||||
async with tractor.get_registry(
|
async with tractor.get_registry(reg_addr) as portal:
|
||||||
addr=reg_addr,
|
|
||||||
) as portal:
|
|
||||||
# runtime needs to be up to call this
|
# runtime needs to be up to call this
|
||||||
actor = tractor.current_actor()
|
actor = tractor.current_actor()
|
||||||
|
|
||||||
|
|
@ -206,7 +190,7 @@ async def spawn_and_check_registry(
|
||||||
|
|
||||||
# ensure current actor is registered
|
# ensure current actor is registered
|
||||||
registry: dict = await get_reg()
|
registry: dict = await get_reg()
|
||||||
assert actor.aid.uid in registry
|
assert actor.uid in registry
|
||||||
|
|
||||||
try:
|
try:
|
||||||
async with tractor.open_nursery() as an:
|
async with tractor.open_nursery() as an:
|
||||||
|
|
@ -254,31 +238,18 @@ async def spawn_and_check_registry(
|
||||||
|
|
||||||
# all subactors should have de-registered
|
# all subactors should have de-registered
|
||||||
registry = await get_reg()
|
registry = await get_reg()
|
||||||
start: float = time.time()
|
assert len(registry) == extra
|
||||||
while (
|
assert actor.uid in registry
|
||||||
not (len(registry) == extra)
|
|
||||||
and
|
|
||||||
(time.time() - start) < 5
|
|
||||||
):
|
|
||||||
print(
|
|
||||||
f'Waiting for remaining subs to dereg..\n'
|
|
||||||
f'{registry!r}\n'
|
|
||||||
)
|
|
||||||
await trio.sleep(0.3)
|
|
||||||
else:
|
|
||||||
assert len(registry) == extra
|
|
||||||
|
|
||||||
assert actor.aid.uid in registry
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize('use_signal', [False, True])
|
@pytest.mark.parametrize('use_signal', [False, True])
|
||||||
@pytest.mark.parametrize('with_streaming', [False, True])
|
@pytest.mark.parametrize('with_streaming', [False, True])
|
||||||
def test_subactors_unregister_on_cancel(
|
def test_subactors_unregister_on_cancel(
|
||||||
debug_mode: bool,
|
debug_mode: bool,
|
||||||
start_method: str,
|
start_method,
|
||||||
use_signal: bool,
|
use_signal,
|
||||||
reg_addr: tuple,
|
reg_addr,
|
||||||
with_streaming: bool,
|
with_streaming,
|
||||||
):
|
):
|
||||||
'''
|
'''
|
||||||
Verify that cancelling a nursery results in all subactors
|
Verify that cancelling a nursery results in all subactors
|
||||||
|
|
@ -303,17 +274,15 @@ def test_subactors_unregister_on_cancel(
|
||||||
def test_subactors_unregister_on_cancel_remote_daemon(
|
def test_subactors_unregister_on_cancel_remote_daemon(
|
||||||
daemon: subprocess.Popen,
|
daemon: subprocess.Popen,
|
||||||
debug_mode: bool,
|
debug_mode: bool,
|
||||||
start_method: str,
|
start_method,
|
||||||
use_signal: bool,
|
use_signal,
|
||||||
reg_addr: tuple,
|
reg_addr,
|
||||||
with_streaming: bool,
|
with_streaming,
|
||||||
):
|
):
|
||||||
'''
|
"""Verify that cancelling a nursery results in all subactors
|
||||||
Verify that cancelling a nursery results in all subactors
|
deregistering themselves with a **remote** (not in the local process
|
||||||
deregistering themselves with a **remote** (not in the local
|
tree) arbiter.
|
||||||
process tree) arbiter.
|
"""
|
||||||
|
|
||||||
'''
|
|
||||||
with pytest.raises(KeyboardInterrupt):
|
with pytest.raises(KeyboardInterrupt):
|
||||||
trio.run(
|
trio.run(
|
||||||
partial(
|
partial(
|
||||||
|
|
@ -398,23 +367,21 @@ async def close_chans_before_nursery(
|
||||||
|
|
||||||
# all subactors should have de-registered
|
# all subactors should have de-registered
|
||||||
registry = await get_reg()
|
registry = await get_reg()
|
||||||
assert portal1.channel.aid.uid not in registry
|
assert portal1.channel.uid not in registry
|
||||||
assert portal2.channel.aid.uid not in registry
|
assert portal2.channel.uid not in registry
|
||||||
assert len(registry) == entries_at_end
|
assert len(registry) == entries_at_end
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize('use_signal', [False, True])
|
@pytest.mark.parametrize('use_signal', [False, True])
|
||||||
def test_close_channel_explicit(
|
def test_close_channel_explicit(
|
||||||
start_method: str,
|
start_method,
|
||||||
use_signal: bool,
|
use_signal,
|
||||||
reg_addr: tuple,
|
reg_addr,
|
||||||
):
|
):
|
||||||
'''
|
"""Verify that closing a stream explicitly and killing the actor's
|
||||||
Verify that closing a stream explicitly and killing the actor's
|
|
||||||
"root nursery" **before** the containing nursery tears down also
|
"root nursery" **before** the containing nursery tears down also
|
||||||
results in subactor(s) deregistering from the arbiter.
|
results in subactor(s) deregistering from the arbiter.
|
||||||
|
"""
|
||||||
'''
|
|
||||||
with pytest.raises(KeyboardInterrupt):
|
with pytest.raises(KeyboardInterrupt):
|
||||||
trio.run(
|
trio.run(
|
||||||
partial(
|
partial(
|
||||||
|
|
@ -429,16 +396,14 @@ def test_close_channel_explicit(
|
||||||
@pytest.mark.parametrize('use_signal', [False, True])
|
@pytest.mark.parametrize('use_signal', [False, True])
|
||||||
def test_close_channel_explicit_remote_arbiter(
|
def test_close_channel_explicit_remote_arbiter(
|
||||||
daemon: subprocess.Popen,
|
daemon: subprocess.Popen,
|
||||||
start_method: str,
|
start_method,
|
||||||
use_signal: bool,
|
use_signal,
|
||||||
reg_addr: tuple,
|
reg_addr,
|
||||||
):
|
):
|
||||||
'''
|
"""Verify that closing a stream explicitly and killing the actor's
|
||||||
Verify that closing a stream explicitly and killing the actor's
|
|
||||||
"root nursery" **before** the containing nursery tears down also
|
"root nursery" **before** the containing nursery tears down also
|
||||||
results in subactor(s) deregistering from the arbiter.
|
results in subactor(s) deregistering from the arbiter.
|
||||||
|
"""
|
||||||
'''
|
|
||||||
with pytest.raises(KeyboardInterrupt):
|
with pytest.raises(KeyboardInterrupt):
|
||||||
trio.run(
|
trio.run(
|
||||||
partial(
|
partial(
|
||||||
|
|
|
||||||
|
|
@ -9,17 +9,12 @@ import sys
|
||||||
import subprocess
|
import subprocess
|
||||||
import platform
|
import platform
|
||||||
import shutil
|
import shutil
|
||||||
from typing import Callable
|
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
import tractor
|
|
||||||
from tractor._testing import (
|
from tractor._testing import (
|
||||||
examples_dir,
|
examples_dir,
|
||||||
)
|
)
|
||||||
|
|
||||||
_non_linux: bool = platform.system() != 'Linux'
|
|
||||||
_friggin_macos: bool = platform.system() == 'Darwin'
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def run_example_in_subproc(
|
def run_example_in_subproc(
|
||||||
|
|
@ -106,10 +101,8 @@ def run_example_in_subproc(
|
||||||
ids=lambda t: t[1],
|
ids=lambda t: t[1],
|
||||||
)
|
)
|
||||||
def test_example(
|
def test_example(
|
||||||
run_example_in_subproc: Callable,
|
run_example_in_subproc,
|
||||||
example_script: str,
|
example_script,
|
||||||
test_log: tractor.log.StackLevelAdapter,
|
|
||||||
ci_env: bool,
|
|
||||||
):
|
):
|
||||||
'''
|
'''
|
||||||
Load and run scripts from this repo's ``examples/`` dir as a user
|
Load and run scripts from this repo's ``examples/`` dir as a user
|
||||||
|
|
@ -123,32 +116,9 @@ def test_example(
|
||||||
'''
|
'''
|
||||||
ex_file: str = os.path.join(*example_script)
|
ex_file: str = os.path.join(*example_script)
|
||||||
|
|
||||||
if (
|
if 'rpc_bidir_streaming' in ex_file and sys.version_info < (3, 9):
|
||||||
'rpc_bidir_streaming' in ex_file
|
|
||||||
and
|
|
||||||
sys.version_info < (3, 9)
|
|
||||||
):
|
|
||||||
pytest.skip("2-way streaming example requires py3.9 async with syntax")
|
pytest.skip("2-way streaming example requires py3.9 async with syntax")
|
||||||
|
|
||||||
if (
|
|
||||||
'full_fledged_streaming_service' in ex_file
|
|
||||||
and
|
|
||||||
_friggin_macos
|
|
||||||
and
|
|
||||||
ci_env
|
|
||||||
):
|
|
||||||
pytest.skip(
|
|
||||||
'Streaming example is too flaky in CI\n'
|
|
||||||
'AND their competitor runs this CI service..\n'
|
|
||||||
'This test does run just fine "in person" however..'
|
|
||||||
)
|
|
||||||
|
|
||||||
timeout: float = (
|
|
||||||
60
|
|
||||||
if ci_env and _non_linux
|
|
||||||
else 16
|
|
||||||
)
|
|
||||||
|
|
||||||
with open(ex_file, 'r') as ex:
|
with open(ex_file, 'r') as ex:
|
||||||
code = ex.read()
|
code = ex.read()
|
||||||
|
|
||||||
|
|
@ -156,12 +126,9 @@ def test_example(
|
||||||
err = None
|
err = None
|
||||||
try:
|
try:
|
||||||
if not proc.poll():
|
if not proc.poll():
|
||||||
_, err = proc.communicate(timeout=timeout)
|
_, err = proc.communicate(timeout=15)
|
||||||
|
|
||||||
except subprocess.TimeoutExpired as e:
|
except subprocess.TimeoutExpired as e:
|
||||||
test_log.exception(
|
|
||||||
f'Example failed to finish within {timeout}s ??\n'
|
|
||||||
)
|
|
||||||
proc.kill()
|
proc.kill()
|
||||||
err = e.stderr
|
err = e.stderr
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -47,11 +47,12 @@ async def sleep_and_err(
|
||||||
|
|
||||||
# just signature placeholders for compat with
|
# just signature placeholders for compat with
|
||||||
# ``to_asyncio.open_channel_from()``
|
# ``to_asyncio.open_channel_from()``
|
||||||
chan: to_asyncio.LinkedTaskChannel|None = None,
|
to_trio: trio.MemorySendChannel|None = None,
|
||||||
|
from_trio: asyncio.Queue|None = None,
|
||||||
|
|
||||||
):
|
):
|
||||||
if chan:
|
if to_trio:
|
||||||
chan.started_nowait('start')
|
to_trio.send_nowait('start')
|
||||||
|
|
||||||
await asyncio.sleep(sleep_for)
|
await asyncio.sleep(sleep_for)
|
||||||
assert 0
|
assert 0
|
||||||
|
|
@ -237,7 +238,7 @@ async def trio_ctx(
|
||||||
trio.open_nursery() as tn,
|
trio.open_nursery() as tn,
|
||||||
tractor.to_asyncio.open_channel_from(
|
tractor.to_asyncio.open_channel_from(
|
||||||
sleep_and_err,
|
sleep_and_err,
|
||||||
) as (chan, first),
|
) as (first, chan),
|
||||||
):
|
):
|
||||||
|
|
||||||
assert first == 'start'
|
assert first == 'start'
|
||||||
|
|
@ -398,7 +399,7 @@ async def no_to_trio_in_args():
|
||||||
|
|
||||||
async def push_from_aio_task(
|
async def push_from_aio_task(
|
||||||
sequence: Iterable,
|
sequence: Iterable,
|
||||||
chan: to_asyncio.LinkedTaskChannel,
|
to_trio: trio.abc.SendChannel,
|
||||||
expect_cancel: False,
|
expect_cancel: False,
|
||||||
fail_early: bool,
|
fail_early: bool,
|
||||||
exit_early: bool,
|
exit_early: bool,
|
||||||
|
|
@ -406,12 +407,15 @@ async def push_from_aio_task(
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
# print('trying breakpoint')
|
||||||
|
# breakpoint()
|
||||||
|
|
||||||
# sync caller ctx manager
|
# sync caller ctx manager
|
||||||
chan.started_nowait(True)
|
to_trio.send_nowait(True)
|
||||||
|
|
||||||
for i in sequence:
|
for i in sequence:
|
||||||
print(f'asyncio sending {i}')
|
print(f'asyncio sending {i}')
|
||||||
chan.send_nowait(i)
|
to_trio.send_nowait(i)
|
||||||
await asyncio.sleep(0.001)
|
await asyncio.sleep(0.001)
|
||||||
|
|
||||||
if (
|
if (
|
||||||
|
|
@ -474,7 +478,7 @@ async def stream_from_aio(
|
||||||
trio_exit_early
|
trio_exit_early
|
||||||
))
|
))
|
||||||
|
|
||||||
) as (chan, first):
|
) as (first, chan):
|
||||||
|
|
||||||
assert first is True
|
assert first is True
|
||||||
|
|
||||||
|
|
@ -768,8 +772,8 @@ async def trio_to_aio_echo_server(
|
||||||
async with to_asyncio.open_channel_from(
|
async with to_asyncio.open_channel_from(
|
||||||
aio_echo_server,
|
aio_echo_server,
|
||||||
) as (
|
) as (
|
||||||
chan,
|
|
||||||
first, # value from `chan.started_nowait()` above
|
first, # value from `chan.started_nowait()` above
|
||||||
|
chan,
|
||||||
):
|
):
|
||||||
assert first == 'start'
|
assert first == 'start'
|
||||||
|
|
||||||
|
|
@ -1105,12 +1109,13 @@ async def raise_before_started(
|
||||||
) -> None:
|
) -> None:
|
||||||
'''
|
'''
|
||||||
`asyncio.Task` entry point which RTEs before calling
|
`asyncio.Task` entry point which RTEs before calling
|
||||||
`chan.started_nowait()`.
|
`to_trio.send_nowait()`.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
await asyncio.sleep(0.2)
|
await asyncio.sleep(0.2)
|
||||||
raise RuntimeError('Some shite went wrong before `.send_nowait()`!!')
|
raise RuntimeError('Some shite went wrong before `.send_nowait()`!!')
|
||||||
|
|
||||||
|
# to_trio.send_nowait('Uhh we shouldve RTE-d ^^ ??')
|
||||||
chan.started_nowait('Uhh we shouldve RTE-d ^^ ??')
|
chan.started_nowait('Uhh we shouldve RTE-d ^^ ??')
|
||||||
await asyncio.sleep(float('inf'))
|
await asyncio.sleep(float('inf'))
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -581,7 +581,7 @@ def test_peer_canceller(
|
||||||
assert (
|
assert (
|
||||||
re.canceller
|
re.canceller
|
||||||
==
|
==
|
||||||
root.aid.uid
|
root.uid
|
||||||
)
|
)
|
||||||
|
|
||||||
else: # the other 2 ctxs
|
else: # the other 2 ctxs
|
||||||
|
|
@ -590,7 +590,7 @@ def test_peer_canceller(
|
||||||
and (
|
and (
|
||||||
re.canceller
|
re.canceller
|
||||||
==
|
==
|
||||||
canceller.channel.aid.uid
|
canceller.channel.uid
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -745,7 +745,7 @@ def test_peer_canceller(
|
||||||
# -> each context should have received
|
# -> each context should have received
|
||||||
# a silently absorbed context cancellation
|
# a silently absorbed context cancellation
|
||||||
# in its remote nursery scope.
|
# in its remote nursery scope.
|
||||||
# assert ctx.chan.aid.uid == ctx.canceller
|
# assert ctx.chan.uid == ctx.canceller
|
||||||
|
|
||||||
# NOTE: when an inter-peer cancellation
|
# NOTE: when an inter-peer cancellation
|
||||||
# occurred, we DO NOT expect this
|
# occurred, we DO NOT expect this
|
||||||
|
|
@ -802,7 +802,7 @@ async def basic_echo_server(
|
||||||
|
|
||||||
'''
|
'''
|
||||||
actor: Actor = tractor.current_actor()
|
actor: Actor = tractor.current_actor()
|
||||||
uid: tuple = actor.aid.uid
|
uid: tuple = actor.uid
|
||||||
await ctx.started(uid)
|
await ctx.started(uid)
|
||||||
async with ctx.open_stream() as ipc:
|
async with ctx.open_stream() as ipc:
|
||||||
async for msg in ipc:
|
async for msg in ipc:
|
||||||
|
|
@ -857,7 +857,7 @@ async def serve_subactors(
|
||||||
f'|_{peer}\n'
|
f'|_{peer}\n'
|
||||||
)
|
)
|
||||||
await ipc.send((
|
await ipc.send((
|
||||||
peer.chan.aid.uid,
|
peer.chan.uid,
|
||||||
peer.chan.raddr.unwrap(),
|
peer.chan.raddr.unwrap(),
|
||||||
))
|
))
|
||||||
|
|
||||||
|
|
@ -992,7 +992,7 @@ async def tell_little_bro(
|
||||||
sub_ctx.open_stream() as echo_ipc,
|
sub_ctx.open_stream() as echo_ipc,
|
||||||
):
|
):
|
||||||
actor: Actor = current_actor()
|
actor: Actor = current_actor()
|
||||||
uid: tuple = actor.aid.uid
|
uid: tuple = actor.uid
|
||||||
for i in range(rng_seed):
|
for i in range(rng_seed):
|
||||||
msg: tuple = (
|
msg: tuple = (
|
||||||
uid,
|
uid,
|
||||||
|
|
@ -1097,7 +1097,7 @@ def test_peer_spawns_and_cancels_service_subactor(
|
||||||
) as (client_ctx, client_says),
|
) as (client_ctx, client_says),
|
||||||
):
|
):
|
||||||
root: Actor = current_actor()
|
root: Actor = current_actor()
|
||||||
spawner_uid: tuple = spawn_ctx.chan.aid.uid
|
spawner_uid: tuple = spawn_ctx.chan.uid
|
||||||
print(
|
print(
|
||||||
f'Server says: {first}\n'
|
f'Server says: {first}\n'
|
||||||
f'Client says: {client_says}\n'
|
f'Client says: {client_says}\n'
|
||||||
|
|
@ -1116,7 +1116,7 @@ def test_peer_spawns_and_cancels_service_subactor(
|
||||||
print(
|
print(
|
||||||
'Sub-spawn came online\n'
|
'Sub-spawn came online\n'
|
||||||
f'portal: {sub}\n'
|
f'portal: {sub}\n'
|
||||||
f'.uid: {sub.actor.aid.uid}\n'
|
f'.uid: {sub.actor.uid}\n'
|
||||||
f'chan.raddr: {sub.chan.raddr}\n'
|
f'chan.raddr: {sub.chan.raddr}\n'
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1150,7 +1150,7 @@ def test_peer_spawns_and_cancels_service_subactor(
|
||||||
|
|
||||||
assert isinstance(res, ContextCancelled)
|
assert isinstance(res, ContextCancelled)
|
||||||
assert client_ctx.cancel_acked
|
assert client_ctx.cancel_acked
|
||||||
assert res.canceller == root.aid.uid
|
assert res.canceller == root.uid
|
||||||
assert not raise_sub_spawn_error_after
|
assert not raise_sub_spawn_error_after
|
||||||
|
|
||||||
# cancelling the spawner sub should
|
# cancelling the spawner sub should
|
||||||
|
|
@ -1184,8 +1184,8 @@ def test_peer_spawns_and_cancels_service_subactor(
|
||||||
# little_bro: a `RuntimeError`.
|
# little_bro: a `RuntimeError`.
|
||||||
#
|
#
|
||||||
check_inner_rte(rae)
|
check_inner_rte(rae)
|
||||||
assert rae.relay_uid == client.chan.aid.uid
|
assert rae.relay_uid == client.chan.uid
|
||||||
assert rae.src_uid == sub.chan.aid.uid
|
assert rae.src_uid == sub.chan.uid
|
||||||
|
|
||||||
assert not client_ctx.cancel_acked
|
assert not client_ctx.cancel_acked
|
||||||
assert (
|
assert (
|
||||||
|
|
@ -1214,12 +1214,12 @@ def test_peer_spawns_and_cancels_service_subactor(
|
||||||
except ContextCancelled as ctxc:
|
except ContextCancelled as ctxc:
|
||||||
_ctxc = ctxc
|
_ctxc = ctxc
|
||||||
print(
|
print(
|
||||||
f'{root.aid.uid} caught ctxc from ctx with {client_ctx.chan.aid.uid}\n'
|
f'{root.uid} caught ctxc from ctx with {client_ctx.chan.uid}\n'
|
||||||
f'{repr(ctxc)}\n'
|
f'{repr(ctxc)}\n'
|
||||||
)
|
)
|
||||||
|
|
||||||
if not raise_sub_spawn_error_after:
|
if not raise_sub_spawn_error_after:
|
||||||
assert ctxc.canceller == root.aid.uid
|
assert ctxc.canceller == root.uid
|
||||||
else:
|
else:
|
||||||
assert ctxc.canceller == spawner_uid
|
assert ctxc.canceller == spawner_uid
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,9 @@
|
||||||
"""
|
"""
|
||||||
Streaming via the, now legacy, "async-gen API".
|
Streaming via async gen api
|
||||||
|
|
||||||
"""
|
"""
|
||||||
import time
|
import time
|
||||||
from functools import partial
|
from functools import partial
|
||||||
import platform
|
import platform
|
||||||
from typing import Callable
|
|
||||||
|
|
||||||
import trio
|
import trio
|
||||||
import tractor
|
import tractor
|
||||||
|
|
@ -21,11 +19,7 @@ def test_must_define_ctx():
|
||||||
async def no_ctx():
|
async def no_ctx():
|
||||||
pass
|
pass
|
||||||
|
|
||||||
assert (
|
assert "no_ctx must be `ctx: tractor.Context" in str(err.value)
|
||||||
"no_ctx must be `ctx: tractor.Context"
|
|
||||||
in
|
|
||||||
str(err.value)
|
|
||||||
)
|
|
||||||
|
|
||||||
@tractor.stream
|
@tractor.stream
|
||||||
async def has_ctx(ctx):
|
async def has_ctx(ctx):
|
||||||
|
|
@ -75,14 +69,14 @@ async def stream_from_single_subactor(
|
||||||
async with tractor.open_nursery(
|
async with tractor.open_nursery(
|
||||||
registry_addrs=[reg_addr],
|
registry_addrs=[reg_addr],
|
||||||
start_method=start_method,
|
start_method=start_method,
|
||||||
) as an:
|
) as nursery:
|
||||||
|
|
||||||
async with tractor.find_actor('streamerd') as portals:
|
async with tractor.find_actor('streamerd') as portals:
|
||||||
|
|
||||||
if not portals:
|
if not portals:
|
||||||
|
|
||||||
# no brokerd actor found
|
# no brokerd actor found
|
||||||
portal = await an.start_actor(
|
portal = await nursery.start_actor(
|
||||||
'streamerd',
|
'streamerd',
|
||||||
enable_modules=[__name__],
|
enable_modules=[__name__],
|
||||||
)
|
)
|
||||||
|
|
@ -122,22 +116,11 @@ async def stream_from_single_subactor(
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize(
|
@pytest.mark.parametrize(
|
||||||
'stream_func',
|
'stream_func', [async_gen_stream, context_stream]
|
||||||
[
|
|
||||||
async_gen_stream,
|
|
||||||
context_stream,
|
|
||||||
],
|
|
||||||
ids='stream_func={}'.format
|
|
||||||
)
|
)
|
||||||
def test_stream_from_single_subactor(
|
def test_stream_from_single_subactor(reg_addr, start_method, stream_func):
|
||||||
reg_addr: tuple,
|
"""Verify streaming from a spawned async generator.
|
||||||
start_method: str,
|
"""
|
||||||
stream_func: Callable,
|
|
||||||
):
|
|
||||||
'''
|
|
||||||
Verify streaming from a spawned async generator.
|
|
||||||
|
|
||||||
'''
|
|
||||||
trio.run(
|
trio.run(
|
||||||
partial(
|
partial(
|
||||||
stream_from_single_subactor,
|
stream_from_single_subactor,
|
||||||
|
|
@ -149,9 +132,10 @@ def test_stream_from_single_subactor(
|
||||||
|
|
||||||
|
|
||||||
# this is the first 2 actors, streamer_1 and streamer_2
|
# this is the first 2 actors, streamer_1 and streamer_2
|
||||||
async def stream_data(seed: int):
|
async def stream_data(seed):
|
||||||
|
|
||||||
for i in range(seed):
|
for i in range(seed):
|
||||||
|
|
||||||
yield i
|
yield i
|
||||||
|
|
||||||
# trigger scheduler to simulate practical usage
|
# trigger scheduler to simulate practical usage
|
||||||
|
|
@ -159,17 +143,15 @@ async def stream_data(seed: int):
|
||||||
|
|
||||||
|
|
||||||
# this is the third actor; the aggregator
|
# this is the third actor; the aggregator
|
||||||
async def aggregate(seed: int):
|
async def aggregate(seed):
|
||||||
'''
|
"""Ensure that the two streams we receive match but only stream
|
||||||
Ensure that the two streams we receive match but only stream
|
|
||||||
a single set of values to the parent.
|
a single set of values to the parent.
|
||||||
|
"""
|
||||||
'''
|
async with tractor.open_nursery() as nursery:
|
||||||
async with tractor.open_nursery() as an:
|
|
||||||
portals = []
|
portals = []
|
||||||
for i in range(1, 3):
|
for i in range(1, 3):
|
||||||
# fork point
|
# fork point
|
||||||
portal = await an.start_actor(
|
portal = await nursery.start_actor(
|
||||||
name=f'streamer_{i}',
|
name=f'streamer_{i}',
|
||||||
enable_modules=[__name__],
|
enable_modules=[__name__],
|
||||||
)
|
)
|
||||||
|
|
@ -182,28 +164,20 @@ async def aggregate(seed: int):
|
||||||
async with send_chan:
|
async with send_chan:
|
||||||
|
|
||||||
async with portal.open_stream_from(
|
async with portal.open_stream_from(
|
||||||
stream_data,
|
stream_data, seed=seed,
|
||||||
seed=seed,
|
|
||||||
) as stream:
|
) as stream:
|
||||||
|
|
||||||
async for value in stream:
|
async for value in stream:
|
||||||
# leverage trio's built-in backpressure
|
# leverage trio's built-in backpressure
|
||||||
await send_chan.send(value)
|
await send_chan.send(value)
|
||||||
|
|
||||||
print(
|
print(f"FINISHED ITERATING {portal.channel.uid}")
|
||||||
f'FINISHED ITERATING!\n'
|
|
||||||
f'peer: {portal.channel.aid.uid}'
|
|
||||||
)
|
|
||||||
|
|
||||||
# spawn 2 trio tasks to collect streams and push to a local queue
|
# spawn 2 trio tasks to collect streams and push to a local queue
|
||||||
async with trio.open_nursery() as tn:
|
async with trio.open_nursery() as n:
|
||||||
|
|
||||||
for portal in portals:
|
for portal in portals:
|
||||||
tn.start_soon(
|
n.start_soon(push_to_chan, portal, send_chan.clone())
|
||||||
push_to_chan,
|
|
||||||
portal,
|
|
||||||
send_chan.clone(),
|
|
||||||
)
|
|
||||||
|
|
||||||
# close this local task's reference to send side
|
# close this local task's reference to send side
|
||||||
await send_chan.aclose()
|
await send_chan.aclose()
|
||||||
|
|
@ -220,21 +194,20 @@ async def aggregate(seed: int):
|
||||||
|
|
||||||
print("FINISHED ITERATING in aggregator")
|
print("FINISHED ITERATING in aggregator")
|
||||||
|
|
||||||
await an.cancel()
|
await nursery.cancel()
|
||||||
print("WAITING on `ActorNursery` to finish")
|
print("WAITING on `ActorNursery` to finish")
|
||||||
print("AGGREGATOR COMPLETE!")
|
print("AGGREGATOR COMPLETE!")
|
||||||
|
|
||||||
|
|
||||||
async def a_quadruple_example() -> list[int]:
|
# this is the main actor and *arbiter*
|
||||||
'''
|
async def a_quadruple_example():
|
||||||
Open the root-actor which is also a "registrar".
|
# a nursery which spawns "actors"
|
||||||
|
async with tractor.open_nursery() as nursery:
|
||||||
|
|
||||||
'''
|
|
||||||
async with tractor.open_nursery() as an:
|
|
||||||
seed = int(1e3)
|
seed = int(1e3)
|
||||||
pre_start = time.time()
|
pre_start = time.time()
|
||||||
|
|
||||||
portal = await an.start_actor(
|
portal = await nursery.start_actor(
|
||||||
name='aggregator',
|
name='aggregator',
|
||||||
enable_modules=[__name__],
|
enable_modules=[__name__],
|
||||||
)
|
)
|
||||||
|
|
@ -255,14 +228,8 @@ async def a_quadruple_example() -> list[int]:
|
||||||
return result_stream
|
return result_stream
|
||||||
|
|
||||||
|
|
||||||
async def cancel_after(
|
async def cancel_after(wait, reg_addr):
|
||||||
wait: float,
|
async with tractor.open_root_actor(registry_addrs=[reg_addr]):
|
||||||
reg_addr: tuple,
|
|
||||||
) -> list[int]:
|
|
||||||
|
|
||||||
async with tractor.open_root_actor(
|
|
||||||
registry_addrs=[reg_addr],
|
|
||||||
):
|
|
||||||
with trio.move_on_after(wait):
|
with trio.move_on_after(wait):
|
||||||
return await a_quadruple_example()
|
return await a_quadruple_example()
|
||||||
|
|
||||||
|
|
@ -273,10 +240,6 @@ def time_quad_ex(
|
||||||
ci_env: bool,
|
ci_env: bool,
|
||||||
spawn_backend: str,
|
spawn_backend: str,
|
||||||
):
|
):
|
||||||
non_linux: bool = (_sys := platform.system()) != 'Linux'
|
|
||||||
if ci_env and non_linux:
|
|
||||||
pytest.skip(f'Test is too flaky on {_sys!r} in CI')
|
|
||||||
|
|
||||||
if spawn_backend == 'mp':
|
if spawn_backend == 'mp':
|
||||||
'''
|
'''
|
||||||
no idea but the mp *nix runs are flaking out here often...
|
no idea but the mp *nix runs are flaking out here often...
|
||||||
|
|
@ -284,20 +247,16 @@ def time_quad_ex(
|
||||||
'''
|
'''
|
||||||
pytest.skip("Test is too flaky on mp in CI")
|
pytest.skip("Test is too flaky on mp in CI")
|
||||||
|
|
||||||
timeout = 7 if non_linux else 4
|
timeout = 7 if platform.system() in ('Windows', 'Darwin') else 4
|
||||||
start = time.time()
|
start = time.time()
|
||||||
results: list[int] = trio.run(
|
results = trio.run(cancel_after, timeout, reg_addr)
|
||||||
cancel_after,
|
diff = time.time() - start
|
||||||
timeout,
|
|
||||||
reg_addr,
|
|
||||||
)
|
|
||||||
diff: float = time.time() - start
|
|
||||||
assert results
|
assert results
|
||||||
return results, diff
|
return results, diff
|
||||||
|
|
||||||
|
|
||||||
def test_a_quadruple_example(
|
def test_a_quadruple_example(
|
||||||
time_quad_ex: tuple[list[int], float],
|
time_quad_ex: tuple,
|
||||||
ci_env: bool,
|
ci_env: bool,
|
||||||
spawn_backend: str,
|
spawn_backend: str,
|
||||||
):
|
):
|
||||||
|
|
@ -305,12 +264,13 @@ def test_a_quadruple_example(
|
||||||
This also serves as a kind of "we'd like to be this fast test".
|
This also serves as a kind of "we'd like to be this fast test".
|
||||||
|
|
||||||
'''
|
'''
|
||||||
non_linux: bool = (_sys := platform.system()) != 'Linux'
|
|
||||||
|
|
||||||
results, diff = time_quad_ex
|
results, diff = time_quad_ex
|
||||||
assert results
|
assert results
|
||||||
this_fast = (
|
this_fast = (
|
||||||
6 if non_linux
|
6 if platform.system() in (
|
||||||
|
'Windows',
|
||||||
|
'Darwin',
|
||||||
|
)
|
||||||
else 3
|
else 3
|
||||||
)
|
)
|
||||||
assert diff < this_fast
|
assert diff < this_fast
|
||||||
|
|
@ -321,33 +281,19 @@ def test_a_quadruple_example(
|
||||||
list(map(lambda i: i/10, range(3, 9)))
|
list(map(lambda i: i/10, range(3, 9)))
|
||||||
)
|
)
|
||||||
def test_not_fast_enough_quad(
|
def test_not_fast_enough_quad(
|
||||||
reg_addr: tuple,
|
reg_addr, time_quad_ex, cancel_delay, ci_env, spawn_backend
|
||||||
time_quad_ex: tuple[list[int], float],
|
|
||||||
cancel_delay: float,
|
|
||||||
ci_env: bool,
|
|
||||||
spawn_backend: str,
|
|
||||||
):
|
):
|
||||||
'''
|
"""Verify we can cancel midway through the quad example and all actors
|
||||||
Verify we can cancel midway through the quad example and all
|
cancel gracefully.
|
||||||
actors cancel gracefully.
|
"""
|
||||||
|
|
||||||
'''
|
|
||||||
results, diff = time_quad_ex
|
results, diff = time_quad_ex
|
||||||
delay = max(diff - cancel_delay, 0)
|
delay = max(diff - cancel_delay, 0)
|
||||||
results = trio.run(
|
results = trio.run(cancel_after, delay, reg_addr)
|
||||||
cancel_after,
|
system = platform.system()
|
||||||
delay,
|
if system in ('Windows', 'Darwin') and results is not None:
|
||||||
reg_addr,
|
|
||||||
)
|
|
||||||
system: str = platform.system()
|
|
||||||
if (
|
|
||||||
system in ('Windows', 'Darwin')
|
|
||||||
and
|
|
||||||
results is not None
|
|
||||||
):
|
|
||||||
# In CI envoirments it seems later runs are quicker then the first
|
# In CI envoirments it seems later runs are quicker then the first
|
||||||
# so just ignore these
|
# so just ignore these
|
||||||
print(f'Woa there {system} caught your breath eh?')
|
print(f"Woa there {system} caught your breath eh?")
|
||||||
else:
|
else:
|
||||||
# should be cancelled mid-streaming
|
# should be cancelled mid-streaming
|
||||||
assert results is None
|
assert results is None
|
||||||
|
|
@ -355,24 +301,23 @@ def test_not_fast_enough_quad(
|
||||||
|
|
||||||
@tractor_test
|
@tractor_test
|
||||||
async def test_respawn_consumer_task(
|
async def test_respawn_consumer_task(
|
||||||
reg_addr: tuple,
|
reg_addr,
|
||||||
spawn_backend: str,
|
spawn_backend,
|
||||||
loglevel: str,
|
loglevel,
|
||||||
):
|
):
|
||||||
'''
|
"""Verify that ``._portal.ReceiveStream.shield()``
|
||||||
Verify that ``._portal.ReceiveStream.shield()``
|
|
||||||
sucessfully protects the underlying IPC channel from being closed
|
sucessfully protects the underlying IPC channel from being closed
|
||||||
when cancelling and respawning a consumer task.
|
when cancelling and respawning a consumer task.
|
||||||
|
|
||||||
This also serves to verify that all values from the stream can be
|
This also serves to verify that all values from the stream can be
|
||||||
received despite the respawns.
|
received despite the respawns.
|
||||||
|
|
||||||
'''
|
"""
|
||||||
stream = None
|
stream = None
|
||||||
|
|
||||||
async with tractor.open_nursery() as an:
|
async with tractor.open_nursery() as n:
|
||||||
|
|
||||||
portal = await an.start_actor(
|
portal = await n.start_actor(
|
||||||
name='streamer',
|
name='streamer',
|
||||||
enable_modules=[__name__]
|
enable_modules=[__name__]
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -35,9 +35,6 @@ if TYPE_CHECKING:
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
_non_linux: bool = platform.system() != 'Linux'
|
|
||||||
|
|
||||||
|
|
||||||
def test_abort_on_sigint(
|
def test_abort_on_sigint(
|
||||||
daemon: subprocess.Popen,
|
daemon: subprocess.Popen,
|
||||||
):
|
):
|
||||||
|
|
@ -140,7 +137,6 @@ def test_non_registrar_spawns_child(
|
||||||
reg_addr: UnwrappedAddress,
|
reg_addr: UnwrappedAddress,
|
||||||
loglevel: str,
|
loglevel: str,
|
||||||
debug_mode: bool,
|
debug_mode: bool,
|
||||||
ci_env: bool,
|
|
||||||
):
|
):
|
||||||
'''
|
'''
|
||||||
Ensure a non-regristar (serving) root actor can spawn a sub and
|
Ensure a non-regristar (serving) root actor can spawn a sub and
|
||||||
|
|
@ -152,12 +148,6 @@ def test_non_registrar_spawns_child(
|
||||||
|
|
||||||
'''
|
'''
|
||||||
async def main():
|
async def main():
|
||||||
|
|
||||||
# XXX, since apparently on macos in GH's CI it can be a race
|
|
||||||
# with the `daemon` registrar on grabbing the socket-addr..
|
|
||||||
if ci_env and _non_linux:
|
|
||||||
await trio.sleep(.5)
|
|
||||||
|
|
||||||
async with tractor.open_nursery(
|
async with tractor.open_nursery(
|
||||||
registry_addrs=[reg_addr],
|
registry_addrs=[reg_addr],
|
||||||
loglevel=loglevel,
|
loglevel=loglevel,
|
||||||
|
|
|
||||||
|
|
@ -49,7 +49,7 @@ def test_infected_root_actor(
|
||||||
),
|
),
|
||||||
to_asyncio.open_channel_from(
|
to_asyncio.open_channel_from(
|
||||||
aio_echo_server,
|
aio_echo_server,
|
||||||
) as (chan, first),
|
) as (first, chan),
|
||||||
):
|
):
|
||||||
assert first == 'start'
|
assert first == 'start'
|
||||||
|
|
||||||
|
|
@ -91,12 +91,13 @@ def test_infected_root_actor(
|
||||||
async def sync_and_err(
|
async def sync_and_err(
|
||||||
# just signature placeholders for compat with
|
# just signature placeholders for compat with
|
||||||
# ``to_asyncio.open_channel_from()``
|
# ``to_asyncio.open_channel_from()``
|
||||||
chan: tractor.to_asyncio.LinkedTaskChannel,
|
to_trio: trio.MemorySendChannel,
|
||||||
|
from_trio: asyncio.Queue,
|
||||||
ev: asyncio.Event,
|
ev: asyncio.Event,
|
||||||
|
|
||||||
):
|
):
|
||||||
if chan:
|
if to_trio:
|
||||||
chan.started_nowait('start')
|
to_trio.send_nowait('start')
|
||||||
|
|
||||||
await ev.wait()
|
await ev.wait()
|
||||||
raise RuntimeError('asyncio-side')
|
raise RuntimeError('asyncio-side')
|
||||||
|
|
@ -173,7 +174,7 @@ def test_trio_prestarted_task_bubbles(
|
||||||
sync_and_err,
|
sync_and_err,
|
||||||
ev=aio_ev,
|
ev=aio_ev,
|
||||||
)
|
)
|
||||||
) as (chan, first),
|
) as (first, chan),
|
||||||
):
|
):
|
||||||
|
|
||||||
for i in range(5):
|
for i in range(5):
|
||||||
|
|
|
||||||
|
|
@ -22,10 +22,6 @@ def unlink_file():
|
||||||
async def crash_and_clean_tmpdir(
|
async def crash_and_clean_tmpdir(
|
||||||
tmp_file_path: str,
|
tmp_file_path: str,
|
||||||
error: bool = True,
|
error: bool = True,
|
||||||
rent_cancel: bool = True,
|
|
||||||
|
|
||||||
# XXX unused, but do we really need to test these cases?
|
|
||||||
self_cancel: bool = False,
|
|
||||||
):
|
):
|
||||||
global _file_path
|
global _file_path
|
||||||
_file_path = tmp_file_path
|
_file_path = tmp_file_path
|
||||||
|
|
@ -36,75 +32,43 @@ async def crash_and_clean_tmpdir(
|
||||||
assert os.path.isfile(tmp_file_path)
|
assert os.path.isfile(tmp_file_path)
|
||||||
await trio.sleep(0.1)
|
await trio.sleep(0.1)
|
||||||
if error:
|
if error:
|
||||||
print('erroring in subactor!')
|
|
||||||
assert 0
|
assert 0
|
||||||
|
else:
|
||||||
elif self_cancel:
|
|
||||||
print('SELF-cancelling subactor!')
|
|
||||||
actor.cancel_soon()
|
actor.cancel_soon()
|
||||||
|
|
||||||
elif rent_cancel:
|
|
||||||
await trio.sleep_forever()
|
|
||||||
|
|
||||||
print('subactor exiting task!')
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize(
|
@pytest.mark.parametrize(
|
||||||
'error_in_child',
|
'error_in_child',
|
||||||
[True, False],
|
[True, False],
|
||||||
ids='error_in_child={}'.format,
|
|
||||||
)
|
)
|
||||||
@tractor_test
|
@tractor_test
|
||||||
async def test_lifetime_stack_wipes_tmpfile(
|
async def test_lifetime_stack_wipes_tmpfile(
|
||||||
tmp_path,
|
tmp_path,
|
||||||
error_in_child: bool,
|
error_in_child: bool,
|
||||||
loglevel: str,
|
|
||||||
# log: tractor.log.StackLevelAdapter,
|
|
||||||
# ^TODO, once landed via macos support!
|
|
||||||
):
|
):
|
||||||
child_tmp_file = tmp_path / "child.txt"
|
child_tmp_file = tmp_path / "child.txt"
|
||||||
child_tmp_file.touch()
|
child_tmp_file.touch()
|
||||||
assert child_tmp_file.exists()
|
assert child_tmp_file.exists()
|
||||||
path = str(child_tmp_file)
|
path = str(child_tmp_file)
|
||||||
|
|
||||||
# NOTE, this is expected to cancel the sub
|
|
||||||
# in the `error_in_child=False` case!
|
|
||||||
timeout: float = (
|
|
||||||
1.6 if error_in_child
|
|
||||||
else 1
|
|
||||||
)
|
|
||||||
try:
|
try:
|
||||||
with trio.move_on_after(timeout) as cs:
|
with trio.move_on_after(0.5):
|
||||||
async with tractor.open_nursery(
|
async with tractor.open_nursery() as n:
|
||||||
loglevel=loglevel,
|
await ( # inlined portal
|
||||||
) as an:
|
await n.run_in_actor(
|
||||||
await ( # inlined `tractor.Portal`
|
crash_and_clean_tmpdir,
|
||||||
await an.run_in_actor(
|
tmp_file_path=path,
|
||||||
crash_and_clean_tmpdir,
|
error=error_in_child,
|
||||||
tmp_file_path=path,
|
)
|
||||||
error=error_in_child,
|
).result()
|
||||||
)
|
|
||||||
).result()
|
|
||||||
except (
|
except (
|
||||||
tractor.RemoteActorError,
|
tractor.RemoteActorError,
|
||||||
|
# tractor.BaseExceptionGroup,
|
||||||
BaseExceptionGroup,
|
BaseExceptionGroup,
|
||||||
) as _exc:
|
):
|
||||||
exc = _exc
|
pass
|
||||||
from tractor.log import get_console_log
|
|
||||||
log = get_console_log(
|
|
||||||
level=loglevel,
|
|
||||||
name=__name__,
|
|
||||||
)
|
|
||||||
log.exception(
|
|
||||||
f'Subactor failed as expected with {type(exc)!r}\n'
|
|
||||||
)
|
|
||||||
|
|
||||||
# tmp file should have been wiped by
|
# tmp file should have been wiped by
|
||||||
# teardown stack.
|
# teardown stack.
|
||||||
assert not child_tmp_file.exists()
|
assert not child_tmp_file.exists()
|
||||||
|
|
||||||
if error_in_child:
|
|
||||||
assert not cs.cancel_called
|
|
||||||
else:
|
|
||||||
# expect timeout in some cases?
|
|
||||||
assert cs.cancel_called
|
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,6 @@
|
||||||
Shared mem primitives and APIs.
|
Shared mem primitives and APIs.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
import platform
|
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
# import numpy
|
# import numpy
|
||||||
|
|
@ -54,18 +53,7 @@ def test_child_attaches_alot():
|
||||||
shm_key=shml.key,
|
shm_key=shml.key,
|
||||||
) as (ctx, start_val),
|
) as (ctx, start_val),
|
||||||
):
|
):
|
||||||
assert (_key := shml.key) == start_val
|
assert start_val == key
|
||||||
|
|
||||||
if platform.system() != 'Darwin':
|
|
||||||
# XXX, macOS has a char limit..
|
|
||||||
# see `ipc._shm._shorten_key_for_macos`
|
|
||||||
assert (
|
|
||||||
start_val
|
|
||||||
==
|
|
||||||
key
|
|
||||||
==
|
|
||||||
_key
|
|
||||||
)
|
|
||||||
await ctx.result()
|
await ctx.result()
|
||||||
|
|
||||||
await portal.cancel_actor()
|
await portal.cancel_actor()
|
||||||
|
|
|
||||||
|
|
@ -66,7 +66,7 @@ async def open_actor_cluster(
|
||||||
trio.open_nursery() as tn,
|
trio.open_nursery() as tn,
|
||||||
tractor.trionics.maybe_raise_from_masking_exc()
|
tractor.trionics.maybe_raise_from_masking_exc()
|
||||||
):
|
):
|
||||||
uid = tractor.current_actor().aid.uid
|
uid = tractor.current_actor().uid
|
||||||
|
|
||||||
async def _start(name: str) -> None:
|
async def _start(name: str) -> None:
|
||||||
name = f'{uid[0]}.{name}'
|
name = f'{uid[0]}.{name}'
|
||||||
|
|
|
||||||
|
|
@ -463,11 +463,10 @@ class Context:
|
||||||
|
|
||||||
# self._cancel_called = val
|
# self._cancel_called = val
|
||||||
|
|
||||||
# TODO, use the `Actor.aid: Aid` instead!
|
|
||||||
@property
|
@property
|
||||||
def canceller(self) -> tuple[str, str]|None:
|
def canceller(self) -> tuple[str, str]|None:
|
||||||
'''
|
'''
|
||||||
`Actor.aid.uid: tuple[str, str]` of the (remote)
|
`Actor.uid: tuple[str, str]` of the (remote)
|
||||||
actor-process who's task was cancelled thus causing this
|
actor-process who's task was cancelled thus causing this
|
||||||
(side of the) context to also be cancelled.
|
(side of the) context to also be cancelled.
|
||||||
|
|
||||||
|
|
@ -500,12 +499,12 @@ class Context:
|
||||||
if from_uid := re.src_uid:
|
if from_uid := re.src_uid:
|
||||||
from_uid: tuple = tuple(from_uid)
|
from_uid: tuple = tuple(from_uid)
|
||||||
|
|
||||||
our_uid: tuple = self._actor.aid.uid
|
our_uid: tuple = self._actor.uid
|
||||||
our_canceller = self.canceller
|
our_canceller = self.canceller
|
||||||
|
|
||||||
return bool(
|
return bool(
|
||||||
isinstance((ctxc := re), ContextCancelled)
|
isinstance((ctxc := re), ContextCancelled)
|
||||||
and from_uid == self.chan.aid.uid
|
and from_uid == self.chan.uid
|
||||||
and ctxc.canceller == our_uid
|
and ctxc.canceller == our_uid
|
||||||
and our_canceller == our_uid
|
and our_canceller == our_uid
|
||||||
)
|
)
|
||||||
|
|
@ -516,7 +515,7 @@ class Context:
|
||||||
Records whether the task on the remote side of this IPC
|
Records whether the task on the remote side of this IPC
|
||||||
context acknowledged a cancel request via a relayed
|
context acknowledged a cancel request via a relayed
|
||||||
`ContextCancelled` with the `.canceller` attr set to the
|
`ContextCancelled` with the `.canceller` attr set to the
|
||||||
`Actor.aid.uid` of the local actor who's task entered
|
`Actor.uid` of the local actor who's task entered
|
||||||
`Portal.open_context()`.
|
`Portal.open_context()`.
|
||||||
|
|
||||||
This will only be `True` when `.cancel()` is called and
|
This will only be `True` when `.cancel()` is called and
|
||||||
|
|
@ -790,8 +789,8 @@ class Context:
|
||||||
# appropriately.
|
# appropriately.
|
||||||
log.runtime(
|
log.runtime(
|
||||||
'Setting remote error for ctx\n\n'
|
'Setting remote error for ctx\n\n'
|
||||||
f'<= {self.peer_side!r}: {self.chan.aid.reprol()}\n'
|
f'<= {self.peer_side!r}: {self.chan.uid}\n'
|
||||||
f'=> {self.side!r}: {self._actor.aid.reprol()}\n\n'
|
f'=> {self.side!r}: {self._actor.uid}\n\n'
|
||||||
f'{error!r}'
|
f'{error!r}'
|
||||||
)
|
)
|
||||||
self._remote_error: BaseException = error
|
self._remote_error: BaseException = error
|
||||||
|
|
@ -812,7 +811,7 @@ class Context:
|
||||||
# cancelled.
|
# cancelled.
|
||||||
#
|
#
|
||||||
# !TODO, switching to `Actor.aid` here!
|
# !TODO, switching to `Actor.aid` here!
|
||||||
if (canc := error.canceller) == self._actor.aid.uid:
|
if (canc := error.canceller) == self._actor.uid:
|
||||||
whom: str = 'us'
|
whom: str = 'us'
|
||||||
self._canceller = canc
|
self._canceller = canc
|
||||||
else:
|
else:
|
||||||
|
|
@ -1037,7 +1036,7 @@ class Context:
|
||||||
---------
|
---------
|
||||||
- after the far end cancels, the `.cancel()` calling side
|
- after the far end cancels, the `.cancel()` calling side
|
||||||
should receive a `ContextCancelled` with the
|
should receive a `ContextCancelled` with the
|
||||||
`.canceller: tuple` uid set to the current `Actor.aid.uid`.
|
`.canceller: tuple` uid set to the current `Actor.uid`.
|
||||||
|
|
||||||
- timeout (quickly) on failure to rx this ACK error-msg in
|
- timeout (quickly) on failure to rx this ACK error-msg in
|
||||||
an attempt to sidestep 2-generals when the transport
|
an attempt to sidestep 2-generals when the transport
|
||||||
|
|
@ -1066,9 +1065,9 @@ class Context:
|
||||||
)
|
)
|
||||||
reminfo: str = (
|
reminfo: str = (
|
||||||
# ' =>\n'
|
# ' =>\n'
|
||||||
# f'Context.cancel() => {self.chan.aid.uid}\n'
|
# f'Context.cancel() => {self.chan.uid}\n'
|
||||||
f'\n'
|
f'\n'
|
||||||
f'c)=> {self.chan.aid.reprol()}\n'
|
f'c)=> {self.chan.uid}\n'
|
||||||
f' |_[{self.dst_maddr}\n'
|
f' |_[{self.dst_maddr}\n'
|
||||||
f' >> {self.repr_rpc}\n'
|
f' >> {self.repr_rpc}\n'
|
||||||
# f' >> {self._nsf}() -> {codec}[dict]:\n\n'
|
# f' >> {self._nsf}() -> {codec}[dict]:\n\n'
|
||||||
|
|
@ -1212,7 +1211,7 @@ class Context:
|
||||||
|
|
||||||
'''
|
'''
|
||||||
__tracebackhide__: bool = hide_tb
|
__tracebackhide__: bool = hide_tb
|
||||||
peer_uid: tuple = self.chan.aid.uid
|
peer_uid: tuple = self.chan.uid
|
||||||
|
|
||||||
# XXX NOTE XXX: `ContextCancelled`/`StreamOverrun` absorption
|
# XXX NOTE XXX: `ContextCancelled`/`StreamOverrun` absorption
|
||||||
# for "graceful cancellation" case(s):
|
# for "graceful cancellation" case(s):
|
||||||
|
|
@ -1229,7 +1228,7 @@ class Context:
|
||||||
# (`ContextCancelled`) as an expected
|
# (`ContextCancelled`) as an expected
|
||||||
# error-msg-is-cancellation-ack IFF said
|
# error-msg-is-cancellation-ack IFF said
|
||||||
# `remote_error: ContextCancelled` has `.canceller`
|
# `remote_error: ContextCancelled` has `.canceller`
|
||||||
# set to the `Actor.aid.uid` of THIS task (i.e. the
|
# set to the `Actor.uid` of THIS task (i.e. the
|
||||||
# cancellation requesting task's actor is the actor
|
# cancellation requesting task's actor is the actor
|
||||||
# checking whether it should absorb the ctxc).
|
# checking whether it should absorb the ctxc).
|
||||||
self_ctxc: bool = self._is_self_cancelled(remote_error)
|
self_ctxc: bool = self._is_self_cancelled(remote_error)
|
||||||
|
|
@ -1680,7 +1679,7 @@ class Context:
|
||||||
|
|
||||||
elif self._started_called:
|
elif self._started_called:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f'called `.started()` twice on context with {self.chan.aid.uid}'
|
f'called `.started()` twice on context with {self.chan.uid}'
|
||||||
)
|
)
|
||||||
|
|
||||||
started_msg = Started(
|
started_msg = Started(
|
||||||
|
|
@ -1813,7 +1812,7 @@ class Context:
|
||||||
'''
|
'''
|
||||||
cid: str = self.cid
|
cid: str = self.cid
|
||||||
chan: Channel = self.chan
|
chan: Channel = self.chan
|
||||||
from_uid: tuple[str, str] = chan.aid.uid
|
from_uid: tuple[str, str] = chan.uid
|
||||||
send_chan: trio.MemorySendChannel = self._send_chan
|
send_chan: trio.MemorySendChannel = self._send_chan
|
||||||
nsf: NamespacePath = self._nsf
|
nsf: NamespacePath = self._nsf
|
||||||
|
|
||||||
|
|
@ -1954,22 +1953,20 @@ class Context:
|
||||||
# overrun state and that msg isn't stuck in an
|
# overrun state and that msg isn't stuck in an
|
||||||
# overflow queue what happens?!?
|
# overflow queue what happens?!?
|
||||||
|
|
||||||
local_aid = self._actor.aid
|
local_uid = self._actor.uid
|
||||||
txt: str = (
|
txt: str = (
|
||||||
'on IPC context:\n'
|
'on IPC context:\n'
|
||||||
|
|
||||||
f'<= sender: {from_uid}\n'
|
f'<= sender: {from_uid}\n'
|
||||||
f' |_ {self._nsf}()\n\n'
|
f' |_ {self._nsf}()\n\n'
|
||||||
|
|
||||||
f'=> overrun: {local_aid.reprol()!r}\n'
|
f'=> overrun: {local_uid}\n'
|
||||||
f' |_cid: {cid}\n'
|
f' |_cid: {cid}\n'
|
||||||
f' |_task: {self._task}\n'
|
f' |_task: {self._task}\n'
|
||||||
)
|
)
|
||||||
if not self._stream_opened:
|
if not self._stream_opened:
|
||||||
txt += (
|
txt += (
|
||||||
f'\n'
|
f'\n*** No stream open on `{local_uid[0]}` side! ***\n\n'
|
||||||
f'*** No stream open on `{local_aid.name}` side! ***\n'
|
|
||||||
f'\n'
|
|
||||||
f'{msg}\n'
|
f'{msg}\n'
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -2118,11 +2115,7 @@ async def open_context_from_portal(
|
||||||
# XXX NOTE XXX: currenly we do NOT allow opening a contex
|
# XXX NOTE XXX: currenly we do NOT allow opening a contex
|
||||||
# with "self" since the local feeder mem-chan processing
|
# with "self" since the local feeder mem-chan processing
|
||||||
# is not built for it.
|
# is not built for it.
|
||||||
if (
|
if (uid := portal.channel.uid) == portal.actor.uid:
|
||||||
(uid := portal.channel.aid.uid)
|
|
||||||
==
|
|
||||||
portal.actor.aid.uid
|
|
||||||
):
|
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
'** !! Invalid Operation !! **\n'
|
'** !! Invalid Operation !! **\n'
|
||||||
'Can not open an IPC ctx with the local actor!\n'
|
'Can not open an IPC ctx with the local actor!\n'
|
||||||
|
|
@ -2336,7 +2329,7 @@ async def open_context_from_portal(
|
||||||
and
|
and
|
||||||
ctxc is ctx._remote_error
|
ctxc is ctx._remote_error
|
||||||
and
|
and
|
||||||
ctxc.canceller == portal.actor.aid.uid
|
ctxc.canceller == portal.actor.uid
|
||||||
):
|
):
|
||||||
log.cancel(
|
log.cancel(
|
||||||
f'Context (cid=[{ctx.cid[-6:]}..] cancelled gracefully with:\n'
|
f'Context (cid=[{ctx.cid[-6:]}..] cancelled gracefully with:\n'
|
||||||
|
|
@ -2413,7 +2406,7 @@ async def open_context_from_portal(
|
||||||
logmeth(msg)
|
logmeth(msg)
|
||||||
|
|
||||||
if debug_mode():
|
if debug_mode():
|
||||||
# async with debug.acquire_debug_lock(portal.actor.aid.uid):
|
# async with debug.acquire_debug_lock(portal.actor.uid):
|
||||||
# pass
|
# pass
|
||||||
# TODO: factor ^ into below for non-root cases?
|
# TODO: factor ^ into below for non-root cases?
|
||||||
#
|
#
|
||||||
|
|
|
||||||
|
|
@ -982,7 +982,6 @@ class TransportClosed(Exception):
|
||||||
'''
|
'''
|
||||||
__tracebackhide__: bool = hide_tb
|
__tracebackhide__: bool = hide_tb
|
||||||
message: str = message or self.message
|
message: str = message or self.message
|
||||||
|
|
||||||
# when a cause is set, slap it onto the log emission.
|
# when a cause is set, slap it onto the log emission.
|
||||||
if cause := self.src_exc:
|
if cause := self.src_exc:
|
||||||
cause_tb_str: str = ''.join(
|
cause_tb_str: str = ''.join(
|
||||||
|
|
@ -990,7 +989,7 @@ class TransportClosed(Exception):
|
||||||
)
|
)
|
||||||
message += (
|
message += (
|
||||||
f'{cause_tb_str}\n' # tb
|
f'{cause_tb_str}\n' # tb
|
||||||
f'{cause!r}\n' # exc repr
|
f' {cause}\n' # exc repr
|
||||||
)
|
)
|
||||||
|
|
||||||
getattr(
|
getattr(
|
||||||
|
|
|
||||||
|
|
@ -252,8 +252,8 @@ async def _invoke_non_context(
|
||||||
):
|
):
|
||||||
log.warning(
|
log.warning(
|
||||||
'Failed to send RPC result?\n'
|
'Failed to send RPC result?\n'
|
||||||
f'|_{func}@{actor.aid.reprol()}() -> {ret_msg}\n\n'
|
f'|_{func}@{actor.uid}() -> {ret_msg}\n\n'
|
||||||
f'x=> peer: {chan.aid.reprol()}\n'
|
f'x=> peer: {chan.uid}\n'
|
||||||
)
|
)
|
||||||
|
|
||||||
@acm
|
@acm
|
||||||
|
|
@ -698,7 +698,7 @@ async def _invoke(
|
||||||
# which cancels the scope presuming the input error
|
# which cancels the scope presuming the input error
|
||||||
# is not a `.cancel_acked` pleaser.
|
# is not a `.cancel_acked` pleaser.
|
||||||
if rpc_ctx_cs.cancelled_caught:
|
if rpc_ctx_cs.cancelled_caught:
|
||||||
our_uid: tuple = actor.aid.uid
|
our_uid: tuple = actor.uid
|
||||||
|
|
||||||
# first check for and raise any remote error
|
# first check for and raise any remote error
|
||||||
# before raising any context cancelled case
|
# before raising any context cancelled case
|
||||||
|
|
@ -730,7 +730,7 @@ async def _invoke(
|
||||||
# TODO: determine if the ctx peer task was the
|
# TODO: determine if the ctx peer task was the
|
||||||
# exact task which cancelled, vs. some other
|
# exact task which cancelled, vs. some other
|
||||||
# task in the same actor.
|
# task in the same actor.
|
||||||
elif canceller == ctx.chan.aid.uid:
|
elif canceller == ctx.chan.uid:
|
||||||
explain += f'its {ctx.peer_side!r}-side peer'
|
explain += f'its {ctx.peer_side!r}-side peer'
|
||||||
|
|
||||||
elif canceller == our_uid:
|
elif canceller == our_uid:
|
||||||
|
|
@ -825,7 +825,7 @@ async def _invoke(
|
||||||
# associated child isn't in debug any more
|
# associated child isn't in debug any more
|
||||||
await debug.maybe_wait_for_debugger()
|
await debug.maybe_wait_for_debugger()
|
||||||
ctx: Context = actor._contexts.pop((
|
ctx: Context = actor._contexts.pop((
|
||||||
chan.aid.uid,
|
chan.uid,
|
||||||
cid,
|
cid,
|
||||||
))
|
))
|
||||||
|
|
||||||
|
|
@ -927,7 +927,7 @@ async def try_ship_error_to_remote(
|
||||||
log.critical(
|
log.critical(
|
||||||
'IPC transport failure -> '
|
'IPC transport failure -> '
|
||||||
f'failed to ship error to {remote_descr}!\n\n'
|
f'failed to ship error to {remote_descr}!\n\n'
|
||||||
f'{type(msg)!r}[{msg.boxed_type_str}] X=> {channel.aid.uid}\n'
|
f'{type(msg)!r}[{msg.boxed_type_str}] X=> {channel.uid}\n'
|
||||||
f'\n'
|
f'\n'
|
||||||
# TODO: use `.msg.preetty_struct` for this!
|
# TODO: use `.msg.preetty_struct` for this!
|
||||||
f'{msg}\n'
|
f'{msg}\n'
|
||||||
|
|
@ -1005,7 +1005,7 @@ async def process_messages(
|
||||||
async for msg in chan:
|
async for msg in chan:
|
||||||
log.transport( # type: ignore
|
log.transport( # type: ignore
|
||||||
f'IPC msg from peer\n'
|
f'IPC msg from peer\n'
|
||||||
f'<= {chan.aid.reprol()}\n\n'
|
f'<= {chan.uid}\n\n'
|
||||||
|
|
||||||
# TODO: use of the pprinting of structs is
|
# TODO: use of the pprinting of structs is
|
||||||
# FRAGILE and should prolly not be
|
# FRAGILE and should prolly not be
|
||||||
|
|
@ -1109,7 +1109,7 @@ async def process_messages(
|
||||||
except BaseException:
|
except BaseException:
|
||||||
log.exception(
|
log.exception(
|
||||||
'Failed to cancel task?\n'
|
'Failed to cancel task?\n'
|
||||||
f'<= canceller: {chan.aid.reprol()}\n'
|
f'<= canceller: {chan.uid}\n'
|
||||||
f' |_{chan}\n\n'
|
f' |_{chan}\n\n'
|
||||||
f'=> {actor}\n'
|
f'=> {actor}\n'
|
||||||
f' |_cid: {target_cid}\n'
|
f' |_cid: {target_cid}\n'
|
||||||
|
|
@ -1264,7 +1264,7 @@ async def process_messages(
|
||||||
|
|
||||||
log.transport(
|
log.transport(
|
||||||
'Waiting on next IPC msg from\n'
|
'Waiting on next IPC msg from\n'
|
||||||
f'peer: {chan.aid.reprol()}\n'
|
f'peer: {chan.uid}\n'
|
||||||
f'|_{chan}\n'
|
f'|_{chan}\n'
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1313,10 +1313,12 @@ async def process_messages(
|
||||||
f'peer IPC channel closed abruptly?\n'
|
f'peer IPC channel closed abruptly?\n'
|
||||||
f'\n'
|
f'\n'
|
||||||
f'<=x[\n'
|
f'<=x[\n'
|
||||||
f'{chan}\n'
|
f' {chan}\n'
|
||||||
|
f' |_{chan.raddr}\n\n'
|
||||||
)
|
)
|
||||||
+
|
+
|
||||||
tc.message
|
tc.message
|
||||||
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# transport **WAS** disconnected
|
# transport **WAS** disconnected
|
||||||
|
|
@ -1339,8 +1341,8 @@ async def process_messages(
|
||||||
match err:
|
match err:
|
||||||
case ContextCancelled():
|
case ContextCancelled():
|
||||||
log.cancel(
|
log.cancel(
|
||||||
f'Actor: {actor.aid.reprol()!r} is ctxc with,\n'
|
f'Actor: {actor.uid} was context-cancelled with,\n'
|
||||||
f'{str(err)}'
|
f'str(err)'
|
||||||
)
|
)
|
||||||
case _:
|
case _:
|
||||||
log.exception("Actor errored:")
|
log.exception("Actor errored:")
|
||||||
|
|
|
||||||
|
|
@ -691,7 +691,7 @@ class Actor:
|
||||||
|
|
||||||
'''
|
'''
|
||||||
# ?TODO, use Aid here as well?
|
# ?TODO, use Aid here as well?
|
||||||
actor_uid = chan.aid.uid
|
actor_uid = chan.uid
|
||||||
assert actor_uid
|
assert actor_uid
|
||||||
try:
|
try:
|
||||||
ctx = self._contexts[(
|
ctx = self._contexts[(
|
||||||
|
|
@ -701,7 +701,7 @@ class Actor:
|
||||||
)]
|
)]
|
||||||
log.debug(
|
log.debug(
|
||||||
f'Retreived cached IPC ctx for\n'
|
f'Retreived cached IPC ctx for\n'
|
||||||
f'peer: {chan.aid.uid}\n'
|
f'peer: {chan.uid}\n'
|
||||||
f'cid:{cid}\n'
|
f'cid:{cid}\n'
|
||||||
)
|
)
|
||||||
ctx._allow_overruns: bool = allow_overruns
|
ctx._allow_overruns: bool = allow_overruns
|
||||||
|
|
@ -718,7 +718,7 @@ class Actor:
|
||||||
except KeyError:
|
except KeyError:
|
||||||
log.debug(
|
log.debug(
|
||||||
f'Allocate new IPC ctx for\n'
|
f'Allocate new IPC ctx for\n'
|
||||||
f'peer: {chan.aid.uid}\n'
|
f'peer: {chan.uid}\n'
|
||||||
f'cid: {cid}\n'
|
f'cid: {cid}\n'
|
||||||
)
|
)
|
||||||
ctx = mk_context(
|
ctx = mk_context(
|
||||||
|
|
@ -764,7 +764,7 @@ class Actor:
|
||||||
|
|
||||||
'''
|
'''
|
||||||
cid: str = str(uuid.uuid4())
|
cid: str = str(uuid.uuid4())
|
||||||
assert chan.aid.uid
|
assert chan.uid
|
||||||
ctx = self.get_context(
|
ctx = self.get_context(
|
||||||
chan=chan,
|
chan=chan,
|
||||||
cid=cid,
|
cid=cid,
|
||||||
|
|
@ -791,12 +791,12 @@ class Actor:
|
||||||
ns=ns,
|
ns=ns,
|
||||||
func=func,
|
func=func,
|
||||||
kwargs=kwargs,
|
kwargs=kwargs,
|
||||||
uid=self.aid.uid, # <- !TODO use .aid!
|
uid=self.uid,
|
||||||
cid=cid,
|
cid=cid,
|
||||||
)
|
)
|
||||||
log.runtime(
|
log.runtime(
|
||||||
'Sending RPC `Start`\n\n'
|
'Sending RPC `Start`\n\n'
|
||||||
f'=> peer: {chan.aid.uid}\n'
|
f'=> peer: {chan.uid}\n'
|
||||||
f' |_ {ns}.{func}({kwargs})\n\n'
|
f' |_ {ns}.{func}({kwargs})\n\n'
|
||||||
|
|
||||||
f'{pretty_struct.pformat(msg)}'
|
f'{pretty_struct.pformat(msg)}'
|
||||||
|
|
@ -1252,7 +1252,7 @@ class Actor:
|
||||||
'Cancel request for invalid RPC task.\n'
|
'Cancel request for invalid RPC task.\n'
|
||||||
'The task likely already completed or was never started!\n\n'
|
'The task likely already completed or was never started!\n\n'
|
||||||
f'<= canceller: {requesting_aid}\n'
|
f'<= canceller: {requesting_aid}\n'
|
||||||
f'=> {cid}@{parent_chan.aid.uid}\n'
|
f'=> {cid}@{parent_chan.uid}\n'
|
||||||
f' |_{parent_chan}\n'
|
f' |_{parent_chan}\n'
|
||||||
)
|
)
|
||||||
return True
|
return True
|
||||||
|
|
@ -1389,7 +1389,7 @@ class Actor:
|
||||||
f'Cancelling {descr} RPC tasks\n\n'
|
f'Cancelling {descr} RPC tasks\n\n'
|
||||||
f'<=c) {req_aid} [canceller]\n'
|
f'<=c) {req_aid} [canceller]\n'
|
||||||
f'{rent_chan_repr}'
|
f'{rent_chan_repr}'
|
||||||
f'c)=> {self.aid.uid} [cancellee]\n'
|
f'c)=> {self.uid} [cancellee]\n'
|
||||||
f' |_{self} [with {len(tasks)} tasks]\n'
|
f' |_{self} [with {len(tasks)} tasks]\n'
|
||||||
# f' |_tasks: {len(tasks)}\n'
|
# f' |_tasks: {len(tasks)}\n'
|
||||||
# f'{tasks_str}'
|
# f'{tasks_str}'
|
||||||
|
|
@ -1695,7 +1695,7 @@ async def async_main(
|
||||||
await reg_portal.run_from_ns(
|
await reg_portal.run_from_ns(
|
||||||
'self',
|
'self',
|
||||||
'register_actor',
|
'register_actor',
|
||||||
uid=actor.aid.uid,
|
uid=actor.uid,
|
||||||
addr=accept_addr.unwrap(),
|
addr=accept_addr.unwrap(),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1766,11 +1766,9 @@ async def async_main(
|
||||||
# always!
|
# always!
|
||||||
match internal_err:
|
match internal_err:
|
||||||
case ContextCancelled():
|
case ContextCancelled():
|
||||||
reprol: str = actor.aid.reprol()
|
|
||||||
log.cancel(
|
log.cancel(
|
||||||
f'Actor {reprol!r} was task-ctx-cancelled with,\n'
|
f'Actor: {actor.uid} was task-context-cancelled with,\n'
|
||||||
f'\n'
|
f'str(internal_err)'
|
||||||
f'{internal_err!r}'
|
|
||||||
)
|
)
|
||||||
case _:
|
case _:
|
||||||
log.exception(
|
log.exception(
|
||||||
|
|
@ -1842,7 +1840,7 @@ async def async_main(
|
||||||
await reg_portal.run_from_ns(
|
await reg_portal.run_from_ns(
|
||||||
'self',
|
'self',
|
||||||
'unregister_actor',
|
'unregister_actor',
|
||||||
uid=actor.aid.uid,
|
uid=actor.uid
|
||||||
)
|
)
|
||||||
except OSError:
|
except OSError:
|
||||||
failed = True
|
failed = True
|
||||||
|
|
|
||||||
|
|
@ -151,7 +151,7 @@ async def exhaust_portal(
|
||||||
__tracebackhide__ = True
|
__tracebackhide__ = True
|
||||||
try:
|
try:
|
||||||
log.debug(
|
log.debug(
|
||||||
f'Waiting on final result from {actor.aid.uid}'
|
f'Waiting on final result from {actor.uid}'
|
||||||
)
|
)
|
||||||
|
|
||||||
# XXX: streams should never be reaped here since they should
|
# XXX: streams should never be reaped here since they should
|
||||||
|
|
@ -210,17 +210,17 @@ async def cancel_on_completion(
|
||||||
actor,
|
actor,
|
||||||
)
|
)
|
||||||
if isinstance(result, Exception):
|
if isinstance(result, Exception):
|
||||||
errors[actor.aid.uid]: Exception = result
|
errors[actor.uid]: Exception = result
|
||||||
log.cancel(
|
log.cancel(
|
||||||
'Cancelling subactor runtime due to error:\n\n'
|
'Cancelling subactor runtime due to error:\n\n'
|
||||||
f'Portal.cancel_actor() => {portal.channel.aid}\n\n'
|
f'Portal.cancel_actor() => {portal.channel.uid}\n\n'
|
||||||
f'error: {result}\n'
|
f'error: {result}\n'
|
||||||
)
|
)
|
||||||
|
|
||||||
else:
|
else:
|
||||||
log.runtime(
|
log.runtime(
|
||||||
'Cancelling subactor gracefully:\n\n'
|
'Cancelling subactor gracefully:\n\n'
|
||||||
f'Portal.cancel_actor() => {portal.channel.aid}\n\n'
|
f'Portal.cancel_actor() => {portal.channel.uid}\n\n'
|
||||||
f'result: {result}\n'
|
f'result: {result}\n'
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -308,7 +308,7 @@ async def hard_kill(
|
||||||
# )
|
# )
|
||||||
# with trio.CancelScope(shield=True):
|
# with trio.CancelScope(shield=True):
|
||||||
# async with debug.acquire_debug_lock(
|
# async with debug.acquire_debug_lock(
|
||||||
# subactor_uid=current_actor().aid.uid,
|
# subactor_uid=current_actor().uid,
|
||||||
# ) as _ctx:
|
# ) as _ctx:
|
||||||
# log.warning(
|
# log.warning(
|
||||||
# 'Acquired debug lock, child ready to be killed ??\n'
|
# 'Acquired debug lock, child ready to be killed ??\n'
|
||||||
|
|
@ -483,7 +483,7 @@ async def trio_proc(
|
||||||
# TODO, how to pass this over "wire" encodings like
|
# TODO, how to pass this over "wire" encodings like
|
||||||
# cmdline args?
|
# cmdline args?
|
||||||
# -[ ] maybe we can add an `msgtypes.Aid.min_tuple()` ?
|
# -[ ] maybe we can add an `msgtypes.Aid.min_tuple()` ?
|
||||||
str(subactor.aid.uid),
|
str(subactor.uid),
|
||||||
# Address the child must connect to on startup
|
# Address the child must connect to on startup
|
||||||
"--parent_addr",
|
"--parent_addr",
|
||||||
str(parent_addr)
|
str(parent_addr)
|
||||||
|
|
@ -514,7 +514,7 @@ async def trio_proc(
|
||||||
# channel should have handshake completed by the
|
# channel should have handshake completed by the
|
||||||
# local actor by the time we get a ref to it
|
# local actor by the time we get a ref to it
|
||||||
event, chan = await ipc_server.wait_for_peer(
|
event, chan = await ipc_server.wait_for_peer(
|
||||||
subactor.aid.uid
|
subactor.uid
|
||||||
)
|
)
|
||||||
|
|
||||||
except trio.Cancelled:
|
except trio.Cancelled:
|
||||||
|
|
@ -528,9 +528,7 @@ async def trio_proc(
|
||||||
await debug.maybe_wait_for_debugger()
|
await debug.maybe_wait_for_debugger()
|
||||||
|
|
||||||
elif proc is not None:
|
elif proc is not None:
|
||||||
async with debug.acquire_debug_lock(
|
async with debug.acquire_debug_lock(subactor.uid):
|
||||||
subactor_uid=subactor.aid.uid
|
|
||||||
):
|
|
||||||
# soft wait on the proc to terminate
|
# soft wait on the proc to terminate
|
||||||
with trio.move_on_after(0.5):
|
with trio.move_on_after(0.5):
|
||||||
await proc.wait()
|
await proc.wait()
|
||||||
|
|
@ -540,7 +538,7 @@ async def trio_proc(
|
||||||
assert proc
|
assert proc
|
||||||
|
|
||||||
portal = Portal(chan)
|
portal = Portal(chan)
|
||||||
actor_nursery._children[subactor.aid.uid] = (
|
actor_nursery._children[subactor.uid] = (
|
||||||
subactor,
|
subactor,
|
||||||
proc,
|
proc,
|
||||||
portal,
|
portal,
|
||||||
|
|
@ -565,7 +563,7 @@ async def trio_proc(
|
||||||
|
|
||||||
# track subactor in current nursery
|
# track subactor in current nursery
|
||||||
curr_actor: Actor = current_actor()
|
curr_actor: Actor = current_actor()
|
||||||
curr_actor._actoruid2nursery[subactor.aid.uid] = actor_nursery
|
curr_actor._actoruid2nursery[subactor.uid] = actor_nursery
|
||||||
|
|
||||||
# resume caller at next checkpoint now that child is up
|
# resume caller at next checkpoint now that child is up
|
||||||
task_status.started(portal)
|
task_status.started(portal)
|
||||||
|
|
@ -618,9 +616,7 @@ async def trio_proc(
|
||||||
# don't clobber an ongoing pdb
|
# don't clobber an ongoing pdb
|
||||||
if cancelled_during_spawn:
|
if cancelled_during_spawn:
|
||||||
# Try again to avoid TTY clobbering.
|
# Try again to avoid TTY clobbering.
|
||||||
async with debug.acquire_debug_lock(
|
async with debug.acquire_debug_lock(subactor.uid):
|
||||||
subactor_uid=subactor.aid.uid
|
|
||||||
):
|
|
||||||
with trio.move_on_after(0.5):
|
with trio.move_on_after(0.5):
|
||||||
await proc.wait()
|
await proc.wait()
|
||||||
|
|
||||||
|
|
@ -666,7 +662,7 @@ async def trio_proc(
|
||||||
if not cancelled_during_spawn:
|
if not cancelled_during_spawn:
|
||||||
# pop child entry to indicate we no longer managing this
|
# pop child entry to indicate we no longer managing this
|
||||||
# subactor
|
# subactor
|
||||||
actor_nursery._children.pop(subactor.aid.uid)
|
actor_nursery._children.pop(subactor.uid)
|
||||||
|
|
||||||
|
|
||||||
async def mp_proc(
|
async def mp_proc(
|
||||||
|
|
@ -748,7 +744,7 @@ async def mp_proc(
|
||||||
# register the process before start in case we get a cancel
|
# register the process before start in case we get a cancel
|
||||||
# request before the actor has fully spawned - then we can wait
|
# request before the actor has fully spawned - then we can wait
|
||||||
# for it to fully come up before sending a cancel request
|
# for it to fully come up before sending a cancel request
|
||||||
actor_nursery._children[subactor.aid.uid] = (subactor, proc, None)
|
actor_nursery._children[subactor.uid] = (subactor, proc, None)
|
||||||
|
|
||||||
proc.start()
|
proc.start()
|
||||||
if not proc.is_alive():
|
if not proc.is_alive():
|
||||||
|
|
@ -762,7 +758,7 @@ async def mp_proc(
|
||||||
# channel should have handshake completed by the
|
# channel should have handshake completed by the
|
||||||
# local actor by the time we get a ref to it
|
# local actor by the time we get a ref to it
|
||||||
event, chan = await ipc_server.wait_for_peer(
|
event, chan = await ipc_server.wait_for_peer(
|
||||||
subactor.aid.uid,
|
subactor.uid,
|
||||||
)
|
)
|
||||||
|
|
||||||
# XXX: monkey patch poll API to match the ``subprocess`` API..
|
# XXX: monkey patch poll API to match the ``subprocess`` API..
|
||||||
|
|
@ -775,7 +771,7 @@ async def mp_proc(
|
||||||
# any process we may have started.
|
# any process we may have started.
|
||||||
|
|
||||||
portal = Portal(chan)
|
portal = Portal(chan)
|
||||||
actor_nursery._children[subactor.aid.uid] = (subactor, proc, portal)
|
actor_nursery._children[subactor.uid] = (subactor, proc, portal)
|
||||||
|
|
||||||
# unblock parent task
|
# unblock parent task
|
||||||
task_status.started(portal)
|
task_status.started(portal)
|
||||||
|
|
@ -814,7 +810,7 @@ async def mp_proc(
|
||||||
# tandem if not done already
|
# tandem if not done already
|
||||||
log.warning(
|
log.warning(
|
||||||
"Cancelling existing result waiter task for "
|
"Cancelling existing result waiter task for "
|
||||||
f"{subactor.aid.uid}")
|
f"{subactor.uid}")
|
||||||
nursery.cancel_scope.cancel()
|
nursery.cancel_scope.cancel()
|
||||||
|
|
||||||
finally:
|
finally:
|
||||||
|
|
@ -832,7 +828,7 @@ async def mp_proc(
|
||||||
log.debug(f"Joined {proc}")
|
log.debug(f"Joined {proc}")
|
||||||
|
|
||||||
# pop child entry to indicate we are no longer managing subactor
|
# pop child entry to indicate we are no longer managing subactor
|
||||||
actor_nursery._children.pop(subactor.aid.uid)
|
actor_nursery._children.pop(subactor.uid)
|
||||||
|
|
||||||
# TODO: prolly report to ``mypy`` how this causes all sorts of
|
# TODO: prolly report to ``mypy`` how this causes all sorts of
|
||||||
# false errors..
|
# false errors..
|
||||||
|
|
|
||||||
|
|
@ -171,7 +171,7 @@ def current_actor(
|
||||||
msg += (
|
msg += (
|
||||||
f'Apparently the lact active actor was\n'
|
f'Apparently the lact active actor was\n'
|
||||||
f'|_{last}\n'
|
f'|_{last}\n'
|
||||||
f'|_{last.aid.uid}\n'
|
f'|_{last.uid}\n'
|
||||||
)
|
)
|
||||||
# no actor runtime has (as of yet) ever been started for
|
# no actor runtime has (as of yet) ever been started for
|
||||||
# this process.
|
# this process.
|
||||||
|
|
@ -242,51 +242,27 @@ def current_ipc_ctx(
|
||||||
|
|
||||||
def get_rt_dir(
|
def get_rt_dir(
|
||||||
subdir: str|Path|None = None,
|
subdir: str|Path|None = None,
|
||||||
appname: str = 'tractor',
|
|
||||||
) -> Path:
|
) -> Path:
|
||||||
'''
|
'''
|
||||||
Return the user "runtime dir", the file-sys location where most
|
Return the user "runtime dir", the file-sys location where most
|
||||||
userspace apps stick their IPC and cache related system
|
userspace apps stick their IPC and cache related system
|
||||||
util-files.
|
util-files.
|
||||||
|
|
||||||
On linux we use a `${XDG_RUNTIME_DIR}/tractor/` subdir by
|
On linux we take use a `'${XDG_RUNTIME_DIR}/tractor/` subdir by
|
||||||
default, but equivalents are mapped for each platform using
|
default but equivalents are mapped for each platform using
|
||||||
the lovely `platformdirs` lib.
|
the lovely `platformdirs`.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
rt_dir: Path = Path(
|
rt_dir: Path = Path(
|
||||||
platformdirs.user_runtime_dir(
|
platformdirs.user_runtime_dir(
|
||||||
appname=appname,
|
appname='tractor',
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Normalize and validate that `subdir` is a relative path
|
|
||||||
# without any parent-directory ("..") components, to prevent
|
|
||||||
# escaping the runtime directory.
|
|
||||||
if subdir:
|
if subdir:
|
||||||
subdir_path = (
|
rt_dir: Path = rt_dir / subdir
|
||||||
subdir
|
|
||||||
if isinstance(subdir, Path)
|
|
||||||
else Path(subdir)
|
|
||||||
)
|
|
||||||
if subdir_path.is_absolute():
|
|
||||||
raise ValueError(
|
|
||||||
f'`subdir` must be a relative path!\n'
|
|
||||||
f'{subdir!r}\n'
|
|
||||||
)
|
|
||||||
if any(part == '..' for part in subdir_path.parts):
|
|
||||||
raise ValueError(
|
|
||||||
"`subdir` must not contain '..' components!\n"
|
|
||||||
f'{subdir!r}\n'
|
|
||||||
)
|
|
||||||
|
|
||||||
rt_dir: Path = rt_dir / subdir_path
|
|
||||||
|
|
||||||
if not rt_dir.is_dir():
|
if not rt_dir.is_dir():
|
||||||
rt_dir.mkdir(
|
rt_dir.mkdir()
|
||||||
parents=True,
|
|
||||||
exist_ok=True, # avoid `FileExistsError` from conc calls
|
|
||||||
)
|
|
||||||
|
|
||||||
return rt_dir
|
return rt_dir
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -391,17 +391,15 @@ class ActorNursery:
|
||||||
|
|
||||||
else:
|
else:
|
||||||
if portal is None: # actor hasn't fully spawned yet
|
if portal is None: # actor hasn't fully spawned yet
|
||||||
event: trio.Event = server._peer_connected[
|
event: trio.Event = server._peer_connected[subactor.uid]
|
||||||
subactor.aid.uid
|
|
||||||
]
|
|
||||||
log.warning(
|
log.warning(
|
||||||
f"{subactor.aid.uid} never 't finished spawning?"
|
f"{subactor.uid} never 't finished spawning?"
|
||||||
)
|
)
|
||||||
|
|
||||||
await event.wait()
|
await event.wait()
|
||||||
|
|
||||||
# channel/portal should now be up
|
# channel/portal should now be up
|
||||||
_, _, portal = children[subactor.aid.uid]
|
_, _, portal = children[subactor.uid]
|
||||||
|
|
||||||
# XXX should be impossible to get here
|
# XXX should be impossible to get here
|
||||||
# unless method was called from within
|
# unless method was called from within
|
||||||
|
|
@ -409,7 +407,7 @@ class ActorNursery:
|
||||||
if portal is None:
|
if portal is None:
|
||||||
# cancelled while waiting on the event
|
# cancelled while waiting on the event
|
||||||
# to arrive
|
# to arrive
|
||||||
chan = server._peers[subactor.aid.uid][-1]
|
chan = server._peers[subactor.uid][-1]
|
||||||
if chan:
|
if chan:
|
||||||
portal = Portal(chan)
|
portal = Portal(chan)
|
||||||
else: # there's no other choice left
|
else: # there's no other choice left
|
||||||
|
|
@ -508,7 +506,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
|
||||||
|
|
||||||
except BaseException as _inner_err:
|
except BaseException as _inner_err:
|
||||||
inner_err = _inner_err
|
inner_err = _inner_err
|
||||||
errors[actor.aid.uid] = inner_err
|
errors[actor.uid] = inner_err
|
||||||
|
|
||||||
# If we error in the root but the debugger is
|
# If we error in the root but the debugger is
|
||||||
# engaged we don't want to prematurely kill (and
|
# engaged we don't want to prematurely kill (and
|
||||||
|
|
@ -541,7 +539,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
|
||||||
log.cancel(
|
log.cancel(
|
||||||
f'Actor-nursery cancelled by {etype}\n\n'
|
f'Actor-nursery cancelled by {etype}\n\n'
|
||||||
|
|
||||||
f'{current_actor().aid.uid}\n'
|
f'{current_actor().uid}\n'
|
||||||
f' |_{an}\n\n'
|
f' |_{an}\n\n'
|
||||||
|
|
||||||
# TODO: show tb str?
|
# TODO: show tb str?
|
||||||
|
|
@ -632,7 +630,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
|
||||||
|
|
||||||
# show frame on any (likely) internal error
|
# show frame on any (likely) internal error
|
||||||
if (
|
if (
|
||||||
not an.cancel_called
|
not an.cancelled
|
||||||
and an._scope_error
|
and an._scope_error
|
||||||
):
|
):
|
||||||
__tracebackhide__: bool = False
|
__tracebackhide__: bool = False
|
||||||
|
|
@ -728,7 +726,7 @@ async def open_nursery(
|
||||||
if (
|
if (
|
||||||
an
|
an
|
||||||
and
|
and
|
||||||
not an.cancel_called
|
not an.cancelled
|
||||||
and
|
and
|
||||||
an._scope_error
|
an._scope_error
|
||||||
):
|
):
|
||||||
|
|
|
||||||
|
|
@ -61,11 +61,7 @@ def get_rando_addr(
|
||||||
# NOTE, file-name uniqueness (no-collisions) will be based on
|
# NOTE, file-name uniqueness (no-collisions) will be based on
|
||||||
# the runtime-directory and root (pytest-proc's) pid.
|
# the runtime-directory and root (pytest-proc's) pid.
|
||||||
case 'uds':
|
case 'uds':
|
||||||
from tractor.ipc._uds import UDSAddress
|
testrun_reg_addr = addr_type.get_random().unwrap()
|
||||||
addr: UDSAddress = addr_type.get_random()
|
|
||||||
assert addr.is_valid
|
|
||||||
assert addr.sockpath.resolve()
|
|
||||||
testrun_reg_addr = addr.unwrap()
|
|
||||||
|
|
||||||
# XXX, as sanity it should never the same as the default for the
|
# XXX, as sanity it should never the same as the default for the
|
||||||
# host-singleton registry actor.
|
# host-singleton registry actor.
|
||||||
|
|
|
||||||
|
|
@ -74,7 +74,6 @@ def tractor_test(fn):
|
||||||
reg_addr=None,
|
reg_addr=None,
|
||||||
start_method: str|None = None,
|
start_method: str|None = None,
|
||||||
debug_mode: bool = False,
|
debug_mode: bool = False,
|
||||||
tpt_proto: str|None=None,
|
|
||||||
**kwargs
|
**kwargs
|
||||||
):
|
):
|
||||||
# __tracebackhide__ = True
|
# __tracebackhide__ = True
|
||||||
|
|
@ -103,9 +102,6 @@ def tractor_test(fn):
|
||||||
# set of subprocess spawning backends
|
# set of subprocess spawning backends
|
||||||
kwargs['debug_mode'] = debug_mode
|
kwargs['debug_mode'] = debug_mode
|
||||||
|
|
||||||
if 'tpt_proto' in inspect.signature(fn).parameters:
|
|
||||||
# set of subprocess spawning backends
|
|
||||||
kwargs['tpt_proto'] = tpt_proto
|
|
||||||
|
|
||||||
if kwargs:
|
if kwargs:
|
||||||
|
|
||||||
|
|
@ -181,13 +177,6 @@ def pytest_configure(config):
|
||||||
backend = config.option.spawn_backend
|
backend = config.option.spawn_backend
|
||||||
tractor._spawn.try_set_start_method(backend)
|
tractor._spawn.try_set_start_method(backend)
|
||||||
|
|
||||||
# register custom marks to avoid warnings see,
|
|
||||||
# https://docs.pytest.org/en/stable/how-to/writing_plugins.html#registering-custom-markers
|
|
||||||
config.addinivalue_line(
|
|
||||||
'markers',
|
|
||||||
'no_tpt(proto_key): test will (likely) not behave with tpt backend'
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope='session')
|
@pytest.fixture(scope='session')
|
||||||
def debug_mode(request) -> bool:
|
def debug_mode(request) -> bool:
|
||||||
|
|
@ -236,32 +225,13 @@ def tpt_protos(request) -> list[str]:
|
||||||
autouse=True,
|
autouse=True,
|
||||||
)
|
)
|
||||||
def tpt_proto(
|
def tpt_proto(
|
||||||
request,
|
|
||||||
tpt_protos: list[str],
|
tpt_protos: list[str],
|
||||||
) -> str:
|
) -> str:
|
||||||
proto_key: str = tpt_protos[0]
|
proto_key: str = tpt_protos[0]
|
||||||
|
|
||||||
# ?TODO, but needs a fn-scoped tpt_proto fixture..
|
|
||||||
# @pytest.mark.no_tpt('uds')
|
|
||||||
# node = request.node
|
|
||||||
# markers = node.own_markers
|
|
||||||
# for mark in markers:
|
|
||||||
# if (
|
|
||||||
# mark.name == 'no_tpt'
|
|
||||||
# and
|
|
||||||
# proto_key in mark.args
|
|
||||||
# ):
|
|
||||||
# pytest.skip(
|
|
||||||
# f'Test {node} normally fails with '
|
|
||||||
# f'tpt-proto={proto_key!r}\n'
|
|
||||||
# )
|
|
||||||
|
|
||||||
from tractor import _state
|
from tractor import _state
|
||||||
if _state._def_tpt_proto != proto_key:
|
if _state._def_tpt_proto != proto_key:
|
||||||
_state._def_tpt_proto = proto_key
|
_state._def_tpt_proto = proto_key
|
||||||
_state._runtime_vars['_enable_tpts'] = [
|
|
||||||
proto_key,
|
|
||||||
]
|
|
||||||
|
|
||||||
yield proto_key
|
yield proto_key
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -21,7 +21,6 @@ cancellation during REPL interaction.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
import platform
|
|
||||||
from typing import (
|
from typing import (
|
||||||
TYPE_CHECKING,
|
TYPE_CHECKING,
|
||||||
)
|
)
|
||||||
|
|
@ -50,7 +49,6 @@ if TYPE_CHECKING:
|
||||||
|
|
||||||
log = get_logger()
|
log = get_logger()
|
||||||
|
|
||||||
_is_macos: bool = platform.system() == 'Darwin'
|
|
||||||
_ctlc_ignore_header: str = (
|
_ctlc_ignore_header: str = (
|
||||||
'Ignoring SIGINT while debug REPL in use'
|
'Ignoring SIGINT while debug REPL in use'
|
||||||
)
|
)
|
||||||
|
|
@ -302,11 +300,6 @@ def sigint_shield(
|
||||||
# XXX: yah, mega hack, but how else do we catch this madness XD
|
# XXX: yah, mega hack, but how else do we catch this madness XD
|
||||||
if (
|
if (
|
||||||
repl.shname == 'xonsh'
|
repl.shname == 'xonsh'
|
||||||
or (
|
|
||||||
repl.shname == 'bash'
|
|
||||||
and
|
|
||||||
_is_macos
|
|
||||||
)
|
|
||||||
):
|
):
|
||||||
flush_status += (
|
flush_status += (
|
||||||
'-> ALSO re-flushing due to `xonsh`..\n'
|
'-> ALSO re-flushing due to `xonsh`..\n'
|
||||||
|
|
|
||||||
|
|
@ -94,7 +94,7 @@ class Channel:
|
||||||
self._transport: MsgTransport|None = transport
|
self._transport: MsgTransport|None = transport
|
||||||
|
|
||||||
# set after handshake - always info from peer end
|
# set after handshake - always info from peer end
|
||||||
self._aid: Aid|None = None
|
self.aid: Aid|None = None
|
||||||
|
|
||||||
self._aiter_msgs = self._iter_msgs()
|
self._aiter_msgs = self._iter_msgs()
|
||||||
self._exc: Exception|None = None
|
self._exc: Exception|None = None
|
||||||
|
|
@ -122,14 +122,6 @@ class Channel:
|
||||||
'''
|
'''
|
||||||
return self._cancel_called
|
return self._cancel_called
|
||||||
|
|
||||||
@property
|
|
||||||
def aid(self) -> Aid:
|
|
||||||
'''
|
|
||||||
Peer actor's ID.
|
|
||||||
|
|
||||||
'''
|
|
||||||
return self._aid
|
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def uid(self) -> tuple[str, str]:
|
def uid(self) -> tuple[str, str]:
|
||||||
'''
|
'''
|
||||||
|
|
@ -513,7 +505,7 @@ class Channel:
|
||||||
f'<= {peer_aid.reprol(sin_uuid=False)}\n'
|
f'<= {peer_aid.reprol(sin_uuid=False)}\n'
|
||||||
)
|
)
|
||||||
# NOTE, we always are referencing the remote peer!
|
# NOTE, we always are referencing the remote peer!
|
||||||
self._aid = peer_aid
|
self.aid = peer_aid
|
||||||
return peer_aid
|
return peer_aid
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -23,7 +23,6 @@ considered optional within the context of this runtime-library.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
import hashlib
|
|
||||||
from multiprocessing import shared_memory as shm
|
from multiprocessing import shared_memory as shm
|
||||||
from multiprocessing.shared_memory import (
|
from multiprocessing.shared_memory import (
|
||||||
# SharedMemory,
|
# SharedMemory,
|
||||||
|
|
@ -107,12 +106,11 @@ class NDToken(Struct, frozen=True):
|
||||||
This type is msg safe.
|
This type is msg safe.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
shm_name: str # actual OS-level name (may be shortened on macOS)
|
shm_name: str # this servers as a "key" value
|
||||||
shm_first_index_name: str
|
shm_first_index_name: str
|
||||||
shm_last_index_name: str
|
shm_last_index_name: str
|
||||||
dtype_descr: tuple
|
dtype_descr: tuple
|
||||||
size: int # in struct-array index / row terms
|
size: int # in struct-array index / row terms
|
||||||
key: str|None = None # original descriptive key (for lookup)
|
|
||||||
|
|
||||||
# TODO: use nptyping here on dtypes
|
# TODO: use nptyping here on dtypes
|
||||||
@property
|
@property
|
||||||
|
|
@ -126,41 +124,6 @@ class NDToken(Struct, frozen=True):
|
||||||
def as_msg(self):
|
def as_msg(self):
|
||||||
return to_builtins(self)
|
return to_builtins(self)
|
||||||
|
|
||||||
def __eq__(self, other) -> bool:
|
|
||||||
'''
|
|
||||||
Compare tokens based on shm names and dtype,
|
|
||||||
ignoring the `key` field.
|
|
||||||
|
|
||||||
The `key` field is only used for lookups,
|
|
||||||
not for token identity.
|
|
||||||
|
|
||||||
'''
|
|
||||||
if not isinstance(other, NDToken):
|
|
||||||
return False
|
|
||||||
return (
|
|
||||||
self.shm_name == other.shm_name
|
|
||||||
and self.shm_first_index_name
|
|
||||||
== other.shm_first_index_name
|
|
||||||
and self.shm_last_index_name
|
|
||||||
== other.shm_last_index_name
|
|
||||||
and self.dtype_descr == other.dtype_descr
|
|
||||||
and self.size == other.size
|
|
||||||
)
|
|
||||||
|
|
||||||
def __hash__(self) -> int:
|
|
||||||
'''
|
|
||||||
Hash based on the same fields used
|
|
||||||
in `.__eq__()`.
|
|
||||||
|
|
||||||
'''
|
|
||||||
return hash((
|
|
||||||
self.shm_name,
|
|
||||||
self.shm_first_index_name,
|
|
||||||
self.shm_last_index_name,
|
|
||||||
self.dtype_descr,
|
|
||||||
self.size,
|
|
||||||
))
|
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_msg(cls, msg: dict) -> NDToken:
|
def from_msg(cls, msg: dict) -> NDToken:
|
||||||
if isinstance(msg, NDToken):
|
if isinstance(msg, NDToken):
|
||||||
|
|
@ -197,50 +160,6 @@ def get_shm_token(key: str) -> NDToken | None:
|
||||||
return _known_tokens.get(key)
|
return _known_tokens.get(key)
|
||||||
|
|
||||||
|
|
||||||
def _shorten_key_for_macos(
|
|
||||||
key: str,
|
|
||||||
prefix: str = '',
|
|
||||||
suffix: str = '',
|
|
||||||
) -> str:
|
|
||||||
'''
|
|
||||||
MacOS has a (hillarious) 31 character limit for POSIX shared
|
|
||||||
memory names. Hash long keys to fit within this limit while
|
|
||||||
maintaining uniqueness.
|
|
||||||
|
|
||||||
'''
|
|
||||||
# macOS shm_open() has a 31 char limit (PSHMNAMLEN)
|
|
||||||
# format: /t_<hash16> = 19 chars, well under limit
|
|
||||||
max_len: int = 31
|
|
||||||
if len(key) <= max_len:
|
|
||||||
return key
|
|
||||||
|
|
||||||
_hash: str = hashlib.sha256(
|
|
||||||
key.encode()
|
|
||||||
).hexdigest()
|
|
||||||
|
|
||||||
hash_len: int = (
|
|
||||||
(max_len - 1)
|
|
||||||
- len(prefix)
|
|
||||||
- len(suffix)
|
|
||||||
)
|
|
||||||
key_hash: str = _hash[:hash_len]
|
|
||||||
short_key = (
|
|
||||||
prefix
|
|
||||||
+
|
|
||||||
f'{key_hash}'
|
|
||||||
+
|
|
||||||
suffix
|
|
||||||
)
|
|
||||||
|
|
||||||
log.debug(
|
|
||||||
f'Shortened shm key for macOS:\n'
|
|
||||||
f' original: {key!r} ({len(key)!r} chars)\n'
|
|
||||||
f' shortened: {short_key!r}'
|
|
||||||
f' ({len(short_key)!r} chars)'
|
|
||||||
)
|
|
||||||
return short_key
|
|
||||||
|
|
||||||
|
|
||||||
def _make_token(
|
def _make_token(
|
||||||
key: str,
|
key: str,
|
||||||
size: int,
|
size: int,
|
||||||
|
|
@ -252,32 +171,12 @@ def _make_token(
|
||||||
to access a shared array.
|
to access a shared array.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
# On macOS, shorten keys that exceed the
|
|
||||||
# 31 character limit
|
|
||||||
if platform.system() == 'Darwin':
|
|
||||||
shm_name = _shorten_key_for_macos(
|
|
||||||
key=key,
|
|
||||||
)
|
|
||||||
shm_first = _shorten_key_for_macos(
|
|
||||||
key=key,
|
|
||||||
suffix='_first',
|
|
||||||
)
|
|
||||||
shm_last = _shorten_key_for_macos(
|
|
||||||
key=key,
|
|
||||||
suffix='_last',
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
shm_name = key
|
|
||||||
shm_first = key + '_first'
|
|
||||||
shm_last = key + '_last'
|
|
||||||
|
|
||||||
return NDToken(
|
return NDToken(
|
||||||
shm_name=shm_name,
|
shm_name=key,
|
||||||
shm_first_index_name=shm_first,
|
shm_first_index_name=key + "_first",
|
||||||
shm_last_index_name=shm_last,
|
shm_last_index_name=key + "_last",
|
||||||
dtype_descr=tuple(np.dtype(dtype).descr),
|
dtype_descr=tuple(np.dtype(dtype).descr),
|
||||||
size=size,
|
size=size,
|
||||||
key=key, # store original key for lookup
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -532,17 +431,9 @@ class ShmArray:
|
||||||
|
|
||||||
def destroy(self) -> None:
|
def destroy(self) -> None:
|
||||||
if _USE_POSIX:
|
if _USE_POSIX:
|
||||||
# We manually unlink to bypass all the
|
# We manually unlink to bypass all the "resource tracker"
|
||||||
# "resource tracker" nonsense meant for
|
# nonsense meant for non-SC systems.
|
||||||
# non-SC systems.
|
shm_unlink(self._shm.name)
|
||||||
name = self._shm.name
|
|
||||||
try:
|
|
||||||
shm_unlink(name)
|
|
||||||
except FileNotFoundError:
|
|
||||||
# might be a teardown race here?
|
|
||||||
log.warning(
|
|
||||||
f'Shm for {name} already unlinked?'
|
|
||||||
)
|
|
||||||
|
|
||||||
self._first.destroy()
|
self._first.destroy()
|
||||||
self._last.destroy()
|
self._last.destroy()
|
||||||
|
|
@ -572,16 +463,8 @@ def open_shm_ndarray(
|
||||||
a = np.zeros(size, dtype=dtype)
|
a = np.zeros(size, dtype=dtype)
|
||||||
a['index'] = np.arange(len(a))
|
a['index'] = np.arange(len(a))
|
||||||
|
|
||||||
# Create token first to get the (possibly
|
|
||||||
# shortened) shm name
|
|
||||||
token = _make_token(
|
|
||||||
key=key,
|
|
||||||
size=size,
|
|
||||||
dtype=dtype,
|
|
||||||
)
|
|
||||||
|
|
||||||
shm = SharedMemory(
|
shm = SharedMemory(
|
||||||
name=token.shm_name,
|
name=key,
|
||||||
create=True,
|
create=True,
|
||||||
size=a.nbytes
|
size=a.nbytes
|
||||||
)
|
)
|
||||||
|
|
@ -593,6 +476,12 @@ def open_shm_ndarray(
|
||||||
array[:] = a[:]
|
array[:] = a[:]
|
||||||
array.setflags(write=int(not readonly))
|
array.setflags(write=int(not readonly))
|
||||||
|
|
||||||
|
token = _make_token(
|
||||||
|
key=key,
|
||||||
|
size=size,
|
||||||
|
dtype=dtype,
|
||||||
|
)
|
||||||
|
|
||||||
# create single entry arrays for storing an first and last indices
|
# create single entry arrays for storing an first and last indices
|
||||||
first = SharedInt(
|
first = SharedInt(
|
||||||
shm=SharedMemory(
|
shm=SharedMemory(
|
||||||
|
|
@ -665,23 +554,13 @@ def attach_shm_ndarray(
|
||||||
|
|
||||||
'''
|
'''
|
||||||
token = NDToken.from_msg(token)
|
token = NDToken.from_msg(token)
|
||||||
# Use original key for _known_tokens lookup,
|
key = token.shm_name
|
||||||
# shm_name for OS calls
|
|
||||||
lookup_key = (
|
|
||||||
token.key if token.key
|
|
||||||
else token.shm_name
|
|
||||||
)
|
|
||||||
|
|
||||||
if lookup_key in _known_tokens:
|
if key in _known_tokens:
|
||||||
assert (
|
assert NDToken.from_msg(_known_tokens[key]) == token, "WTF"
|
||||||
NDToken.from_msg(
|
|
||||||
_known_tokens[lookup_key]
|
|
||||||
) == token
|
|
||||||
), 'WTF'
|
|
||||||
|
|
||||||
# XXX: ugh, looks like due to the ``shm_open()``
|
# XXX: ugh, looks like due to the ``shm_open()`` C api we can't
|
||||||
# C api we can't actually place files in a subdir,
|
# actually place files in a subdir, see discussion here:
|
||||||
# see discussion here:
|
|
||||||
# https://stackoverflow.com/a/11103289
|
# https://stackoverflow.com/a/11103289
|
||||||
|
|
||||||
# attach to array buffer and view as per dtype
|
# attach to array buffer and view as per dtype
|
||||||
|
|
@ -689,7 +568,7 @@ def attach_shm_ndarray(
|
||||||
for _ in range(3):
|
for _ in range(3):
|
||||||
try:
|
try:
|
||||||
shm = SharedMemory(
|
shm = SharedMemory(
|
||||||
name=token.shm_name,
|
name=key,
|
||||||
create=False,
|
create=False,
|
||||||
)
|
)
|
||||||
break
|
break
|
||||||
|
|
@ -735,10 +614,10 @@ def attach_shm_ndarray(
|
||||||
sha.array
|
sha.array
|
||||||
|
|
||||||
# Stash key -> token knowledge for future queries
|
# Stash key -> token knowledge for future queries
|
||||||
# via `maybe_open_shm_ndarray()` but only after
|
# via `maybe_opepn_shm_array()` but only after we know
|
||||||
# we know we can attach.
|
# we can attach.
|
||||||
if lookup_key not in _known_tokens:
|
if key not in _known_tokens:
|
||||||
_known_tokens[lookup_key] = token
|
_known_tokens[key] = token
|
||||||
|
|
||||||
# "close" attached shm on actor teardown
|
# "close" attached shm on actor teardown
|
||||||
tractor.current_actor().lifetime_stack.callback(sha.close)
|
tractor.current_actor().lifetime_stack.callback(sha.close)
|
||||||
|
|
@ -782,10 +661,7 @@ def maybe_open_shm_ndarray(
|
||||||
False, # not newly opened
|
False, # not newly opened
|
||||||
)
|
)
|
||||||
except KeyError:
|
except KeyError:
|
||||||
log.warning(
|
log.warning(f"Could not find {key} in shms cache")
|
||||||
f'Could not find key in shms cache,\n'
|
|
||||||
f'key: {key!r}\n'
|
|
||||||
)
|
|
||||||
if dtype:
|
if dtype:
|
||||||
token = _make_token(
|
token = _make_token(
|
||||||
key,
|
key,
|
||||||
|
|
@ -895,7 +771,6 @@ def open_shm_list(
|
||||||
size: int = int(2 ** 10),
|
size: int = int(2 ** 10),
|
||||||
dtype: float | int | bool | str | bytes | None = float,
|
dtype: float | int | bool | str | bytes | None = float,
|
||||||
readonly: bool = True,
|
readonly: bool = True,
|
||||||
prefix: str = 'shml_',
|
|
||||||
|
|
||||||
) -> ShmList:
|
) -> ShmList:
|
||||||
|
|
||||||
|
|
@ -909,12 +784,6 @@ def open_shm_list(
|
||||||
}[dtype]
|
}[dtype]
|
||||||
sequence = [default] * size
|
sequence = [default] * size
|
||||||
|
|
||||||
if platform.system() == 'Darwin':
|
|
||||||
key: str = _shorten_key_for_macos(
|
|
||||||
key=key,
|
|
||||||
prefix=prefix,
|
|
||||||
)
|
|
||||||
|
|
||||||
shml = ShmList(
|
shml = ShmList(
|
||||||
sequence=sequence,
|
sequence=sequence,
|
||||||
name=key,
|
name=key,
|
||||||
|
|
|
||||||
|
|
@ -23,12 +23,12 @@ from contextlib import (
|
||||||
)
|
)
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
import os
|
import os
|
||||||
import sys
|
|
||||||
from socket import (
|
from socket import (
|
||||||
AF_UNIX,
|
AF_UNIX,
|
||||||
SOCK_STREAM,
|
SOCK_STREAM,
|
||||||
|
SO_PASSCRED,
|
||||||
|
SO_PEERCRED,
|
||||||
SOL_SOCKET,
|
SOL_SOCKET,
|
||||||
error as socket_error,
|
|
||||||
)
|
)
|
||||||
import struct
|
import struct
|
||||||
from typing import (
|
from typing import (
|
||||||
|
|
@ -53,7 +53,7 @@ from tractor.log import get_logger
|
||||||
from tractor.ipc._transport import (
|
from tractor.ipc._transport import (
|
||||||
MsgpackTransport,
|
MsgpackTransport,
|
||||||
)
|
)
|
||||||
from tractor._state import (
|
from .._state import (
|
||||||
get_rt_dir,
|
get_rt_dir,
|
||||||
current_actor,
|
current_actor,
|
||||||
is_root_process,
|
is_root_process,
|
||||||
|
|
@ -63,28 +63,6 @@ if TYPE_CHECKING:
|
||||||
from ._runtime import Actor
|
from ._runtime import Actor
|
||||||
|
|
||||||
|
|
||||||
# Platform-specific credential passing constants
|
|
||||||
# See: https://stackoverflow.com/a/7982749
|
|
||||||
if sys.platform == 'linux':
|
|
||||||
from socket import (
|
|
||||||
SO_PASSCRED,
|
|
||||||
SO_PEERCRED,
|
|
||||||
)
|
|
||||||
|
|
||||||
else:
|
|
||||||
# Other (Unix) platforms - though further testing is required and
|
|
||||||
# others may need additional special handling?
|
|
||||||
SO_PASSCRED = None
|
|
||||||
SO_PEERCRED = None
|
|
||||||
|
|
||||||
# NOTE, macOS uses `LOCAL_PEERCRED` instead of `SO_PEERCRED` and
|
|
||||||
# doesn't need `SO_PASSCRED` (credential passing is always enabled).
|
|
||||||
# See code in <sys/un.h>: `#define LOCAL_PEERCRED 0x001`
|
|
||||||
#
|
|
||||||
# XXX INSTEAD we use the (hopefully) more generic
|
|
||||||
# `get_peer_pid()` below for other OSes.
|
|
||||||
|
|
||||||
|
|
||||||
log = get_logger()
|
log = get_logger()
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -187,21 +165,15 @@ class UDSAddress(
|
||||||
err_on_no_runtime=False,
|
err_on_no_runtime=False,
|
||||||
)
|
)
|
||||||
if actor:
|
if actor:
|
||||||
sockname: str = f'{actor.aid.name}@{pid}'
|
sockname: str = '::'.join(actor.uid) + f'@{pid}'
|
||||||
# XXX, orig version which broke both macOS (file-name
|
|
||||||
# length) and `multiaddrs` ('::' invalid separator).
|
|
||||||
# sockname: str = '::'.join(actor.uid) + f'@{pid}'
|
|
||||||
#
|
|
||||||
# ?^TODO, for `multiaddr`'s parser we can't use the `::`
|
# ?^TODO, for `multiaddr`'s parser we can't use the `::`
|
||||||
# above^, SO maybe a `.` or something else here?
|
# above^, SO maybe a `.` or something else here?
|
||||||
# sockname: str = '.'.join(actor.uid) + f'@{pid}'
|
# sockname: str = '.'.join(actor.uid) + f'@{pid}'
|
||||||
# -[ ] CURRENTLY using `.` BREAKS TEST SUITE tho..
|
# -[ ] CURRENTLY using `.` BREAKS TEST SUITE tho..
|
||||||
else:
|
else:
|
||||||
|
prefix: str = '<unknown-actor>'
|
||||||
if is_root_process():
|
if is_root_process():
|
||||||
prefix: str = 'no_runtime_root'
|
prefix: str = 'root'
|
||||||
else:
|
|
||||||
prefix: str = 'no_runtime_actor'
|
|
||||||
|
|
||||||
sockname: str = f'{prefix}@{pid}'
|
sockname: str = f'{prefix}@{pid}'
|
||||||
|
|
||||||
sockpath: Path = Path(f'{sockname}.sock')
|
sockpath: Path = Path(f'{sockname}.sock')
|
||||||
|
|
@ -320,12 +292,7 @@ def close_listener(
|
||||||
|
|
||||||
|
|
||||||
async def open_unix_socket_w_passcred(
|
async def open_unix_socket_w_passcred(
|
||||||
filename: (
|
filename: str|bytes|os.PathLike[str]|os.PathLike[bytes],
|
||||||
str
|
|
||||||
|bytes
|
|
||||||
|os.PathLike[str]
|
|
||||||
|os.PathLike[bytes]
|
|
||||||
),
|
|
||||||
) -> trio.SocketStream:
|
) -> trio.SocketStream:
|
||||||
'''
|
'''
|
||||||
Literally the exact same as `trio.open_unix_socket()` except we set the additiona
|
Literally the exact same as `trio.open_unix_socket()` except we set the additiona
|
||||||
|
|
@ -343,66 +310,21 @@ async def open_unix_socket_w_passcred(
|
||||||
# much more simplified logic vs tcp sockets - one socket type and only one
|
# much more simplified logic vs tcp sockets - one socket type and only one
|
||||||
# possible location to connect to
|
# possible location to connect to
|
||||||
sock = trio.socket.socket(AF_UNIX, SOCK_STREAM)
|
sock = trio.socket.socket(AF_UNIX, SOCK_STREAM)
|
||||||
|
sock.setsockopt(SOL_SOCKET, SO_PASSCRED, 1)
|
||||||
# Only set SO_PASSCRED on Linux (not needed/available on macOS)
|
|
||||||
if SO_PASSCRED is not None:
|
|
||||||
sock.setsockopt(SOL_SOCKET, SO_PASSCRED, 1)
|
|
||||||
|
|
||||||
with close_on_error(sock):
|
with close_on_error(sock):
|
||||||
await sock.connect(os.fspath(filename))
|
await sock.connect(os.fspath(filename))
|
||||||
|
|
||||||
return trio.SocketStream(sock)
|
return trio.SocketStream(sock)
|
||||||
|
|
||||||
|
|
||||||
def get_peer_pid(sock) -> int|None:
|
def get_peer_info(sock: trio.socket.socket) -> tuple[
|
||||||
'''
|
|
||||||
Gets the PID of the process connected to the other end of a Unix
|
|
||||||
domain socket on macOS, or `None` if that fails.
|
|
||||||
|
|
||||||
NOTE, should work on MacOS (and others?).
|
|
||||||
|
|
||||||
'''
|
|
||||||
# try to get the peer PID using a naive soln found from,
|
|
||||||
# https://stackoverflow.com/a/67971484
|
|
||||||
#
|
|
||||||
# NOTE, a more correct soln is likely needed here according to
|
|
||||||
# the complaints of `copilot` which led to digging into the
|
|
||||||
# underlying `go`lang issue linked from the above SO answer,
|
|
||||||
|
|
||||||
# XXX, darwin-xnu kernel srces defining these constants,
|
|
||||||
# - SOL_LOCAL
|
|
||||||
# |_https://github.com/apple/darwin-xnu/blob/main/bsd/sys/un.h#L85
|
|
||||||
# - LOCAL_PEERPID
|
|
||||||
# |_https://github.com/apple/darwin-xnu/blob/main/bsd/sys/un.h#L89
|
|
||||||
#
|
|
||||||
SOL_LOCAL: int = 0
|
|
||||||
LOCAL_PEERPID: int = 0x002
|
|
||||||
|
|
||||||
try:
|
|
||||||
pid: int = sock.getsockopt(
|
|
||||||
SOL_LOCAL,
|
|
||||||
LOCAL_PEERPID,
|
|
||||||
)
|
|
||||||
return pid
|
|
||||||
except socket_error as e:
|
|
||||||
log.exception(
|
|
||||||
f"Failed to get peer PID: {e}"
|
|
||||||
)
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def get_peer_info(
|
|
||||||
sock: trio.socket.socket,
|
|
||||||
) -> tuple[
|
|
||||||
int, # pid
|
int, # pid
|
||||||
int, # uid
|
int, # uid
|
||||||
int, # guid
|
int, # guid
|
||||||
]:
|
]:
|
||||||
'''
|
'''
|
||||||
Deliver the connecting peer's "credentials"-info as defined in
|
Deliver the connecting peer's "credentials"-info as defined in
|
||||||
a platform-specific way.
|
a very Linux specific way..
|
||||||
|
|
||||||
Linux-ONLY, uses SO_PEERCRED.
|
|
||||||
|
|
||||||
For more deats see,
|
For more deats see,
|
||||||
- `man accept`,
|
- `man accept`,
|
||||||
|
|
@ -415,11 +337,6 @@ def get_peer_info(
|
||||||
- https://stackoverflow.com/a/7982749
|
- https://stackoverflow.com/a/7982749
|
||||||
|
|
||||||
'''
|
'''
|
||||||
if SO_PEERCRED is None:
|
|
||||||
raise RuntimeError(
|
|
||||||
f'Peer credential retrieval not supported on {sys.platform}!'
|
|
||||||
)
|
|
||||||
|
|
||||||
creds: bytes = sock.getsockopt(
|
creds: bytes = sock.getsockopt(
|
||||||
SOL_SOCKET,
|
SOL_SOCKET,
|
||||||
SO_PEERCRED,
|
SO_PEERCRED,
|
||||||
|
|
@ -523,37 +440,13 @@ class MsgpackUDSStream(MsgpackTransport):
|
||||||
match (peername, sockname):
|
match (peername, sockname):
|
||||||
case (str(), bytes()):
|
case (str(), bytes()):
|
||||||
sock_path: Path = Path(peername)
|
sock_path: Path = Path(peername)
|
||||||
|
|
||||||
case (bytes(), str()):
|
case (bytes(), str()):
|
||||||
sock_path: Path = Path(sockname)
|
sock_path: Path = Path(sockname)
|
||||||
|
(
|
||||||
case (str(), str()): # XXX, likely macOS
|
peer_pid,
|
||||||
sock_path: Path = Path(peername)
|
_,
|
||||||
|
_,
|
||||||
case _:
|
) = get_peer_info(sock)
|
||||||
raise TypeError(
|
|
||||||
f'Failed to match (peername, sockname) types?\n'
|
|
||||||
f'peername: {peername!r}\n'
|
|
||||||
f'sockname: {sockname!r}\n'
|
|
||||||
)
|
|
||||||
|
|
||||||
if sys.platform == 'linux':
|
|
||||||
(
|
|
||||||
peer_pid,
|
|
||||||
_,
|
|
||||||
_,
|
|
||||||
) = get_peer_info(sock)
|
|
||||||
|
|
||||||
# NOTE known to at least works on,
|
|
||||||
# - macos
|
|
||||||
else:
|
|
||||||
peer_pid: int|None = get_peer_pid(sock)
|
|
||||||
if peer_pid is None:
|
|
||||||
log.warning(
|
|
||||||
f'Unable to get peer PID?\n'
|
|
||||||
f'sock: {sock!r}\n'
|
|
||||||
f'peer_pid: {peer_pid!r}\n'
|
|
||||||
)
|
|
||||||
|
|
||||||
filedir, filename = unwrap_sockpath(sock_path)
|
filedir, filename = unwrap_sockpath(sock_path)
|
||||||
laddr = UDSAddress(
|
laddr = UDSAddress(
|
||||||
|
|
|
||||||
|
|
@ -293,7 +293,7 @@ _conc_name_getters = {
|
||||||
'task': pformat_task_uid,
|
'task': pformat_task_uid,
|
||||||
'actor': lambda: _curr_actor_no_exc(),
|
'actor': lambda: _curr_actor_no_exc(),
|
||||||
'actor_name': lambda: current_actor().name,
|
'actor_name': lambda: current_actor().name,
|
||||||
'actor_uid': lambda: current_actor().aid.uuid[:6],
|
'actor_uid': lambda: current_actor().uid[1][:6],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -324,8 +324,6 @@ class Start(
|
||||||
# => SEE ABOVE <=
|
# => SEE ABOVE <=
|
||||||
kwargs: dict[str, Any]
|
kwargs: dict[str, Any]
|
||||||
uid: tuple[str, str] # (calling) actor-id
|
uid: tuple[str, str] # (calling) actor-id
|
||||||
# aid: Aid
|
|
||||||
# ^TODO, convert stat!
|
|
||||||
|
|
||||||
# TODO: enforcing a msg-spec in terms `Msg.pld`
|
# TODO: enforcing a msg-spec in terms `Msg.pld`
|
||||||
# parameterizable msgs to be used in the appls IPC dialog.
|
# parameterizable msgs to be used in the appls IPC dialog.
|
||||||
|
|
|
||||||
|
|
@ -272,16 +272,18 @@ class LinkedTaskChannel(
|
||||||
'''
|
'''
|
||||||
Receive a value `asyncio.Task` <- `trio.Task`.
|
Receive a value `asyncio.Task` <- `trio.Task`.
|
||||||
|
|
||||||
This is equiv to `await self._to_aio.get()`.
|
This is equiv to `await self._from_trio.get()`.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
return await self._to_aio.get()
|
return await self._to_aio.get()
|
||||||
|
|
||||||
async def send(self, item: Any) -> None:
|
async def send(self, item: Any) -> None:
|
||||||
'''
|
'''
|
||||||
Send a value `trio.Task` -> `asyncio.Task`
|
Send a value through `trio.Task` -> `asyncio.Task`
|
||||||
by enqueuing `item` onto the internal
|
presuming
|
||||||
`asyncio.Queue` via `put_nowait()`.
|
it defines a `from_trio` argument or makes calls
|
||||||
|
to `chan.get()` , if it does not
|
||||||
|
this method will raise an error.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
self._to_aio.put_nowait(item)
|
self._to_aio.put_nowait(item)
|
||||||
|
|
@ -1302,14 +1304,13 @@ async def open_channel_from(
|
||||||
tuple[LinkedTaskChannel, Any]
|
tuple[LinkedTaskChannel, Any]
|
||||||
]:
|
]:
|
||||||
'''
|
'''
|
||||||
Start an `asyncio.Task` as `target()` and open an
|
Start an `asyncio.Task` as `target()` and open an inter-loop
|
||||||
inter-loop (linked) channel for streaming between
|
(linked) channel for streaming between it and the current
|
||||||
it and the current `trio.Task`.
|
`trio.Task`.
|
||||||
|
|
||||||
A pair `(chan: LinkedTaskChannel, Any)` is delivered
|
A pair `(chan: LinkedTaskChannel, Any)` is delivered to the caller
|
||||||
to the caller where the 2nd element is the value
|
where the 2nd element is the value provided by the
|
||||||
provided by the `asyncio.Task`'s unblocking call
|
`asyncio.Task`'s unblocking call to `chan.started_nowait()`.
|
||||||
to `chan.started_nowait()`.
|
|
||||||
|
|
||||||
'''
|
'''
|
||||||
chan: LinkedTaskChannel = _run_asyncio_task(
|
chan: LinkedTaskChannel = _run_asyncio_task(
|
||||||
|
|
@ -1333,7 +1334,8 @@ async def open_channel_from(
|
||||||
first = await chan.receive()
|
first = await chan.receive()
|
||||||
|
|
||||||
# deliver stream handle upward
|
# deliver stream handle upward
|
||||||
yield chan, first
|
yield first, chan
|
||||||
|
# ^TODO! swap these!!
|
||||||
except trio.Cancelled as taskc:
|
except trio.Cancelled as taskc:
|
||||||
if cs.cancel_called:
|
if cs.cancel_called:
|
||||||
if isinstance(chan._trio_to_raise, AsyncioCancelled):
|
if isinstance(chan._trio_to_raise, AsyncioCancelled):
|
||||||
|
|
|
||||||
11
uv.lock
11
uv.lock
|
|
@ -1,15 +1,6 @@
|
||||||
version = 1
|
version = 1
|
||||||
revision = 3
|
revision = 3
|
||||||
requires-python = ">=3.12, <3.14"
|
requires-python = ">=3.12"
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "async-generator"
|
|
||||||
version = "1.10"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/ce/b6/6fa6b3b598a03cba5e80f829e0dadbb49d7645f523d209b2fb7ea0bbb02a/async_generator-1.10.tar.gz", hash = "sha256:6ebb3d106c12920aaae42ccb6f787ef5eefdcdd166ea3d628fa8476abe712144", size = 29870, upload-time = "2018-08-01T03:36:21.69Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/71/52/39d20e03abd0ac9159c162ec24b93fbcaa111e8400308f2465432495ca2b/async_generator-1.10-py3-none-any.whl", hash = "sha256:01c7bf666359b4967d2cda0000cc2e4af16a0ae098cbffcb8472fb9e8ad6585b", size = 18857, upload-time = "2018-08-01T03:36:20.029Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "async-generator"
|
name = "async-generator"
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue