Compare commits
No commits in common. "mtf_backend" and "main" have entirely different histories.
mtf_backen
...
main
|
|
@ -1,38 +0,0 @@
|
|||
# Docs TODOs
|
||||
|
||||
## Auto-sync README code examples with source
|
||||
|
||||
The `docs/README.rst` has inline code blocks that
|
||||
duplicate actual example files (e.g.
|
||||
`examples/infected_asyncio_echo_server.py`). Every time
|
||||
the public API changes we have to manually sync both.
|
||||
|
||||
Sphinx's `literalinclude` directive can pull code directly
|
||||
from source files:
|
||||
|
||||
```rst
|
||||
.. literalinclude:: ../examples/infected_asyncio_echo_server.py
|
||||
:language: python
|
||||
:caption: examples/infected_asyncio_echo_server.py
|
||||
```
|
||||
|
||||
Or to include only a specific function/section:
|
||||
|
||||
```rst
|
||||
.. literalinclude:: ../examples/infected_asyncio_echo_server.py
|
||||
:language: python
|
||||
:pyobject: aio_echo_server
|
||||
```
|
||||
|
||||
This way the docs always reflect the actual code without
|
||||
manual syncing.
|
||||
|
||||
### Considerations
|
||||
- `README.rst` is also rendered on GitHub/PyPI which do
|
||||
NOT support `literalinclude` - so we'd need a build
|
||||
step or a separate `_sphinx_readme.rst` (which already
|
||||
exists at `docs/github_readme/_sphinx_readme.rst`).
|
||||
- Could use a pre-commit hook or CI step to extract code
|
||||
from examples into the README for GitHub rendering.
|
||||
- Another option: `sphinx-autodoc` style approach where
|
||||
docstrings from the actual module are pulled in.
|
||||
|
|
@ -1,125 +0,0 @@
|
|||
# `RuntimeVars` env-var lift — design plan
|
||||
|
||||
Status: **draft, awaiting user edits**
|
||||
|
||||
## Goal
|
||||
|
||||
Consolidate the sprawl of pytest CLI flags + ad-hoc env vars +
|
||||
hardcoded fixture defaults into a *single* env-var-encoded
|
||||
runtime-vars envelope, with a typed in-memory representation
|
||||
(`tractor.runtime._state.RuntimeVars`) as the sole source of
|
||||
truth.
|
||||
|
||||
## Why now
|
||||
|
||||
- `--tpt-proto`, `--spawn-backend`, `--diag-on-hang`,
|
||||
`--diag-capture-delay` and (soon) `TRACTOR_REG_ADDR` etc. are
|
||||
proliferating. Each adds a parsing seam.
|
||||
- `tests/devx/test_debugger.py` invokes example scripts as
|
||||
separate subprocesses; they currently can't see the
|
||||
fixture-allocated `reg_addr` at all (root cause of why
|
||||
parametrizing devx scripts on `reg_addr` is on your TODO).
|
||||
- Concurrent pytest sessions on the same host collide on
|
||||
shared defaults (the `registry@1616` race we just fixed is
|
||||
one symptom; per-session unique addr is the structural
|
||||
fix).
|
||||
- `tractor.runtime._state.RuntimeVars: Struct` is already
|
||||
defined and **unused** — its docstring even says it
|
||||
"should be utilized as possible for future calls."
|
||||
|
||||
## Design
|
||||
|
||||
### Module: `tractor/_testing/_rtvars.py`
|
||||
|
||||
Lifted from `modden.runtime.env`, ~50 LOC, no new deps.
|
||||
|
||||
```python
|
||||
_TRACTOR_RT_VARS_OSENV: str = '_TRACTOR_RT_VARS'
|
||||
|
||||
def dump_rtvars(rtvars: RuntimeVars|dict) -> tuple[str, str]:
|
||||
'''str-serialize via `str(dict)` — ast.literal_eval-able'''
|
||||
|
||||
def load_rtvars(env: dict) -> RuntimeVars:
|
||||
'''ast.literal_eval the env-var value, hydrate to struct'''
|
||||
|
||||
def get_rtvars(proc: psutil.Process|None = None) -> RuntimeVars:
|
||||
'''read the var from a target proc's env (or current)'''
|
||||
|
||||
def update_rtvars(
|
||||
rtvars: RuntimeVars|dict|None = None,
|
||||
update_osenv: bool|dict = True,
|
||||
) -> tuple[str, str]:
|
||||
'''mutate + re-encode + (optionally) write to os.environ'''
|
||||
```
|
||||
|
||||
### Encoding choice: `str(dict)` + `ast.literal_eval`
|
||||
|
||||
Pros:
|
||||
- stdlib only
|
||||
- handles all the types tractor's tests need: `str`, `int`,
|
||||
`float`, `bool`, `None`, `list`, `tuple`, `dict`
|
||||
- human-readable in the env (greppable, inspectable via
|
||||
`cat /proc/<pid>/environ | tr '\0' '\n'`)
|
||||
|
||||
Cons:
|
||||
- non-stdlib types (msgspec Structs, `Path`, custom classes)
|
||||
must be lowered first — fine for the test fixture set
|
||||
- not stable across Python versions for esoteric repr cases
|
||||
(we don't hit any)
|
||||
|
||||
Alternatives considered:
|
||||
- **msgpack**: adds a dep + binary form is ungreppable
|
||||
- **json**: doesn't preserve tuples (becomes lists), which is
|
||||
a common type for `reg_addr`
|
||||
- **toml/yaml**: heavier deps, no real benefit
|
||||
|
||||
### `RuntimeVars` becomes the single source of truth
|
||||
|
||||
The legacy `_runtime_vars: dict[str, Any]` global in
|
||||
`runtime/_state.py` becomes a *cached view* of a
|
||||
`RuntimeVars` singleton instance:
|
||||
|
||||
- `get_runtime_vars()` returns either the struct or a
|
||||
`.to_dict()` view depending on caller's preference
|
||||
- `set_runtime_vars(...)` validates against the struct schema
|
||||
- spawn-time SpawnSpec sends the struct (already does
|
||||
conceptually — just gets typed)
|
||||
- `__setattr__` `breakpoint()` debug instrumentation gets
|
||||
removed (unrelated cleanup, mentioned in conversation)
|
||||
|
||||
### Migration path
|
||||
|
||||
**Phase 0** *(prep)*: strip the stray `breakpoint()` from
|
||||
`RuntimeVars.__setattr__`.
|
||||
|
||||
**Phase 1**: land `_rtvars.py` as a leaf module, used only by
|
||||
test infra. Subprocess-spawned scripts in `tests/devx/`
|
||||
read `_TRACTOR_RT_VARS` on startup → reconstruct
|
||||
`RuntimeVars` → call `tractor.open_root_actor(**rtvars.as_kwargs())`.
|
||||
Concurrent runs become deterministic-isolated because each
|
||||
session writes a unique `_registry_addrs` into the env.
|
||||
|
||||
**Phase 2**: migrate runtime callers (`_state.get_runtime_vars`,
|
||||
spawn `SpawnSpec`, `Actor.async_main`) to operate on the
|
||||
struct directly, with the dict as a compat view that gets
|
||||
deprecated.
|
||||
|
||||
**Phase 3** *(structural)*: per-session bindspace subdir
|
||||
`/run/user/<uid>/tractor/<session_uuid>/` — encoded in the
|
||||
rt-vars envelope, picked up by every subactor automatically.
|
||||
Obsoletes the entire bindspace-leak warning class.
|
||||
|
||||
## Open design questions (user input wanted)
|
||||
|
||||
- (placeholder for your edits)
|
||||
- (placeholder)
|
||||
- (placeholder)
|
||||
|
||||
## Out-of-scope for this lift
|
||||
|
||||
- Anything in `modden.runtime.env` related to `Spawn`,
|
||||
`WmCtl`, `Wks` — that's a workspace orchestration layer,
|
||||
not an env-var helper. We only lift the four utility
|
||||
functions + the var name constant.
|
||||
- Switching to msgpack/json — explicitly chosen against
|
||||
above.
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(cp .claude/*)",
|
||||
"Read(.claude/**)",
|
||||
"Read(.claude/skills/run-tests/**)",
|
||||
"Write(.claude/**/*commit_msg*)",
|
||||
"Write(.claude/git_commit_msg_LATEST.md)",
|
||||
"Skill(run-tests)",
|
||||
"Skill(close-wkt)",
|
||||
"Skill(open-wkt)",
|
||||
"Skill(prompt-io)",
|
||||
"Bash(date *)",
|
||||
"Bash(git diff *)",
|
||||
"Bash(git log *)",
|
||||
"Bash(git status)",
|
||||
"Bash(git remote:*)",
|
||||
"Bash(git stash:*)",
|
||||
"Bash(git mv:*)",
|
||||
"Bash(git rev-parse:*)",
|
||||
"Bash(test:*)",
|
||||
"Bash(ls:*)",
|
||||
"Bash(grep:*)",
|
||||
"Bash(find:*)",
|
||||
"Bash(ln:*)",
|
||||
"Bash(cat:*)",
|
||||
"Bash(mkdir:*)",
|
||||
"Bash(gh pr:*)",
|
||||
"Bash(gh api:*)",
|
||||
"Bash(gh issue:*)",
|
||||
"Bash(UV_PROJECT_ENVIRONMENT=py* uv sync:*)",
|
||||
"Bash(UV_PROJECT_ENVIRONMENT=py* uv run:*)",
|
||||
"Bash(echo EXIT:$?:*)",
|
||||
"Bash(echo \"EXIT=$?\")",
|
||||
"Read(/tmp/**)"
|
||||
],
|
||||
"deny": [],
|
||||
"ask": []
|
||||
},
|
||||
"prefersReducedMotion": false,
|
||||
"outputStyle": "default"
|
||||
}
|
||||
|
|
@ -1,225 +0,0 @@
|
|||
# Commit Message Style Guide for `tractor`
|
||||
|
||||
Analysis based on 500 recent commits from the `tractor` repository.
|
||||
|
||||
## Core Principles
|
||||
|
||||
Write commit messages that are technically precise yet casual in
|
||||
tone. Use abbreviations and informal language while maintaining
|
||||
clarity about what changed and why.
|
||||
|
||||
## Subject Line Format
|
||||
|
||||
### Length and Structure
|
||||
- Target: ~50 chars with a hard-max of 67.
|
||||
- Use backticks around code elements (72.2% of commits)
|
||||
- Rarely use colons (5.2%), except for file prefixes
|
||||
- End with '?' for uncertain changes (rare: 0.8%)
|
||||
- End with '!' for important changes (rare: 2.0%)
|
||||
|
||||
### Opening Verbs (Present Tense)
|
||||
|
||||
Most common verbs from analysis:
|
||||
- `Add` (14.4%) - wholly new features/functionality
|
||||
- `Use` (4.4%) - adopt new approach/tool
|
||||
- `Drop` (3.6%) - remove code/feature
|
||||
- `Fix` (2.4%) - bug fixes
|
||||
- `Move`/`Mv` (3.6%) - relocate code
|
||||
- `Adjust` (2.0%) - minor tweaks
|
||||
- `Update` (1.6%) - enhance existing feature
|
||||
- `Bump` (1.2%) - dependency updates
|
||||
- `Rename` (1.2%) - identifier changes
|
||||
- `Set` (1.2%) - configuration changes
|
||||
- `Handle` (1.0%) - add handling logic
|
||||
- `Raise` (1.0%) - add error raising
|
||||
- `Pass` (0.8%) - pass parameters/values
|
||||
- `Support` (0.8%) - add support for something
|
||||
- `Hide` (1.4%) - make private/internal
|
||||
- `Always` (1.4%) - enforce consistent behavior
|
||||
- `Mk` (1.4%) - make/create (abbreviated)
|
||||
- `Start` (1.0%) - begin implementation
|
||||
|
||||
Other frequent verbs: `More`, `Change`, `Extend`, `Disable`, `Log`,
|
||||
`Enable`, `Ensure`, `Expose`, `Allow`
|
||||
|
||||
### Backtick Usage
|
||||
|
||||
Always use backticks for:
|
||||
- Module names: `trio`, `asyncio`, `msgspec`, `greenback`, `stackscope`
|
||||
- Class names: `Context`, `Actor`, `Address`, `PldRx`, `SpawnSpec`
|
||||
- Method names: `.pause_from_sync()`, `._pause()`, `.cancel()`
|
||||
- Function names: `breakpoint()`, `collapse_eg()`, `open_root_actor()`
|
||||
- Decorators: `@acm`, `@context`
|
||||
- Exceptions: `Cancelled`, `TransportClosed`, `MsgTypeError`
|
||||
- Keywords: `finally`, `None`, `False`
|
||||
- Variable names: `tn`, `debug_mode`
|
||||
- Complex expressions: `trio.Cancelled`, `asyncio.Task`
|
||||
|
||||
Most backticked terms in tractor:
|
||||
`trio`, `asyncio`, `Context`, `.pause_from_sync()`, `tn`,
|
||||
`._pause()`, `breakpoint()`, `collapse_eg()`, `Actor`, `@acm`,
|
||||
`.cancel()`, `Cancelled`, `open_root_actor()`, `greenback`
|
||||
|
||||
### Examples
|
||||
|
||||
Good subject lines:
|
||||
```
|
||||
Add `uds` to `._multiaddr`, tweak typing
|
||||
Drop `DebugStatus.shield` attr, add `.req_finished`
|
||||
Use `stackscope` for all actor-tree rendered "views"
|
||||
Fix `.to_asyncio` inter-task-cancellation!
|
||||
Bump `ruff.toml` to target py313
|
||||
Mv `load_module_from_path()` to new `._code_load` submod
|
||||
Always use `tuple`-cast for singleton parent addrs
|
||||
```
|
||||
|
||||
## Body Format
|
||||
|
||||
### General Structure
|
||||
- 43.2% of commits have no body (simple changes)
|
||||
- Use blank line after subject
|
||||
- Max line length: 67 chars
|
||||
- Use `-` bullets for lists (28.0% of commits)
|
||||
- Rarely use `*` bullets (2.4%)
|
||||
|
||||
### Section Markers
|
||||
|
||||
Use these markers to organize longer commit bodies:
|
||||
- `Also,` (most common: 26 occurrences)
|
||||
- `Other,` (13 occurrences)
|
||||
- `Deats,` (11 occurrences) - for implementation details
|
||||
- `Further,` (7 occurrences)
|
||||
- `TODO,` (3 occurrences)
|
||||
- `Impl details,` (2 occurrences)
|
||||
- `Notes,` (1 occurrence)
|
||||
|
||||
### Common Abbreviations
|
||||
|
||||
Use these freely (sorted by frequency):
|
||||
- `msg` (63) - message
|
||||
- `bg` (37) - background
|
||||
- `ctx` (30) - context
|
||||
- `impl` (27) - implementation
|
||||
- `mod` (26) - module
|
||||
- `obvi` (17) - obviously
|
||||
- `tn` (16) - task name
|
||||
- `fn` (15) - function
|
||||
- `vs` (15) - versus
|
||||
- `bc` (14) - because
|
||||
- `var` (14) - variable
|
||||
- `prolly` (9) - probably
|
||||
- `ep` (6) - entry point
|
||||
- `OW` (5) - otherwise
|
||||
- `rn` (4) - right now
|
||||
- `sig` (4) - signal/signature
|
||||
- `deps` (3) - dependencies
|
||||
- `iface` (2) - interface
|
||||
- `subproc` (2) - subprocess
|
||||
- `tho` (2) - though
|
||||
- `ofc` (2) - of course
|
||||
|
||||
### Tone and Style
|
||||
|
||||
- Casual but technical (use `XD` for humor: 23 times)
|
||||
- Use `..` for trailing thoughts (108 occurrences)
|
||||
- Use `Woops,` to acknowledge mistakes (4 subject lines)
|
||||
- Don't be afraid to show personality while being precise
|
||||
|
||||
### Example Bodies
|
||||
|
||||
Simple with bullets:
|
||||
```
|
||||
Add `multiaddr` and bump up some deps
|
||||
|
||||
Since we're planning to use it for (discovery)
|
||||
addressing, allowing replacement of the hacky (pretend)
|
||||
attempt in `tractor._multiaddr` Bp
|
||||
|
||||
Also pin some deps,
|
||||
- make us py312+
|
||||
- use `pdbp` with my frame indexing fix.
|
||||
- mv to latest `xonsh` for fancy cmd/suggestion injections.
|
||||
|
||||
Bump lock file to match obvi!
|
||||
```
|
||||
|
||||
With section markers:
|
||||
```
|
||||
Use `stackscope` for all actor-tree rendered "views"
|
||||
|
||||
Instead of the (much more) limited and hacky `.devx._code`
|
||||
impls, move to using the new `.devx._stackscope` API which
|
||||
wraps the `stackscope` project.
|
||||
|
||||
Deats,
|
||||
- make new `stackscope.extract_stack()` wrapper
|
||||
- port over frame-descing to `_stackscope.pformat_stack()`
|
||||
- move `PdbREPL` to use `stackscope` render approach
|
||||
- update tests for new stack output format
|
||||
|
||||
Also,
|
||||
- tweak log formatting for consistency
|
||||
- add typing hints throughout
|
||||
```
|
||||
|
||||
## Special Patterns
|
||||
|
||||
### WIP Commits
|
||||
Rare (0.2%) - avoid committing WIP if possible
|
||||
|
||||
### Merge Commits
|
||||
Auto-generated (4.4%), don't worry about style
|
||||
|
||||
### File References
|
||||
- Use `module.py` or `.submodule` style
|
||||
- Rarely use `file.py:line` references (0 in analysis)
|
||||
|
||||
### Links
|
||||
- GitHub links used sparingly (3 total)
|
||||
- Prefer code references over external links
|
||||
|
||||
## Footer
|
||||
|
||||
The default footer should credit `claude` (you) for helping generate
|
||||
the commit msg content:
|
||||
|
||||
```
|
||||
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
|
||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
||||
```
|
||||
|
||||
Further, if the patch was solely or in part written
|
||||
by `claude`, instead add:
|
||||
|
||||
```
|
||||
(this patch was generated in some part by [`claude-code`][claude-code-gh])
|
||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
||||
```
|
||||
|
||||
## Summary Checklist
|
||||
|
||||
Before committing, verify:
|
||||
- [ ] Subject line uses present tense verb
|
||||
- [ ] Subject line ~50 chars (hard max 67)
|
||||
- [ ] Code elements wrapped in backticks
|
||||
- [ ] Body lines ≤67 chars
|
||||
- [ ] Abbreviations used where natural
|
||||
- [ ] Casual yet precise tone
|
||||
- [ ] Section markers if body >3 paragraphs
|
||||
- [ ] Technical accuracy maintained
|
||||
|
||||
## Analysis Metadata
|
||||
|
||||
```
|
||||
Source: tractor repository
|
||||
Commits analyzed: 500
|
||||
Date range: 2019-2025
|
||||
Analysis date: 2026-02-08
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
(this style guide was generated by [`claude-code`][claude-code-gh]
|
||||
analyzing commit history)
|
||||
|
||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
||||
|
|
@ -1,297 +0,0 @@
|
|||
---
|
||||
name: conc-anal
|
||||
description: >
|
||||
Concurrency analysis for tractor's trio-based
|
||||
async primitives. Trace task scheduling across
|
||||
checkpoint boundaries, identify race windows in
|
||||
shared mutable state, and verify synchronization
|
||||
correctness. Invoke on code segments the user
|
||||
points at, OR proactively when reviewing/writing
|
||||
concurrent cache, lock, or multi-task acm code.
|
||||
argument-hint: "[file:line-range or function name]"
|
||||
allowed-tools:
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
- Task
|
||||
---
|
||||
|
||||
Perform a structured concurrency analysis on the
|
||||
target code. This skill should be invoked:
|
||||
|
||||
- **On demand**: user points at a code segment
|
||||
(file:lines, function name, or pastes a snippet)
|
||||
- **Proactively**: when writing or reviewing code
|
||||
that touches shared mutable state across trio
|
||||
tasks — especially `_Cache`, locks, events, or
|
||||
multi-task `@acm` lifecycle management
|
||||
|
||||
## 0. Identify the target
|
||||
|
||||
If the user provides a file:line-range or function
|
||||
name, read that code. If not explicitly provided,
|
||||
identify the relevant concurrent code from context
|
||||
(e.g. the current diff, a failing test, or the
|
||||
function under discussion).
|
||||
|
||||
## 1. Inventory shared mutable state
|
||||
|
||||
List every piece of state that is accessed by
|
||||
multiple tasks. For each, note:
|
||||
|
||||
- **What**: the variable/dict/attr (e.g.
|
||||
`_Cache.values`, `_Cache.resources`,
|
||||
`_Cache.users`)
|
||||
- **Scope**: class-level, module-level, or
|
||||
closure-captured
|
||||
- **Writers**: which tasks/code-paths mutate it
|
||||
- **Readers**: which tasks/code-paths read it
|
||||
- **Guarded by**: which lock/event/ordering
|
||||
protects it (or "UNGUARDED" if none)
|
||||
|
||||
Format as a table:
|
||||
|
||||
```
|
||||
| State | Writers | Readers | Guard |
|
||||
|---------------------|-----------------|-----------------|----------------|
|
||||
| _Cache.values | run_ctx, moc¹ | moc | ctx_key lock |
|
||||
| _Cache.resources | run_ctx, moc | moc, run_ctx | UNGUARDED |
|
||||
```
|
||||
|
||||
¹ `moc` = `maybe_open_context`
|
||||
|
||||
## 2. Map checkpoint boundaries
|
||||
|
||||
For each code path through the target, mark every
|
||||
**checkpoint** — any `await` expression where trio
|
||||
can switch to another task. Use line numbers:
|
||||
|
||||
```
|
||||
L325: await lock.acquire() ← CHECKPOINT
|
||||
L395: await service_tn.start(...) ← CHECKPOINT
|
||||
L411: lock.release() ← (not a checkpoint, but changes lock state)
|
||||
L414: yield (False, yielded) ← SUSPEND (caller runs)
|
||||
L485: no_more_users.set() ← (wakes run_ctx, no switch yet)
|
||||
```
|
||||
|
||||
**Key trio scheduling rules to apply:**
|
||||
- `Event.set()` makes waiters *ready* but does NOT
|
||||
switch immediately
|
||||
- `lock.release()` is not a checkpoint
|
||||
- `await sleep(0)` IS a checkpoint
|
||||
- Code in `finally` blocks CAN have checkpoints
|
||||
(unlike asyncio)
|
||||
- `await` inside `except` blocks can be
|
||||
`trio.Cancelled`-masked
|
||||
|
||||
## 3. Trace concurrent task schedules
|
||||
|
||||
Write out the **interleaved execution trace** for
|
||||
the problematic scenario. Number each step and tag
|
||||
which task executes it:
|
||||
|
||||
```
|
||||
[Task A] 1. acquires lock
|
||||
[Task A] 2. cache miss → allocates resources
|
||||
[Task A] 3. releases lock
|
||||
[Task A] 4. yields to caller
|
||||
[Task A] 5. caller exits → finally runs
|
||||
[Task A] 6. users-- → 0, sets no_more_users
|
||||
[Task A] 7. pops lock from _Cache.locks
|
||||
[run_ctx] 8. wakes from no_more_users.wait()
|
||||
[run_ctx] 9. values.pop(ctx_key)
|
||||
[run_ctx] 10. acm __aexit__ → CHECKPOINT
|
||||
[Task B] 11. creates NEW lock (old one popped)
|
||||
[Task B] 12. acquires immediately
|
||||
[Task B] 13. values[ctx_key] → KeyError
|
||||
[Task B] 14. resources[ctx_key] → STILL EXISTS
|
||||
[Task B] 15. 💥 RuntimeError
|
||||
```
|
||||
|
||||
Identify the **race window**: the range of steps
|
||||
where state is inconsistent. In the example above,
|
||||
steps 9–10 are the window (values gone, resources
|
||||
still alive).
|
||||
|
||||
## 4. Classify the bug
|
||||
|
||||
Categorize what kind of concurrency issue this is:
|
||||
|
||||
- **TOCTOU** (time-of-check-to-time-of-use): state
|
||||
changes between a check and the action based on it
|
||||
- **Stale reference**: a task holds a reference to
|
||||
state that another task has invalidated
|
||||
- **Lifetime mismatch**: a synchronization primitive
|
||||
(lock, event) has a shorter lifetime than the
|
||||
state it's supposed to protect
|
||||
- **Missing guard**: shared state is accessed
|
||||
without any synchronization
|
||||
- **Atomicity gap**: two operations that should be
|
||||
atomic have a checkpoint between them
|
||||
|
||||
## 5. Propose fixes
|
||||
|
||||
For each proposed fix, provide:
|
||||
|
||||
- **Sketch**: pseudocode or diff showing the change
|
||||
- **How it closes the window**: which step(s) from
|
||||
the trace it eliminates or reorders
|
||||
- **Tradeoffs**: complexity, perf, new edge cases,
|
||||
impact on other code paths
|
||||
- **Risk**: what could go wrong (deadlocks, new
|
||||
races, cancellation issues)
|
||||
|
||||
Rate each fix: `[simple|moderate|complex]` impl
|
||||
effort.
|
||||
|
||||
## 6. Output format
|
||||
|
||||
Structure the full analysis as:
|
||||
|
||||
```markdown
|
||||
## Concurrency analysis: `<target>`
|
||||
|
||||
### Shared state
|
||||
<table from step 1>
|
||||
|
||||
### Checkpoints
|
||||
<list from step 2>
|
||||
|
||||
### Race trace
|
||||
<interleaved trace from step 3>
|
||||
|
||||
### Classification
|
||||
<bug type from step 4>
|
||||
|
||||
### Fixes
|
||||
<proposals from step 5>
|
||||
```
|
||||
|
||||
## Tractor-specific patterns to watch
|
||||
|
||||
These are known problem areas in tractor's
|
||||
concurrency model. Flag them when encountered:
|
||||
|
||||
### `_Cache` lock vs `run_ctx` lifetime
|
||||
|
||||
The `_Cache.locks` entry is managed by
|
||||
`maybe_open_context` callers, but `run_ctx` runs
|
||||
in `service_tn` — a different task tree. Lock
|
||||
pop/release in the caller's `finally` does NOT
|
||||
wait for `run_ctx` to finish tearing down. Any
|
||||
state that `run_ctx` cleans up in its `finally`
|
||||
(e.g. `resources.pop()`) is vulnerable to
|
||||
re-entry races after the lock is popped.
|
||||
|
||||
### `values.pop()` → acm `__aexit__` → `resources.pop()` gap
|
||||
|
||||
In `_Cache.run_ctx`, the inner `finally` pops
|
||||
`values`, then the acm's `__aexit__` runs (which
|
||||
has checkpoints), then the outer `finally` pops
|
||||
`resources`. This creates a window where `values`
|
||||
is gone but `resources` still exists — a classic
|
||||
atomicity gap.
|
||||
|
||||
### Global vs per-key counters
|
||||
|
||||
`_Cache.users` as a single `int` (pre-fix) meant
|
||||
that users of different `ctx_key`s inflated each
|
||||
other's counts, preventing teardown when one key's
|
||||
users hit zero. Always verify that per-key state
|
||||
(`users`, `locks`) is actually keyed on `ctx_key`
|
||||
and not on `fid` or some broader key.
|
||||
|
||||
### `Event.set()` wakes but doesn't switch
|
||||
|
||||
`trio.Event.set()` makes waiting tasks *ready* but
|
||||
the current task continues executing until its next
|
||||
checkpoint. Code between `.set()` and the next
|
||||
`await` runs atomically from the scheduler's
|
||||
perspective. Use this to your advantage (or watch
|
||||
for bugs where code assumes the woken task runs
|
||||
immediately).
|
||||
|
||||
### `except` block checkpoint masking
|
||||
|
||||
`await` expressions inside `except` handlers can
|
||||
be masked by `trio.Cancelled`. If a `finally`
|
||||
block runs from an `except` and contains
|
||||
`lock.release()`, the release happens — but any
|
||||
`await` after it in the same `except` may be
|
||||
swallowed. This is why `maybe_open_context`'s
|
||||
cache-miss path does `lock.release()` in a
|
||||
`finally` inside the `except KeyError`.
|
||||
|
||||
### Cancellation in `finally`
|
||||
|
||||
Unlike asyncio, trio allows checkpoints in
|
||||
`finally` blocks. This means `finally` cleanup
|
||||
that does `await` can itself be cancelled (e.g.
|
||||
by nursery shutdown). Watch for cleanup code that
|
||||
assumes it will run to completion.
|
||||
|
||||
### Unbounded waits in cleanup paths
|
||||
|
||||
Any `await <event>.wait()` in a teardown path is
|
||||
a latent deadlock unless the event's setter is
|
||||
GUARANTEED to fire. If the setter depends on
|
||||
external state (peer disconnects, child process
|
||||
exit, subsequent task completion) that itself
|
||||
depends on the current task's progress, you have
|
||||
a mutual wait.
|
||||
|
||||
Rule: **bound every `await X.wait()` in cleanup
|
||||
paths with `trio.move_on_after()`** unless you
|
||||
can prove the setter is unconditionally reachable
|
||||
from the state at the await site. Concrete recent
|
||||
example: `ipc_server.wait_for_no_more_peers()` in
|
||||
`async_main`'s finally (see
|
||||
`ai/conc-anal/subint_forkserver_test_cancellation_leak_issue.md`
|
||||
"probe iteration 3") — it was unbounded, and when
|
||||
one peer-handler was stuck the wait-for-no-more-
|
||||
peers event never fired, deadlocking the whole
|
||||
actor-tree teardown cascade.
|
||||
|
||||
### The capture-pipe-fill hang pattern (grep this first)
|
||||
|
||||
When investigating any hang in the test suite
|
||||
**especially under fork-based backends**, first
|
||||
check whether the hang reproduces under `pytest
|
||||
-s` (`--capture=no`). If `-s` makes it go away
|
||||
you're not looking at a trio concurrency bug —
|
||||
you're looking at a Linux pipe-buffer fill.
|
||||
|
||||
Mechanism: pytest replaces fds 1,2 with pipe
|
||||
write-ends. Fork-child subactors inherit those
|
||||
fds. High-volume error-log tracebacks (cancel
|
||||
cascade spew) fill the 64KB pipe buffer. Child
|
||||
`write()` blocks. Child can't exit. Parent's
|
||||
`waitpid`/pidfd wait blocks. Deadlock cascades up
|
||||
the tree.
|
||||
|
||||
Pre-existing guards in `tests/conftest.py` encode
|
||||
this knowledge — grep these BEFORE blaming
|
||||
concurrency:
|
||||
|
||||
```python
|
||||
# tests/conftest.py:258
|
||||
if loglevel in ('trace', 'debug'):
|
||||
# XXX: too much logging will lock up the subproc (smh)
|
||||
loglevel: str = 'info'
|
||||
|
||||
# tests/conftest.py:316
|
||||
# can lock up on the `_io.BufferedReader` and hang..
|
||||
stderr: str = proc.stderr.read().decode()
|
||||
```
|
||||
|
||||
Full post-mortem +
|
||||
`ai/conc-anal/subint_forkserver_test_cancellation_leak_issue.md`
|
||||
for the canonical reproduction. Cost several
|
||||
investigation sessions before catching it —
|
||||
because the capture-pipe symptom was masked by
|
||||
deeper cascade-deadlocks. Once the cascades were
|
||||
fixed, the tree tore down enough to generate
|
||||
pipe-filling log volume → capture-pipe finally
|
||||
surfaced. Grep-note for future-self: **if a
|
||||
multi-subproc tractor test hangs, `pytest -s`
|
||||
first, conc-anal second.**
|
||||
|
|
@ -1,241 +0,0 @@
|
|||
# PR/Patch-Request Description Format Reference
|
||||
|
||||
Canonical structure for `tractor` patch-request
|
||||
descriptions, designed to work across GitHub,
|
||||
Gitea, SourceHut, and GitLab markdown renderers.
|
||||
|
||||
**Line length: wrap at 72 chars** for all prose
|
||||
content (Summary bullets, Motivation paragraphs,
|
||||
Scopes bullets, etc.). Fill lines *to* 72 — don't
|
||||
stop short at 50-65. Only raw URLs in
|
||||
reference-link definitions may exceed this.
|
||||
|
||||
## Template
|
||||
|
||||
```markdown
|
||||
<!-- pr-msg-meta
|
||||
branch: <branch-name>
|
||||
base: <base-branch>
|
||||
submitted:
|
||||
github: ___
|
||||
gitea: ___
|
||||
srht: ___
|
||||
-->
|
||||
|
||||
## <Title: present-tense verb + backticked code>
|
||||
|
||||
### Summary
|
||||
- [<hash>][<hash>] Description of change ending
|
||||
with period.
|
||||
- [<hash>][<hash>] Another change description
|
||||
ending with period.
|
||||
- [<hash>][<hash>] [<hash>][<hash>] Multi-commit
|
||||
change description.
|
||||
|
||||
### Motivation
|
||||
<1-2 paragraphs: problem/limitation first,
|
||||
then solution. Hard-wrap at 72 chars.>
|
||||
|
||||
### Scopes changed
|
||||
- [<hash>][<hash>] `pkg.mod.func()` — what
|
||||
changed.
|
||||
* [<hash>][<hash>] Also adjusts
|
||||
`.related_thing()` in same module.
|
||||
- [<hash>][<hash>] `tests.test_mod` — new/changed
|
||||
test coverage.
|
||||
|
||||
<!--
|
||||
### Cross-references
|
||||
Also submitted as
|
||||
[github-pr][] | [gitea-pr][] | [srht-patch][].
|
||||
|
||||
### Links
|
||||
- [relevant-issue-or-discussion](url)
|
||||
- [design-doc-or-screenshot](url)
|
||||
-->
|
||||
|
||||
(this pr content was generated in some part by
|
||||
[`claude-code`][claude-code-gh])
|
||||
|
||||
[<hash>]: https://<service>/<owner>/<repo>/commit/<hash>
|
||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
||||
|
||||
<!-- cross-service pr refs (fill after submit):
|
||||
[github-pr]: https://github.com/<owner>/<repo>/pull/___
|
||||
[gitea-pr]: https://<host>/<owner>/<repo>/pulls/___
|
||||
[srht-patch]: https://git.sr.ht/~<owner>/<repo>/patches/___
|
||||
-->
|
||||
```
|
||||
|
||||
## Markdown Reference-Link Strategy
|
||||
|
||||
Use reference-style links for ALL commit hashes
|
||||
and cross-service PR refs to ensure cross-service
|
||||
compatibility:
|
||||
|
||||
**Inline usage** (in bullets):
|
||||
```markdown
|
||||
- [f3726cf9][f3726cf9] Add `reg_err_types()`
|
||||
for custom exc lookup.
|
||||
```
|
||||
|
||||
**Definition** (bottom of document):
|
||||
```markdown
|
||||
[f3726cf9]: https://github.com/goodboy/tractor/commit/f3726cf9
|
||||
```
|
||||
|
||||
### Why reference-style?
|
||||
- Keeps prose readable without long inline URLs.
|
||||
- All URLs in one place — trivially swappable
|
||||
per-service.
|
||||
- Most git services auto-link bare SHAs anyway,
|
||||
but explicit refs guarantee it works in *any*
|
||||
md renderer.
|
||||
- The `[hash][hash]` form is self-documenting —
|
||||
display text matches the ref ID.
|
||||
- Cross-service PR refs use the same mechanism:
|
||||
`[github-pr][]` resolves via a ref-link def
|
||||
at the bottom, trivially fillable post-submit.
|
||||
|
||||
## Cross-Service PR Placeholder Mechanism
|
||||
|
||||
The generated description includes three layers
|
||||
of cross-service support, all using native md
|
||||
reference-links:
|
||||
|
||||
### 1. Metadata comment (top of file)
|
||||
|
||||
```markdown
|
||||
<!-- pr-msg-meta
|
||||
branch: remote_exc_type_registry
|
||||
base: main
|
||||
submitted:
|
||||
github: ___
|
||||
gitea: ___
|
||||
srht: ___
|
||||
-->
|
||||
```
|
||||
|
||||
A YAML-ish HTML comment block. The `___`
|
||||
placeholders get filled with PR/patch numbers
|
||||
after submission. Machine-parseable for tooling
|
||||
(e.g. `gish`) but invisible in rendered md.
|
||||
|
||||
### 2. Cross-references section (in body)
|
||||
|
||||
```markdown
|
||||
<!--
|
||||
### Cross-references
|
||||
Also submitted as
|
||||
[github-pr][] | [gitea-pr][] | [srht-patch][].
|
||||
-->
|
||||
```
|
||||
|
||||
Commented out at generation time. After submitting
|
||||
to multiple services, uncomment and the ref-links
|
||||
resolve via the stubs at the bottom.
|
||||
|
||||
### 3. Ref-link stubs (bottom of file)
|
||||
|
||||
```markdown
|
||||
<!-- cross-service pr refs (fill after submit):
|
||||
[github-pr]: https://github.com/goodboy/tractor/pull/___
|
||||
[gitea-pr]: https://pikers.dev/goodboy/tractor/pulls/___
|
||||
[srht-patch]: https://git.sr.ht/~goodboy/tractor/patches/___
|
||||
-->
|
||||
```
|
||||
|
||||
Commented out with `___` number placeholders.
|
||||
After submission: uncomment, replace `___` with
|
||||
the actual number. Each service-specific copy
|
||||
fills in all services' numbers so any copy can
|
||||
cross-reference the others.
|
||||
|
||||
### Post-submission file layout
|
||||
|
||||
```
|
||||
pr_msg_LATEST.md # latest draft (skill root)
|
||||
msgs/
|
||||
20260325T002027Z_mybranch_pr_msg.md # timestamped
|
||||
github/
|
||||
42_pr_msg.md # github PR #42
|
||||
gitea/
|
||||
17_pr_msg.md # gitea PR #17
|
||||
srht/
|
||||
5_pr_msg.md # srht patch #5
|
||||
```
|
||||
|
||||
Each `<service>/<num>_pr_msg.md` is a copy with:
|
||||
- metadata `submitted:` fields filled in
|
||||
- cross-references section uncommented
|
||||
- ref-link stubs uncommented with real numbers
|
||||
- all services cross-linked in each copy
|
||||
|
||||
This mirrors the `gish` skill's
|
||||
`<backend>/<num>.md` pattern.
|
||||
|
||||
## Commit-Link URL Patterns by Service
|
||||
|
||||
| Service | Pattern |
|
||||
|-----------|-------------------------------------|
|
||||
| GitHub | `https://github.com/<o>/<r>/commit/<h>` |
|
||||
| Gitea | `https://<host>/<o>/<r>/commit/<h>` |
|
||||
| SourceHut | `https://git.sr.ht/~<o>/<r>/commit/<h>` |
|
||||
| GitLab | `https://gitlab.com/<o>/<r>/-/commit/<h>` |
|
||||
|
||||
## PR/Patch URL Patterns by Service
|
||||
|
||||
| Service | Pattern |
|
||||
|-----------|-------------------------------------|
|
||||
| GitHub | `https://github.com/<o>/<r>/pull/<n>` |
|
||||
| Gitea | `https://<host>/<o>/<r>/pulls/<n>` |
|
||||
| SourceHut | `https://git.sr.ht/~<o>/<r>/patches/<n>` |
|
||||
| GitLab | `https://gitlab.com/<o>/<r>/-/merge_requests/<n>` |
|
||||
|
||||
## Scope Naming Convention
|
||||
|
||||
Use Python namespace-resolution syntax for
|
||||
referencing changed code scopes:
|
||||
|
||||
| File path | Scope reference |
|
||||
|---------------------------|-------------------------------|
|
||||
| `tractor/_exceptions.py` | `tractor._exceptions` |
|
||||
| `tractor/_state.py` | `tractor._state` |
|
||||
| `tests/test_foo.py` | `tests.test_foo` |
|
||||
| Function in module | `tractor._exceptions.func()` |
|
||||
| Method on class | `.RemoteActorError.src_type` |
|
||||
| Class | `tractor._exceptions.RAE` |
|
||||
|
||||
Prefix with the package path for top-level refs;
|
||||
use leading-dot shorthand (`.ClassName.method()`)
|
||||
for sub-bullets where the parent module is already
|
||||
established.
|
||||
|
||||
## Title Conventions
|
||||
|
||||
Same verb vocabulary as commit messages:
|
||||
- `Add` — wholly new feature/API
|
||||
- `Fix` — bug fix
|
||||
- `Drop` — removal
|
||||
- `Use` — adopt new approach
|
||||
- `Move`/`Mv` — relocate code
|
||||
- `Adjust` — minor tweak
|
||||
- `Update` — enhance existing feature
|
||||
- `Support` — add support for something
|
||||
|
||||
Target 50 chars, hard max 70. Always backtick
|
||||
code elements.
|
||||
|
||||
## Tone
|
||||
|
||||
Casual yet technically precise — matching the
|
||||
project's commit-msg style. Terse but every bullet
|
||||
carries signal. Use project abbreviations freely
|
||||
(msg, bg, ctx, impl, mod, obvi, fn, bc, var,
|
||||
prolly, ep, etc.).
|
||||
|
||||
---
|
||||
|
||||
(this format reference was generated by
|
||||
[`claude-code`][claude-code-gh])
|
||||
[claude-code-gh]: https://github.com/anthropics/claude-code
|
||||
|
|
@ -1,632 +0,0 @@
|
|||
---
|
||||
name: run-tests
|
||||
description: >
|
||||
Run tractor test suite (or subsets). Use when the user wants
|
||||
to run tests, verify changes, or check for regressions.
|
||||
argument-hint: "[test-path-or-pattern] [--opts]"
|
||||
allowed-tools:
|
||||
- Bash(python -m pytest *)
|
||||
- Bash(python -c *)
|
||||
- Bash(python --version *)
|
||||
- Bash(UV_PROJECT_ENVIRONMENT=py* uv run python *)
|
||||
- Bash(UV_PROJECT_ENVIRONMENT=py* uv run pytest *)
|
||||
- Bash(UV_PROJECT_ENVIRONMENT=py* uv sync *)
|
||||
- Bash(UV_PROJECT_ENVIRONMENT=py* uv pip show *)
|
||||
- Bash(git rev-parse *)
|
||||
- Bash(ls *)
|
||||
- Bash(cat *)
|
||||
- Bash(jq * .pytest_cache/*)
|
||||
# process inspection + SIGINT-first cleanup ladder (see
|
||||
# the zombie-actor pre-flight / teardown steps below).
|
||||
- Bash(ss *)
|
||||
- Bash(pgrep *)
|
||||
- Bash(pkill *)
|
||||
- Bash(sleep *)
|
||||
- Bash(rm -f /tmp/registry@*.sock)
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
- Task
|
||||
- AskUserQuestion
|
||||
---
|
||||
|
||||
Run the `tractor` test suite using `pytest`. Follow this
|
||||
process:
|
||||
|
||||
## 1. Parse user intent
|
||||
|
||||
From the user's message and any arguments, determine:
|
||||
|
||||
- **scope**: full suite, specific file(s), specific
|
||||
test(s), or a keyword pattern (`-k`).
|
||||
- **transport**: which IPC transport protocol to test
|
||||
against (default: `tcp`, also: `uds`).
|
||||
- **options**: any extra pytest flags the user wants
|
||||
(e.g. `--ll debug`, `--tpdb`, `-x`, `-v`).
|
||||
|
||||
If the user provides a bare path or pattern as argument,
|
||||
treat it as the test target. Examples:
|
||||
|
||||
- `/run-tests` → full suite
|
||||
- `/run-tests test_local.py` → single file
|
||||
- `/run-tests test_registrar -v` → file + verbose
|
||||
- `/run-tests -k cancel` → keyword filter
|
||||
- `/run-tests tests/ipc/ --tpt-proto uds` → subdir + UDS
|
||||
|
||||
## 2. Construct the pytest command
|
||||
|
||||
Base command:
|
||||
```
|
||||
python -m pytest
|
||||
```
|
||||
|
||||
### Default flags (always include unless user overrides):
|
||||
- `-x` (stop on first failure)
|
||||
- `--tb=short` (concise tracebacks)
|
||||
- `--no-header` (reduce noise)
|
||||
|
||||
### Path resolution:
|
||||
- If the user gives a bare filename like `test_local.py`,
|
||||
resolve it under `tests/`.
|
||||
- If the user gives a subdirectory like `ipc/`, resolve
|
||||
under `tests/ipc/`.
|
||||
- Glob if needed: `tests/**/test_*<pattern>*.py`
|
||||
|
||||
### Key pytest options for this project:
|
||||
|
||||
| Flag | Purpose |
|
||||
|---|---|
|
||||
| `--ll <level>` | Set tractor log level (e.g. `debug`, `info`, `runtime`) |
|
||||
| `--tpdb` / `--debug-mode` | Enable tractor's multi-proc debugger |
|
||||
| `--tpt-proto <key>` | IPC transport: `tcp` (default) or `uds` |
|
||||
| `--spawn-backend <be>` | Spawn method: `trio` (default), `mp_spawn`, `mp_forkserver` |
|
||||
| `-k <expr>` | pytest keyword filter |
|
||||
| `-v` / `-vv` | Verbosity |
|
||||
| `-s` | No output capture (useful with `--tpdb`) |
|
||||
|
||||
### Common combos:
|
||||
```sh
|
||||
# quick smoke test of core modules
|
||||
python -m pytest tests/test_local.py tests/test_rpc.py -x --tb=short --no-header
|
||||
|
||||
# full suite, stop on first failure
|
||||
python -m pytest tests/ -x --tb=short --no-header
|
||||
|
||||
# specific test with debug
|
||||
python -m pytest tests/discovery/test_registrar.py::test_reg_then_unreg -x -s --tpdb --ll debug
|
||||
|
||||
# run with UDS transport
|
||||
python -m pytest tests/ -x --tb=short --no-header --tpt-proto uds
|
||||
|
||||
# keyword filter
|
||||
python -m pytest tests/ -x --tb=short --no-header -k "cancel and not slow"
|
||||
```
|
||||
|
||||
## 3. Pre-flight: venv detection (MANDATORY)
|
||||
|
||||
**Always verify a `uv` venv is active before running
|
||||
`python` or `pytest`.** This project uses
|
||||
`UV_PROJECT_ENVIRONMENT=py<MINOR>` naming (e.g.
|
||||
`py313`) — never `.venv`.
|
||||
|
||||
### Step 1: detect active venv
|
||||
|
||||
Run this check first:
|
||||
|
||||
```sh
|
||||
python -c "
|
||||
import sys, os
|
||||
venv = os.environ.get('VIRTUAL_ENV', '')
|
||||
prefix = sys.prefix
|
||||
print(f'VIRTUAL_ENV={venv}')
|
||||
print(f'sys.prefix={prefix}')
|
||||
print(f'executable={sys.executable}')
|
||||
"
|
||||
```
|
||||
|
||||
### Step 2: interpret results
|
||||
|
||||
**Case A — venv is active** (`VIRTUAL_ENV` is set
|
||||
and points to a `py<MINOR>/` dir under the project
|
||||
root or worktree):
|
||||
|
||||
Use bare `python` / `python -m pytest` for all
|
||||
commands. This is the normal, fast path.
|
||||
|
||||
**Case B — no venv active** (`VIRTUAL_ENV` is empty
|
||||
or `sys.prefix` points to a system Python):
|
||||
|
||||
Use `AskUserQuestion` to ask the user:
|
||||
|
||||
> "No uv venv is active. Should I activate one
|
||||
> via `UV_PROJECT_ENVIRONMENT=py<MINOR> uv sync`,
|
||||
> or would you prefer to activate your shell venv
|
||||
> first?"
|
||||
|
||||
Options:
|
||||
1. **"Create/sync venv"** — run
|
||||
`UV_PROJECT_ENVIRONMENT=py<MINOR> uv sync` where
|
||||
`<MINOR>` is detected from `python --version`
|
||||
(e.g. `313` for 3.13). Then use
|
||||
`py<MINOR>/bin/python` for all subsequent
|
||||
commands in this session.
|
||||
2. **"I'll activate it myself"** — stop and let the
|
||||
user `source py<MINOR>/bin/activate` or similar.
|
||||
|
||||
**Case C — inside a git worktree** (`git rev-parse
|
||||
--git-common-dir` differs from `--git-dir`):
|
||||
|
||||
Verify Python resolves from the **worktree's own
|
||||
venv**, not the main repo's:
|
||||
|
||||
```sh
|
||||
python -c "import tractor; print(tractor.__file__)"
|
||||
```
|
||||
|
||||
If the path points outside the worktree, create a
|
||||
worktree-local venv:
|
||||
|
||||
```sh
|
||||
UV_PROJECT_ENVIRONMENT=py<MINOR> uv sync
|
||||
```
|
||||
|
||||
Then use `py<MINOR>/bin/python` for all commands.
|
||||
|
||||
**Why this matters**: without the correct venv,
|
||||
subprocesses spawned by tractor resolve modules
|
||||
from the wrong editable install, causing spurious
|
||||
`AttributeError` / `ModuleNotFoundError`.
|
||||
|
||||
### Fallback: `uv run`
|
||||
|
||||
If the user can't or won't activate a venv, all
|
||||
`python` and `pytest` commands can be prefixed
|
||||
with `UV_PROJECT_ENVIRONMENT=py<MINOR> uv run`:
|
||||
|
||||
```sh
|
||||
# instead of: python -m pytest tests/ -x
|
||||
UV_PROJECT_ENVIRONMENT=py313 uv run pytest tests/ -x
|
||||
|
||||
# instead of: python -c 'import tractor'
|
||||
UV_PROJECT_ENVIRONMENT=py313 uv run python -c 'import tractor'
|
||||
```
|
||||
|
||||
`uv run` auto-discovers the project and venv,
|
||||
but is slower than a pre-activated venv due to
|
||||
lock-file resolution on each invocation. Prefer
|
||||
activating the venv when possible.
|
||||
|
||||
### Step 3: import + collection checks
|
||||
|
||||
After venv is confirmed, always run these
|
||||
(especially after refactors or module moves):
|
||||
|
||||
```sh
|
||||
# 1. package import smoke check
|
||||
python -c 'import tractor; print(tractor)'
|
||||
|
||||
# 2. verify all tests collect (no import errors)
|
||||
python -m pytest tests/ -x -q --co 2>&1 | tail -5
|
||||
```
|
||||
|
||||
If either fails, fix the import error before running
|
||||
any actual tests.
|
||||
|
||||
### Step 4: zombie-actor / stale-registry check (MANDATORY)
|
||||
|
||||
The tractor runtime's default registry address is
|
||||
**`127.0.0.1:1616`** (TCP) / `/tmp/registry@1616.sock`
|
||||
(UDS). Whenever any prior test run — especially one
|
||||
using a fork-based backend like `subint_forkserver` —
|
||||
leaks a child actor process, that zombie keeps the
|
||||
registry port bound and **every subsequent test
|
||||
session fails to bind**, often presenting as 50+
|
||||
unrelated failures ("all tests broken"!) across
|
||||
backends.
|
||||
|
||||
**This has to be checked before the first run AND
|
||||
after any cancelled/SIGINT'd run** — signal failures
|
||||
in the middle of a test can leave orphan children.
|
||||
|
||||
```sh
|
||||
# 1. TCP registry — any listener on :1616? (primary signal)
|
||||
ss -tlnp 2>/dev/null | grep ':1616' || echo 'TCP :1616 free'
|
||||
|
||||
# 2. leftover actor/forkserver procs — scoped to THIS
|
||||
# repo's python path, so we don't false-flag legit
|
||||
# long-running tractor-using apps (e.g. `piker`,
|
||||
# downstream projects that embed tractor).
|
||||
pgrep -af "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv" \
|
||||
| grep -v 'grep\|pgrep' \
|
||||
|| echo 'no leaked actor procs from this repo'
|
||||
|
||||
# 3. stale UDS registry sockets
|
||||
ls -la /tmp/registry@*.sock 2>/dev/null \
|
||||
|| echo 'no leaked UDS registry sockets'
|
||||
```
|
||||
|
||||
**Interpretation:**
|
||||
|
||||
- **TCP :1616 free AND no stale sockets** → clean,
|
||||
proceed. The actor-procs probe is secondary — false
|
||||
positives are common (piker, any other tractor-
|
||||
embedding app); only cleanup if `:1616` is bound or
|
||||
sockets linger.
|
||||
- **TCP :1616 bound OR stale sockets present** →
|
||||
surface PIDs + cmdlines to the user, offer cleanup:
|
||||
|
||||
```sh
|
||||
# 1. GRACEFUL FIRST (tractor is structured concurrent — it
|
||||
# catches SIGINT as an OS-cancel in `_trio_main` and
|
||||
# cascades Portal.cancel_actor via IPC to every descendant.
|
||||
# So always try SIGINT first with a bounded timeout; only
|
||||
# escalate to SIGKILL if graceful cleanup doesn't complete).
|
||||
pkill -INT -f "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv"
|
||||
|
||||
# 2. bounded wait for graceful teardown (usually sub-second).
|
||||
# Loop until the processes exit, or timeout. Keep the
|
||||
# bound tight — hung/abrupt-killed descendants usually
|
||||
# hang forever, so don't wait more than a few seconds.
|
||||
for i in $(seq 1 10); do
|
||||
pgrep -f "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv" >/dev/null || break
|
||||
sleep 0.3
|
||||
done
|
||||
|
||||
# 3. ESCALATE TO SIGKILL only if graceful didn't finish.
|
||||
if pgrep -f "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv" >/dev/null; then
|
||||
echo 'graceful teardown timed out — escalating to SIGKILL'
|
||||
pkill -9 -f "$(pwd)/py[0-9]*/bin/python.*_actor_child_main|subint-forkserv"
|
||||
fi
|
||||
|
||||
# 4. if a test zombie holds :1616 specifically and doesn't
|
||||
# match the above pattern, find its PID the hard way:
|
||||
ss -tlnp 2>/dev/null | grep ':1616' # prints `users:(("<name>",pid=NNNN,...))`
|
||||
# then (same SIGINT-first ladder):
|
||||
# kill -INT <NNNN>; sleep 1; kill -9 <NNNN> 2>/dev/null
|
||||
|
||||
# 5. remove stale UDS sockets
|
||||
rm -f /tmp/registry@*.sock
|
||||
|
||||
# 6. re-verify
|
||||
ss -tlnp 2>/dev/null | grep ':1616' || echo 'TCP :1616 now free'
|
||||
```
|
||||
|
||||
**Never ignore stale registry state.** If you see the
|
||||
"all tests failing" pattern — especially
|
||||
`trio.TooSlowError` / connection refused / address in
|
||||
use on many unrelated tests — check registry **before**
|
||||
spelunking into test code. The failure signature will
|
||||
be identical across backends because they're all
|
||||
fighting for the same port.
|
||||
|
||||
**False-positive warning for step 2:** a plain
|
||||
`pgrep -af '_actor_child_main'` will also match
|
||||
legit long-running tractor-embedding apps (e.g.
|
||||
`piker` at `~/repos/piker/py*/bin/python3 -m
|
||||
tractor._child ...`). Always scope to the current
|
||||
repo's python path, or only use step 1 (`:1616`) as
|
||||
the authoritative signal.
|
||||
|
||||
## 4. Run and report
|
||||
|
||||
- Run the constructed command.
|
||||
- Use a timeout of **600000ms** (10min) for full suite
|
||||
runs, **120000ms** (2min) for single-file runs.
|
||||
- If the suite is large (full `tests/`), consider running
|
||||
in the background and checking output when done.
|
||||
- Use `--lf` (last-failed) to re-run only previously
|
||||
failing tests when iterating on a fix.
|
||||
|
||||
### On failure:
|
||||
- Show the failing test name(s) and short traceback.
|
||||
- If the failure looks related to recent changes, point
|
||||
out the likely cause and suggest a fix.
|
||||
- **Check the known-flaky list** (section 8) before
|
||||
investigating — don't waste time on pre-existing
|
||||
timeout issues.
|
||||
- **NEVER auto-commit fixes.** If you apply a code fix
|
||||
during test iteration, leave it unstaged. Tell the
|
||||
user what changed and suggest they review the
|
||||
worktree state, stage files manually, and use
|
||||
`/commit-msg` (inline or in a separate session) to
|
||||
generate the commit message. The human drives all
|
||||
`git add` and `git commit` operations.
|
||||
|
||||
### On success:
|
||||
- Report the pass/fail/skip counts concisely.
|
||||
|
||||
## 5. Test directory layout (reference)
|
||||
|
||||
```
|
||||
tests/
|
||||
├── conftest.py # root fixtures, daemon, signals
|
||||
├── devx/ # debugger/tooling tests
|
||||
├── ipc/ # transport protocol tests
|
||||
├── msg/ # messaging layer tests
|
||||
├── discovery/ # discovery subsystem tests
|
||||
│ ├── test_multiaddr.py # multiaddr construction
|
||||
│ └── test_registrar.py # registry/discovery protocol
|
||||
├── test_local.py # registrar + local actor basics
|
||||
├── test_rpc.py # RPC error handling
|
||||
├── test_spawning.py # subprocess spawning
|
||||
├── test_multi_program.py # multi-process tree tests
|
||||
├── test_cancellation.py # cancellation semantics
|
||||
├── test_context_stream_semantics.py # ctx streaming
|
||||
├── test_inter_peer_cancellation.py # peer cancel
|
||||
├── test_infected_asyncio.py # trio-in-asyncio
|
||||
└── ...
|
||||
```
|
||||
|
||||
## 6. Change-type → test mapping
|
||||
|
||||
After modifying specific modules, run the corresponding
|
||||
test subset first for fast feedback:
|
||||
|
||||
| Changed module(s) | Run these tests first |
|
||||
|---|---|
|
||||
| `runtime/_runtime.py`, `runtime/_state.py` | `test_local.py test_rpc.py test_spawning.py test_root_runtime.py` |
|
||||
| `discovery/` (`_registry`, `_discovery`, `_addr`) | `tests/discovery/ test_multi_program.py test_local.py` |
|
||||
| `_context.py`, `_streaming.py` | `test_context_stream_semantics.py test_advanced_streaming.py` |
|
||||
| `ipc/` (`_chan`, `_server`, `_transport`) | `tests/ipc/ test_2way.py` |
|
||||
| `runtime/_portal.py`, `runtime/_rpc.py` | `test_rpc.py test_cancellation.py` |
|
||||
| `spawn/` (`_spawn`, `_entry`) | `test_spawning.py test_multi_program.py` |
|
||||
| `devx/debug/` | `tests/devx/test_debugger.py` (slow!) |
|
||||
| `to_asyncio.py` | `test_infected_asyncio.py test_root_infect_asyncio.py` |
|
||||
| `msg/` | `tests/msg/` |
|
||||
| `_exceptions.py` | `test_remote_exc_relay.py test_inter_peer_cancellation.py` |
|
||||
| `runtime/_supervise.py` | `test_cancellation.py test_spawning.py` |
|
||||
|
||||
## 7. Quick-check shortcuts
|
||||
|
||||
### After refactors (fastest first-pass):
|
||||
```sh
|
||||
# import + collect check
|
||||
python -c 'import tractor' && python -m pytest tests/ -x -q --co 2>&1 | tail -3
|
||||
|
||||
# core subset (~10s)
|
||||
python -m pytest tests/test_local.py tests/test_rpc.py tests/test_spawning.py tests/discovery/test_registrar.py -x --tb=short --no-header
|
||||
```
|
||||
|
||||
### Inspect last failures (without re-running):
|
||||
|
||||
When the user asks "what failed?", "show failures",
|
||||
or wants to check the last-failed set before
|
||||
re-running — read the pytest cache directly. This
|
||||
is instant and avoids test collection overhead.
|
||||
|
||||
```sh
|
||||
python -c "
|
||||
import json, pathlib, sys
|
||||
p = pathlib.Path('.pytest_cache/v/cache/lastfailed')
|
||||
if not p.exists():
|
||||
print('No lastfailed cache found.'); sys.exit()
|
||||
data = json.loads(p.read_text())
|
||||
# filter to real test node IDs (ignore junk
|
||||
# entries that can accumulate from system paths)
|
||||
tests = sorted(k for k in data if k.startswith('tests/'))
|
||||
if not tests:
|
||||
print('No failures recorded.')
|
||||
else:
|
||||
print(f'{len(tests)} last-failed test(s):')
|
||||
for t in tests:
|
||||
print(f' {t}')
|
||||
"
|
||||
```
|
||||
|
||||
**Why not `--cache-show` or `--co --lf`?**
|
||||
|
||||
- `pytest --cache-show 'cache/lastfailed'` works
|
||||
but dumps raw dict repr including junk entries
|
||||
(stale system paths that leak into the cache).
|
||||
- `pytest --co --lf` actually *collects* tests which
|
||||
triggers import resolution and is slow (~0.5s+).
|
||||
Worse, when cached node IDs don't exactly match
|
||||
current parametrize IDs (e.g. param names changed
|
||||
between runs), pytest falls back to collecting
|
||||
the *entire file*, giving false positives.
|
||||
- Reading the JSON directly is instant, filterable
|
||||
to `tests/`-prefixed entries, and shows exactly
|
||||
what pytest recorded — no interpretation.
|
||||
|
||||
**After inspecting**, re-run the failures:
|
||||
```sh
|
||||
python -m pytest --lf -x --tb=short --no-header
|
||||
```
|
||||
|
||||
### Full suite in background:
|
||||
When core tests pass and you want full coverage while
|
||||
continuing other work, run in background:
|
||||
```sh
|
||||
python -m pytest tests/ -x --tb=short --no-header -q
|
||||
```
|
||||
(use `run_in_background=true` on the Bash tool)
|
||||
|
||||
## 8. Known flaky tests
|
||||
|
||||
These tests have **pre-existing** timing/environment
|
||||
sensitivity. If they fail with `TooSlowError` or
|
||||
pexpect `TIMEOUT`, they are almost certainly NOT caused
|
||||
by your changes — note them and move on.
|
||||
|
||||
| Test | Typical error | Notes |
|
||||
|---|---|---|
|
||||
| `devx/test_debugger.py::test_multi_nested_subactors_error_through_nurseries` | pexpect TIMEOUT | Debugger pexpect timing |
|
||||
| `test_cancellation.py::test_cancel_via_SIGINT_other_task` | TooSlowError | Signal handling race |
|
||||
| `test_inter_peer_cancellation.py::test_peer_spawns_and_cancels_service_subactor` | TooSlowError | Async timing (both param variants) |
|
||||
| `test_docs_examples.py::test_example[we_are_processes.py]` | `assert None == 0` | `__main__` missing `__file__` in subproc |
|
||||
|
||||
**Rule of thumb**: if a test fails with `TooSlowError`,
|
||||
`trio.TooSlowError`, or `pexpect.TIMEOUT` and you didn't
|
||||
touch the relevant code path, it's flaky — skip it.
|
||||
|
||||
## 9. The pytest-capture hang pattern (CHECK THIS FIRST)
|
||||
|
||||
**Symptom:** a tractor test hangs indefinitely under
|
||||
default `pytest` but passes instantly when you add
|
||||
`-s` (`--capture=no`).
|
||||
|
||||
**Cause:** tractor subactors (especially under fork-
|
||||
based backends) inherit pytest's stdout/stderr
|
||||
capture pipes via fds 1,2. Under high-volume error
|
||||
logging (e.g. multi-level cancel cascade, nested
|
||||
`run_in_actor` failures, anything triggering
|
||||
`RemoteActorError` + `ExceptionGroup` traceback
|
||||
spew), the **64KB Linux pipe buffer fills** faster
|
||||
than pytest drains it. Subactor writes block → can't
|
||||
finish exit → parent's `waitpid`/pidfd wait blocks →
|
||||
deadlock cascades up the tree.
|
||||
|
||||
**Pre-existing guards in the tractor harness** that
|
||||
encode this same knowledge — grep these FIRST
|
||||
before spelunking:
|
||||
|
||||
- `tests/conftest.py:258-260` (in the `daemon`
|
||||
fixture): `# XXX: too much logging will lock up
|
||||
the subproc (smh)` — downgrades `trace`/`debug`
|
||||
loglevel to `info` to prevent the hang.
|
||||
- `tests/conftest.py:316`: `# can lock up on the
|
||||
_io.BufferedReader and hang..` — noted on the
|
||||
`proc.stderr.read()` post-SIGINT.
|
||||
|
||||
**Debug recipe (in priority order):**
|
||||
|
||||
1. **Try `-s` first.** If the hang disappears with
|
||||
`pytest -s`, you've confirmed it's capture-pipe
|
||||
fill. Skip spelunking.
|
||||
2. **Lower the loglevel.** Default `--ll=error` on
|
||||
this project; if you've bumped it to `debug` /
|
||||
`info`, try dropping back. Each log level
|
||||
multiplies pipe-pressure under fault cascades.
|
||||
3. **If you MUST use default capture + high log
|
||||
volume**, redirect subactor stdout/stderr in the
|
||||
child prelude (e.g.
|
||||
`tractor.spawn._subint_forkserver._child_target`
|
||||
post-`_close_inherited_fds`) to `/dev/null` or a
|
||||
file.
|
||||
|
||||
**Signature tells you it's THIS bug (vs. a real
|
||||
code hang):**
|
||||
|
||||
- Multi-actor test under fork-based backend
|
||||
(`subint_forkserver`, eventually `trio_proc` too
|
||||
under enough log volume).
|
||||
- Multiple `RemoteActorError` / `ExceptionGroup`
|
||||
tracebacks in the error path.
|
||||
- Test passes with `-s` in the 5-10s range, hangs
|
||||
past pytest-timeout (usually 30+ s) without `-s`.
|
||||
- Subactor processes visible via `pgrep -af
|
||||
subint-forkserv` or similar after the hang —
|
||||
they're alive but blocked on `write()` to an
|
||||
inherited stdout fd.
|
||||
|
||||
**Historical reference:** this deadlock cost a
|
||||
multi-session investigation (4 genuine cascade
|
||||
fixes landed along the way) that only surfaced the
|
||||
capture-pipe issue AFTER the deeper fixes let the
|
||||
tree actually tear down enough to produce pipe-
|
||||
filling log volume. Full post-mortem in
|
||||
`ai/conc-anal/subint_forkserver_test_cancellation_leak_issue.md`.
|
||||
Lesson codified here so future-me grep-finds the
|
||||
workaround before digging.
|
||||
|
||||
## 10. Reaping zombie subactors (`tractor-reap`)
|
||||
|
||||
**Symptom:** after a `pytest` run crashes, times out,
|
||||
or is `Ctrl+C`'d, subactor forks (esp. under
|
||||
`subint_forkserver`) can be reparented to `init`
|
||||
(PPid==1) and linger. They hold onto ports, inherit
|
||||
pytest's capture-pipe fds, and flakify later
|
||||
sessions.
|
||||
|
||||
**Two layers of defense:**
|
||||
|
||||
### a) Session-scoped auto-fixture (always on)
|
||||
|
||||
`tractor/_testing/pytest.py::_reap_orphaned_subactors`
|
||||
runs at pytest session teardown. It walks `/proc` for
|
||||
direct descendants of the pytest pid, SIGINTs them,
|
||||
waits up to 3s, then SIGKILLs survivors. SC-polite:
|
||||
gives the subactor runtime a chance to run its trio
|
||||
cancel shield + IPC teardown before escalation.
|
||||
|
||||
This is *autouse* and session-scoped — you don't need
|
||||
to do anything. It just runs.
|
||||
|
||||
### b) `scripts/tractor-reap` CLI (manual reap)
|
||||
|
||||
For the **pytest-died-mid-session** case (Ctrl+C, OOM
|
||||
kill, hung process you had to `kill -9`), the fixture
|
||||
never ran. Reach for the CLI:
|
||||
|
||||
```sh
|
||||
# default: orphans (PPid==1, cwd==repo, cmd contains python)
|
||||
scripts/tractor-reap
|
||||
|
||||
# descendant-mode: from a still-live supervisor
|
||||
scripts/tractor-reap --parent <pytest-pid>
|
||||
|
||||
# see what would be reaped, don't signal
|
||||
scripts/tractor-reap -n
|
||||
|
||||
# tune the SIGINT → SIGKILL grace window
|
||||
scripts/tractor-reap --grace 5
|
||||
```
|
||||
|
||||
Exit code: `0` if everyone exited on SIGINT, `1` if
|
||||
SIGKILL had to escalate — so you can chain it in CI
|
||||
health-checks (`scripts/tractor-reap || <alert>`).
|
||||
|
||||
**What it matches** (orphan-mode):
|
||||
- `PPid == 1` (reparented to init → definitely
|
||||
orphaned, not just a currently-running child)
|
||||
- `cwd == <repo-root>` (keeps the sweep scoped; won't
|
||||
touch unrelated init-children elsewhere)
|
||||
- `python` in cmdline
|
||||
|
||||
**What it does not do:** kill anything whose PPid is
|
||||
still a live tractor parent. If the parent is alive
|
||||
it's not an orphan; use `--parent <pid>` if you need
|
||||
to force-reap under a still-live supervisor.
|
||||
|
||||
**When NOT to run it:** while a pytest session is
|
||||
active in another terminal. It's safe (won't touch
|
||||
that session's live children in orphan-mode) but can
|
||||
race if the target session is mid-teardown.
|
||||
|
||||
### c) `--shm` / `--shm-only`: orphan-segment sweep
|
||||
|
||||
Because `tractor.ipc._mp_bs.disable_mantracker()`
|
||||
turns off `mp.resource_tracker` (see
|
||||
`ai/conc-anal/subint_forkserver_mp_shared_memory_issue.md`),
|
||||
a hard-crashing actor can leave `/dev/shm/<key>`
|
||||
segments behind that nothing else GCs.
|
||||
|
||||
```sh
|
||||
# process reap THEN shm sweep
|
||||
scripts/tractor-reap --shm
|
||||
|
||||
# shm sweep only (skip process phase)
|
||||
scripts/tractor-reap --shm-only
|
||||
|
||||
# dry-run: list candidates, don't unlink
|
||||
scripts/tractor-reap --shm -n
|
||||
```
|
||||
|
||||
**Match criteria** (very conservative — this is a
|
||||
shared-system path, can't be wrong):
|
||||
- segment is a regular file under `/dev/shm`,
|
||||
- owned by the **current uid** (`stat.st_uid`),
|
||||
- AND **no live process holds it open** —
|
||||
enumerated by walking every readable
|
||||
`/proc/<pid>/maps` (post-mmap mappings) AND
|
||||
`/proc/<pid>/fd/*` (pre-mmap shm-opened fds).
|
||||
|
||||
The "nobody has it open" check is the
|
||||
kernel-canonical "is this leaked?" test — same
|
||||
answer `lsof /dev/shm/<key>` would give. No
|
||||
reliance on tractor-specific naming, so it works
|
||||
for any tractor app. Critically, it WILL NOT touch
|
||||
segments held by other apps you have running
|
||||
(e.g. `piker`, `lttng-ust-*`, `aja-shm-*` —
|
||||
verified locally with 81 in-use segments correctly
|
||||
preserved).
|
||||
|
|
@ -1,18 +1,10 @@
|
|||
name: CI
|
||||
|
||||
# NOTE distilled from,
|
||||
# https://github.com/orgs/community/discussions/26276
|
||||
on:
|
||||
# any time a new update to 'main'
|
||||
# any time someone pushes a new branch to origin
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
|
||||
# for on all (forked) PRs to repo
|
||||
# NOTE, use a draft PR if you just want CI triggered..
|
||||
pull_request:
|
||||
|
||||
# to run workflow manually from the "Actions" tab
|
||||
# Allows you to run this workflow manually from the Actions tab
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
|
|
@ -37,12 +29,7 @@ jobs:
|
|||
run: uv build --sdist --python=3.13
|
||||
|
||||
- name: Install sdist from .tar.gz
|
||||
# XXX must install under py3.13 (matching the build's
|
||||
# `--python=3.13`); the runner's default `python` is 3.12
|
||||
# which our `requires-python = ">=3.13"` now rejects.
|
||||
run: |
|
||||
uv venv --python 3.13
|
||||
uv pip install dist/*.tar.gz
|
||||
run: python -m pip install dist/*.tar.gz
|
||||
|
||||
# ------ type-check ------
|
||||
# mypy:
|
||||
|
|
@ -87,81 +74,24 @@ jobs:
|
|||
# run: mypy tractor/ --ignore-missing-imports --show-traceback
|
||||
|
||||
|
||||
testing:
|
||||
name: '${{ matrix.os }} Python${{ matrix.python-version }} spawn_backend=${{ matrix.spawn_backend }} tpt_proto=${{ matrix.tpt_proto }} capture=${{ matrix.capture }}'
|
||||
timeout-minutes: 20
|
||||
testing-linux:
|
||||
name: '${{ matrix.os }} Python ${{ matrix.python }} - ${{ matrix.spawn_backend }}'
|
||||
timeout-minutes: 10
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
# NOTE on the matrix shape — the `capture=` mode follows
|
||||
# `spawn_backend`:
|
||||
#
|
||||
# - `trio` / `mp_*` backends use `--capture=fd` (default)
|
||||
# for per-test attribution of subactor *raw-fd* output
|
||||
# in failure reports.
|
||||
# - Fork-based backends (`main_thread_forkserver`,
|
||||
# `subint_forkserver`) REQUIRE `--capture=sys` because
|
||||
# fork-child × `--capture=fd` is a known deadlock
|
||||
# pattern. See the long NOTE in `tractor._testing.pytest`'s
|
||||
# `pytest_load_initial_conftests` for the mechanism +
|
||||
# tradeoff write-up.
|
||||
#
|
||||
# If a future matrix row adds a fork-spawn backend
|
||||
# WITHOUT setting `capture: 'sys'`, the
|
||||
# `pytest_load_initial_conftests` hook fail-fasts on `CI=1`
|
||||
# with a clear error msg. So the matrix is self-policing.
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [
|
||||
ubuntu-latest,
|
||||
macos-latest,
|
||||
]
|
||||
python-version: [
|
||||
'3.13',
|
||||
# '3.14',
|
||||
]
|
||||
os: [ubuntu-latest]
|
||||
python-version: ['3.13']
|
||||
spawn_backend: [
|
||||
'trio',
|
||||
# 'mp_spawn',
|
||||
# 'mp_forkserver',
|
||||
# ?TODO^ is it worth it to get these running again?
|
||||
#
|
||||
# - [ ] next-gen backends, on 3.13+
|
||||
# https://github.com/goodboy/tractor/issues/379
|
||||
# 'subinterpreter',
|
||||
# 'subint',
|
||||
]
|
||||
tpt_proto: [
|
||||
'tcp',
|
||||
'uds',
|
||||
]
|
||||
capture: [
|
||||
'fd', # default for non-fork backends
|
||||
]
|
||||
|
||||
# Fork-based backends — added via `include:` so each
|
||||
# cell carries its REQUIRED `capture: 'sys'` mode.
|
||||
# Linux-only for now; macOS coverage TBD pending
|
||||
# local validation.
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
python-version: '3.13'
|
||||
spawn_backend: 'main_thread_forkserver'
|
||||
tpt_proto: 'tcp'
|
||||
capture: 'sys'
|
||||
- os: ubuntu-latest
|
||||
python-version: '3.13'
|
||||
spawn_backend: 'main_thread_forkserver'
|
||||
tpt_proto: 'uds'
|
||||
capture: 'sys'
|
||||
|
||||
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
|
||||
exclude:
|
||||
# don't do UDS run on macOS (for now)
|
||||
- os: macos-latest
|
||||
tpt_proto: 'uds'
|
||||
|
||||
steps:
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: 'Install uv + py-${{ matrix.python-version }}'
|
||||
|
|
@ -188,18 +118,7 @@ jobs:
|
|||
run: uv tree
|
||||
|
||||
- name: Run tests
|
||||
run: >
|
||||
uv run
|
||||
pytest
|
||||
tests/
|
||||
-rsx
|
||||
--spawn-backend=${{ matrix.spawn_backend }}
|
||||
--tpt-proto=${{ matrix.tpt_proto }}
|
||||
--capture=${{ matrix.capture }}
|
||||
# NOTE: capture mode is matrix-driven — `fd` for
|
||||
# non-fork backends (per-test fd attribution),
|
||||
# `sys` for fork-based (avoids fork-child x
|
||||
# capture-fd deadlock). See matrix-NOTE above.
|
||||
run: uv run pytest tests/ --spawn-backend=${{ matrix.spawn_backend }} -rsx
|
||||
|
||||
# XXX legacy NOTE XXX
|
||||
#
|
||||
|
|
|
|||
|
|
@ -1,63 +0,0 @@
|
|||
name: docs
|
||||
|
||||
# build sphinx docs on every PR + push to main;
|
||||
# deploy to gh-pages only from main pushes.
|
||||
# (see goodboy/tractor#123 for the original ask)
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
# to run workflow manually from the "Actions" tab
|
||||
workflow_dispatch:
|
||||
|
||||
# needed by actions/deploy-pages
|
||||
permissions:
|
||||
contents: read
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: 'sphinx build'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Install latest uv
|
||||
uses: astral-sh/setup-uv@v6
|
||||
|
||||
# NOTE, no `d2` bin is installed in CI (yet) so
|
||||
# the pre-rendered + committed SVGs under
|
||||
# `docs/_diagrams/` are used as-is; see
|
||||
# `docs/_ext/d2diagrams.py` for the fallback
|
||||
# policy.
|
||||
- name: Build html docs
|
||||
run: |
|
||||
uv sync --no-dev --group docs
|
||||
uv run --no-dev --group docs make -C docs html
|
||||
|
||||
- name: Upload pages artifact
|
||||
uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: docs/_build/html
|
||||
|
||||
deploy:
|
||||
name: 'deploy to gh-pages'
|
||||
if: ${{ github.ref == 'refs/heads/main' && github.event_name == 'push' }}
|
||||
needs: build
|
||||
# serialize deploys but NEVER cancel an in-flight one
|
||||
# mid-upload; queue the next instead (scoped to this job so
|
||||
# PR builds can't cancel a production deploy).
|
||||
concurrency:
|
||||
group: 'pages'
|
||||
cancel-in-progress: false
|
||||
runs-on: ubuntu-latest
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- name: Deploy
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
|
|
@ -102,69 +102,3 @@ venv.bak/
|
|||
|
||||
# mypy
|
||||
.mypy_cache/
|
||||
|
||||
# all files under
|
||||
.git/
|
||||
|
||||
# require very explicit staging for anything we **really**
|
||||
# want put/kept in repo.
|
||||
notes_to_self/
|
||||
snippets/
|
||||
|
||||
# ------- AI shiz -------
|
||||
# `ai.skillz` symlinks,
|
||||
# (machine-local, deploy via deploy-skill.sh)
|
||||
.claude/skills/py-codestyle
|
||||
.claude/skills/close-wkt
|
||||
.claude/skills/plan-io
|
||||
.claude/skills/prompt-io
|
||||
.claude/skills/resolve-conflicts
|
||||
.claude/skills/inter-skill-review
|
||||
|
||||
# /open-wkt specifics
|
||||
.claude/skills/open-wkt
|
||||
.claude/wkts/
|
||||
claude_wkts
|
||||
|
||||
# /code-review-changes specifics
|
||||
.claude/skills/code-review-changes
|
||||
# review-skill ephemeral ctx (per-PR, single-use)
|
||||
.claude/review_context.md
|
||||
.claude/review_regression.md
|
||||
|
||||
# /pr-msg specifics
|
||||
.claude/skills/pr-msg/*
|
||||
# repo-specific
|
||||
!.claude/skills/pr-msg/format-reference.md
|
||||
# XXX, so u can nvim-telescope this file.
|
||||
# !.claude/skills/pr-msg/pr_msg_LATEST.md
|
||||
|
||||
# /commit-msg specifics
|
||||
# - any commit-msg gen tmp files
|
||||
.claude/*_commit_*.md
|
||||
.claude/*_commit*.txt
|
||||
.claude/skills/commit-msg/*
|
||||
!.claude/skills/commit-msg/style-duie-reference.md
|
||||
|
||||
# use prompt-io instead?
|
||||
.claude/plans
|
||||
|
||||
# nix develop --profile .nixdev
|
||||
.nixdev*
|
||||
|
||||
# :Obsession .
|
||||
Session.vim
|
||||
|
||||
# `gish` local `.md`-files
|
||||
# TODO? better all around automation!
|
||||
# -[ ] it'd be handy to also commit and sync with wtv git service?
|
||||
# -[ ] everything should be put under a `.gish/` no?
|
||||
gitea/
|
||||
gh/
|
||||
|
||||
# ------ macOS ------
|
||||
# Finder metadata
|
||||
**/.DS_Store
|
||||
|
||||
# LLM conversations that should remain private
|
||||
docs/conversations/
|
||||
|
|
|
|||
1
NEWS.rst
1
NEWS.rst
|
|
@ -78,7 +78,6 @@ Bug Fixes
|
|||
for ``asyncio``-side errors to not propagate due to a race condition.
|
||||
|
||||
The implementation fix summary is:
|
||||
|
||||
- add state to signal the end of the ``trio`` side task to be
|
||||
read by the ``asyncio`` side and always cancel any ongoing
|
||||
task in such cases.
|
||||
|
|
|
|||
|
|
@ -1,314 +0,0 @@
|
|||
# Cancel-cascade `trio.TooSlowError` flakes under `main_thread_forkserver`
|
||||
|
||||
## Symptom
|
||||
|
||||
Running the full test suite under
|
||||
|
||||
```bash
|
||||
./py313/bin/python -m pytest tests/ \
|
||||
--tpt-proto=tcp \
|
||||
--spawn-backend=main_thread_forkserver
|
||||
```
|
||||
|
||||
surfaces a single, **rotating** `trio.TooSlowError`
|
||||
failure each run. The failure isn't deterministic on
|
||||
test identity — different test each run — but it
|
||||
ALWAYS looks like:
|
||||
|
||||
```
|
||||
FAILED tests/<file>::test_<name> - trio.TooSlowError
|
||||
==== 1 failed, 373 passed, 17 skipped, 11–12 xfailed,
|
||||
0–1 xpassed, ~550 warnings in ~6min ====
|
||||
```
|
||||
|
||||
Pass rate: **~99.7%** (373 of 374 non-skip tests).
|
||||
Wall-clock per full run: 5–6 min.
|
||||
|
||||
## Tests observed flaking so far
|
||||
|
||||
Each row was the SOLE failure in a separate run:
|
||||
|
||||
| run # | test |
|
||||
|---|---|
|
||||
| 1 | `tests/test_advanced_streaming.py::test_dynamic_pub_sub[KeyboardInterrupt]` |
|
||||
| 2 | `tests/test_infected_asyncio.py::test_context_spawns_aio_task_that_errors[parent_actor_cancels_child=False]` |
|
||||
|
||||
Both share the same shape:
|
||||
|
||||
- **Cancel cascade** of N subactors back to a parent root actor.
|
||||
- N ≥ `multiprocessing.cpu_count()` for `test_dynamic_pub_sub`
|
||||
(it spawns `cpus - 1` consumers + publisher + dynamic-consumer).
|
||||
- N ≈ 2 for `test_context_spawns_aio_task_that_errors` —
|
||||
but each subactor is `infect_asyncio=True`, so each
|
||||
cancel involves the trio↔asyncio guest-run unwind
|
||||
which is structurally heavier than pure-trio.
|
||||
- Test wraps the cascade in `trio.fail_after(N seconds)`
|
||||
and the cap fires before the cascade completes.
|
||||
|
||||
The exact failing test rotates because each test is
|
||||
independently close to the cap; whichever happens to
|
||||
be unlucky in scheduling/CPU-contention on a given run
|
||||
is the one that times out.
|
||||
|
||||
## Root-cause family
|
||||
|
||||
`hard_kill` (`tractor/spawn/_spawn.py:hard_kill`) runs
|
||||
the SC-graceful teardown ladder per subactor:
|
||||
|
||||
1. `Portal.cancel_actor()` — graceful IPC cancel-req.
|
||||
2. Wait `terminate_after=1.6s` for sub to exit.
|
||||
3. If still alive: `proc.kill()` (SIGKILL).
|
||||
4. (NEW) `_unlink_uds_bind_addrs()` — post-mortem
|
||||
sock-file cleanup for UDS leaks (issue #452 fix).
|
||||
|
||||
For a cascade of N subactors, each pays steps 1–4. If
|
||||
graceful-cancel doesn't complete within 1.6s for ANY
|
||||
sub, that sub eats a full 1.6s of `move_on_after` plus
|
||||
the `proc.wait()` post-SIGKILL.
|
||||
|
||||
Worst case under fork backend with N=cpus subs:
|
||||
- N × 1.6s = 16s+ on a 10-core box just for the
|
||||
graceful timeout phase
|
||||
- Plus per-spawn fork-IPC handshake cost compounds
|
||||
during teardown (each sub's IPC cleanup goes through
|
||||
the same forkserver coordinator)
|
||||
- Plus the new autouse fixtures
|
||||
(`_track_orphaned_uds_per_test`,
|
||||
`_detect_runaway_subactors_per_test`,
|
||||
`_reap_orphaned_subactors`) all run at test
|
||||
teardown, adding small (10s of ms) but cumulative
|
||||
overhead
|
||||
|
||||
Current cap: 30s (`fail_after_s = 30 if
|
||||
is_forking_spawner else 12`). Empirically fits the
|
||||
median run but the tail breaks ~0.3% of the time.
|
||||
|
||||
## NOT regressing
|
||||
|
||||
To confirm this is a flake and not a regression:
|
||||
|
||||
- Pre-`WakeupSocketpair`-patch baseline: tests
|
||||
HUNG INDEFINITELY (busy-loop never released).
|
||||
- Post-patch: pass-or-fail-fast, ~99.7% pass, the
|
||||
occasional cap-hit fails in bounded time (<60s for
|
||||
the offending test).
|
||||
- Same test PASSES under `--spawn-backend=trio`
|
||||
(no fork, no hard-kill compounding).
|
||||
|
||||
So the suite is dramatically better than before; the
|
||||
remaining flake is a known-tolerable steady-state.
|
||||
|
||||
## Possible mitigations (ranked)
|
||||
|
||||
### A. Bump the cap further
|
||||
|
||||
Cheapest. Change the per-test `fail_after_s` from 30
|
||||
to e.g. 60 for fork backends. Pros: trivial. Cons:
|
||||
masks any genuine slowness regression we'd want to
|
||||
catch.
|
||||
|
||||
### B. CPU-count-aware cap
|
||||
|
||||
For tests whose N scales with `cpu_count()`, scale
|
||||
the cap too:
|
||||
|
||||
```python
|
||||
fail_after_s = (
|
||||
max(30, cpu_count() * 3) # 3s/actor floor
|
||||
if is_forking_spawner
|
||||
else 12
|
||||
)
|
||||
```
|
||||
|
||||
Pros: scales with the actual cancel-cascade work.
|
||||
Cons: still arbitrary multiplier.
|
||||
|
||||
### C. `pytest-rerunfailures` for these tests only
|
||||
|
||||
Mark the known-flaky tests with
|
||||
`@pytest.mark.flaky(reruns=1)` (needs
|
||||
`pytest-rerunfailures` dep). Single retry hides
|
||||
genuine ~0.3% transient flakes.
|
||||
|
||||
Pros: no cap change, surfaces persistent failures
|
||||
loudly. Cons: adds a dep, retries can mask real bugs
|
||||
if used widely.
|
||||
|
||||
### D. Reduce `hard_kill`'s `terminate_after`
|
||||
|
||||
Drop from 1.6s → 0.8s. Cuts the worst-case cascade
|
||||
time roughly in half. Risks: fewer subs get a chance
|
||||
to run their cleanup before SIGKILL → more orphaned
|
||||
state for the autouse reapers to handle (ironically,
|
||||
adds back overhead elsewhere).
|
||||
|
||||
### E. Profile + targeted fix
|
||||
|
||||
Add `log.devx()` markers in `hard_kill` to time each
|
||||
phase. Identify if any subactor is consistently
|
||||
hitting the 1.6s cap (vs. exiting in <0.1s). If so,
|
||||
that sub has a teardown bug worth fixing at source.
|
||||
Pros: actually fixes the underlying slowness. Cons:
|
||||
real investigation work, deferred from this round.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Land this issue-doc as the tracker. Apply **(B)** as
|
||||
a small follow-up — cheap and proportional. If it
|
||||
still flakes, escalate to **(E)** with a `log.devx()`
|
||||
profile-pass.
|
||||
|
||||
`(C)` is a backstop if `(B)` doesn't quite get there
|
||||
and we need green CI faster than (E) can deliver.
|
||||
|
||||
## Verification protocol
|
||||
|
||||
After applying any mitigation:
|
||||
|
||||
```bash
|
||||
# Run the suite N times back-to-back, count failures.
|
||||
# A persistent failure on the SAME test == real bug.
|
||||
# Failures rotating across tests == still cap-related.
|
||||
|
||||
for i in $(seq 1 5); do
|
||||
./py313/bin/python -m pytest tests/ \
|
||||
--tpt-proto=tcp \
|
||||
--spawn-backend=main_thread_forkserver \
|
||||
-q 2>&1 | tail -2
|
||||
done
|
||||
```
|
||||
|
||||
Target: 0 failures across 5 runs ⇒ ship. 1–2 failures
|
||||
still rotating ⇒ apply (C). Same test failing twice
|
||||
⇒ escalate to (E).
|
||||
|
||||
## Snapshot evidence (2026-05-13)
|
||||
|
||||
After landing the `fail_after_w_trace` /
|
||||
`afk_alarm_w_trace` capture-on-timeout helpers
|
||||
(`tractor._testing.trace`), `test_nested_multierrors`
|
||||
on the `main_thread_forkserver` backend produces
|
||||
**reproducible diag snapshots** at
|
||||
`$XDG_CACHE_HOME/tractor/hung-dumps/test_nested_multierrors_start_method_main_thread_forkserver__<iso-ts>/`.
|
||||
|
||||
### Reproduction
|
||||
|
||||
```bash
|
||||
pytest \
|
||||
-v --verbose --durations=10 \
|
||||
--spawn-backend=main_thread_forkserver \
|
||||
--tpt-proto=uds \
|
||||
--capture=sys --show-capture=stderr -rxX \
|
||||
tests/test_cancellation.py::test_nested_multierrors
|
||||
```
|
||||
|
||||
The test is `xfail(strict=False)` for MTF — it RUNS
|
||||
each invocation so snapshots accumulate, but doesn't
|
||||
break `--lf` workflow.
|
||||
|
||||
### Consistent shape across runs
|
||||
|
||||
5+ snapshots taken back-to-back show the SAME pattern:
|
||||
|
||||
- **Timing:** ~10s wall-clock total. Inner
|
||||
`fail_after_w_trace(10)` fires at exactly T=10s;
|
||||
cascade's `nursery.__aexit__` takes ~0.6s more to
|
||||
gather + propagate the resulting
|
||||
`BaseExceptionGroup`. **Trio backend completes the
|
||||
SAME test in <6s** — so the MTF cascade is ~2x
|
||||
slower at minimum.
|
||||
|
||||
- **`BaseExceptionGroup` shape:** mixed
|
||||
`[RemoteActorError, Cancelled]`. The first
|
||||
subactor's natural error-propagation (`assert 0`
|
||||
raised → `RemoteActorError` portal-result)
|
||||
arrives before T=10s; the OTHER subactor's
|
||||
portal-wait is still in flight at T=10s, gets
|
||||
cancelled by `fail_after_w_trace`'s scope-cancel
|
||||
→ returns `Cancelled` instead.
|
||||
|
||||
- **Orphan-spawn skew:** snapshot's `orphans` bucket
|
||||
(after the `_is_tractor_subactor` cgroup-slice
|
||||
override fix) consistently shows 2-4 init-adopted
|
||||
procs at `depth_3` and `depth_1` levels — these
|
||||
are the leaves whose parent (`depth_2` spawner)
|
||||
was killed mid-cascade but who hadn't yet seen
|
||||
the cancel signal themselves.
|
||||
|
||||
- **UDS sock-leak:** 2-6 dead-orphan socks per run
|
||||
(varies with cascade timing). The
|
||||
`track_orphaned_uds_per_test` fixture reaps them
|
||||
post-test → contamination is isolated per-invocation.
|
||||
|
||||
### Capture mechanism
|
||||
|
||||
`fail_after_w_trace` covers two firing paths:
|
||||
|
||||
1. **`trio.TooSlowError`** raised at scope-exit
|
||||
(body returned cleanly past deadline) — direct
|
||||
`except` handler captures.
|
||||
|
||||
2. **Scope-cancel + body raises non-`Cancelled` exc**
|
||||
(e.g. `nursery.__aexit__` wraps timeout-induced
|
||||
`Cancelled` into a `BaseExceptionGroup` that
|
||||
escapes before `trio.fail_after`'s exit-check
|
||||
could fire `TooSlowError`) — body-raise `except`
|
||||
handler checks `scope.cancel_called` and
|
||||
captures if True. This path catches the
|
||||
`test_nested_multierrors` shape specifically (see
|
||||
"BaseExceptionGroup shape" above).
|
||||
|
||||
The snapshot dir contains:
|
||||
- `trace.txt` — `ptree` + `hung_state` (kernel
|
||||
`wchan`/`stack` + `py-spy dump --locals` when
|
||||
sudo cached), with `include_strays=True`
|
||||
surfacing any cross-test ghost subactor trees in
|
||||
the `orphans` bucket.
|
||||
- `bindspace.txt` — UDS bindspace classification
|
||||
(live-active / orphaned-alive / orphaned-dead).
|
||||
- `meta.json` — `{pid, label, captured_at, sudo_cached}`.
|
||||
|
||||
The end-of-session `pytest_terminal_summary` hook
|
||||
in `tractor._testing.pytest` lists every snapshot
|
||||
dir from the run so you don't have to scroll back
|
||||
through captured-stderr lines:
|
||||
|
||||
```
|
||||
========================= tractor hang-snapshot index ==========================
|
||||
N `fail_after_w_trace` / `afk_alarm_w_trace` snapshot(s) captured this session:
|
||||
<test-id>
|
||||
→ /home/.../.cache/tractor/hung-dumps/<label>__<ts>
|
||||
```
|
||||
|
||||
### Caveats
|
||||
|
||||
The snapshot fires AFTER the body-raise (not at the
|
||||
exact moment of scope-cancel), so the parent's
|
||||
py-spy frames show `_do_capture_snapshot` itself
|
||||
running, NOT the cancel-cascade hang frame. To see
|
||||
the actual hang state, manual `acli.ptree` /
|
||||
`acli.hung_dump` from a second terminal at T=10s
|
||||
would be needed — **not currently possible**
|
||||
because per-test reaper fixtures clean up ~0.6s
|
||||
post-timeout. See follow-up TODO in
|
||||
`tractor/_testing/trace.py` for a
|
||||
`TRACTOR_TRACE_HOLD=1` env-var pause mode.
|
||||
|
||||
## See also
|
||||
|
||||
- [#452](https://github.com/goodboy/tractor/issues/452) —
|
||||
UDS sock-file leak (related — `hard_kill`'s
|
||||
cleanup phase contributes to cascade time)
|
||||
- `ai/conc-anal/trio_wakeup_socketpair_busy_loop_under_fork_issue.md`
|
||||
— the upstream-trio fix that turned this from a
|
||||
100% hang into a 0.3% flake
|
||||
- `ai/conc-anal/infected_asyncio_under_main_thread_forkserver_hang_issue.md`
|
||||
— the asyncio variant which contributes to one of
|
||||
the rotating failures
|
||||
- `tractor/spawn/_spawn.py::hard_kill` — the SIGKILL
|
||||
cascade source
|
||||
- `tractor/_testing/_reap.py::_track_orphaned_uds_per_test`,
|
||||
`_detect_runaway_subactors_per_test`,
|
||||
`_reap_orphaned_subactors` — autouse cleanup
|
||||
fixtures whose cumulative teardown overhead
|
||||
contributes to the cascade time
|
||||
|
|
@ -1,281 +0,0 @@
|
|||
# `fork()` in a multi-threaded program — execution-side vs. memory-side of the same coin
|
||||
|
||||
A reference doc for readers who've encountered one of two
|
||||
opposite-sounding framings of POSIX `fork()` semantics in a
|
||||
multi-threaded program and are confused by the other.
|
||||
|
||||
This is a sibling to
|
||||
`subint_fork_blocked_by_cpython_post_fork_issue.md` — that
|
||||
doc covers a CPython-level refusal of fork-from-subint;
|
||||
this one covers the more general POSIX layer, since
|
||||
tractor's main-thread forkserver design rests on it.
|
||||
|
||||
## TL;DR
|
||||
|
||||
POSIX `fork()` only preserves the *calling* thread as a
|
||||
runnable thread in the child — every other thread in the
|
||||
parent simply never executes another instruction in the
|
||||
child. trio's docs call this "leaked"; tractor's
|
||||
`_main_thread_forkserver.py` docstring calls it "gone".
|
||||
Both are correct: "gone" is the *execution* side (no
|
||||
scheduler entry, no instructions retired), "leaked" is the
|
||||
*memory* side (the dead threads' stacks and per-thread
|
||||
heap structures still ride into the child's address space
|
||||
as orphaned COW pages with no owner and no cleanup hook).
|
||||
Same POSIX reality, two halves of the same coin.
|
||||
|
||||
## The two framings
|
||||
|
||||
[python-trio/trio#1614][trio-1614] (the canonical "trio +
|
||||
fork" hazards thread) puts it this way:
|
||||
|
||||
> If you use `fork()` in a process with multiple threads,
|
||||
> all the other thread stacks are just leaked: there's
|
||||
> nothing else you can reasonably do with them.
|
||||
|
||||
`tractor.spawn._main_thread_forkserver`'s module docstring
|
||||
(specifically the "What survives the fork? — POSIX
|
||||
semantics" section) puts it this way:
|
||||
|
||||
> POSIX `fork()` only preserves the *calling* thread as a
|
||||
> runnable thread in the child. Every other thread in the
|
||||
> parent — trio's runner thread, any `to_thread` cache
|
||||
> threads, anything else — never executes another
|
||||
> instruction post-fork.
|
||||
|
||||
A reader bouncing between the two can be forgiven for
|
||||
asking: well, *which* is it — leaked or gone?
|
||||
|
||||
The answer is "yes". They're describing the same POSIX
|
||||
behavior from two different angles:
|
||||
|
||||
- trio is talking about the **bytes** the dead threads
|
||||
leave behind — stacks, TLS slots, per-thread arena
|
||||
metadata — and the fact that nothing in the child can
|
||||
drive them forward, free them, or even safely walk
|
||||
them. That's a memory leak in the strict sense: held
|
||||
but unreachable.
|
||||
- tractor is talking about the **execution** side
|
||||
relevant to the forkserver design: which threads
|
||||
retire instructions in the child? Exactly one — the
|
||||
one that called `fork()`. Everything else, regardless
|
||||
of the bytes left behind, is dead in a scheduler
|
||||
sense.
|
||||
|
||||
Neither framing is wrong; they're just answering
|
||||
different questions.
|
||||
|
||||
## POSIX `fork()` in a multi-threaded program — what actually happens
|
||||
|
||||
Per POSIX (and concretely on Linux glibc), the contract
|
||||
of `fork()` in a multi-threaded process is:
|
||||
|
||||
1. The kernel creates a new process whose virtual
|
||||
address space is a COW copy of the parent's. *All*
|
||||
pages map across — code, heap, every thread's stack,
|
||||
every malloc arena, every mmap region.
|
||||
2. Of the parent's N threads, exactly **one** is
|
||||
reified in the child as a runnable kernel task: the
|
||||
thread that called `fork()`. The other N-1 threads
|
||||
have *no* corresponding task in the child kernel. They
|
||||
were never scheduled, never `clone()`d for the child,
|
||||
never exist as runnable entities.
|
||||
3. Their **memory artifacts** — pthread stacks, TLS,
|
||||
`pthread_t` structures, glibc per-thread arena
|
||||
bookkeeping — are still mapped in the child's address
|
||||
space, because (1) duplicates *everything* page-wise.
|
||||
They sit there as inert COW bytes.
|
||||
4. The kernel does not clean those bytes up. There is no
|
||||
"phantom-thread cleanup" pass post-fork. The kernel
|
||||
doesn't know which mapped pages "belonged to" which
|
||||
thread — at the kernel level mappings are
|
||||
process-scoped, not thread-scoped.
|
||||
5. The surviving thread (the caller of `fork()`) cannot
|
||||
safely access those leaked bytes either. Any state
|
||||
they encoded — held mutexes, in-flight syscalls,
|
||||
half-updated invariants — is frozen at whatever
|
||||
instant the parent's fork-syscall observed it. Some
|
||||
of those mutexes may even still be locked from the
|
||||
child's POV (the canonical "fork-in-multithreaded-
|
||||
program-deadlocks" hazard; see `man pthread_atfork`).
|
||||
|
||||
So: from the kernel's PoV, the child has one thread.
|
||||
From the address-space's PoV, the child has all the
|
||||
parent's bytes — including the corpses of the N-1 dead
|
||||
threads' stacks. Both true simultaneously.
|
||||
|
||||
## Why trio says "leaked"
|
||||
|
||||
trio's framing makes sense from the parent's
|
||||
PoV, looking at *what those threads were doing*. In a
|
||||
running `trio.run()` process you typically have:
|
||||
|
||||
- The trio runner thread itself — owns the `selectors`
|
||||
epoll fd, the signal-wakeup-fd, the run-queue.
|
||||
- Threadpool worker threads (`trio.to_thread`'s cache)
|
||||
— blocked in `wait()` on the threadpool's work
|
||||
condvar.
|
||||
- Whatever other ad-hoc threads the application
|
||||
started.
|
||||
|
||||
Each of those threads owns *real work-state*: epoll
|
||||
registrations, file descriptors held in
|
||||
soon-to-be-completed reads, half-released locks, posted
|
||||
but unconsumed wakeups. After fork, that state is still
|
||||
encoded in the child's memory. None of it is invalid in
|
||||
a well-formed-bytes sense. It's just that:
|
||||
|
||||
- The thread that was driving it is gone.
|
||||
- Nothing else in the child knows the layout well
|
||||
enough to take over.
|
||||
- Even if it did, the kernel objects backing the work
|
||||
(epoll fd, signalfd) have separate post-fork
|
||||
semantics that don't compose with userland trio
|
||||
state.
|
||||
|
||||
So the bytes are *held* (they're in the child's
|
||||
address space, they count against RSS, they survive
|
||||
until something clobbers them), and they're
|
||||
*unreachable* in any meaningful sense — no thread can
|
||||
safely drive them forward. That is the textbook
|
||||
definition of a leak.
|
||||
|
||||
trio's quote is reminding the user that `fork()` from a
|
||||
multi-threaded process is a one-way memory hazard:
|
||||
whatever those threads were doing, that work-state is
|
||||
now garbage you happen to still be carrying.
|
||||
|
||||
## Why tractor says "gone"
|
||||
|
||||
tractor's `_main_thread_forkserver` framing is concerned
|
||||
with a different question: *which thread executes in the
|
||||
child, and is it safe?*
|
||||
|
||||
The forkserver design rests on POSIX's "calling thread
|
||||
is the sole survivor" guarantee. We pick that calling
|
||||
thread very deliberately: a dedicated worker that has
|
||||
provably never entered trio. So the thread that *does*
|
||||
run in the child is one whose locals, TLS, and stack
|
||||
contain nothing trio-related. Trio's runner thread —
|
||||
the one that owned the epoll fd and the run-queue — is
|
||||
*gone* from the child in the execution sense. It will
|
||||
never run another instruction. The fact that its stack
|
||||
bytes still exist in the child's address space (the
|
||||
"leaked" view) is irrelevant to the forkserver, because
|
||||
nothing in the child reads or writes those pages.
|
||||
|
||||
So when the docstring says "Every other thread … is
|
||||
gone the instant `fork()` returns in the child", it's
|
||||
being precise about the surface that matters for the
|
||||
backend: scheduler-level liveness. Nothing schedules
|
||||
those threads ever again. Whether their bytes are
|
||||
hanging around is a separate (and, for the design,
|
||||
non-load-bearing) fact.
|
||||
|
||||
## Cross-table
|
||||
|
||||
The same tabular layout the `_main_thread_forkserver`
|
||||
docstring uses, expanded with a fourth "what handles
|
||||
it" column:
|
||||
|
||||
| thread | parent | child (executing) | child (memory) | what handles it |
|
||||
|---------------------|-----------|-------------------|------------------------------|-----------------------------|
|
||||
| forkserver worker | continues | sole survivor | live stack | runs the child's bootstrap |
|
||||
| `trio.run()` thread | continues | not running | leaked stack (zombie bytes) | overwritten by child's fresh `trio.run()` |
|
||||
| any other thread | continues | not running | leaked stack (zombie bytes) | overwritten / GC'd / clobbered by `exec()` if used |
|
||||
|
||||
The "child (executing)" column is the *execution* side
|
||||
of the coin — what tractor cares about. The "child
|
||||
(memory)" column is the *memory* side — what trio
|
||||
cares about.
|
||||
|
||||
The "what handles it" column is the deliberate punchline
|
||||
of the design: nothing has to handle the leaked bytes
|
||||
*explicitly*. They get clobbered by ordinary forward
|
||||
progress in the child:
|
||||
|
||||
- The fresh `trio.run()` the child boots up allocates
|
||||
its own stack, scheduler, and run-queue, which over
|
||||
time overlaps and overwrites the inherited zombie
|
||||
pages.
|
||||
- Python's GC walks live objects only; the dead-thread
|
||||
Python frames aren't reachable from any
|
||||
`PyThreadState`, so they get freed at the next
|
||||
collection cycle.
|
||||
- If the child eventually `exec()`s, the entire address
|
||||
space is replaced and the leak vanishes.
|
||||
|
||||
## What this means for the forkserver design
|
||||
|
||||
The crucial point is that **the design doesn't and
|
||||
*can't* prevent the leak**. There is no userland fix
|
||||
for COW thread stacks. The kernel hands the child a
|
||||
duplicated address space; that's what `fork()` *is*. No
|
||||
amount of pre-fork hookery, `pthread_atfork()`
|
||||
gymnastics, or post-fork cleanup can un-COW the dead
|
||||
threads' pages without unmapping them, and unmapping
|
||||
arbitrary regions of a duplicated address space is
|
||||
neither portable nor safe.
|
||||
|
||||
What the design *does* ensure is the orthogonal
|
||||
property: the survivor thread is one that doesn't need
|
||||
any of that leaked state to function. Concretely:
|
||||
|
||||
- Survivor is the forkserver worker thread.
|
||||
- That worker has provably never imported, called into,
|
||||
or held any reference to `trio`. (Enforced by keeping
|
||||
the worker's lifecycle entirely in
|
||||
`_main_thread_forkserver.py` and never letting trio
|
||||
task-state cross into it.)
|
||||
- So the leaked pages — trio runner stack, threadpool
|
||||
caches, etc. — are inert relative to the survivor.
|
||||
No code path in the child references them.
|
||||
- The child then boots its own fresh `trio.run()`,
|
||||
which allocates new state in new pages. Over the
|
||||
child's lifetime the COW'd zombie pages get
|
||||
overwritten, GC'd, or (if the child eventually
|
||||
`exec()`s) discarded wholesale.
|
||||
|
||||
The "leak" is real but inert. It costs RSS until
|
||||
clobbered; it doesn't cost correctness. That's exactly
|
||||
the property the forkserver pattern is built on, and
|
||||
it's also why the design needs the "calling thread is
|
||||
trio-free" precondition to be airtight: if the survivor
|
||||
were a trio thread, it *would* try to drive the leaked
|
||||
trio state, and the leak would no longer be inert.
|
||||
|
||||
## See also
|
||||
|
||||
- `tractor/spawn/_main_thread_forkserver.py` — module
|
||||
docstring's "What survives the fork? — POSIX
|
||||
semantics" section is the in-tree, code-adjacent
|
||||
prose this doc expands on. The cross-table here is a
|
||||
fourth-column expansion of the table there.
|
||||
|
||||
- [python-trio/trio#1614][trio-1614] — the trio issue
|
||||
with the "leaked" framing, and the canonical thread
|
||||
for trio + `fork()` hazards more broadly.
|
||||
|
||||
- [`subint_fork_blocked_by_cpython_post_fork_issue.md`](./subint_fork_blocked_by_cpython_post_fork_issue.md)
|
||||
— sibling analysis covering CPython's *post-fork*
|
||||
hooks (`PyOS_AfterFork_Child`,
|
||||
`_PyInterpreterState_DeleteExceptMain`) and why
|
||||
fork-from-non-main-subint is a CPython-level hard
|
||||
refusal. Complementary axis: this doc is about POSIX
|
||||
semantics; that doc is about the CPython runtime
|
||||
layer that runs *after* POSIX `fork()` returns in
|
||||
the child.
|
||||
|
||||
- `man pthread_atfork(3)` — canonical "fork in a
|
||||
multithreaded process is dangerous" reference.
|
||||
Especially the rationale section, which is the
|
||||
closest thing to a normative statement of "the
|
||||
surviving thread cannot safely use anything the dead
|
||||
threads were touching."
|
||||
|
||||
- `man fork(2)` (Linux) — "Other than [the calling
|
||||
thread], … no other threads are replicated …"
|
||||
paragraph is the kernel-side statement of the
|
||||
execution-side framing this doc opens with.
|
||||
|
||||
[trio-1614]: https://github.com/python-trio/trio/issues/1614
|
||||
|
|
@ -1,142 +0,0 @@
|
|||
# Spawn-time boot-death (`rc=2`) under rapid same-name spawn against a registrar
|
||||
|
||||
## Symptom
|
||||
|
||||
Spawning N (≥4) sub-actors with the **same name** in tight
|
||||
succession against a daemon registrar surfaces as
|
||||
`ActorFailure: Sub-actor (...) died during boot (rc=2)
|
||||
before completing parent-handshake`.
|
||||
|
||||
```
|
||||
tests/discovery/test_multi_program.py
|
||||
::test_dup_name_cancel_cascade_escalates_to_hard_kill[n_dups=4]
|
||||
```
|
||||
|
||||
```
|
||||
tractor._exceptions.ActorFailure:
|
||||
Sub-actor ('doggy', '<uuid>') died during boot (rc=2)
|
||||
before completing parent-handshake.
|
||||
proc: <_ForkedProc pid=<n> returncode=None>
|
||||
```
|
||||
|
||||
The `proc` repr shows `returncode=None` because the repr is
|
||||
captured before `proc.wait()` returns; the actual
|
||||
`os.WEXITSTATUS == 2` is reported via `result['died']` in the
|
||||
race-helper.
|
||||
|
||||
## When it surfaces
|
||||
|
||||
- N=2 (`n_dups=2`): **always passes**.
|
||||
- N=4 (`n_dups=4`): **consistent fail** under both `tpt-proto=tcp`
|
||||
and `tpt-proto=uds`, MTF backend.
|
||||
- N=8 (`n_dups=8`): **passes** (counter-intuitive — see "racing
|
||||
windows").
|
||||
- Non-MTF backends: not yet exercised systematically.
|
||||
|
||||
## What previously masked it
|
||||
|
||||
Pre the spawn-time `wait_for_peer_or_proc_death` race-helper
|
||||
(in `tractor.spawn._spawn`), the parent's `start_actor` flow
|
||||
ended with a bare:
|
||||
|
||||
```python
|
||||
event, chan = await ipc_server.wait_for_peer(uid)
|
||||
```
|
||||
|
||||
That awaits an unsignalled `trio.Event` on `_peer_connected[uid]`.
|
||||
If the sub-actor process **dies during boot** (before its
|
||||
runtime executes the parent-callback handshake that sets the
|
||||
event), the wait parks forever. The dead proc becomes a zombie
|
||||
because no one ever calls `proc.wait()` to reap it.
|
||||
|
||||
In test contexts the failure presented as a hang or a much
|
||||
later `trio.TooSlowError` from an outer `fail_after`. In
|
||||
production it'd present as a parent that never makes progress
|
||||
past `start_actor`. The death itself was silently masked.
|
||||
|
||||
## What surfaces it now
|
||||
|
||||
`tractor.spawn._spawn.wait_for_peer_or_proc_death` (used by
|
||||
`_main_thread_forkserver_proc`) races the handshake-wait
|
||||
against `proc.wait()`. The race-helper raises `ActorFailure`
|
||||
on death-first instead of parking, exposing the rc=2.
|
||||
|
||||
## Hypothesis: registrar-side same-name contention
|
||||
|
||||
The test spawns N actors with name `doggy` sequentially:
|
||||
|
||||
```python
|
||||
for i in range(n_dups):
|
||||
p: Portal = await an.start_actor('doggy')
|
||||
portals.append(p)
|
||||
```
|
||||
|
||||
Each spawned doggy:
|
||||
|
||||
1. Forks via the forkserver.
|
||||
2. Boots its runtime in `_actor_child_main`.
|
||||
3. Connects back to the parent for handshake.
|
||||
4. Connects to the daemon registrar to call `register_actor`.
|
||||
5. Enters its RPC msg-loop.
|
||||
|
||||
Step (4) is where the same-name contention lives. The
|
||||
registrar's `register_actor` (in
|
||||
`tractor.discovery._registry`) accepts duplicate names
|
||||
(stores `(name, uuid) -> addr`), but its internal bookkeeping
|
||||
may have a non-trivial check (e.g. `wait_for_actor` resolution,
|
||||
`_addrs2aids` map updates) that errors out under specific
|
||||
ordering between the existing entry and the incoming one.
|
||||
|
||||
`rc=2 == os.WEXITSTATUS == 2` corresponds to `sys.exit(2)`
|
||||
in the doggy process — typically reached via an unhandled
|
||||
exception that's translated to exit code 2 by Python's top-
|
||||
level (e.g. `argparse` errors use 2; `SystemExit(2)` etc.).
|
||||
So the doggy is hitting an explicit exit path during
|
||||
`register_actor` or just-after.
|
||||
|
||||
The non-monotonic shape (N=2 OK, N=4 BAD, N=8 OK) suggests a
|
||||
specific timing window — likely "the 3rd register-RPC arrives
|
||||
while the 1st-or-2nd is in some intermediate state". With
|
||||
N=8, the additional procs widen the registration spread
|
||||
enough that no two land in the conflicting window.
|
||||
|
||||
## Where to dig next
|
||||
|
||||
- Add per-actor logging in `_actor_child_main` and
|
||||
`register_actor` to surface the actual exception that
|
||||
triggers the rc=2 exit. Currently the doggy dies before
|
||||
the parent ever sees its stderr (forkserver doesn't
|
||||
marshal child stdio back).
|
||||
- Race-test the registrar's `register_actor` /
|
||||
`unregister_actor` / `wait_for_actor` against same-name
|
||||
concurrent calls in isolation (no spawn).
|
||||
- Consider whether `register_actor` should be idempotent
|
||||
under same-name re-register or should explicitly reject
|
||||
same-name (and ideally with a clear `RemoteActorError`,
|
||||
not `sys.exit(2)`).
|
||||
|
||||
## Test-suite handling
|
||||
|
||||
Currently:
|
||||
|
||||
- `tests/discovery/test_multi_program.py
|
||||
::test_dup_name_cancel_cascade_escalates_to_hard_kill[n_dups=4]`
|
||||
is `pytest.mark.xfail(strict=False, reason=...)` to keep
|
||||
the suite green while this issue is investigated.
|
||||
- `n_dups=2` and `n_dups=8` continue to validate the
|
||||
cancel-cascade hard-kill escalation.
|
||||
|
||||
Once the underlying race is understood + fixed, drop the
|
||||
xfail.
|
||||
|
||||
## Related work
|
||||
|
||||
- The cancel-cascade fix that introduced this regression
|
||||
test:
|
||||
`tractor/_exceptions.py:ActorTooSlowError`,
|
||||
`tractor/runtime/_supervise.py:_try_cancel_then_kill`,
|
||||
`tractor/runtime/_portal.py:Portal.cancel_actor(
|
||||
raise_on_timeout=...)`.
|
||||
- The spawn-time death-detection that exposed this:
|
||||
`tractor/spawn/_spawn.py:wait_for_peer_or_proc_death`,
|
||||
used by `tractor/spawn/_main_thread_forkserver.py`.
|
||||
|
|
@ -1,161 +0,0 @@
|
|||
# `subint` backend: parent trio loop parks after subint teardown (Ctrl-C works; not a CPython-level issue)
|
||||
|
||||
Follow-up to the Phase B subint spawn-backend PR (see
|
||||
`tractor.spawn._subint`, issue #379). Distinct from the
|
||||
`subint_sigint_starvation_issue.md` (SIGINT-unresponsive
|
||||
starvation hang): this one is **Ctrl-C-able**, which means
|
||||
it's *not* the shared-GIL-hostage class and is ours to fix
|
||||
from inside tractor rather than waiting on upstream CPython
|
||||
/ msgspec progress.
|
||||
|
||||
## TL;DR
|
||||
|
||||
After a stuck-subint subactor is torn down via the
|
||||
hard-kill path, a parent-side trio task parks on an
|
||||
*orphaned resource* (most likely a `chan.recv()` /
|
||||
`process_messages` loop on the now-dead subint's IPC
|
||||
channel) and waits forever for bytes that can't arrive —
|
||||
because the channel was torn down without emitting a clean
|
||||
EOF/`BrokenResourceError` to the waiting receiver.
|
||||
|
||||
Unlike `subint_sigint_starvation_issue.md`, the main trio
|
||||
loop **is** iterating normally — SIGINT delivers cleanly
|
||||
and the test unhangs. But absent Ctrl-C, the test suite
|
||||
wedges indefinitely.
|
||||
|
||||
## Symptom
|
||||
|
||||
Running `test_subint_non_checkpointing_child` under
|
||||
`--spawn-backend=subint` (in
|
||||
`tests/test_subint_cancellation.py`):
|
||||
|
||||
1. Test spawns a subactor whose main task runs
|
||||
`threading.Event.wait(1.0)` in a loop — releases the
|
||||
GIL but never inserts a trio checkpoint.
|
||||
2. Parent does `an.cancel_scope.cancel()`. Our
|
||||
`subint_proc` cancel path fires: soft-kill sends
|
||||
`Portal.cancel_actor()` over the live IPC channel →
|
||||
subint's trio loop *should* process the cancel msg on
|
||||
its IPC dispatcher task (since the GIL releases are
|
||||
happening).
|
||||
3. Expected: subint's `trio.run()` unwinds, driver thread
|
||||
exits naturally, parent returns.
|
||||
4. Actual: parent `trio.run()` never completes. Test
|
||||
hangs past its `trio.fail_after()` deadline.
|
||||
|
||||
## Evidence
|
||||
|
||||
### `strace` on the hung pytest process during SIGINT
|
||||
|
||||
```
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(17, "\2", 1) = 1
|
||||
```
|
||||
|
||||
Contrast with the SIGINT-starvation hang (see
|
||||
`subint_sigint_starvation_issue.md`) where that same
|
||||
`write()` returned `EAGAIN`. Here the SIGINT byte is
|
||||
written successfully → Python's signal handler pipe is
|
||||
being drained → main trio loop **is** iterating → SIGINT
|
||||
gets turned into `trio.Cancelled` → the test unhangs (if
|
||||
the operator happens to be there to hit Ctrl-C).
|
||||
|
||||
### Stack dump (via `tractor.devx.dump_on_hang`)
|
||||
|
||||
Single main thread visible, parked in
|
||||
`trio._core._io_epoll.get_events` inside `trio.run` at the
|
||||
test's `trio.run(...)` call site. No subint driver thread
|
||||
(subint was destroyed successfully — this is *after* the
|
||||
hard-kill path, not during it).
|
||||
|
||||
## Root cause hypothesis
|
||||
|
||||
Most consistent with the evidence: a parent-side trio
|
||||
task is awaiting a `chan.recv()` / `process_messages` loop
|
||||
on the dead subint's IPC channel. The sequence:
|
||||
|
||||
1. Soft-kill in `subint_proc` sends `Portal.cancel_actor()`
|
||||
over the channel. The subint's trio dispatcher *may* or
|
||||
may not have processed the cancel msg before the subint
|
||||
was destroyed — timing-dependent.
|
||||
2. Hard-kill timeout fires (because the subint's main
|
||||
task was in `threading.Event.wait()` with no trio
|
||||
checkpoint — cancel-msg processing couldn't race the
|
||||
timeout).
|
||||
3. Driver thread abandoned, `_interpreters.destroy()`
|
||||
runs. Subint is gone.
|
||||
4. But the parent-side trio task holding a
|
||||
`chan.recv()` / `process_messages` loop against that
|
||||
channel was **not** explicitly cancelled. The channel's
|
||||
underlying socket got torn down, but without a clean
|
||||
EOF delivered to the waiting recv, the task parks
|
||||
forever on `trio.lowlevel.wait_readable` (or similar).
|
||||
|
||||
This matches the "main loop fine, task parked on
|
||||
orphaned I/O" signature.
|
||||
|
||||
## Why this is ours to fix (not CPython's)
|
||||
|
||||
- Main trio loop iterates normally → GIL isn't starved.
|
||||
- SIGINT is deliverable → not a signal-pipe-full /
|
||||
wakeup-fd contention scenario.
|
||||
- The hang is in *our* supervision code, specifically in
|
||||
how `subint_proc` tears down its side of the IPC when
|
||||
the subint is abandoned/destroyed.
|
||||
|
||||
## Possible fix directions
|
||||
|
||||
1. **Explicit parent-side channel abort on subint
|
||||
abandon.** In `subint_proc`'s teardown block, after the
|
||||
hard-kill timeout fires, explicitly close the parent's
|
||||
end of the IPC channel to the subint. Any waiting
|
||||
`chan.recv()` / `process_messages` task sees
|
||||
`BrokenResourceError` (or `ClosedResourceError`) and
|
||||
unwinds.
|
||||
2. **Cancel parent-side RPC tasks tied to the dead
|
||||
subint's channel.** The `Actor._rpc_tasks` / nursery
|
||||
machinery should have a handle on any
|
||||
`process_messages` loops bound to a specific peer
|
||||
channel. Iterate those and cancel explicitly.
|
||||
3. **Bound the top-level `await actor_nursery
|
||||
._join_procs.wait()` shield in `subint_proc`** (same
|
||||
pattern as the other bounded shields the hard-kill
|
||||
patch added). If the nursery never sets `_join_procs`
|
||||
because a child task is parked, the bound would at
|
||||
least let the teardown proceed.
|
||||
|
||||
Of these, (1) is the most surgical and directly addresses
|
||||
the root cause. (2) is a defense-in-depth companion. (3)
|
||||
is a band-aid but cheap to add.
|
||||
|
||||
## Current workaround
|
||||
|
||||
None in-tree. The test's `trio.fail_after()` bound
|
||||
currently fires and raises `TooSlowError`, so the test
|
||||
visibly **fails** rather than hangs — which is
|
||||
intentional (an unbounded cancellation-audit test would
|
||||
defeat itself). But in interactive test runs the operator
|
||||
has to hit Ctrl-C to move past the parked state before
|
||||
pytest reports the failure.
|
||||
|
||||
## Reproducer
|
||||
|
||||
```
|
||||
./py314/bin/python -m pytest \
|
||||
tests/test_subint_cancellation.py::test_subint_non_checkpointing_child \
|
||||
--spawn-backend=subint --tb=short --no-header -v
|
||||
```
|
||||
|
||||
Expected: hangs until `trio.fail_after(15)` fires, or
|
||||
Ctrl-C unwedges it manually.
|
||||
|
||||
## References
|
||||
|
||||
- `tractor.spawn._subint.subint_proc` — current subint
|
||||
teardown code; see the `_HARD_KILL_TIMEOUT` bounded
|
||||
shields + `daemon=True` driver-thread abandonment
|
||||
(commit `b025c982`).
|
||||
- `ai/conc-anal/subint_sigint_starvation_issue.md` — the
|
||||
sibling CPython-level hang (GIL-starvation,
|
||||
SIGINT-unresponsive) which is **not** this issue.
|
||||
- Phase B tracking: issue #379.
|
||||
|
|
@ -1,337 +0,0 @@
|
|||
# `os.fork()` from a non-main sub-interpreter aborts the child (CPython refuses post-fork cleanup)
|
||||
|
||||
Third `subint`-class analysis in this project. Unlike its
|
||||
two siblings (`subint_sigint_starvation_issue.md`,
|
||||
`subint_cancel_delivery_hang_issue.md`), this one is not a
|
||||
hang — it's a **hard CPython-level refusal** of an
|
||||
experimental spawn strategy we wanted to try.
|
||||
|
||||
## TL;DR
|
||||
|
||||
An in-process sub-interpreter cannot be used as a
|
||||
"launchpad" for `os.fork()` on current CPython. The fork
|
||||
syscall succeeds in the parent, but the forked CHILD
|
||||
process is aborted immediately by CPython's post-fork
|
||||
cleanup with:
|
||||
|
||||
```
|
||||
Fatal Python error: _PyInterpreterState_DeleteExceptMain: not main interpreter
|
||||
```
|
||||
|
||||
This is enforced by a hard `PyStatus_ERR` gate in
|
||||
`Python/pystate.c`. The CPython devs acknowledge the
|
||||
fragility with an in-source comment (`// Ideally we could
|
||||
guarantee tstate is running main.`) but provide no
|
||||
mechanism to satisfy the precondition from user code.
|
||||
|
||||
**Implication for tractor**: the `subint_fork` backend
|
||||
sketched in `tractor.spawn._subint_fork` is structurally
|
||||
dead on current CPython. The submodule is kept as
|
||||
documentation of the attempt; `--spawn-backend=subint_fork`
|
||||
raises `NotImplementedError` pointing here.
|
||||
|
||||
## Context — why we tried this
|
||||
|
||||
The motivation is issue #379's "Our own thoughts, ideas
|
||||
for `fork()`-workaround/hacks..." section. The existing
|
||||
trio-backend (`tractor.spawn._trio.trio_proc`) spawns
|
||||
subactors via `trio.lowlevel.open_process()` → ultimately
|
||||
`posix_spawn()` or `fork+exec`, from the parent's main
|
||||
interpreter that is currently running `trio.run()`. This
|
||||
brushes against a known-fragile interaction between
|
||||
`trio` and `fork()` tracked in
|
||||
[python-trio/trio#1614](https://github.com/python-trio/trio/issues/1614)
|
||||
and siblings — mostly mitigated in `tractor`'s case only
|
||||
incidentally (we `exec()` immediately post-fork).
|
||||
|
||||
The idea was:
|
||||
|
||||
1. Create a subint that has *never* imported `trio`.
|
||||
2. From a worker thread in that subint, call `os.fork()`.
|
||||
3. In the child, `execv()` back into
|
||||
`python -m tractor._child` — same as `trio_proc` does.
|
||||
4. The fork is from a trio-free context → trio+fork
|
||||
hazards avoided regardless of downstream behavior.
|
||||
|
||||
The parent-side orchestration (`ipc_server.wait_for_peer`,
|
||||
`SpawnSpec`, `Portal` yield) would reuse
|
||||
`trio_proc`'s flow verbatim, with only the subproc-spawn
|
||||
mechanics swapped.
|
||||
|
||||
## Symptom
|
||||
|
||||
Running the prototype (`tractor.spawn._subint_fork.subint_fork_proc`,
|
||||
see git history prior to the stub revert) on py3.14:
|
||||
|
||||
```
|
||||
Fatal Python error: _PyInterpreterState_DeleteExceptMain: not main interpreter
|
||||
Python runtime state: initialized
|
||||
|
||||
Current thread 0x00007f6b71a456c0 [subint-fork-lau] (most recent call first):
|
||||
File "<script>", line 2 in <module>
|
||||
<script>:2: DeprecationWarning: This process (pid=802985) is multi-threaded, use of fork() may lead to deadlocks in the child.
|
||||
```
|
||||
|
||||
Key clues:
|
||||
|
||||
- The **`DeprecationWarning`** fires in the parent (before
|
||||
fork completes) — fork *is* executing, we get that far.
|
||||
- The **`Fatal Python error`** comes from the child — it
|
||||
aborts during CPython's post-fork C initialization
|
||||
before any user Python runs in the child.
|
||||
- The thread name `subint-fork-lau[nchpad]` is ours —
|
||||
confirms the fork is being called from the launchpad
|
||||
subint's driver thread.
|
||||
|
||||
## CPython source walkthrough
|
||||
|
||||
### Call site — `Modules/posixmodule.c:728-793`
|
||||
|
||||
The post-fork-child hook CPython runs in the child process:
|
||||
|
||||
```c
|
||||
void
|
||||
PyOS_AfterFork_Child(void)
|
||||
{
|
||||
PyStatus status;
|
||||
_PyRuntimeState *runtime = &_PyRuntime;
|
||||
|
||||
// re-creates runtime->interpreters.mutex (HEAD_UNLOCK)
|
||||
status = _PyRuntimeState_ReInitThreads(runtime);
|
||||
...
|
||||
|
||||
PyThreadState *tstate = _PyThreadState_GET();
|
||||
_Py_EnsureTstateNotNULL(tstate);
|
||||
|
||||
...
|
||||
|
||||
// Ideally we could guarantee tstate is running main. ← !!!
|
||||
_PyInterpreterState_ReinitRunningMain(tstate);
|
||||
|
||||
status = _PyEval_ReInitThreads(tstate);
|
||||
...
|
||||
|
||||
status = _PyInterpreterState_DeleteExceptMain(runtime);
|
||||
if (_PyStatus_EXCEPTION(status)) {
|
||||
goto fatal_error;
|
||||
}
|
||||
...
|
||||
|
||||
fatal_error:
|
||||
Py_ExitStatusException(status);
|
||||
}
|
||||
```
|
||||
|
||||
The `// Ideally we could guarantee tstate is running
|
||||
main.` comment is a flashing warning sign — the CPython
|
||||
devs *know* this path is fragile when fork is called from
|
||||
a non-main subint, but they've chosen to abort rather than
|
||||
silently corrupt state. Arguably the right call.
|
||||
|
||||
### The refusal — `Python/pystate.c:1035-1075`
|
||||
|
||||
```c
|
||||
/*
|
||||
* Delete all interpreter states except the main interpreter. If there
|
||||
* is a current interpreter state, it *must* be the main interpreter.
|
||||
*/
|
||||
PyStatus
|
||||
_PyInterpreterState_DeleteExceptMain(_PyRuntimeState *runtime)
|
||||
{
|
||||
struct pyinterpreters *interpreters = &runtime->interpreters;
|
||||
|
||||
PyThreadState *tstate = _PyThreadState_Swap(runtime, NULL);
|
||||
if (tstate != NULL && tstate->interp != interpreters->main) {
|
||||
return _PyStatus_ERR("not main interpreter"); ← our error
|
||||
}
|
||||
|
||||
HEAD_LOCK(runtime);
|
||||
PyInterpreterState *interp = interpreters->head;
|
||||
interpreters->head = NULL;
|
||||
while (interp != NULL) {
|
||||
if (interp == interpreters->main) {
|
||||
interpreters->main->next = NULL;
|
||||
interpreters->head = interp;
|
||||
interp = interp->next;
|
||||
continue;
|
||||
}
|
||||
|
||||
// XXX Won't this fail since PyInterpreterState_Clear() requires
|
||||
// the "current" tstate to be set?
|
||||
PyInterpreterState_Clear(interp); // XXX must activate?
|
||||
zapthreads(interp);
|
||||
...
|
||||
}
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
The comment in the docstring (`If there is a current
|
||||
interpreter state, it *must* be the main interpreter.`) is
|
||||
the formal API contract. The `XXX` comments further in
|
||||
suggest the CPython team is already aware this function
|
||||
has latent issues even in the happy path.
|
||||
|
||||
## Chain summary
|
||||
|
||||
1. Our launchpad subint's driver OS-thread calls
|
||||
`os.fork()`.
|
||||
2. `fork()` succeeds. Child wakes up with:
|
||||
- The parent's full memory image (including all
|
||||
subints).
|
||||
- Only the *calling* thread alive (the driver thread).
|
||||
- `_PyThreadState_GET()` on that thread returns the
|
||||
**launchpad subint's tstate**, *not* main's.
|
||||
3. CPython runs `PyOS_AfterFork_Child()`.
|
||||
4. It reaches `_PyInterpreterState_DeleteExceptMain()`.
|
||||
5. Gate check fails: `tstate->interp != interpreters->main`.
|
||||
6. `PyStatus_ERR("not main interpreter")` → `fatal_error`
|
||||
goto → `Py_ExitStatusException()` → child aborts.
|
||||
|
||||
Parent-side consequence: `os.fork()` in the subint
|
||||
bootstrap returned successfully with the child's PID, but
|
||||
the child died before connecting back. Our parent's
|
||||
`ipc_server.wait_for_peer(uid)` would hang forever — the
|
||||
child never gets to `_actor_child_main`.
|
||||
|
||||
## Definitive answer to "Open Question 1"
|
||||
|
||||
From the (now-stub) `subint_fork_proc` docstring:
|
||||
|
||||
> Does CPython allow `os.fork()` from a non-main
|
||||
> sub-interpreter under the legacy config?
|
||||
|
||||
**No.** Not in a usable-by-user-code sense. The fork
|
||||
syscall is not blocked, but the child cannot survive
|
||||
CPython's post-fork initialization. This is enforced, not
|
||||
accidental, and the CPython devs have acknowledged the
|
||||
fragility in-source.
|
||||
|
||||
## What we'd need from CPython to unblock
|
||||
|
||||
Any one of these, from least-to-most invasive:
|
||||
|
||||
1. **A pre-fork hook mechanism** that lets user code (or
|
||||
tractor itself via `os.register_at_fork(before=...)`)
|
||||
swap the current tstate to main before fork runs. The
|
||||
swap would need to work across the subint→main
|
||||
boundary, which is the actual hard part —
|
||||
`_PyThreadState_Swap()` exists but is internal.
|
||||
|
||||
2. **A `_PyInterpreterState_DeleteExceptFor(tstate->interp)`
|
||||
variant** that cleans up all *other* subints while
|
||||
preserving the calling subint's state. Lets the child
|
||||
continue executing in the subint after fork; a
|
||||
subsequent `execv()` clears everything at the OS
|
||||
level anyway.
|
||||
|
||||
3. **A cleaner error** than `Fatal Python error` aborting
|
||||
the child. Even without fixing the underlying
|
||||
capability, a raised Python-level exception in the
|
||||
parent's `fork()` call (rather than a silent child
|
||||
abort) would at least make the failure mode
|
||||
debuggable.
|
||||
|
||||
## Upstream-report draft (for CPython issue tracker)
|
||||
|
||||
### Title
|
||||
|
||||
> `os.fork()` from a non-main sub-interpreter aborts the
|
||||
> child with a fatal error in `PyOS_AfterFork_Child`; can
|
||||
> we at least make it a clean `RuntimeError` in the
|
||||
> parent?
|
||||
|
||||
### Body
|
||||
|
||||
> **Version**: Python 3.14.x
|
||||
>
|
||||
> **Summary**: Calling `os.fork()` from a thread currently
|
||||
> executing inside a sub-interpreter causes the forked
|
||||
> child process to abort during CPython's post-fork
|
||||
> cleanup, with the following output in the child:
|
||||
>
|
||||
> ```
|
||||
> Fatal Python error: _PyInterpreterState_DeleteExceptMain: not main interpreter
|
||||
> ```
|
||||
>
|
||||
> From the **parent's** point of view the fork succeeded
|
||||
> (returned a valid child PID). The failure is completely
|
||||
> opaque to parent-side Python code — unless the parent
|
||||
> does `os.waitpid()` it won't even notice the child
|
||||
> died.
|
||||
>
|
||||
> **Root cause** (as I understand it from reading sources):
|
||||
> `Modules/posixmodule.c::PyOS_AfterFork_Child()` calls
|
||||
> `_PyInterpreterState_DeleteExceptMain()` with a
|
||||
> precondition that `_PyThreadState_GET()->interp` be the
|
||||
> main interpreter. When `fork()` is called from a thread
|
||||
> executing inside a subinterpreter, the child wakes up
|
||||
> with its tstate still pointing at the subint, and the
|
||||
> gate in `Python/pystate.c:1044-1047` fails.
|
||||
>
|
||||
> A comment in the source
|
||||
> (`Modules/posixmodule.c:753` — `// Ideally we could
|
||||
> guarantee tstate is running main.`) suggests this is a
|
||||
> known-fragile path rather than an intentional
|
||||
> invariant.
|
||||
>
|
||||
> **Use case**: I was experimenting with using a
|
||||
> sub-interpreter as a "fork launchpad" — have a subint
|
||||
> that has never imported `trio`, call `os.fork()` from
|
||||
> that subint's thread, and in the child `execv()` back
|
||||
> into a fresh Python interpreter process. The goal was
|
||||
> to sidestep known issues with `trio` + `fork()`
|
||||
> interaction (see
|
||||
> [python-trio/trio#1614](https://github.com/python-trio/trio/issues/1614))
|
||||
> by guaranteeing the forking context had never been
|
||||
> "contaminated" by trio's imports or globals. This
|
||||
> approach would allow `trio`-using applications to
|
||||
> combine `fork`-based subprocess spawning with
|
||||
> per-worker `trio.run()` runtimes — a fairly common
|
||||
> pattern that currently requires workarounds.
|
||||
>
|
||||
> **Request**:
|
||||
>
|
||||
> Ideally: make fork-from-subint work (e.g., by swapping
|
||||
> the caller's tstate to main in the pre-fork hook), or
|
||||
> provide a `_PyInterpreterState_DeleteExceptFor(interp)`
|
||||
> variant that permits the caller's subint to survive
|
||||
> post-fork so user code can subsequently `execv()`.
|
||||
>
|
||||
> Minimally: convert the fatal child-side abort into a
|
||||
> clean `RuntimeError` (or similar) raised in the
|
||||
> parent's `fork()` call. Even if the capability isn't
|
||||
> expanded, the failure mode should be debuggable by
|
||||
> user-code in the parent — right now it's a silent
|
||||
> child death with an error message buried in the
|
||||
> child's stderr that parent code can't programmatically
|
||||
> see.
|
||||
>
|
||||
> **Related**: PEP 684 (per-interpreter GIL), PEP 734
|
||||
> (`concurrent.interpreters` public API). The private
|
||||
> `_interpreters` module is what I used to create the
|
||||
> launchpad — behavior is the same whether using
|
||||
> `_interpreters.create('legacy')` or
|
||||
> `concurrent.interpreters.create()` (the latter was not
|
||||
> tested but the gate is identical).
|
||||
>
|
||||
> Happy to contribute a minimal reproducer + test case if
|
||||
> this is something the team wants to pursue.
|
||||
|
||||
## References
|
||||
|
||||
- `Modules/posixmodule.c:728` —
|
||||
[`PyOS_AfterFork_Child`](https://github.com/python/cpython/blob/main/Modules/posixmodule.c#L728)
|
||||
- `Python/pystate.c:1040` —
|
||||
[`_PyInterpreterState_DeleteExceptMain`](https://github.com/python/cpython/blob/main/Python/pystate.c#L1040)
|
||||
- PEP 684 (per-interpreter GIL):
|
||||
<https://peps.python.org/pep-0684/>
|
||||
- PEP 734 (`concurrent.interpreters` public API):
|
||||
<https://peps.python.org/pep-0734/>
|
||||
- [python-trio/trio#1614](https://github.com/python-trio/trio/issues/1614)
|
||||
— the original motivation for the launchpad idea.
|
||||
- tractor issue #379 — "Our own thoughts, ideas for
|
||||
`fork()`-workaround/hacks..." section where this was
|
||||
first sketched.
|
||||
- `tractor.spawn._subint_fork` — in-tree stub preserving
|
||||
the attempted impl's shape in git history.
|
||||
|
|
@ -1,375 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
'''
|
||||
Standalone CPython-level feasibility check for the "main-interp
|
||||
worker-thread forkserver + subint-hosted trio" architecture
|
||||
proposed as a workaround to the CPython-level refusal
|
||||
documented in
|
||||
`ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`.
|
||||
|
||||
Purpose
|
||||
-------
|
||||
Deliberately NOT a `tractor` test. Zero `tractor` imports.
|
||||
Uses `_interpreters` (private stdlib) + `os.fork()` directly so
|
||||
the signal is unambiguous — pass/fail here is a property of
|
||||
CPython alone, independent of our runtime.
|
||||
|
||||
Run each scenario in isolation; the child's fate is observable
|
||||
only via `os.waitpid()` of the parent and the scenario's own
|
||||
status prints.
|
||||
|
||||
Scenarios (pick one with `--scenario <name>`)
|
||||
---------------------------------------------
|
||||
|
||||
- `control_subint_thread_fork` — the KNOWN-BROKEN case we
|
||||
documented in `subint_fork_blocked_by_cpython_post_fork_issue.md`:
|
||||
drive a subint from a thread, call `os.fork()` inside its
|
||||
`_interpreters.exec()`, watch the child abort. **Included as
|
||||
a control** — if this scenario DOESN'T abort the child, our
|
||||
analysis is wrong and we should re-check everything.
|
||||
|
||||
- `main_thread_fork` — baseline sanity. Call `os.fork()` from
|
||||
the process's main thread. Must always succeed; if this
|
||||
fails something much bigger is broken.
|
||||
|
||||
- `worker_thread_fork` — the architectural assertion. Spawn a
|
||||
regular `threading.Thread` (attached to main interp, NOT a
|
||||
subint), have IT call `os.fork()`. Child should survive
|
||||
post-fork cleanup.
|
||||
|
||||
- `full_architecture` — end-to-end: main-interp worker thread
|
||||
forks. In the child, fork-thread (still main-interp) creates
|
||||
a subint, drives a second worker thread inside it that runs
|
||||
a trivial `trio.run()`. Validates the "root runtime lives in
|
||||
a subint in the child" piece of the proposed arch.
|
||||
|
||||
All scenarios print a self-contained pass/fail banner. Exit
|
||||
code 0 on expected outcome (which for `control_*` means "child
|
||||
aborted", not "child succeeded"!).
|
||||
|
||||
Requires Python 3.14+.
|
||||
|
||||
Usage
|
||||
-----
|
||||
::
|
||||
|
||||
python subint_fork_from_main_thread_smoketest.py \\
|
||||
--scenario main_thread_fork
|
||||
|
||||
python subint_fork_from_main_thread_smoketest.py \\
|
||||
--scenario full_architecture
|
||||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
|
||||
# Hard-require py3.14 for the public `concurrent.interpreters`
|
||||
# API (we still drop to `_interpreters` internally, same as
|
||||
# `tractor.spawn._subint`).
|
||||
try:
|
||||
from concurrent import interpreters as _public_interpreters # noqa: F401
|
||||
import _interpreters # type: ignore
|
||||
except ImportError:
|
||||
print(
|
||||
'FAIL (setup): requires Python 3.14+ '
|
||||
'(missing `concurrent.interpreters`)',
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
# The actual primitives this script exercises live in
|
||||
# `tractor.spawn._subint_forkserver` — we re-import them here
|
||||
# rather than inlining so the module and the validation stay
|
||||
# in sync. (Early versions of this file had them inline for
|
||||
# the "zero tractor imports" isolation guarantee; now that
|
||||
# CPython-level feasibility is confirmed, the validated
|
||||
# primitives have moved into tractor proper.)
|
||||
from tractor.spawn._main_thread_forkserver import (
|
||||
fork_from_worker_thread,
|
||||
wait_child,
|
||||
)
|
||||
from tractor.spawn._subint_forkserver import (
|
||||
run_subint_in_worker_thread,
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# small observability helpers (test-harness only)
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
|
||||
def _banner(title: str) -> None:
|
||||
line = '=' * 60
|
||||
print(f'\n{line}\n{title}\n{line}', flush=True)
|
||||
|
||||
|
||||
def _report(
|
||||
label: str,
|
||||
*,
|
||||
ok: bool,
|
||||
status_str: str,
|
||||
expect_exit_ok: bool,
|
||||
) -> None:
|
||||
verdict: str = 'PASS' if ok else 'FAIL'
|
||||
expected_str: str = (
|
||||
'normal exit (rc=0)'
|
||||
if expect_exit_ok
|
||||
else 'abnormal death (signal or nonzero exit)'
|
||||
)
|
||||
print(
|
||||
f'[{verdict}] {label}: '
|
||||
f'expected {expected_str}; observed {status_str}',
|
||||
flush=True,
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# scenario: `control_subint_thread_fork` (known-broken)
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
|
||||
def scenario_control_subint_thread_fork() -> int:
|
||||
_banner(
|
||||
'[control] fork from INSIDE a subint (expected: child aborts)'
|
||||
)
|
||||
interp_id = _interpreters.create('legacy')
|
||||
print(f' created subint {interp_id}', flush=True)
|
||||
|
||||
# Shared flag: child writes a sentinel file we can detect from
|
||||
# the parent. If the child manages to write this, CPython's
|
||||
# post-fork refusal is NOT happening → analysis is wrong.
|
||||
sentinel = '/tmp/subint_fork_smoketest_control_child_ran'
|
||||
try:
|
||||
os.unlink(sentinel)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
|
||||
bootstrap = (
|
||||
'import os\n'
|
||||
'pid = os.fork()\n'
|
||||
'if pid == 0:\n'
|
||||
# child — if CPython's refusal fires this code never runs
|
||||
f' with open({sentinel!r}, "w") as f:\n'
|
||||
' f.write("ran")\n'
|
||||
' os._exit(0)\n'
|
||||
'else:\n'
|
||||
# parent side (inside the launchpad subint) — stash the
|
||||
# forked PID on a shareable dict so we can waitpid()
|
||||
# from the outer main interp. We can't just return it;
|
||||
# _interpreters.exec() returns nothing useful.
|
||||
' import builtins\n'
|
||||
' builtins._forked_child_pid = pid\n'
|
||||
)
|
||||
|
||||
# NOTE, we can't easily pull state back from the subint.
|
||||
# For the CONTROL scenario we just time-bound the fork +
|
||||
# check the sentinel. If sentinel exists → child ran →
|
||||
# analysis wrong. If not → child aborted → analysis
|
||||
# confirmed.
|
||||
done = threading.Event()
|
||||
|
||||
def _drive() -> None:
|
||||
try:
|
||||
_interpreters.exec(interp_id, bootstrap)
|
||||
except Exception as err:
|
||||
print(
|
||||
f' subint bootstrap raised (expected on some '
|
||||
f'CPython versions): {type(err).__name__}: {err}',
|
||||
flush=True,
|
||||
)
|
||||
finally:
|
||||
done.set()
|
||||
|
||||
t = threading.Thread(
|
||||
target=_drive,
|
||||
name='control-subint-fork-launchpad',
|
||||
daemon=True,
|
||||
)
|
||||
t.start()
|
||||
done.wait(timeout=5.0)
|
||||
t.join(timeout=2.0)
|
||||
|
||||
# Give the (possibly-aborted) child a moment to die.
|
||||
time.sleep(0.5)
|
||||
|
||||
sentinel_present = os.path.exists(sentinel)
|
||||
verdict = (
|
||||
# "PASS" for our analysis means sentinel NOT present.
|
||||
'PASS' if not sentinel_present else 'FAIL (UNEXPECTED)'
|
||||
)
|
||||
print(
|
||||
f'[{verdict}] control: sentinel present={sentinel_present} '
|
||||
f'(analysis predicts False — child should abort before '
|
||||
f'writing)',
|
||||
flush=True,
|
||||
)
|
||||
if sentinel_present:
|
||||
os.unlink(sentinel)
|
||||
|
||||
try:
|
||||
_interpreters.destroy(interp_id)
|
||||
except _interpreters.InterpreterError:
|
||||
pass
|
||||
|
||||
return 0 if not sentinel_present else 1
|
||||
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# scenario: `main_thread_fork` (baseline sanity)
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
|
||||
def scenario_main_thread_fork() -> int:
|
||||
_banner(
|
||||
'[baseline] fork from MAIN thread (expected: child exits normally)'
|
||||
)
|
||||
|
||||
pid = os.fork()
|
||||
if pid == 0:
|
||||
os._exit(0)
|
||||
|
||||
return 0 if _wait_child(
|
||||
pid,
|
||||
label='main_thread_fork',
|
||||
expect_exit_ok=True,
|
||||
) else 1
|
||||
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# scenario: `worker_thread_fork` (architectural assertion)
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
|
||||
def _run_worker_thread_fork_scenario(
|
||||
label: str,
|
||||
*,
|
||||
child_target=None,
|
||||
) -> int:
|
||||
'''
|
||||
Thin wrapper: delegate the actual fork to the
|
||||
`tractor.spawn._subint_forkserver` primitive, then wait
|
||||
on the child and render a pass/fail banner.
|
||||
|
||||
'''
|
||||
try:
|
||||
pid: int = fork_from_worker_thread(
|
||||
child_target=child_target,
|
||||
thread_name=f'worker-fork-thread[{label}]',
|
||||
)
|
||||
except RuntimeError as err:
|
||||
print(f'[FAIL] {label}: {err}', flush=True)
|
||||
return 1
|
||||
print(f' forked child pid={pid}', flush=True)
|
||||
ok, status_str = wait_child(pid, expect_exit_ok=True)
|
||||
_report(
|
||||
label,
|
||||
ok=ok,
|
||||
status_str=status_str,
|
||||
expect_exit_ok=True,
|
||||
)
|
||||
return 0 if ok else 1
|
||||
|
||||
|
||||
def scenario_worker_thread_fork() -> int:
|
||||
_banner(
|
||||
'[arch] fork from MAIN-INTERP WORKER thread '
|
||||
'(expected: child exits normally — this is the one '
|
||||
'that matters)'
|
||||
)
|
||||
return _run_worker_thread_fork_scenario(
|
||||
'worker_thread_fork',
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# scenario: `full_architecture`
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
|
||||
_CHILD_TRIO_BOOTSTRAP: str = (
|
||||
'import trio\n'
|
||||
'async def _main():\n'
|
||||
' await trio.sleep(0.05)\n'
|
||||
' return 42\n'
|
||||
'result = trio.run(_main)\n'
|
||||
'assert result == 42, f"trio.run returned {result}"\n'
|
||||
'print(" CHILD subint: trio.run OK, result=42", '
|
||||
'flush=True)\n'
|
||||
)
|
||||
|
||||
|
||||
def _child_trio_in_subint() -> int:
|
||||
'''
|
||||
CHILD-side `child_target`: drive a trivial `trio.run()`
|
||||
inside a fresh legacy-config subint on a worker thread,
|
||||
using the `tractor.spawn._subint_forkserver.run_subint_in_worker_thread`
|
||||
primitive. Returns 0 on success.
|
||||
|
||||
'''
|
||||
try:
|
||||
run_subint_in_worker_thread(
|
||||
_CHILD_TRIO_BOOTSTRAP,
|
||||
thread_name='child-subint-trio-thread',
|
||||
)
|
||||
except RuntimeError as err:
|
||||
print(
|
||||
f' CHILD: run_subint_in_worker_thread timed out / thread '
|
||||
f'never returned: {err}',
|
||||
flush=True,
|
||||
)
|
||||
return 3
|
||||
except BaseException as err:
|
||||
print(
|
||||
f' CHILD: subint bootstrap raised: '
|
||||
f'{type(err).__name__}: {err}',
|
||||
flush=True,
|
||||
)
|
||||
return 4
|
||||
return 0
|
||||
|
||||
|
||||
def scenario_full_architecture() -> int:
|
||||
_banner(
|
||||
'[arch-full] worker-thread fork + child runs trio in a '
|
||||
'subint (end-to-end proposed arch)'
|
||||
)
|
||||
return _run_worker_thread_fork_scenario(
|
||||
'full_architecture',
|
||||
child_target=_child_trio_in_subint,
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# main
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
|
||||
SCENARIOS: dict[str, Callable[[], int]] = {
|
||||
'control_subint_thread_fork': scenario_control_subint_thread_fork,
|
||||
'main_thread_fork': scenario_main_thread_fork,
|
||||
'worker_thread_fork': scenario_worker_thread_fork,
|
||||
'full_architecture': scenario_full_architecture,
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(
|
||||
description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
ap.add_argument(
|
||||
'--scenario',
|
||||
choices=sorted(SCENARIOS.keys()),
|
||||
required=True,
|
||||
)
|
||||
args = ap.parse_args()
|
||||
return SCENARIOS[args.scenario]()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
|
|
@ -1,385 +0,0 @@
|
|||
# `subint_forkserver` backend: orphaned-subactor SIGINT wedged in `epoll_wait`
|
||||
|
||||
Follow-up to the Phase C `subint_forkserver` spawn-backend
|
||||
PR (see `tractor.spawn._subint_forkserver`, issue #379).
|
||||
Surfaced by the xfail'd
|
||||
`tests/spawn/test_subint_forkserver.py::test_orphaned_subactor_sigint_cleanup_DRAFT`.
|
||||
|
||||
Related-but-distinct from
|
||||
`subint_cancel_delivery_hang_issue.md` (orphaned-channel
|
||||
park AFTER subint teardown) and
|
||||
`subint_sigint_starvation_issue.md` (GIL-starvation,
|
||||
SIGINT never delivered): here the SIGINT IS delivered,
|
||||
trio's handler IS installed, but trio's event loop never
|
||||
wakes — so the KBI-at-checkpoint → `_trio_main` catch path
|
||||
(which is the runtime's *intentional* OS-cancel design)
|
||||
never fires.
|
||||
|
||||
## TL;DR
|
||||
|
||||
When a `subint_forkserver`-spawned subactor is orphaned
|
||||
(parent `SIGKILL`'d, no IPC cancel path available) and then
|
||||
externally `SIGINT`'d, the subactor hangs in
|
||||
`trio/_core/_io_epoll.py::get_events` (epoll_wait)
|
||||
indefinitely — even though:
|
||||
|
||||
1. `threading.current_thread() is threading.main_thread()`
|
||||
post-fork (CPython 3.14 re-designates correctly).
|
||||
2. Trio's SIGINT handler IS installed in the subactor
|
||||
(`signal.getsignal(SIGINT)` returns
|
||||
`<function KIManager.install.<locals>.handler at 0x...>`).
|
||||
3. The kernel does deliver SIGINT — the signal arrives at
|
||||
the only thread in the process (the fork-inherited
|
||||
worker which IS now "main" per Python).
|
||||
|
||||
Yet `epoll_wait` does not return. Trio's wakeup-fd mechanism
|
||||
— the machinery that turns SIGINT into an epoll-wake — is
|
||||
somehow not firing the wakeup. Until that's fixed, the
|
||||
intentional "KBI-as-OS-cancel" path in
|
||||
`tractor/spawn/_entry.py::_trio_main:164` is unreachable
|
||||
for forkserver-spawned subactors whose parent dies.
|
||||
|
||||
## Symptom
|
||||
|
||||
Test: `tests/spawn/test_subint_forkserver.py::test_orphaned_subactor_sigint_cleanup_DRAFT`
|
||||
(currently marked `@pytest.mark.xfail(strict=True)`).
|
||||
|
||||
1. Harness subprocess brings up a tractor root actor +
|
||||
one `run_in_actor(_sleep_forever)` subactor via
|
||||
`try_set_start_method('subint_forkserver')`.
|
||||
2. Harness prints `CHILD_PID` (subactor) and
|
||||
`PARENT_READY` (root actor) markers to stdout.
|
||||
3. Test `os.kill(parent_pid, SIGKILL)` + `proc.wait()`
|
||||
to fully reap the root-actor harness.
|
||||
4. Child (now reparented to pid 1) is still alive.
|
||||
5. Test `os.kill(child_pid, SIGINT)` and polls
|
||||
`os.kill(child_pid, 0)` for up to 10s.
|
||||
6. **Observed**: the child is still alive at deadline —
|
||||
SIGINT did not unwedge the trio loop.
|
||||
|
||||
## What the "intentional" cancel path IS
|
||||
|
||||
`tractor/spawn/_entry.py::_trio_main:157-186` —
|
||||
|
||||
```python
|
||||
try:
|
||||
if infect_asyncio:
|
||||
actor._infected_aio = True
|
||||
run_as_asyncio_guest(trio_main)
|
||||
else:
|
||||
trio.run(trio_main)
|
||||
|
||||
except KeyboardInterrupt:
|
||||
logmeth = log.cancel
|
||||
exit_status: str = (
|
||||
'Actor received KBI (aka an OS-cancel)\n'
|
||||
...
|
||||
)
|
||||
```
|
||||
|
||||
The "KBI == OS-cancel" mapping IS the runtime's
|
||||
deliberate, documented design. An OS-level SIGINT should
|
||||
flow as: kernel → trio handler → KBI at trio checkpoint
|
||||
→ unwinds `async_main` → surfaces at `_trio_main`'s
|
||||
`except KeyboardInterrupt:` → `log.cancel` + clean `rc=0`.
|
||||
|
||||
**So fixing this hang is not "add a new SIGINT behavior" —
|
||||
it's "make the existing designed behavior actually fire in
|
||||
this backend config".** That's why option (B) ("fix root
|
||||
cause") is aligned with existing design intent, not a
|
||||
scope expansion.
|
||||
|
||||
## Evidence
|
||||
|
||||
### Positive control: standalone fork-from-worker + `trio.run(sleep_forever)` + SIGINT WORKS
|
||||
|
||||
```python
|
||||
import os, signal, time, trio
|
||||
from tractor.spawn._subint_forkserver import (
|
||||
fork_from_worker_thread, wait_child,
|
||||
)
|
||||
|
||||
def child_target() -> int:
|
||||
async def _main():
|
||||
try:
|
||||
await trio.sleep_forever()
|
||||
except KeyboardInterrupt:
|
||||
print('CHILD: caught KBI — trio SIGINT works!')
|
||||
return
|
||||
trio.run(_main)
|
||||
return 0
|
||||
|
||||
pid = fork_from_worker_thread(child_target, thread_name='trio-sigint-test')
|
||||
time.sleep(1.0)
|
||||
os.kill(pid, signal.SIGINT)
|
||||
wait_child(pid)
|
||||
```
|
||||
|
||||
Result: `CHILD: caught KBI — trio SIGINT works!` + clean
|
||||
exit. So the fork-child + trio signal plumbing IS healthy
|
||||
in isolation. The hang appears only with the full tractor
|
||||
subactor runtime on top.
|
||||
|
||||
### Negative test: full tractor subactor + orphan-SIGINT
|
||||
|
||||
Equivalent to the xfail test. Traceback dump via
|
||||
`faulthandler.register(SIGUSR1, all_threads=True)` at the
|
||||
stuck moment:
|
||||
|
||||
```
|
||||
Current thread 0x00007... [subint-forkserv] (most recent call first):
|
||||
File ".../trio/_core/_io_epoll.py", line 245 in get_events
|
||||
File ".../trio/_core/_run.py", line 2415 in run
|
||||
File "tractor/spawn/_entry.py", line 162 in _trio_main
|
||||
File "tractor/_child.py", line 72 in _actor_child_main
|
||||
File "tractor/spawn/_subint_forkserver.py", line 650 in _child_target
|
||||
File "tractor/spawn/_subint_forkserver.py", line 308 in _worker
|
||||
File ".../threading.py", line 1024 in run
|
||||
```
|
||||
|
||||
### Thread + signal-mask inventory of the stuck subactor
|
||||
|
||||
Single thread (`tid == pid`, comm `'subint-forkserv'`,
|
||||
which IS `threading.main_thread()` post-fork):
|
||||
|
||||
```
|
||||
SigBlk: 0000000000000000 # nothing blocked
|
||||
SigIgn: 0000000001001000 # SIGPIPE etc (Python defaults)
|
||||
SigCgt: 0000000108000202 # bit 1 = SIGINT caught
|
||||
```
|
||||
|
||||
Bit 1 set in `SigCgt` → SIGINT handler IS installed. So
|
||||
trio's handler IS in place at the kernel level — not a
|
||||
"handler missing" situation.
|
||||
|
||||
### Handler identity
|
||||
|
||||
Inside the subactor's RPC body, `signal.getsignal(SIGINT)`
|
||||
returns `<function KIManager.install.<locals>.handler at
|
||||
0x...>` — trio's own `KIManager` handler. tractor's only
|
||||
SIGINT touches are `signal.getsignal()` *reads* (to stash
|
||||
into `debug.DebugStatus._trio_handler`); nothing writes
|
||||
over trio's handler outside the debug-REPL shielding path
|
||||
(`devx/debug/_tty_lock.py::shield_sigint`) which isn't
|
||||
engaged here (no debug_mode).
|
||||
|
||||
## Ruled out
|
||||
|
||||
- **GIL starvation / signal-pipe-full** (class A,
|
||||
`subint_sigint_starvation_issue.md`): subactor runs on
|
||||
its own GIL (separate OS process), not sharing with the
|
||||
parent → no cross-process GIL contention. And `strace`-
|
||||
equivalent in the signal mask shows SIGINT IS caught,
|
||||
not queued.
|
||||
- **Orphaned channel park** (`subint_cancel_delivery_hang_issue.md`):
|
||||
different failure mode — that one has trio iterating
|
||||
normally and getting wedged on an orphaned
|
||||
`chan.recv()` AFTER teardown. Here trio's event loop
|
||||
itself never wakes.
|
||||
- **Tractor explicitly catching + swallowing KBI**:
|
||||
greppable — the one `except KeyboardInterrupt:` in the
|
||||
runtime is the INTENTIONAL cancel-path catch at
|
||||
`_trio_main:164`. `async_main` uses `except Exception`
|
||||
(not BaseException), so KBI should propagate through
|
||||
cleanly if it ever fires.
|
||||
- **Missing `signal.set_wakeup_fd` (main-thread
|
||||
restriction)**: post-fork, the fork-worker thread IS
|
||||
`threading.main_thread()`, so trio's main-thread check
|
||||
passes and its wakeup-fd install should succeed.
|
||||
|
||||
## Root cause hypothesis (unverified)
|
||||
|
||||
The SIGINT handler fires but trio's wakeup-fd write does
|
||||
not wake `epoll_wait`. Candidate causes, ranked by
|
||||
plausibility:
|
||||
|
||||
1. **Wakeup-fd lifecycle race around tractor IPC setup.**
|
||||
`async_main` spins up an IPC server + `process_messages`
|
||||
loops early. Somewhere in that path the wakeup-fd that
|
||||
trio registered with its epoll instance may be
|
||||
closed/replaced/clobbered, so subsequent SIGINT writes
|
||||
land on an fd that's no longer in the epoll set.
|
||||
Evidence needed: compare
|
||||
`signal.set_wakeup_fd(-1)` return value inside a
|
||||
post-tractor-bringup RPC body vs. a pre-bringup
|
||||
equivalent. If they differ, that's it.
|
||||
2. **Shielded cancel scope around `process_messages`.**
|
||||
The RPC message loop is likely wrapped in a trio cancel
|
||||
scope; if that scope is `shield=True` at any outer
|
||||
layer, KBI scheduled at a checkpoint could be absorbed
|
||||
by the shield and never bubble out to `_trio_main`.
|
||||
3. **Pre-fork wakeup-fd inheritance.** trio in the PARENT
|
||||
process registered a wakeup-fd with its own epoll. The
|
||||
child inherits the fd number but not the parent's
|
||||
epoll instance — if tractor/trio re-uses the parent's
|
||||
stale fd number anywhere, writes would go to a no-op
|
||||
fd. (This is the least likely — `trio.run()` on the
|
||||
child calls `KIManager.install` which should install a
|
||||
fresh wakeup-fd from scratch.)
|
||||
|
||||
## Cross-backend scope question
|
||||
|
||||
**Untested**: does the same orphan-SIGINT hang reproduce
|
||||
against the `trio_proc` backend (stock subprocess + exec)?
|
||||
If yes → pre-existing tractor bug, independent of
|
||||
`subint_forkserver`. If no → something specific to the
|
||||
fork-from-worker path (e.g. inherited fds, mid-epoll-setup
|
||||
interference).
|
||||
|
||||
**Quick repro for trio_proc**:
|
||||
|
||||
```python
|
||||
# save as /tmp/trio_proc_orphan_sigint_repro.py
|
||||
import os, sys, signal, time, glob
|
||||
import subprocess as sp
|
||||
|
||||
SCRIPT = '''
|
||||
import os, sys, trio, tractor
|
||||
async def _sleep_forever():
|
||||
print(f"CHILD_PID={os.getpid()}", flush=True)
|
||||
await trio.sleep_forever()
|
||||
|
||||
async def _main():
|
||||
async with (
|
||||
tractor.open_root_actor(registry_addrs=[("127.0.0.1", 12350)]),
|
||||
tractor.open_nursery() as an,
|
||||
):
|
||||
await an.run_in_actor(_sleep_forever, name="sf-child")
|
||||
print(f"PARENT_READY={os.getpid()}", flush=True)
|
||||
await trio.sleep_forever()
|
||||
|
||||
trio.run(_main)
|
||||
'''
|
||||
|
||||
proc = sp.Popen(
|
||||
[sys.executable, '-c', SCRIPT],
|
||||
stdout=sp.PIPE, stderr=sp.STDOUT,
|
||||
)
|
||||
# parse CHILD_PID + PARENT_READY off proc.stdout ...
|
||||
# SIGKILL parent, SIGINT child, poll.
|
||||
```
|
||||
|
||||
If that hangs too, open a broader issue; if not, this is
|
||||
`subint_forkserver`-specific (likely fd-inheritance-related).
|
||||
|
||||
## Why this is ours to fix (not CPython's)
|
||||
|
||||
- Signal IS delivered (`SigCgt` bitmask confirms).
|
||||
- Handler IS installed (trio's `KIManager`).
|
||||
- Thread identity is correct post-fork.
|
||||
- `_trio_main` already has the intentional KBI→clean-exit
|
||||
path waiting to fire.
|
||||
|
||||
Every CPython-level precondition is met. Something in
|
||||
tractor's runtime or trio's integration with it is
|
||||
breaking the SIGINT→wakeup→event-loop-wake pipeline.
|
||||
|
||||
## Possible fix directions
|
||||
|
||||
1. **Audit the wakeup-fd across tractor's IPC bringup.**
|
||||
Add a trio startup hook that captures
|
||||
`signal.set_wakeup_fd(-1)` at `_trio_main` entry,
|
||||
after `async_main` enters, and periodically — assert
|
||||
it's unchanged. If it moves, track down the writer.
|
||||
2. **Explicit `signal.set_wakeup_fd` reset after IPC
|
||||
setup.** Brute force: re-install a fresh wakeup-fd
|
||||
mid-bringup. Band-aid, but fast to try.
|
||||
3. **Ensure no `shield=True` cancel scope envelopes the
|
||||
RPC-message-loop / IPC-server task.** If one does,
|
||||
KBI-at-checkpoint never escapes.
|
||||
4. **Once fixed, the `child_sigint='trio'` mode on
|
||||
`subint_forkserver_proc`** becomes effectively a no-op
|
||||
or a doc-only mode — trio's natural handler already
|
||||
does the right thing. Might end up removing the flag
|
||||
entirely if there's no behavioral difference between
|
||||
modes.
|
||||
|
||||
## Current workaround
|
||||
|
||||
None; `child_sigint` defaults to `'ipc'` (IPC cancel is
|
||||
the only reliable cancel path today), and the xfail test
|
||||
documents the gap. Operators hitting orphan-SIGINT get a
|
||||
hung process that needs `SIGKILL`.
|
||||
|
||||
## Reproducer
|
||||
|
||||
Inline, standalone (no pytest):
|
||||
|
||||
```python
|
||||
# save as /tmp/orphan_sigint_repro.py (py3.14+)
|
||||
import os, sys, signal, time, glob, trio
|
||||
import tractor
|
||||
from tractor.spawn._subint_forkserver import (
|
||||
fork_from_worker_thread,
|
||||
)
|
||||
|
||||
async def _sleep_forever():
|
||||
print(f'SUBACTOR[{os.getpid()}]', flush=True)
|
||||
await trio.sleep_forever()
|
||||
|
||||
async def _main():
|
||||
async with (
|
||||
tractor.open_root_actor(
|
||||
registry_addrs=[('127.0.0.1', 12349)],
|
||||
),
|
||||
tractor.open_nursery() as an,
|
||||
):
|
||||
await an.run_in_actor(_sleep_forever, name='sf-child')
|
||||
await trio.sleep_forever()
|
||||
|
||||
def child_target() -> int:
|
||||
from tractor.spawn._spawn import try_set_start_method
|
||||
try_set_start_method('subint_forkserver')
|
||||
trio.run(_main)
|
||||
return 0
|
||||
|
||||
pid = fork_from_worker_thread(child_target, thread_name='repro')
|
||||
time.sleep(3.0)
|
||||
|
||||
# find the subactor pid via /proc
|
||||
children = []
|
||||
for path in glob.glob(f'/proc/{pid}/task/*/children'):
|
||||
with open(path) as f:
|
||||
children.extend(int(x) for x in f.read().split() if x)
|
||||
subactor_pid = children[0]
|
||||
|
||||
# SIGKILL root → orphan the subactor
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
os.waitpid(pid, 0)
|
||||
time.sleep(0.3)
|
||||
|
||||
# SIGINT the orphan — should cause clean trio exit
|
||||
os.kill(subactor_pid, signal.SIGINT)
|
||||
|
||||
# poll for exit
|
||||
for _ in range(100):
|
||||
try:
|
||||
os.kill(subactor_pid, 0)
|
||||
time.sleep(0.1)
|
||||
except ProcessLookupError:
|
||||
print('HARNESS: subactor exited cleanly ✔')
|
||||
sys.exit(0)
|
||||
os.kill(subactor_pid, signal.SIGKILL)
|
||||
print('HARNESS: subactor hung — reproduced')
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
Expected (current): `HARNESS: subactor hung — reproduced`.
|
||||
|
||||
After fix: `HARNESS: subactor exited cleanly ✔`.
|
||||
|
||||
## References
|
||||
|
||||
- `tractor/spawn/_entry.py::_trio_main:157-186` — the
|
||||
intentional KBI→clean-exit path this bug makes
|
||||
unreachable.
|
||||
- `tractor/spawn/_subint_forkserver` — the backend whose
|
||||
orphan cancel-robustness this blocks.
|
||||
- `tests/spawn/test_subint_forkserver.py::test_orphaned_subactor_sigint_cleanup_DRAFT`
|
||||
— the xfail'd reproducer in the test suite.
|
||||
- `ai/conc-anal/subint_cancel_delivery_hang_issue.md` —
|
||||
sibling "orphaned channel park" hang (different class).
|
||||
- `ai/conc-anal/subint_sigint_starvation_issue.md` —
|
||||
sibling "GIL starvation SIGINT drop" hang (different
|
||||
class).
|
||||
- tractor issue #379 — subint backend tracking.
|
||||
|
|
@ -1,851 +0,0 @@
|
|||
# `subint_forkserver` backend: `test_cancellation.py` multi-level cancel cascade hang
|
||||
|
||||
> **Tracked at:** [#449](https://github.com/goodboy/tractor/issues/449)
|
||||
|
||||
Follow-up tracker: surfaced while wiring the new
|
||||
`subint_forkserver` spawn backend into the full tractor
|
||||
test matrix (step 2 of the post-backend-lands plan).
|
||||
See also
|
||||
`ai/conc-anal/subint_forkserver_orphan_sigint_hang_issue.md`
|
||||
— sibling tracker for a different forkserver-teardown
|
||||
class which probably shares the same fundamental root
|
||||
cause (fork-FD-inheritance across nested spawns).
|
||||
|
||||
## TL;DR
|
||||
|
||||
`tests/test_cancellation.py::test_nested_multierrors[subint_forkserver]`
|
||||
hangs indefinitely under our new backend. The hang is
|
||||
**inside the graceful IPC cancel cascade** — every actor
|
||||
in the multi-level tree parks in `epoll_wait` waiting
|
||||
for IPC messages that never arrive. Not a hard-kill /
|
||||
tree-reap issue (we don't reach the hard-kill fallback
|
||||
path at all).
|
||||
|
||||
Working hypothesis (unverified): **`os.fork()` from a
|
||||
subactor inherits the root parent's IPC listener socket
|
||||
FDs**. When a first-level subactor forkserver-spawns a
|
||||
grandchild, that grandchild inherits both its direct
|
||||
spawner's FDs AND the root's FDs — IPC message routing
|
||||
becomes ambiguous (or silently sends to the wrong
|
||||
channel), so the cancel cascade can't reach its target.
|
||||
|
||||
## Corrected diagnosis vs. earlier draft
|
||||
|
||||
An earlier version of this doc claimed the root cause
|
||||
was **"forkserver teardown doesn't tree-kill
|
||||
descendants"** (SIGKILL only reaches the direct child,
|
||||
grandchildren survive and hold TCP `:1616`). That
|
||||
diagnosis was **wrong**, caused by conflating two
|
||||
observations:
|
||||
|
||||
1. *5-zombie leak holding :1616* — happened in my own
|
||||
workflow when I aborted a bg pytest task with
|
||||
`pkill` (SIGTERM/SIGKILL, not SIGINT). The abrupt
|
||||
kill skipped the graceful `ActorNursery.__aexit__`
|
||||
cancel cascade entirely, orphaning descendants to
|
||||
init. **This was my cleanup bug, not a forkserver
|
||||
teardown bug.** Codified the fix (SIGINT-first +
|
||||
bounded wait before SIGKILL) in
|
||||
`feedback_sc_graceful_cancel_first.md` +
|
||||
`.claude/skills/run-tests/SKILL.md`.
|
||||
2. *`test_nested_multierrors` hangs indefinitely* —
|
||||
the real, separate, forkserver-specific bug
|
||||
captured by this doc.
|
||||
|
||||
The two symptoms are unrelated. The tree-kill / setpgrp
|
||||
fix direction proposed earlier would not help (1) (SC-
|
||||
graceful-cleanup is the right answer there) and would
|
||||
not help (2) (the hang is in the cancel cascade, not
|
||||
in the hard-kill fallback).
|
||||
|
||||
## Symptom
|
||||
|
||||
Reproducer (py3.14, clean env):
|
||||
|
||||
```sh
|
||||
# preflight: ensure clean env
|
||||
ss -tlnp 2>/dev/null | grep ':1616' && echo 'FOUL — cleanup first!' || echo 'clean'
|
||||
|
||||
./py314/bin/python -m pytest --spawn-backend=subint_forkserver \
|
||||
'tests/test_cancellation.py::test_nested_multierrors[subint_forkserver]' \
|
||||
--timeout=30 --timeout-method=thread --tb=short -v
|
||||
```
|
||||
|
||||
Expected: `pytest-timeout` fires at 30s with a thread-
|
||||
dump banner, but the process itself **remains alive
|
||||
after timeout** and doesn't unwedge on subsequent
|
||||
SIGINT. Requires SIGKILL to reap.
|
||||
|
||||
## Evidence (tree structure at hang point)
|
||||
|
||||
All 5 processes are kernel-level `S` (sleeping) in
|
||||
`do_epoll_wait` (trio's event loop waiting on I/O):
|
||||
|
||||
```
|
||||
PID PPID THREADS NAME ROLE
|
||||
333986 1 2 subint-forkserv pytest main (the test body)
|
||||
333993 333986 3 subint-forkserv "child 1" spawner subactor
|
||||
334003 333993 1 subint-forkserv grandchild errorer under child-1
|
||||
334014 333993 1 subint-forkserv grandchild errorer under child-1
|
||||
333999 333986 1 subint-forkserv "child 2" spawner subactor (NO grandchildren!)
|
||||
```
|
||||
|
||||
### Asymmetric tree depth
|
||||
|
||||
The test's `spawn_and_error(breadth=2, depth=3)` should
|
||||
have BOTH direct children spawning 2 grandchildren
|
||||
each, going 3 levels deep. Reality:
|
||||
|
||||
- Child 1 (333993, 3 threads) DID spawn its two
|
||||
grandchildren as expected — fully booted trio
|
||||
runtime.
|
||||
- Child 2 (333999, 1 thread) did NOT spawn any
|
||||
grandchildren — clearly never completed its
|
||||
nursery's first `run_in_actor`. Its 1-thread state
|
||||
suggests the runtime never fully booted (no trio
|
||||
worker threads for `waitpid`/IPC).
|
||||
|
||||
This asymmetry is the key clue: the two direct
|
||||
children started identically but diverged. Probably a
|
||||
race around fork-inherited state (listener FDs,
|
||||
subactor-nursery channel state) that happens to land
|
||||
differently depending on spawn ordering.
|
||||
|
||||
### Parent-side state
|
||||
|
||||
Thread-dump of pytest main (333986) at the hang:
|
||||
|
||||
- Main trio thread — parked in
|
||||
`trio._core._io_epoll.get_events` (epoll_wait on
|
||||
its event loop). Waiting for IPC from children.
|
||||
- Two trio-cache worker threads — each parked in
|
||||
`outcome.capture(sync_fn)` calling
|
||||
`os.waitpid(child_pid, 0)`. These are our
|
||||
`_ForkedProc.wait()` off-loads. They're waiting for
|
||||
the direct children to exit — but children are
|
||||
stuck in their own epoll_wait waiting for IPC from
|
||||
the parent.
|
||||
|
||||
**It's a deadlock, not a leak:** the parent is
|
||||
correctly running `soft_kill(proc, _ForkedProc.wait,
|
||||
portal)` (graceful IPC cancel via
|
||||
`Portal.cancel_actor()`), but the children never
|
||||
acknowledge the cancel message (or the message never
|
||||
reaches them through the tangled post-fork IPC).
|
||||
|
||||
## What's NOT the cause (ruled out)
|
||||
|
||||
- **`_ForkedProc.kill()` only SIGKILLs direct pid /
|
||||
missing tree-kill**: doesn't apply — we never reach
|
||||
the hard-kill path. The deadlock is in the graceful
|
||||
cancel cascade.
|
||||
- **Port `:1616` contention**: ruled out after the
|
||||
`reg_addr` fixture-wiring fix; each test session
|
||||
gets a unique port now.
|
||||
- **GIL starvation / SIGINT pipe filling** (class-A,
|
||||
`subint_sigint_starvation_issue.md`): doesn't apply
|
||||
— each subactor is its own OS process with its own
|
||||
GIL (not legacy-config subint).
|
||||
- **Child-side `_trio_main` absorbing KBI**: grep
|
||||
confirmed; `_trio_main` only catches KBI at the
|
||||
`trio.run()` callsite, which is reached only if the
|
||||
trio loop exits normally. The children here never
|
||||
exit trio.run() — they're wedged inside.
|
||||
|
||||
## Hypothesis: FD inheritance across nested forks
|
||||
|
||||
`subint_forkserver_proc` calls
|
||||
`fork_from_worker_thread()` which ultimately does
|
||||
`os.fork()` from a dedicated worker thread. Standard
|
||||
Linux/POSIX fork semantics: **the child inherits ALL
|
||||
open FDs from the parent**, including listener
|
||||
sockets, epoll fds, trio wakeup pipes, and the
|
||||
parent's IPC channel sockets.
|
||||
|
||||
At root-actor fork-spawn time, the root's IPC server
|
||||
listener FDs are open in the parent. Those get
|
||||
inherited by child 1. Child 1 then forkserver-spawns
|
||||
its OWN subactor (grandchild). The grandchild
|
||||
inherits FDs from child 1 — but child 1's address
|
||||
space still contains **the root's IPC listener FDs
|
||||
too** (inherited at first fork). So the grandchild
|
||||
has THREE sets of FDs:
|
||||
|
||||
1. Its own (created after becoming a subactor).
|
||||
2. Its direct parent child-1's.
|
||||
3. The ROOT's (grandparent's) — inherited transitively.
|
||||
|
||||
IPC message routing may be ambiguous in this tangled
|
||||
state. Or a listener socket that the root thinks it
|
||||
owns is actually open in multiple processes, and
|
||||
messages sent to it go to an arbitrary one. That
|
||||
would exactly match the observed "graceful cancel
|
||||
never propagates".
|
||||
|
||||
This hypothesis predicts the bug **scales with fork
|
||||
depth**: single-level forkserver spawn
|
||||
(`test_subint_forkserver_spawn_basic`) works
|
||||
perfectly, but any test that spawns a second level
|
||||
deadlocks. Matches observations so far.
|
||||
|
||||
## Fix directions (to validate)
|
||||
|
||||
### 1. `close_fds=True` equivalent in `fork_from_worker_thread()`
|
||||
|
||||
`subprocess.Popen` / `trio.lowlevel.open_process` have
|
||||
`close_fds=True` by default on POSIX — they
|
||||
enumerate open FDs in the child post-fork and close
|
||||
everything except stdio + any explicitly-passed FDs.
|
||||
Our raw `os.fork()` doesn't. Adding the equivalent to
|
||||
our `_worker` prelude would isolate each fork
|
||||
generation's FD set.
|
||||
|
||||
Implementation sketch in
|
||||
`tractor.spawn._subint_forkserver.fork_from_worker_thread._worker`:
|
||||
|
||||
```python
|
||||
def _worker() -> None:
|
||||
pid: int = os.fork()
|
||||
if pid == 0:
|
||||
# CHILD: close inherited FDs except stdio + the
|
||||
# pid-pipe we just opened.
|
||||
keep: set[int] = {0, 1, 2, rfd, wfd}
|
||||
import resource
|
||||
soft, _ = resource.getrlimit(resource.RLIMIT_NOFILE)
|
||||
os.closerange(3, soft) # blunt; or enumerate /proc/self/fd
|
||||
# ... then child_target() as before
|
||||
```
|
||||
|
||||
Problem: overly aggressive — closes FDs the
|
||||
grandchild might legitimately need (e.g. its parent's
|
||||
IPC channel for the spawn-spec handshake, if we rely
|
||||
on that). Needs thought about which FDs are
|
||||
"inheritable and safe" vs. "inherited by accident".
|
||||
|
||||
### 2. Cloexec on tractor's own FDs
|
||||
|
||||
Set `FD_CLOEXEC` on tractor-created sockets (listener
|
||||
sockets, IPC channel sockets, pipes). This flag
|
||||
causes automatic close on `execve`, but since we
|
||||
`fork()` without `exec()`, this alone doesn't help.
|
||||
BUT — combined with a child-side explicit close-
|
||||
non-cloexec loop, it gives us a way to mark "my
|
||||
private FDs" vs. "safe to inherit". Most robust, but
|
||||
requires tractor-wide audit.
|
||||
|
||||
### 3. Explicit FD cleanup in `_ForkedProc`/`_child_target`
|
||||
|
||||
Have `subint_forkserver_proc`'s `_child_target`
|
||||
closure explicitly close the parent-side IPC listener
|
||||
FDs before calling `_actor_child_main`. Requires
|
||||
being able to enumerate "the parent's listener FDs
|
||||
that the child shouldn't keep" — plausible via
|
||||
`Actor.ipc_server`'s socket objects.
|
||||
|
||||
### 4. Use `os.posix_spawn` with explicit `file_actions`
|
||||
|
||||
Instead of raw `os.fork()`, use `os.posix_spawn()`
|
||||
which supports explicit file-action specifications
|
||||
(close this FD, dup2 that FD). Cleaner semantics, but
|
||||
probably incompatible with our "no exec" requirement
|
||||
(subint_forkserver is a fork-without-exec design).
|
||||
|
||||
**Likely correct answer: (3) — targeted FD cleanup
|
||||
via `actor.ipc_server` handle.** (1) is too blunt,
|
||||
(2) is too wide-ranging, (4) changes the spawn
|
||||
mechanism.
|
||||
|
||||
## Reproducer (standalone, no pytest)
|
||||
|
||||
```python
|
||||
# save as /tmp/forkserver_nested_hang_repro.py (py3.14+)
|
||||
import trio, tractor
|
||||
|
||||
async def assert_err():
|
||||
assert 0
|
||||
|
||||
async def spawn_and_error(breadth: int = 2, depth: int = 1):
|
||||
async with tractor.open_nursery() as n:
|
||||
for i in range(breadth):
|
||||
if depth > 0:
|
||||
await n.run_in_actor(
|
||||
spawn_and_error,
|
||||
breadth=breadth,
|
||||
depth=depth - 1,
|
||||
name=f'spawner_{i}_{depth}',
|
||||
)
|
||||
else:
|
||||
await n.run_in_actor(
|
||||
assert_err,
|
||||
name=f'errorer_{i}',
|
||||
)
|
||||
|
||||
async def _main():
|
||||
async with tractor.open_nursery() as n:
|
||||
for i in range(2):
|
||||
await n.run_in_actor(
|
||||
spawn_and_error,
|
||||
name=f'top_{i}',
|
||||
breadth=2,
|
||||
depth=1,
|
||||
)
|
||||
|
||||
if __name__ == '__main__':
|
||||
from tractor.spawn._spawn import try_set_start_method
|
||||
try_set_start_method('subint_forkserver')
|
||||
with trio.fail_after(20):
|
||||
trio.run(_main)
|
||||
```
|
||||
|
||||
Expected (current): hangs on `trio.fail_after(20)`
|
||||
— children never ack the error-propagation cancel
|
||||
cascade. Pattern: top 2 direct children, 4
|
||||
grandchildren, 1 errorer deadlocks while trying to
|
||||
unwind through its parent chain.
|
||||
|
||||
After fix: `trio.TooSlowError`-free completion; the
|
||||
root's `open_nursery` receives the
|
||||
`BaseExceptionGroup` containing the `AssertionError`
|
||||
from the errorer and unwinds cleanly.
|
||||
|
||||
## Update — 2026-04-23: partial fix landed, deeper layer surfaced
|
||||
|
||||
Three improvements landed as separate commits in the
|
||||
`subint_forkserver_backend` branch (see `git log`):
|
||||
|
||||
1. **`_close_inherited_fds()` in fork-child prelude**
|
||||
(`tractor/spawn/_subint_forkserver.py`). POSIX
|
||||
close-fds-equivalent enumeration via
|
||||
`/proc/self/fd` (or `RLIMIT_NOFILE` fallback), keep
|
||||
only stdio. This is fix-direction (1) from the list
|
||||
above — went with the blunt form rather than the
|
||||
targeted enum-via-`actor.ipc_server` form, turns
|
||||
out the aggressive close is safe because every
|
||||
inheritable resource the fresh child needs
|
||||
(IPC-channel socket, etc.) is opened AFTER the
|
||||
fork anyway.
|
||||
2. **`_ForkedProc.wait()` via `os.pidfd_open()` +
|
||||
`trio.lowlevel.wait_readable()`** — matches the
|
||||
`trio.Process.wait` / `mp.Process.sentinel` pattern
|
||||
used by `trio_proc` and `proc_waiter`. Gives us
|
||||
fully trio-cancellable child-wait (prior impl
|
||||
blocked a cache thread on a sync `os.waitpid` that
|
||||
was NOT trio-cancellable due to
|
||||
`abandon_on_cancel=False`).
|
||||
3. **`_parent_chan_cs` wiring** in
|
||||
`tractor/runtime/_runtime.py`: capture the shielded
|
||||
`loop_cs` for the parent-channel `process_messages`
|
||||
task in `async_main`; explicitly cancel it in
|
||||
`Actor.cancel()` teardown. This breaks the shield
|
||||
during teardown so the parent-chan loop exits when
|
||||
cancel is issued, instead of parking on a parent-
|
||||
socket EOF that might never arrive under fork
|
||||
semantics.
|
||||
|
||||
**Concrete wins from (1):** the sibling
|
||||
`subint_forkserver_orphan_sigint_hang_issue.md` class
|
||||
is **now fixed** — `test_orphaned_subactor_sigint_cleanup_DRAFT`
|
||||
went from strict-xfail to pass. The xfail mark was
|
||||
removed; the test remains as a regression guard.
|
||||
|
||||
**test_nested_multierrors STILL hangs** though.
|
||||
|
||||
### Updated diagnosis (narrowed)
|
||||
|
||||
DIAGDEBUG instrumentation of `process_messages` ENTER/
|
||||
EXIT pairs + `_parent_chan_cs.cancel()` call sites
|
||||
showed (captured during a 20s-timeout repro):
|
||||
|
||||
- 80 `process_messages` ENTERs, 75 EXITs → 5 stuck.
|
||||
- **All 40 `shield=True` ENTERs matched EXIT** — every
|
||||
shielded parent-chan loop exits cleanly. The
|
||||
`_parent_chan_cs` wiring works as intended.
|
||||
- **The 5 stuck loops are all `shield=False`** — peer-
|
||||
channel handlers (inbound connections handled by
|
||||
`handle_stream_from_peer` in stream_handler_tn).
|
||||
- After our `_parent_chan_cs.cancel()` fires, NEW
|
||||
shielded process_messages loops start (on the
|
||||
session reg_addr port — probably discovery-layer
|
||||
reconnection attempts). These don't block teardown
|
||||
(they all exit) but indicate the cancel cascade has
|
||||
more moving parts than expected.
|
||||
|
||||
### Remaining unknown
|
||||
|
||||
Why don't the 5 peer-channel loops exit when
|
||||
`service_tn.cancel_scope.cancel()` fires? They're in
|
||||
`stream_handler_tn` which IS `service_tn` in the
|
||||
current configuration (`open_ipc_server(parent_tn=
|
||||
service_tn, stream_handler_tn=service_tn)`). A
|
||||
standard nursery-scope-cancel should propagate through
|
||||
them — no shield, no special handler. Something
|
||||
specific to the fork-spawned configuration keeps them
|
||||
alive.
|
||||
|
||||
Candidate follow-up experiments:
|
||||
|
||||
- Dump the trio task tree at the hang point (via
|
||||
`stackscope` or direct trio introspection) to see
|
||||
what each stuck loop is awaiting. `chan.__anext__`
|
||||
on a socket recv? An inner lock? A shielded sub-task?
|
||||
- Compare peer-channel handler lifecycle under
|
||||
`trio_proc` vs `subint_forkserver` with equivalent
|
||||
logging to spot the divergence.
|
||||
- Investigate whether the peer handler is caught in
|
||||
the `except trio.Cancelled:` path at
|
||||
`tractor/ipc/_server.py:448` that re-raises — but
|
||||
re-raise means it should still exit. Unless
|
||||
something higher up swallows it.
|
||||
|
||||
### Attempted fix (DID NOT work) — hypothesis (3)
|
||||
|
||||
Tried: in `_serve_ipc_eps` finally, after closing
|
||||
listeners, also iterate `server._peers` and
|
||||
sync-close each peer channel's underlying stream
|
||||
socket fd:
|
||||
|
||||
```python
|
||||
for _uid, _chans in list(server._peers.items()):
|
||||
for _chan in _chans:
|
||||
try:
|
||||
_stream = _chan._transport.stream if _chan._transport else None
|
||||
if _stream is not None:
|
||||
_stream.socket.close() # sync fd close
|
||||
except (AttributeError, OSError):
|
||||
pass
|
||||
```
|
||||
|
||||
Theory: closing the socket fd from outside the stuck
|
||||
recv task would make the recv see EBADF /
|
||||
ClosedResourceError and unblock.
|
||||
|
||||
Result: `test_nested_multierrors[subint_forkserver]`
|
||||
still hangs identically. Either:
|
||||
- The sync `socket.close()` doesn't propagate into
|
||||
trio's in-flight `recv_some()` the way I expected
|
||||
(trio may hold an internal reference that keeps the
|
||||
fd open even after an external close), or
|
||||
- The stuck recv isn't even the root blocker and the
|
||||
peer handlers never reach the finally for some
|
||||
reason I haven't understood yet.
|
||||
|
||||
Either way, the sync-close hypothesis is **ruled
|
||||
out**. Reverted the experiment, restored the skip-
|
||||
mark on the test.
|
||||
|
||||
### Aside: `-s` flag does NOT change `test_nested_multierrors` behavior
|
||||
|
||||
Tested explicitly: both with and without `-s`, the
|
||||
test hangs identically. So the capture-pipe-fill
|
||||
hypothesis is **ruled out** for this test.
|
||||
|
||||
The earlier `test_context_stream_semantics.py` `-s`
|
||||
observation was most likely caused by a competing
|
||||
pytest run in my session (confirmed via process list
|
||||
— my leftover pytest was alive at that time and
|
||||
could have been holding state on the default
|
||||
registry port).
|
||||
|
||||
## Update — 2026-04-23 (late): cancel delivery ruled in, nursery-wait ruled BLOCKER
|
||||
|
||||
**New diagnostic run** instrumented
|
||||
`handle_stream_from_peer` at ENTER / `except
|
||||
trio.Cancelled:` / finally, plus `Actor.cancel()`
|
||||
just before `self._parent_chan_cs.cancel()`. Result:
|
||||
|
||||
- **40 `handle_stream_from_peer` ENTERs**.
|
||||
- **0 `except trio.Cancelled:` hits** — cancel
|
||||
never fires on any peer-handler.
|
||||
- **35 finally hits** — those handlers exit via
|
||||
peer-initiated EOF (normal return), NOT cancel.
|
||||
- **5 handlers never reach finally** — stuck forever.
|
||||
- **`Actor.cancel()` fired in 12 PIDs** — but the
|
||||
PIDs with peer handlers that DIDN'T fire
|
||||
Actor.cancel are exactly **root + 2 direct
|
||||
spawners**. These 3 actors have peer handlers
|
||||
(for their own subactors) that stay stuck because
|
||||
**`Actor.cancel()` at these levels never runs**.
|
||||
|
||||
### The actual deadlock shape
|
||||
|
||||
`Actor.cancel()` lives in
|
||||
`open_root_actor.__aexit__` / `async_main` teardown.
|
||||
That only runs when the enclosing `async with
|
||||
tractor.open_nursery()` exits. The nursery's
|
||||
`__aexit__` calls the backend `*_proc` spawn target's
|
||||
teardown, which does `soft_kill() →
|
||||
_ForkedProc.wait()` on its child PID. That wait is
|
||||
trio-cancellable via pidfd now (good) — but nothing
|
||||
CANCELS it because the outer scope only cancels when
|
||||
`Actor.cancel()` runs, which only runs when the
|
||||
nursery completes, which waits on the child.
|
||||
|
||||
It's a **multi-level mutual wait**:
|
||||
|
||||
```
|
||||
root blocks on spawner.wait()
|
||||
spawner blocks on grandchild.wait()
|
||||
grandchild blocks on errorer.wait()
|
||||
errorer Actor.cancel() ran, but process
|
||||
may not have fully exited yet
|
||||
(something in root_tn holding on?)
|
||||
```
|
||||
|
||||
Each level waits for the level below. The bottom
|
||||
level (errorer) reaches Actor.cancel(), but its
|
||||
process may not fully exit — meaning its pidfd
|
||||
doesn't go readable, meaning the grandchild's
|
||||
waitpid doesn't return, meaning the grandchild's
|
||||
nursery doesn't unwind, etc. all the way up.
|
||||
|
||||
### Refined question
|
||||
|
||||
**Why does an errorer process not exit after its
|
||||
`Actor.cancel()` completes?**
|
||||
|
||||
Possibilities:
|
||||
1. `_parent_chan_cs.cancel()` fires (shielded
|
||||
parent-chan loop unshielded), but the task is
|
||||
stuck INSIDE the shielded loop's recv in a way
|
||||
that cancel still can't break.
|
||||
2. After `Actor.cancel()` returns, `async_main`
|
||||
still has other tasks in `root_tn` waiting for
|
||||
something that never arrives (e.g. outbound
|
||||
IPC reply delivery).
|
||||
3. The `os._exit(rc)` in `_worker` (at
|
||||
`_subint_forkserver.py`) doesn't run because
|
||||
`_child_target` never returns.
|
||||
|
||||
Next-session candidate probes (in priority order):
|
||||
|
||||
1. **Instrument `_worker`'s fork-child branch** to
|
||||
confirm whether `child_target()` returns (and
|
||||
thus `os._exit(rc)` is reached) for errorer
|
||||
PIDs. If yes → process should die; if no →
|
||||
trace back into `_actor_child_main` /
|
||||
`_trio_main` / `async_main` to find the stuck
|
||||
spot.
|
||||
2. **Instrument `async_main`'s final unwind** to
|
||||
see which await in the teardown doesn't
|
||||
complete.
|
||||
3. **Compare under `trio_proc` backend** at the
|
||||
same `_worker`-equivalent level to see where
|
||||
the flows diverge.
|
||||
|
||||
### Rule-out: NOT a stuck peer-chan recv
|
||||
|
||||
Earlier hypothesis was that the 5 stuck peer-chan
|
||||
loops were blocked on a socket recv that cancel
|
||||
couldn't interrupt. This pass revealed the real
|
||||
cause: cancel **never reaches those tasks** because
|
||||
their owning actor's `Actor.cancel()` never runs.
|
||||
The recvs are fine — they're just parked because
|
||||
nothing is telling them to stop.
|
||||
|
||||
## Update — 2026-04-23 (very late): leaves exit, middle actors stuck in `trio.run`
|
||||
|
||||
Yet another instrumentation pass — this time
|
||||
printing at:
|
||||
|
||||
- `_worker` child branch: `pre child_target()` /
|
||||
`child_target RETURNED rc=N` / `about to
|
||||
os._exit(rc)`
|
||||
- `_trio_main`: `about to trio.run` /
|
||||
`trio.run RETURNED NORMALLY` / `FINALLY`
|
||||
|
||||
**Fresh-run results** (`test_nested_multierrors[
|
||||
subint_forkserver]`, depth=1/breadth=2, 1 root + 14
|
||||
forked = 15 actors total):
|
||||
|
||||
- **9 processes completed the full flow** —
|
||||
`trio.run RETURNED NORMALLY` → `child_target
|
||||
RETURNED rc=0` → `about to os._exit(0)`. These
|
||||
are the LEAVES of the tree (errorer actors) plus
|
||||
their direct parents (depth-0 spawners). They
|
||||
actually exit their processes.
|
||||
- **5 processes are stuck INSIDE `trio.run(trio_main)`**
|
||||
— they hit "about to trio.run" but NEVER see
|
||||
"trio.run RETURNED NORMALLY". These are root +
|
||||
top-level spawners + one intermediate.
|
||||
|
||||
**What this means:** `async_main` itself is the
|
||||
deadlock holder, not the peer-channel loops.
|
||||
Specifically, the outer `async with root_tn:` in
|
||||
`async_main` never exits for the 5 stuck actors.
|
||||
Their `trio.run` never returns → `_trio_main`
|
||||
catch/finally never runs → `_worker` never reaches
|
||||
`os._exit(rc)` → the PROCESS never dies → its
|
||||
parent's `_ForkedProc.wait()` blocks → parent's
|
||||
nursery hangs → parent's `async_main` hangs → ...
|
||||
|
||||
### The new precise question
|
||||
|
||||
**What task in the 5 stuck actors' `async_main`
|
||||
never completes?** Candidates:
|
||||
|
||||
1. The shielded parent-chan `process_messages`
|
||||
task in `root_tn` — but we explicitly cancel it
|
||||
via `_parent_chan_cs.cancel()` in `Actor.cancel()`.
|
||||
However, `Actor.cancel()` only runs during
|
||||
`open_root_actor.__aexit__`, which itself runs
|
||||
only after `async_main`'s outer unwind — which
|
||||
doesn't happen. So the shield isn't broken.
|
||||
|
||||
2. `await actor_nursery._join_procs.wait()` or
|
||||
similar in the inline backend `*_proc` flow.
|
||||
|
||||
3. `_ForkedProc.wait()` on a grandchild that
|
||||
actually DID exit — but the pidfd_open watch
|
||||
didn't fire for some reason (race between
|
||||
pidfd_open and the child exiting?).
|
||||
|
||||
The most specific next probe: **add DIAG around
|
||||
`_ForkedProc.wait()` enter/exit** to see whether
|
||||
the pidfd-based wait returns for every grandchild
|
||||
exit. If a stuck parent's `_ForkedProc.wait()`
|
||||
NEVER returns despite its child exiting, the
|
||||
pidfd mechanism has a race bug under nested
|
||||
forkserver.
|
||||
|
||||
Alternative probe: instrument `async_main`'s outer
|
||||
nursery exits to find which nursery's `__aexit__`
|
||||
is stuck, drilling down from `trio.run` to the
|
||||
specific `async with` that never completes.
|
||||
|
||||
### Cascade summary (updated tree view)
|
||||
|
||||
```
|
||||
ROOT (pytest) STUCK in trio.run
|
||||
├── top_0 (spawner, d=1) STUCK in trio.run
|
||||
│ ├── spawner_0_d1_0 (d=0) exited (os._exit 0)
|
||||
│ │ ├── errorer_0_0 exited (os._exit 0)
|
||||
│ │ └── errorer_0_1 exited (os._exit 0)
|
||||
│ └── spawner_0_d1_1 (d=0) exited (os._exit 0)
|
||||
│ ├── errorer_0_2 exited (os._exit 0)
|
||||
│ └── errorer_0_3 exited (os._exit 0)
|
||||
└── top_1 (spawner, d=1) STUCK in trio.run
|
||||
├── spawner_1_d1_0 (d=0) STUCK in trio.run (sibling race?)
|
||||
│ ├── errorer_1_0 exited
|
||||
│ └── errorer_1_1 exited
|
||||
└── spawner_1_d1_1 (d=0) STUCK in trio.run
|
||||
├── errorer_1_2 exited
|
||||
└── errorer_1_3 exited
|
||||
```
|
||||
|
||||
Grandchildren (d=0 spawners) exit OR stick —
|
||||
asymmetric. Not purely depth-determined. Some race
|
||||
condition in nursery teardown when multiple
|
||||
siblings error simultaneously.
|
||||
|
||||
## Update — 2026-04-23 (late, probe iteration 3): hang pinpointed to `wait_for_no_more_peers()`
|
||||
|
||||
Further DIAGDEBUG at every milestone in `async_main`
|
||||
(runtime UP / EXITED service_tn / EXITED root_tn /
|
||||
FINALLY ENTER / RETURNING) plus `_ForkedProc.wait`
|
||||
ENTER/RETURNED per-pidfd. Result:
|
||||
|
||||
**Every stuck actor reaches `async_main: FINALLY
|
||||
ENTER` but NOT `async_main: RETURNING`.**
|
||||
|
||||
That isolates the hang to a specific await in
|
||||
`async_main`'s finally block at
|
||||
`tractor/runtime/_runtime.py:1837+`. The suspect:
|
||||
|
||||
```python
|
||||
# Ensure all peers (actors connected to us as clients) are finished
|
||||
if ipc_server := actor.ipc_server and ipc_server.has_peers(check_chans=True):
|
||||
...
|
||||
await ipc_server.wait_for_no_more_peers() # ← UNBOUNDED, blocks forever
|
||||
```
|
||||
|
||||
`_no_more_peers` is an `Event` set only when
|
||||
`server._peers` empties (see
|
||||
`ipc/_server.py:526-530`). If ANY peer-handler is
|
||||
stuck (the 5 unclosed loops from the earlier pass),
|
||||
it keeps its channel in `server._peers`, so the
|
||||
event never fires, so the wait hangs.
|
||||
|
||||
### Applied fix (partial, landed as defensive-in-depth)
|
||||
|
||||
`tractor/runtime/_runtime.py:1981` —
|
||||
`wait_for_no_more_peers()` call now wrapped in
|
||||
`trio.move_on_after(3.0)` + a warning log when the
|
||||
timeout fires. Commented with the full rationale.
|
||||
|
||||
**Verified:** with this fix, ALL 15 actors reach
|
||||
`async_main: RETURNING` cleanly (up from 10/15
|
||||
reaching end before).
|
||||
|
||||
**Unfortunately:** the test still hangs past 45s
|
||||
total — meaning there's YET ANOTHER unbounded wait
|
||||
downstream of `async_main`. The bounded
|
||||
`wait_for_no_more_peers` unblocks one level, but
|
||||
the cascade has another level above it.
|
||||
|
||||
### Candidates for the remaining hang
|
||||
|
||||
1. `open_root_actor`'s own finally / post-
|
||||
`async_main` flow in `_root.py` — specifically
|
||||
`await actor.cancel(None)` which has its own
|
||||
internal waits.
|
||||
2. The `trio.run()` itself doesn't return even
|
||||
after the root task completes because trio's
|
||||
nursery still has background tasks running.
|
||||
3. Maybe `_serve_ipc_eps`'s finally has an await
|
||||
that blocks when peers aren't clearing.
|
||||
|
||||
### Current stance
|
||||
|
||||
- Defensive `wait_for_no_more_peers` bound landed
|
||||
(good hygiene regardless). Revealing a real
|
||||
deadlock-avoidance gap in tractor's cleanup.
|
||||
- Test still hangs → skip-mark restored on
|
||||
`test_nested_multierrors[subint_forkserver]`.
|
||||
- The full chain of unbounded waits needs another
|
||||
session of drilling, probably at
|
||||
`open_root_actor` / `actor.cancel` level.
|
||||
|
||||
### Summary of this investigation's wins
|
||||
|
||||
1. **FD hygiene fix** (`_close_inherited_fds`) —
|
||||
correct, closed orphan-SIGINT sibling issue.
|
||||
2. **pidfd-based `_ForkedProc.wait`** — cancellable,
|
||||
matches trio_proc pattern.
|
||||
3. **`_parent_chan_cs` wiring** —
|
||||
`Actor.cancel()` now breaks the shielded parent-
|
||||
chan `process_messages` loop.
|
||||
4. **`wait_for_no_more_peers` bounded** —
|
||||
prevents the actor-level finally hang.
|
||||
5. **Ruled-out hypotheses:** tree-kill missing
|
||||
(wrong), stuck socket recv (wrong).
|
||||
6. **Pinpointed remaining unknown:** at least one
|
||||
more unbounded wait in the teardown cascade
|
||||
above `async_main`. Concrete candidates
|
||||
enumerated above.
|
||||
|
||||
## Update — 2026-04-23 (VERY late): pytest capture pipe IS the final gate
|
||||
|
||||
After landing fixes 1-4 and instrumenting every
|
||||
layer down to `tractor_test`'s `trio.run(_main)`:
|
||||
|
||||
**Empirical result: with `pytest -s` the test PASSES
|
||||
in 6.20s.** Without `-s` (default `--capture=fd`) it
|
||||
hangs forever.
|
||||
|
||||
DIAG timeline for the root pytest PID (with `-s`
|
||||
implied from later verification):
|
||||
|
||||
```
|
||||
tractor_test: about to trio.run(_main)
|
||||
open_root_actor: async_main task started, yielding to test body
|
||||
_main: about to await wrapped test fn
|
||||
_main: wrapped RETURNED cleanly ← test body completed!
|
||||
open_root_actor: about to actor.cancel(None)
|
||||
Actor.cancel ENTER req_chan=False
|
||||
Actor.cancel RETURN
|
||||
open_root_actor: actor.cancel RETURNED
|
||||
open_root_actor: outer FINALLY
|
||||
open_root_actor: finally END (returning from ctxmgr)
|
||||
tractor_test: trio.run FINALLY (returned or raised) ← trio.run fully returned!
|
||||
```
|
||||
|
||||
`trio.run()` fully returns. The test body itself
|
||||
completes successfully (pytest.raises absorbed the
|
||||
expected `BaseExceptionGroup`). What blocks is
|
||||
**pytest's own stdout/stderr capture** — under
|
||||
`--capture=fd` default, pytest replaces the parent
|
||||
process's fd 1,2 with pipe write-ends it's reading
|
||||
from. Fork children inherit those pipe fds
|
||||
(because `_close_inherited_fds` correctly preserves
|
||||
stdio). High-volume subactor error-log tracebacks
|
||||
(7+ actors each logging multiple
|
||||
`RemoteActorError`/`ExceptionGroup` tracebacks on
|
||||
the error-propagation cascade) fill the 64KB Linux
|
||||
pipe buffer. Subactor writes block. Subactor can't
|
||||
progress. Process doesn't exit. Parent's
|
||||
`_ForkedProc.wait` (now pidfd-based and
|
||||
cancellable, but nothing's cancelling here since
|
||||
the test body already completed) keeps the pipe
|
||||
reader alive... but pytest isn't draining its end
|
||||
fast enough because test-teardown/fixture-cleanup
|
||||
is in progress.
|
||||
|
||||
**Actually** the exact mechanism is slightly
|
||||
different: pytest's capture fixture MIGHT be
|
||||
actively reading, but faster-than-writer subactors
|
||||
overflow its internal buffer. Or pytest might be
|
||||
blocked itself on the finalization step.
|
||||
|
||||
Either way, `-s` conclusively fixes it.
|
||||
|
||||
### Why I ruled this out earlier (and shouldn't have)
|
||||
|
||||
Earlier in this investigation I tested
|
||||
`test_nested_multierrors` with/without `-s` and
|
||||
both hung. That's because AT THAT TIME, fixes 1-4
|
||||
weren't all in place yet. The test was hanging at
|
||||
multiple deeper levels long before reaching the
|
||||
"generate lots of error-log output" phase. Once
|
||||
the cascade actually tore down cleanly, enough
|
||||
output was produced to hit the capture-pipe limit.
|
||||
|
||||
**Classic order-of-operations mistake in
|
||||
debugging:** ruling something out too early based
|
||||
on a test that was actually failing for a
|
||||
different reason.
|
||||
|
||||
### Fix direction (next session)
|
||||
|
||||
Redirect subactor stdout/stderr to `/dev/null` (or
|
||||
a session-scoped log file) in the fork-child
|
||||
prelude, right after `_close_inherited_fds()`. This
|
||||
severs the inherited pytest-capture pipes and lets
|
||||
subactor output flow elsewhere. Under normal
|
||||
production use (non-pytest), stdout/stderr would
|
||||
be the TTY — we'd want to keep that. So the
|
||||
redirect should be conditional or opt-in via the
|
||||
`child_sigint`/proc_kwargs flag family.
|
||||
|
||||
Alternative: document as a gotcha and recommend
|
||||
`pytest -s` for any tests using the
|
||||
`subint_forkserver` backend with multi-level actor
|
||||
trees. Simpler, user-visible, no code change.
|
||||
|
||||
### Current state
|
||||
|
||||
- Skip-mark on `test_nested_multierrors[subint_forkserver]`
|
||||
restored with reason pointing here.
|
||||
- Test confirmed passing with `-s` after all 4
|
||||
cascade fixes applied.
|
||||
- The 4 cascade fixes are NOT wasted — they're
|
||||
correct hardening regardless of the capture-pipe
|
||||
issue, AND without them we'd never reach the
|
||||
"actually produces enough output to fill the
|
||||
pipe" state.
|
||||
|
||||
## Stopgap (landed)
|
||||
|
||||
`test_nested_multierrors` skip-marked under
|
||||
`subint_forkserver` via
|
||||
`@pytest.mark.skipon_spawn_backend('subint_forkserver',
|
||||
reason='...')`, cross-referenced to this doc. Mark
|
||||
should be dropped once the peer-channel-loop exit
|
||||
issue is fixed.
|
||||
|
||||
## References
|
||||
|
||||
- `tractor/spawn/_subint_forkserver.py::fork_from_worker_thread`
|
||||
— the primitive whose post-fork FD hygiene is
|
||||
probably the culprit.
|
||||
- `tractor/spawn/_subint_forkserver.py::subint_forkserver_proc`
|
||||
— the backend function that orchestrates the
|
||||
graceful cancel path hitting this bug.
|
||||
- `tractor/spawn/_subint_forkserver.py::_ForkedProc`
|
||||
— the `trio.Process`-compatible shim; NOT the
|
||||
failing component (confirmed via thread-dump).
|
||||
- `tests/test_cancellation.py::test_nested_multierrors`
|
||||
— the test that surfaced the hang.
|
||||
- `ai/conc-anal/subint_forkserver_orphan_sigint_hang_issue.md`
|
||||
— sibling hang class; probably same underlying
|
||||
fork-FD-inheritance root cause.
|
||||
- tractor issue #379 — subint backend tracking.
|
||||
|
|
@ -1,186 +0,0 @@
|
|||
# Revisit `subint_forkserver` thread-cache constraints once msgspec PEP 684 support lands
|
||||
|
||||
> **Tracked at:** [#450](https://github.com/goodboy/tractor/issues/450)
|
||||
|
||||
Follow-up tracker for cleanup work gated on the msgspec
|
||||
PEP 684 adoption upstream ([jcrist/msgspec#563](https://github.com/jcrist/msgspec/issues/563)).
|
||||
|
||||
Context — why this exists
|
||||
-------------------------
|
||||
|
||||
The `tractor.spawn._subint_forkserver` submodule currently
|
||||
carries two "non-trio" thread-hygiene constraints whose
|
||||
necessity is tangled with issues that *should* dissolve
|
||||
under PEP 684 isolated-mode subinterpreters:
|
||||
|
||||
1. `fork_from_worker_thread()` / `run_subint_in_worker_thread()`
|
||||
internally allocate a **dedicated `threading.Thread`**
|
||||
rather than using `trio.to_thread.run_sync()`.
|
||||
2. The test helper is named
|
||||
`run_fork_in_non_trio_thread()` — the
|
||||
`non_trio` qualifier is load-bearing today.
|
||||
|
||||
This doc catalogs *why* those constraints exist, which of
|
||||
them isolated-mode would fix, and what the
|
||||
audit-and-cleanup path looks like once msgspec #563 is
|
||||
resolved.
|
||||
|
||||
The three reasons the constraints exist
|
||||
---------------------------------------
|
||||
|
||||
### 1. GIL-starvation class → fixed by PEP 684 isolated mode
|
||||
|
||||
The class-A hang documented in
|
||||
`subint_sigint_starvation_issue.md` is entirely about
|
||||
legacy-config subints **sharing the main GIL**. Once
|
||||
msgspec #563 lands and tractor flips
|
||||
`tractor.spawn._subint` to
|
||||
`concurrent.interpreters.create()` (isolated config), each
|
||||
subint gets its own GIL. Abandoned subint threads can't
|
||||
contend for main's GIL → can't starve the main trio loop
|
||||
→ signal-wakeup-pipe drains normally → no SIGINT-drop.
|
||||
|
||||
This class of hazard **dissolves entirely**. The
|
||||
non-trio-thread requirement for *this reason* disappears.
|
||||
|
||||
### 2. Destroy race / tstate-recycling → orthogonal; unclear
|
||||
|
||||
The `subint_proc` dedicated-thread fix (commit `26fb8206`)
|
||||
addressed a different issue: `_interpreters.destroy(interp_id)`
|
||||
was blocking on a trio-cache worker that had run an
|
||||
earlier `interp.exec()` for that subint. Working
|
||||
hypothesis at the time was "the cached thread retains the
|
||||
subint's tstate".
|
||||
|
||||
But tstate-handling is **not specific to GIL mode** —
|
||||
`_PyXI_Enter` / `_PyXI_Exit` (the C-level machinery both
|
||||
configs use to enter/leave a subint from a thread) should
|
||||
restore the caller's tstate regardless of GIL config. So
|
||||
isolated mode **doesn't obviously fix this**. It might be:
|
||||
|
||||
- A py3.13 bug fixed in later versions — we saw the race
|
||||
first on 3.13 and never re-tested on 3.14 after moving
|
||||
to dedicated threads.
|
||||
- A genuine CPython quirk around cached threads that
|
||||
exec'd into a subint, persisting across GIL modes.
|
||||
- Something else we misdiagnosed — the empirical fix
|
||||
(dedicated thread) worked but the analysis may have
|
||||
been incomplete.
|
||||
|
||||
Only way to know: once we're on isolated mode, empirically
|
||||
retry `trio.to_thread.run_sync(interp.exec, ...)` and see
|
||||
if `destroy()` still blocks. If it does, keep the
|
||||
dedicated thread; if not, one constraint relaxed.
|
||||
|
||||
### 3. Fork-from-main-interp-tstate (the constraint in this module's helper names)
|
||||
|
||||
The fork-from-main-interp-tstate invariant — CPython's
|
||||
`PyOS_AfterFork_Child` →
|
||||
`_PyInterpreterState_DeleteExceptMain` gate documented in
|
||||
`subint_fork_blocked_by_cpython_post_fork_issue.md` — is
|
||||
about the calling thread's **current** tstate at the
|
||||
moment `os.fork()` runs. If trio's cache threads never
|
||||
enter subints at all, their tstate is plain main-interp,
|
||||
and fork from them would be fine.
|
||||
|
||||
The reason the smoke test +
|
||||
`run_fork_in_non_trio_thread` test helper
|
||||
currently use a dedicated `threading.Thread` is narrow:
|
||||
**we don't want to risk a trio cache thread that has
|
||||
previously been used as a subint driver being the one that
|
||||
picks up the fork job**. If cached tstate doesn't get
|
||||
cleared (back to reason #2), the fork's child-side
|
||||
post-init would see the wrong interp and abort.
|
||||
|
||||
In an isolated-mode world where msgspec works:
|
||||
|
||||
- `subint_proc` would use the public
|
||||
`concurrent.interpreters.create()` + `Interpreter.exec()`
|
||||
/ `Interpreter.close()` — which *should* handle tstate
|
||||
cleanly (they're the "blessed" API).
|
||||
- If so, trio's cache threads are safe to fork from
|
||||
regardless of whether they've previously driven subints.
|
||||
- → the `non_trio` qualifier in
|
||||
`run_fork_in_non_trio_thread` becomes
|
||||
*overcautious* rather than load-bearing, and the
|
||||
dedicated-thread primitives in `_subint_forkserver.py`
|
||||
can likely be replaced with straight
|
||||
`trio.to_thread.run_sync()` wrappers.
|
||||
|
||||
TL;DR
|
||||
-----
|
||||
|
||||
| constraint | fixed by isolated mode? |
|
||||
|---|---|
|
||||
| GIL-starvation (class A) | **yes** |
|
||||
| destroy race on cached worker | unclear — empirical test on py3.14 + isolated API required |
|
||||
| fork-from-main-tstate requirement on worker | **probably yes, conditional on the destroy-race question above** |
|
||||
|
||||
If #2 also resolves on py3.14+ with isolated mode,
|
||||
tractor could drop the `non_trio` qualifier from the fork
|
||||
helper's name and just use `trio.to_thread.run_sync(...)`
|
||||
for everything. But **we shouldn't do that preemptively**
|
||||
— the current cautious design is cheap (one dedicated
|
||||
thread per fork / per subint-exec) and correct.
|
||||
|
||||
Audit plan when msgspec #563 lands
|
||||
----------------------------------
|
||||
|
||||
Assuming msgspec grows `Py_mod_multiple_interpreters`
|
||||
support:
|
||||
|
||||
1. **Flip `tractor.spawn._subint` to isolated mode.** Drop
|
||||
the `_interpreters.create('legacy')` call in favor of
|
||||
the public API (`concurrent.interpreters.create()` +
|
||||
`Interpreter.exec()` / `Interpreter.close()`). Run the
|
||||
three `ai/conc-anal/subint_*_issue.md` reproducers —
|
||||
class-A (`test_stale_entry_is_deleted` etc.) should
|
||||
pass without the `skipon_spawn_backend('subint')` marks
|
||||
(revisit the marker inventory).
|
||||
|
||||
2. **Empirical destroy-race retest.** In `subint_proc`,
|
||||
swap the dedicated `threading.Thread` back to
|
||||
`trio.to_thread.run_sync(Interpreter.exec, ...,
|
||||
abandon_on_cancel=False)` and run the full subint test
|
||||
suite. If `Interpreter.close()` (or the backing
|
||||
destroy) blocks the same way as the legacy version
|
||||
did, revert and keep the dedicated thread.
|
||||
|
||||
3. **If #2 clean**, audit `_subint_forkserver.py`:
|
||||
- Rename `run_fork_in_non_trio_thread` → drop the
|
||||
`_non_trio_` qualifier (e.g. `run_fork_in_thread`) or
|
||||
inline the two-line `trio.to_thread.run_sync` call at
|
||||
the call sites and drop the helper entirely.
|
||||
- Consider whether `fork_from_worker_thread` +
|
||||
`run_subint_in_worker_thread` still warrant being
|
||||
separate module-level primitives or whether they
|
||||
collapse into a compound
|
||||
`trio.to_thread.run_sync`-driven pattern inside the
|
||||
(future) `subint_forkserver_proc` backend.
|
||||
|
||||
4. **Doc fallout.** `subint_sigint_starvation_issue.md`
|
||||
and `subint_cancel_delivery_hang_issue.md` both cite
|
||||
the legacy-GIL-sharing architecture as the root cause.
|
||||
Close them with commit-refs to the isolated-mode
|
||||
migration. This doc itself should get a closing
|
||||
post-mortem section noting which of #1/#2/#3 actually
|
||||
resolved vs persisted.
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
- `tractor.spawn._subint_forkserver` — the in-tree module
|
||||
whose constraints this doc catalogs.
|
||||
- `ai/conc-anal/subint_sigint_starvation_issue.md` — the
|
||||
GIL-starvation class.
|
||||
- `ai/conc-anal/subint_cancel_delivery_hang_issue.md` —
|
||||
sibling Ctrl-C-able hang class.
|
||||
- `ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`
|
||||
— why fork-from-subint is blocked (this drives the
|
||||
forkserver-via-non-subint-thread workaround).
|
||||
- `ai/conc-anal/subint_fork_from_main_thread_smoketest.py`
|
||||
— empirical validation for the workaround.
|
||||
- [PEP 684 — per-interpreter GIL](https://peps.python.org/pep-0684/)
|
||||
- [PEP 734 — `concurrent.interpreters` public API](https://peps.python.org/pep-0734/)
|
||||
- [jcrist/msgspec#563 — PEP 684 support tracker](https://github.com/jcrist/msgspec/issues/563)
|
||||
- tractor issue #379 — subint backend tracking.
|
||||
|
|
@ -1,350 +0,0 @@
|
|||
# `subint` backend: abandoned-subint thread can wedge main trio event loop (Ctrl-C unresponsive)
|
||||
|
||||
Follow-up to the Phase B subint spawn-backend PR (see
|
||||
`tractor.spawn._subint`, issue #379). The hard-kill escape
|
||||
hatch we landed (`_HARD_KILL_TIMEOUT`, bounded shields,
|
||||
`daemon=True` driver-thread abandonment) handles *most*
|
||||
stuck-subint scenarios cleanly, but there's one class of
|
||||
hang that can't be fully escaped from within tractor: a
|
||||
still-running abandoned sub-interpreter can starve the
|
||||
**parent's** trio event loop to the point where **SIGINT is
|
||||
effectively dropped by the kernel ↔ Python boundary** —
|
||||
making the pytest process un-Ctrl-C-able.
|
||||
|
||||
## Symptom
|
||||
|
||||
Running `test_stale_entry_is_deleted[subint]` under
|
||||
`--spawn-backend=subint`:
|
||||
|
||||
1. Test spawns a subactor (`transport_fails_actor`) which
|
||||
kills its own IPC server and then
|
||||
`trio.sleep_forever()`.
|
||||
2. Parent tries `Portal.cancel_actor()` → channel
|
||||
disconnected → fast return.
|
||||
3. Nursery teardown triggers our `subint_proc` cancel path.
|
||||
Portal-cancel fails (dead channel),
|
||||
`_HARD_KILL_TIMEOUT` fires, driver thread is abandoned
|
||||
(`daemon=True`), `_interpreters.destroy(interp_id)`
|
||||
raises `InterpreterError` (because the subint is still
|
||||
running).
|
||||
4. Test appears to hang indefinitely at the *outer*
|
||||
`async with tractor.open_nursery() as an:` exit.
|
||||
5. `Ctrl-C` at the terminal does nothing. The pytest
|
||||
process is un-interruptable.
|
||||
|
||||
## Evidence
|
||||
|
||||
### `strace` on the hung pytest process
|
||||
|
||||
```
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(37, "\2", 1) = -1 EAGAIN (Resource temporarily unavailable)
|
||||
rt_sigreturn({mask=[WINCH]}) = 140585542325792
|
||||
```
|
||||
|
||||
Translated:
|
||||
|
||||
- Kernel delivers `SIGINT` to pytest.
|
||||
- CPython's C-level signal handler fires and tries to
|
||||
write the signal number byte (`0x02` = SIGINT) to fd 37
|
||||
— the **Python signal-wakeup fd** (set via
|
||||
`signal.set_wakeup_fd()`, which trio uses to wake its
|
||||
event loop on signals).
|
||||
- Write returns `EAGAIN` — **the pipe is full**. Nothing
|
||||
is draining it.
|
||||
- `rt_sigreturn` with the signal masked off — signal is
|
||||
"handled" from the kernel's perspective but the actual
|
||||
Python-level handler (and therefore trio's
|
||||
`KeyboardInterrupt` delivery) never runs.
|
||||
|
||||
### Stack dump (via `tractor.devx.dump_on_hang`)
|
||||
|
||||
At 20s into the hang, only the **main thread** is visible:
|
||||
|
||||
```
|
||||
Thread 0x...7fdca0191780 [python] (most recent call first):
|
||||
File ".../trio/_core/_io_epoll.py", line 245 in get_events
|
||||
File ".../trio/_core/_run.py", line 2415 in run
|
||||
File ".../tests/discovery/test_registrar.py", line 575 in test_stale_entry_is_deleted
|
||||
...
|
||||
```
|
||||
|
||||
No driver thread shows up. The abandoned-legacy-subint
|
||||
thread still exists from the OS's POV (it's still running
|
||||
inside `_interpreters.exec()` driving the subint's
|
||||
`trio.run()` on `trio.sleep_forever()`) but the **main
|
||||
interp's faulthandler can't see threads currently executing
|
||||
inside a sub-interpreter's tstate**. Concretely: the thread
|
||||
is alive, holding state we can't introspect from here.
|
||||
|
||||
## Root cause analysis
|
||||
|
||||
The most consistent explanation for both observations:
|
||||
|
||||
1. **Legacy-config subinterpreters share the main GIL.**
|
||||
PEP 734's public `concurrent.interpreters.create()`
|
||||
defaults to `'isolated'` (per-interp GIL), but tractor
|
||||
uses `_interpreters.create('legacy')` as a workaround
|
||||
for C extensions that don't yet support PEP 684
|
||||
(notably `msgspec`, see
|
||||
[jcrist/msgspec#563](https://github.com/jcrist/msgspec/issues/563)).
|
||||
Legacy-mode subints share process-global state
|
||||
including the GIL.
|
||||
|
||||
2. **Our abandoned subint thread never exits.** After our
|
||||
hard-kill timeout, `driver_thread.join()` is abandoned
|
||||
via `abandon_on_cancel=True` and the thread is
|
||||
`daemon=True` so proc-exit won't block on it — but the
|
||||
thread *itself* is still alive inside
|
||||
`_interpreters.exec()`, driving a `trio.run()` that
|
||||
will never return (the subint actor is in
|
||||
`trio.sleep_forever()`).
|
||||
|
||||
3. **`_interpreters.destroy()` cannot force-stop a running
|
||||
subint.** It raises `InterpreterError` on any
|
||||
still-running subinterpreter; there is no public
|
||||
CPython API to force-destroy one.
|
||||
|
||||
4. **Shared-GIL + non-terminating subint thread → main
|
||||
trio loop starvation.** Under enough load (the subint's
|
||||
trio event loop iterating in the background, IPC-layer
|
||||
tasks still in the subint, etc.) the main trio event
|
||||
loop can fail to iterate frequently enough to drain its
|
||||
wakeup pipe. Once that pipe fills, `SIGINT` writes from
|
||||
the C signal handler return `EAGAIN` and signals are
|
||||
silently dropped — exactly what `strace` shows.
|
||||
|
||||
The shielded
|
||||
`await actor_nursery._join_procs.wait()` at the top of
|
||||
`subint_proc` (inherited unchanged from the `trio_proc`
|
||||
pattern) is structurally involved too: if main trio *does*
|
||||
get a schedule slice, it'd find the `subint_proc` task
|
||||
parked on `_join_procs` under shield — which traps whatever
|
||||
`Cancelled` arrives. But that's a second-order effect; the
|
||||
signal-pipe-full condition is the primary "Ctrl-C doesn't
|
||||
work" cause.
|
||||
|
||||
## Why we can't fix this from inside tractor
|
||||
|
||||
- **No force-destroy API.** CPython provides neither a
|
||||
`_interpreters.force_destroy()` nor a thread-
|
||||
cancellation primitive (`pthread_cancel` is actively
|
||||
discouraged and unavailable on Windows). A subint stuck
|
||||
in pure-Python loops (or worse, C code that doesn't poll
|
||||
for signals) is structurally unreachable from outside.
|
||||
- **Shared GIL is the root scheduling issue.** As long as
|
||||
we're forced into legacy-mode subints for `msgspec`
|
||||
compatibility, the abandoned-thread scenario is
|
||||
fundamentally a process-global GIL-starvation window.
|
||||
- **`signal.set_wakeup_fd()` is process-global.** Even if
|
||||
we wanted to put our own drainer on the wakeup pipe,
|
||||
only one party owns it at a time.
|
||||
|
||||
## Current workaround
|
||||
|
||||
- **Fixture-side SIGINT loop on the `daemon` subproc** (in
|
||||
this test's `daemon: subprocess.Popen` fixture in
|
||||
`tests/conftest.py`). The daemon dying closes its end of
|
||||
the registry IPC, which unblocks a pending recv in main
|
||||
trio's IPC-server task, which lets the event loop
|
||||
iterate, which drains the wakeup pipe, which finally
|
||||
delivers the test-harness SIGINT.
|
||||
- **Module-level skip on py3.13**
|
||||
(`pytest.importorskip('concurrent.interpreters')`) — the
|
||||
private `_interpreters` C module exists on 3.13 but the
|
||||
multi-trio-task interaction hangs silently there
|
||||
independently of this issue.
|
||||
|
||||
## Path forward
|
||||
|
||||
1. **Primary**: upstream `msgspec` PEP 684 adoption
|
||||
([jcrist/msgspec#563](https://github.com/jcrist/msgspec/issues/563)).
|
||||
Unlocks `concurrent.interpreters.create()` isolated
|
||||
mode → per-interp GIL → abandoned subint threads no
|
||||
longer starve the parent's main trio loop. At that
|
||||
point we can flip `_subint.py` back to the public API
|
||||
(`create()` / `Interpreter.exec()` / `Interpreter.close()`)
|
||||
and drop the private `_interpreters` path.
|
||||
|
||||
2. **Secondary**: watch CPython for a public
|
||||
force-destroy primitive. If something like
|
||||
`Interpreter.close(force=True)` lands, we can use it as
|
||||
a hard-kill final stage and actually tear down
|
||||
abandoned subints.
|
||||
|
||||
3. **Harness-level**: document the fixture-side SIGINT
|
||||
loop pattern as the "known workaround" for subint-
|
||||
backend tests that can leave background state holding
|
||||
the main event loop hostage.
|
||||
|
||||
## References
|
||||
|
||||
- PEP 734 (`concurrent.interpreters`):
|
||||
<https://peps.python.org/pep-0734/>
|
||||
- PEP 684 (per-interpreter GIL):
|
||||
<https://peps.python.org/pep-0684/>
|
||||
- `msgspec` PEP 684 tracker:
|
||||
<https://github.com/jcrist/msgspec/issues/563>
|
||||
- CPython `_interpretersmodule.c` source:
|
||||
<https://github.com/python/cpython/blob/main/Modules/_interpretersmodule.c>
|
||||
- `tractor.spawn._subint` module docstring (in-tree
|
||||
explanation of the legacy-mode choice and its
|
||||
tradeoffs).
|
||||
|
||||
## Reproducer
|
||||
|
||||
```
|
||||
./py314/bin/python -m pytest \
|
||||
tests/discovery/test_registrar.py::test_stale_entry_is_deleted \
|
||||
--spawn-backend=subint \
|
||||
--tb=short --no-header -v
|
||||
```
|
||||
|
||||
Hangs indefinitely without the fixture-side SIGINT loop;
|
||||
with the loop, the test completes (albeit with the
|
||||
abandoned-thread warning in logs).
|
||||
|
||||
## Additional known-hanging tests (same class)
|
||||
|
||||
All three tests below exhibit the same
|
||||
signal-wakeup-fd-starvation fingerprint (`write() → EAGAIN`
|
||||
on the wakeup pipe after enough SIGINT attempts) and
|
||||
share the same structural cause — abandoned legacy-subint
|
||||
driver threads contending with the main interpreter for
|
||||
the shared GIL until the main trio loop can no longer
|
||||
drain its wakeup pipe fast enough to deliver signals.
|
||||
|
||||
They're listed separately because each exposes the class
|
||||
under a different load pattern worth documenting.
|
||||
|
||||
### `tests/discovery/test_registrar.py::test_stale_entry_is_deleted[subint]`
|
||||
|
||||
Original exemplar — see the **Symptom** and **Evidence**
|
||||
sections above. One abandoned subint
|
||||
(`transport_fails_actor`, stuck in `trio.sleep_forever()`
|
||||
after self-cancelling its IPC server) is sufficient to
|
||||
tip main into starvation once the harness's `daemon`
|
||||
fixture subproc keeps its half of the registry IPC alive.
|
||||
|
||||
### `tests/test_cancellation.py::test_cancel_while_childs_child_in_sync_sleep[subint-False]`
|
||||
|
||||
Cancel a grandchild that's in sync Python sleep from 2
|
||||
nurseries up. The test's own docstring declares the
|
||||
dependency: "its parent should issue a 'zombie reaper' to
|
||||
hard kill it after sufficient timeout" — which for
|
||||
`trio`/`mp_*` is an OS-level `SIGKILL` of the grandchild
|
||||
subproc. **Under `subint` there's no equivalent** (no
|
||||
public CPython API to force-destroy a running
|
||||
sub-interpreter), so the grandchild's sync-sleeping
|
||||
`trio.run()` persists inside its abandoned driver thread
|
||||
indefinitely. The nested actor-tree (parent → child →
|
||||
grandchild, all subints) means a single cancel triggers
|
||||
multiple concurrent hard-kill abandonments, each leaving
|
||||
a live driver thread.
|
||||
|
||||
This test often only manifests the starvation under
|
||||
**full-suite runs** rather than solo execution —
|
||||
earlier-in-session subint tests also leave abandoned
|
||||
driver threads behind, and the combined population is
|
||||
what actually tips main trio into starvation. Solo runs
|
||||
may stay Ctrl-C-able with fewer abandoned threads in the
|
||||
mix.
|
||||
|
||||
### `tests/test_cancellation.py::test_multierror_fast_nursery[subint-25-0.5]`
|
||||
|
||||
Nursery-error-path throughput stress-test parametrized
|
||||
for **25 concurrent subactors**. When the multierror
|
||||
fires and the nursery cancels, every subactor goes
|
||||
through our `subint_proc` teardown. The bounded
|
||||
hard-kills run in parallel (all `subint_proc` tasks are
|
||||
sibling trio tasks), so the timeout budget is ~3s total
|
||||
rather than 3s × 25. After that, **25 abandoned
|
||||
`daemon=True` driver threads are simultaneously alive** —
|
||||
an extreme pressure multiplier on the same mechanism.
|
||||
|
||||
The `strace` fingerprint is striking under this load: six
|
||||
or more **successful** `write(16, "\2", 1) = 1` calls
|
||||
(main trio getting brief GIL slices, each long enough to
|
||||
drain exactly one wakeup-pipe byte) before finally
|
||||
saturating with `EAGAIN`:
|
||||
|
||||
```
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(16, "\2", 1) = 1
|
||||
rt_sigreturn({mask=[WINCH]}) = 140141623162400
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(16, "\2", 1) = 1
|
||||
rt_sigreturn({mask=[WINCH]}) = 140141623162400
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(16, "\2", 1) = 1
|
||||
rt_sigreturn({mask=[WINCH]}) = 140141623162400
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(16, "\2", 1) = 1
|
||||
rt_sigreturn({mask=[WINCH]}) = 140141623162400
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(16, "\2", 1) = 1
|
||||
rt_sigreturn({mask=[WINCH]}) = 140141623162400
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(16, "\2", 1) = 1
|
||||
rt_sigreturn({mask=[WINCH]}) = 140141623162400
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(16, "\2", 1) = -1 EAGAIN (Resource temporarily unavailable)
|
||||
rt_sigreturn({mask=[WINCH]}) = 140141623162400
|
||||
```
|
||||
|
||||
Those successful writes indicate CPython's
|
||||
`sys.getswitchinterval()`-based GIL round-robin *is*
|
||||
giving main brief slices — just never long enough to run
|
||||
the Python-level signal handler through to the point
|
||||
where trio converts the delivered SIGINT into a
|
||||
`Cancelled` on the appropriate scope. Once the
|
||||
accumulated write rate outpaces main's drain rate, the
|
||||
pipe saturates and subsequent signals are silently
|
||||
dropped.
|
||||
|
||||
The `pstree` below (pid `530060` = hung `pytest`) shows
|
||||
the subint-driver thread population at the moment of
|
||||
capture. Even with fewer than the full 25 shown (pstree
|
||||
truncates thread names to `subint-driver[<interp_id>` —
|
||||
interpreters `3` and `4` visible across 16 thread
|
||||
entries), the GIL-contender count is more than enough to
|
||||
explain the starvation:
|
||||
|
||||
```
|
||||
>>> pstree -snapt 530060
|
||||
systemd,1 --switched-root --system --deserialize=40
|
||||
└─login,1545 --
|
||||
└─bash,1872
|
||||
└─sway,2012
|
||||
└─alacritty,70471 -e xonsh
|
||||
└─xonsh,70487 .../bin/xonsh
|
||||
└─uv,70955 run xonsh
|
||||
└─xonsh,70959 .../py314/bin/xonsh
|
||||
└─python,530060 .../py314/bin/pytest -v tests/test_cancellation.py --spawn-backend=subint
|
||||
├─{subint-driver[3},531857
|
||||
├─{subint-driver[3},531860
|
||||
├─{subint-driver[3},531862
|
||||
├─{subint-driver[3},531866
|
||||
├─{subint-driver[3},531877
|
||||
├─{subint-driver[3},531882
|
||||
├─{subint-driver[3},531884
|
||||
├─{subint-driver[3},531945
|
||||
├─{subint-driver[3},531950
|
||||
├─{subint-driver[3},531952
|
||||
├─{subint-driver[4},531956
|
||||
├─{subint-driver[4},531959
|
||||
├─{subint-driver[4},531961
|
||||
├─{subint-driver[4},531965
|
||||
├─{subint-driver[4},531968
|
||||
└─{subint-driver[4},531979
|
||||
```
|
||||
|
||||
(`pstree` uses `{...}` to denote threads rather than
|
||||
processes — these are all the **driver OS-threads** our
|
||||
`subint_proc` creates with name
|
||||
`f'subint-driver[{interp_id}]'`. Every one of them is
|
||||
still alive, executing `_interpreters.exec()` inside a
|
||||
sub-interpreter our hard-kill has abandoned. At 16+
|
||||
abandoned driver threads competing for the main GIL, the
|
||||
main-interpreter trio loop gets starved and signal
|
||||
delivery stalls.)
|
||||
|
|
@ -1,273 +0,0 @@
|
|||
# `test_register_duplicate_name` racy connect-failure on `daemon` fixture readiness
|
||||
|
||||
## Symptom
|
||||
|
||||
`tests/test_multi_program.py::test_register_duplicate_name`
|
||||
fails intermittently under BOTH transports + ALL spawn
|
||||
backends with connect-refused errors:
|
||||
|
||||
```
|
||||
# under --tpt-proto=uds
|
||||
FAILED tests/test_multi_program.py::test_register_duplicate_name
|
||||
- ConnectionRefusedError: [Errno 111] Connection refused
|
||||
( ^^^ this exc was collapsed from a group ^^^ )
|
||||
|
||||
# under --tpt-proto=tcp
|
||||
FAILED tests/test_multi_program.py::test_register_duplicate_name
|
||||
- OSError: all attempts to connect to 127.0.0.1:36003 failed
|
||||
( ^^^ this exc was collapsed from a group ^^^ )
|
||||
```
|
||||
|
||||
Distinct from the cancel-cascade `TooSlowError` flake
|
||||
class — see
|
||||
`cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
|
||||
This is a **connect-time race** before the daemon is
|
||||
fully ready to `accept()`, not a teardown-cascade
|
||||
slowness.
|
||||
|
||||
## Root cause: blind `time.sleep()` in `daemon` fixture
|
||||
|
||||
`tests/conftest.py::daemon` boots a sub-py-process via
|
||||
`subprocess.Popen([python, '-c', 'tractor.run_daemon(...)'])`,
|
||||
then **blindly sleeps** a fixed delay before yielding
|
||||
`proc` to the test:
|
||||
|
||||
```python
|
||||
# excerpt from tests/conftest.py::daemon
|
||||
proc = subprocess.Popen([
|
||||
sys.executable, '-c', code,
|
||||
])
|
||||
|
||||
bg_daemon_spawn_delay: float = _PROC_SPAWN_WAIT # 0.6
|
||||
if tpt_proto == 'uds':
|
||||
bg_daemon_spawn_delay += 1.6
|
||||
if _non_linux and ci_env:
|
||||
bg_daemon_spawn_delay += 1
|
||||
|
||||
# XXX, allow time for the sub-py-proc to boot up.
|
||||
# !TODO, see ping-polling ideas above!
|
||||
time.sleep(bg_daemon_spawn_delay)
|
||||
|
||||
assert not proc.returncode
|
||||
yield proc
|
||||
```
|
||||
|
||||
Inherent fragility: the delay is "long enough on dev
|
||||
boxes most of the time" but has no actual
|
||||
synchronization with the daemon's `bind()` + `listen()`
|
||||
completion. Under any of:
|
||||
|
||||
- Loaded box (CI parallelism, big rebuild in
|
||||
background, low-cpu-freq)
|
||||
- Cold first-run (`importlib` cache miss, JIT warmup)
|
||||
- Higher-than-expected `tractor` import cost
|
||||
- Filesystem latency (UDS sockfile create, slow
|
||||
tmpfs)
|
||||
|
||||
...the sleep finishes BEFORE the daemon has bound its
|
||||
listen socket → first test client call to
|
||||
`tractor.find_actor()` / `wait_for_actor()` /
|
||||
`open_nursery(registry_addrs=[reg_addr])`'s implicit
|
||||
connect → `ConnectionRefusedError` (TCP) or
|
||||
`FileNotFoundError`/`ConnectionRefusedError` (UDS).
|
||||
|
||||
## Reproducer
|
||||
|
||||
Easiest: run the suite under load.
|
||||
|
||||
```bash
|
||||
# create CPU pressure on another core in parallel
|
||||
stress-ng --cpu 2 --timeout 600s &
|
||||
|
||||
./py313/bin/python -m pytest \
|
||||
tests/test_multi_program.py::test_register_duplicate_name \
|
||||
--spawn-backend=main_thread_forkserver \
|
||||
--tpt-proto=tcp -v
|
||||
```
|
||||
|
||||
Reproduces ~30-50% of the time on a dev laptop. On a
|
||||
quiet idle box, may need 5-10 runs to hit.
|
||||
|
||||
## Why the existing `_PROC_SPAWN_WAIT` tuning is
|
||||
inadequate
|
||||
|
||||
Recent `bg_daemon_spawn_delay` rename
|
||||
(de-monotonic-grow fix) just-shipped removed the
|
||||
*accumulation* bug where each invocation made the
|
||||
NEXT test's wait longer too. Net effect: every
|
||||
invocation now uses the SAME `0.6 + 1.6` (UDS) or
|
||||
`0.6` (TCP) sleep, no growth. Good — but does
|
||||
NOTHING for the underlying race. Each individual
|
||||
test still relies on a blind sleep that may or may
|
||||
not be sufficient.
|
||||
|
||||
Bumping the constant higher pushes flake rate down
|
||||
but never to zero AND adds dead time to every
|
||||
non-flaking run. Not a fix, just a knob.
|
||||
|
||||
## Side effects
|
||||
|
||||
- **Inter-test cascade**: a single failure can cascade
|
||||
via leaked subprocesses (the `daemon` fixture's
|
||||
cleanup may not fully tear down a daemon that never
|
||||
reached "ready"). The `_reap_orphaned_subactors`
|
||||
session-end + `_track_orphaned_uds_per_test`
|
||||
per-test fixtures handle most of this now, but the
|
||||
affected test itself still fails.
|
||||
- **Worsens under fork-spawn backends**: the daemon
|
||||
has more init work
|
||||
(`_main_thread_forkserver`-coordinator-thread
|
||||
startup, etc.) so the sleep has to cover MORE.
|
||||
|
||||
## Fix design — replace blind sleep with active poll
|
||||
|
||||
The right primitive is **poll the daemon's bind
|
||||
address until it accepts a connection or we time
|
||||
out**, with the timeout being a hard ceiling rather
|
||||
than a baseline. Two implementation paths:
|
||||
|
||||
### Path A — TCP/UDS connect-poll loop
|
||||
|
||||
Try `socket.connect(reg_addr)` in a tight loop with
|
||||
short backoff (~50ms), succeed on the first non-error
|
||||
return, fail-loud on a hard cap (e.g. 10s). Same
|
||||
primitive works for both transports because both use
|
||||
`socket.connect()` semantics.
|
||||
|
||||
Rough shape:
|
||||
|
||||
```python
|
||||
def _wait_for_daemon_ready(
|
||||
reg_addr,
|
||||
tpt_proto: str,
|
||||
timeout: float = 10.0,
|
||||
poll_interval: float = 0.05,
|
||||
) -> None:
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
if tpt_proto == 'tcp':
|
||||
sock = socket.socket(socket.AF_INET)
|
||||
target = reg_addr # (host, port)
|
||||
else: # uds
|
||||
sock = socket.socket(socket.AF_UNIX)
|
||||
target = os.path.join(*reg_addr)
|
||||
try:
|
||||
sock.settimeout(poll_interval)
|
||||
sock.connect(target)
|
||||
except (
|
||||
ConnectionRefusedError,
|
||||
FileNotFoundError,
|
||||
socket.timeout,
|
||||
) as exc:
|
||||
if time.monotonic() >= deadline:
|
||||
raise TimeoutError(
|
||||
f'Daemon never accepted on {target!r} '
|
||||
f'within {timeout}s'
|
||||
) from exc
|
||||
time.sleep(poll_interval)
|
||||
else:
|
||||
sock.close()
|
||||
return
|
||||
```
|
||||
|
||||
Pros: trivial primitive, no tractor-runtime
|
||||
dependency, works pre-yield in the fixture body,
|
||||
fail-fast on truly-broken daemon.
|
||||
Cons: doesn't actually do an IPC handshake, just
|
||||
proves listen-side is up. A daemon that bound but
|
||||
hasn't initialized its registrar table yet would
|
||||
still race.
|
||||
|
||||
### Path B — `tractor.find_actor()` poll
|
||||
|
||||
Use the actual discovery API the test would call:
|
||||
|
||||
```python
|
||||
async def _wait_for_daemon_ready_via_discovery(
|
||||
reg_addr,
|
||||
timeout: float = 10.0,
|
||||
poll_interval: float = 0.05,
|
||||
):
|
||||
deadline = trio.current_time() + timeout
|
||||
async with tractor.open_root_actor(
|
||||
registry_addrs=[reg_addr],
|
||||
# ephemeral root just for the probe
|
||||
):
|
||||
while True:
|
||||
try:
|
||||
async with tractor.find_actor(
|
||||
'registrar', # daemon's own name
|
||||
registry_addrs=[reg_addr],
|
||||
) as portal:
|
||||
if portal is not None:
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
if trio.current_time() >= deadline:
|
||||
raise TimeoutError(...)
|
||||
await trio.sleep(poll_interval)
|
||||
```
|
||||
|
||||
Pros: actually proves the discovery path works,
|
||||
handles the "bound but not ready" case naturally.
|
||||
Cons: requires booting an ephemeral root actor JUST
|
||||
for the probe (overhead), more code, and runs in trio
|
||||
which complicates the sync-fixture context. Need a
|
||||
`trio.run()` wrapper.
|
||||
|
||||
### Recommended: Path A with optional handshake check
|
||||
|
||||
Path A is much simpler + handles 95% of the bug
|
||||
class. If "bound-but-not-ready" turns out to still
|
||||
race (it shouldn't — `tractor.run_daemon` doesn't
|
||||
return from `bind()` until the registrar is
|
||||
fully populated), escalate to Path B as a focused
|
||||
follow-up.
|
||||
|
||||
## Workarounds (until fix lands)
|
||||
|
||||
1. **Bump `_PROC_SPAWN_WAIT`** higher (current: 0.6).
|
||||
2.0–3.0 hides most flakes at the cost of adding
|
||||
dead time to every test. Not a fix but reduces
|
||||
blast radius while the proper poll lands.
|
||||
2. **`pytest-rerunfailures`** with `reruns=1` on the
|
||||
`daemon` fixture's tests specifically. Hides the
|
||||
flake but doesn't address it.
|
||||
3. **Mark known-affected tests as `xfail(strict=False)`**
|
||||
under `--ci`. Lets CI go green at the cost of
|
||||
silently hiding regressions.
|
||||
|
||||
(Recommend skipping all three — implement the active
|
||||
poll instead.)
|
||||
|
||||
## Investigation next steps
|
||||
|
||||
1. Implement Path A as a `_wait_for_daemon_ready()`
|
||||
helper in `tests/conftest.py`. Replace the
|
||||
`time.sleep(bg_daemon_spawn_delay)` call with it.
|
||||
2. Drop the `_PROC_SPAWN_WAIT` constant entirely
|
||||
(active poll obsoletes blind sleep).
|
||||
3. Run the suite 5-10 times to validate flake rate
|
||||
drops to 0.
|
||||
4. If flakes persist, profile whether the daemon
|
||||
process exits with non-zero before the poll's
|
||||
deadline hits — that'd be a different bug
|
||||
(daemon startup crash) that the blind sleep was
|
||||
masking.
|
||||
5. Cross-check `tests/test_multi_program.py::test_*`
|
||||
— multiple tests use the `daemon` fixture; all
|
||||
should benefit from the same poll primitive.
|
||||
|
||||
## Related
|
||||
|
||||
- `tests/conftest.py::daemon` — the fixture under
|
||||
fix
|
||||
- `tests/conftest.py::_PROC_SPAWN_WAIT` — the
|
||||
constant to drop
|
||||
- `cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`
|
||||
— distinct flake class (cancel-cascade
|
||||
`TooSlowError` at teardown, not connect-time race)
|
||||
- `trio_wakeup_socketpair_busy_loop_under_fork_issue.md`
|
||||
— different bug entirely; this race was masked
|
||||
pre-WakeupSocketpair-patch by the busy-loop
|
||||
hangs.
|
||||
|
|
@ -1,102 +0,0 @@
|
|||
# `trio` 0.29 -> 0.33 slows the depth=3 cancel-cascade
|
||||
|
||||
## Symptom
|
||||
|
||||
After locking to `trio==0.33.0` (commit `c7741bba`, was
|
||||
`0.29.0`), this test reliably trips its `fail_after`
|
||||
deadline on the **`trio`** backend:
|
||||
|
||||
```
|
||||
FAILED tests/test_cancellation.py::test_nested_multierrors[start_method=trio-depth=3]
|
||||
- AssertionError: assert False
|
||||
where False = isinstance(
|
||||
Cancelled(source='deadline', source_task=None, reason=None),
|
||||
tractor.RemoteActorError,
|
||||
)
|
||||
```
|
||||
|
||||
A `fail_after_w_trace` hang-snapshot is captured for the
|
||||
test each run (deadline-injected `Cancelled` wrapped into
|
||||
the actor-nursery `BaseExceptionGroup`).
|
||||
|
||||
## Root cause (immediate)
|
||||
|
||||
The test budgets `fail_after(6)` for the `trio` backend.
|
||||
That 6s was chosen (commit `32955db0`, while `trio==0.29`)
|
||||
with the assertion that trio finishes "well under" 6s.
|
||||
The `trio` 0.29 -> 0.33 bump slowed the depth=3 cascade
|
||||
past that budget, so the 6s deadline now fires mid-cascade.
|
||||
|
||||
trio 0.33 added **cancel-reason tracking** — every
|
||||
`Cancelled` now carries `(source=, reason=, source_task=)`.
|
||||
The injected exc is `Cancelled(source='deadline')`, i.e.
|
||||
trio itself naming our `fail_after(6)` scope as the cancel
|
||||
origin. When that `Cancelled` collapses one branch of the
|
||||
nursery BEG, the test's `isinstance(subexc,
|
||||
RemoteActorError)` assertion fails. The healthy outcome is
|
||||
`BEG = [RemoteActorError, RemoteActorError]`; the
|
||||
`Cancelled` is purely an artifact of the deadline cutting
|
||||
the cascade short.
|
||||
|
||||
## Measurements (standalone, this machine)
|
||||
|
||||
```
|
||||
depth=1 trio ~3.15s PASS (keeps 6s budget)
|
||||
depth=3 trio ~6.8-8.2s FAIL @ 6s (now bumped to 12s)
|
||||
```
|
||||
|
||||
depth=1 still fits comfortably; only depth=3 (deeper
|
||||
recursive spawn-and-error tree => more actors to reap)
|
||||
exceeds the old budget. The ~2s/depth-level cost looks
|
||||
like serialized per-actor reap / `terminate_after` waits.
|
||||
|
||||
## Mitigation applied
|
||||
|
||||
`test_nested_multierrors` now splits the `trio` budget:
|
||||
|
||||
```python
|
||||
case ('trio', 1):
|
||||
timeout = 6
|
||||
case ('trio', 3):
|
||||
timeout = 12 # was 6; see this doc
|
||||
```
|
||||
|
||||
This stops the deadline from firing so the cascade
|
||||
completes naturally to `[RAE, RAE]`.
|
||||
|
||||
## Also affected — same root cause, different test
|
||||
|
||||
`test_echoserver_detailed_mechanics[trio-raise_error=KeyboardInterrupt]`
|
||||
(`tests/test_infected_asyncio.py`) tripped the *same*
|
||||
slowdown via its much tighter `trio` budget of `1s`. The
|
||||
single-aio-subactor teardown now takes ~1s, so the `1s`
|
||||
`fail_after` raced the deadline (PASS at 0.99s / FAIL at
|
||||
1.03s across back-to-back standalone runs). On a deadline-
|
||||
fire the injected `Cancelled(source='deadline')` wraps the
|
||||
mid-stream `KeyboardInterrupt` into a `BaseExceptionGroup`,
|
||||
which is NOT a `KeyboardInterrupt` so the bare
|
||||
`pytest.raises(KeyboardInterrupt)` fails. (The sibling
|
||||
`raise_error=Exception` variant only "passes" by accident:
|
||||
an `ExceptionGroup` *is-a* `Exception`, so its
|
||||
`pytest.raises(Exception)` still matches even when wrapped.)
|
||||
|
||||
Mitigation: bump that `trio` budget `1 -> 4s` (matching the
|
||||
forking-spawner case). Without a deadline-fire the KBI
|
||||
propagates bare and the assertion passes.
|
||||
|
||||
## Open follow-up (the actual regression)
|
||||
|
||||
The budget bump is a band-aid — the underlying question is
|
||||
**why** the depth=3 `trio` cancel-cascade went from <6s to
|
||||
~7-8s across `trio` 0.29 -> 0.33. Candidate avenues:
|
||||
|
||||
- which scope owns the per-actor `terminate_after` wait,
|
||||
and are the tree's reaps concurrent or serialized?
|
||||
- did trio 0.33's abort/reschedule or cancel-reason
|
||||
bookkeeping change checkpoint timing on the cancel path?
|
||||
|
||||
If/when the cascade speeds back up under-budget, depth=3
|
||||
will start completing well under 12s — at which point the
|
||||
budget can be tightened back toward 6s as a regression
|
||||
tripwire. Related (different backend, same cascade class):
|
||||
`cancel_cascade_too_slow_under_main_thread_forkserver_issue.md`.
|
||||
|
|
@ -1,221 +0,0 @@
|
|||
# trio `WakeupSocketpair.drain()` busy-loop in forked child (peer-closed missed-EOF)
|
||||
|
||||
## Reproducer
|
||||
|
||||
```bash
|
||||
./py313/bin/python -m pytest \
|
||||
tests/test_multi_program.py::test_register_duplicate_name \
|
||||
--tpt-proto=tcp \
|
||||
--spawn-backend=main_thread_forkserver \
|
||||
-v --capture=sys
|
||||
```
|
||||
|
||||
Subactor pegs a CPU core indefinitely; parent test
|
||||
hangs waiting for the subactor.
|
||||
|
||||
## Empirical evidence (caught alive)
|
||||
|
||||
```
|
||||
$ sudo strace -p <subactor-pid>
|
||||
recvfrom(6, "", 65536, 0, NULL, NULL) = 0
|
||||
recvfrom(6, "", 65536, 0, NULL, NULL) = 0
|
||||
recvfrom(6, "", 65536, 0, NULL, NULL) = 0
|
||||
... (no `epoll_wait`, no other syscalls, just this back-to-back)
|
||||
```
|
||||
|
||||
Pattern: tight C-level `recvfrom` loop returning 0
|
||||
each call. No `epoll_wait` between iterations →
|
||||
**not trio's task scheduler**. Pure synchronous C
|
||||
loop.
|
||||
|
||||
```
|
||||
$ sudo readlink /proc/<subactor-pid>/fd/6
|
||||
socket:[<inode>]
|
||||
|
||||
$ sudo lsof -p <subactor-pid> | grep ' 6u'
|
||||
<cmd> <pid> goodboy 6u unix 0xffff... 0t0 <inode> type=STREAM (CONNECTED)
|
||||
```
|
||||
|
||||
fd=6 is an **AF_UNIX socket** in CONNECTED state.
|
||||
Even though the test uses `--tpt-proto=tcp`, this fd
|
||||
is NOT a tractor IPC channel — it's an internal
|
||||
trio socketpair.
|
||||
|
||||
## Root-cause: `WakeupSocketpair.drain()`
|
||||
|
||||
`/site-packages/trio/_core/_wakeup_socketpair.py`:
|
||||
|
||||
```python
|
||||
class WakeupSocketpair:
|
||||
def __init__(self) -> None:
|
||||
self.wakeup_sock, self.write_sock = socket.socketpair()
|
||||
self.wakeup_sock.setblocking(False)
|
||||
self.write_sock.setblocking(False)
|
||||
...
|
||||
|
||||
def drain(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
self.wakeup_sock.recv(2**16)
|
||||
except BlockingIOError:
|
||||
pass
|
||||
```
|
||||
|
||||
`socket.socketpair()` on Linux defaults to AF_UNIX
|
||||
SOCK_STREAM. Both ends non-blocking. Normal flow:
|
||||
|
||||
1. Signal/wake event → `write_sock.send(b'\x00')`
|
||||
queues a byte.
|
||||
2. `wakeup_sock` becomes readable → trio's epoll
|
||||
triggers.
|
||||
3. Trio calls `drain()` to flush the buffer.
|
||||
4. drain loops on `wakeup_sock.recv(64KB)`.
|
||||
5. Eventually buffer empty → non-blocking socket
|
||||
raises `BlockingIOError` → except → break.
|
||||
|
||||
**Bug surface — peer-closed missed-EOF**:
|
||||
|
||||
Non-blocking socket semantics:
|
||||
- buffer has data → `recv` returns N>0 bytes (loop continues)
|
||||
- buffer empty → `recv` raises `BlockingIOError`
|
||||
- **peer FIN'd → `recv` returns 0 bytes (NEITHER exception NOR
|
||||
break — infinite tight loop)**
|
||||
|
||||
`drain()` does not handle the `b''` return-value
|
||||
(EOF) case. If `write_sock` has been closed (or the
|
||||
process holding it is gone), every iteration returns
|
||||
0 → infinite loop → 100% CPU on a single core.
|
||||
|
||||
## Why this triggers under `main_thread_forkserver`
|
||||
|
||||
Under `os.fork()` from the forkserver-worker thread:
|
||||
|
||||
1. Parent has a `WakeupSocketpair` instance with
|
||||
`wakeup_sock=fdN`, `write_sock=fdM`. Both fds
|
||||
open in parent.
|
||||
2. Fork → child inherits BOTH fds (kernel-level fd
|
||||
table dup).
|
||||
3. `_close_inherited_fds()` runs in child →
|
||||
closes everything except stdio. `wakeup_sock` and
|
||||
`write_sock` of the parent's `WakeupSocketpair`
|
||||
ARE closed in child.
|
||||
4. Child's trio (running fresh) creates its OWN
|
||||
`WakeupSocketpair` → NEW fd numbers (e.g. fd 6, 7).
|
||||
5. **In `infect_asyncio` mode** the asyncio loop is
|
||||
the host; trio runs as guest via
|
||||
`start_guest_run`. trio still creates its
|
||||
`WakeupSocketpair` in the I/O manager but its
|
||||
role is different.
|
||||
|
||||
The race window: somewhere between (3) and (5), if a
|
||||
`WakeupSocketpair` Python object reference inherited
|
||||
via COW (from parent's pre-fork heap) survives long
|
||||
enough that `drain()` is called on it AFTER its fds
|
||||
were closed but BEFORE the child's NEW socketpair
|
||||
takes over the recycled fd numbers — the recycled fd
|
||||
will be one of the child's NEW socketpair ends, whose
|
||||
peer might be FIN-flagged (e.g. parent-process
|
||||
peer-end is closed).
|
||||
|
||||
Or simpler: the `wait_for_actor`/`find_actor` discovery
|
||||
flow in `test_register_duplicate_name` triggers an
|
||||
unusual code path where a stale `WakeupSocketpair`
|
||||
gets `drain()`-called on a fd whose peer has already
|
||||
closed.
|
||||
|
||||
## Why `drain()` shouldn't loop indefinitely on EOF
|
||||
(upstream trio bug)
|
||||
|
||||
Even WITHOUT fork, `drain()` should treat `b''` as
|
||||
EOF and break. The current code is correct for the
|
||||
"buffer drained on a healthy socketpair" scenario but
|
||||
incorrect for the "peer is gone" scenario. It's a
|
||||
defensive-programming gap in trio.
|
||||
|
||||
A one-line patch upstream:
|
||||
|
||||
```python
|
||||
def drain(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
data = self.wakeup_sock.recv(2**16)
|
||||
if not data:
|
||||
break # peer-closed; nothing more to drain
|
||||
except BlockingIOError:
|
||||
pass
|
||||
```
|
||||
|
||||
## Workarounds (until the underlying issue lands)
|
||||
|
||||
1. **Skip-mark on the fork backend**:
|
||||
`tests/test_multi_program.py` →
|
||||
`pytest.mark.skipon_spawn_backend('main_thread_forkserver',
|
||||
reason='trio WakeupSocketpair.drain busy-loop, see ai/conc-anal/trio_wakeup_socketpair_busy_loop_under_fork_issue.md')`.
|
||||
|
||||
2. **Defensive monkey-patch in tractor's
|
||||
forkserver-child prelude** — wrap
|
||||
`WakeupSocketpair.drain` to handle `b''`:
|
||||
|
||||
```python
|
||||
# in `_actor_child_main` or `_close_inherited_fds`'s
|
||||
# post-fork prelude:
|
||||
from trio._core._wakeup_socketpair import WakeupSocketpair
|
||||
_orig_drain = WakeupSocketpair.drain
|
||||
def _safe_drain(self):
|
||||
try:
|
||||
while True:
|
||||
data = self.wakeup_sock.recv(2**16)
|
||||
if not data:
|
||||
return # peer closed
|
||||
except BlockingIOError:
|
||||
pass
|
||||
WakeupSocketpair.drain = _safe_drain
|
||||
```
|
||||
|
||||
Tracks upstream — remove once trio fixes.
|
||||
|
||||
3. **Upstream the fix**: 1-line PR to `python-trio/trio`
|
||||
adding `if not data: break` to `drain()`.
|
||||
|
||||
## Investigation next steps
|
||||
|
||||
1. **Confirm via py-spy**: when caught alive, detach
|
||||
strace first then
|
||||
`sudo py-spy dump --pid <subactor> --locals`. The
|
||||
busy thread should show `drain` from `WakeupSocketpair`
|
||||
in the call chain.
|
||||
2. **Identify which write-end peer is closed**: from
|
||||
the inode of fd 6, look up the matching peer
|
||||
inode via `ss -xp` and see whose process it
|
||||
was/is.
|
||||
3. **Verify the missed-EOF hypothesis**: hand-craft a
|
||||
minimal `WakeupSocketpair` repro:
|
||||
|
||||
```python
|
||||
from trio._core._wakeup_socketpair import WakeupSocketpair
|
||||
ws = WakeupSocketpair()
|
||||
ws.write_sock.close() # simulate peer-gone
|
||||
ws.drain() # should hang forever
|
||||
```
|
||||
|
||||
## Sibling bug
|
||||
|
||||
`tests/test_infected_asyncio.py::test_aio_simple_error`
|
||||
hangs under the same backend with a DIFFERENT
|
||||
fingerprint (Mode-A deadlock, both parties in
|
||||
`epoll_wait`, no busy-loop). Distinct root cause —
|
||||
see `infected_asyncio_under_main_thread_forkserver_hang_issue.md`.
|
||||
|
||||
Both share the broader theme: **trio internal-state
|
||||
initialization isn't fully fork-safe under
|
||||
`main_thread_forkserver`** for the more exotic
|
||||
dispatch paths.
|
||||
|
||||
## See also
|
||||
|
||||
- [#379](https://github.com/goodboy/tractor/issues/379) — subint umbrella
|
||||
- python-trio/trio#1614 — trio + fork hazards
|
||||
- `trio._core._wakeup_socketpair.WakeupSocketpair`
|
||||
source (the smoking gun)
|
||||
- `ai/conc-anal/fork_thread_semantics_execution_vs_memory.md`
|
||||
- `ai/conc-anal/infected_asyncio_under_main_thread_forkserver_hang_issue.md`
|
||||
|
|
@ -1,54 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
session: (ad-hoc, not tracked via conf.toml)
|
||||
timestamp: 2026-04-06T17:28:48Z
|
||||
git_ref: 02b2ef1
|
||||
scope: tests
|
||||
substantive: true
|
||||
raw_file: 20260406T172848Z_02b2ef1_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
User asked to extend `tests/test_resource_cache.py` with a test
|
||||
that reproduces the edge case fixed in commit `02b2ef18` (per-key
|
||||
locking+user tracking in `maybe_open_context()`). The bug was
|
||||
originally triggered in piker's `brokerd.kraken` backend where the
|
||||
same `acm_func` was called with different kwargs, and the old
|
||||
global `_Cache.users` counter caused:
|
||||
|
||||
- teardown skipped for one `ctx_key` bc another key's users kept
|
||||
the global count > 0
|
||||
- re-entry hitting `assert not resources.get(ctx_key)` during the
|
||||
teardown window
|
||||
|
||||
User requested a test that would fail under the old code and pass
|
||||
with the fix.
|
||||
|
||||
## Response summary
|
||||
|
||||
Designed and implemented `test_per_ctx_key_resource_lifecycle`
|
||||
which verifies per-`ctx_key` resource isolation by:
|
||||
|
||||
1. Holding resource `'a'` open in a bg task
|
||||
2. Opening+closing resource `'b'` (same `acm_func`, different
|
||||
kwargs) while `'a'` is still alive
|
||||
3. Re-opening `'b'` and asserting cache MISS — proving `'b'` was
|
||||
torn down independently despite `'a'` keeping its own user
|
||||
count > 0
|
||||
|
||||
With the old global counter, phase 3 would produce a stale cache
|
||||
HIT (leaked resource) or crash on the assert.
|
||||
|
||||
Also added a trivial `acm_with_resource(resource_id)` ACM helper
|
||||
at module level.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tests/test_resource_cache.py` — add `acm_with_resource` ACM +
|
||||
`test_per_ctx_key_resource_lifecycle` test fn
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated (pending user review).
|
||||
|
|
@ -1,57 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
session: (ad-hoc, not tracked via conf.toml)
|
||||
timestamp: 2026-04-06T19:31:25Z
|
||||
git_ref: 85f9c5d
|
||||
scope: tests
|
||||
substantive: true
|
||||
raw_file: 20260406T193125Z_85f9c5d_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
User asked to reproduce the exact `assert not resources.get(ctx_key)`
|
||||
crash originally seen in piker's `brokerd.kraken` backend via
|
||||
`open_cached_client('kraken')`. Key constraints from user:
|
||||
|
||||
- In piker, kwargs were the **same** (empty) so all callers
|
||||
share one `ctx_key = (fid, ())`
|
||||
- The root issue is `_Cache.locks` being indexed by `fid`
|
||||
rather than `ctx_key`, plus the race window between
|
||||
`values.pop()` and `resources.pop()` in `_Cache.run_ctx`
|
||||
(the acm `__aexit__` has checkpoints in between)
|
||||
- Mark the test `@pytest.mark.xfail` so it can be committed
|
||||
against the unpatched code and later adjusted once the fix
|
||||
from `9e49eddd` is applied
|
||||
|
||||
User pointed to piker's `open_cached_client('kraken')` usage
|
||||
as the real-world pattern to model.
|
||||
|
||||
## Response summary
|
||||
|
||||
Added `test_moc_reentry_during_teardown` which deterministically
|
||||
reproduces the teardown race:
|
||||
|
||||
1. A `cached_client()` acm (no params, like
|
||||
`kraken.api.get_client()`) signals via `in_aexit` event
|
||||
when its `__aexit__` starts, then sleeps to hold the
|
||||
window open
|
||||
2. `use_and_exit` task enters+exits the cached resource
|
||||
3. `reenter_during_teardown` waits on `in_aexit` then
|
||||
re-enters — landing exactly in the window where
|
||||
`values` is popped but `resources` still exists
|
||||
4. The old `assert not resources.get(ctx_key)` fires
|
||||
5. `@pytest.mark.xfail(raises=AssertionError)` catches it
|
||||
|
||||
Test confirmed: `XFAIL` with the exact traceback at
|
||||
`_mngrs.py:327`.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tests/test_resource_cache.py` — add
|
||||
`test_moc_reentry_during_teardown` xfail test fn
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated (pending user review).
|
||||
|
|
@ -1,52 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
session: (ad-hoc, not tracked via conf.toml)
|
||||
timestamp: 2026-04-06T22:26:48Z
|
||||
git_ref: f086222
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260406T222648Z_f086222_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
User applied the per-key locking patch (`f086222d`),
|
||||
removed the `@pytest.mark.xfail` from
|
||||
`test_moc_reentry_during_teardown`, and found the
|
||||
test STILL fails — now with `RuntimeError` instead
|
||||
of `AssertionError`. Asked to diagnose why the fix
|
||||
doesn't actually resolve this race.
|
||||
|
||||
## Response summary
|
||||
|
||||
Traced the exact scheduling sequence across three
|
||||
concurrent tasks to show that the per-`ctx_key`
|
||||
locking change doesn't close the fundamental gap:
|
||||
|
||||
- `maybe_open_context`'s finally pops the lock and
|
||||
sets `no_more_users` in the **exiting caller's
|
||||
task**
|
||||
- But the actual resource cleanup (`values.pop`,
|
||||
acm `__aexit__`, `resources.pop`) runs in the
|
||||
**`run_ctx` task** inside `service_tn`
|
||||
- A re-entering task creates a fresh lock (old one
|
||||
popped), acquires immediately, and hits stale
|
||||
`resources[ctx_key]` while `run_ctx` is still in
|
||||
`__aexit__`
|
||||
|
||||
Identified three possible fix strategies:
|
||||
1. Let `run_ctx` own lock release/pop after
|
||||
`resources.pop()`
|
||||
2. Add a "teardown complete" event that new callers
|
||||
wait on
|
||||
3. Move `resources.pop()` into `run_ctx`'s inner
|
||||
finally (before acm `__aexit__`)
|
||||
|
||||
## Files changed
|
||||
|
||||
No code changes — analysis only.
|
||||
|
||||
## Human edits
|
||||
|
||||
N/A — diagnostic output, no code generated.
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
timestamp: 2026-04-06T22:26:48Z
|
||||
git_ref: f086222
|
||||
diff_cmd: git diff f086222~1..f086222
|
||||
---
|
||||
|
||||
Here's the problem. The per-key locking doesn't fix this race because the **lock lifetime is decoupled from `run_ctx`'s lifetime**. Trace through what happens:
|
||||
|
||||
**`maybe_open_context` finally** (lines 468-495):
|
||||
1. `users[ctx_key] -= 1` → 0
|
||||
2. `no_more_users.set()` → wakes `run_ctx` (but it doesn't run yet, just becomes ready)
|
||||
3. `_Cache.locks.pop(ctx_key)` → **lock is gone**
|
||||
4. `maybe_open_context` exits
|
||||
|
||||
**`run_ctx`** now gets scheduled (lines 237-241):
|
||||
5. Inner finally: `values.pop(ctx_key)` → values gone
|
||||
6. `mng.__aexit__()` runs → `in_aexit.set()`, `await trio.sleep(10)` → **CHECKPOINT**
|
||||
|
||||
**`reenter_during_teardown`** wakes:
|
||||
7. Enters `maybe_open_context` → no lock exists for `ctx_key` → creates **fresh** lock, acquires immediately
|
||||
8. `values[ctx_key]` → `KeyError`
|
||||
9. `resources.get(ctx_key)` → **still exists!** (step 6 hasn't reached `resources.pop()` in `run_ctx`'s outer finally yet)
|
||||
10. `raise RuntimeError('Caching resources ALREADY exist?!')`
|
||||
|
||||
The core issue: `no_more_users.set()` (step 2) and `locks.pop()` (step 3) happen in the **exiting caller's task**, but the actual resource cleanup (`values.pop`, acm `__aexit__`, `resources.pop`) happens in the **`run_ctx` task** inside `service_tn`. There's no synchronization between "lock removed, new callers welcome" and "run_ctx has actually finished tearing down". The per-`ctx_key` lock change just made the lock more granular — it didn't close this gap.
|
||||
|
||||
The fix needs to ensure a new caller can't acquire a lock for `ctx_key` until `run_ctx` has fully exited (including `resources.pop()`). Either:
|
||||
- **don't pop the lock in `maybe_open_context`** — let `run_ctx` release/pop it after `resources.pop()`
|
||||
- **wait on a "teardown complete" event** in the cache-miss path when stale `resources` exist
|
||||
- **pop `resources` before the acm `__aexit__`** (move `resources.pop()` into the inner finally alongside `values.pop()`)
|
||||
|
|
@ -1,74 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
session: (ad-hoc, not tracked via conf.toml)
|
||||
timestamp: 2026-04-13T19:21:16Z
|
||||
git_ref: f851f28
|
||||
scope: code
|
||||
substantive: true
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
User provided a detailed implementation plan for
|
||||
a new `test_tpt_bind_addrs.py` test suite covering
|
||||
`open_root_actor(tpt_bind_addrs=...)` — the three
|
||||
runtime code paths in `_root.py:385-450`:
|
||||
|
||||
1. Non-registrar, no explicit bind -> random addrs
|
||||
2. Registrar, no explicit bind -> registry_addrs
|
||||
3. Explicit bind given -> `wrap_address()` + merge
|
||||
|
||||
Plan specified 6 test functions (~10 parametrized
|
||||
variants), predicted a type-mixing bug at line 446,
|
||||
and asked for an in-flight fix if confirmed.
|
||||
|
||||
## Response summary
|
||||
|
||||
Created `tests/discovery/test_tpt_bind_addrs.py`
|
||||
with 9 collected test variants across 6 functions:
|
||||
|
||||
- `test_registrar_root_tpt_bind_addrs` (3 variants:
|
||||
`bind-eq-reg`, `bind-subset-reg`,
|
||||
`bind-disjoint-reg`)
|
||||
- `test_non_registrar_root_tpt_bind_addrs`
|
||||
(2 variants: `bind-same-bindspace`,
|
||||
`bind-disjoint`)
|
||||
- `test_non_registrar_default_random_bind`
|
||||
(baseline, no explicit bind)
|
||||
- `test_tpt_bind_addrs_as_maddr_str`
|
||||
(multiaddr string input)
|
||||
- `test_registrar_merge_binds_union`
|
||||
(registrar + disjoint bind -> union)
|
||||
- `test_open_nursery_forwards_tpt_bind_addrs`
|
||||
(`open_nursery(**kwargs)` forwarding)
|
||||
|
||||
Confirmed and fixed the predicted bug at
|
||||
`_root.py:446`: the registrar merge path mixed
|
||||
`Address` objects (`tpt_bind_addrs`) with raw tuples
|
||||
(`uw_reg_addrs`) inside `set()`, preventing
|
||||
deduplication and causing double-bind `OSError`.
|
||||
|
||||
Fix: wrap `uw_reg_addrs` before the set union:
|
||||
```python
|
||||
# before (broken)
|
||||
tpt_bind_addrs = list(set(
|
||||
tpt_bind_addrs + uw_reg_addrs
|
||||
))
|
||||
# after (fixed)
|
||||
tpt_bind_addrs = list(set(
|
||||
tpt_bind_addrs
|
||||
+ [wrap_address(a) for a in uw_reg_addrs]
|
||||
))
|
||||
```
|
||||
|
||||
All 9 tests pass after the fix.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tests/discovery/test_tpt_bind_addrs.py` (new)
|
||||
- `tractor/_root.py:446` (bug fix, 1 line)
|
||||
|
||||
## Human edits
|
||||
|
||||
N/A — pending review.
|
||||
|
|
@ -1,50 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
session: 76154e65-d8e1-4b5f-9275-0ea45ba7e98a
|
||||
timestamp: 2026-04-13T20:50:48Z
|
||||
git_ref: 269d939c
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260413T205048Z_269d939c_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Implement a `parse_endpoints()` API in
|
||||
`tractor.discovery._multiaddr` that lets downstream
|
||||
projects declare per-actor transport bind addresses
|
||||
as a service table of actor-name -> multiaddr
|
||||
strings (loaded from config, e.g. TOML `[network]`
|
||||
section). Add type aliases `EndpointsTable` and
|
||||
`ParsedEndpoints`, plus 7 unit tests covering TCP,
|
||||
UDS, tuples, mixed input, unsupported protocols, and
|
||||
edge cases.
|
||||
|
||||
A detailed implementation plan was provided
|
||||
specifying insertion points, type signatures, test
|
||||
names, and verification commands.
|
||||
|
||||
## Response summary
|
||||
|
||||
Add `parse_endpoints()` function with two supporting
|
||||
type aliases to `_multiaddr.py`. The function
|
||||
iterates a `dict[str, list[str|tuple]]` service
|
||||
table and delegates each entry to the existing
|
||||
`wrap_address()` helper, which already handles maddr
|
||||
strings, raw tuples, and pre-wrapped `Address`
|
||||
objects. Seven pure unit tests (no trio runtime)
|
||||
validate all input variants and edge cases. All 22
|
||||
multiaddr tests pass.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/discovery/_multiaddr.py` — add
|
||||
`EndpointsTable`, `ParsedEndpoints` type aliases
|
||||
and `parse_endpoints()` function after line 133
|
||||
- `tests/discovery/test_multiaddr.py` — add
|
||||
`parse_endpoints` import and 7 new test functions
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated.
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
timestamp: 2026-04-13T20:50:48Z
|
||||
git_ref: 269d939c
|
||||
diff_cmd: git diff 269d939c~1..269d939c
|
||||
---
|
||||
|
||||
## Implementation output
|
||||
|
||||
### `tractor/discovery/_multiaddr.py`
|
||||
|
||||
> `git diff 269d939c~1..269d939c -- tractor/discovery/_multiaddr.py`
|
||||
|
||||
Type aliases `EndpointsTable` and `ParsedEndpoints`
|
||||
plus `parse_endpoints()` helper that wraps each
|
||||
entry via `wrap_address()`.
|
||||
|
||||
### `tests/discovery/test_multiaddr.py`
|
||||
|
||||
> `git diff 269d939c~1..269d939c -- tests/discovery/test_multiaddr.py`
|
||||
|
||||
Add import of `parse_endpoints` to existing import
|
||||
block, then 7 new test functions.
|
||||
|
||||
### Test run output
|
||||
|
||||
```
|
||||
22 passed, 1 warning in 0.05s
|
||||
```
|
||||
|
||||
All 22 tests pass (15 existing + 7 new).
|
||||
|
|
@ -1,38 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
session: multiaddr-support-rename-prefer
|
||||
timestamp: 2026-04-14T16:33:00Z
|
||||
git_ref: befedc49
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260414T163300Z_befedc49_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Create a helper function that determines the best transport given
|
||||
actor locality (distributed vs same host). Use PID/hostname
|
||||
comparison for locality detection, apply at registry addr selection
|
||||
only (not spawn-time).
|
||||
|
||||
## Response summary
|
||||
|
||||
New `prefer_addr()` + `_is_local_addr()` helpers
|
||||
in `_api.py` using `socket.getaddrinfo()` and
|
||||
`ipaddress` for PID/hostname locality detection.
|
||||
Preference: UDS > local TCP > remote TCP.
|
||||
Integrated into `query_actor()` and
|
||||
`wait_for_actor()`. Also changed
|
||||
`Registrar.find_actor()` to return full addr list
|
||||
so callers can apply preference.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/discovery/_discovery.py` → `_api.py`
|
||||
— renamed + added `prefer_addr()`,
|
||||
`_is_local_addr()`; updated `query_actor()` and
|
||||
`wait_for_actor()` call sites
|
||||
- `tractor/discovery/_registry.py`
|
||||
— `Registrar.find_actor()` returns
|
||||
`list[UnwrappedAddress]|None`
|
||||
|
|
@ -1,62 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-6
|
||||
service: claude
|
||||
timestamp: 2026-04-14T16:33:00Z
|
||||
git_ref: befedc49
|
||||
diff_cmd: git diff befedc49~1..befedc49
|
||||
---
|
||||
|
||||
### `tractor/discovery/_api.py`
|
||||
|
||||
> `git diff befedc49~1..befedc49 -- tractor/discovery/_api.py`
|
||||
|
||||
Add `_is_local_addr()` and `prefer_addr()` transport
|
||||
preference helpers.
|
||||
|
||||
#### `_is_local_addr(addr: Address) -> bool`
|
||||
|
||||
Determines whether an `Address` is reachable on the
|
||||
local host:
|
||||
|
||||
- `UDSAddress`: always returns `True`
|
||||
(filesystem-bound, inherently local)
|
||||
- `TCPAddress`: checks if `._host` is a loopback IP
|
||||
via `ipaddress.ip_address().is_loopback`, then
|
||||
falls back to comparing against the machine's own
|
||||
interface IPs via
|
||||
`socket.getaddrinfo(socket.gethostname(), None)`
|
||||
|
||||
#### `prefer_addr(addrs: list[UnwrappedAddress]) -> UnwrappedAddress`
|
||||
|
||||
Selects the "best" transport address from a
|
||||
multihomed actor's address list. Wraps each
|
||||
candidate via `wrap_address()` to get typed
|
||||
`Address` objects, then classifies into three tiers:
|
||||
|
||||
1. **UDS** (same-host guaranteed, lowest overhead)
|
||||
2. **TCP loopback / same-host IP** (local network)
|
||||
3. **TCP remote** (only option for distributed)
|
||||
|
||||
Within each tier, the last-registered (latest) entry
|
||||
is preferred. Falls back to `addrs[-1]` if no
|
||||
heuristic matches.
|
||||
|
||||
### `tractor/discovery/_registry.py`
|
||||
|
||||
> `git diff befedc49~1..befedc49 -- tractor/discovery/_registry.py`
|
||||
|
||||
`Registrar.find_actor()` return type broadened from
|
||||
single addr to `list[UnwrappedAddress]|None` — full
|
||||
addr list lets callers apply transport preference.
|
||||
|
||||
#### Integration
|
||||
|
||||
`query_actor()` and `wait_for_actor()` now call
|
||||
`prefer_addr(addrs)` instead of `addrs[-1]`.
|
||||
|
||||
### Verification
|
||||
|
||||
All discovery tests pass (13/13 non-daemon).
|
||||
`test_local.py` and `test_multi_program.py` also
|
||||
pass (daemon fixture teardown failures are
|
||||
pre-existing and unrelated).
|
||||
|
|
@ -1,101 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
session: subints-spawner-design-kickoff
|
||||
timestamp: 2026-04-17T03:49:18Z
|
||||
git_ref: 9703210
|
||||
scope: docs
|
||||
substantive: true
|
||||
raw_file: 20260417T034918Z_9703210_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Drive the "first big boi, from GH issue" task seeded by
|
||||
`ai/prompt-io/prompts/subints_spawner.md`: design, plan
|
||||
and implement sub-interpreter (subint) spawn-backend
|
||||
support per issue #379, including (1) modularizing
|
||||
`tractor.spawn._spawn` into per-backend submods, (2) a new
|
||||
`._subint` backend, and (3) harness parametrization via the
|
||||
existing `--spawn-backend` / `start_method` pytest fixture
|
||||
in `tractor._testing.pytest`.
|
||||
|
||||
Follow-up clarifications from the user (this turn):
|
||||
1. Pin `<3.15` on this dev branch and feature-gate subint
|
||||
tests — chose option (a).
|
||||
2. Split Phase A (modularization) into its own PR first.
|
||||
3. Defer the `fork()`-via-subint hack to a follow-up.
|
||||
4. Harness flag is `pytest --spawn-backend <key>` CLI →
|
||||
`start_method` fixture (prompt file updated to match).
|
||||
|
||||
## Response summary
|
||||
|
||||
Produced a three-phase plan and a concrete Phase A (pure
|
||||
modularization) file-split plan for user review; no code
|
||||
written yet — the green-light to start Phase A was given
|
||||
in this same turn conditional on logging this prompt-io
|
||||
entry first.
|
||||
|
||||
Phases:
|
||||
- **A — modularize** `tractor/spawn/_spawn.py` (847 LOC):
|
||||
keep generic machinery in `_spawn.py`, extract
|
||||
`trio_proc` → `spawn/_trio.py`, `mp_proc` →
|
||||
`spawn/_mp.py`. No pin bump.
|
||||
- **B — `_subint` backend**: bump `pyproject.toml`
|
||||
`requires-python` upper to `<3.15`; add `'subint'` to
|
||||
`SpawnMethodKey`; reuse existing UDS transport; shm
|
||||
escape-hatch deferred.
|
||||
- **C — harness**: drive the valid-backend tuple in
|
||||
`tractor/_testing/pytest.py:345-349` from
|
||||
`typing.get_args(SpawnMethodKey)`; skip subint tests on
|
||||
Python < 3.14.
|
||||
|
||||
Key findings surfaced to the user:
|
||||
- `pyproject.toml:12` currently pins `<3.14`; PEP 734
|
||||
`concurrent.interpreters` only ships in 3.14 — the
|
||||
load-bearing constraint.
|
||||
- `_testing/pytest.py:345-349` hardcodes valid backends
|
||||
as a string tuple (`'mp_spawn'`, `'mp_forkserver'`,
|
||||
`'trio'`) — should be `get_args(SpawnMethodKey)`.
|
||||
- `_testing/pytest.py:228` already imports
|
||||
`try_set_start_method` from `tractor.spawn._spawn` —
|
||||
keeping the `_spawn.py` path as the "core" module
|
||||
avoids breaking external refs during Phase A.
|
||||
|
||||
Phase A commit breakdown (each green via `/run-tests`):
|
||||
1. Drive `_testing/pytest.py` backend-tuple from
|
||||
`get_args(SpawnMethodKey)` — prep.
|
||||
2. Extract `trio_proc` → `spawn/_trio.py`; `_methods`
|
||||
registry imports it.
|
||||
3. Extract `mp_proc` → `spawn/_mp.py`; `_methods` imports
|
||||
it.
|
||||
4. Tidy `spawn/__init__.py` docstring for the new layout.
|
||||
|
||||
Workflow: Phase A runs in a new worktree via `/open-wkt
|
||||
spawn_modularize` (branch `wkt/spawn_modularize`) and
|
||||
lands as its own PR before Phase B/C returns to the
|
||||
current `claude_bun_opus4.7_subint_spawner_backend`
|
||||
branch.
|
||||
|
||||
## Files changed
|
||||
|
||||
None this turn — design/planning only. The user updated
|
||||
`ai/prompt-io/prompts/subints_spawner.md` themselves to
|
||||
complete the truncated sentence on line 32 (harness-flag
|
||||
reference).
|
||||
|
||||
Upcoming (Phase A, next turn):
|
||||
- `tractor/spawn/_spawn.py` — trimmed to "core"
|
||||
- `tractor/spawn/_trio.py` — new; holds `trio_proc()`
|
||||
- `tractor/spawn/_mp.py` — new; holds `mp_proc()`
|
||||
- `tractor/spawn/__init__.py` — docstring refresh
|
||||
- `tractor/_testing/pytest.py` — backend tuple from
|
||||
`get_args(SpawnMethodKey)`
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated. User completed the
|
||||
truncated sentence on line 32 of
|
||||
`ai/prompt-io/prompts/subints_spawner.md` themselves
|
||||
*before* this prompt-io log was written; that edit was
|
||||
the user's authoring, not an edit to AI output.
|
||||
|
|
@ -1,117 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
timestamp: 2026-04-17T03:49:18Z
|
||||
git_ref: 9703210
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
No code was produced in this turn — this is a design/planning
|
||||
exchange for subint spawner support per issue #379. Captured
|
||||
verbatim below are the two substantive AI responses, which
|
||||
together fix the phased plan (A: modularize, B: subint, C:
|
||||
harness) and the concrete Phase A file split before any
|
||||
implementation begins.
|
||||
|
||||
## Turn 1 — initial triage + gating questions
|
||||
|
||||
> Read: issue #379, current `spawn/_spawn.py` (847 lines),
|
||||
> and pyproject pins.
|
||||
>
|
||||
> ## Task summary
|
||||
> Design/plan/implement subint backend per #379, plus
|
||||
> modularize `spawn/_spawn.py` into per-backend submods, plus
|
||||
> harness flag for test parametrization.
|
||||
>
|
||||
> ## Key finding
|
||||
> `pyproject.toml:12` pins `requires-python = ">=3.12, <3.14"`
|
||||
> — but `concurrent.interpreters` / PEP 734 **only lands in
|
||||
> 3.14**. This is the load-bearing design question before
|
||||
> coding.
|
||||
>
|
||||
> ## Proposed phasing (incremental commits)
|
||||
> - **Phase A — modularize** `_spawn.py`: extract `trio_proc`
|
||||
> → `spawn/_trio.py`, `mp_proc` → `spawn/_mp.py`, keep
|
||||
> generic helpers (`hard_kill`/`soft_kill`/`new_proc`
|
||||
> dispatcher/`SpawnMethodKey`) in a slim core. Tests green
|
||||
> at each step.
|
||||
> - **Phase B — `_subint` backend**: extend `SpawnMethodKey`
|
||||
> with `'subint'`; reuse UDS transport by default; shm is
|
||||
> escape-hatch follow-up.
|
||||
> - **Phase C — harness**: add backend-parametrize flag, skip
|
||||
> subint when Python < 3.14.
|
||||
>
|
||||
> ## Gating questions before I start
|
||||
> 1. **Python pin**: bump `<3.15` and feature-gate subint
|
||||
> tests (a), require 3.14 outright (b), or backport shim
|
||||
> (c)? I'd recommend (a).
|
||||
> 2. **PR strategy**: one branch, commits per phase (current
|
||||
> `claude_bun_opus4.7_subint_spawner_backend`)? Or split
|
||||
> Phase A as its own PR first?
|
||||
> 3. **`fork()`-via-subint hack** from the issue's "Our own
|
||||
> thoughts" section — v1 or follow-up? I'd defer to
|
||||
> follow-up.
|
||||
> 4. **Prompt file line 32 is truncated** — what's the
|
||||
> existing harness flag name you had in mind ("using the
|
||||
> existing ___")?
|
||||
>
|
||||
> Which of these are already decided vs. open?
|
||||
|
||||
## User answers (for the raw record)
|
||||
|
||||
1. bump `<3.15`, feature-gate — option (a).
|
||||
2. split Phase A as its own PR first.
|
||||
3. defer `fork()`-via-subint hack.
|
||||
4. harness flag is `--spawn-backend` CLI → `start_method`
|
||||
fixture, per `tractor._testing._pytest`. Prompt file
|
||||
updated accordingly.
|
||||
|
||||
## Turn 2 — concrete Phase A plan
|
||||
|
||||
> Good — prompt file now ends cleanly; harness flag
|
||||
> confirmed as `--spawn-backend` CLI → `start_method`
|
||||
> fixture (via `_testing/pytest.py:193,355`). Also noted
|
||||
> `pytest.py:345-349` hardcodes valid backends as a string
|
||||
> tuple — `typing.get_args(SpawnMethodKey)` should drive
|
||||
> that list.
|
||||
>
|
||||
> ## Phase A — concrete plan (separate PR)
|
||||
>
|
||||
> **File split** in `tractor/spawn/`:
|
||||
> - `_spawn.py` stays as "core": keeps `SpawnMethodKey`,
|
||||
> `_spawn_method`, `proc_waiter`, `try_set_start_method`,
|
||||
> `exhaust_portal`, `cancel_on_completion`, `hard_kill`,
|
||||
> `soft_kill`, `new_proc` dispatcher, `_methods`
|
||||
> registry. Kept at this path so `_testing/pytest.py:228`
|
||||
> and other external imports don't break.
|
||||
> - `_trio.py` ← `trio_proc()` (lines 448–671, ~224 lines)
|
||||
> - `_mp.py` ← `mp_proc()` (lines 672–842, ~171 lines)
|
||||
> - `_methods` dict imports the two backend fns from the
|
||||
> new modules.
|
||||
>
|
||||
> **Commits** (each green via `/run-tests`):
|
||||
> 1. Drive `_testing/pytest.py` backend-tuple from
|
||||
> `get_args(SpawnMethodKey)` — tiny prep.
|
||||
> 2. Extract `trio_proc` → `spawn/_trio.py`; `_methods`
|
||||
> imports it.
|
||||
> 3. Extract `mp_proc` → `spawn/_mp.py`; `_methods`
|
||||
> imports it.
|
||||
> 4. Tidy `spawn/__init__.py` docstring to describe new
|
||||
> layout.
|
||||
>
|
||||
> No pyproject pin bump in Phase A (that's Phase B).
|
||||
>
|
||||
> **Workflow**: I'd run this in a new worktree via
|
||||
> `/open-wkt spawn_modularize` (branch
|
||||
> `wkt/spawn_modularize`), then open a PR before returning
|
||||
> to the current `claude_bun_opus4.7_subint_spawner_backend`
|
||||
> branch for Phase B/C.
|
||||
>
|
||||
> Green-light to `/open-wkt` and start Phase A?
|
||||
|
||||
## User final response
|
||||
|
||||
> all this first then yes, green light.
|
||||
|
||||
— log this `/prompt-io` entry first, then proceed with
|
||||
Phase A in a worktree.
|
||||
|
|
@ -1,86 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
session: subints-spawner-phase-a-impl
|
||||
timestamp: 2026-04-17T03:58:05Z
|
||||
git_ref: 61a73ba
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260417T035800Z_61a73ba_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Follow-up from the design session logged in
|
||||
`20260417T034918Z_9703210_prompt_io.md`: the user
|
||||
greenlit Phase A after `/prompt-io` tracking, so
|
||||
execute the Phase A "modularize `spawn._spawn`" plan
|
||||
end-to-end in a new worktree — four logical steps
|
||||
(pytest harness prep, `trio_proc` extraction,
|
||||
`mp_proc` extraction, docstring refresh), each green
|
||||
via `/run-tests`.
|
||||
|
||||
User directives during execution:
|
||||
- **Option 3** for the test cadence ("continue A.2–A.4
|
||||
first, then run the full suite once at the end of
|
||||
Phase A").
|
||||
- **One commit** for the whole phase ("can't we just
|
||||
commit the whole patch in one commit?") instead of
|
||||
the 3/4-commit split I initially proposed.
|
||||
- **Don't pre-draft** commit messages — wait for the
|
||||
user to invoke `/commit-msg` (captured as feedback
|
||||
memory `feedback_no_auto_draft_commit_msgs.md`).
|
||||
|
||||
## Response summary
|
||||
|
||||
Produced the cohesive Phase A modularization patch,
|
||||
landed as commit `61a73bae` (subject: `Mv
|
||||
trio_proc`/`mp_proc` to per-backend submods`). Five
|
||||
files changed, +565 / -418 lines.
|
||||
|
||||
Key pieces of the patch (generated by claude,
|
||||
reviewed by the human before commit):
|
||||
- `tractor/spawn/_trio.py` — **new**; receives
|
||||
`trio_proc()` verbatim from `_spawn.py`; imports
|
||||
cross-backend helpers back from `._spawn`.
|
||||
- `tractor/spawn/_mp.py` — **new**; receives
|
||||
`mp_proc()` verbatim; uses `from . import _spawn`
|
||||
for late-binding access to the mutable `_ctx` /
|
||||
`_spawn_method` globals (design decision made
|
||||
during impl, not the original plan).
|
||||
- `tractor/spawn/_spawn.py` — shrunk 847 → 448 LOC;
|
||||
import pruning; bottom-of-module late imports for
|
||||
`trio_proc` / `mp_proc` with a one-line comment
|
||||
explaining the circular-dep reason.
|
||||
- `tractor/spawn/__init__.py` — docstring refresh
|
||||
describing the new layout.
|
||||
- `tractor/_testing/pytest.py` — the valid-backend
|
||||
set now comes from `typing.get_args(SpawnMethodKey)`
|
||||
so future additions (`'subint'`) don't need harness
|
||||
edits.
|
||||
|
||||
## Files changed
|
||||
|
||||
See `git diff 61a73ba~1..61a73ba --stat`:
|
||||
|
||||
```
|
||||
tractor/_testing/pytest.py | 12 +-
|
||||
tractor/spawn/__init__.py | 31 +++-
|
||||
tractor/spawn/_mp.py | 235 ++++++++++++++++++++++++
|
||||
tractor/spawn/_spawn.py | 413 +-------------------------------
|
||||
tractor/spawn/_trio.py | 292 ++++++++++++++++++++++++++++
|
||||
5 files changed, 565 insertions(+), 418 deletions(-)
|
||||
```
|
||||
|
||||
Validation:
|
||||
- import probe + `_methods` wiring check — OK
|
||||
- spawn-relevant test subset — 37 passed, 1 skipped
|
||||
- full suite — 350 passed, 14 skipped, 7 xfailed, 1
|
||||
xpassed
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated by claude (no diff
|
||||
between `.claude/git_commit_msg_LATEST.md` and the
|
||||
committed body, as far as the assistant could
|
||||
observe).
|
||||
|
|
@ -1,138 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
timestamp: 2026-04-17T03:58:05Z
|
||||
git_ref: 61a73ba
|
||||
diff_cmd: git diff 61a73ba~1..61a73ba
|
||||
---
|
||||
|
||||
Code generated in this turn was committed verbatim as
|
||||
`61a73bae` ("Mv `trio_proc`/`mp_proc` to per-backend
|
||||
submods"). Per diff-ref mode, per-file code is captured
|
||||
via the pointers below, each followed by a prose
|
||||
summary of what the AI generated. Non-code output
|
||||
(sanity-check results, design rationale) is included
|
||||
verbatim.
|
||||
|
||||
## Per-file generated content
|
||||
|
||||
### `tractor/spawn/_trio.py` (new, 292 lines)
|
||||
|
||||
> `git diff 61a73ba~1..61a73ba -- tractor/spawn/_trio.py`
|
||||
|
||||
Pure lift-and-shift of `trio_proc()` out of
|
||||
`tractor/spawn/_spawn.py` (previously lines 448–670).
|
||||
Added AGPL header + module docstring describing the
|
||||
backend; imports include local `from ._spawn import
|
||||
cancel_on_completion, hard_kill, soft_kill` which
|
||||
creates the bottom-of-module late-import pattern in
|
||||
the core file to avoid a cycle. All call sites,
|
||||
log-format strings, and body logic are byte-identical
|
||||
to the originals — no semantic change.
|
||||
|
||||
### `tractor/spawn/_mp.py` (new, 235 lines)
|
||||
|
||||
> `git diff 61a73ba~1..61a73ba -- tractor/spawn/_mp.py`
|
||||
|
||||
Pure lift-and-shift of `mp_proc()` out of
|
||||
`tractor/spawn/_spawn.py` (previously lines 672–842).
|
||||
Same AGPL header convention. Key difference from
|
||||
`_trio.py`: uses `from . import _spawn` (module
|
||||
import, not from-import) for `_ctx` and
|
||||
`_spawn_method` references — these are mutated at
|
||||
runtime by `try_set_start_method()`, so late binding
|
||||
via `_spawn._ctx` / `_spawn._spawn_method` is required
|
||||
for correctness. Also imports `cancel_on_completion`,
|
||||
`soft_kill`, `proc_waiter` from `._spawn`.
|
||||
|
||||
### `tractor/spawn/_spawn.py` (modified, 847 → 448 LOC)
|
||||
|
||||
> `git diff 61a73ba~1..61a73ba -- tractor/spawn/_spawn.py`
|
||||
|
||||
- removed `trio_proc()` body (moved to `_trio.py`)
|
||||
- removed `mp_proc()` body (moved to `_mp.py`)
|
||||
- pruned imports now unused in core: `sys`,
|
||||
`is_root_process`, `current_actor`,
|
||||
`is_main_process`, `_mp_main`, `ActorFailure`,
|
||||
`pretty_struct`, `_pformat`
|
||||
- added bottom-of-file late imports
|
||||
`from ._trio import trio_proc` and
|
||||
`from ._mp import mp_proc` with a one-line
|
||||
comment explaining why (circular dep)
|
||||
- `_methods` dict unchanged structurally; still binds
|
||||
`'trio' → trio_proc`, `'mp_spawn' → mp_proc`,
|
||||
`'mp_forkserver' → mp_proc`
|
||||
|
||||
### `tractor/spawn/__init__.py` (modified)
|
||||
|
||||
> `git diff 61a73ba~1..61a73ba -- tractor/spawn/__init__.py`
|
||||
|
||||
Docstring rewrite only — no code. New text describes
|
||||
the per-backend `Layout` with `- ._spawn` (core),
|
||||
`- ._trio` (default), `- ._mp` (two mp variants), and
|
||||
reiterates the "no eager import" NOTE.
|
||||
|
||||
### `tractor/_testing/pytest.py` (modified)
|
||||
|
||||
> `git diff 61a73ba~1..61a73ba -- tractor/_testing/pytest.py`
|
||||
|
||||
Two-line logical change (plus a comment):
|
||||
- `from typing import` grows `get_args`
|
||||
- `pytest_generate_tests()` replaces the hardcoded
|
||||
`('mp_spawn','mp_forkserver','trio')` tuple with
|
||||
`get_args(SpawnMethodKey)` via lazy import
|
||||
`from tractor.spawn._spawn import SpawnMethodKey`
|
||||
inside the function body (matching the project's
|
||||
existing lazy-import convention for the spawn
|
||||
module)
|
||||
|
||||
## Non-code output (verbatim)
|
||||
|
||||
### Design rationale for the file split
|
||||
|
||||
From my Phase A plan turn (already captured in
|
||||
`20260417T034918Z_9703210_prompt_io.md`):
|
||||
- `_spawn.py` stays as the "core" rather than renamed
|
||||
to `_core.py` so external imports
|
||||
(`_testing/pytest.py:228` imports
|
||||
`try_set_start_method` from `tractor.spawn._spawn`)
|
||||
keep working without churn.
|
||||
- Per-backend extraction chosen over alternatives
|
||||
(e.g. splitting generic helpers further) because
|
||||
the immediate motivation is hosting a 3rd
|
||||
`_subint.py` sibling cleanly in Phase B.
|
||||
|
||||
### Sanity-check output (verbatim terminal excerpts)
|
||||
|
||||
Post-extraction import probe:
|
||||
```
|
||||
extraction OK
|
||||
_methods: {'trio': 'tractor.spawn._trio.trio_proc',
|
||||
'mp_spawn': 'tractor.spawn._mp.mp_proc',
|
||||
'mp_forkserver': 'tractor.spawn._mp.mp_proc'}
|
||||
```
|
||||
|
||||
Spawn-relevant test subset (`tests/test_local.py
|
||||
test_rpc.py test_spawning.py test_multi_program.py
|
||||
test_discovery.py`):
|
||||
```
|
||||
37 passed, 1 skipped, 14 warnings in 55.37s
|
||||
```
|
||||
|
||||
Full suite:
|
||||
```
|
||||
350 passed, 14 skipped, 7 xfailed, 1 xpassed,
|
||||
151 warnings in 437.73s (0:07:17)
|
||||
```
|
||||
|
||||
No regressions vs. `main`. One transient `-x`
|
||||
early-stop `ERROR` on
|
||||
`test_close_channel_explicit_remote_registrar[trio-True]`
|
||||
was flaky (passed solo, passed without `-x`), not
|
||||
caused by this refactor.
|
||||
|
||||
### Commit message
|
||||
|
||||
Also AI-drafted (via `/commit-msg`) — the 40-line
|
||||
message on commit `61a73bae` itself. Not reproduced
|
||||
here; see `git log -1 61a73bae`.
|
||||
|
|
@ -1,97 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
session: subints-spawner-phase-b1-impl
|
||||
timestamp: 2026-04-17T12:44:37Z
|
||||
git_ref: 5cd6df5
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260417T124437Z_5cd6df5_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Continuing the Phase B work from the design sessions
|
||||
logged in `20260417T034918Z_9703210_prompt_io.md` and
|
||||
the Phase A impl in `20260417T035800Z_61a73ba_...`.
|
||||
|
||||
User direction at this step:
|
||||
- "ok now let's continue with the subints part of the
|
||||
plan ya?" → kick off Phase B.
|
||||
- After I proposed a 4-sub-phase breakdown (B.1
|
||||
scaffolding, B.2 min-viable spawn, B.3 lifecycle,
|
||||
B.4 test enablement), user: "ok can we continue
|
||||
with B" → approving the phasing and design
|
||||
defaults (thread-per-subint, UDS default,
|
||||
code-string bootstrap via `interp.exec()`).
|
||||
- User switched the worktree to
|
||||
`subint_spawner_backend` (instead of opening a
|
||||
fresh worktree as I'd proposed): "i already
|
||||
switched branches in the wkt, you should work off
|
||||
`subint_spawner_backend` now".
|
||||
|
||||
Scope of this turn: **B.1 scaffolding only** —
|
||||
wire the `'subint'` name through the spawn-method
|
||||
registry + harness, add a feature-detected stub
|
||||
module, bump pyproject. No actual spawn flow.
|
||||
|
||||
## Response summary
|
||||
|
||||
Produced the Phase B.1 scaffolding patch, landed as
|
||||
commit `5cd6df58` (subject: `Add `'subint'` spawn
|
||||
backend scaffold (#379)`). Four files changed,
|
||||
+124 / -2 lines.
|
||||
|
||||
Key pieces (all generated by claude, reviewed by
|
||||
human before commit):
|
||||
- `tractor/spawn/_subint.py` — **new**; feature-
|
||||
detects `concurrent.interpreters`; `subint_proc()`
|
||||
stub raises `RuntimeError` on py<3.14 or
|
||||
`NotImplementedError` with issue-#379 URL on
|
||||
py≥3.14. Signature mirrors `trio_proc`/`mp_proc`
|
||||
so B.2 can drop the impl in without touching
|
||||
`_methods`.
|
||||
- `tractor/spawn/_spawn.py` — adds `'subint'` to
|
||||
`SpawnMethodKey`, grows a `case 'subint'` arm in
|
||||
`try_set_start_method()` with feature-gate, re-
|
||||
imports `sys` for the gate-error msg, adds late
|
||||
`from ._subint import subint_proc` import and
|
||||
`_methods` entry.
|
||||
- `tractor/_testing/pytest.py` — converts the
|
||||
gate-error into `pytest.UsageError` via a
|
||||
`try/except` around `try_set_start_method()` so
|
||||
`--spawn-backend=subint` on py<3.14 prints a
|
||||
clean banner instead of a traceback.
|
||||
- `pyproject.toml` — pin `requires-python` `<3.14`
|
||||
→ `<3.15`, add `3.14` trove classifier.
|
||||
|
||||
## Files changed
|
||||
|
||||
See `git diff 5cd6df5~1..5cd6df5 --stat`:
|
||||
|
||||
```
|
||||
pyproject.toml | 3 +-
|
||||
tractor/_testing/pytest.py | 8 +++-
|
||||
tractor/spawn/_spawn.py | 15 ++++++
|
||||
tractor/spawn/_subint.py | 100 +++++++++++++++++++++++++
|
||||
4 files changed, 124 insertions(+), 2 deletions(-)
|
||||
```
|
||||
|
||||
Validation on py3.13:
|
||||
- registry/gate probe — OK; `_has_subints = False`,
|
||||
`subint_proc()` raises `RuntimeError` as expected.
|
||||
- `try_set_start_method('subint')` — raises cleanly.
|
||||
- `pytest --spawn-backend=subint` — exits with
|
||||
`pytest.UsageError` banner.
|
||||
- Spawn-relevant test subset — 69 passed, 1 skipped.
|
||||
- Full suite on py3.13 deferred (scaffolding is
|
||||
no-op on <3.14).
|
||||
- Full-suite on py3.14 pending user setting up the
|
||||
venv.
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated by claude. `uv.lock`
|
||||
was intentionally left unstaged by the user (pin-
|
||||
bump regenerated cp314 wheel entries but they were
|
||||
deferred to a follow-up).
|
||||
|
|
@ -1,168 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
timestamp: 2026-04-17T12:44:37Z
|
||||
git_ref: 5cd6df5
|
||||
diff_cmd: git diff 5cd6df5~1..5cd6df5
|
||||
---
|
||||
|
||||
Code generated in this turn was committed verbatim as
|
||||
`5cd6df58` ("Add `'subint'` spawn backend scaffold
|
||||
(#379)"). Per diff-ref mode, per-file code is captured
|
||||
via the pointers below, each followed by a prose
|
||||
summary. Non-code output (sanity-check results,
|
||||
design rationale) is included verbatim.
|
||||
|
||||
## Per-file generated content
|
||||
|
||||
### `tractor/spawn/_subint.py` (new, 100 lines)
|
||||
|
||||
> `git diff 5cd6df5~1..5cd6df5 -- tractor/spawn/_subint.py`
|
||||
|
||||
New scaffolding module for the PEP 734 subinterpreter
|
||||
backend. Contents:
|
||||
- AGPL header + module docstring (describes backend
|
||||
intent, 3.14+ availability gate, and explicit
|
||||
"SCAFFOLDING STUB" status pointing at issue #379).
|
||||
- Top-level `try/except ImportError` wrapping
|
||||
`from concurrent import interpreters as
|
||||
_interpreters` → sets module-global
|
||||
`_has_subints: bool`. This lets the registry stay
|
||||
introspectable on py<3.14 while spawn-time still
|
||||
fails cleanly.
|
||||
- `subint_proc()` coroutine with signature matching
|
||||
`trio_proc`/`mp_proc` exactly (same param names,
|
||||
defaults, and `TaskStatus[Portal]` typing) —
|
||||
intentional so Phase B.2 can drop the impl in
|
||||
without touching `_methods` or changing call-site
|
||||
binding.
|
||||
- Body raises `RuntimeError` on py<3.14 (with
|
||||
`sys.version` printed) or `NotImplementedError`
|
||||
with issue-#379 URL on py≥3.14.
|
||||
|
||||
### `tractor/spawn/_spawn.py` (modified, +15 LOC)
|
||||
|
||||
> `git diff 5cd6df5~1..5cd6df5 -- tractor/spawn/_spawn.py`
|
||||
|
||||
- `import sys` re-added (pruned during Phase A, now
|
||||
needed again for the py-version string in the
|
||||
`'subint'` gate-error).
|
||||
- `SpawnMethodKey = Literal[...]` grows `'subint'` as
|
||||
the 4th member, with inline comment `# py3.14+ via
|
||||
`concurrent.interpreters` (PEP 734)`.
|
||||
- `try_set_start_method()` match-block grows a new
|
||||
`case 'subint':` arm that imports
|
||||
`from ._subint import _has_subints` lazily and
|
||||
raises `RuntimeError` with a multi-line gate msg
|
||||
if unavailable.
|
||||
- Bottom-of-module late-import section grows
|
||||
`from ._subint import subint_proc` alongside the
|
||||
existing `_trio` / `_mp` imports.
|
||||
- `_methods` dict grows `'subint': subint_proc`.
|
||||
|
||||
### `tractor/_testing/pytest.py` (modified, +8 LOC)
|
||||
|
||||
> `git diff 5cd6df5~1..5cd6df5 -- tractor/_testing/pytest.py`
|
||||
|
||||
`pytest_configure()` wraps the
|
||||
`try_set_start_method(backend)` call in a
|
||||
`try/except RuntimeError` that re-raises as
|
||||
`pytest.UsageError(str(err))`. Rationale: the gate
|
||||
error on py<3.14 is legitimately a configuration
|
||||
problem, not a test failure, so pytest's UsageError
|
||||
path (exit code 4) gives a clean single-line banner
|
||||
instead of a traceback.
|
||||
|
||||
### `pyproject.toml` (modified, +2 / -1)
|
||||
|
||||
> `git diff 5cd6df5~1..5cd6df5 -- pyproject.toml`
|
||||
|
||||
- `requires-python` pin relaxed `>=3.12, <3.14` →
|
||||
`>=3.12, <3.15` to admit 3.14 as a valid target.
|
||||
- Added `"Programming Language :: Python :: 3.14"`
|
||||
to the trove classifiers.
|
||||
|
||||
## Non-code output (verbatim)
|
||||
|
||||
### Design choices captured in the prior turn
|
||||
|
||||
(Previously confirmed by the user on April 17 in this
|
||||
session — relevant excerpts captured here for
|
||||
provenance since they drove the shape of the impl):
|
||||
|
||||
1. **Python pin**: `<3.15` + feature-gate the subint
|
||||
backend at spawn time — user answered option (a)
|
||||
from the design triage.
|
||||
2. **Deferred `fork()`-via-subint hack** from issue
|
||||
#379's "Our own thoughts" section.
|
||||
3. **Phase B phasing**: user approved the B.1 / B.2 /
|
||||
B.3 / B.4 breakdown — this commit is strictly B.1
|
||||
(scaffolding only, no spawn-flow impl).
|
||||
4. **Option (B) worktree strategy**: new worktree
|
||||
branched from `wkt/spawn_modularize`. *(Amended by
|
||||
user at runtime: user switched the existing
|
||||
`spawn_modularize` worktree to the
|
||||
`subint_spawner_backend` branch instead.)*
|
||||
|
||||
### Sanity-check output (verbatim terminal excerpts)
|
||||
|
||||
Registry / feature-gate verification on py3.13:
|
||||
```
|
||||
SpawnMethodKey values: ('trio', 'mp_spawn',
|
||||
'mp_forkserver', 'subint')
|
||||
_methods keys: ['trio', 'mp_spawn',
|
||||
'mp_forkserver', 'subint']
|
||||
_has_subints: False (py version: (3, 13) )
|
||||
[expected] RuntimeError: The 'subint' spawn backend
|
||||
requires Python 3.14+ (stdlib
|
||||
`concurrent.interpreters`, PEP 734).
|
||||
```
|
||||
|
||||
`try_set_start_method('subint')` gate on py3.13:
|
||||
```
|
||||
[expected] RuntimeError: Spawn method 'subint'
|
||||
requires Python 3.14+ (stdlib
|
||||
`concurrent.interpreters`, PEP 734).
|
||||
```
|
||||
|
||||
Pytest `--spawn-backend=subint` on py3.13 (the new
|
||||
UsageError wrapper kicking in):
|
||||
```
|
||||
ERROR: Spawn method 'subint' requires Python 3.14+
|
||||
(stdlib `concurrent.interpreters`, PEP 734).
|
||||
Current runtime: 3.13.11 (main, Dec 5 2025,
|
||||
16:06:33) [GCC 15.2.0]
|
||||
```
|
||||
|
||||
Collection probe: `404 tests collected in 0.18s`
|
||||
(no import errors from the new module).
|
||||
|
||||
Spawn-relevant test subset (`tests/test_local.py
|
||||
test_rpc.py test_spawning.py test_multi_program.py
|
||||
tests/discovery/`):
|
||||
```
|
||||
69 passed, 1 skipped, 10 warnings in 61.38s
|
||||
```
|
||||
|
||||
Full suite was **not** run on py3.13 for this commit
|
||||
— the scaffolding is no-op on <3.14 and full-suite
|
||||
validation under py3.14 is pending that venv being
|
||||
set up by the user.
|
||||
|
||||
### Commit message
|
||||
|
||||
Also AI-drafted (via `/commit-msg`, with the prose
|
||||
rewrapped through `/home/goodboy/.claude/skills/pr-msg/
|
||||
scripts/rewrap.py --width 67`) — the 33-line message
|
||||
on commit `5cd6df58` itself. Not reproduced here; see
|
||||
`git log -1 5cd6df58`.
|
||||
|
||||
### Known follow-ups flagged to user
|
||||
|
||||
- **`uv.lock` deferred**: pin-bump regenerated cp314
|
||||
wheel entries in `uv.lock`, but the user chose to
|
||||
not stage `uv.lock` for this commit. Warned
|
||||
explicitly.
|
||||
- **Phase B.2 needs py3.14 venv** — running the
|
||||
actual subint impl requires it; user said they'd
|
||||
set it up separately.
|
||||
|
|
@ -1,117 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
session: subints-phase-b2-destroy-race-fix
|
||||
timestamp: 2026-04-18T04:25:26Z
|
||||
git_ref: 26fb820
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260418T042526Z_26fb820_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Follow-up to Phase B.2 (`5cd6df58`) after the user
|
||||
observed intermittent mid-suite hangs when running
|
||||
the tractor test suite under `--spawn-backend=subint`
|
||||
on py3.14. The specific sequence of prompts over
|
||||
several turns:
|
||||
|
||||
1. User pointed at the `test_context_stream_semantics.py`
|
||||
suite as the first thing to make run clean under
|
||||
`--spawn-backend=subint`.
|
||||
2. After a series of `timeout`-terminated runs that
|
||||
gave no diagnostic info, user nudged me to stop
|
||||
relying on `timeout` and get actual runtime
|
||||
diagnostics ("the suite hangs indefinitely, so i
|
||||
don't think this `timeout 30` is helping you at
|
||||
all.."). Switched to
|
||||
`faulthandler.dump_traceback_later(...)` and a
|
||||
resource-tracker fixture to rule out leaks.
|
||||
3. Captured a stack pinning the hang on
|
||||
`_interpreters.destroy(interp_id)` in the subint
|
||||
teardown finally block.
|
||||
4. Proposed dedicated-OS-thread fix. User greenlit.
|
||||
5. Implemented + verified on-worktree; user needed
|
||||
to be pointed at the *worktree*'s `./py313` venv
|
||||
because bare `pytest` was picking up the main
|
||||
repo's venv (running un-patched `_subint.py`) and
|
||||
still hanging.
|
||||
|
||||
Running theme over the whole exchange: this patch
|
||||
only closes the *destroy race*. The user and I also
|
||||
traced through the deeper cancellation story — SIGINT
|
||||
can't reach subints, legacy-mode shares the GIL,
|
||||
portal-cancel dies when the IPC channel is already
|
||||
broken — and agreed the next step is a bounded
|
||||
hard-kill in `subint_proc`'s teardown plus a
|
||||
dedicated cancellation test suite. Those land as
|
||||
separate commits.
|
||||
|
||||
## Response summary
|
||||
|
||||
Produced the `tractor/spawn/_subint.py` patch landed
|
||||
as commit `26fb8206` ("Fix subint destroy race via
|
||||
dedicated OS thread"). One file, +110/-84 LOC.
|
||||
|
||||
Mechanism: swap `trio.to_thread.run_sync(_interpreters
|
||||
.exec, ...)` for a plain `threading.Thread(target=...
|
||||
, daemon=False)`. The trio thread cache recycles
|
||||
workers — so the OS thread that ran `_interpreters
|
||||
.exec()` remained alive in the cache holding a
|
||||
stale subint tstate, blocking
|
||||
`_interpreters.destroy()` in the finally indefinitely.
|
||||
A dedicated one-shot thread exits naturally after
|
||||
the sync target returns, releasing tstate and
|
||||
unblocking destroy.
|
||||
|
||||
Coordination across the trio↔thread boundary:
|
||||
- `trio.lowlevel.current_trio_token()` captured at
|
||||
`subint_proc` entry
|
||||
- driver thread signals `subint_exited.set()` back
|
||||
to parent trio via `trio.from_thread.run_sync(...,
|
||||
trio_token=token)` (synchronous from the thread's
|
||||
POV; the call returns after trio has run `.set()`)
|
||||
- `trio.RunFinishedError` swallowed in that path for
|
||||
the process-teardown case where parent trio already
|
||||
exited
|
||||
- teardown `finally` off-loads the sync
|
||||
`driver_thread.join()` via `to_thread.run_sync` (a
|
||||
cache thread carries no subint tstate — safe)
|
||||
|
||||
## Files changed
|
||||
|
||||
See `git diff 26fb820~1..26fb820 --stat`:
|
||||
|
||||
```
|
||||
tractor/spawn/_subint.py | 194 +++++++++++++++++++------------
|
||||
1 file changed, 110 insertions(+), 84 deletions(-)
|
||||
```
|
||||
|
||||
Validation:
|
||||
- `test_parent_cancels[chk_ctx_result_before_exit=True-
|
||||
cancel_method=ctx-child_returns_early=False]`
|
||||
(the specific test that was hanging for the user)
|
||||
— passed in 1.06s.
|
||||
- Full `tests/test_context_stream_semantics.py` under
|
||||
subint — 61 passed in 100.35s (clean-cache re-run:
|
||||
100.82s).
|
||||
- Trio backend regression subset — 69 passed / 1
|
||||
skipped / 89.19s — no regressions from this change.
|
||||
|
||||
## Files changed
|
||||
|
||||
Beyond the `_subint.py` patch, the raw log also
|
||||
records the cancellation-semantics research that
|
||||
spanned this conversation but did not ship as code
|
||||
in *this* commit. Preserving it inline under "Non-
|
||||
code output" because it directly informs the
|
||||
Phase B.3 hard-kill impl that will follow (and any
|
||||
upstream CPython bug reports we end up filing).
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated. The commit message
|
||||
itself was also AI-drafted via `/commit-msg` and
|
||||
rewrapped via the project's `rewrap.py --width 67`
|
||||
tooling; user landed it without edits.
|
||||
|
|
@ -1,220 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
timestamp: 2026-04-18T04:25:26Z
|
||||
git_ref: 26fb820
|
||||
diff_cmd: git diff 26fb820~1..26fb820
|
||||
---
|
||||
|
||||
Code generated in this turn was committed as `26fb8206`
|
||||
("Fix subint destroy race via dedicated OS thread").
|
||||
Per diff-ref mode, per-file code is captured via the
|
||||
pointer below; the rest is verbatim non-code output
|
||||
— test-session diagnostics, CPython-internals
|
||||
reasoning, and the design synthesis that's worth
|
||||
keeping alongside the code since it informs the
|
||||
still-pending Phase B.3 hard-kill work.
|
||||
|
||||
## Per-file generated content
|
||||
|
||||
### `tractor/spawn/_subint.py` (modified, +110/-84 LOC)
|
||||
|
||||
> `git diff 26fb820~1..26fb820 -- tractor/spawn/_subint.py`
|
||||
|
||||
Rewrites the subint driver-thread strategy:
|
||||
- replaces `trio.to_thread.run_sync(_interpreters.exec, ...)`
|
||||
with a plain `threading.Thread(target=_subint_target,
|
||||
daemon=False)` so the OS thread truly exits after
|
||||
`_interpreters.exec()` returns
|
||||
- captures a `trio.lowlevel.current_trio_token()` at
|
||||
`subint_proc` entry; the driver thread signals
|
||||
completion back via `trio.from_thread.run_sync(
|
||||
subint_exited.set, trio_token=...)`
|
||||
- swallows `trio.RunFinishedError` in the signal path
|
||||
for the case where the parent trio loop has already
|
||||
exited (process teardown)
|
||||
- in the teardown `finally` off-loads the sync
|
||||
`driver_thread.join()` call to `trio.to_thread.run_sync`
|
||||
(a cache thread w/ no subint tstate — so no cache
|
||||
conflict) to wait for the driver thread to fully
|
||||
exit before calling `_interpreters.destroy()`
|
||||
|
||||
## Non-code output (verbatim) — the CPython-internals research
|
||||
|
||||
### What went wrong before this commit
|
||||
|
||||
Under `--spawn-backend=subint` on py3.14, most single
|
||||
tests passed but longer runs hung intermittently. The
|
||||
position of the hang moved between runs (test #22 on
|
||||
one run, test #53 on another) suggesting a timing-
|
||||
dependent race rather than a deterministic bug.
|
||||
|
||||
`faulthandler.dump_traceback_later()` eventually
|
||||
caught a stack with the main thread blocked in
|
||||
`_interpreters.destroy(interp_id)` at `_subint.py:293`.
|
||||
Only 2 threads were alive:
|
||||
- main thread waiting in `_interpreters.destroy()`
|
||||
- one idle trio thread-cache worker in
|
||||
`trio._core._thread_cache._work`
|
||||
|
||||
No subint was still running (`_interpreters.list_all()`
|
||||
showed only the main interp). A resource-tracker
|
||||
pytest fixture confirmed threads/subints did NOT
|
||||
accumulate across tests — this was not a leak but a
|
||||
specific "destroy blocks on cached thread w/ stale
|
||||
tstate" race.
|
||||
|
||||
### Why the race exists
|
||||
|
||||
`trio.to_thread.run_sync` uses a thread *cache* to
|
||||
avoid OS-thread creation overhead. When the sync
|
||||
callable returns, the OS thread is NOT terminated —
|
||||
it's parked in `_thread_cache._work` waiting for the
|
||||
next job. CPython's subinterpreter implementation
|
||||
attaches a **tstate** (thread-state object) to each
|
||||
OS thread that ever entered a subint via
|
||||
`_interpreters.exec()`. That tstate is released
|
||||
lazily — either when the thread picks up a new job
|
||||
(which re-attaches a new tstate, evicting the old
|
||||
one) or when the thread truly exits.
|
||||
|
||||
`_interpreters.destroy(interp_id)` waits for *all*
|
||||
tstates associated w/ that subint to be released
|
||||
before it can proceed. If the cached worker is idle
|
||||
holding the stale tstate, destroy blocks indefinitely.
|
||||
Whether the race manifests depends on timing — if
|
||||
the cached thread happens to pick up another job
|
||||
quickly, destroy unblocks; if it sits idle, we hang.
|
||||
|
||||
### Why a dedicated `threading.Thread` fixes it
|
||||
|
||||
A plain `threading.Thread(target=_subint_target,
|
||||
daemon=False)` runs its target once and exits. When
|
||||
the target returns, OS-thread teardown (`_bootstrap_inner`
|
||||
→ `_bootstrap`) fires and CPython releases the
|
||||
tstate for that thread. `_interpreters.destroy()`
|
||||
then has no blocker.
|
||||
|
||||
### Diagnostic tactics that actually helped
|
||||
|
||||
1. `faulthandler.dump_traceback_later(n, repeat=False,
|
||||
file=open(path, 'w'))` for captured stack dumps on
|
||||
hang. Critically, pipe to a `file=` not stderr —
|
||||
pytest captures stderr weirdly and the dump is
|
||||
easy to miss.
|
||||
2. A resource-tracker autouse fixture printing
|
||||
per-test `threading.active_count()` +
|
||||
`len(_interpreters.list_all())` deltas → ruled out
|
||||
leak-accumulation theories quickly.
|
||||
3. Running the hanging test *solo* vs in-suite —
|
||||
when solo passes but in-suite hangs, you know
|
||||
it's a cross-test state-transfer bug rather than
|
||||
a test-internal bug.
|
||||
|
||||
### Design synthesis — SIGINT + subints + SC
|
||||
|
||||
The user and I walked through the cancellation
|
||||
semantics of PEP 684/734 subinterpreters in detail.
|
||||
Key findings we want to preserve:
|
||||
|
||||
**Signal delivery in subints (stdlib limitation).**
|
||||
CPython's signal machinery only delivers signals
|
||||
(SIGINT included) to the *main thread of the main
|
||||
interpreter*. Subints cannot install signal handlers
|
||||
that will ever fire. This is an intentional design
|
||||
choice in PEP 684 and not expected to change. For
|
||||
tractor's subint actors, this means:
|
||||
|
||||
- Ctrl-C never reaches a subint directly.
|
||||
- `trio.run()` running on a worker thread (as we do
|
||||
for subints) already skips SIGINT handler install
|
||||
because `signal.signal()` raises on non-main
|
||||
threads.
|
||||
- The only cancellation surface into a subint is
|
||||
our IPC `Portal.cancel_actor()`.
|
||||
|
||||
**Legacy-mode subints share the main GIL** (which
|
||||
our impl uses since `msgspec` lacks PEP 684 support
|
||||
per `jcrist/msgspec#563`). This means a stuck subint
|
||||
thread can starve the parent's trio loop during
|
||||
cancellation — the parent can't even *start* its
|
||||
teardown handling until the subint yields the GIL.
|
||||
|
||||
**Failure modes identified for Phase B.3 audit:**
|
||||
|
||||
1. Portal cancel lands cleanly → subint unwinds →
|
||||
thread exits → destroy succeeds. (Happy path.)
|
||||
2. IPC channel is already broken when we try to
|
||||
send cancel (e.g., `test_ipc_channel_break_*`)
|
||||
→ cancel raises `BrokenResourceError` → subint
|
||||
keeps running unaware → parent hangs waiting for
|
||||
`subint_exited`. This is what breaks
|
||||
`test_advanced_faults.py` under subint.
|
||||
3. Subint is stuck in non-checkpointing Python code
|
||||
→ portal-cancel msg queued but never processed.
|
||||
4. Subint is in a shielded cancel scope when cancel
|
||||
arrives → delay until shield exits.
|
||||
|
||||
**Current teardown has a shield-bug too:**
|
||||
`trio.CancelScope(shield=True)` wrapping the `finally`
|
||||
block absorbs Ctrl-C, so even when the user tries
|
||||
to break out they can't. This is the reason
|
||||
`test_ipc_channel_break_during_stream[break_parent-...
|
||||
no_msgstream_aclose]` locks up unkillable.
|
||||
|
||||
**B.3 hard-kill fix plan (next commit):**
|
||||
|
||||
1. Bound `driver_thread.join()` with
|
||||
`trio.move_on_after(HARD_KILL_TIMEOUT)`.
|
||||
2. If it times out, log a warning naming the
|
||||
`interp_id` and switch the driver thread to
|
||||
`daemon=True` mode (not actually possible after
|
||||
start — so instead create as daemon=True upfront
|
||||
and accept the tradeoff of proc-exit not waiting
|
||||
for a stuck subint).
|
||||
3. Best-effort `_interpreters.destroy()`; catch the
|
||||
`InterpreterError` if the subint is still running.
|
||||
4. Document that the leak is real and the only
|
||||
escape hatch we have without upstream cooperation.
|
||||
|
||||
**Test plan for Phase B.3:**
|
||||
|
||||
New `tests/test_subint_cancellation.py` covering:
|
||||
- SIGINT at spawn
|
||||
- SIGINT mid-portal-RPC
|
||||
- SIGINT during shielded section in subint
|
||||
- Dead-channel cancel (mirror of `test_ipc_channel_
|
||||
break_during_stream` minimized)
|
||||
- Non-checkpointing subint (tight `while True` in
|
||||
user code)
|
||||
- Per-test `pytest-timeout`-style bounds so the
|
||||
tests visibly fail instead of wedging the runner
|
||||
|
||||
### Sanity-check output (verbatim terminal excerpts)
|
||||
|
||||
Post-fix single-test validation:
|
||||
```
|
||||
1 passed, 1 warning in 1.06s
|
||||
```
|
||||
(same test that was hanging pre-fix:
|
||||
`test_parent_cancels[...cancel_method=ctx-...False]`)
|
||||
|
||||
Full `tests/test_context_stream_semantics.py`
|
||||
under subint:
|
||||
```
|
||||
61 passed, 1 warning in 100.35s (0:01:40)
|
||||
```
|
||||
and a clean-cache re-run:
|
||||
```
|
||||
61 passed, 1 warning in 100.82s (0:01:40)
|
||||
```
|
||||
|
||||
No regressions on trio backend (same subset):
|
||||
```
|
||||
69 passed, 1 skipped, 3 warnings in 89.19s
|
||||
```
|
||||
|
||||
### Commit msg
|
||||
|
||||
Also AI-drafted via `/commit-msg` + `rewrap.py
|
||||
--width 67`. See `git log -1 26fb820`.
|
||||
|
|
@ -1,111 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
session: subint-phase-b-hang-classification
|
||||
timestamp: 2026-04-20T19:27:39Z
|
||||
git_ref: HEAD (pre-commit; on branch `subint_spawner_backend`)
|
||||
scope: docs
|
||||
substantive: true
|
||||
raw_file: 20260420T192739Z_5e8cd8b2_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Collab between user (`goodboy`) and `claude` to
|
||||
disambiguate two distinct hang modes hit during
|
||||
Phase B subint-spawn-backend bringup (issue #379).
|
||||
The user ran the failing suites, captured `strace`
|
||||
evidence on hung pytest pids, and set the framing:
|
||||
|
||||
> "These cannot be the same bug. Different fix
|
||||
> paths. Write them up separately or we'll keep
|
||||
> conflating them."
|
||||
|
||||
Follow-on asks:
|
||||
- Cross-link each doc from its reproducer test so
|
||||
a future triager lands on the analysis without
|
||||
needing to grep `ai/conc-anal/`.
|
||||
- On `test_stale_entry_is_deleted`: arm
|
||||
`dump_on_hang(seconds=20, path=...)` so a
|
||||
regression captures a stack dump (pytest stderr
|
||||
capture otherwise eats `faulthandler` output).
|
||||
- Keep `test_stale_entry_is_deleted` un-`skip`ped
|
||||
so the dump file remains inspectable.
|
||||
|
||||
Defer the actual fix for the class-2 hang
|
||||
(Ctrl-C-able, tractor-side bug) to a later commit
|
||||
per the incremental-commits preference.
|
||||
|
||||
## Response summary
|
||||
|
||||
Wrote two new `ai/conc-anal/` classification docs
|
||||
and cross-linked them from the reproducer tests:
|
||||
|
||||
1. **`subint_sigint_starvation_issue.md`** —
|
||||
class 1: abandoned-legacy-subint thread +
|
||||
shared GIL starves main trio loop →
|
||||
signal-wakeup-fd pipe fills → SIGINT silently
|
||||
dropped (`write() = EAGAIN`). Pytest process
|
||||
un-Ctrl-C-able. Structurally a CPython limit;
|
||||
blocked on `msgspec` PEP 684 support
|
||||
(jcrist/msgspec#563). Reproducer:
|
||||
`test_stale_entry_is_deleted[subint]`.
|
||||
|
||||
2. **`subint_cancel_delivery_hang_issue.md`** —
|
||||
class 2: parent-side trio task parks on an
|
||||
orphaned IPC channel after subint teardown;
|
||||
no clean EOF delivered to waiting receiver.
|
||||
Ctrl-C-able (main trio loop iterating fine).
|
||||
OUR bug to fix. Candidate fix: explicit
|
||||
parent-side channel abort in `subint_proc`'s
|
||||
hard-kill teardown. Reproducer:
|
||||
`test_subint_non_checkpointing_child`.
|
||||
|
||||
Test-side cross-links:
|
||||
- `tests/discovery/test_registrar.py`:
|
||||
`test_stale_entry_is_deleted` → `trio.run(main)`
|
||||
wrapped in `dump_on_hang(seconds=20,
|
||||
path=<per-method-tmp>)`; long inline comment
|
||||
summarizes `strace` evidence + root-cause chain
|
||||
and points at both docs.
|
||||
- `tests/test_subint_cancellation.py`:
|
||||
`test_subint_non_checkpointing_child` docstring
|
||||
extended with "KNOWN ISSUE (Ctrl-C-able hang)"
|
||||
section pointing at the class-2 doc + noting
|
||||
the class-1 doc is NOT what this test hits.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `ai/conc-anal/subint_sigint_starvation_issue.md`
|
||||
— new, 205 LOC
|
||||
- `ai/conc-anal/subint_cancel_delivery_hang_issue.md`
|
||||
— new, 161 LOC
|
||||
- `tests/discovery/test_registrar.py` — +52/-1
|
||||
(arm `dump_on_hang`, inline-comment cross-link)
|
||||
- `tests/test_subint_cancellation.py` — +26
|
||||
(docstring "KNOWN ISSUE" block)
|
||||
|
||||
## Human edits
|
||||
|
||||
Substantive collab — prose was jointly iterated:
|
||||
|
||||
- User framed the two-doc split, set the
|
||||
classification criteria (Ctrl-C-able vs not),
|
||||
and provided the `strace` evidence.
|
||||
- User decided to keep `test_stale_entry_is_deleted`
|
||||
un-`skip`ped (my initial suggestion was
|
||||
`pytestmark.skipif(spawn_backend=='subint')`).
|
||||
- User chose the candidate fix ordering for
|
||||
class 2 and marked "explicit parent-side channel
|
||||
abort" as the surgical preferred fix.
|
||||
- User picked the file naming convention
|
||||
(`subint_<hang-shape>_issue.md`) over my initial
|
||||
`hang_class_{1,2}.md`.
|
||||
- Assistant drafted the prose, aggregated prior-
|
||||
session root-cause findings from Phase B.2/B.3
|
||||
bringup, and wrote the test-side cross-linking
|
||||
comments.
|
||||
|
||||
No further mechanical edits expected before
|
||||
commit; user may still rewrap via
|
||||
`scripts/rewrap.py` if preferred.
|
||||
|
|
@ -1,198 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
timestamp: 2026-04-20T19:27:39Z
|
||||
git_ref: HEAD (pre-commit; will land on branch `subint_spawner_backend`)
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
Collab between `goodboy` (user) and `claude` (this
|
||||
assistant) spanning multiple test-run iterations on
|
||||
branch `subint_spawner_backend`. The user ran the
|
||||
failing suites, captured `strace` evidence on the
|
||||
hung pytest pids, and set the direction ("these are
|
||||
two different hangs — write them up separately so
|
||||
we don't re-confuse ourselves later"). The assistant
|
||||
aggregated prior-session findings (Phase B.2/B.3
|
||||
bringup) into two classification docs + test-side
|
||||
cross-links. All prose was jointly iterated; the
|
||||
user had final say on framing and decided which
|
||||
candidate fix directions to list.
|
||||
|
||||
## Per-file generated content
|
||||
|
||||
### `ai/conc-anal/subint_sigint_starvation_issue.md` (new, 205 LOC)
|
||||
|
||||
> `git diff HEAD~1..HEAD -- ai/conc-anal/subint_sigint_starvation_issue.md`
|
||||
|
||||
Writes up the "abandoned-legacy-subint thread wedges
|
||||
the parent trio loop" class. Key sections:
|
||||
|
||||
- **Symptom** — `test_stale_entry_is_deleted[subint]`
|
||||
hangs indefinitely AND is un-Ctrl-C-able.
|
||||
- **Evidence** — annotated `strace` excerpt showing
|
||||
SIGINT delivered to pytest, C-level signal handler
|
||||
tries to write to the signal-wakeup-fd pipe, gets
|
||||
`write() = -1 EAGAIN (Resource temporarily
|
||||
unavailable)`. Pipe is full because main trio loop
|
||||
isn't iterating often enough to drain it.
|
||||
- **Root-cause chain** — our hard-kill abandons the
|
||||
`daemon=True` driver OS thread after
|
||||
`_HARD_KILL_TIMEOUT`; the subint *inside* that
|
||||
thread is still running `trio.run()`;
|
||||
`_interpreters.destroy()` cannot force-stop a
|
||||
running subint (raises `InterpreterError`); legacy
|
||||
subints share the main GIL → abandoned subint
|
||||
starves main trio loop → wakeup-fd fills → SIGINT
|
||||
silently dropped.
|
||||
- **Why it's structurally a CPython limit** — no
|
||||
public force-destroy primitive for a running
|
||||
subint; the only escape is per-interpreter GIL
|
||||
isolation, gated on msgspec PEP 684 adoption
|
||||
(jcrist/msgspec#563).
|
||||
- **Current escape hatch** — harness-side SIGINT
|
||||
loop in the `daemon` fixture teardown that kills
|
||||
the bg registrar subproc, eventually unblocking
|
||||
a parent-side recv enough for the main loop to
|
||||
drain the wakeup pipe.
|
||||
|
||||
### `ai/conc-anal/subint_cancel_delivery_hang_issue.md` (new, 161 LOC)
|
||||
|
||||
> `git diff HEAD~1..HEAD -- ai/conc-anal/subint_cancel_delivery_hang_issue.md`
|
||||
|
||||
Writes up the *sibling* hang class — same subint
|
||||
backend, distinct root cause:
|
||||
|
||||
- **TL;DR** — Ctrl-C-able, so NOT the SIGINT-
|
||||
starvation class; main trio loop iterates fine;
|
||||
ours to fix.
|
||||
- **Symptom** — `test_subint_non_checkpointing_child`
|
||||
hangs past the expected `_HARD_KILL_TIMEOUT`
|
||||
budget even after the subint is torn down.
|
||||
- **Diagnosis** — a parent-side trio task (likely
|
||||
a `chan.recv()` in `process_messages`) parks on
|
||||
an orphaned IPC channel; channel was torn down
|
||||
without emitting a clean EOF /
|
||||
`BrokenResourceError` to the waiting receiver.
|
||||
- **Candidate fix directions** — listed in rough
|
||||
order of preference:
|
||||
1. Explicit parent-side channel abort in
|
||||
`subint_proc`'s hard-kill teardown (surgical;
|
||||
most likely).
|
||||
2. Audit `process_messages` to add a timeout or
|
||||
cancel-scope protection that catches the
|
||||
orphaned-recv state.
|
||||
3. Wrap subint IPC channel construction in a
|
||||
sentinel that can force-close from the parent
|
||||
side regardless of subint liveness.
|
||||
|
||||
### `tests/discovery/test_registrar.py` (modified, +52/-1 LOC)
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/discovery/test_registrar.py`
|
||||
|
||||
Wraps the `trio.run(main)` call at the bottom of
|
||||
`test_stale_entry_is_deleted` in
|
||||
`dump_on_hang(seconds=20, path=<per-method-tmp>)`.
|
||||
Adds a long inline comment that:
|
||||
- Enumerates variant-by-variant status
|
||||
(`[trio]`/`[mp_*]` = clean; `[subint]` = hangs
|
||||
+ un-Ctrl-C-able)
|
||||
- Summarizes the `strace` evidence and root-cause
|
||||
chain inline (so a future reader hitting this
|
||||
test doesn't need to cross-ref the doc to
|
||||
understand the hang shape)
|
||||
- Points at
|
||||
`ai/conc-anal/subint_sigint_starvation_issue.md`
|
||||
for full analysis
|
||||
- Cross-links to the *sibling*
|
||||
`subint_cancel_delivery_hang_issue.md` so
|
||||
readers can tell the two classes apart
|
||||
- Explains why it's kept un-`skip`ped: the dump
|
||||
file is useful if the hang ever returns after
|
||||
a refactor. pytest stderr capture would
|
||||
otherwise eat `faulthandler` output, hence the
|
||||
file path.
|
||||
|
||||
### `tests/test_subint_cancellation.py` (modified, +26 LOC)
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_subint_cancellation.py`
|
||||
|
||||
Extends the docstring of
|
||||
`test_subint_non_checkpointing_child` with a
|
||||
"KNOWN ISSUE (Ctrl-C-able hang)" block:
|
||||
- Describes the current hang: parent-side orphaned
|
||||
IPC recv after hard-kill; distinct from the
|
||||
SIGINT-starvation sibling class.
|
||||
- Cites `strace` distinguishing signal: wakeup-fd
|
||||
`write() = 1` (not `EAGAIN`) — i.e. main loop
|
||||
iterating.
|
||||
- Points at
|
||||
`ai/conc-anal/subint_cancel_delivery_hang_issue.md`
|
||||
for full analysis + candidate fix directions.
|
||||
- Clarifies that the *other* sibling doc
|
||||
(SIGINT-starvation) is NOT what this test hits.
|
||||
|
||||
## Non-code output
|
||||
|
||||
### Classification reasoning (why two docs, not one)
|
||||
|
||||
The user and I converged on the two-doc split after
|
||||
running the suites and noticing two *qualitatively
|
||||
different* hang symptoms:
|
||||
|
||||
1. `test_stale_entry_is_deleted[subint]` — pytest
|
||||
process un-Ctrl-C-able. Ctrl-C at the terminal
|
||||
does nothing. Must kill-9 from another shell.
|
||||
2. `test_subint_non_checkpointing_child` — pytest
|
||||
process Ctrl-C-able. One Ctrl-C at the prompt
|
||||
unblocks cleanly and the test reports a hang
|
||||
via pytest-timeout.
|
||||
|
||||
From the user: "These cannot be the same bug.
|
||||
Different fix paths. Write them up separately or
|
||||
we'll keep conflating them."
|
||||
|
||||
`strace` on the `[subint]` hang gave the decisive
|
||||
signal for the first class:
|
||||
|
||||
```
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(5, "\2", 1) = -1 EAGAIN (Resource temporarily unavailable)
|
||||
```
|
||||
|
||||
fd 5 is Python's signal-wakeup-fd pipe. `EAGAIN`
|
||||
on a `write()` of 1 byte to a pipe means the pipe
|
||||
buffer is full → reader side (main Python thread
|
||||
inside `trio.run()`) isn't consuming. That's the
|
||||
GIL-hostage signature.
|
||||
|
||||
The second class's `strace` showed `write(5, "\2",
|
||||
1) = 1` — clean drain — so the main trio loop was
|
||||
iterating and the hang had to be on the application
|
||||
side of things, not the kernel-↔-Python signal
|
||||
boundary.
|
||||
|
||||
### Why the candidate fix for class 2 is "explicit parent-side channel abort"
|
||||
|
||||
The second hang class has the trio loop alive. A
|
||||
parked `chan.recv()` that will never get bytes is
|
||||
fundamentally a tractor-side resource-lifetime bug
|
||||
— the IPC channel was torn down (subint destroyed)
|
||||
but no one explicitly raised
|
||||
`BrokenResourceError` at the parent-side receiver.
|
||||
The `subint_proc` hard-kill path is the natural
|
||||
place to add that notification, because it already
|
||||
knows the subint is unreachable at that point.
|
||||
|
||||
Alternative fix paths (blanket timeouts on
|
||||
`process_messages`, sentinel-wrapped channels) are
|
||||
less surgical and risk masking unrelated bugs —
|
||||
hence the preference ordering in the doc.
|
||||
|
||||
### Why we're not just patching the code now
|
||||
|
||||
The user explicitly deferred the fix to a later
|
||||
commit: "Document both classes now, land the fix
|
||||
for class 2 separately so the diff reviews clean."
|
||||
This matches the incremental-commits preference
|
||||
from memory.
|
||||
|
|
@ -1,155 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
session: subints-phase-b-hardening-and-fork-block
|
||||
timestamp: 2026-04-22T20:07:23Z
|
||||
git_ref: 797f57c
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260422T200723Z_797f57c_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Session-spanning work on the Phase B `subint` spawn-backend.
|
||||
Three distinct sub-phases in one log:
|
||||
|
||||
1. **Py3.13 gate tightening** — diagnose a reproducible hang
|
||||
of subint spawn flow under py3.13 (works on py3.14), trace
|
||||
to a private `_interpreters` module vintage issue, tighten
|
||||
our feature gate from "`_interpreters` present" to "public
|
||||
`concurrent.interpreters` present" (i.e. py3.14+).
|
||||
2. **Test-harness hardening** — add `pytest-timeout` dep, put
|
||||
`@pytest.mark.timeout(30, method='thread')` on the
|
||||
three known-hanging subint tests cataloged in
|
||||
`ai/conc-anal/subint_sigint_starvation_issue.md`. Separately,
|
||||
code-review the user's in-flight `skipon_spawn_backend`
|
||||
marker implementation; find four bugs; refactor to use
|
||||
`item.iter_markers()`.
|
||||
3. **`subint_fork` prototype → CPython-block finding** — draft
|
||||
a WIP `subint_fork_proc` backend using a sub-interpreter as
|
||||
a launchpad for `os.fork()` (to sidestep trio#1614). User
|
||||
tests on py3.14, hits
|
||||
`Fatal Python error: _PyInterpreterState_DeleteExceptMain:
|
||||
not main interpreter`. Walk CPython sources (local clone at
|
||||
`~/repos/cpython/`) to pinpoint the refusal
|
||||
(`Modules/posixmodule.c:728` → `Python/pystate.c:1040`).
|
||||
Revert implementation to a `NotImplementedError` stub in a
|
||||
new `_subint_fork.py` submodule, document the finding in a
|
||||
third `conc-anal/` doc with an upstream-report draft for
|
||||
the CPython issue tracker. Finally, discuss user's proposed
|
||||
workaround architecture (main-interp worker-thread
|
||||
forkserver) and draft a standalone smoke-test script for
|
||||
feasibility validation.
|
||||
|
||||
## Response summary
|
||||
|
||||
All three sub-phases landed concrete artifacts:
|
||||
|
||||
**Sub-phase 1** — `_subint.py` + `_spawn.py` gates + error
|
||||
messages updated to require py3.14+ via the public
|
||||
`concurrent.interpreters` module presence check. Module
|
||||
docstring revised to explain the empirical reason
|
||||
(py3.13's private `_interpreters` vintage wedges under
|
||||
multi-trio-task usage even though minimal standalone
|
||||
reproducers work fine there). Test-module
|
||||
`pytest.importorskip` likewise switched.
|
||||
|
||||
**Sub-phase 2** — `pytest-timeout>=2.3` added to `testing`
|
||||
dep group. `@pytest.mark.timeout(30, method='thread')`
|
||||
applied on:
|
||||
- `tests/discovery/test_registrar.py::test_stale_entry_is_deleted`
|
||||
- `tests/test_cancellation.py::test_cancel_while_childs_child_in_sync_sleep`
|
||||
- `tests/test_cancellation.py::test_multierror_fast_nursery`
|
||||
- `tests/test_subint_cancellation.py::test_subint_non_checkpointing_child`
|
||||
|
||||
`method='thread'` documented inline as load-bearing — the
|
||||
GIL-starvation path that drops `SIGINT` would equally drop
|
||||
`SIGALRM`, so only a watchdog-thread timeout can reliably
|
||||
escape.
|
||||
|
||||
`skipon_spawn_backend` plugin refactored into a single
|
||||
`iter_markers`-driven loop in `pytest_collection_modifyitems`
|
||||
(~30 LOC replacing ~30 LOC of nested conditionals). Four
|
||||
bugs dissolved: wrong `.get()` key, module-level `pytestmark`
|
||||
suppressing per-test marks, unhandled `pytestmark = [list]`
|
||||
form, `pytest.Makr` typo. Marker help text updated to
|
||||
document the variadic backend-list + `reason=` kwarg
|
||||
surface.
|
||||
|
||||
**Sub-phase 3** — Prototype drafted (then reverted):
|
||||
|
||||
- `tractor/spawn/_subint_fork.py` — new dedicated submodule
|
||||
housing the `subint_fork_proc` stub. Module docstring +
|
||||
fn docstring explain the attempt, the CPython-level
|
||||
block, and the reason for keeping the stub in-tree
|
||||
(documentation of the attempt + starting point if CPython
|
||||
ever lifts the restriction).
|
||||
- `tractor/spawn/_spawn.py` — `'subint_fork'` registered as a
|
||||
`SpawnMethodKey` literal + in `_methods`, so
|
||||
`--spawn-backend=subint_fork` routes to a clean
|
||||
`NotImplementedError` pointing at the analysis doc rather
|
||||
than an "invalid backend" error.
|
||||
- `ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md` —
|
||||
third sibling conc-anal doc. Full annotated CPython
|
||||
source walkthrough from user-visible
|
||||
`Fatal Python error` → `Modules/posixmodule.c:728
|
||||
PyOS_AfterFork_Child()` → `Python/pystate.c:1040
|
||||
_PyInterpreterState_DeleteExceptMain()` gate. Includes a
|
||||
copy-paste-ready upstream-report draft for the CPython
|
||||
issue tracker with a two-tier ask (ideally "make it work",
|
||||
minimally "cleaner error than `Fatal Python error`
|
||||
aborting the child").
|
||||
- `ai/conc-anal/subint_fork_from_main_thread_smoketest.py` —
|
||||
standalone zero-tractor-import CPython-level smoke test
|
||||
for the user's proposed workaround architecture
|
||||
(forkserver on a main-interp worker thread). Four
|
||||
argparse-driven scenarios: `control_subint_thread_fork`
|
||||
(reproduces the known-broken case as a test-harness
|
||||
sanity), `main_thread_fork` (baseline), `worker_thread_fork`
|
||||
(architectural assertion), `full_architecture`
|
||||
(end-to-end trio-in-subint in forked child). User will
|
||||
run on py3.14 next.
|
||||
|
||||
## Files changed
|
||||
|
||||
See `git log 26fb820..HEAD --stat` for the canonical list.
|
||||
New files this session:
|
||||
- `tractor/spawn/_subint_fork.py`
|
||||
- `ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`
|
||||
- `ai/conc-anal/subint_fork_from_main_thread_smoketest.py`
|
||||
|
||||
Modified (diff pointers in raw log):
|
||||
- `tractor/spawn/_subint.py` (py3.14 gate)
|
||||
- `tractor/spawn/_spawn.py` (`subint_fork` registration)
|
||||
- `tractor/_testing/pytest.py` (`skipon_spawn_backend` refactor)
|
||||
- `pyproject.toml` (`pytest-timeout` dep)
|
||||
- `tests/discovery/test_registrar.py`,
|
||||
`tests/test_cancellation.py`,
|
||||
`tests/test_subint_cancellation.py` (timeout marks,
|
||||
cross-refs to conc-anal docs)
|
||||
|
||||
## Human edits
|
||||
|
||||
Several back-and-forth iterations with user-driven
|
||||
adjustments during the session:
|
||||
|
||||
- User corrected my initial mis-classification of
|
||||
`test_cancel_while_childs_child_in_sync_sleep[subint-False]`
|
||||
as Ctrl-C-able — second strace showed `EAGAIN`, putting
|
||||
it squarely in class A (GIL-starvation). Re-analysis
|
||||
preserved in the raw log.
|
||||
- User independently fixed the `.get(reason)` → `.get('reason', reason)`
|
||||
bug in the marker plugin before my review; preserved their
|
||||
fix.
|
||||
- User suggested moving the `subint_fork_proc` stub from
|
||||
the bottom of `_subint.py` into its own
|
||||
`_subint_fork.py` submodule — applied.
|
||||
- User asked to keep the forkserver-architecture
|
||||
discussion as background for the smoke-test rather than
|
||||
committing to a tractor-side refactor until the smoke
|
||||
test validates the CPython-level assumptions.
|
||||
|
||||
Commit messages in this range (b025c982 … 797f57c) were
|
||||
drafted via `/commit-msg` + `rewrap.py --width 67`; user
|
||||
landed them with the usual review.
|
||||
|
|
@ -1,343 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
timestamp: 2026-04-22T20:07:23Z
|
||||
git_ref: 797f57c
|
||||
diff_cmd: git log 26fb820..HEAD # all session commits since the destroy-race fix log
|
||||
---
|
||||
|
||||
Session-spanning conversation covering the Phase B hardening
|
||||
of the `subint` spawn-backend and an investigation into a
|
||||
proposed `subint_fork` follow-up which turned out to be
|
||||
blocked at the CPython level. This log is a narrative capture
|
||||
of the substantive turns (not every message) and references
|
||||
the concrete code + docs the session produced. Per diff-ref
|
||||
mode the actual code diffs are pointed at via `git log` on
|
||||
each ref rather than duplicated inline.
|
||||
|
||||
## Narrative of the substantive turns
|
||||
|
||||
### Py3.13 hang / gate tightening
|
||||
|
||||
Diagnosed a reproducible hang of the `subint` backend under
|
||||
py3.13 (test_spawning tests wedge after root-actor bringup).
|
||||
Root cause: py3.13's vintage of the private `_interpreters` C
|
||||
module has a latent thread/subint-interaction issue that
|
||||
`_interpreters.exec()` silently fails to progress under
|
||||
tractor's multi-trio usage pattern — even though a minimal
|
||||
standalone `threading.Thread` + `_interpreters.exec()`
|
||||
reproducer works fine on the same Python. Empirically
|
||||
py3.14 fixes it.
|
||||
|
||||
Fix (from this session): tighten the `_has_subints` gate in
|
||||
`tractor.spawn._subint` from "private module importable" to
|
||||
"public `concurrent.interpreters` present" — which is 3.14+
|
||||
only. This leaves `subint_proc()` unchanged in behavior (we
|
||||
still call the *private* `_interpreters.create('legacy')`
|
||||
etc. under the hood) but refuses to engage on 3.13.
|
||||
|
||||
Also tightened the matching gate in
|
||||
`tractor.spawn._spawn.try_set_start_method('subint')` and
|
||||
rev'd the corresponding error messages from "3.13+" to
|
||||
"3.14+" with a sentence explaining why. Test-module
|
||||
`pytest.importorskip` switched from `_interpreters` →
|
||||
`concurrent.interpreters` to match.
|
||||
|
||||
### `pytest-timeout` dep + `skipon_spawn_backend` marker plumbing
|
||||
|
||||
Added `pytest-timeout>=2.3` to the `testing` dep group with
|
||||
an inline comment pointing at the `ai/conc-anal/*.md` docs.
|
||||
Applied `@pytest.mark.timeout(30, method='thread')` (the
|
||||
`method='thread'` is load-bearing — `signal`-method
|
||||
`SIGALRM` suffers the same GIL-starvation path that drops
|
||||
`SIGINT` in the class-A hang pattern) to the three known-
|
||||
hanging subint tests cataloged in
|
||||
`subint_sigint_starvation_issue.md`.
|
||||
|
||||
Separately code-reviewed the user's newly-staged
|
||||
`skipon_spawn_backend` pytest marker implementation in
|
||||
`tractor/_testing/pytest.py`. Found four bugs:
|
||||
|
||||
1. `modmark.kwargs.get(reason)` called `.get()` with the
|
||||
*variable* `reason` as the dict key instead of the string
|
||||
`'reason'` — user-supplied `reason=` was never picked up.
|
||||
(User had already fixed this locally via `.get('reason',
|
||||
reason)` by the time my review happened — preserved that
|
||||
fix.)
|
||||
2. The module-level `pytestmark` branch suppressed per-test
|
||||
marker handling (the `else:` was an `else:` rather than
|
||||
independent iteration).
|
||||
3. `mod_pytestmark.mark` assumed a single
|
||||
`MarkDecorator` — broke on the valid-pytest `pytestmark =
|
||||
[mark, mark]` list form.
|
||||
4. Typo: `pytest.Makr` → `pytest.Mark`.
|
||||
|
||||
Refactored the hook to use `item.iter_markers(name=...)`
|
||||
which walks function + class + module scopes uniformly and
|
||||
handles both `pytestmark` forms natively. ~30 LOC replaced
|
||||
the original ~30 LOC of nested conditionals, all four bugs
|
||||
dissolved. Also updated the marker help string to reflect
|
||||
the variadic `*start_methods` + `reason=` surface.
|
||||
|
||||
### `subint_fork_proc` prototype attempt
|
||||
|
||||
User's hypothesis: the known trio+`fork()` issues
|
||||
(python-trio/trio#1614) could be sidestepped by using a
|
||||
sub-interpreter purely as a launchpad — `os.fork()` from a
|
||||
subint that has never imported trio → child is in a
|
||||
trio-free context. In the child `execv()` back into
|
||||
`python -m tractor._child` and the downstream handshake
|
||||
matches `trio_proc()` identically.
|
||||
|
||||
Drafted the prototype at `tractor/spawn/_subint.py`'s bottom
|
||||
(originally — later moved to its own submod, see below):
|
||||
launchpad-subint creation, bootstrap code-string with
|
||||
`os.fork()` + `execv()`, driver-thread orchestration,
|
||||
parent-side `ipc_server.wait_for_peer()` dance. Registered
|
||||
`'subint_fork'` as a new `SpawnMethodKey` literal, added
|
||||
`case 'subint' | 'subint_fork':` feature-gate arm in
|
||||
`try_set_start_method()`, added entry in `_methods` dict.
|
||||
|
||||
### CPython-level block discovered
|
||||
|
||||
User tested on py3.14 and saw:
|
||||
|
||||
```
|
||||
Fatal Python error: _PyInterpreterState_DeleteExceptMain: not main interpreter
|
||||
Python runtime state: initialized
|
||||
|
||||
Current thread 0x00007f6b71a456c0 [subint-fork-lau] (most recent call first):
|
||||
File "<script>", line 2 in <module>
|
||||
<script>:2: DeprecationWarning: This process (pid=802985) is multi-threaded, use of fork() may lead to deadlocks in the child.
|
||||
```
|
||||
|
||||
Walked CPython sources (local clone at `~/repos/cpython/`):
|
||||
|
||||
- **`Modules/posixmodule.c:728` `PyOS_AfterFork_Child()`** —
|
||||
post-fork child-side cleanup. Calls
|
||||
`_PyInterpreterState_DeleteExceptMain(runtime)` with
|
||||
`goto fatal_error` on non-zero status. Has the
|
||||
`// Ideally we could guarantee tstate is running main.`
|
||||
self-acknowledging-fragile comment directly above.
|
||||
|
||||
- **`Python/pystate.c:1040`
|
||||
`_PyInterpreterState_DeleteExceptMain()`** — the
|
||||
refusal. Hard `PyStatus_ERR("not main interpreter")` gate
|
||||
when `tstate->interp != interpreters->main`. Docstring
|
||||
formally declares the precondition ("If there is a
|
||||
current interpreter state, it *must* be the main
|
||||
interpreter"). `XXX` comments acknowledge further latent
|
||||
issues within.
|
||||
|
||||
Definitive answer to "Open Question 1" of the prototype
|
||||
docstring: **no, CPython does not support `os.fork()` from
|
||||
a non-main sub-interpreter**. Not because the fork syscall
|
||||
is blocked (it isn't — the parent returns a valid pid),
|
||||
but because the child cannot survive CPython's post-fork
|
||||
initialization. This is an enforced invariant, not an
|
||||
incidental limitation.
|
||||
|
||||
### Revert: move to stub submod + doc the finding
|
||||
|
||||
Per user request:
|
||||
|
||||
1. Reverted the working `subint_fork_proc` body to a
|
||||
`NotImplementedError` stub, MOVED to its own submod
|
||||
`tractor/spawn/_subint_fork.py` (keeps `_subint.py`
|
||||
focused on the working `subint_proc` backend).
|
||||
2. Updated `_spawn.py` to import the stub from the new
|
||||
submod path; kept `'subint_fork'` in `SpawnMethodKey` +
|
||||
`_methods` so `--spawn-backend=subint_fork` routes to a
|
||||
clean `NotImplementedError` with pointer to the analysis
|
||||
doc rather than an "invalid backend" error.
|
||||
3. Wrote
|
||||
`ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`
|
||||
with the full annotated CPython walkthrough + an
|
||||
upstream-report draft for the CPython issue tracker.
|
||||
Draft has a two-tier ask: ideally "make it work"
|
||||
(pre-fork tstate-swap hook or `DeleteExceptFor(interp)`
|
||||
variant), minimally "give us a clean `RuntimeError` in
|
||||
the parent instead of a `Fatal Python error` aborting
|
||||
the child silently".
|
||||
|
||||
### Design discussion — main-interp-thread forkserver workaround
|
||||
|
||||
User proposed: set up a "subint forking server" that fork()s
|
||||
on behalf of subint callers. Core insight: the CPython gate
|
||||
is on `tstate->interp`, not thread identity, so **any thread
|
||||
whose tstate is main-interp** can fork cleanly. A worker
|
||||
thread attached to main-interp (never entering a subint)
|
||||
satisfies the precondition.
|
||||
|
||||
Structurally this is `mp.forkserver` (which tractor already
|
||||
has as `mp_forkserver`) but **in-process**: instead of a
|
||||
separate Python subproc as the fork server, we'd put the
|
||||
forkserver on a thread in the tractor parent process. Pros:
|
||||
faster spawn (no IPC marshalling to external server + no
|
||||
separate Python startup), inherits already-imported modules
|
||||
for free. Cons: less crash isolation (forkserver failure
|
||||
takes the whole process).
|
||||
|
||||
Required tractor-side refactor: move the root actor's
|
||||
`trio.run()` off main-interp-main-thread (so main-thread can
|
||||
run the forkserver loop). Nontrivial; approximately the same
|
||||
magnitude as "Phase C".
|
||||
|
||||
The design would also not fully resolve the class-A
|
||||
GIL-starvation issue because child actors' trio still runs
|
||||
inside subints (legacy config, msgspec PEP 684 pending).
|
||||
Would mitigate SIGINT-starvation specifically if signal
|
||||
handling moves to the forkserver thread.
|
||||
|
||||
Recommended pre-commitment: a standalone CPython-only smoke
|
||||
test validating the four assumptions the arch rests on,
|
||||
before any tractor-side work.
|
||||
|
||||
### Smoke-test script drafted
|
||||
|
||||
Wrote `ai/conc-anal/subint_fork_from_main_thread_smoketest.py`:
|
||||
argparse-driven, four scenarios (`control_subint_thread_fork`
|
||||
reproducing the known-broken case, `main_thread_fork`
|
||||
baseline, `worker_thread_fork` the architectural assertion,
|
||||
`full_architecture` end-to-end with trio in a subint in the
|
||||
forked child). No `tractor` imports; pure CPython + `_interpreters`
|
||||
+ `trio`. Bails cleanly on py<3.14. Pass/fail banners per
|
||||
scenario.
|
||||
|
||||
User will validate on their py3.14 env next.
|
||||
|
||||
## Per-code-artifact provenance
|
||||
|
||||
### `tractor/spawn/_subint_fork.py` (new submod)
|
||||
|
||||
> `git show 797f57c -- tractor/spawn/_subint_fork.py`
|
||||
|
||||
NotImplementedError stub for the subint-fork backend. Module
|
||||
docstring + fn docstring explain the attempt, the CPython
|
||||
block, and why the stub is kept in-tree. No runtime behavior
|
||||
beyond raising with a pointer at the conc-anal doc.
|
||||
|
||||
### `tractor/spawn/_spawn.py` (modified)
|
||||
|
||||
> `git log 26fb820..HEAD -- tractor/spawn/_spawn.py`
|
||||
|
||||
- Added `'subint_fork'` to `SpawnMethodKey` literal with a
|
||||
block comment explaining the CPython-level block.
|
||||
- Generalized the `case 'subint':` arm to `case 'subint' |
|
||||
'subint_fork':` since both use the same py3.14+ gate.
|
||||
- Registered `subint_fork_proc` in `_methods` with a
|
||||
pointer-comment at the analysis doc.
|
||||
|
||||
### `tractor/spawn/_subint.py` (modified across session)
|
||||
|
||||
> `git log 26fb820..HEAD -- tractor/spawn/_subint.py`
|
||||
|
||||
- Tightened `_has_subints` gate: dual-requires public
|
||||
`concurrent.interpreters` + private `_interpreters`
|
||||
(tests for py3.14-or-newer on the public-API presence,
|
||||
then uses the private one for legacy-config subints
|
||||
because `msgspec` still blocks the public isolated mode
|
||||
per jcrist/msgspec#563).
|
||||
- Updated module docstring, `subint_proc()` docstring, and
|
||||
gate-error messages to reflect the 3.14+ requirement and
|
||||
the reason (py3.13 wedges under multi-trio usage even
|
||||
though the private module exists there).
|
||||
|
||||
### `tractor/_testing/pytest.py` (modified)
|
||||
|
||||
> `git log 26fb820..HEAD -- tractor/_testing/pytest.py`
|
||||
|
||||
- New `skipon_spawn_backend(*start_methods, reason=...)`
|
||||
pytest marker expanded into `pytest.mark.skip(reason=...)`
|
||||
at collection time via
|
||||
`pytest_collection_modifyitems()`.
|
||||
- Implementation uses `item.iter_markers(name=...)` which
|
||||
walks function + class + module scopes uniformly and
|
||||
handles both `pytestmark = <single Mark>` and
|
||||
`pytestmark = [mark, ...]` forms natively. ~30-LOC
|
||||
single-loop refactor replacing a prior nested
|
||||
conditional that had four bugs (see "Review" narrative
|
||||
above).
|
||||
- Added `pytest.Config` / `pytest.Function` /
|
||||
`pytest.FixtureRequest` type annotations on fixture
|
||||
signatures while touching the file.
|
||||
|
||||
### `pyproject.toml` (modified)
|
||||
|
||||
> `git log 26fb820..HEAD -- pyproject.toml`
|
||||
|
||||
Added `pytest-timeout>=2.3` to `testing` dep group with
|
||||
comment pointing at the `ai/conc-anal/` docs.
|
||||
|
||||
### `tests/discovery/test_registrar.py`,
|
||||
`tests/test_subint_cancellation.py`,
|
||||
`tests/test_cancellation.py` (modified)
|
||||
|
||||
> `git log 26fb820..HEAD -- tests/`
|
||||
|
||||
Applied `@pytest.mark.timeout(30, method='thread')` on
|
||||
known-hanging subint tests. Extended comments to cross-
|
||||
reference the `ai/conc-anal/*.md` docs. `method='thread'`
|
||||
is documented inline as load-bearing (`signal`-method
|
||||
SIGALRM suffers the same GIL-starvation path that drops
|
||||
SIGINT).
|
||||
|
||||
### `ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md` (new)
|
||||
|
||||
> `git show 797f57c -- ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`
|
||||
|
||||
Third sibling doc under `conc-anal/`. Structure: TL;DR,
|
||||
context ("what we tried"), symptom (the user's exact
|
||||
`Fatal Python error` output), CPython source walkthrough
|
||||
with excerpted snippets from `posixmodule.c` +
|
||||
`pystate.c`, chain summary, definitive answer to Open
|
||||
Question 1, `## Upstream-report draft (for CPython issue
|
||||
tracker)` section with a two-tier ask, references.
|
||||
|
||||
### `ai/conc-anal/subint_fork_from_main_thread_smoketest.py` (new, THIS turn)
|
||||
|
||||
Zero-tractor-import smoke test for the proposed workaround
|
||||
architecture. Four argparse-driven scenarios covering the
|
||||
control case + baseline + arch-critical case + end-to-end.
|
||||
Pass/fail banners per scenario; clean `--help` output;
|
||||
py3.13 early-exit.
|
||||
|
||||
## Non-code output (verbatim)
|
||||
|
||||
### The `strace` signature that kicked off the CPython
|
||||
walkthrough
|
||||
|
||||
```
|
||||
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
|
||||
write(16, "\2", 1) = -1 EAGAIN (Resource temporarily unavailable)
|
||||
rt_sigreturn({mask=[WINCH]}) = 139801964688928
|
||||
```
|
||||
|
||||
### Key user quotes framing the direction
|
||||
|
||||
> ok actually we get this [fatal error] ... see if you can
|
||||
> take a look at what's going on, in particular wrt to
|
||||
> cpython's sources. pretty sure there's a local copy at
|
||||
> ~/repos/cpython/
|
||||
|
||||
(Drove the CPython walkthrough that produced the
|
||||
definitive refusal chain.)
|
||||
|
||||
> is there any reason we can't just sidestep this "must fork
|
||||
> from main thread in main subint" issue by simply ensuring
|
||||
> a "subint forking server" is always setup prior to
|
||||
> invoking trio in a non-main-thread subint ...
|
||||
|
||||
(Drove the main-interp-thread-forkserver architectural
|
||||
discussion + smoke-test script design.)
|
||||
|
||||
### CPython source tags for quick jump-back
|
||||
|
||||
```
|
||||
Modules/posixmodule.c:728 PyOS_AfterFork_Child()
|
||||
Modules/posixmodule.c:753 // Ideally we could guarantee tstate is running main.
|
||||
Modules/posixmodule.c:778 status = _PyInterpreterState_DeleteExceptMain(runtime);
|
||||
|
||||
Python/pystate.c:1040 _PyInterpreterState_DeleteExceptMain()
|
||||
Python/pystate.c:1044-1047 tstate->interp != main → PyStatus_ERR("not main interpreter")
|
||||
```
|
||||
|
|
@ -1,146 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
session: trio-0.33-subproc-supervisor-retroactive
|
||||
timestamp: 2026-06-01T23:14:29Z
|
||||
git_ref: 0e3e008b
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260601T231429Z_0e3e008b_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
**RETROACTIVE LOG** — original session prompts not
|
||||
preserved; reconstructed from the staged work product.
|
||||
|
||||
The work designs a `trio.Nursery.start()`-style wrapper
|
||||
around `trio.run_process()` for SC-friendly subprocess
|
||||
supervision. From the resulting code shape, the
|
||||
prompting intent was:
|
||||
|
||||
1. Surface rc!=0 `CalledProcessError` DETERMINISTICALLY,
|
||||
without the nursery-eg-wrapping that complicates
|
||||
`collapse_eg()` usage and races the relay reader on
|
||||
trio's `check=True`-driven cancel cascade.
|
||||
2. ALWAYS isolate the parent controlling-tty so a
|
||||
spawned child can't emit terminal control-seqs onto
|
||||
the launching tty (clobbering scrollback). Default
|
||||
`stdin=DEVNULL`; default `stdout=DEVNULL` unless
|
||||
explicitly relayed/overridden; distinguish "caller
|
||||
passed nothing" from "caller passed `None` for
|
||||
inherit".
|
||||
3. Optional live per-line relay of child std-streams to
|
||||
the `tractor` log — STREAMED (not
|
||||
buffered-until-exit) so long-lived daemon output is
|
||||
visible during the run. Pick a custom log level that
|
||||
shows at usual `info`/`devx` console levels but is
|
||||
separately filterable.
|
||||
4. Concurrent pipe-drain reader MANDATORY when piping
|
||||
without `capture_*` — without it the child blocks on
|
||||
`write()` once the OS pipe buffer fills (~64KiB),
|
||||
causing deadlocks on output bursts.
|
||||
5. Non-blocking `tn.start()` semantics: hand the live
|
||||
`trio.Process` to the parent immediately;
|
||||
supervise/relay run to completion in the supervisor
|
||||
coro.
|
||||
6. Hermetic `trio`-only unit tests (no actor-runtime)
|
||||
covering each of: per-line relay, tty isolation,
|
||||
no-deadlock on >64KiB unnewlined output, CPE
|
||||
rebuild w/ stderr relay, CPE rebuild on the silent
|
||||
drain+capture path.
|
||||
|
||||
## Response summary
|
||||
|
||||
Adds `tractor/trionics/_subproc.py` (296 LOC) +
|
||||
`tests/trionics/test_subproc.py` (230 LOC) + a
|
||||
re-export in `tractor/trionics/__init__.py`.
|
||||
|
||||
**`supervise_run_process()`** (public, re-exported)
|
||||
- `check=False` is forced to `trio.run_process`; the
|
||||
rc-check runs in the supervisor coro AFTER `own_tn`
|
||||
unwinds (both the child AND the relay readers have
|
||||
hit EOF + fully drained). A BARE
|
||||
`subprocess.CalledProcessError` is rebuilt + raised
|
||||
from there, with `.stderr` bytes passed in the
|
||||
constructor AND attached as an `add_note()`'d
|
||||
`|_.stderr:` block for legible teardown logs.
|
||||
- `stdin=DEVNULL` always. `stdout` default chosen via a
|
||||
`_UNSET` sentinel: `relay_stdout=True` → PIPE,
|
||||
explicit `stdout=...` → as given, else `DEVNULL`.
|
||||
`stderr` defaults to PIPE whenever we relay OR need
|
||||
the CPE note (when `check=True`), else `DEVNULL`.
|
||||
- `relay_level='io'` (custom level 21; sorts just
|
||||
above stdlib `INFO`=20 so it shows at usual
|
||||
`info`/`devx` levels and stays separately
|
||||
filterable). `runtime`=15 would silently filter at
|
||||
default levels, so it's rejected as a default.
|
||||
- `task_status.started(trio_proc)` delivers the live
|
||||
process immediately. The internal `own_tn`
|
||||
supervises `trio.run_process` + any relay readers to
|
||||
completion.
|
||||
- `**run_process_kwargs` forward verbatim;
|
||||
`stdin/stdout/stderr/check` are MANAGED keys
|
||||
(override on conflict).
|
||||
- Crash-handling deliberately NOT baked in — compose
|
||||
`maybe_open_crash_handler()` on top at the call-site.
|
||||
|
||||
**`_relay_stream_lines()`** (internal helper)
|
||||
- Three modes (combinable): `emit`-only (live per-line
|
||||
relay), `accum`-only (silent drain+capture for a CPE
|
||||
note), or both (live relay AND capture).
|
||||
- Per-line split handles cross-chunk residuals via a
|
||||
rolling `residual` bytes buffer; flushes any trailing
|
||||
un-newline-term'd line at EOF.
|
||||
- `async with stream:` ensures aclose at EOF/cancel
|
||||
(mirrors trio's internal `_subprocess` drain idiom).
|
||||
|
||||
**`_add_stderr_note()`** (internal helper)
|
||||
- `add_note()`s a `textwrap.indent(...)`'d
|
||||
`|_.stderr:` block onto a `CalledProcessError` for
|
||||
teardown logs.
|
||||
|
||||
**Tests** (5 hermetic, trio-only) — `_capture_relay`
|
||||
fixture monkeypatches `_subproc.log.<level>` to a list:
|
||||
- `test_stdout_relayed_per_line`: per-line stdout
|
||||
relay carries each `line=N` to the records.
|
||||
- `test_parent_tty_isolated`: `readlink /proc/self/fd/0`
|
||||
and `fd/1` from the child show `pipe:` (fd1) +
|
||||
`/dev/null` (fd0); NO `/dev/pts/*`.
|
||||
- `test_no_deadlock_on_big_unnewlined_output`: 200KiB
|
||||
of `x` with no newlines completes inside
|
||||
`fail_after(2)` — exercises the concurrent drain.
|
||||
- `test_stderr_relay_and_cpe_rebuild`: rc=3 with
|
||||
`relay_stderr=True` raises bare CPE
|
||||
(via `collapse_eg()`) with `b'boom' in cpe.stderr`,
|
||||
the note attached, AND per-line live relay.
|
||||
- `test_nonrelay_cpe_note`: rc=7 with no relay still
|
||||
produces CPE with `.stderr` + note via the silent
|
||||
drain+capture path.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/trionics/_subproc.py` — NEW. Public
|
||||
`supervise_run_process()` + helpers
|
||||
`_relay_stream_lines()` / `_add_stderr_note()` + the
|
||||
`_UNSET` sentinel.
|
||||
- `tests/trionics/test_subproc.py` — NEW. 5 hermetic
|
||||
trio-only tests + `_capture_relay` monkeypatch
|
||||
fixture.
|
||||
- `tractor/trionics/__init__.py` — re-export
|
||||
`supervise_run_process`.
|
||||
|
||||
## Human edits
|
||||
|
||||
**RETROACTIVE**: this log is being written from the
|
||||
staged diff, not from a live session. The code as
|
||||
staged is the canonical artifact; any human edits the
|
||||
user made during the originating design session are
|
||||
already integrated and cannot be separated post-hoc.
|
||||
The `.raw.md` sibling is a diff-pointer placeholder,
|
||||
NOT a pre-edit transcript.
|
||||
|
||||
Future prompt-io entries for in-flight work should be
|
||||
written DURING the design session per the skill
|
||||
contract so the pre-edit `.raw.md` captures the
|
||||
unedited model output for genuine provenance.
|
||||
|
|
@ -1,106 +0,0 @@
|
|||
---
|
||||
model: claude-opus-4-7[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-01T23:14:29Z
|
||||
git_ref: 0e3e008b
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
# RETROACTIVE — original model output not preserved
|
||||
|
||||
This `.raw.md` would normally contain the verbatim
|
||||
pre-human-edit response from the design session that
|
||||
produced the staged `_subproc.py` module + tests. That
|
||||
session's transcript is not available, so this file
|
||||
serves as a diff-pointer placeholder + transparency
|
||||
note.
|
||||
|
||||
## Authoritative artifact
|
||||
|
||||
The committed code IS the artifact of record. Once the
|
||||
companion commit lands, the unified diff is:
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tractor/trionics/_subproc.py`
|
||||
> `git diff HEAD~1..HEAD -- tests/trionics/test_subproc.py`
|
||||
> `git diff HEAD~1..HEAD -- tractor/trionics/__init__.py`
|
||||
|
||||
Before committing, substitute `--cached` for the
|
||||
pre-commit form.
|
||||
|
||||
## What is NOT here
|
||||
|
||||
Because this is retroactive:
|
||||
- No verbatim chain-of-thought / discussion prose from
|
||||
the design session.
|
||||
- No rejected alternatives the model considered before
|
||||
arriving at the final shape (e.g. whether the
|
||||
rc-check should live inside `own_tn` vs after it; the
|
||||
`_UNSET` sentinel vs a `None`-means-DEVNULL
|
||||
convention; `io` vs `info` as the default relay
|
||||
level).
|
||||
- No pre-edit code blocks as the model first emitted
|
||||
them, separable from any user cleanup applied before
|
||||
the diff was staged.
|
||||
|
||||
## Inferred design choices visible in the final code
|
||||
|
||||
(Documented here because they're the kind of decision
|
||||
detail an unedited raw transcript would have captured.)
|
||||
|
||||
1. **Post-drain rc-check in the supervisor coro body,
|
||||
AFTER `own_tn.__aexit__`.** Placing the
|
||||
`CalledProcessError` raise here (not inside
|
||||
`own_tn`) means the EG-unwrap happens at the OUTER
|
||||
`tn.start()` boundary — callers do `collapse_eg()`
|
||||
if they want bare. Doing the raise INSIDE `own_tn`
|
||||
would cancel the still-draining relay reader
|
||||
mid-flight and lose stderr lines.
|
||||
|
||||
2. **`_UNSET` sentinel for `stdout`.** A plain default
|
||||
of `None` couldn't distinguish "use the safe
|
||||
`DEVNULL` default" from "caller explicitly passed
|
||||
`None` (inherit, presumably knowingly)". The
|
||||
sentinel keeps the SAFE default while letting power
|
||||
users opt into inherit.
|
||||
|
||||
3. **`relay_level='io'` (custom level 21).** Chosen to
|
||||
sort just above stdlib `INFO`=20 so a default
|
||||
`--ll info` shows the relay, but it remains a
|
||||
distinct level so users can filter
|
||||
`tractor.trionics:io` separately. Picking
|
||||
`runtime`=15 would have made the relay invisible at
|
||||
default verbosity (a footgun for daemon supervisors
|
||||
whose whole point is "I want to see this output").
|
||||
|
||||
4. **Reader is MANDATORY, not opt-in cosmetic.** With
|
||||
`stdout=PIPE` / `stderr=PIPE` we OWN the drain
|
||||
responsibility — there's no `trio.capture_*` running
|
||||
under the hood here. The ~64KiB OS pipe buffer
|
||||
means a child writing more than that without us
|
||||
reading hangs at `write()` — a deadlock that won't
|
||||
show up in small-output tests, which is why the
|
||||
200KiB-no-newline test is in the suite.
|
||||
|
||||
5. **`task_status.started(trio_proc)` BEFORE the
|
||||
`own_tn` exits.** Without this, `tn.start()` would
|
||||
block until the child exits — losing the "start a
|
||||
long-lived daemon and continue with parent work"
|
||||
use case. With it, the parent gets the live process
|
||||
handle immediately and the supervise+relay tasks
|
||||
run in the supervisor coro until the child exits.
|
||||
|
||||
6. **`__notes__` via `add_note()` for the CPE
|
||||
`.stderr`.** The `.stderr` attribute is what
|
||||
`subprocess` callers expect; the `add_note()` is
|
||||
what trio's exception-rendering shows. Both wired so
|
||||
programmatic AND human consumers see the stderr at
|
||||
teardown.
|
||||
|
||||
## Honesty statement
|
||||
|
||||
This file's content is RECONSTRUCTED from the staged
|
||||
code, not extracted from a verbatim model transcript.
|
||||
The prompt-io skill's intent is for the `.raw.md` to
|
||||
be a pre-edit fossil; that's not possible here. Future
|
||||
work should write the prompt-io entry DURING the
|
||||
design session.
|
||||
|
|
@ -1,93 +0,0 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 0780a862-e19a-4f0a-86cd-c8afc0997757
|
||||
timestamp: 2026-06-11T17:51:52Z
|
||||
git_ref: 8526985c
|
||||
scope: docs+code+config
|
||||
substantive: true
|
||||
raw_file: 20260611T175152Z_8526985c_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
> we need big boi docs; ours are way out of date and generally
|
||||
> terrible. i'd like to use a sphinx theme in the vein and/or
|
||||
> one-of/nearly-the-same-as used in one or all of the `msgspec`,
|
||||
> `numpy`, `ray`, `xonsh`, `polars` projects.
|
||||
>
|
||||
> see the following oustanding but now very old issues to guide
|
||||
> you: #175, #126, #123, #157
|
||||
>
|
||||
> we should try to target a 3 columned sphinx theme with,
|
||||
> - document index-nav on left,
|
||||
> - body content in middle,
|
||||
> - diagrams (ideally in d2lang if possible) on RHS
|
||||
>
|
||||
> optimize for (if possible),
|
||||
> - reusing examples code without duplication in docs,
|
||||
> - generating new examples from todos throughout code base,
|
||||
> - distilling the essence of SC across processes in the simplest
|
||||
> and most friendly way possible.
|
||||
>
|
||||
> do this work in a /open-wkt and do you best without my guidance
|
||||
> for the first major pass - we will refine asap once you are
|
||||
> complete.
|
||||
|
||||
Session settings: `/effort max` + ultracode (multi-agent workflow
|
||||
orchestration). Fully autonomous first pass; two orchestrated
|
||||
agent fleets were used (6-agent recon survey, then 10-agent
|
||||
content fan-out: 9 section writers + 1 examples smith), with the
|
||||
orchestrator authoring the landing page, SC essay, sphinx
|
||||
scaffold, d2 pipeline + diagram sources, CI workflow and all
|
||||
integration/fix passes directly.
|
||||
|
||||
## Response summary
|
||||
|
||||
Complete sphinx docs revamp on branch `wkt/big_boi_docs`:
|
||||
`pydata-sphinx-theme` 0.18 (3-column: left nav / content /
|
||||
page-toc, with d2 diagrams + asides floated into the RHS margin
|
||||
via custom CSS), sphinx 9.1, a local `.. d2::` directive
|
||||
rendering `docs/diagrams/*.d2` sources with committed-SVG
|
||||
fallback, a vendored `.. margin::` directive, ~25 new doc pages
|
||||
(landing, start/, explain/, 12 guides, 10 api-ref pages,
|
||||
project/), 5 new auto-tested examples + 3 modernized + 1
|
||||
renamed, and a gh-pages deploy workflow (issue #123). All
|
||||
example code is `literalinclude`d from `examples/` (zero
|
||||
duplication, CI-verified). Build: green, 24 warnings all
|
||||
pre-existing-docstring/NEWS sourced.
|
||||
|
||||
## Files changed
|
||||
|
||||
See the branch diff (uncommitted at entry-write time):
|
||||
|
||||
> `git diff test_cpu_throttling..wkt/big_boi_docs`
|
||||
> `git -C <wkt> status --short` (pre-commit working tree)
|
||||
|
||||
- `docs/conf.py` — full rewrite for pydata theme + ext stack
|
||||
- `docs/_ext/d2diagrams.py` — new `.. d2::` sphinx directive
|
||||
- `docs/_ext/marginalia.py` — new `.. margin::` directive
|
||||
- `docs/_static/css/custom.css` — b&w skin + RHS margin floats
|
||||
- `docs/diagrams/*.d2` (7) — diagram sources (sketch/grayscale)
|
||||
- `docs/_diagrams/*.svg` (7) — committed rendered fallbacks
|
||||
- `docs/index.rst` — new landing (replaces dead-API doc)
|
||||
- `docs/start/*.rst` (3), `docs/explain/*.rst` (3),
|
||||
`docs/guide/*.rst` (13), `docs/api/*.rst` (10),
|
||||
`docs/project/*.rst` (3) — new content tree
|
||||
- `docs/dev_tips.rst` — removed (ported to project/dev-tips)
|
||||
- `examples/{typed_payloads,nested_actor_tree,
|
||||
service_daemon_discovery,uds_transport_actor_tree,
|
||||
streaming_broadcast_fanout}.py` — new, smoke-tested
|
||||
- `examples/{a_trynamic_first_scene,
|
||||
actor_spawning_and_causality,parallelism/single_func}.py` —
|
||||
`.result()` -> `.wait_for_result()` modernization
|
||||
- `examples/parallelism/concurrent_futures_primes.py` — renamed
|
||||
from leading-underscore + trio-runner shim added
|
||||
- `pyproject.toml` — `docs` dependency-group filled in
|
||||
- `uv.lock` — relock for docs group
|
||||
- `.github/workflows/docs.yml` — build + gh-pages deploy
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — entry written pre-commit; the author reviews, stages
|
||||
and commits manually (per repo workflow policy).
|
||||
|
|
@ -1,49 +0,0 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-11T17:51:52Z
|
||||
git_ref: 8526985c
|
||||
diff_cmd: git diff test_cpu_throttling..wkt/big_boi_docs
|
||||
---
|
||||
|
||||
# Raw output pointers (diff-ref mode)
|
||||
|
||||
All generated content is code/config/docs committed alongside
|
||||
this entry on branch `wkt/big_boi_docs`; per the prompt-io
|
||||
diff-ref decision rule each file's verbatim content is the diff
|
||||
itself rather than a copy here:
|
||||
|
||||
> `git diff test_cpu_throttling..wkt/big_boi_docs -- docs/`
|
||||
> `git diff test_cpu_throttling..wkt/big_boi_docs -- examples/`
|
||||
> `git diff test_cpu_throttling..wkt/big_boi_docs -- pyproject.toml uv.lock .github/`
|
||||
|
||||
## Generation notes (non-code output summary)
|
||||
|
||||
- Theme research (web, agent-verified 2026-06-11): msgspec=furo,
|
||||
xonsh=furo, numpy/ray/polars=pydata-sphinx-theme (ray migrated
|
||||
off sphinx-book-theme); sphinx-book-theme 1.2.0 hard-pins
|
||||
pydata 0.16.1 (stale) -> chose pydata 0.18 + sphinx 9.1.
|
||||
- d2 ecosystem: no production-grade pypi extension exists
|
||||
(sphinxcontrib-d2lang 0.0.5 ignores returncodes, uuid4 output
|
||||
names; sphinx-d2 is an empty stub) -> wrote local
|
||||
`docs/_ext/d2diagrams.py` (~230 LOC) with D2_BIN env
|
||||
discovery, mtime caching, committed-SVG fallback and
|
||||
literal-block last resort.
|
||||
- Diagrams authored in d2 (theme-id 1 "Neutral Grey" + sketch
|
||||
mode + ELK layout, validated by render + headless-firefox
|
||||
screenshot loop): actor_tree, context_handshake (real
|
||||
msg-spec names Start/StartAck/Started/Yield/Stop/Return),
|
||||
streaming_pipeline, runtime_stack, debug_lock,
|
||||
error_propagation, infected_aio.
|
||||
- API truth enforced from a 6-agent recon pass over the
|
||||
reorganized package tree (runtime/, discovery/, spawn/, ipc/,
|
||||
msg/, devx/, trionics/): docs teach `.wait_for_result()`,
|
||||
registrar (not arbiter) naming, `@tractor.context` +
|
||||
`open_context()` as the core model, `run_in_actor()` as
|
||||
convenience only.
|
||||
- All ~30 literalincluded example scripts verified present; 9
|
||||
touched/new example files smoke-run green (exit 0, <16s).
|
||||
- Final build: `sphinx-build -b html` succeeded; 24 residual
|
||||
warnings, every one sourced from pre-existing library
|
||||
docstring rst-isms or legacy NEWS.rst content (left untouched
|
||||
by design; flagged for a follow-up docstring lint pass).
|
||||
|
|
@ -1,65 +0,0 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 638d5a98-abd2-4217-a447-19f2330eb334
|
||||
timestamp: 2026-06-25T23:25:06Z
|
||||
git_ref: 0a3c48fc
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260625T232506Z_0a3c48fc_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Same `/code-review` follow-up session (PR #460). Finding #10:
|
||||
the new `api/` autodoc reference pages surfaced 22 docutils
|
||||
warnings sourced from informal reST in public docstrings. The
|
||||
user opted to fix it on this branch,
|
||||
|
||||
> well we're on a clean main so why not just do these now too?
|
||||
|
||||
Goal: drive the docs build warning-free (24 -> 0) without any
|
||||
code/behaviour change — docstring content only.
|
||||
|
||||
## Response summary
|
||||
|
||||
Surgical docstring-RST lint across 9 library modules + the one
|
||||
offending `NEWS.rst` changelog entry,
|
||||
|
||||
- bullet lists given a blank line + base-column indent
|
||||
(`Context`, `Context.cancel_called`/`.cancelled_caught`/
|
||||
`.outcome`, `ActorNursery.cancel_called`, `query_actor`,
|
||||
`open_crash_handler`, the `#318` `NEWS.rst` entry),
|
||||
- the under-short `Behaviour:` underline in `Context.cancel`
|
||||
demoted to a `**bold**` label,
|
||||
- unbalanced/pluralized-role backticks fixed — closed the
|
||||
`wait_for_actor` summary backtick; applied the `` `role`\ s ``
|
||||
escaped-plural idiom in `gather_contexts`, `mk_pdb`,
|
||||
`MsgCodec`, msg `Error`, `open_context_from_portal`,
|
||||
- the `|_` method-tree in `ContextCancelled.canceller` made a
|
||||
literal block (bare `|` was parsed as a substitution ref).
|
||||
|
||||
Verified: build `24 -> 0` warnings; `import tractor` clean under
|
||||
`-W error::SyntaxWarning`; the `\ s` idiom renders as e.g.
|
||||
"acms" (no backslash/space leak in HTML); `ruff` clean; diff
|
||||
confirmed docstring-content-only.
|
||||
|
||||
Initial generation was delegated to a constrained subagent
|
||||
(docstring-only edits + a rebuild-to-zero verification gate);
|
||||
the orchestrator independently re-built, reviewed the full
|
||||
diff, and added the `NEWS.rst` fix to reach zero.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/_context.py`, `tractor/_exceptions.py`,
|
||||
`tractor/devx/debug/_post_mortem.py`,
|
||||
`tractor/devx/debug/_repl.py`, `tractor/discovery/_api.py`,
|
||||
`tractor/msg/_codec.py`, `tractor/msg/types.py`,
|
||||
`tractor/runtime/_supervise.py`,
|
||||
`tractor/trionics/_mngrs.py` — docstring reST fixes,
|
||||
- `NEWS.rst` — blank line before a bullet list in the `#318`
|
||||
entry.
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated (`0a3c48fc`).
|
||||
|
|
@ -1,35 +0,0 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-25T23:25:06Z
|
||||
git_ref: 0a3c48fc
|
||||
diff_cmd: git diff 0a3c48fc~1..0a3c48fc
|
||||
---
|
||||
|
||||
# Raw output pointers (diff-ref mode)
|
||||
|
||||
The generated patch is committed; the verbatim content is the
|
||||
diff:
|
||||
|
||||
> `git diff 0a3c48fc~1..0a3c48fc`
|
||||
|
||||
## Generation notes (non-code, verbatim)
|
||||
|
||||
- Pure docstring-content edits; no signatures, logic, or
|
||||
non-docstring lines touched (confirmed via `git diff` review).
|
||||
- reST fix patterns applied: blank-line + base-column indent
|
||||
for bullet lists ("Unexpected indentation" / "Definition
|
||||
list ..." / "Block quote ..."); `**bold**` in place of an
|
||||
under-length section underline ("Title underline too short");
|
||||
closing/escaping backticks ("Inline interpreted text ...
|
||||
without end-string"); literal-block for an ASCII tree whose
|
||||
`|` chars tripped "Inline substitution_reference ...".
|
||||
- The pluralized-role idiom is written `\\ s` in the Python
|
||||
source (so the runtime docstring holds `` `role`\ s ``); a
|
||||
bare `\ ` would raise a 3.13 invalid-escape SyntaxWarning.
|
||||
Verified clean via `python -W error::SyntaxWarning -c
|
||||
"import tractor"`.
|
||||
- Verification gate: `sphinx-build` warning count 24 -> 2
|
||||
(subagent, docstrings only) -> 0 after the orchestrator added
|
||||
the `NEWS.rst` blank-line fix; rendered-HTML spot-check
|
||||
confirmed no literal `\ s` leak; `ruff check` clean.
|
||||
|
|
@ -1,64 +0,0 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 638d5a98-abd2-4217-a447-19f2330eb334
|
||||
timestamp: 2026-06-25T23:25:06Z
|
||||
git_ref: 3a5cfde0
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260625T232506Z_3a5cfde0_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Follow-up to a `/code-review` of PR #460 (this branch) which
|
||||
surfaced lower-severity findings in the `.. d2::` sphinx
|
||||
directive. The user's driving instructions across the session,
|
||||
|
||||
> yup do both *at least*, then report back.
|
||||
|
||||
(re: "implement #7 (+ optionally #8) on the branch") and,
|
||||
|
||||
> well we're on a clean main so why not just do these now too?
|
||||
|
||||
(re: also doing #9). The three findings being addressed,
|
||||
|
||||
- #7: a `d2` render that is *attempted and fails* (binary
|
||||
present but errors on the source) silently degraded to the
|
||||
stale committed SVG and the build still exited 0,
|
||||
- #8: `parallel_*_safe = True` was declared while `run()`
|
||||
writes the shared output SVG during the read phase (torn
|
||||
write under `sphinx-build -j`),
|
||||
- #9: output keyed only by `src.stem` so two `.d2` sources
|
||||
with the same stem collide.
|
||||
|
||||
## Response summary
|
||||
|
||||
Reworked `docs/_ext/d2diagrams.py`,
|
||||
|
||||
- `render_svg()` now returns a `RenderResult` tristate
|
||||
(`OK` / `FELL_BACK` / `NO_OUTPUT` / `FAILED`) so `run()` can
|
||||
distinguish a graceful no-binary fallback from a real render
|
||||
failure; `FAILED` emits a `reporter.error` (build fails under
|
||||
`-W`),
|
||||
- the render writes into a sibling temp-file then `os.replace()`
|
||||
(atomic) so a failed/torn render can never clobber a good
|
||||
committed SVG,
|
||||
- a per-build `_seen_outputs` map (reset on `builder-inited`)
|
||||
errors on a same-stem output collision.
|
||||
|
||||
Verified: a corrupted `.d2` errors under `-W` (exit 1) with the
|
||||
committed SVG byte-unchanged; the collision guard fires on a
|
||||
duplicate stem; `nix run nixpkgs#d2` render path + atomic swap
|
||||
leave no temp residue; `ruff` clean.
|
||||
|
||||
## Response summary (cont.) — Files changed
|
||||
|
||||
- `docs/_ext/d2diagrams.py` — tristate render result, atomic
|
||||
temp-file render, output-collision guard, doc-string policy
|
||||
update.
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated (`3a5cfde0`). The user noted
|
||||
follow-up refinements may come in later commits.
|
||||
|
|
@ -1,35 +0,0 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-25T23:25:06Z
|
||||
git_ref: 3a5cfde0
|
||||
diff_cmd: git diff 3a5cfde0~1..3a5cfde0
|
||||
---
|
||||
|
||||
# Raw output pointers (diff-ref mode)
|
||||
|
||||
The generated patch is committed; per the diff-ref decision
|
||||
rule the verbatim content is the diff itself:
|
||||
|
||||
> `git diff 3a5cfde0~1..3a5cfde0 -- docs/_ext/d2diagrams.py`
|
||||
|
||||
## Generation notes (non-code, verbatim)
|
||||
|
||||
- `RenderResult` enum added; `render_svg()` return type changed
|
||||
`bool -> RenderResult`. Mapping in `run()`: `FAILED` ->
|
||||
`state_machine.reporter.error(...)`, `NO_OUTPUT` -> raw `.d2`
|
||||
source as a `literal_block`, `OK`/`FELL_BACK` -> `image`.
|
||||
- Atomicity via `tempfile.mkstemp(dir=out.parent, ...)` +
|
||||
`os.replace(tmp, out)`; temp unlinked on any failure path.
|
||||
- Collision guard: module-level `_seen_outputs: dict[str,str]`
|
||||
keyed by output basename, cleared by a `builder-inited`
|
||||
handler connected in `setup()`.
|
||||
- `parallel_read_safe`/`parallel_write_safe` kept `True` but
|
||||
now justified by the atomic swap (documented inline).
|
||||
- Empirical verification performed before commit:
|
||||
- broken `.d2` + `sphinx-build -W` -> exit 1, ERROR node
|
||||
rendered, committed `actor_tree.svg` md5 unchanged,
|
||||
- duplicate-stem orphan page -> "d2 output collision" error,
|
||||
- `nix run nixpkgs#d2` forced re-render -> deterministic
|
||||
(no git drift), no `.tmp` residue,
|
||||
- `ruff check` clean.
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
# AI Prompt I/O Log — claude
|
||||
|
||||
This directory tracks prompt inputs and model
|
||||
outputs for AI-assisted development using
|
||||
`claude` (Claude Code).
|
||||
|
||||
## Policy
|
||||
|
||||
Prompt logging follows the
|
||||
[NLNet generative AI policy][nlnet-ai].
|
||||
All substantive AI contributions are logged
|
||||
with:
|
||||
- Model name and version
|
||||
- Timestamps
|
||||
- The prompts that produced the output
|
||||
- Unedited model output (`.raw.md` files)
|
||||
|
||||
[nlnet-ai]: https://nlnet.nl/foundation/policies/generativeAI/
|
||||
|
||||
## Usage
|
||||
|
||||
Entries are created by the `/prompt-io` skill
|
||||
or automatically via `/commit-msg` integration.
|
||||
|
||||
Human contributors remain accountable for all
|
||||
code decisions. AI-generated content is never
|
||||
presented as human-authored work.
|
||||
|
|
@ -1,76 +0,0 @@
|
|||
ok now i want you to take a look at the most recent commit adding
|
||||
a `tpt_bind_addrs` to `open_root_actor()` and extend the existing
|
||||
tests/discovery/test_multiaddr* and friends to use this new param in
|
||||
at least one suite with parametrizations over,
|
||||
|
||||
- `registry_addrs == tpt_bind_addrs`, as in both inputs are the same.
|
||||
- `set(registry_addrs) >= set(tpt_bind_addrs)`, as in the registry
|
||||
addrs include the bind set.
|
||||
- `registry_addrs != tpt_bind_addrs`, where the reg set is disjoint from
|
||||
the bind set in all possible combos you can imagine.
|
||||
|
||||
All of the ^above cases should further be parametrized over,
|
||||
- the root being the registrar,
|
||||
- a non-registrar root using our bg `daemon` fixture.
|
||||
|
||||
once we have a fairly thorough test suite and have flushed out all
|
||||
bugs and edge cases we want to design a wrapping API which allows
|
||||
declaring full tree's of actors tpt endpoints using multiaddrs such
|
||||
that a `dict[str, list[str]]` of actor-name -> multiaddr can be used
|
||||
to configure a tree of actors-as-services given such an input
|
||||
"endpoints-table" can be matched with the number of appropriately
|
||||
named subactore spawns in a `tractor` user-app.
|
||||
|
||||
Here is a small example from piker,
|
||||
|
||||
- in piker's root conf.toml we define a `[network]` section which can
|
||||
define various actor-service-daemon names set to a maddr
|
||||
(multiaddress str).
|
||||
|
||||
- each actor whether part of the `pikerd` tree (as a sub) or spawned
|
||||
in other non-registrar rooted trees (such as `piker chart`) should
|
||||
configurable in terms of its `tractor` tpt bind addresses via
|
||||
a simple service lookup table,
|
||||
|
||||
```toml
|
||||
[network]
|
||||
pikerd = [
|
||||
'/ip4/127.0.0.1/tcp/6116', # std localhost daemon-actor tree
|
||||
'/uds/run/user/1000/piker/pikerd@6116.sock', # same but serving UDS
|
||||
]
|
||||
chart = [
|
||||
'/ip4/127.0.0.1/tcp/3333', # std localhost daemon-actor tree
|
||||
'/uds/run/user/1000/piker/chart@3333.sock',
|
||||
]
|
||||
```
|
||||
|
||||
We should take whatever common API is needed to support this and
|
||||
distill it into a
|
||||
```python
|
||||
tractor.discovery.parse_endpoints(
|
||||
) -> dict[
|
||||
str,
|
||||
list[Address]
|
||||
|dict[str, list[Address]]
|
||||
# ^recursive case, see below
|
||||
]:
|
||||
```
|
||||
|
||||
style API which can,
|
||||
|
||||
- be re-used easily across dependent projects.
|
||||
- correctly raise tpt-backend support errors when a maddr specifying
|
||||
a unsupport proto is passed.
|
||||
- be used to handle "tunnelled" maddrs per
|
||||
https://github.com/multiformats/py-multiaddr/#tunneling such that
|
||||
for any such tunneled maddr-`str`-entry we deliver a data-structure
|
||||
which can easily be passed to nested `@acm`s which consecutively
|
||||
setup nested net bindspaces for binding the endpoint addrs using
|
||||
a combo of our `.ipc.*` machinery and, say for example something like
|
||||
https://github.com/svinota/pyroute2, more precisely say for
|
||||
managing tunnelled wireguard eps within network-namespaces,
|
||||
* https://docs.pyroute2.org/
|
||||
* https://docs.pyroute2.org/netns.html
|
||||
|
||||
remember to include use of all default `.claude/skills` throughout
|
||||
this work!
|
||||
|
|
@ -1,34 +0,0 @@
|
|||
This is your first big boi, "from GH issue" design, plan and
|
||||
implement task.
|
||||
|
||||
We need to try and add sub-interpreter (aka subint) support per the
|
||||
issue,
|
||||
|
||||
https://github.com/goodboy/tractor/issues/379
|
||||
|
||||
Part of this work should include,
|
||||
|
||||
- modularizing and thus better organizing the `.spawn.*` subpkg by
|
||||
breaking up various backends currently in `spawn._spawn` into
|
||||
separate submods where it makes sense.
|
||||
|
||||
- add a new `._subint` backend which tries to keep as much of the
|
||||
inter-process-isolation machinery in use as possible but with plans
|
||||
to optimize for localhost only benefits as offered by python's
|
||||
subints where possible.
|
||||
|
||||
* utilizing localhost-only tpts like UDS, shm-buffers for
|
||||
performant IPC between subactors but also leveraging the benefits from
|
||||
the traditional OS subprocs mem/storage-domain isolation, linux
|
||||
namespaces where possible and as available/permitted by whatever
|
||||
is happening under the hood with how cpython implements subints.
|
||||
|
||||
* default configuration should encourage state isolation as with
|
||||
subprocs, but explicit public escape hatches to enable rigorously
|
||||
managed shm channels for high performance apps.
|
||||
|
||||
- all tests should be (able to be) parameterized to use the new
|
||||
`subints` backend and enabled by flag in the harness using the
|
||||
existing `pytest --spawn-backend <spawn-backend>` support offered in
|
||||
the `open_root_actor()` and `.testing._pytest` harness override
|
||||
fixture.
|
||||
|
|
@ -1,159 +0,0 @@
|
|||
# Logging-spec leaf-module granularity — "Route B" (decouple
|
||||
# logger-*identity* from console-*display*)
|
||||
|
||||
Follow-up notes recording the breaking-changes / costs of the
|
||||
deeper fix that would give the `tractor.log` logging-spec (see
|
||||
`LogSpec`/`apply_logspec()`) true **per-leaf-MODULE** level
|
||||
control — deliberately *not* taken (for now) in favour of the
|
||||
smaller sub-PACKAGE fix already landed.
|
||||
|
||||
## Status / what already shipped
|
||||
|
||||
The cheap, contained fix is **done**: `get_logger()`'s "strip
|
||||
#2" (`log.py`, the `pkg_path = subpkg_path` collapse) no longer
|
||||
eats a real sub-package component. It now strips the trailing
|
||||
token *only* when it duplicates the caller's leaf-*module*
|
||||
filename (which the header already shows via `{filename}`).
|
||||
|
||||
Result:
|
||||
|
||||
- `devx.debug` resolves to `tractor.devx.debug`, **distinct**
|
||||
from a bare `devx` -> `tractor.devx` (its parent). So the
|
||||
logging-spec can dial sub-package levels at any nesting depth
|
||||
(`devx.debug:runtime` ≠ `devx:cancel`).
|
||||
- The `get_logger(__name__)` cosmetic ("don't repeat the leaf
|
||||
module in `{name}` since `{filename}` shows it") is preserved.
|
||||
|
||||
What is **still NOT addressable** after that fix:
|
||||
|
||||
- **Per-leaf-MODULE** levels. Every module in a (sub-)pkg shares
|
||||
that pkg's logger, because `get_logger()` drops the leaf
|
||||
module-name from the logger key by design.
|
||||
- **Top-level lib modules** (eg. `tractor.to_asyncio`,
|
||||
`__package__ == 'tractor'`) emit on the *root* `tractor`
|
||||
logger, so a `to_asyncio:<lvl>` spec entry hits a phantom
|
||||
child -> no-op.
|
||||
|
||||
## What "Route B" is
|
||||
|
||||
Make the logger's *identity* the **full dotted module path**
|
||||
(incl. the leaf module + top-level modules), eg.
|
||||
`tractor.devx.debug._tty_lock` and `tractor.to_asyncio`, and
|
||||
move the cosmetic leaf-trim out of logger-naming and into the
|
||||
**formatter's `{name}` rendering**.
|
||||
|
||||
Net effect:
|
||||
|
||||
- Real per-module `Logger` nodes exist in the hierarchy ->
|
||||
the spec can target ANY module; stdlib level-inheritance and
|
||||
propagation "just work" top-down.
|
||||
- Console headers stay clean because the formatter computes a
|
||||
trimmed display string (drop the trailing token that equals
|
||||
`{filename}`'s stem) instead of the logger doing it.
|
||||
|
||||
## Why it's "broad" — breaking changes / costs
|
||||
|
||||
The logger *name* is currently load-bearing well beyond
|
||||
display; changing it ripples:
|
||||
|
||||
1. **Every logger name changes.**
|
||||
Today (post sub-pkg fix) names collapse to the sub-package;
|
||||
Route B = full module path. This touches:
|
||||
- handler attachment points + the `getChild()` hierarchy,
|
||||
- any `logging.getLogger('tractor.X')` string lookups,
|
||||
- any name-based filtering,
|
||||
- the dedup / `_strict_debug` warning logic *inside*
|
||||
`get_logger()` itself — the `pkg_name in name`,
|
||||
`leaf_mod in pkg_path`, "duplicate pkg-name" branches all
|
||||
key off the *name shape* and would need re-derivation.
|
||||
|
||||
2. **Formatter rewrite.**
|
||||
`LOG_FORMAT` uses `{name}` == `record.name` (the full logger
|
||||
name). To keep headers clean we must compute a *display*
|
||||
name and inject it as a record attr (eg. `record.pkg_ns`)
|
||||
via a `logging.Filter` or a `colorlog.ColoredFormatter`
|
||||
subclass overriding `.format()`, then point `LOG_FORMAT` at
|
||||
that field. The `{filename}` vs `{name}` de-dup intent has
|
||||
to be re-implemented per-record rather than per-logger.
|
||||
|
||||
3. **Propagation / double-emit surface grows.**
|
||||
Full-depth loggers mean more intermediate nodes
|
||||
(`...debug._tty_lock` -> `.debug` -> `.devx` -> `tractor`).
|
||||
If more than one level carries a handler (spec sub-handlers
|
||||
+ a root console), records double-emit. The
|
||||
`propagate=False` trick we already use for filter-targeted
|
||||
sub-loggers (`apply_logspec()`) must be applied carefully
|
||||
across a deeper tree — more levels == more places to leak a
|
||||
dup.
|
||||
|
||||
4. **Level-inheritance semantics shift.**
|
||||
Today setting a level on `tractor.devx` gates *all* devx
|
||||
emits (they share that logger). Post-Route-B,
|
||||
`tractor.devx.debug._tty_lock` is its own `NOTSET` logger
|
||||
that *inherits* the effective level from ancestors —
|
||||
functionally similar via inheritance, BUT any code that does
|
||||
`log.setLevel(...)` / reads `log.level` on a (previously
|
||||
collapsed) logger now only affects that exact node. All
|
||||
`setLevel`/`.level =` call sites need an audit (eg.
|
||||
`get_logger()`'s own `log.level = rlog.level` line).
|
||||
|
||||
5. **Downstream contract churn.**
|
||||
`modden` / `piker` call `get_logger()` / `get_console_log()`
|
||||
and may depend on current names — including
|
||||
`modden.runtime.daemon.setup_tractor_logging()` which
|
||||
asserts `'tractor' not in name` on spec parts. The header
|
||||
`{name}` field is user-visible in everyone's logs + CI
|
||||
output. Changing the canonical names is a public-ish
|
||||
behavior change -> needs a version note + downstream
|
||||
coordination (or a formatter trim that keeps the *displayed*
|
||||
string byte-identical to today).
|
||||
|
||||
6. **`get_logger()` refactor risk.**
|
||||
The fn tangles two concerns: compute logger *identity* and
|
||||
compute the *display* string. Route B forces splitting them
|
||||
inside a ~300-line fn with multiple `_strict_debug`
|
||||
branches, dup-warnings, and the `name=__name__` convenience.
|
||||
High chance of subtle regressions without an exhaustive
|
||||
name-derivation test matrix.
|
||||
|
||||
## Migration / test plan (if pursued)
|
||||
|
||||
- Extract a pure helper
|
||||
`_mk_logger_name(pkg_name, mod_name, mod_pkg) -> (logger_name,
|
||||
display_name)` and cover it with an exhaustive unit matrix:
|
||||
auto vs explicit vs `__name__`; package-`__init__` vs leaf
|
||||
module; nested vs flat; `pkg_name in name` vs not; top-level
|
||||
module (`__package__ == pkg_name`).
|
||||
- Switch `get_logger()` to use it for *identity*; switch the
|
||||
formatter to use `display_name` (via a record attr).
|
||||
- Re-run the full suite + golden-diff a sample of rendered log
|
||||
headers to confirm zero cosmetic churn.
|
||||
- Coordinate the name change with `modden`/`piker`; bump +
|
||||
CHANGES note.
|
||||
|
||||
## Cheaper alternative — "Route A" (record-filter)
|
||||
|
||||
If per-leaf control is wanted *before* committing to Route B:
|
||||
keep names collapsed, add a `logging.Filter` on the configured
|
||||
handler keyed on `record.module` / `record.pathname` that maps
|
||||
each record's source module -> its spec level. Set the base
|
||||
logger to the *minimum* level in the spec (so records aren't
|
||||
pre-dropped by the logger), and let the filter discriminate
|
||||
up/down within that floor.
|
||||
|
||||
- Pros: no name churn, no formatter change, fully contained
|
||||
next to `apply_logspec()`.
|
||||
- Cons: a filter can only discriminate *within* what the logger
|
||||
admits -> base must be permissive, so `at_least_level()`
|
||||
expensive-work guards over-admit; matching dotted spec names
|
||||
to a `pathname` is fiddly; doesn't clean up the hierarchy
|
||||
itself.
|
||||
|
||||
## Recommendation
|
||||
|
||||
- Defer Route B unless true per-module loggers are wanted as a
|
||||
first-class feature.
|
||||
- If per-leaf control is needed soon, prefer **Route A**
|
||||
(filter) — lower risk.
|
||||
- The shipped sub-PACKAGE fix already covers the common ask
|
||||
(`devx.debug` vs `devx`).
|
||||
602
docs/README.rst
602
docs/README.rst
|
|
@ -1,4 +1,4 @@
|
|||
|logo| ``tractor``: distributed structured concurrency
|
||||
|logo| ``tractor``: distributed structurred concurrency
|
||||
|
||||
``tractor`` is a `structured concurrency`_ (SC), multi-processing_ runtime built on trio_.
|
||||
|
||||
|
|
@ -23,11 +23,16 @@ model" looks like, and that's **intentional**.
|
|||
|
||||
Where do i start!?
|
||||
------------------
|
||||
New to ``trio`` and **structured concurrency**? Our docs collect the
|
||||
best starting points and then walk you straight into a hands-on
|
||||
quickstart:
|
||||
The first step to grok ``tractor`` is to get an intermediate
|
||||
knowledge of ``trio`` and **structured concurrency** B)
|
||||
|
||||
Some great places to start are,
|
||||
|
||||
- the seminal `blog post`_
|
||||
- obviously the `trio docs`_
|
||||
- wikipedia's nascent SC_ page
|
||||
- the fancy diagrams @ libdill-docs_
|
||||
|
||||
https://goodboy.github.io/tractor/start/quickstart.html
|
||||
|
||||
Features
|
||||
--------
|
||||
|
|
@ -52,7 +57,6 @@ Features
|
|||
`discovery`_ sys with plans to support multiple `modern protocol`_
|
||||
approaches.
|
||||
- Various ``trio`` extension APIs via ``tractor.trionics`` such as,
|
||||
|
||||
- task fan-out `broadcasting`_,
|
||||
- multi-task-single-resource-caching and fan-out-to-multi
|
||||
``__aenter__()`` APIs for ``@acm`` functions,
|
||||
|
|
@ -94,7 +98,7 @@ the code base::
|
|||
|
||||
# but @goodboy prefers the more explicit (and shell agnostic)
|
||||
# https://docs.astral.sh/uv/configuration/environment/#uv_project_environment
|
||||
UV_PROJECT_ENVIRONMENT="tractor_py313"
|
||||
UV_PROJECT_ENVIRONMENT="tractor_py313
|
||||
|
||||
# hint hint, enter @goodboy's fave shell B)
|
||||
uv run --dev xonsh
|
||||
|
|
@ -106,53 +110,545 @@ Alongside all this we ofc offer "releases" on PyPi::
|
|||
Just note that YMMV since the main git branch is often much further
|
||||
ahead then any latest release.
|
||||
|
||||
Hacking on the docs themselves? The build + live-preview one-liners
|
||||
(incl. nix-shell specifics) are collected in `notes_to_self/howtodocs.md
|
||||
<https://github.com/goodboy/tractor/blob/main/notes_to_self/howtodocs.md>`_,
|
||||
and rendered as the "Building these docs" section of our dev-tips guide.
|
||||
|
||||
|
||||
Example codez
|
||||
-------------
|
||||
We prefer to point you at the runnable scripts under ``examples/``
|
||||
- each is CI-run and ``literalinclude``-d straight into the docs, so
|
||||
what you read there is what actually runs - rather than inline a
|
||||
pile of them here. The one-minute pitch: spawn a subactor per core,
|
||||
open a ``Context`` into each, then crash the root *on purpose* and
|
||||
watch the runtime reap the whole tree - zero zombies, guaranteed (if
|
||||
you can make a zombie child without a system signal, it **is a
|
||||
bug**).
|
||||
In ``tractor``'s (very lacking) documention we prefer to point to
|
||||
example scripts in the repo over duplicating them in docs, but with
|
||||
that in mind here are some definitive snippets to try and hook you
|
||||
into digging deeper.
|
||||
|
||||
See it run - plus the full tour (the flagship multi-process
|
||||
debugger, bidirectional streaming over a ``Context``, cancellation,
|
||||
discovery, "infected ``asyncio``", typed messaging and worker-pool /
|
||||
cluster patterns) - in the docs:
|
||||
|
||||
- docs: https://goodboy.github.io/tractor/
|
||||
- examples: https://github.com/goodboy/tractor/tree/main/examples
|
||||
Run a func in a process
|
||||
***********************
|
||||
Use ``trio``'s style of focussing on *tasks as functions*:
|
||||
|
||||
.. code:: python
|
||||
|
||||
"""
|
||||
Run with a process monitor from a terminal using::
|
||||
|
||||
$TERM -e watch -n 0.1 "pstree -a $$" \
|
||||
& python examples/parallelism/single_func.py \
|
||||
&& kill $!
|
||||
|
||||
"""
|
||||
import os
|
||||
|
||||
import tractor
|
||||
import trio
|
||||
|
||||
|
||||
async def burn_cpu():
|
||||
|
||||
pid = os.getpid()
|
||||
|
||||
# burn a core @ ~ 50kHz
|
||||
for _ in range(50000):
|
||||
await trio.sleep(1/50000/50)
|
||||
|
||||
return os.getpid()
|
||||
|
||||
|
||||
async def main():
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
|
||||
portal = await n.run_in_actor(burn_cpu)
|
||||
|
||||
# burn rubber in the parent too
|
||||
await burn_cpu()
|
||||
|
||||
# wait on result from target function
|
||||
pid = await portal.result()
|
||||
|
||||
# end of nursery block
|
||||
print(f"Collected subproc {pid}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
trio.run(main)
|
||||
|
||||
|
||||
This runs ``burn_cpu()`` in a new process and reaps it on completion
|
||||
of the nursery block.
|
||||
|
||||
If you only need to run a sync function and retreive a single result, you
|
||||
might want to check out `trio-parallel`_.
|
||||
|
||||
|
||||
Zombie safe: self-destruct a process tree
|
||||
*****************************************
|
||||
``tractor`` tries to protect you from zombies, no matter what.
|
||||
|
||||
.. code:: python
|
||||
|
||||
"""
|
||||
Run with a process monitor from a terminal using::
|
||||
|
||||
$TERM -e watch -n 0.1 "pstree -a $$" \
|
||||
& python examples/parallelism/we_are_processes.py \
|
||||
&& kill $!
|
||||
|
||||
"""
|
||||
from multiprocessing import cpu_count
|
||||
import os
|
||||
|
||||
import tractor
|
||||
import trio
|
||||
|
||||
|
||||
async def target():
|
||||
print(
|
||||
f"Yo, i'm '{tractor.current_actor().name}' "
|
||||
f"running in pid {os.getpid()}"
|
||||
)
|
||||
|
||||
await trio.sleep_forever()
|
||||
|
||||
|
||||
async def main():
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
|
||||
for i in range(cpu_count()):
|
||||
await n.run_in_actor(target, name=f'worker_{i}')
|
||||
|
||||
print('This process tree will self-destruct in 1 sec...')
|
||||
await trio.sleep(1)
|
||||
|
||||
# raise an error in root actor/process and trigger
|
||||
# reaping of all minions
|
||||
raise Exception('Self Destructed')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try:
|
||||
trio.run(main)
|
||||
except Exception:
|
||||
print('Zombies Contained')
|
||||
|
||||
|
||||
If you can create zombie child processes (without using a system signal)
|
||||
it **is a bug**.
|
||||
|
||||
|
||||
"Native" multi-process debugging
|
||||
********************************
|
||||
Using the magic of `pdbp`_ and our internal IPC, we've
|
||||
been able to create a native feeling debugging experience for
|
||||
any (sub-)process in your ``tractor`` tree.
|
||||
|
||||
.. code:: python
|
||||
|
||||
from os import getpid
|
||||
|
||||
import tractor
|
||||
import trio
|
||||
|
||||
|
||||
async def breakpoint_forever():
|
||||
"Indefinitely re-enter debugger in child actor."
|
||||
while True:
|
||||
yield 'yo'
|
||||
await tractor.breakpoint()
|
||||
|
||||
|
||||
async def name_error():
|
||||
"Raise a ``NameError``"
|
||||
getattr(doggypants)
|
||||
|
||||
|
||||
async def main():
|
||||
"""Test breakpoint in a streaming actor.
|
||||
"""
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
loglevel='error',
|
||||
) as n:
|
||||
|
||||
p0 = await n.start_actor('bp_forever', enable_modules=[__name__])
|
||||
p1 = await n.start_actor('name_error', enable_modules=[__name__])
|
||||
|
||||
# retreive results
|
||||
stream = await p0.run(breakpoint_forever)
|
||||
await p1.run(name_error)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
trio.run(main)
|
||||
|
||||
|
||||
You can run this with::
|
||||
|
||||
>>> python examples/debugging/multi_daemon_subactors.py
|
||||
|
||||
And, yes, there's a built-in crash handling mode B)
|
||||
|
||||
We're hoping to add a respawn-from-repl system soon!
|
||||
|
||||
|
||||
SC compatible bi-directional streaming
|
||||
**************************************
|
||||
Yes, you saw it here first; we provide 2-way streams
|
||||
with reliable, transitive setup/teardown semantics.
|
||||
|
||||
Our nascent api is remniscent of ``trio.Nursery.start()``
|
||||
style invocation:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import trio
|
||||
import tractor
|
||||
|
||||
|
||||
@tractor.context
|
||||
async def simple_rpc(
|
||||
|
||||
ctx: tractor.Context,
|
||||
data: int,
|
||||
|
||||
) -> None:
|
||||
'''Test a small ping-pong 2-way streaming server.
|
||||
|
||||
'''
|
||||
# signal to parent that we're up much like
|
||||
# ``trio_typing.TaskStatus.started()``
|
||||
await ctx.started(data + 1)
|
||||
|
||||
async with ctx.open_stream() as stream:
|
||||
|
||||
count = 0
|
||||
async for msg in stream:
|
||||
|
||||
assert msg == 'ping'
|
||||
await stream.send('pong')
|
||||
count += 1
|
||||
|
||||
else:
|
||||
assert count == 10
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
|
||||
portal = await n.start_actor(
|
||||
'rpc_server',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
|
||||
# XXX: this syntax requires py3.9
|
||||
async with (
|
||||
|
||||
portal.open_context(
|
||||
simple_rpc,
|
||||
data=10,
|
||||
) as (ctx, sent),
|
||||
|
||||
ctx.open_stream() as stream,
|
||||
):
|
||||
|
||||
assert sent == 11
|
||||
|
||||
count = 0
|
||||
# receive msgs using async for style
|
||||
await stream.send('ping')
|
||||
|
||||
async for msg in stream:
|
||||
assert msg == 'pong'
|
||||
await stream.send('ping')
|
||||
count += 1
|
||||
|
||||
if count >= 9:
|
||||
break
|
||||
|
||||
|
||||
# explicitly teardown the daemon-actor
|
||||
await portal.cancel_actor()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
trio.run(main)
|
||||
|
||||
|
||||
See original proposal and discussion in `#53`_ as well
|
||||
as follow up improvements in `#223`_ that we'd love to
|
||||
hear your thoughts on!
|
||||
|
||||
.. _#53: https://github.com/goodboy/tractor/issues/53
|
||||
.. _#223: https://github.com/goodboy/tractor/issues/223
|
||||
|
||||
|
||||
Worker poolz are easy peasy
|
||||
***************************
|
||||
The initial ask from most new users is *"how do I make a worker
|
||||
pool thing?"*.
|
||||
|
||||
``tractor`` is built to handle any SC (structured concurrent) process
|
||||
tree you can imagine; a "worker pool" pattern is a trivial special
|
||||
case.
|
||||
|
||||
We have a `full worker pool re-implementation`_ of the std-lib's
|
||||
``concurrent.futures.ProcessPoolExecutor`` example for reference.
|
||||
|
||||
You can run it like so (from this dir) to see the process tree in
|
||||
real time::
|
||||
|
||||
$TERM -e watch -n 0.1 "pstree -a $$" \
|
||||
& python examples/parallelism/concurrent_actors_primes.py \
|
||||
&& kill $!
|
||||
|
||||
This uses no extra threads, fancy semaphores or futures; all we need
|
||||
is ``tractor``'s IPC!
|
||||
|
||||
"Infected ``asyncio``" mode
|
||||
***************************
|
||||
Have a bunch of ``asyncio`` code you want to force to be SC at the process level?
|
||||
|
||||
Check out our experimental system for `guest`_-mode controlled
|
||||
``asyncio`` actors:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import asyncio
|
||||
from statistics import mean
|
||||
import time
|
||||
|
||||
import trio
|
||||
import tractor
|
||||
|
||||
|
||||
async def aio_echo_server(
|
||||
to_trio: trio.MemorySendChannel,
|
||||
from_trio: asyncio.Queue,
|
||||
) -> None:
|
||||
|
||||
# a first message must be sent **from** this ``asyncio``
|
||||
# task or the ``trio`` side will never unblock from
|
||||
# ``tractor.to_asyncio.open_channel_from():``
|
||||
to_trio.send_nowait('start')
|
||||
|
||||
# XXX: this uses an ``from_trio: asyncio.Queue`` currently but we
|
||||
# should probably offer something better.
|
||||
while True:
|
||||
# echo the msg back
|
||||
to_trio.send_nowait(await from_trio.get())
|
||||
await asyncio.sleep(0)
|
||||
|
||||
|
||||
@tractor.context
|
||||
async def trio_to_aio_echo_server(
|
||||
ctx: tractor.Context,
|
||||
):
|
||||
# this will block until the ``asyncio`` task sends a "first"
|
||||
# message.
|
||||
async with tractor.to_asyncio.open_channel_from(
|
||||
aio_echo_server,
|
||||
) as (first, chan):
|
||||
|
||||
assert first == 'start'
|
||||
await ctx.started(first)
|
||||
|
||||
async with ctx.open_stream() as stream:
|
||||
|
||||
async for msg in stream:
|
||||
await chan.send(msg)
|
||||
|
||||
out = await chan.receive()
|
||||
# echo back to parent actor-task
|
||||
await stream.send(out)
|
||||
|
||||
|
||||
async def main():
|
||||
|
||||
async with tractor.open_nursery() as n:
|
||||
p = await n.start_actor(
|
||||
'aio_server',
|
||||
enable_modules=[__name__],
|
||||
infect_asyncio=True,
|
||||
)
|
||||
async with p.open_context(
|
||||
trio_to_aio_echo_server,
|
||||
) as (ctx, first):
|
||||
|
||||
assert first == 'start'
|
||||
|
||||
count = 0
|
||||
async with ctx.open_stream() as stream:
|
||||
|
||||
delays = []
|
||||
send = time.time()
|
||||
|
||||
await stream.send(count)
|
||||
async for msg in stream:
|
||||
recv = time.time()
|
||||
delays.append(recv - send)
|
||||
assert msg == count
|
||||
count += 1
|
||||
send = time.time()
|
||||
await stream.send(count)
|
||||
|
||||
if count >= 1e3:
|
||||
break
|
||||
|
||||
print(f'mean round trip rate (Hz): {1/mean(delays)}')
|
||||
await p.cancel_actor()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
trio.run(main)
|
||||
|
||||
|
||||
Yes, we spawn a python process, run ``asyncio``, start ``trio`` on the
|
||||
``asyncio`` loop, then send commands to the ``trio`` scheduled tasks to
|
||||
tell ``asyncio`` tasks what to do XD
|
||||
|
||||
We need help refining the `asyncio`-side channel API to be more
|
||||
`trio`-like. Feel free to sling your opinion in `#273`_!
|
||||
|
||||
|
||||
.. _#273: https://github.com/goodboy/tractor/issues/273
|
||||
|
||||
|
||||
Higher level "cluster" APIs
|
||||
***************************
|
||||
To be extra terse the ``tractor`` devs have started hacking some "higher
|
||||
level" APIs for managing actor trees/clusters. These interfaces should
|
||||
generally be condsidered provisional for now but we encourage you to try
|
||||
them and provide feedback. Here's a new API that let's you quickly
|
||||
spawn a flat cluster:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import trio
|
||||
import tractor
|
||||
|
||||
|
||||
async def sleepy_jane():
|
||||
uid = tractor.current_actor().uid
|
||||
print(f'Yo i am actor {uid}')
|
||||
await trio.sleep_forever()
|
||||
|
||||
|
||||
async def main():
|
||||
'''
|
||||
Spawn a flat actor cluster, with one process per
|
||||
detected core.
|
||||
|
||||
'''
|
||||
portal_map: dict[str, tractor.Portal]
|
||||
results: dict[str, str]
|
||||
|
||||
# look at this hip new syntax!
|
||||
async with (
|
||||
|
||||
tractor.open_actor_cluster(
|
||||
modules=[__name__]
|
||||
) as portal_map,
|
||||
|
||||
trio.open_nursery() as n,
|
||||
):
|
||||
|
||||
for (name, portal) in portal_map.items():
|
||||
n.start_soon(portal.run, sleepy_jane)
|
||||
|
||||
await trio.sleep(0.5)
|
||||
|
||||
# kill the cluster with a cancel
|
||||
raise KeyboardInterrupt
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try:
|
||||
trio.run(main)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
|
||||
|
||||
.. _full worker pool re-implementation: https://github.com/goodboy/tractor/blob/master/examples/parallelism/concurrent_actors_primes.py
|
||||
|
||||
|
||||
Under the hood
|
||||
--------------
|
||||
``tractor`` is an attempt to pair trionic_ `structured
|
||||
concurrency`_ with distributed Python - think of it as ``trio``
|
||||
*-across-processes*, or as an opinionated replacement for the
|
||||
stdlib's ``multiprocessing`` built on async primitives from the
|
||||
ground up. But really it **is just** ``trio``: nurseries that
|
||||
spawn *processes* and cancel-able streaming IPC between them. If
|
||||
you can drive ``trio``, you can drive ``tractor``.
|
||||
``tractor`` is an attempt to pair trionic_ `structured concurrency`_ with
|
||||
distributed Python. You can think of it as a ``trio``
|
||||
*-across-processes* or simply as an opinionated replacement for the
|
||||
stdlib's ``multiprocessing`` but built on async programming primitives
|
||||
from the ground up.
|
||||
|
||||
"But wait - don't 'actors' have mailboxes and messages and
|
||||
stuff?!" Well, we've got (well referenced) opinions on what an "actor
|
||||
model" **actually is** (tl;dr: the `3 axioms`_, not the cultural
|
||||
baggage) - that whole riff lives in our docs:
|
||||
Don't be scared off by this description. ``tractor`` **is just** ``trio``
|
||||
but with nurseries for process management and cancel-able streaming IPC.
|
||||
If you understand how to work with ``trio``, ``tractor`` will give you
|
||||
the parallelism you may have been needing.
|
||||
|
||||
https://goodboy.github.io/tractor/explain/sc-distributed.html#hold-up-is-this-an-actor-model
|
||||
|
||||
What's on the TODO
|
||||
------------------
|
||||
The roadmap lives with our docs - see `what the future holds
|
||||
<https://goodboy.github.io/tractor/project/index.html#what-the-future-holds>`_
|
||||
for where ``tractor`` is headed.
|
||||
Wait, huh?! I thought "actors" have messages, and mailboxes and stuff?!
|
||||
***********************************************************************
|
||||
Let's stop and ask how many canon actor model papers have you actually read ;)
|
||||
|
||||
From our experience many "actor systems" aren't really "actor models"
|
||||
since they **don't adhere** to the `3 axioms`_ and pay even less
|
||||
attention to the problem of *unbounded non-determinism* (which was the
|
||||
whole point for creation of the model in the first place).
|
||||
|
||||
From the author's mouth, **the only thing required** is `adherance to`_
|
||||
the `3 axioms`_, *and that's it*.
|
||||
|
||||
``tractor`` adheres to said base requirements of an "actor model"::
|
||||
|
||||
In response to a message, an actor may:
|
||||
|
||||
- send a finite number of new messages
|
||||
- create a finite number of new actors
|
||||
- designate a new behavior to process subsequent messages
|
||||
|
||||
|
||||
**and** requires *no further api changes* to accomplish this.
|
||||
|
||||
If you want do debate this further please feel free to chime in on our
|
||||
chat or discuss on one of the following issues *after you've read
|
||||
everything in them*:
|
||||
|
||||
- https://github.com/goodboy/tractor/issues/210
|
||||
- https://github.com/goodboy/tractor/issues/18
|
||||
|
||||
|
||||
Let's clarify our parlance
|
||||
**************************
|
||||
Whether or not ``tractor`` has "actors" underneath should be mostly
|
||||
irrelevant to users other then for referring to the interactions of our
|
||||
primary runtime primitives: each Python process + ``trio.run()``
|
||||
+ surrounding IPC machinery. These are our high level, base
|
||||
*runtime-units-of-abstraction* which both *are* (as much as they can
|
||||
be in Python) and will be referred to as our *"actors"*.
|
||||
|
||||
The main goal of ``tractor`` is is to allow for highly distributed
|
||||
software that, through the adherence to *structured concurrency*,
|
||||
results in systems which fail in predictable, recoverable and maybe even
|
||||
understandable ways; being an "actor model" is just one way to describe
|
||||
properties of the system.
|
||||
|
||||
|
||||
What's on the TODO:
|
||||
-------------------
|
||||
Help us push toward the future of distributed `Python`.
|
||||
|
||||
- Erlang-style supervisors via composed context managers (see `#22
|
||||
<https://github.com/goodboy/tractor/issues/22>`_)
|
||||
- Typed messaging protocols (ex. via ``msgspec.Struct``, see `#36
|
||||
<https://github.com/goodboy/tractor/issues/36>`_)
|
||||
- Typed capability-based (dialog) protocols ( see `#196
|
||||
<https://github.com/goodboy/tractor/issues/196>`_ with draft work
|
||||
started in `#311 <https://github.com/goodboy/tractor/pull/311>`_)
|
||||
- We **recently disabled CI-testing on windows** and need help getting
|
||||
it running again! (see `#327
|
||||
<https://github.com/goodboy/tractor/pull/327>`_). **We do have windows
|
||||
support** (and have for quite a while) but since no active hacker
|
||||
exists in the user-base to help test on that OS, for now we're not
|
||||
actively maintaining testing due to the added hassle and general
|
||||
latency..
|
||||
|
||||
|
||||
Feel like saying hi?
|
||||
--------------------
|
||||
|
|
@ -163,24 +659,36 @@ say hi, please feel free to reach us in our `matrix channel`_. If
|
|||
matrix seems too hip, we're also mostly all in the the `trio gitter
|
||||
channel`_!
|
||||
|
||||
.. _structured concurrent: https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228
|
||||
.. _distributed: https://en.wikipedia.org/wiki/Distributed_computing
|
||||
.. _multi-processing: https://en.wikipedia.org/wiki/Multiprocessing
|
||||
.. _trio: https://github.com/python-trio/trio
|
||||
.. _nurseries: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/#nurseries-a-structured-replacement-for-go-statements
|
||||
.. _actor model: https://en.wikipedia.org/wiki/Actor_model
|
||||
.. _trionic: https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles
|
||||
.. _async sandwich: https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich
|
||||
.. _3 axioms: https://www.youtube.com/watch?v=7erJ1DV_Tlo&t=162s
|
||||
.. .. _3 axioms: https://en.wikipedia.org/wiki/Actor_model#Fundamental_concepts
|
||||
.. _adherance to: https://www.youtube.com/watch?v=7erJ1DV_Tlo&t=1821s
|
||||
.. _trio gitter channel: https://gitter.im/python-trio/general
|
||||
.. _matrix channel: https://matrix.to/#/!tractor:matrix.org
|
||||
.. _broadcasting: https://github.com/goodboy/tractor/pull/229
|
||||
.. _modern procotol: https://en.wikipedia.org/wiki/Rendezvous_protocol
|
||||
.. _pdbp: https://github.com/mdmintz/pdbp
|
||||
.. _pdb++: https://github.com/pdbpp/pdbpp
|
||||
.. _cheap or nasty: https://zguide.zeromq.org/docs/chapter7/#The-Cheap-or-Nasty-Pattern
|
||||
.. _(un)protocol: https://zguide.zeromq.org/docs/chapter7/#Unprotocols
|
||||
.. _discovery: https://zguide.zeromq.org/docs/chapter8/#Discovery
|
||||
.. _modern protocol: https://en.wikipedia.org/wiki/Rendezvous_protocol
|
||||
.. _messages: https://en.wikipedia.org/wiki/Message_passing
|
||||
.. _trio docs: https://trio.readthedocs.io/en/latest/
|
||||
.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
|
||||
.. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||
.. _SC: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||
.. _libdill-docs: https://sustrik.github.io/libdill/structured-concurrency.html
|
||||
.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony
|
||||
.. _async generators: https://www.python.org/dev/peps/pep-0525/
|
||||
.. _trio-parallel: https://github.com/richardsheridan/trio-parallel
|
||||
.. _uv: https://docs.astral.sh/uv/
|
||||
.. _msgspec: https://jcristharif.com/msgspec/
|
||||
.. _guest: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
|
||||
|
|
@ -191,10 +699,10 @@ channel`_!
|
|||
.. |gh_actions| image:: https://github.com/goodboy/tractor/actions/workflows/ci.yml/badge.svg?branch=main
|
||||
:target: https://github.com/goodboy/tractor/actions/workflows/ci.yml
|
||||
|
||||
.. |docs| image:: https://github.com/goodboy/tractor/actions/workflows/docs.yml/badge.svg?branch=main
|
||||
:target: https://goodboy.github.io/tractor/
|
||||
:alt: Documentation
|
||||
.. |docs| image:: https://readthedocs.org/projects/tractor/badge/?version=latest
|
||||
:target: https://tractor.readthedocs.io/en/latest/?badge=latest
|
||||
:alt: Documentation Status
|
||||
|
||||
.. |logo| image:: _static/tractor_logo_wire.svg
|
||||
.. |logo| image:: _static/tractor_logo_side.svg
|
||||
:width: 250
|
||||
:align: middle
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 98 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 93 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 80 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 80 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 98 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 61 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 90 KiB |
|
|
@ -1,387 +0,0 @@
|
|||
# tractor: distributed structured concurrency.
|
||||
'''
|
||||
``.. d2::`` - embed d2lang_ diagrams in sphinx docs
|
||||
with build-time rendering and a committed-SVG
|
||||
fallback.
|
||||
|
||||
Rendering policy,
|
||||
|
||||
- when a ``d2`` binary is found (see discovery order
|
||||
below) any out-of-date SVG is (re)rendered from its
|
||||
``.d2`` source (normally kept in ``docs/diagrams/``)
|
||||
into ``docs/_diagrams/``,
|
||||
- otherwise any pre-rendered (and git committed) SVG
|
||||
already in ``docs/_diagrams/`` is used as-is,
|
||||
- when neither is possible the diagram *source* is
|
||||
emitted as a literal block so no content is ever
|
||||
silently dropped.
|
||||
|
||||
A render that is *attempted and fails* (a ``d2`` bin is
|
||||
present but errors on the source) is NOT silently
|
||||
degraded to the stale committed SVG: it raises a
|
||||
docutils error (so ``sphinx-build -W`` fails the
|
||||
build). The render is also atomic — a failed/torn
|
||||
render can never clobber a good committed SVG — so the
|
||||
last-good diagram survives a bad edit.
|
||||
|
||||
Binary discovery order,
|
||||
|
||||
- the ``D2_BIN`` env var; may contain args which are
|
||||
split via `shlex`, eg.
|
||||
``D2_BIN='nix run nixpkgs#d2 --'``,
|
||||
- the ``d2_bin`` sphinx config value,
|
||||
- ``shutil.which('d2')``.
|
||||
|
||||
Usage,
|
||||
|
||||
.. code:: rst
|
||||
|
||||
.. d2:: diagrams/actor_tree.d2
|
||||
:caption: A tree of ``trio``-task-trees.
|
||||
:margin:
|
||||
|
||||
.. _d2lang: https://d2lang.com
|
||||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
|
||||
import enum
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shlex
|
||||
import shutil
|
||||
import subprocess as sp
|
||||
import tempfile
|
||||
|
||||
from docutils import nodes
|
||||
from docutils.parsers.rst import directives
|
||||
from sphinx.application import Sphinx
|
||||
from sphinx.util import logging
|
||||
from sphinx.util.docutils import SphinxDirective
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# subdir (under the sphinx srcdir) holding rendered,
|
||||
# git-committed, fallback SVG outputs.
|
||||
_outdir: str = '_diagrams'
|
||||
|
||||
# per-build map of {output-svg-name: source-relpath} used
|
||||
# to detect 2 distinct `.d2` sources colliding on a single
|
||||
# output stem; reset on each `builder-inited` (see `setup`).
|
||||
_seen_outputs: dict[str, str] = {}
|
||||
|
||||
|
||||
class RenderResult(enum.Enum):
|
||||
'''
|
||||
Outcome of a `render_svg()` call.
|
||||
|
||||
'''
|
||||
OK = 'ok' # fresh SVG exists on disk
|
||||
FELL_BACK = 'fell_back' # no bin; serving committed SVG
|
||||
NO_OUTPUT = 'no_output' # no bin AND no committed SVG
|
||||
FAILED = 'failed' # bin present, render errored
|
||||
|
||||
|
||||
def find_d2(
|
||||
app: Sphinx,
|
||||
) -> list[str]|None:
|
||||
'''
|
||||
Resolve the d2 render command as an argv list or
|
||||
`None` when no binary can be found.
|
||||
|
||||
'''
|
||||
if env_bin := os.environ.get('D2_BIN'):
|
||||
return shlex.split(env_bin)
|
||||
if cfg_bin := app.config.d2_bin:
|
||||
return shlex.split(cfg_bin)
|
||||
if path_bin := shutil.which('d2'):
|
||||
return [path_bin]
|
||||
return None
|
||||
|
||||
|
||||
def render_svg(
|
||||
app: Sphinx,
|
||||
src: Path,
|
||||
out: Path,
|
||||
) -> RenderResult:
|
||||
'''
|
||||
Maybe (re)render `src` -> `out` and report the
|
||||
outcome as a `RenderResult`.
|
||||
|
||||
A render is performed only when `out` is missing or
|
||||
older than `src` AND a `d2` binary is available. The
|
||||
write is atomic (temp-file + `os.replace`) so a
|
||||
failed or torn render never clobbers an existing
|
||||
(committed) SVG.
|
||||
|
||||
'''
|
||||
fresh: bool = (
|
||||
out.exists()
|
||||
and
|
||||
src.stat().st_mtime <= out.stat().st_mtime
|
||||
)
|
||||
if fresh:
|
||||
return RenderResult.OK
|
||||
d2cmd: list[str]|None = find_d2(app)
|
||||
if d2cmd is None:
|
||||
# no binary: fall back to whatever is committed,
|
||||
# else signal the caller to emit the raw source.
|
||||
if out.exists():
|
||||
log.info(
|
||||
f'no d2 binary; using committed svg '
|
||||
f'for {src.name}'
|
||||
)
|
||||
return RenderResult.FELL_BACK
|
||||
return RenderResult.NO_OUTPUT
|
||||
out.parent.mkdir(
|
||||
parents=True,
|
||||
exist_ok=True,
|
||||
)
|
||||
# render into a sibling temp-file first so a bad
|
||||
# `d2` exit (or a crash mid-write) leaves any prior
|
||||
# committed SVG fully intact.
|
||||
fd, tmpname = tempfile.mkstemp(
|
||||
dir=str(out.parent),
|
||||
prefix=f'.{out.stem}.',
|
||||
suffix='.svg.tmp',
|
||||
)
|
||||
os.close(fd)
|
||||
tmp = Path(tmpname)
|
||||
argv: list[str] = (
|
||||
d2cmd
|
||||
+ list(app.config.d2_args)
|
||||
+ [str(src), str(tmp)]
|
||||
)
|
||||
try:
|
||||
proc = sp.run(
|
||||
argv,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
)
|
||||
except (
|
||||
OSError,
|
||||
sp.TimeoutExpired,
|
||||
) as err:
|
||||
tmp.unlink(missing_ok=True)
|
||||
log.warning(
|
||||
f'd2 invocation failed for {src.name}: '
|
||||
f'{err}'
|
||||
)
|
||||
return RenderResult.FAILED
|
||||
if proc.returncode != 0:
|
||||
tmp.unlink(missing_ok=True)
|
||||
log.warning(
|
||||
f'd2 render error for {src.name}:\n'
|
||||
f'{proc.stderr}'
|
||||
)
|
||||
return RenderResult.FAILED
|
||||
# atomic swap-in of the freshly rendered SVG.
|
||||
os.replace(tmp, out)
|
||||
return RenderResult.OK
|
||||
|
||||
|
||||
class D2Diagram(SphinxDirective):
|
||||
'''
|
||||
Render a ``.d2`` source file (path relative to
|
||||
the sphinx srcdir) as an SVG figure.
|
||||
|
||||
'''
|
||||
required_arguments = 1
|
||||
has_content = False
|
||||
option_spec = {
|
||||
'caption': directives.unchanged,
|
||||
'alt': directives.unchanged,
|
||||
'width': (
|
||||
directives.length_or_percentage_or_unitless
|
||||
),
|
||||
'margin': directives.flag,
|
||||
'class': directives.class_option,
|
||||
'name': directives.unchanged,
|
||||
}
|
||||
|
||||
def run(self) -> list[nodes.Node]:
|
||||
relsrc: str = self.arguments[0]
|
||||
srcdir = Path(self.env.srcdir)
|
||||
src: Path = srcdir / relsrc
|
||||
self.env.note_dependency(relsrc)
|
||||
if not src.exists():
|
||||
err = self.state_machine.reporter.error(
|
||||
f'd2 source not found: {relsrc}',
|
||||
line=self.lineno,
|
||||
)
|
||||
return [err]
|
||||
out: Path = (
|
||||
srcdir
|
||||
/ _outdir
|
||||
/ f'{src.stem}.svg'
|
||||
)
|
||||
# collision guard: two distinct sources must not
|
||||
# map onto the same output stem within a build.
|
||||
prior: str|None = _seen_outputs.get(out.name)
|
||||
if (
|
||||
prior is not None
|
||||
and
|
||||
prior != relsrc
|
||||
):
|
||||
err = self.state_machine.reporter.error(
|
||||
f'd2 output collision: {relsrc!r} and '
|
||||
f'{prior!r} both render to '
|
||||
f'{_outdir}/{out.name}; rename one '
|
||||
f'`.d2` stem',
|
||||
line=self.lineno,
|
||||
)
|
||||
return [err]
|
||||
_seen_outputs[out.name] = relsrc
|
||||
|
||||
result: RenderResult = render_svg(
|
||||
self.env.app,
|
||||
src,
|
||||
out,
|
||||
)
|
||||
if result is RenderResult.FAILED:
|
||||
# loud, build-failing (under `-W`) signal; the
|
||||
# prior committed SVG, if any, is untouched.
|
||||
err = self.state_machine.reporter.error(
|
||||
f'd2 render failed for {relsrc} (see '
|
||||
f'build log); the committed svg, if any, '
|
||||
f'was left untouched',
|
||||
line=self.lineno,
|
||||
)
|
||||
return [err]
|
||||
if result is RenderResult.NO_OUTPUT:
|
||||
# last resort: emit the raw d2 source so no
|
||||
# content is ever silently dropped.
|
||||
log.warning(
|
||||
f'no svg available for {relsrc}; '
|
||||
f'emitting d2 source as literal block'
|
||||
)
|
||||
src_text: str = src.read_text()
|
||||
literal = nodes.literal_block(
|
||||
src_text,
|
||||
src_text,
|
||||
)
|
||||
literal['language'] = 'text'
|
||||
return [literal]
|
||||
# OK | FELL_BACK -> embed the SVG figure.
|
||||
img = nodes.image(
|
||||
uri=f'/{_outdir}/{out.name}',
|
||||
alt=self.options.get(
|
||||
'alt',
|
||||
f'd2 diagram: {src.stem}',
|
||||
),
|
||||
)
|
||||
if width := self.options.get('width'):
|
||||
img['width'] = width
|
||||
# content-hash the rendered svg so the served
|
||||
# `<img src>` carries a `?v=<hash>` cache-buster
|
||||
# (sphinx tags css/js this way but not images);
|
||||
# without it an edited diagram keeps showing the
|
||||
# browser's stale copy at the stable
|
||||
# `_images/<stem>.svg` url on autobuild reload.
|
||||
if out.exists():
|
||||
img['d2_stem'] = out.stem
|
||||
img['d2_cachebust'] = hashlib.sha256(
|
||||
out.read_bytes()
|
||||
).hexdigest()[:8]
|
||||
fig = nodes.figure()
|
||||
fig += img
|
||||
classes: list[str] = (
|
||||
['d2-diagram']
|
||||
+ self.options.get('class', [])
|
||||
)
|
||||
if 'margin' in self.options:
|
||||
# NB: the bare 'margin' class is what
|
||||
# book-style themes key off for
|
||||
# right-margin placement; our custom css
|
||||
# uses 'd2-margin'.
|
||||
classes += [
|
||||
'margin',
|
||||
'd2-margin',
|
||||
]
|
||||
fig['classes'] += classes
|
||||
if caption_txt := self.options.get('caption'):
|
||||
(
|
||||
inline_nodes,
|
||||
_msgs,
|
||||
) = self.state.inline_text(
|
||||
caption_txt,
|
||||
self.lineno,
|
||||
)
|
||||
caption = nodes.caption(
|
||||
caption_txt,
|
||||
'',
|
||||
*inline_nodes,
|
||||
)
|
||||
fig += caption
|
||||
self.add_name(fig)
|
||||
return [fig]
|
||||
|
||||
|
||||
def _reset_seen(
|
||||
app: Sphinx,
|
||||
) -> None:
|
||||
'''
|
||||
Clear the per-build output-collision map at the
|
||||
start of each build.
|
||||
|
||||
'''
|
||||
_seen_outputs.clear()
|
||||
|
||||
|
||||
def _cachebust_d2_images(
|
||||
app: Sphinx,
|
||||
pagename: str,
|
||||
templatename: str,
|
||||
context: dict,
|
||||
doctree,
|
||||
) -> None:
|
||||
'''
|
||||
Append a ``?v=<content-hash>`` query to every d2
|
||||
diagram ``<img src>`` in the rendered page ``body``
|
||||
so a browser re-fetches an *edited* SVG instead of
|
||||
its stale cached copy at the stable
|
||||
``_images/<stem>.svg`` url (sphinx cache-busts
|
||||
css/js this way, but not images).
|
||||
|
||||
Scoped strictly to d2 images (those carrying the
|
||||
`d2_cachebust` attr set in `D2Diagram.run()`); all
|
||||
other markup is left untouched.
|
||||
|
||||
'''
|
||||
if doctree is None or 'body' not in context:
|
||||
return
|
||||
busts: dict[str, str] = {}
|
||||
for img in doctree.findall(nodes.image):
|
||||
if ver := img.get('d2_cachebust'):
|
||||
busts[img['d2_stem']] = ver
|
||||
if not busts:
|
||||
return
|
||||
body: str = context['body']
|
||||
for stem, ver in busts.items():
|
||||
body = re.sub(
|
||||
r'(src="[^"]*?' + re.escape(stem) + r'\.svg)(")',
|
||||
rf'\1?v={ver}\2',
|
||||
body,
|
||||
)
|
||||
context['body'] = body
|
||||
|
||||
|
||||
def setup(app: Sphinx) -> dict:
|
||||
app.add_config_value('d2_bin', None, 'env')
|
||||
app.add_config_value('d2_args', [], 'env')
|
||||
app.add_directive('d2', D2Diagram)
|
||||
app.connect('builder-inited', _reset_seen)
|
||||
app.connect('html-page-context', _cachebust_d2_images)
|
||||
return {
|
||||
'version': '0.1.0',
|
||||
# NB: run() writes the rendered SVG during the
|
||||
# READ phase; the temp-file + `os.replace` swap
|
||||
# keeps that atomic so concurrent renders of the
|
||||
# same diagram (under `sphinx-build -j`) can't
|
||||
# tear the output file.
|
||||
'parallel_read_safe': True,
|
||||
'parallel_write_safe': True,
|
||||
}
|
||||
|
|
@ -1,51 +0,0 @@
|
|||
# tractor: distributed structured concurrency.
|
||||
'''
|
||||
``.. margin::`` - prose-anchored, right-margin asides.
|
||||
|
||||
A theme-agnostic vendoring of `sphinx_book_theme`'s
|
||||
`Margin` directive: a `docutils` `Sidebar` subclass
|
||||
which tags the node with a ``margin`` class; placement
|
||||
is then pure CSS (see ``_static/css/custom.css``)
|
||||
allowing use on any theme incl. our
|
||||
`pydata_sphinx_theme`.
|
||||
|
||||
Usage,
|
||||
|
||||
.. code:: rst
|
||||
|
||||
.. margin:: An optional title
|
||||
|
||||
Aside content; text, figures, whatever.
|
||||
|
||||
'''
|
||||
from docutils import nodes
|
||||
from docutils.parsers.rst.directives.body import (
|
||||
Sidebar,
|
||||
)
|
||||
from sphinx.application import Sphinx
|
||||
|
||||
|
||||
class Margin(Sidebar):
|
||||
'''
|
||||
Notes/figures placed in the right margin, anchored
|
||||
at the current point in the prose flow.
|
||||
|
||||
'''
|
||||
required_arguments = 0
|
||||
optional_arguments = 1
|
||||
|
||||
def run(self) -> list[nodes.Node]:
|
||||
if not self.arguments:
|
||||
self.arguments = ['']
|
||||
out: list[nodes.Node] = super().run()
|
||||
out[0].attributes['classes'].append('margin')
|
||||
return out
|
||||
|
||||
|
||||
def setup(app: Sphinx) -> dict:
|
||||
app.add_directive('margin', Margin)
|
||||
return {
|
||||
'version': '0.1.0',
|
||||
'parallel_read_safe': True,
|
||||
'parallel_write_safe': True,
|
||||
}
|
||||
|
|
@ -1,185 +0,0 @@
|
|||
/* tractor docs: a minimal black + white skin over
|
||||
* `pydata-sphinx-theme` plus RHS-marginalia + d2
|
||||
* diagram styling.
|
||||
*/
|
||||
html[data-theme="light"] {
|
||||
--pst-color-primary: #000000;
|
||||
--pst-color-secondary: #3d3d3d;
|
||||
--pst-color-accent: #5a5a5a;
|
||||
--pst-color-link: #000000;
|
||||
--pst-color-link-hover: #5a5a5a;
|
||||
--pst-color-inline-code: #1a1a1a;
|
||||
--pst-color-inline-code-links: #000000;
|
||||
}
|
||||
html[data-theme="dark"] {
|
||||
--pst-color-primary: #ffffff;
|
||||
--pst-color-secondary: #c9c9c9;
|
||||
--pst-color-accent: #a8a8a8;
|
||||
--pst-color-link: #ffffff;
|
||||
--pst-color-link-hover: #bdbdbd;
|
||||
--pst-color-inline-code: #e8e8e8;
|
||||
--pst-color-inline-code-links: #ffffff;
|
||||
}
|
||||
/* mono-chrome links: rely on underline for
|
||||
* affordance instead of color.
|
||||
*/
|
||||
.bd-content a:not(.headerlink) {
|
||||
text-decoration: underline;
|
||||
text-underline-offset: 0.18em;
|
||||
}
|
||||
/* d2 diagram figures */
|
||||
figure.d2-diagram {
|
||||
text-align: center;
|
||||
}
|
||||
figure.d2-diagram img {
|
||||
max-width: 100%;
|
||||
height: auto;
|
||||
}
|
||||
/* keep light-rendered svgs legible in dark mode */
|
||||
html[data-theme="dark"] figure.d2-diagram img {
|
||||
background: #ffffff;
|
||||
border-radius: 6px;
|
||||
padding: 6px;
|
||||
}
|
||||
/* prose-anchored right-margin asides (tufte-ish):
|
||||
* float right within the content column on wide
|
||||
* screens, collapse inline on narrow ones.
|
||||
*/
|
||||
@media (min-width: 960px) {
|
||||
aside.margin,
|
||||
figure.margin,
|
||||
div.margin,
|
||||
figure.d2-margin {
|
||||
float: right;
|
||||
clear: right;
|
||||
width: 44%;
|
||||
margin: 0.2rem 0 1rem 1.4rem;
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
}
|
||||
/* strip docutils sidebar chrome from margin asides */
|
||||
aside.sidebar.margin {
|
||||
border: none;
|
||||
background: transparent;
|
||||
padding: 0;
|
||||
}
|
||||
aside.sidebar.margin > p.sidebar-title {
|
||||
font-weight: 600;
|
||||
font-size: 0.9rem;
|
||||
margin-bottom: 0.3rem;
|
||||
}
|
||||
/* landing-page hero logo (inline svg): the linework uses
|
||||
* `fill: currentColor` so it inherits the theme text colour
|
||||
* (auto-flips light<->dark), and the faces are transparent so
|
||||
* the page background shows through. */
|
||||
svg.hero-logo {
|
||||
display: block;
|
||||
max-width: 360px;
|
||||
height: auto;
|
||||
color: var(--pst-color-text-base);
|
||||
}
|
||||
/* 3-up landing hero row: name | logo | tagline, with a centered
|
||||
* sub-tagline beneath. */
|
||||
.hero-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
/* left-anchored single row; tighter gap so the logo sits over
|
||||
* the sub-line and the tagline over the prose below it (stack
|
||||
* only on mobile, via the media query). */
|
||||
justify-content: flex-start;
|
||||
flex-wrap: nowrap;
|
||||
gap: clamp(0.5rem, 2vw, 1.5rem);
|
||||
margin: 0 0 0.25rem;
|
||||
}
|
||||
.hero-row svg.hero-logo {
|
||||
/* override the stacked-hero sizing for the inline row */
|
||||
width: clamp(240px, 34vw, 360px);
|
||||
margin: 0;
|
||||
}
|
||||
.hero-row .hero-tag {
|
||||
margin: 0;
|
||||
/* keep it on a single line beside the logo */
|
||||
max-width: none;
|
||||
white-space: nowrap;
|
||||
font-size: clamp(1rem, 2.1vw, 1.4rem);
|
||||
font-weight: 400;
|
||||
line-height: 1.15;
|
||||
color: var(--pst-color-text-base);
|
||||
}
|
||||
.hero-row .hero-tag {
|
||||
min-width: 0;
|
||||
}
|
||||
.hero-sub {
|
||||
/* wrap at the prose width (not the logo's), with a line of
|
||||
* space before the prose section below. */
|
||||
margin: 0 0 4rem;
|
||||
max-width: none;
|
||||
text-align: left;
|
||||
font-size: 1.05rem;
|
||||
color: var(--pst-color-text-muted);
|
||||
}
|
||||
/* the page carries a single rst <h1> ("tractor") for semantics
|
||||
* + SEO, but the visual title is the hero — so hide the doc
|
||||
* title accessibly (still read by screen readers + search). */
|
||||
#tractor > h1 {
|
||||
position: absolute !important;
|
||||
width: 1px;
|
||||
height: 1px;
|
||||
margin: -1px;
|
||||
padding: 0;
|
||||
overflow: hidden;
|
||||
clip: rect(0 0 0 0);
|
||||
clip-path: inset(50%);
|
||||
white-space: nowrap;
|
||||
border: 0;
|
||||
}
|
||||
/* smaller section headings on the landing only */
|
||||
#sixty-seconds-of-why > :is(h1, h2),
|
||||
#dig-in > :is(h1, h2),
|
||||
#features > :is(h1, h2),
|
||||
#where-do-i-start > :is(h1, h2) {
|
||||
font-size: 1.75rem;
|
||||
}
|
||||
/* extra breathing room before the first landing section */
|
||||
#sixty-seconds-of-why {
|
||||
margin-top: 4rem;
|
||||
}
|
||||
/* pull the section's `tl;dr` margin-note up so its top edge is
|
||||
* level with the heading (default floats it beside the body). */
|
||||
#sixty-seconds-of-why aside.margin {
|
||||
margin-top: -3.75rem;
|
||||
}
|
||||
/* stack the 3-up hero only on real mobile */
|
||||
@media (max-width: 640px) {
|
||||
.hero-row {
|
||||
flex-direction: column;
|
||||
}
|
||||
.hero-row .hero-tag {
|
||||
max-width: none;
|
||||
}
|
||||
}
|
||||
/* navbar brand: the "distributed SC" tagline next to the logo
|
||||
* (the logo `text` value), kept small + muted like a sub-line. */
|
||||
.navbar-brand .title {
|
||||
/* match the centered nav tabs (Bootstrap leaves the nav-link
|
||||
* size/weight vars empty → ~1rem / normal); `inline-block` is
|
||||
* the real underline fix — it stops the parent brand-link's
|
||||
* :hover underline from propagating onto the text (a block
|
||||
* child's `text-decoration:none` can't cancel an ancestor's
|
||||
* underline). */
|
||||
display: inline-block;
|
||||
font-size: 1rem;
|
||||
font-weight: 400;
|
||||
text-decoration: none;
|
||||
}
|
||||
/* it's a brand mark, not a content link — never underline the
|
||||
* brand text (base / hover / focus). */
|
||||
.navbar-brand,
|
||||
.navbar-brand:hover,
|
||||
.navbar-brand:focus,
|
||||
.navbar-brand:visited:hover,
|
||||
.navbar-brand .title,
|
||||
.navbar-brand:hover .title {
|
||||
text-decoration: none !important;
|
||||
border-bottom: 0 !important;
|
||||
}
|
||||
|
|
@ -1,462 +0,0 @@
|
|||
<div class="hero-row">
|
||||
<svg class="hero-logo" role="img" aria-label="tractor" version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
viewBox="0 0 2670 1980" style="enable-background:new 0 0 2670 1980;" xml:space="preserve">
|
||||
<style type="text/css">
|
||||
.st0{fill:currentColor;}
|
||||
.st1{fill:none;}
|
||||
</style>
|
||||
<g>
|
||||
<path class="st0" d="M2275.3,1000.1c-0.1,2.3,0,4.7,0,7c0,189.3,0,378.7,0,568c0,6,0.1,12.1-0.8,17.9c-2.4,15.2-12.9,17.4-23.6,10
|
||||
c-45.4-31-91.2-61.4-136.9-91.9l-30.1-20.3c-9.8-6.3-19.2-13.2-29.2-19.3c-6.7-4.1-9.8-9.7-9.9-17.2c-0.1-9.3-0.6-18.7-0.3-28
|
||||
c0.2-7.3-1.7-13-6.5-19c-18.6-23.3-36.2-47.3-56.9-69c-18-18.8-36.7-36.8-56.4-53.8c-31.8-27.5-65.4-52.7-101-74.9
|
||||
c-32.7-20.4-66.8-38.4-102.2-54c-50.7-22.4-103.2-39.2-157.1-51.5c-36.7-8.4-73.8-14.4-111.2-18.5c-34.5-3.8-69-6.5-103.7-5.8
|
||||
c-23.1,0.5-46.2-1.3-69.3,0.4c-22.9,1.8-45.9,3-68.7,5.6c-72.7,8.4-143.7,24.6-212.1,50.6c-65.5,24.8-127.4,57-184.9,97.4
|
||||
c-46.8,32.8-90.1,69.8-129.5,111.1c-18.1,19-35.1,39.2-50.5,60.6c-3.9,5.5-5.5,11-5.3,17.6c0.4,9.3,0,18.7,0,28
|
||||
c0,9.3-3.1,16.4-11.8,21.4c-10.4,5.9-19.9,13.4-30.1,19.8l-25.4,17.2c-11.6,7.8-23.3,15.5-34.9,23.3c-34.3,23-68.6,46-102.9,69
|
||||
c-14.6,9.7-23.6,5.5-25.2-12.1c-0.5-5.3-0.5-10.7-0.5-16c0-189.7,0-379.3,0-569l-0.1-26.1c0.2-190.3,0.1-380.7,0.2-571
|
||||
c0-5.7,0.4-11.3,0.5-17c0.1-3.7,1.2-7.3,3-10.5c3.4-5.9,9.3-7.7,15.4-4.8c2.7,1.3,5.1,3.1,7.6,4.8c44.3,29.6,88.5,59.3,132.8,88.9
|
||||
l29.7,20.1c10.8,6.6,20.9,14.3,31.7,20.9c6.7,4.1,9.7,9.8,9.8,17.3c0.2,11,0.4,22,0.2,33c-0.1,5.2,1.2,9.6,4.2,13.9
|
||||
c15.4,22.2,33,42.8,51.6,62.2c53.9,56.4,114.1,105.1,181.4,144.6c38.4,22.6,78.3,42.4,119.9,58.6c40,15.6,80.9,28.3,122.8,38
|
||||
c52.7,12.1,106,19.3,160,22.6c18.3,1.1,36.6,2.1,54.9,1.5c17.7-0.6,35.5,1.1,53.2-0.1c17.6-1.1,35.2-2.3,52.8-3.6
|
||||
c35.9-2.5,71.4-8.5,106.7-15.1c53.4-9.9,105.5-25,156-45.1c53.8-21.5,105.1-48.2,153.2-80.9c43.4-29.5,83.6-62.7,121.1-99.3
|
||||
c27.3-26.6,50.3-56.8,73.9-86.6c2.9-3.6,3.6-7.6,3.6-12.1c-0.1-10.7,0.2-21.3,0.4-32c0.1-7.5,3.2-13.1,9.9-17.2
|
||||
c10.2-6.2,19.9-13.4,30-19.8l23.2-16.3c4.2-1.3,7.4-4.3,11-6.7c42.9-28.7,85.8-57.4,128.7-86.1c1.1-0.7,2.3-1.4,3.3-2.3
|
||||
c10.9-9.1,21.4-4.7,23.8,10.7c0.8,5.6,0.6,11.3,0.6,17c0,189.3,0,378.7,0,568L2275.3,1000.1z M740,1203.1c-2-0.2-4-0.3-6-0.5
|
||||
c-0.5,2.5-4.8,2-4.2,5.8c2.9-1,6.4-1.2,4.2-5.8C735.8,1206,738.1,1202.3,740,1203.1c-0.6-0.7-0.5-1.3,0.2-1.7
|
||||
c-1.1-0.3-1.1-0.1-0.4,0.8C740,1202.3,739.9,1202.8,740,1203.1z M1935.5,780.5c-0.2,0.2-0.4,0.4-0.6,0.4c-0.3,0.1-0.6,0-0.8,0
|
||||
c0.4-0.2,0.8-0.5,1.1-0.8c1.4-0.9,2-1.9,0.5-3.3c3.2,1.1,6.1,1.2,7.9-2.8c-3.3-0.7-4.6,3.8-7.7,2.9
|
||||
C1935.7,778.2,1935.6,779.4,1935.5,780.5z M1934.8,1202.5c-0.1-0.2-0.1-0.4-0.2-0.6c0,0,0.1-0.1,0.1-0.1c0,0.2,0.1,0.3,0.1,0.5
|
||||
c3.1,1.6,5,5.7,11.3,5.1C1941.3,1205,1938.9,1202,1934.8,1202.5z M2072.4,516.6c-1,2.3-3.4,3-5.3,4.3c-7.4,5.1-8.2,7.2-5.3,17.8
|
||||
c4.2-6.5,8-12.6,11.9-18.6c1.1-1.7,0.9-3.1-1.3-3.6c0,1,0.3,1.8,1.4,2c0.2,0,0.4-0.4,0.6-0.7C2073.8,517.4,2073.1,517,2072.4,516.6
|
||||
z M599.6,517.5c-0.4-0.8-1.1-1.7-1.8-0.8c-0.8,0.8,0.1,1.5,1,1.6c2.4,8.9,7.5,16.2,13.4,24.1c2.3-16.5,1.9-17.3-10.7-24.4
|
||||
C600.9,517.6,600.2,517.6,599.6,517.5z M534.9,1353.5c-20,21.8-37.3,44.1-53.4,67.4c-16.1,23.4-26.8,49.5-41.8,73.5
|
||||
c9-21,17.4-42.2,28.9-61.9c16.9-28.7,35.4-56.4,57.5-81.5c0.9-1,1.6-2.1,2.6-3.1c2.9-2.9,3.3-6,2.3-10
|
||||
c-9.1-33.8-14.6-68.3-19.2-103c-5-38-8.3-76.1-10.6-114.3c-2.5-39.9-3.4-79.9-3.9-119.8c-0.1-5.1-1.4-6.8-6.7-6.9
|
||||
c-21.6-0.2-43.3-0.8-64.9-1.3c-2,0-4-0.4-6,0.8c0,195.2,0,390.5,0,585.6c2.1,0.6,3.1-0.4,4.1-1.1c52.7-33.8,105.3-67.7,158-101.4
|
||||
c3.6-2.3,5.7-5.4,7.3-9.1c6-13,12.5-25.8,20.8-37.4c4.2-5.9,3.9-11.9,2.5-19.1c-2.4,1.8-4,2.9-5.5,4.2
|
||||
c-13.9,12.6-32.1,8.7-42.7-2.9c-7.3-8-12.4-17.2-17.1-26.8C542.3,1375.7,539.4,1365.1,534.9,1353.5z M534.6,627.2
|
||||
c3.6-5,4.3-10.7,6.3-15.9c6.2-15.8,12.6-31.5,25.2-43.7c10.8-10.5,26.8-12.6,38.9-2.8c2.3,1.9,4.3,5,7.9,5
|
||||
c1.6-8.3-0.2-15.3-5.1-21.9c-7.4-9.9-12.7-21-17.8-32.3c-2.4-5.3-5.8-9.2-10.8-12.4c-50.5-32.2-100.9-64.7-151.3-97.1
|
||||
c-2.4-1.5-4.5-3.9-8.7-3.9c0,3.7,0,7.2,0,10.7c0,187.9,0,375.8,0,563.7c0,2,0.2,4,0,6c-0.6,4.9,1.4,6.3,6.2,6.1
|
||||
c13.6-0.6,27.3-0.7,41-1c8.3-0.2,16.7-0.6,25-0.4c4.9,0.1,6.7-1.8,6.1-6.4c-0.1-1,0-2,0-3c1.2-75,4.3-149.8,13.7-224.3
|
||||
c4.6-36.3,10-72.5,19.6-107.9c1.6-5.7,0.4-9.9-3.4-14.4c-10.7-12.8-21.7-25.4-31-39.1c-19-28.1-36.8-57-49.6-88.7
|
||||
c-2.2-5.4-5.5-10.4-6.2-16.6C465,538.3,494.6,586,534.6,627.2z M2060.9,1411.1c-2.3,8.5-0.1,15.1,4,21.4
|
||||
c5.9,9.3,13.4,18.1,17.1,28.2c3.9,10.7,10.9,16.6,19.9,22.4c47.8,30.3,95.3,61.1,143,91.7c2.7,1.7,5,4.3,9.5,4.5c0-3.4,0-6.6,0-9.8
|
||||
c0-188.3,0-376.6,0-565c0-2-0.1-4,0-6c0.2-3.4-1.1-5-4.6-4.8c-2.3,0.1-4.7,0-7,0.1c-20,0.6-40,1.2-60,1.9c-6.7,0.2-6.7,0.4-7,7.4
|
||||
c0,1.3,0,2.7-0.1,4c-0.4,60-3,119.9-8.9,179.7c-5,50.1-11.3,99.9-24,148.7c-1.4,5.2,0.1,8.3,3.5,12.3c15,17.8,29.6,36.1,42.4,55.5
|
||||
c20,30.4,36.1,62.9,49.3,96.8c3.2,8.2,5.7,16.6,6.8,25.4c-33.9-90.8-82.6-145-107.2-169.7c-4.1,11-7.5,22.1-12.7,32.5
|
||||
c-4.9,9.9-10.3,19.5-18.8,26.8c-7.1,6.1-15.1,8.9-24.8,7.5C2073.2,1421.4,2068.3,1415.5,2060.9,1411.1z M2244.8,454.3
|
||||
c0.6,4.5-0.2,6.7-0.8,8.9c-6.5,24.2-17,47-28.2,69.2c-18.9,37.2-43,71.2-70.4,102.8c-2.7,3.2-3.6,5.8-2.5,9.8
|
||||
c7.9,29.6,13.1,59.8,17.3,90.2c6.4,45.6,10.2,91.3,12.8,137.3c2,35.3,2.6,70.6,3.1,105.9c0.1,7.6,0.2,7.6,7.6,7.8
|
||||
c17,0.5,34,1,51,1.4c5,0.1,10,0.2,15,0.5c3.2,0.1,5.1-0.9,4.9-4.5c-0.1-1.7,0-3.3,0-5c0-189.6,0-379.3-0.1-568.9
|
||||
c0-2.4,1.1-5.2-1.2-7.3c-0.8,0.1-1.6,0-2.1,0.4c-52.7,33.8-105.3,67.7-158,101.5c-2,1.3-3.7,2.6-4.7,4.9c-6.9,16-16.9,30.1-25.8,45
|
||||
c-3.1,5.2-3.4,10.2-2,15.7c2.6,0.1,3.5-1.6,4.6-2.6c15.7-13.9,33-10.6,45.6,3.8c6,6.8,10.4,14.6,14.4,22.7
|
||||
c5.1,10.3,8.3,21.3,12.8,32.9C2185.2,576.1,2221.7,520.5,2244.8,454.3z M553.7,662.8c-0.3,1.3-0.7,2.8-1,4.3
|
||||
c-7.1,31.2-11.7,62.8-15.4,94.5c-7.1,59.9-10.3,120.1-11.8,180.4c-0.3,13-0.2,26-0.5,39c-0.1,4.4,1.5,5.7,5.7,5.6
|
||||
c25-0.5,50-0.7,74.9-1.1c7.4-0.1,7.6-0.2,7.6-7.8c0-85.6,0-171.2,0.1-256.8c0-5.3-1.7-8.5-5.8-11.7c-16.6-12.8-32.4-26.6-46.9-41.9
|
||||
C559,665.6,557.6,663,553.7,662.8z M553.6,1319.7c2.1-1,2.8-1.2,3.2-1.6c16.4-17.1,34-32.9,52.7-47.4c3-2.4,3.8-5,3.8-8.6
|
||||
c-0.1-13.3-0.1-26.7-0.1-40c0-72.3,0-144.7,0-217c0-10.3,0.4-9-9.2-9.2c-23.3-0.4-46.7-0.8-70-1.1c-2.8,0-5.9-1-9.3,1.5
|
||||
C526.2,1104.4,531.1,1212.3,553.6,1319.7z M2119.8,1319.2c4.8-20.9,8.3-40.2,11.1-59.6c5.8-39.5,9.8-79.2,12.5-119.1
|
||||
c3.1-45.5,4.7-91.1,5.1-136.7c0.1-7.6-0.1-7.7-7.6-7.7c-24.3,0.1-48.6,0.3-72.9,0.5c-7.9,0.1-8,0.1-8,8.5c0,84.6,0,169.1-0.1,253.7
|
||||
c0,5.6,1.7,9.3,5.9,12.7c12.1,9.9,24.2,19.9,36,30.1C2107.7,1306.9,2113.1,1312.7,2119.8,1319.2z M2120.4,664.4
|
||||
c-2.1-0.6-2.9-0.1-3.6,0.7c-15.2,16.5-33.1,29.9-50.2,44.2c-4.8,4-6.8,8-6.8,14.3c0.3,32,0.2,64,0.2,96c0,52.3,0,104.6,0,157
|
||||
c0,8.1,0.1,8.2,8.2,8.2c18.3,0.2,36.7,0.2,55,0.4c6.7,0.1,13.3,0.1,20,0.3c3.7,0.1,5.7-1.1,5.4-5.1c-0.2-2.3,0-4.7-0.1-7
|
||||
c-0.5-45-2.1-89.9-5.2-134.8C2139.2,780.1,2132.7,721.9,2120.4,664.4z M878.7,1138.5c-36,15-69.8,32.9-102.3,53.1
|
||||
c-3.4,2.1-4.9,4.7-5.7,8.6c-2.8,14.4-5.7,28.8-12.2,42c-3.6,7.3-7.7,14.4-17.3,14.4c-9.6,0-13.4-7.4-17.4-14.4
|
||||
c-1.9-3.2-2-7.6-5.4-9.8c-22.3,17.4-41.6,37.2-58.8,58.9c-5.8,7.3-10.2,15.3-12,24.8c-3.8,20.3-9.1,40.2-16.7,59.5
|
||||
c-2.6,6.7-1.3,13.3-1.5,20.3c1.5-0.4,2-0.3,2.1-0.5c3.2-3.8,6.4-7.6,9.4-11.6c31.2-40.5,66-77.4,105.4-110.2
|
||||
c36.9-30.8,75.6-59.1,117.2-83.1c54.3-31.2,110.9-57.4,170.8-76.4c38.4-12.2,77.4-21.8,117-29.2c9.5-1.8,19.1-3,28.6-4.6
|
||||
c1.5-0.3,3.8,0.3,4.4-1.5c1.4-3.9,0.6-8-0.4-11.7c-1-3.6-4.5-1.7-6.9-1.5c-40.6,3.2-80.4,11-120.4,18c-7,1.2-10.4,4.1-10.9,11.2
|
||||
c-0.3,3.8-0.5,9.1-5.8,9c-4.6-0.1-4.7-5-5.3-8.5c-0.7-4.5-2.4-6-7.3-4.9c-39.3,9.1-78,19.9-115.5,34.7c-4.6,1.8-7.1,4.2-7.8,8.9
|
||||
c-0.9,5.9-2.2,11.8-5.2,17.1c-4.1,7.1-10.6,7.2-14.6,0C881.9,1147.7,880.7,1143.4,878.7,1138.5z M1032.8,891.9
|
||||
c0.9-3.8,1.6-6.7,2.4-9.5c0.7-2.3,1.7-4.6,4.4-4.9c3.4-0.3,4.4,2.3,5.3,4.9c0.4,1.3,0.8,2.6,0.8,3.9c0.1,7.7,4.5,10.4,11.7,11.6
|
||||
c32.5,5.5,64.8,11.9,97.5,15.7c7.9,0.9,15.9,1.7,23.8,2.4c1.9,0.2,4,0.7,5.3-1.6c3.5-6.5,0.3-13-7-14.2c-1-0.2-2-0.2-3-0.4
|
||||
c-42.6-5.8-84.2-15.9-125.4-28.2c-41.8-12.4-82.5-28.2-121.6-47.1c-51-24.6-99.8-53.3-144.7-87.9c-20.6-15.8-40.9-32.1-59.9-49.7
|
||||
c-29.1-26.9-56.4-55.7-80.1-87.7c-3.7-4.9-7.7-9.6-11.6-14.3c-0.8,7.6-2.4,14.7,0.5,21.8c7.4,18.2,12.4,37.2,16,56.5
|
||||
c1.7,9.1,5,17.3,10.6,24.4c9.2,11.4,18.5,22.8,28.7,33.3c9.9,10.1,21.1,19,32.2,28.9c3.8-6.5,5.6-13,9.8-18.2
|
||||
c7.7-9.4,17.7-9.5,25.3-0.1c3.6,4.5,5.7,9.7,7.8,15c4.3,10.9,6.8,22.3,8.9,33.7c0.9,5.2,3.3,8.1,7.5,10.7
|
||||
c21.9,13.4,44.1,26,67.3,37.1c10.9,5.2,22.1,10,33.5,15.2c1.8-4.8,3.1-8.6,4.7-12.2c1.6-3.5,4.2-6.3,8.3-6.1
|
||||
c3.6,0.2,5.8,2.9,7.3,6.1c2.4,5.2,3.8,10.6,4.8,16.2c0.8,4.6,2.9,7.2,7.7,9C950.6,871.5,990.8,883,1032.8,891.9z M1953.8,1233.7
|
||||
c-2.9,5.8-4.9,11.5-8.8,16.2c-7.6,9.3-17.8,9.3-25.3,0c-3.4-4.2-5.5-9-7.5-14c-4.4-11.5-7.2-23.5-9.4-35.7
|
||||
c-0.7-3.9-2.4-6.4-5.8-8.6c-30.4-19.4-62.4-35.6-95-50.8c-6.9-3.3-7-3-9.9,4.7c-0.7,1.9-1.5,3.7-2.3,5.5c-1.6,3.1-3.7,5.6-7.6,5.6
|
||||
c-3.8,0-5.9-2.5-7.6-5.6c-2.9-5.4-4.3-11.3-5.1-17.2c-0.7-5-3-7.3-7.5-9.1c-37.8-15.2-77.3-24.6-116.7-34.3c-3.8-0.9-5.4,0.2-6.2,4
|
||||
c-0.7,3.9-0.2,9.9-6.3,9.5c-5-0.3-5.1-6.1-5.3-9.6c-0.4-6.6-3.3-8.9-9.6-10c-32.8-5.6-65.4-12.2-98.6-15.7
|
||||
c-8.3-0.9-16.5-2.1-24.8-2.7c-5.4-0.4-7.1,1.2-7.5,6.6c-0.3,3.9-1.3,8,5.2,8.4c6.3,0.4,12.4,2.1,18.7,3.1
|
||||
c47.7,7.5,94.4,19.1,140.3,34.3c42.4,14,83.5,31.2,123,52c46.3,24.4,90.2,52.6,131,85.6c48.5,39.1,93.7,81.2,129.4,132.6
|
||||
c1.9,2.8,4.8,7.5,7.3,6.7c4.5-1.5,1.4-6.8,2.1-10.4c0.6-3.4-0.5-6.6-1.8-9.7c-7.8-19.5-12.6-40-17.1-60.4c-1.1-5.1-3-9.4-6-13.5
|
||||
c-17-22.8-36.7-43.1-57.6-62.2C1959.6,1237.1,1957.6,1234.7,1953.8,1233.7z M2043.6,585.1c-4.2,2.7-6.2,6.2-8.5,9.2
|
||||
c-29.9,40.3-64.6,76.1-102.5,109c-33.8,29.3-69.5,56.2-107.6,79.4c-51.8,31.6-106.3,57.7-163.9,77.3
|
||||
c-40.7,13.8-82.1,24.9-124.2,33.1c-13.7,2.7-27.6,4.7-41.3,7c-2.2,0.4-5.3-0.1-6.2,2.4c-1.4,3.8-1.1,8.2,0.3,11.7
|
||||
c1.4,3.5,5.3,1.6,8.1,1.3c40.1-4.2,79.8-10.7,119.5-18.1c6.2-1.2,10.4-2.7,10.5-10.2c0-3.5,0.3-9.3,5.4-9.5c6-0.3,5.5,5.7,6.2,9.6
|
||||
c0.7,3.9,2.3,4.9,6.1,3.9c10-2.7,20-5.1,30-7.5c28.5-6.9,56.4-15.6,83.9-25.7c6.8-2.5,10.2-6.3,11-13.3c0.5-5,2-9.8,4.5-14.2
|
||||
c4.2-7.6,10.8-7.6,15.1,0.1c1.6,2.9,2.5,6.2,3.4,9.3c0.9,2.9,2.5,3.5,5.2,2.2c2.4-1.2,5-1.9,7.3-3.1c26.6-13.1,53.2-26,78.7-41.2
|
||||
c11.9-7.1,19.6-14.7,21.3-29.3c1.4-12.4,5.5-24.8,12.8-35.4c8.1-11.7,19.5-11.7,27.5,0.1c3.1,4.6,5.4,9.7,8.2,14.9
|
||||
c1.9-1.4,3.6-2.5,5.1-3.8c22.2-19.8,42.7-41.1,60.6-64.9c2.6-3.5,4.3-7.3,5.2-11.7c4.5-21.2,9.6-42.2,17.6-62.4
|
||||
C2045.3,599.2,2044.5,592.6,2043.6,585.1z M777.2,1163.8c1.2,0.2,1.6,0.4,1.9,0.3c30.4-14.3,60.9-28.3,92.6-39.6
|
||||
c4.2-1.5,4.4-3.7,3.9-7.5c-4.5-37.4-6.3-75-6.5-112.6c0-7.8-0.2-7.9-8.5-7.9c-22,0-44,0-65.9,0c-8.6,0-8.8,0.1-8.8,8.7
|
||||
c-0.4,45.3-2.2,90.5-6.6,135.6C778.4,1148.4,776.3,1155.9,777.2,1163.8z M776.8,817.8c0.3,3.8,0.5,7.1,0.9,10.4
|
||||
c2.5,21.2,4.1,42.4,5.5,63.7c1.8,28.6,2.3,57.2,2.7,85.9c0.1,7.1,0.3,7.3,7.2,7.3c22.6,0.1,45.3,0.1,67.9,0.1
|
||||
c8.8,0,8.1-0.2,8.1-8.3c0.1-37,1.9-73.9,6.4-110.6c0.7-5.8-0.5-8.2-6-10.1c-29.5-10.3-58-23.3-86.2-36.8
|
||||
C781.5,818.4,779.9,817.2,776.8,817.8z M1896.4,1163.9c1-2.7,0.2-5.3-0.1-7.9c-2.4-19.8-3.9-39.8-5.3-59.7
|
||||
c-2.1-30.3-2.5-60.6-3.3-90.9c-0.2-8.7-0.2-8.8-8.6-8.8c-16.7-0.1-33.3,0-50-0.1c-6.3,0-12.7,0.2-19-0.1c-3.9-0.1-5.5,1.4-5.2,5.3
|
||||
c0.1,1.7,0.1,3.3,0,5c-1.1,36-2,72-6.4,107.7c-0.9,7.7-1.1,7.9,6.3,10.6c28.5,10.4,56.1,22.9,83.3,36.1
|
||||
C1890.6,1162.4,1893.1,1164.4,1896.4,1163.9z M1896.8,817.7c-2.5-0.3-4.3,0.6-6.1,1.5c-6,2.9-12,5.6-18,8.6
|
||||
c-22.1,10.9-44.9,20.1-67.9,28.7c-7.5,2.8-7.3,3-6.5,10.6c1,8.9,1.9,17.9,2.7,26.8c2.4,28.2,3.2,56.5,3.6,84.7
|
||||
c0.1,5.4,1.8,6.7,6.8,6.6c22.6-0.2,45.3-0.1,67.9-0.2c1,0,2-0.1,3,0c3.8,0.3,5.6-1.2,5.2-5.1c-0.2-1.6,0-3.3,0.1-5
|
||||
c0.7-38.6,1.7-77.2,5.2-115.6C1894.1,845.4,1895.5,831.6,1896.8,817.7z M908.3,1109.6c1.4,0,2.4,0.2,3.2,0
|
||||
c37.8-11.7,75.5-24,114.2-32.5c4.5-1,5.8-2.9,5.5-7.4c-1.3-22-2.5-43.9-2.3-65.9c0.1-7.3-0.2-7.4-8-7.4c-33.3,0-66.7,0.1-100,0.2
|
||||
c-7.7,0-7.9,0.1-7.9,7.5c0,31-1.6,61.9-4,92.9C908.8,1101,907.3,1105.2,908.3,1109.6z M1765.3,1110.4c0-2.5,0.1-3.5,0-4.4
|
||||
c-3.1-33.9-4.7-67.8-4.8-101.8c0-7.4-0.4-7.6-8.2-7.7c-33.3-0.1-66.7-0.2-100-0.2c-7.3,0-7.2,0.1-7.5,7.9c-0.7,21-1.1,42-2.3,62.9
|
||||
c-0.5,9-1.3,8.3,7.6,10.4c29.8,7.1,59.4,14.9,88.5,24.5C1747.2,1104.8,1755.7,1107.4,1765.3,1110.4z M908.8,872.2
|
||||
c-0.2,0.3-0.6,0.6-0.6,0.8c2.8,35.5,5.2,71,4.8,106.7c-0.1,4.6,1.9,5.6,6,5.6c34.7-0.1,69.3-0.1,104,0.1c4.9,0,6.2-1.8,6.1-6.4
|
||||
c-0.5-22,0.8-44,2.1-65.9c0.3-5.2-0.9-7.7-6.4-8.8c-36.9-7.6-72.4-20.1-108.5-30.5C913.8,873,911.6,871.1,908.8,872.2z
|
||||
M1765.6,872.4c-3.2-0.6-5.1,0.3-7,0.9c-22.2,6.7-44.2,14.5-66.7,20.3c-14.5,3.7-29,7.3-43.5,10.8c-4.4,1.1-6.5,2.6-6.2,8
|
||||
c1.3,21.9,1.8,43.9,2.6,65.8c0.2,7,0.2,7.1,7.1,7.1c33.6,0,67.2,0,100.8-0.1c7.5,0,7.7-0.2,7.7-7.8c0.2-24.6,0.9-49.2,2.6-73.8
|
||||
C1763.8,893.4,1764.7,883.1,1765.6,872.4z M725.8,1192.5c8.7-4.8,15.8-9.8,23.8-12.8c9.6-3.7,12.7-10.5,13-19.7
|
||||
c0.1-1.3,0.2-2.7,0.4-4c3.4-27.1,5.6-54.3,6.7-81.5c1-23.6,1.5-47.2,2-70.8c0.2-7-0.1-7-7.3-7.1c-6.7-0.1-13.3,0-20,0
|
||||
c-9,0-18,0.2-27,0c-4.2-0.1-6.2,1.3-5.9,5.7c0.2,2.3,0,4.7,0.1,7c0.8,27.6,1.2,55.2,3,82.8C717,1125.1,719.9,1158.2,725.8,1192.5z
|
||||
M1946.2,1191.7c2.9-2.2,2.4-4.7,2.8-6.9c4.5-26.6,6.9-53.5,8.9-80.4c2.5-33.9,3.2-67.9,3.7-101.8c0.1-5-1.8-6.2-6.4-6.1
|
||||
c-15.3,0.2-30.6,0-46,0.1c-7.6,0-7.8,0.2-7.7,7.8c0.7,44,2,87.9,6.7,131.6c1.1,10.3,2.2,20.5,3.5,30.7c0.4,2.8,0.2,6,3.5,7.8
|
||||
C1925.8,1180.2,1936.2,1186.1,1946.2,1191.7z M1947.7,789.1c-9.1,5-16.9,10.1-25.3,13.8c-8.2,3.6-10.5,9.5-11.5,17.8
|
||||
c-3,27.1-5.6,54.3-6.9,81.6c-1.2,25.3-1.5,50.6-2.5,75.9c-0.2,5.1,1.5,7,6.7,7c15-0.3,30-0.1,45-0.2c8.6,0,8.4,0.6,8.3-8.9
|
||||
c-0.3-30.6-1-61.3-3.1-91.9C1956.3,852.9,1953.6,821.8,1947.7,789.1z M726,789.2c-3.3,16.4-5.1,31.3-6.7,46.2
|
||||
c-4.6,42.4-6.3,85-7.3,127.7c-0.1,5.7,0,11.3-0.2,17c-0.1,3.4,1.4,4.8,4.7,4.8c17,0,34,0,51,0.1c3.6,0,4.6-1.9,4.4-5
|
||||
c-0.1-2.3-0.1-4.7-0.1-7c-0.5-37.3-1.7-74.6-5-111.8c-1.4-15.9-2.8-31.9-5.5-47.6c-0.4-2.6-0.3-5.2-3.2-6.8
|
||||
C747.6,801.1,737.2,795.4,726,789.2z M1625.1,910.5c-9.8,1.8-18,3.4-26.2,4.6c-23.3,3.5-46.2,9.9-70,10
|
||||
c-10.9,0.1-21.8,2.4-32.7,3.7c-3.5,0.4-6.2,1.1-6,6c0.5,15,0.7,30,0.7,44.9c0,4.4,1.7,5.8,5.9,5.7c8.3-0.2,16.6,0,25-0.1
|
||||
c30.6,0,61.3,0,91.9-0.1c9,0,9-0.1,9.2-9.4c0.2-13,0.2-26,0.6-38.9C1623.7,928.4,1624.5,919.8,1625.1,910.5z M1625.2,1071.1
|
||||
c-2.2-23.8-2.3-46.1-2.4-68.4c0-4.5-1-6.6-6.1-6.6c-39.9,0.1-79.9,0.1-119.8-0.1c-5.1,0-6.1,2-6.1,6.5c0,14.6-0.1,29.3-0.6,43.9
|
||||
c-0.1,4.5,1.6,5.6,5.6,6.1c12.6,1.4,25,3.8,37.7,4c8,0.1,15.9,1.1,23.8,2.5C1579.5,1063,1601.7,1066.9,1625.2,1071.1z
|
||||
M1049.3,1070.4c2.6,1,4.9-0.1,7-0.6c23-5.5,46.3-8.9,69.7-12.3c16.7-2.5,33.7-3.8,50.6-5.2c5.1-0.4,6.8-1.9,6.6-7
|
||||
c-0.4-14-0.3-27.9-0.5-41.9c-0.1-7-0.2-7.2-7.2-7.2c-39.3,0-78.5,0-117.8,0.1c-6.7,0-6.8,0.2-6.8,7.5c0,16.3-0.5,32.6-1.4,48.9
|
||||
C1049.3,1058.6,1048.4,1064.5,1049.3,1070.4z M1049.8,910.2c-0.4,1.2-0.8,1.8-0.8,2.4c0.6,22.6,2.5,45.2,1.9,67.8
|
||||
c-0.1,4.2,2.1,4.8,5.6,4.8c39.9,0,79.9-0.1,119.8,0.1c5.5,0,6.7-2.3,6.6-7.2c-0.2-11,0.1-22,0.2-32.9c0.1-15.9,0.1-15.5-16.3-17.4
|
||||
c-16.5-1.9-33.1-2.4-49.6-5.2C1094.7,918.9,1072,915.9,1049.8,910.2z M1260.5,985.4c21.6,0,43.2,0,64.9,0c9.1,0,9.1-0.1,9.1-9.4
|
||||
c0-10.3,0-20.6-0.1-30.9c-0.1-7.7-0.1-7.7-7.6-7.9c-2.3-0.1-4.7,0-7,0c-41.9,0-83.8-2.2-125.5-6.2c-4.6-0.4-5.9,0.9-6,5.1
|
||||
c-0.2,13.6-0.6,27.3-0.8,40.9c-0.1,8.3,0.1,8.4,8.1,8.4C1217.3,985.4,1238.9,985.4,1260.5,985.4z M1485,931.5
|
||||
c-3.1-1.4-5.4-0.6-7.7-0.4c-26.2,1.9-52.4,4.3-78.7,5.1c-17.3,0.5-34.6,1.3-51.9,1.1c-7.4-0.1-7.5,0.1-7.6,7.8
|
||||
c-0.1,11.7,0,23.3-0.1,35c0,3.7,1,5.8,5,5.5c2-0.1,4,0,6,0c41.6,0,83.2,0,124.9,0c2,0,4-0.1,6,0c3.3,0.1,5-1.2,4.9-4.6
|
||||
C1485.5,964.2,1485.3,947.6,1485,931.5z M1485,1051.1c0.3-17,0.5-33.6,0.9-50.1c0.1-4.2-2.2-4.8-5.7-4.8c-45,0.1-89.9,0.1-134.9,0
|
||||
c-4.7,0-6.5,1.4-6.3,6.2c0.3,12,0.4,24,0,36c-0.2,5.2,2.1,6,6.5,6c11.6-0.1,23.3,0.1,34.9,0.4
|
||||
C1415,1045.8,1449.5,1048.1,1485,1051.1z M1188.4,1050.6c47.5-3.4,93.2-6.8,139.2-6.4c0.7,0,1.3,0,2,0c3.2,0.2,4.9-0.9,4.9-4.5
|
||||
c-0.1-13,0-25.9,0-38.9c0-3.4-1.6-4.8-4.9-4.6c-1.7,0.1-3.3,0-5,0c-42.6,0-85.1,0-127.7,0c-1.7,0-3.3,0.1-5,0
|
||||
c-2.8-0.1-4.4,1.1-4.4,4.1C1187.9,1016.8,1188.1,1033.4,1188.4,1050.6z M2010.7,1230.1c0.9-7.3-0.9-13.8-1.5-20.3
|
||||
c-6.8-66-10.3-132.2-10.7-198.6c-0.1-16.1-0.3-16.3-16.8-14.7c-2.9,0.3-4.2,1.1-4.1,4.1c0.1,3,0,6,0,9
|
||||
c-0.3,41.3-2.1,82.6-6.1,123.8c-2,20.2-4.3,40.4-8.2,60.4c-0.9,4.8-0.2,7.8,4.5,10.4c10.2,5.7,19.9,12.2,29.8,18.3
|
||||
C2001.8,1225,2005.6,1228.2,2010.7,1230.1z M2011.2,752.5c-2.7-0.8-3.6,0.3-4.7,1c-13.3,8.3-26.4,16.7-39.8,24.7
|
||||
c-3.6,2.1-4.2,4.5-3.5,8.2c1.4,7.8,2.8,15.7,3.9,23.6c7.4,53.8,10.1,107.9,10.6,162.2c0.1,14.1-2.6,12.8,13.7,12.8
|
||||
c7,0,7-0.2,7.1-7.3c0.4-27.3,0.6-54.6,1.8-81.8c1.4-30.9,3.1-61.8,6.1-92.6C2008,786.3,2009.6,769.5,2011.2,752.5z M662.1,1231.5
|
||||
c3.7-2.4,5.6-3.7,7.5-4.9c11.6-7.7,22.9-16.2,35.5-22.2c5.3-2.5,6.2-5.5,5.1-11c-4.6-22.5-7-45.4-9-68.3
|
||||
c-3.5-39.9-5-79.8-5.4-119.8c-0.1-8.6-0.2-8.6-8.8-8.7c-1.3,0-2.7,0-4,0c-7.9,0.1-8,0.1-7.9,8.4c0,24.7-0.6,49.3-1.6,74
|
||||
c-1.3,32-3,63.9-6,95.8C665.8,1192.9,664.1,1211.1,662.1,1231.5z M662.9,752.1c-1,1.4-0.4,3-0.3,4.6c2,20.5,4.6,41.1,6,61.7
|
||||
c3.8,52.5,6.5,105.1,6.4,157.8c0,8.8,0.1,8.9,8.5,8.9c14.5-0.1,12.1,1.3,12.3-12.1c1.1-62.7,3.5-125.2,14.8-187
|
||||
c0.8-4.1-0.9-6-4.2-7.7c-13.4-6.8-25.6-15.5-38-23.9C666.8,753.1,665.5,751.1,662.9,752.1z M2044.3,728.6c-5.4,1.1-7.1,3.9-7.7,8.3
|
||||
c-1.7,14.9-4,29.7-5.6,44.6c-4.6,42.1-7.2,84.2-8.9,126.5c-1,25.3-1.3,50.5-1.3,76.8c6.5,0,12.4-0.1,18.3,0
|
||||
c3.8,0.1,5.5-1.4,5.2-5.3c-0.2-1.7,0-3.3,0-5c0-78.9,0-157.9,0-236.8C2044.3,734.9,2044.3,732,2044.3,728.6z M2044.3,1253
|
||||
c0-3.7,0-6.4,0-9c0-78.6,0-157.2,0-235.8c0-13,1.3-11.5-11.9-11.5c-13,0-11.5-1.8-11.5,11.8c0,59.3,3.1,118.5,8.7,177.6
|
||||
c1.9,19.5,4,39.1,7,58.5C2037.3,1248.9,2038.8,1251.8,2044.3,1253z M629.2,728.7c0,4.3,0,8,0,11.6c0,76.5,0,153.1,0,229.6
|
||||
c0,15.8,0,15.8,16,15.3c7.6-0.2,7.6-0.2,7.7-7.7c0.1-3.3,0-6.7,0-10c-0.8-55.6-3.1-111.1-8.5-166.4c-2-21.2-5-42.3-7.3-63.4
|
||||
C636.6,733.3,635,730.3,629.2,728.7z M629.2,1253.2c5.3-2.1,7.4-4.5,7.9-9.2c1.6-14.9,4-29.7,5.5-44.6
|
||||
c6.5-62.4,9.5-124.9,10.3-187.6c0.2-15.6,0.1-15.6-15-15.6c-9,0-8.8-0.6-8.7,8.6c0,2,0,4,0,6c0,75.7,0,151.3,0,227
|
||||
C629.2,1242.5,629.2,1247.2,629.2,1253.2z M2060,697.5c19-15.9,36.7-29.3,52.3-45.3c3.5-3.5,3.7-6.7,2.4-10.7
|
||||
c-2.8-8.9-5.4-17.8-8.7-26.5c-3.5-9.3-7.4-18.5-13.7-26.4c-5.3-6.6-8-6.6-13.4-0.4c-1.3,1.5-2.7,3-3.6,4.7
|
||||
c-8.9,15.7-17.3,31.5-15.3,50.5c0.4,3.6,0.1,7.3,0.1,11C2060,668,2060,681.5,2060,697.5z M613.2,1283.1
|
||||
c-20.8,13.6-36.9,29.9-53.8,45.4c-2.8,2.6-2.3,5.1-1.5,8.1c4.1,14.7,8.6,29.3,15.3,43.2c2.6,5.4,5.4,10.7,9.8,15
|
||||
c4.3,4.2,6.2,4.6,10,0.4c10.7-11.9,18.4-26.3,19.7-41.7C614.7,1330.6,613.2,1307.6,613.2,1283.1z M2060,1284.4
|
||||
c0,19.5,0.7,36.8-0.2,54c-1.1,19.1,6.3,35.2,15.8,50.5c7.4,12.1,12.1,11.7,19.8-0.6c9.7-15.4,14.6-32.7,19.7-50
|
||||
c1.1-3.7,0-6.2-2.6-8.8C2096.8,1313.6,2079.3,1299.9,2060,1284.4z M611.9,698.2c2.1-3.3,1.3-5.9,1.3-8.4c0.1-16-0.8-32,0.3-47.9
|
||||
c1.2-18.4-6.2-33.8-15.1-48.6c-7.6-12.7-12.6-12.4-20.4,0.3c-10.1,16.3-15.1,34.6-20.4,52.8c-1,3.6,0.6,5.5,3,7.7
|
||||
c8.8,8.1,17.3,16.7,26.3,24.5C594.8,685.4,603.3,691.5,611.9,698.2z M1788.2,864c-12.5,3.2-12.6,3.2-13.9,13.8
|
||||
c-0.9,7.6-1.6,15.2-2.1,22.8c-1.8,26.2-2.9,52.5-3.2,78.8c0,4,0.8,6.3,5.4,6c4.6-0.3,9.3-0.2,14,0c5.1,0.1,6.9-2.1,6.7-7.3
|
||||
c-0.5-11.3-0.2-22.6-0.7-33.9C1793.2,917.5,1792.3,890.9,1788.2,864z M885.3,1118.4c12.8-3,12.5-3,13.8-13.9
|
||||
c3.5-29.5,4.6-59.1,5-88.8c0.3-23.5,5.2-18.7-19.8-19.4c-4.4-0.1-5.7,1.5-5.7,5.8C878.8,1040.8,880.3,1079.4,885.3,1118.4z
|
||||
M885.7,863.5c-0.2,0.9-0.6,1.9-0.7,2.9c-4.3,33.7-6,67.5-6.1,101.4c0,5.5-2.5,12.6,1.3,16.2c3.8,3.6,10.8,1.1,16.3,1.2
|
||||
c7.9,0.1,8.1-0.1,7.9-8.4c-0.6-33.2-1.5-66.4-5.3-99.4C897.9,865.6,898,865.6,885.7,863.5z M1788.4,1117.5
|
||||
c4.4-34.4,5.9-68.6,6.5-102.9c0.1-5.6,2.4-12.7-0.9-16.5c-3.9-4.3-11.3-1.4-17.2-1.6c-7.6-0.2-7.7,0.1-7.7,7.6
|
||||
c0,24.7,1.3,49.3,2.8,73.9c0.6,10.3,2.2,20.5,2.9,30.8c0.3,3.6,1.7,5.5,4.9,6.5C1782.2,1116,1784.4,1117.7,1788.4,1117.5z
|
||||
M1481.1,1078.1c0-2.7,0.1-4.4,0-6c-0.3-7.8-0.3-8.1-8.2-8.7c-22.9-1.7-45.8-3.3-68.7-4.7c-19.9-1.2-39.9-0.9-59.9-1
|
||||
c-4.6,0-5.5,2-5.4,6c0.1,3.7,0.3,6.7,5.2,6.2c1.3-0.1,2.7,0,4,0c34.3-0.3,68.4,1.9,102.5,5.6
|
||||
C1460.6,1076.6,1470.4,1078.5,1481.1,1078.1z M1480,903c-15.3,1.6-30.8,3.6-46.3,4.8c-29.5,2.4-59.1,4.7-88.8,3.9
|
||||
c-5.1-0.1-6.1,2-6,6.5c0.1,4.1,1.1,6.3,5.6,5.8c1.3-0.2,2.7,0,4,0c42.3,0.3,84.5-2.6,126.7-5.9c4.2-0.3,6.2-1.6,5.8-5.9
|
||||
C1480.7,909.3,1482,906.3,1480,903z M1192.2,1078.2c3.1-0.2,5.4-0.3,7.6-0.5c37-4.6,74-7.6,111.3-7.8c6,0,12-0.1,18,0
|
||||
c3.7,0.1,5.3-1.1,5.4-5.1c0.1-6.5,0-6.9-7.3-7c-40.6-0.8-81.1,1.7-121.6,4.6C1193.1,1063.3,1193.2,1063.6,1192.2,1078.2z
|
||||
M1192.6,903.2c0.2,14.4,0.2,14.4,12.8,16c3,0.4,6,0.6,8.9,0.8c19.3,1.1,38.5,2,57.8,3.2c18.6,1.2,37.2,0.8,55.9,0.8
|
||||
c6.4,0,6.3-0.5,6.6-6.8c0.2-5.8-3.3-5.5-7.1-5.4c-28.3,0.8-56.5-1.5-84.7-3.6C1226.3,907,1209.8,905.4,1192.6,903.2z M653.6,694.6
|
||||
c2,12.3,3.8,22.4,5.1,32.6c0.5,3.7,1.7,6.1,4.8,8c14.2,8.8,27.5,18.8,42.2,26.6c2.7,1.4,6.2,6.3,8.9,2.4c2.7-3.7-2.6-6.2-5-8.5
|
||||
c-19-17.3-36.7-35.8-51.8-56.7C657,698.1,656,697.3,653.6,694.6z M654,1284.6c0.8,0,1.3,0.1,1.5,0c1.4-1.4,2.9-2.7,4-4.3
|
||||
c14.6-20.8,32.8-38.3,51.2-55.5c1.2-1.1,2.6-2.1,3.6-3.4c1.4-1.7,1.5-3.6-0.2-5.7c-2.9,0.3-5.3,2.1-7.8,3.6
|
||||
c-10.2,6.3-19.8,13.5-30.5,18.8c-12.9,6.4-19.3,15.7-19.2,30.2C656.6,1273.7,654.9,1279.1,654,1284.6z M2019.1,698.8
|
||||
c-0.5-0.3-1-0.6-1.5-0.9c-1,1.2-2,2.4-3,3.6c-6.8,9-13,18.4-21.4,26.1c-11,10.1-20.5,21.7-32.4,30.8c-1.9,1.5-3.6,3.2-2.1,5.9
|
||||
c2,3.5,4.1,0.5,5.7-0.4c15.4-9.7,30.8-19.6,46.1-29.5c2.3-1.5,4-3.1,4.4-6.2C2016.2,718.5,2017.7,708.6,2019.1,698.8z
|
||||
M2017.1,1283.9c0.7-0.2,1.4-0.5,2.1-0.7c-1.5-10.1-3-20.2-4.4-30.4c-0.4-2.6-1.9-4.1-3.9-5.4c-15.6-10-31.3-20.1-46.9-30
|
||||
c-1.5-1-3.3-2.8-5-0.5c-1.7,2.4-0.3,4.2,1.7,5.9c4,3.4,8.1,6.9,11.8,10.6C1988.4,1249.3,2005.3,1264.3,2017.1,1283.9z M1044.6,1034
|
||||
c0.2,0,0.3,0,0.5,0c0-11-0.1-21.9,0.1-32.9c0.1-4.2-2-4.8-5.5-4.9c-4.5-0.1-4.6,2.6-4.6,5.9c0.1,22.3,1.2,44.5,2.4,66.8
|
||||
c0.1,1,0.2,2,0.4,3c0.2,1.2,1.2,1.8,2.1,1.7c0.8-0.1,1.9-1,2.2-1.7c0.5-1.5,0.7-3.2,0.8-4.9C1043.5,1055.9,1044,1045,1044.6,1034z
|
||||
M1636.4,1033.8c0.7,0,1.4,0,2.1,0c0-10.6,0-21.2,0-31.8c0-2.7,0.2-5.3-3.6-5.8c-4.7-0.6-6,0.4-6,6.1c-0.1,21.9,0.2,43.7,2.1,65.6
|
||||
c0.1,1.3,0.1,2.7,0.6,3.9c0.3,0.7,1.5,1.6,2.2,1.5c0.7,0,1.7-1.1,1.9-1.8c0.5-1.9,0.7-3.9,0.7-5.9
|
||||
C1636.4,1055,1636.4,1044.4,1636.4,1033.8z M1638.5,948.1c-0.2,0-0.5,0-0.7,0c-0.5-11.6-1-23.3-1.6-34.9c-0.1-1.9,0.2-4.8-2.3-4.9
|
||||
c-3.3-0.1-2.8,3-3,5.2c-0.3,3-0.4,6-0.7,9c-1.5,19-1.4,37.9-1.4,56.9c0,5.3,2,6.8,6.4,5.8c3.8-0.8,3.3-3.6,3.3-6.2
|
||||
C1638.5,968.8,1638.5,958.4,1638.5,948.1z M1045.1,948c-0.1,0-0.3,0-0.4,0c-0.6-11.9-1.2-23.9-1.9-35.8c-0.1-1.9-0.4-4.3-3.2-3.9
|
||||
c-2.2,0.3-1.9,2.4-2,4c-1.3,22.5-2.3,45.1-2.5,67.7c0,3.7,1.2,5.4,5.1,5.4c4.1,0.1,5-1.9,5-5.5C1045,969.3,1045.1,958.7,1045.1,948
|
||||
z M741.9,740.1c-4.9,5.5-6.3,11-8.3,16.2c-0.8,2-0.8,4,1.2,5.3c6.2,4.1,11.6,9.4,19.7,13C751.6,762.1,748.8,750.9,741.9,740.1z
|
||||
M1931.7,1241.3c11.7-16.6,11.5-18.2-3-28.3c-2.7-1.9-4.9-4.5-9.1-4.9C1921.9,1219.6,1924.7,1230.7,1931.7,1241.3z M1931.6,740
|
||||
c-6.6,10.9-9.8,22-12.1,33.9c4-0.9,6.1-3.1,8.5-4.8C1943.5,758.3,1943.6,757.2,1931.6,740z M742.1,1241.3
|
||||
c6.6-11.1,9.8-21.9,12.2-34.3c-8,3.6-13.2,8.8-19.2,12.6c-2,1.3-2.3,3.1-1.6,5.1C735.6,1230.2,736.7,1236.2,742.1,1241.3z
|
||||
M597.6,1464.4c16.9-6.5,17.3-8.9,14-24.4C605.8,1447.2,601.7,1455,597.6,1464.4z M2061.5,1442.5c-5.1,14.3,5.6,17.3,12.5,22.8
|
||||
C2072.1,1457,2066.2,1451,2061.5,1442.5z M1781.6,836.9c-1,5.2-3.6,8.5-1.8,12.6C1785.7,846.2,1785.7,846.2,1781.6,836.9z
|
||||
M895.4,849.3c-1.5-4.5-1.6-7.9-3.8-11.1C887.9,846.3,887.9,846.3,895.4,849.3z M630.1,1284.6c-0.4-5.3,4.1-10.2,1.8-17
|
||||
C627.4,1273.6,629.9,1279.2,630.1,1284.6z M629.9,697.1c0.2,5.4-3,11.4,2.4,16.6C633.8,707.4,630.4,702.5,629.9,697.1z
|
||||
M2044.5,1284.5c-1-5.2,1.5-10.3-2.8-15C2040.3,1275.4,2042.4,1279.6,2044.5,1284.5z M2041.9,712.1c4.1-4.8,1.6-10,2.2-14.6
|
||||
C2043,702.1,2039.8,706.2,2041.9,712.1z M892.2,1143.9c0.4-4.2,3.5-7.3,1.6-11.5C887.6,1136,887.6,1136,892.2,1143.9z
|
||||
M1779.6,1132.2c-1,2.9-0.4,4.8,0.4,6.5c0.7,1.5,0.4,3.6,2.3,4.6C1785.7,1135.3,1785.7,1135.3,1779.6,1132.2z M729.5,774
|
||||
c1.5,1.6,2.6,4,5.8,3.2C733.9,774.7,732.2,773.5,729.5,774z M737.9,778.4c-0.2,0.2-0.5,0.4-0.7,0.6c0.2,0.2,0.4,0.6,0.6,0.6
|
||||
c0.2,0,0.5-0.3,0.7-0.5C738.4,778.8,738.1,778.6,737.9,778.4z"/>
|
||||
<path class="st1" d="M534.9,1353.5c4.5,11.6,7.4,22.2,12.2,32c4.7,9.6,9.8,18.8,17.1,26.8c10.6,11.6,28.8,15.5,42.7,2.9
|
||||
c1.4-1.3,3.1-2.4,5.5-4.2c1.4,7.2,1.7,13.2-2.5,19.1c-8.4,11.7-14.8,24.4-20.8,37.4c-1.7,3.7-3.7,6.7-7.3,9.1
|
||||
c-52.7,33.7-105.3,67.6-158,101.4c-1.1,0.7-2.1,1.7-4.1,1.1c0-195.1,0-390.4,0-585.6c2.1-1.2,4.1-0.9,6-0.8
|
||||
c21.6,0.5,43.3,1.1,64.9,1.3c5.3,0,6.7,1.8,6.7,6.9c0.5,40,1.4,79.9,3.9,119.8c2.4,38.2,5.6,76.4,10.6,114.3
|
||||
c4.6,34.7,10.1,69.1,19.2,103c1.1,4,0.6,7.1-2.3,10c-0.9,0.9-1.7,2.1-2.6,3.1c-22.1,25.1-40.6,52.7-57.5,81.5
|
||||
c-11.6,19.7-19.9,41-28.9,61.9c14.9-23.9,25.7-50.1,41.8-73.5C497.6,1397.6,514.9,1375.3,534.9,1353.5z"/>
|
||||
<path class="st1" d="M534.6,627.2c-40-41.2-69.6-89-94.2-140.1c0.7,6.2,4,11.2,6.2,16.6c12.7,31.7,30.5,60.6,49.6,88.7
|
||||
c9.3,13.7,20.4,26.3,31,39.1c3.8,4.5,5,8.7,3.4,14.4c-9.6,35.4-15,71.6-19.6,107.9c-9.4,74.5-12.5,149.3-13.7,224.3c0,1-0.1,2,0,3
|
||||
c0.6,4.7-1.2,6.5-6.1,6.4c-8.3-0.2-16.7,0.2-25,0.4c-13.7,0.3-27.3,0.4-41,1c-4.8,0.2-6.8-1.2-6.2-6.1c0.3-2,0-4,0-6
|
||||
c0-187.9,0-375.8,0-563.7c0-3.5,0-7.1,0-10.7c4.2-0.1,6.3,2.3,8.7,3.9c50.4,32.4,100.8,64.9,151.3,97.1c5,3.2,8.4,7.1,10.8,12.4
|
||||
c5.1,11.2,10.4,22.4,17.8,32.3c4.9,6.6,6.7,13.5,5.1,21.9c-3.6-0.1-5.6-3.1-7.9-5c-12-9.8-28.1-7.7-38.9,2.8
|
||||
c-12.6,12.2-19,27.9-25.2,43.7C538.8,616.5,538.2,622.2,534.6,627.2z"/>
|
||||
<path class="st1" d="M2060.9,1411.1c7.4,4.3,12.3,10.3,20.6,11.4c9.8,1.4,17.7-1.4,24.8-7.5c8.5-7.3,13.9-16.9,18.8-26.8
|
||||
c5.1-10.4,8.6-21.5,12.7-32.5c24.7,24.7,73.3,78.9,107.2,169.7c-1.1-8.8-3.5-17.2-6.8-25.4c-13.2-33.9-29.3-66.4-49.3-96.8
|
||||
c-12.8-19.5-27.4-37.7-42.4-55.5c-3.4-4-4.8-7.1-3.5-12.3c12.8-48.8,19.1-98.6,24-148.7c5.9-59.7,8.4-119.6,8.9-179.7
|
||||
c0-1.3,0-2.7,0.1-4c0.2-7,0.2-7.1,7-7.4c20-0.7,40-1.3,60-1.9c2.3-0.1,4.7,0,7-0.1c3.5-0.2,4.8,1.4,4.6,4.8c-0.1,2,0,4,0,6
|
||||
c0,188.3,0,376.6,0,565c0,3.2,0,6.4,0,9.8c-4.5-0.2-6.9-2.8-9.5-4.5c-47.7-30.6-95.2-61.4-143-91.7c-9-5.7-16-11.6-19.9-22.4
|
||||
c-3.7-10.1-11.2-18.9-17.1-28.2C2060.8,1426.2,2058.6,1419.6,2060.9,1411.1z"/>
|
||||
<path class="st1" d="M2244.8,454.3c-23,66.2-59.6,121.8-106.6,172.4c-4.5-11.6-7.7-22.6-12.8-32.9c-4-8.1-8.4-15.8-14.4-22.7
|
||||
c-12.6-14.4-29.9-17.7-45.6-3.8c-1.1,1-2,2.7-4.6,2.6c-1.4-5.5-1.1-10.5,2-15.7c8.8-14.9,18.9-29,25.8-45c1-2.3,2.7-3.7,4.7-4.9
|
||||
c52.7-33.9,105.3-67.7,158-101.5c0.5-0.3,1.3-0.2,2.1-0.4c2.3,2.1,1.2,4.9,1.2,7.3c0.1,189.6,0.1,379.3,0.1,568.9
|
||||
c0,1.7-0.1,3.3,0,5c0.2,3.6-1.7,4.6-4.9,4.5c-5-0.2-10-0.3-15-0.5c-17-0.5-34-1-51-1.4c-7.4-0.2-7.5-0.2-7.6-7.8
|
||||
c-0.5-35.3-1.1-70.6-3.1-105.9c-2.6-45.9-6.4-91.7-12.8-137.3c-4.2-30.3-9.4-60.5-17.3-90.2c-1.1-4-0.2-6.6,2.5-9.8
|
||||
c27.4-31.6,51.5-65.6,70.4-102.8c11.3-22.2,21.7-44.9,28.2-69.2C2244.6,460.9,2245.4,458.8,2244.8,454.3z"/>
|
||||
<path class="st1" d="M553.7,662.8c3.8,0.2,5.2,2.8,6.9,4.6c14.5,15.2,30.3,29,46.9,41.9c4.1,3.2,5.8,6.4,5.8,11.7
|
||||
c-0.2,85.6-0.1,171.2-0.1,256.8c0,7.5-0.1,7.7-7.6,7.8c-25,0.4-50,0.6-74.9,1.1c-4.3,0.1-5.9-1.2-5.7-5.6c0.4-13,0.2-26,0.5-39
|
||||
c1.5-60.3,4.7-120.5,11.8-180.4c3.7-31.7,8.3-63.3,15.4-94.5C553,665.6,553.4,664.1,553.7,662.8z"/>
|
||||
<path class="st1" d="M553.6,1319.7c-22.4-107.4-27.4-215.3-28.7-323.3c3.3-2.5,6.4-1.5,9.3-1.5c23.3,0.3,46.7,0.7,70,1.1
|
||||
c9.5,0.1,9.2-1.2,9.2,9.2c0,72.3,0,144.7,0,217c0,13.3-0.1,26.7,0.1,40c0,3.6-0.7,6.2-3.8,8.6c-18.7,14.5-36.3,30.3-52.7,47.4
|
||||
C556.3,1318.5,555.6,1318.7,553.6,1319.7z"/>
|
||||
<path class="st1" d="M2119.8,1319.2c-6.7-6.5-12.1-12.3-18-17.5c-11.8-10.2-23.8-20.2-36-30.1c-4.2-3.4-5.9-7.1-5.9-12.7
|
||||
c0.2-84.6,0.1-169.1,0.1-253.7c0-8.4,0.1-8.5,8-8.5c24.3-0.2,48.6-0.4,72.9-0.5c7.5,0,7.7,0,7.6,7.7c-0.3,45.6-1.9,91.2-5.1,136.7
|
||||
c-2.7,39.9-6.7,79.6-12.5,119.1C2128.1,1279,2124.6,1298.3,2119.8,1319.2z"/>
|
||||
<path class="st1" d="M2120.4,664.4c12.3,57.5,18.7,115.7,22.8,174.2c3.1,44.9,4.7,89.8,5.2,134.8c0,2.3-0.1,4.7,0.1,7
|
||||
c0.3,4-1.7,5.2-5.4,5.1c-6.7-0.2-13.3-0.3-20-0.3c-18.3-0.1-36.7-0.2-55-0.4c-8-0.1-8.2-0.1-8.2-8.2c0-52.3,0-104.6,0-157
|
||||
c0-32,0.2-64-0.2-96c-0.1-6.3,2-10.3,6.8-14.3c17.1-14.3,35.1-27.7,50.2-44.2C2117.5,664.3,2118.3,663.8,2120.4,664.4z"/>
|
||||
<path class="st1" d="M878.7,1138.5c2,4.9,3.2,9.1,5.3,12.8c4,7.2,10.5,7.1,14.6,0c3-5.3,4.3-11.2,5.2-17.1c0.8-4.7,3.2-7.1,7.8-8.9
|
||||
c37.6-14.8,76.3-25.6,115.5-34.7c5-1.1,6.6,0.4,7.3,4.9c0.6,3.5,0.7,8.4,5.3,8.5c5.3,0.1,5.5-5.2,5.8-9c0.5-7.1,3.9-10,10.9-11.2
|
||||
c40-7,79.8-14.9,120.4-18c2.4-0.2,5.9-2.1,6.9,1.5c1,3.7,1.8,7.8,0.4,11.7c-0.7,1.8-2.9,1.2-4.4,1.5c-9.5,1.6-19.1,2.8-28.6,4.6
|
||||
c-39.6,7.4-78.6,17-117,29.2c-59.9,19-116.5,45.2-170.8,76.4c-41.6,24-80.2,52.3-117.2,83.1c-39.4,32.8-74.3,69.6-105.4,110.2
|
||||
c-3,3.9-6.2,7.7-9.4,11.6c-0.1,0.2-0.6,0.2-2.1,0.5c0.2-7-1.1-13.6,1.5-20.3c7.6-19.2,12.9-39.2,16.7-59.5
|
||||
c1.8-9.6,6.2-17.5,12-24.8c17.2-21.7,36.5-41.5,58.8-58.9c3.4,2.2,3.5,6.6,5.4,9.8c4,7,7.8,14.4,17.4,14.4
|
||||
c9.6,0,13.8-7.2,17.3-14.4c6.5-13.3,9.4-27.7,12.2-42c0.8-3.9,2.2-6.5,5.7-8.6C808.9,1171.4,842.7,1153.5,878.7,1138.5z"/>
|
||||
<path class="st1" d="M1032.8,891.9c-42-9-82.2-20.4-121.3-35.7c-4.8-1.9-6.9-4.4-7.7-9c-0.9-5.6-2.3-11.1-4.8-16.2
|
||||
c-1.5-3.2-3.7-5.9-7.3-6.1c-4.1-0.2-6.8,2.5-8.3,6.1c-1.6,3.6-2.8,7.4-4.7,12.2c-11.4-5.2-22.6-10-33.5-15.2
|
||||
c-23.1-11.1-45.4-23.7-67.3-37.1c-4.3-2.6-6.6-5.5-7.5-10.7c-2.1-11.4-4.6-22.8-8.9-33.7c-2.1-5.3-4.2-10.5-7.8-15
|
||||
c-7.6-9.4-17.5-9.4-25.3,0.1c-4.2,5.2-6,11.7-9.8,18.2c-11.1-9.9-22.3-18.8-32.2-28.9c-10.2-10.4-19.5-21.8-28.7-33.3
|
||||
c-5.6-7-8.9-15.3-10.6-24.4c-3.7-19.3-8.6-38.3-16-56.5c-2.9-7.1-1.4-14.1-0.5-21.8c3.9,4.8,7.9,9.4,11.6,14.3
|
||||
c23.7,32,51,60.7,80.1,87.7c19,17.6,39.4,33.9,59.9,49.7c45,34.7,93.8,63.3,144.7,87.9c39.2,18.9,79.8,34.7,121.6,47.1
|
||||
c41.2,12.2,82.8,22.3,125.4,28.2c1,0.1,2,0.2,3,0.4c7.3,1.2,10.5,7.7,7,14.2c-1.3,2.3-3.4,1.7-5.3,1.6c-8-0.7-15.9-1.5-23.8-2.4
|
||||
c-32.8-3.7-65.1-10.2-97.5-15.7c-7.1-1.2-11.5-3.9-11.7-11.6c0-1.3-0.4-2.6-0.8-3.9c-0.8-2.6-1.9-5.3-5.3-4.9
|
||||
c-2.7,0.3-3.7,2.6-4.4,4.9C1034.4,885.3,1033.7,888.2,1032.8,891.9z"/>
|
||||
<path class="st1" d="M1953.8,1233.7c3.8,1,5.8,3.4,8,5.3c21,19.1,40.7,39.4,57.6,62.2c3,4.1,4.9,8.4,6,13.5
|
||||
c4.5,20.5,9.3,40.9,17.1,60.4c1.3,3.1,2.4,6.3,1.8,9.7c-0.6,3.6,2.4,8.9-2.1,10.4c-2.5,0.8-5.4-3.9-7.3-6.7
|
||||
c-35.7-51.5-81-93.6-129.4-132.6c-40.8-32.9-84.7-61.2-131-85.6c-39.5-20.8-80.6-37.9-123-52c-45.9-15.2-92.6-26.8-140.3-34.3
|
||||
c-6.2-1-12.4-2.7-18.7-3.1c-6.5-0.4-5.5-4.5-5.2-8.4c0.4-5.4,2.1-7.1,7.5-6.6c8.3,0.7,16.5,1.9,24.8,2.7
|
||||
c33.2,3.4,65.8,10.1,98.6,15.7c6.3,1.1,9.2,3.4,9.6,10c0.2,3.6,0.3,9.3,5.3,9.6c6,0.4,5.5-5.6,6.3-9.5c0.7-3.9,2.4-5,6.2-4
|
||||
c39.4,9.7,78.9,19.2,116.7,34.3c4.5,1.8,6.8,4.2,7.5,9.1c0.9,5.9,2.2,11.8,5.1,17.2c1.7,3.1,3.8,5.6,7.6,5.6c3.8,0,6-2.5,7.6-5.6
|
||||
c0.9-1.8,1.7-3.6,2.3-5.5c2.8-7.7,2.9-7.9,9.9-4.7c32.6,15.3,64.6,31.4,95,50.8c3.4,2.2,5.1,4.7,5.8,8.6
|
||||
c2.1,12.1,4.9,24.1,9.4,35.7c1.9,5,4.1,9.8,7.5,14c7.5,9.4,17.7,9.4,25.3,0C1948.9,1245.3,1950.9,1239.5,1953.8,1233.7z"/>
|
||||
<path class="st1" d="M2043.6,585.1c0.9,7.5,1.7,14.1-0.8,20.4c-8,20.2-13.1,41.2-17.6,62.4c-0.9,4.4-2.6,8.3-5.2,11.7
|
||||
c-17.8,23.8-38.4,45.2-60.6,64.9c-1.5,1.3-3.2,2.3-5.1,3.8c-2.9-5.3-5.1-10.4-8.2-14.9c-8-11.8-19.3-11.8-27.5-0.1
|
||||
c-7.4,10.6-11.4,23-12.8,35.4c-1.7,14.6-9.4,22.2-21.3,29.3c-25.5,15.2-52.1,28.1-78.7,41.2c-2.4,1.2-5,2-7.3,3.1
|
||||
c-2.7,1.3-4.3,0.7-5.2-2.2c-1-3.2-1.9-6.4-3.4-9.3c-4.2-7.7-10.8-7.7-15.1-0.1c-2.5,4.4-3.9,9.3-4.5,14.2c-0.8,7-4.2,10.7-11,13.3
|
||||
c-27.5,10.2-55.4,18.9-83.9,25.7c-10,2.4-20.1,4.8-30,7.5c-3.8,1-5.4,0-6.1-3.9c-0.7-4-0.2-9.9-6.2-9.6c-5.1,0.2-5.3,6-5.4,9.5
|
||||
c-0.1,7.5-4.3,9-10.5,10.2c-39.6,7.4-79.4,13.9-119.5,18.1c-2.8,0.3-6.7,2.3-8.1-1.3c-1.4-3.6-1.7-7.9-0.3-11.7
|
||||
c0.9-2.4,3.9-2,6.2-2.4c13.8-2.4,27.6-4.4,41.3-7c42.1-8.2,83.5-19.3,124.2-33.1c57.6-19.6,112.2-45.7,163.9-77.3
|
||||
c38.1-23.2,73.8-50.1,107.6-79.4c37.9-32.8,72.6-68.6,102.5-109C2037.5,591.3,2039.4,587.8,2043.6,585.1z"/>
|
||||
<path class="st1" d="M777.2,1163.8c-0.9-7.9,1.2-15.4,2-23c4.4-45.1,6.3-90.3,6.6-135.6c0.1-8.6,0.2-8.7,8.8-8.7
|
||||
c22-0.1,44-0.1,65.9,0c8.3,0,8.4,0.1,8.5,7.9c0.2,37.7,2,75.2,6.5,112.6c0.5,3.8,0.3,6-3.9,7.5c-31.7,11.3-62.2,25.3-92.6,39.6
|
||||
C778.8,1164.3,778.4,1164.1,777.2,1163.8z"/>
|
||||
<path class="st1" d="M776.8,817.8c3.1-0.6,4.7,0.6,6.5,1.4c28.2,13.5,56.6,26.5,86.2,36.8c5.4,1.9,6.7,4.3,6,10.1
|
||||
c-4.5,36.7-6.3,73.6-6.4,110.6c0,8.1,0.7,8.3-8.1,8.3c-22.6-0.1-45.3,0-67.9-0.1c-7,0-7.1-0.2-7.2-7.3c-0.4-28.6-0.9-57.3-2.7-85.9
|
||||
c-1.3-21.3-2.9-42.5-5.5-63.7C777.2,824.9,777.1,821.6,776.8,817.8z"/>
|
||||
<path class="st1" d="M1896.4,1163.9c-3.3,0.5-5.7-1.5-8.4-2.8c-27.2-13.3-54.8-25.7-83.3-36.1c-7.4-2.7-7.2-2.9-6.3-10.6
|
||||
c4.4-35.8,5.3-71.8,6.4-107.7c0.1-1.7,0.1-3.3,0-5c-0.3-3.8,1.3-5.4,5.2-5.3c6.3,0.2,12.7,0.1,19,0.1c16.7,0,33.3,0,50,0.1
|
||||
c8.4,0,8.4,0.1,8.6,8.8c0.8,30.3,1.2,60.6,3.3,90.9c1.4,19.9,2.9,39.9,5.3,59.7C1896.5,1158.6,1897.4,1161.2,1896.4,1163.9z"/>
|
||||
<path class="st1" d="M1896.8,817.7c-1.3,13.9-2.7,27.7-3.9,41.6c-3.5,38.5-4.5,77-5.2,115.6c0,1.7-0.2,3.3-0.1,5
|
||||
c0.4,4-1.5,5.5-5.2,5.1c-1-0.1-2,0-3,0c-22.6,0-45.3-0.1-67.9,0.2c-5,0-6.8-1.2-6.8-6.6c-0.4-28.3-1.2-56.5-3.6-84.7
|
||||
c-0.8-8.9-1.6-17.9-2.7-26.8c-0.9-7.6-1.1-7.8,6.5-10.6c23-8.7,45.8-17.8,67.9-28.7c6-2.9,12-5.7,18-8.6
|
||||
C1892.5,818.3,1894.3,817.4,1896.8,817.7z"/>
|
||||
<path class="st1" d="M908.3,1109.6c-1-4.4,0.4-8.6,0.8-12.8c2.5-30.9,4-61.8,4-92.9c0-7.4,0.2-7.5,7.9-7.5
|
||||
c33.3-0.1,66.7-0.2,100-0.2c7.8,0,8.1,0.1,8,7.4c-0.3,22,1,44,2.3,65.9c0.3,4.5-1,6.5-5.5,7.4c-38.7,8.5-76.4,20.9-114.2,32.5
|
||||
C910.7,1109.8,909.7,1109.6,908.3,1109.6z"/>
|
||||
<path class="st1" d="M1765.3,1110.4c-9.6-3-18.1-5.6-26.6-8.4c-29.1-9.6-58.7-17.4-88.5-24.5c-8.9-2.1-8.1-1.4-7.6-10.4
|
||||
c1.2-21,1.6-42,2.3-62.9c0.2-7.7,0.2-7.9,7.5-7.9c33.3,0,66.7,0.1,100,0.2c7.8,0,8.2,0.3,8.2,7.7c0.2,34,1.7,68,4.8,101.8
|
||||
C1765.4,1106.9,1765.3,1107.9,1765.3,1110.4z"/>
|
||||
<path class="st1" d="M908.8,872.2c2.8-1.1,5,0.8,7.5,1.5c36.1,10.3,71.6,22.9,108.5,30.5c5.5,1.1,6.7,3.6,6.4,8.8
|
||||
c-1.3,21.9-2.6,43.9-2.1,65.9c0.1,4.7-1.2,6.5-6.1,6.4c-34.7-0.2-69.3-0.2-104-0.1c-4.1,0-6.1-1-6-5.6c0.4-35.6-2-71.2-4.8-106.7
|
||||
C908.2,872.8,908.6,872.5,908.8,872.2z"/>
|
||||
<path class="st1" d="M1765.6,872.4c-0.9,10.7-1.8,20.9-2.5,31.2c-1.7,24.6-2.4,49.2-2.6,73.8c-0.1,7.7-0.2,7.8-7.7,7.8
|
||||
c-33.6,0.1-67.2,0.1-100.8,0.1c-6.9,0-6.9-0.2-7.1-7.1c-0.8-21.9-1.3-43.9-2.6-65.8c-0.3-5.4,1.8-7,6.2-8
|
||||
c14.5-3.5,29-7.1,43.5-10.8c22.5-5.7,44.5-13.5,66.7-20.3C1760.5,872.7,1762.3,871.8,1765.6,872.4z"/>
|
||||
<path class="st1" d="M725.8,1192.5c-5.9-34.3-8.8-67.4-11-100.6c-1.8-27.6-2.2-55.2-3-82.8c-0.1-2.3,0.1-4.7-0.1-7
|
||||
c-0.4-4.4,1.6-5.8,5.9-5.7c9,0.2,18,0,27,0c6.7,0,13.3-0.1,20,0c7.2,0.1,7.4,0.2,7.3,7.1c-0.5,23.6-1,47.2-2,70.8
|
||||
c-1.2,27.2-3.3,54.4-6.7,81.5c-0.2,1.3-0.3,2.6-0.4,4c-0.4,9.2-3.4,16-13,19.7C741.7,1182.7,734.5,1187.7,725.8,1192.5z"/>
|
||||
<path class="st1" d="M1946.2,1191.7c-10.1-5.6-20.4-11.5-30.8-17.2c-3.3-1.8-3.1-5-3.5-7.8c-1.4-10.2-2.4-20.5-3.5-30.7
|
||||
c-4.7-43.8-6-87.7-6.7-131.6c-0.1-7.7,0.1-7.8,7.7-7.8c15.3-0.1,30.6,0.1,46-0.1c4.6-0.1,6.5,1.1,6.4,6.1
|
||||
c-0.5,34-1.2,67.9-3.7,101.8c-2,26.9-4.5,53.7-8.9,80.4C1948.7,1187,1949.1,1189.5,1946.2,1191.7z"/>
|
||||
<path class="st1" d="M1947.7,789.1c5.8,32.7,8.6,63.8,10.8,95c2.1,30.6,2.8,61.2,3.1,91.9c0.1,9.6,0.3,8.9-8.3,8.9
|
||||
c-15,0-30-0.1-45,0.2c-5.3,0.1-6.9-1.8-6.7-7c0.9-25.3,1.3-50.6,2.5-75.9c1.3-27.3,3.9-54.4,6.9-81.6c0.9-8.3,3.3-14.2,11.5-17.8
|
||||
C1930.8,799.2,1938.6,794.1,1947.7,789.1z"/>
|
||||
<path class="st1" d="M726,789.2c11.3,6.2,21.7,11.9,32.2,17.5c2.9,1.6,2.8,4.2,3.2,6.8c2.7,15.8,4.1,31.7,5.5,47.6
|
||||
c3.3,37.2,4.5,74.5,5,111.8c0,2.3,0,4.7,0.1,7c0.1,3.1-0.8,5-4.4,5c-17-0.1-34-0.1-51-0.1c-3.3,0-4.8-1.5-4.7-4.8
|
||||
c0.2-5.7,0-11.3,0.2-17c1-42.6,2.7-85.2,7.3-127.7C720.9,820.5,722.6,805.7,726,789.2z"/>
|
||||
<path class="st1" d="M1625.1,910.5c-0.6,9.4-1.4,18-1.7,26.5c-0.4,13-0.4,26-0.6,38.9c-0.1,9.3-0.2,9.4-9.2,9.4
|
||||
c-30.6,0.1-61.3,0.1-91.9,0.1c-8.3,0-16.7-0.1-25,0.1c-4.2,0.1-5.9-1.3-5.9-5.7c0-15-0.1-30-0.7-44.9c-0.2-4.8,2.5-5.5,6-6
|
||||
c10.9-1.3,21.8-3.6,32.7-3.7c23.9-0.2,46.7-6.5,70-10C1607.1,913.9,1615.2,912.2,1625.1,910.5z"/>
|
||||
<path class="st1" d="M1625.2,1071.1c-23.5-4.2-45.7-8.1-67.9-12.1c-7.9-1.4-15.8-2.4-23.8-2.5c-12.7-0.1-25.1-2.6-37.7-4
|
||||
c-4-0.4-5.8-1.6-5.6-6.1c0.5-14.6,0.6-29.3,0.6-43.9c0-4.5,1-6.5,6.1-6.5c39.9,0.2,79.9,0.2,119.8,0.1c5,0,6.1,2.1,6.1,6.6
|
||||
C1622.9,1025,1623,1047.3,1625.2,1071.1z"/>
|
||||
<path class="st1" d="M1049.3,1070.4c-1-5.9-0.1-11.8,0.3-17.7c0.9-16.3,1.4-32.6,1.4-48.9c0-7.3,0.2-7.5,6.8-7.5
|
||||
c39.3-0.1,78.5-0.1,117.8-0.1c7,0,7.1,0.2,7.2,7.2c0.2,14,0,27.9,0.5,41.9c0.2,5.2-1.5,6.6-6.6,7c-16.9,1.3-33.8,2.7-50.6,5.2
|
||||
c-23.3,3.4-46.7,6.8-69.7,12.3C1054.2,1070.3,1051.9,1071.4,1049.3,1070.4z"/>
|
||||
<path class="st1" d="M1049.8,910.2c22.1,5.7,44.8,8.7,67.4,12.5c16.4,2.8,33.1,3.3,49.6,5.2c16.4,1.9,16.4,1.5,16.3,17.4
|
||||
c-0.1,11-0.3,22-0.2,32.9c0.1,4.9-1,7.2-6.6,7.2c-39.9-0.2-79.9-0.2-119.8-0.1c-3.4,0-5.7-0.6-5.6-4.8c0.6-22.6-1.3-45.2-1.9-67.8
|
||||
C1049,912,1049.4,911.4,1049.8,910.2z"/>
|
||||
<path class="st1" d="M1260.5,985.4c-21.6,0-43.2,0-64.9,0c-8,0-8.2-0.1-8.1-8.4c0.2-13.6,0.6-27.3,0.8-40.9c0.1-4.2,1.4-5.6,6-5.1
|
||||
c41.7,4.1,83.5,6.2,125.5,6.2c2.3,0,4.7,0,7,0c7.5,0.2,7.5,0.2,7.6,7.9c0.1,10.3,0.1,20.6,0.1,30.9c0,9.3,0,9.4-9.1,9.4
|
||||
C1303.8,985.4,1282.2,985.4,1260.5,985.4z"/>
|
||||
<path class="st1" d="M1485,931.5c0.3,16.2,0.5,32.7,0.8,49.3c0.1,3.4-1.7,4.8-4.9,4.6c-2-0.1-4,0-6,0c-41.6,0-83.2,0-124.9,0
|
||||
c-2,0-4-0.1-6,0c-4,0.3-5.1-1.8-5-5.5c0.1-11.7,0-23.3,0.1-35c0.1-7.6,0.1-7.8,7.6-7.8c17.3,0.2,34.6-0.6,51.9-1.1
|
||||
c26.3-0.7,52.5-3.1,78.7-5.1C1479.6,930.9,1481.8,930,1485,931.5z"/>
|
||||
<path class="st1" d="M1485,1051.1c-35.4-3-69.9-5.3-104.5-6.4c-11.6-0.4-23.3-0.5-34.9-0.4c-4.4,0-6.7-0.8-6.5-6
|
||||
c0.4-12,0.3-24,0-36c-0.1-4.8,1.6-6.3,6.3-6.2c45,0.1,89.9,0.1,134.9,0c3.5,0,5.8,0.7,5.7,4.8
|
||||
C1485.5,1017.5,1485.3,1034.1,1485,1051.1z"/>
|
||||
<path class="st1" d="M1188.4,1050.6c-0.3-17.2-0.5-33.7-0.8-50.3c0-2.9,1.6-4.1,4.4-4.1c1.7,0,3.3,0,5,0c42.6,0,85.1,0,127.7,0
|
||||
c1.7,0,3.3,0.1,5,0c3.3-0.2,4.9,1.2,4.9,4.6c-0.1,13-0.1,25.9,0,38.9c0,3.6-1.7,4.7-4.9,4.5c-0.7,0-1.3,0-2,0
|
||||
C1281.6,1043.7,1235.9,1047.2,1188.4,1050.6z"/>
|
||||
<path class="st1" d="M2010.7,1230.1c-5.1-1.9-9-5.1-13.1-7.7c-9.9-6.1-19.6-12.6-29.8-18.3c-4.8-2.6-5.5-5.6-4.5-10.4
|
||||
c3.9-20,6.2-40.1,8.2-60.4c4-41.2,5.8-82.4,6.1-123.8c0-3,0.1-6,0-9c-0.1-3,1.1-3.8,4.1-4.1c16.5-1.5,16.7-1.4,16.8,14.7
|
||||
c0.4,66.4,3.9,132.5,10.7,198.6C2009.9,1216.3,2011.6,1222.7,2010.7,1230.1z"/>
|
||||
<path class="st1" d="M2011.2,752.5c-1.6,16.9-3.2,33.8-4.8,50.6c-3,30.8-4.6,61.7-6.1,92.6c-1.3,27.3-1.5,54.5-1.8,81.8
|
||||
c-0.1,7.1-0.2,7.3-7.1,7.3c-16.3,0-13.6,1.3-13.7-12.8c-0.5-54.3-3.2-108.4-10.6-162.2c-1.1-7.9-2.5-15.8-3.9-23.6
|
||||
c-0.7-3.6-0.1-6,3.5-8.2c13.4-8,26.6-16.4,39.8-24.7C2007.6,752.8,2008.5,751.7,2011.2,752.5z"/>
|
||||
<path class="st1" d="M662.1,1231.5c1.9-20.4,3.7-38.6,5.4-56.8c3-31.9,4.7-63.8,6-95.8c1-24.6,1.6-49.3,1.6-74
|
||||
c0-8.3,0.1-8.3,7.9-8.4c1.3,0,2.7,0,4,0c8.6,0.1,8.7,0.1,8.8,8.7c0.4,40,1.8,80,5.4,119.8c2,22.9,4.4,45.8,9,68.3
|
||||
c1.1,5.5,0.2,8.5-5.1,11c-12.7,6-23.9,14.5-35.5,22.2C667.7,1227.9,665.8,1229.1,662.1,1231.5z"/>
|
||||
<path class="st1" d="M662.9,752.1c2.6-0.9,3.9,1,5.4,2.1c12.4,8.4,24.6,17.1,38,23.9c3.3,1.7,5,3.6,4.2,7.7
|
||||
c-11.3,61.8-13.7,124.4-14.8,187c-0.2,13.5,2.2,12.1-12.3,12.1c-8.4,0-8.5-0.1-8.5-8.9c0.1-52.7-2.6-105.3-6.4-157.8
|
||||
c-1.5-20.6-4-41.1-6-61.7C662.5,755.1,661.9,753.4,662.9,752.1z"/>
|
||||
<path class="st1" d="M2044.3,728.6c0,3.4,0,6.3,0,9.2c0,78.9,0,157.9,0,236.8c0,1.7-0.1,3.3,0,5c0.4,3.9-1.3,5.4-5.2,5.3
|
||||
c-5.9-0.2-11.9,0-18.3,0c0-26.3,0.3-51.6,1.3-76.8c1.7-42.3,4.4-84.5,8.9-126.5c1.6-14.9,3.9-29.7,5.6-44.6
|
||||
C2037.2,732.4,2038.9,729.7,2044.3,728.6z"/>
|
||||
<path class="st1" d="M2044.3,1253c-5.6-1.2-7-4.1-7.7-8.6c-3.1-19.4-5.2-38.9-7-58.5c-5.6-59-8.7-118.2-8.7-177.6
|
||||
c0-13.6-1.4-11.7,11.5-11.8c13.3,0,11.9-1.5,11.9,11.5c0,78.6,0,157.2,0,235.8C2044.3,1246.6,2044.3,1249.3,2044.3,1253z"/>
|
||||
<path class="st1" d="M629.2,728.7c5.8,1.6,7.4,4.6,7.9,9c2.3,21.2,5.3,42.2,7.3,63.4c5.3,55.3,7.7,110.8,8.5,166.4
|
||||
c0,3.3,0.1,6.7,0,10c-0.1,7.5-0.1,7.5-7.7,7.7c-16,0.5-16,0.5-16-15.3c0-76.5,0-153.1,0-229.6C629.2,736.7,629.2,733,629.2,728.7z"
|
||||
/>
|
||||
<path class="st1" d="M629.2,1253.2c0-6,0-10.6,0-15.3c0-75.7,0-151.3,0-227c0-2,0-4,0-6c-0.1-9.2-0.3-8.7,8.7-8.6
|
||||
c15.1,0,15.2,0,15,15.6c-0.7,62.7-3.8,125.2-10.3,187.6c-1.5,14.9-3.9,29.7-5.5,44.6C636.6,1248.7,634.5,1251.1,629.2,1253.2z"/>
|
||||
<path class="st1" d="M2060,697.5c0-15.9,0-29.5,0-43c0-3.7,0.3-7.3-0.1-11c-2-19,6.4-34.8,15.3-50.5c1-1.7,2.3-3.2,3.6-4.7
|
||||
c5.4-6.2,8.1-6.2,13.4,0.4c6.3,7.8,10.3,17.1,13.7,26.4c3.2,8.7,5.9,17.6,8.7,26.5c1.3,4,1,7.1-2.4,10.7
|
||||
C2096.7,668.2,2079,681.6,2060,697.5z"/>
|
||||
<path class="st1" d="M613.2,1283.1c0,24.5,1.5,47.6-0.5,70.3c-1.4,15.4-9,29.8-19.7,41.7c-3.8,4.2-5.7,3.9-10-0.4
|
||||
c-4.3-4.3-7.2-9.6-9.8-15c-6.7-13.8-11.2-28.4-15.3-43.2c-0.8-3-1.3-5.6,1.5-8.1C576.3,1313,592.4,1296.7,613.2,1283.1z"/>
|
||||
<path class="st1" d="M2060,1284.4c19.3,15.5,36.8,29.3,52.5,45.2c2.6,2.6,3.7,5.1,2.6,8.8c-5.1,17.2-10,34.5-19.7,50
|
||||
c-7.7,12.2-12.4,12.7-19.8,0.6c-9.5-15.4-16.9-31.4-15.8-50.5C2060.7,1321.2,2060,1303.9,2060,1284.4z"/>
|
||||
<path class="st1" d="M611.9,698.2c-8.5-6.7-17.1-12.8-25-19.7c-9.1-7.8-17.5-16.4-26.3-24.5c-2.3-2.2-4-4.1-3-7.7
|
||||
c5.3-18.2,10.3-36.5,20.4-52.8c7.8-12.7,12.8-13,20.4-0.3c8.9,14.9,16.3,30.2,15.1,48.6c-1,15.9-0.2,31.9-0.3,47.9
|
||||
C613.2,692.2,614,694.9,611.9,698.2z"/>
|
||||
<path class="st1" d="M1788.2,864c4.1,27,4.9,53.5,6.1,80.1c0.5,11.3,0.2,22.6,0.7,33.9c0.2,5.2-1.6,7.4-6.7,7.3
|
||||
c-4.7-0.1-9.3-0.3-14,0c-4.6,0.3-5.5-2-5.4-6c0.2-26.3,1.3-52.5,3.2-78.8c0.5-7.6,1.2-15.2,2.1-22.8
|
||||
C1775.6,867.2,1775.7,867.2,1788.2,864z"/>
|
||||
<path class="st1" d="M885.3,1118.4c-5.1-39.1-6.6-77.6-6.6-116.3c0-4.2,1.3-5.9,5.7-5.8c25,0.7,20.1-4.1,19.8,19.4
|
||||
c-0.4,29.6-1.5,59.3-5,88.8C897.9,1115.4,898.1,1115.4,885.3,1118.4z"/>
|
||||
<path class="st1" d="M885.7,863.5c12.4,2.1,12.2,2.1,13.5,13.8c3.8,33,4.6,66.2,5.3,99.4c0.2,8.3,0,8.5-7.9,8.4
|
||||
c-5.6-0.1-12.6,2.4-16.3-1.2c-3.8-3.6-1.3-10.7-1.3-16.2c0.1-33.9,1.8-67.7,6.1-101.4C885.1,865.4,885.4,864.5,885.7,863.5z"/>
|
||||
<path class="st1" d="M1788.4,1117.5c-4,0.3-6.2-1.5-8.7-2.2c-3.2-0.9-4.6-2.9-4.9-6.5c-0.7-10.3-2.2-20.5-2.9-30.8
|
||||
c-1.5-24.6-2.8-49.2-2.8-73.9c0-7.5,0.1-7.8,7.7-7.6c5.9,0.2,13.3-2.7,17.2,1.6c3.4,3.7,1,10.9,0.9,16.5
|
||||
C1794.3,1048.8,1792.8,1083.1,1788.4,1117.5z"/>
|
||||
<path class="st1" d="M1481.1,1078.1c-10.7,0.3-20.5-1.5-30.3-2.6c-34.1-3.7-68.2-5.9-102.5-5.6c-1.3,0-2.7-0.1-4,0
|
||||
c-4.9,0.5-5.1-2.5-5.2-6.2c-0.1-4,0.7-6,5.4-6c20,0.1,39.9-0.2,59.9,1c22.9,1.4,45.8,3,68.7,4.7c7.9,0.6,7.8,0.9,8.2,8.7
|
||||
C1481.2,1073.7,1481.1,1075.4,1481.1,1078.1z"/>
|
||||
<path class="st1" d="M1480,903c2,3.3,0.7,6.3,1,9.2c0.4,4.3-1.6,5.6-5.8,5.9c-42.2,3.3-84.3,6.2-126.7,5.9c-1.3,0-2.7-0.1-4,0
|
||||
c-4.5,0.5-5.5-1.7-5.6-5.8c-0.1-4.5,0.9-6.6,6-6.5c29.7,0.7,59.2-1.5,88.8-3.9C1449.3,906.5,1464.7,904.6,1480,903z"/>
|
||||
<path class="st1" d="M1192.2,1078.2c1-14.7,0.9-15,13.5-15.8c40.5-2.9,80.9-5.4,121.6-4.6c7.3,0.1,7.4,0.5,7.3,7
|
||||
c-0.1,4-1.7,5.2-5.4,5.1c-6-0.2-12-0.1-18,0c-37.3,0.2-74.3,3.2-111.3,7.8C1197.6,1078,1195.3,1078,1192.2,1078.2z"/>
|
||||
<path class="st1" d="M1192.6,903.2c17.2,2.2,33.7,3.8,50.2,5c28.2,2.1,56.4,4.4,84.7,3.6c3.8-0.1,7.3-0.4,7.1,5.4
|
||||
c-0.3,6.3-0.2,6.8-6.6,6.8c-18.6,0-37.2,0.4-55.9-0.8c-19.2-1.2-38.5-2.1-57.8-3.2c-3-0.2-6-0.4-8.9-0.8
|
||||
C1192.8,917.6,1192.8,917.6,1192.6,903.2z"/>
|
||||
<path class="st1" d="M653.6,694.6c2.5,2.7,3.4,3.5,4.2,4.6c15.1,20.9,32.8,39.4,51.8,56.7c2.5,2.3,7.7,4.8,5,8.5
|
||||
c-2.7,3.9-6.2-1-8.9-2.4c-14.7-7.8-28.1-17.9-42.2-26.6c-3.1-1.9-4.3-4.4-4.8-8C657.4,717,655.5,706.9,653.6,694.6z"/>
|
||||
<path class="st1" d="M654,1284.6c0.9-5.5,2.7-11,2.6-16.4c-0.1-14.4,6.3-23.8,19.2-30.2c10.6-5.3,20.3-12.5,30.5-18.8
|
||||
c2.5-1.5,4.9-3.3,7.8-3.6c1.6,2.1,1.5,4,0.2,5.7c-1,1.3-2.4,2.3-3.6,3.4c-18.4,17.3-36.5,34.8-51.2,55.5c-1.1,1.6-2.6,2.9-4,4.3
|
||||
C655.3,1284.8,654.8,1284.6,654,1284.6z"/>
|
||||
<path class="st1" d="M2019.1,698.8c-1.4,9.8-3,19.6-4.2,29.5c-0.4,3-2.2,4.7-4.4,6.2c-15.3,9.9-30.7,19.7-46.1,29.5
|
||||
c-1.5,1-3.7,4-5.7,0.4c-1.5-2.6,0.2-4.4,2.1-5.9c11.9-9.2,21.5-20.7,32.4-30.8c8.4-7.7,14.5-17.2,21.4-26.1c0.9-1.2,2-2.4,3-3.6
|
||||
C2018.1,698.2,2018.6,698.5,2019.1,698.8z"/>
|
||||
<path class="st1" d="M2017.1,1283.9c-11.8-19.6-28.7-34.6-44.6-50.5c-3.7-3.7-7.8-7.2-11.8-10.6c-1.9-1.7-3.4-3.4-1.7-5.9
|
||||
c1.7-2.4,3.5-0.5,5,0.5c15.7,10,31.3,20,46.9,30c2,1.3,3.5,2.9,3.9,5.4c1.4,10.1,2.9,20.2,4.4,30.4
|
||||
C2018.5,1283.4,2017.8,1283.7,2017.1,1283.9z"/>
|
||||
<path class="st1" d="M1044.6,1034c-0.6,11-1.1,21.9-1.7,32.9c-0.1,1.6-0.3,3.3-0.8,4.9c-0.3,0.8-1.4,1.6-2.2,1.7
|
||||
c-0.9,0.1-1.9-0.5-2.1-1.7c-0.1-1-0.3-2-0.4-3c-1.2-22.2-2.3-44.5-2.4-66.8c0-3.2,0-6,4.6-5.9c3.5,0.1,5.6,0.7,5.5,4.9
|
||||
c-0.2,11-0.1,21.9-0.1,32.9C1044.9,1034,1044.8,1034,1044.6,1034z"/>
|
||||
<path class="st1" d="M1636.4,1033.8c0,10.6,0,21.2,0,31.8c0,2-0.3,4-0.7,5.9c-0.2,0.8-1.2,1.8-1.9,1.8c-0.8,0-1.9-0.8-2.2-1.5
|
||||
c-0.5-1.2-0.5-2.6-0.6-3.9c-1.9-21.8-2.2-43.7-2.1-65.6c0-5.7,1.3-6.7,6-6.1c3.9,0.5,3.6,3.1,3.6,5.8c0,10.6,0,21.2,0,31.8
|
||||
C1637.8,1033.8,1637.1,1033.8,1636.4,1033.8z"/>
|
||||
<path class="st1" d="M1638.5,948.1c0,10.3,0,20.6,0,31c0,2.6,0.5,5.4-3.3,6.2c-4.4,0.9-6.4-0.5-6.4-5.8c0-19,0-38,1.4-56.9
|
||||
c0.2-3,0.4-6,0.7-9c0.2-2.2-0.3-5.4,3-5.2c2.5,0.1,2.2,3,2.3,4.9c0.6,11.6,1.1,23.3,1.6,34.9C1638,948.1,1638.2,948.1,1638.5,948.1
|
||||
z"/>
|
||||
<path class="st1" d="M1045.1,948c0,10.6-0.1,21.2,0,31.9c0,3.6-0.9,5.5-5,5.5c-3.9-0.1-5.1-1.7-5.1-5.4c0.1-22.6,1.2-45.1,2.5-67.7
|
||||
c0.1-1.6-0.1-3.7,2-4c2.8-0.4,3.1,1.9,3.2,3.9c0.7,11.9,1.3,23.9,1.9,35.8C1044.8,948,1044.9,948,1045.1,948z"/>
|
||||
<path class="st1" d="M741.9,740.1c7,10.8,9.7,22,12.6,34.5c-8.1-3.6-13.5-8.9-19.7-13c-2-1.3-2-3.3-1.2-5.3
|
||||
C735.6,751.1,737,745.6,741.9,740.1z"/>
|
||||
<path class="st1" d="M1931.7,1241.3c-6.9-10.6-9.7-21.7-12.1-33.3c4.2,0.4,6.4,3.1,9.1,4.9
|
||||
C1943.2,1223.1,1943.4,1224.7,1931.7,1241.3z"/>
|
||||
<path class="st1" d="M1931.6,740c12,17.2,11.9,18.3-3.6,29.1c-2.4,1.7-4.5,3.9-8.5,4.8C1921.8,761.9,1925,750.9,1931.6,740z"/>
|
||||
<path class="st1" d="M742.1,1241.3c-5.3-5.2-6.5-11.1-8.6-16.5c-0.8-2-0.5-3.8,1.6-5.1c6-3.9,11.2-9.1,19.2-12.6
|
||||
C751.8,1219.5,748.7,1230.3,742.1,1241.3z"/>
|
||||
<path class="st1" d="M599.6,517.5c0.6,0.1,1.4,0.1,1.9,0.4c12.6,7.1,13,7.9,10.7,24.4c-5.9-7.9-11-15.2-13.4-24.1
|
||||
C599,518,599.3,517.7,599.6,517.5z"/>
|
||||
<path class="st1" d="M597.6,1464.4c4-9.4,8.1-17.2,14-24.4C614.9,1455.5,614.5,1457.9,597.6,1464.4z"/>
|
||||
<path class="st1" d="M2072.4,516.4c2.2,0.6,2.3,2,1.3,3.7c-3.8,6.1-7.7,12.1-11.9,18.6c-2.8-10.6-2.1-12.7,5.3-17.8
|
||||
c1.9-1.3,4.2-2,5.3-4.3L2072.4,516.4z"/>
|
||||
<path class="st1" d="M2061.5,1442.5c4.7,8.5,10.5,14.5,12.5,22.8C2067.2,1459.8,2056.4,1456.8,2061.5,1442.5z"/>
|
||||
<path class="st1" d="M1781.6,836.9c4.2,9.3,4.2,9.3-1.8,12.6C1777.9,845.4,1780.6,842.1,1781.6,836.9z"/>
|
||||
<path class="st1" d="M895.4,849.3c-7.5-2.9-7.5-2.9-3.8-11.1C893.8,841.3,893.9,844.7,895.4,849.3z"/>
|
||||
<path class="st1" d="M630.1,1284.6c-0.2-5.4-2.7-11,1.8-17C634.2,1274.4,629.7,1279.3,630.1,1284.6z"/>
|
||||
<path class="st1" d="M629.9,697.1c0.5,5.4,3.9,10.3,2.4,16.6C626.9,708.5,630.1,702.5,629.9,697.1z"/>
|
||||
<path class="st1" d="M2044.5,1284.5c-2.2-5-4.3-9.2-2.8-15C2046.1,1274.3,2043.5,1279.4,2044.5,1284.5z"/>
|
||||
<path class="st1" d="M2041.9,712.1c-2.2-6,1.1-10.1,2.2-14.6C2043.5,702.2,2046,707.4,2041.9,712.1z"/>
|
||||
<path class="st1" d="M892.2,1143.9c-4.6-7.9-4.6-7.9,1.6-11.5C895.7,1136.5,892.6,1139.7,892.2,1143.9z"/>
|
||||
<path class="st1" d="M1779.6,1132.2c6.1,3.1,6.1,3.1,2.7,11.1c-1.9-1-1.7-3.1-2.3-4.6C1779.2,1137,1778.6,1135.1,1779.6,1132.2z"/>
|
||||
<path class="st1" d="M1934.8,1202.5c4.1-0.6,6.5,2.5,11.4,4.9c-6.3,0.5-8.2-3.5-11.3-5.1L1934.8,1202.5z"/>
|
||||
<path class="st1" d="M1935.8,777c3.1,0.9,4.5-3.5,7.8-2.9c-1.8,4-4.7,3.9-7.9,2.8L1935.8,777z"/>
|
||||
<path class="st1" d="M729.5,774c2.7-0.5,4.4,0.6,5.8,3.2C732.1,778,731,775.7,729.5,774z"/>
|
||||
<path class="st1" d="M734,1202.5c2.3,4.6-1.2,4.8-4.2,5.9c-0.6-3.7,3.7-3.3,4.2-5.7C734,1202.7,734,1202.5,734,1202.5z"/>
|
||||
<path class="st1" d="M1935.2,780.3c-0.4,0.2-0.8,0.5-1.2,0.7c0.3,0,0.6,0.1,0.8,0c0.2-0.1,0.4-0.3,0.6-0.5
|
||||
C1935.5,780.5,1935.2,780.3,1935.2,780.3z"/>
|
||||
<path class="st1" d="M740,1203.1c-1.9-0.8-4.2,2.9-6-0.5c0,0,0,0.2,0,0.1C736,1202.8,738,1202.9,740,1203.1L740,1203.1z"/>
|
||||
<path class="st1" d="M1935.7,777c1.4,1.3,0.9,2.4-0.5,3.3c0,0,0.3,0.3,0.3,0.3c0.1-1.2,0.3-2.3,0.4-3.5
|
||||
C1935.8,777,1935.7,777,1935.7,777z"/>
|
||||
<path class="st1" d="M737.9,778.4c0.2,0.2,0.4,0.5,0.6,0.7c-0.2,0.2-0.5,0.5-0.7,0.5c-0.2,0-0.4-0.4-0.6-0.6
|
||||
C737.4,778.8,737.7,778.6,737.9,778.4z"/>
|
||||
<path class="st1" d="M740,1203.1c0-0.3,0-0.8-0.2-1c-0.8-0.8-0.7-1.1,0.4-0.8C739.5,1201.9,739.4,1202.4,740,1203.1
|
||||
C740,1203.1,740,1203.1,740,1203.1z"/>
|
||||
<path class="st1" d="M1934.9,1202.3c0-0.2-0.1-0.3-0.1-0.5c0,0-0.1,0.1-0.1,0.1c0.1,0.2,0.1,0.4,0.2,0.6
|
||||
C1934.8,1202.5,1934.9,1202.3,1934.9,1202.3z"/>
|
||||
<path class="st1" d="M2072.4,516.6c0.7,0.4,1.4,0.8,2.1,1.2c-0.2,0.2-0.4,0.7-0.6,0.7c-1.1-0.2-1.5-1-1.4-2
|
||||
C2072.4,516.4,2072.4,516.6,2072.4,516.6z"/>
|
||||
<path class="st1" d="M598.7,518.3c-1-0.2-1.8-0.8-1-1.6c0.8-0.9,1.4,0,1.8,0.8C599.3,517.7,599,518,598.7,518.3z"/>
|
||||
</g>
|
||||
</svg>
|
||||
<p class="hero-tag">distributed structured concurrency</p>
|
||||
</div>
|
||||
<p class="hero-sub">a multi-processing runtime built on (and shaped
|
||||
entirely like) <a class="reference external"
|
||||
href="https://github.com/python-trio/trio">trio</a>.</p>
|
||||
|
|
@ -1,456 +0,0 @@
|
|||
<svg class="hero-logo" role="img" aria-label="tractor" version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
viewBox="0 0 2670 1980" style="enable-background:new 0 0 2670 1980;" xml:space="preserve">
|
||||
<style type="text/css">
|
||||
.st0{fill:currentColor;}
|
||||
.st1{fill:none;}
|
||||
</style>
|
||||
<g>
|
||||
<path class="st0" d="M2275.3,1000.1c-0.1,2.3,0,4.7,0,7c0,189.3,0,378.7,0,568c0,6,0.1,12.1-0.8,17.9c-2.4,15.2-12.9,17.4-23.6,10
|
||||
c-45.4-31-91.2-61.4-136.9-91.9l-30.1-20.3c-9.8-6.3-19.2-13.2-29.2-19.3c-6.7-4.1-9.8-9.7-9.9-17.2c-0.1-9.3-0.6-18.7-0.3-28
|
||||
c0.2-7.3-1.7-13-6.5-19c-18.6-23.3-36.2-47.3-56.9-69c-18-18.8-36.7-36.8-56.4-53.8c-31.8-27.5-65.4-52.7-101-74.9
|
||||
c-32.7-20.4-66.8-38.4-102.2-54c-50.7-22.4-103.2-39.2-157.1-51.5c-36.7-8.4-73.8-14.4-111.2-18.5c-34.5-3.8-69-6.5-103.7-5.8
|
||||
c-23.1,0.5-46.2-1.3-69.3,0.4c-22.9,1.8-45.9,3-68.7,5.6c-72.7,8.4-143.7,24.6-212.1,50.6c-65.5,24.8-127.4,57-184.9,97.4
|
||||
c-46.8,32.8-90.1,69.8-129.5,111.1c-18.1,19-35.1,39.2-50.5,60.6c-3.9,5.5-5.5,11-5.3,17.6c0.4,9.3,0,18.7,0,28
|
||||
c0,9.3-3.1,16.4-11.8,21.4c-10.4,5.9-19.9,13.4-30.1,19.8l-25.4,17.2c-11.6,7.8-23.3,15.5-34.9,23.3c-34.3,23-68.6,46-102.9,69
|
||||
c-14.6,9.7-23.6,5.5-25.2-12.1c-0.5-5.3-0.5-10.7-0.5-16c0-189.7,0-379.3,0-569l-0.1-26.1c0.2-190.3,0.1-380.7,0.2-571
|
||||
c0-5.7,0.4-11.3,0.5-17c0.1-3.7,1.2-7.3,3-10.5c3.4-5.9,9.3-7.7,15.4-4.8c2.7,1.3,5.1,3.1,7.6,4.8c44.3,29.6,88.5,59.3,132.8,88.9
|
||||
l29.7,20.1c10.8,6.6,20.9,14.3,31.7,20.9c6.7,4.1,9.7,9.8,9.8,17.3c0.2,11,0.4,22,0.2,33c-0.1,5.2,1.2,9.6,4.2,13.9
|
||||
c15.4,22.2,33,42.8,51.6,62.2c53.9,56.4,114.1,105.1,181.4,144.6c38.4,22.6,78.3,42.4,119.9,58.6c40,15.6,80.9,28.3,122.8,38
|
||||
c52.7,12.1,106,19.3,160,22.6c18.3,1.1,36.6,2.1,54.9,1.5c17.7-0.6,35.5,1.1,53.2-0.1c17.6-1.1,35.2-2.3,52.8-3.6
|
||||
c35.9-2.5,71.4-8.5,106.7-15.1c53.4-9.9,105.5-25,156-45.1c53.8-21.5,105.1-48.2,153.2-80.9c43.4-29.5,83.6-62.7,121.1-99.3
|
||||
c27.3-26.6,50.3-56.8,73.9-86.6c2.9-3.6,3.6-7.6,3.6-12.1c-0.1-10.7,0.2-21.3,0.4-32c0.1-7.5,3.2-13.1,9.9-17.2
|
||||
c10.2-6.2,19.9-13.4,30-19.8l23.2-16.3c4.2-1.3,7.4-4.3,11-6.7c42.9-28.7,85.8-57.4,128.7-86.1c1.1-0.7,2.3-1.4,3.3-2.3
|
||||
c10.9-9.1,21.4-4.7,23.8,10.7c0.8,5.6,0.6,11.3,0.6,17c0,189.3,0,378.7,0,568L2275.3,1000.1z M740,1203.1c-2-0.2-4-0.3-6-0.5
|
||||
c-0.5,2.5-4.8,2-4.2,5.8c2.9-1,6.4-1.2,4.2-5.8C735.8,1206,738.1,1202.3,740,1203.1c-0.6-0.7-0.5-1.3,0.2-1.7
|
||||
c-1.1-0.3-1.1-0.1-0.4,0.8C740,1202.3,739.9,1202.8,740,1203.1z M1935.5,780.5c-0.2,0.2-0.4,0.4-0.6,0.4c-0.3,0.1-0.6,0-0.8,0
|
||||
c0.4-0.2,0.8-0.5,1.1-0.8c1.4-0.9,2-1.9,0.5-3.3c3.2,1.1,6.1,1.2,7.9-2.8c-3.3-0.7-4.6,3.8-7.7,2.9
|
||||
C1935.7,778.2,1935.6,779.4,1935.5,780.5z M1934.8,1202.5c-0.1-0.2-0.1-0.4-0.2-0.6c0,0,0.1-0.1,0.1-0.1c0,0.2,0.1,0.3,0.1,0.5
|
||||
c3.1,1.6,5,5.7,11.3,5.1C1941.3,1205,1938.9,1202,1934.8,1202.5z M2072.4,516.6c-1,2.3-3.4,3-5.3,4.3c-7.4,5.1-8.2,7.2-5.3,17.8
|
||||
c4.2-6.5,8-12.6,11.9-18.6c1.1-1.7,0.9-3.1-1.3-3.6c0,1,0.3,1.8,1.4,2c0.2,0,0.4-0.4,0.6-0.7C2073.8,517.4,2073.1,517,2072.4,516.6
|
||||
z M599.6,517.5c-0.4-0.8-1.1-1.7-1.8-0.8c-0.8,0.8,0.1,1.5,1,1.6c2.4,8.9,7.5,16.2,13.4,24.1c2.3-16.5,1.9-17.3-10.7-24.4
|
||||
C600.9,517.6,600.2,517.6,599.6,517.5z M534.9,1353.5c-20,21.8-37.3,44.1-53.4,67.4c-16.1,23.4-26.8,49.5-41.8,73.5
|
||||
c9-21,17.4-42.2,28.9-61.9c16.9-28.7,35.4-56.4,57.5-81.5c0.9-1,1.6-2.1,2.6-3.1c2.9-2.9,3.3-6,2.3-10
|
||||
c-9.1-33.8-14.6-68.3-19.2-103c-5-38-8.3-76.1-10.6-114.3c-2.5-39.9-3.4-79.9-3.9-119.8c-0.1-5.1-1.4-6.8-6.7-6.9
|
||||
c-21.6-0.2-43.3-0.8-64.9-1.3c-2,0-4-0.4-6,0.8c0,195.2,0,390.5,0,585.6c2.1,0.6,3.1-0.4,4.1-1.1c52.7-33.8,105.3-67.7,158-101.4
|
||||
c3.6-2.3,5.7-5.4,7.3-9.1c6-13,12.5-25.8,20.8-37.4c4.2-5.9,3.9-11.9,2.5-19.1c-2.4,1.8-4,2.9-5.5,4.2
|
||||
c-13.9,12.6-32.1,8.7-42.7-2.9c-7.3-8-12.4-17.2-17.1-26.8C542.3,1375.7,539.4,1365.1,534.9,1353.5z M534.6,627.2
|
||||
c3.6-5,4.3-10.7,6.3-15.9c6.2-15.8,12.6-31.5,25.2-43.7c10.8-10.5,26.8-12.6,38.9-2.8c2.3,1.9,4.3,5,7.9,5
|
||||
c1.6-8.3-0.2-15.3-5.1-21.9c-7.4-9.9-12.7-21-17.8-32.3c-2.4-5.3-5.8-9.2-10.8-12.4c-50.5-32.2-100.9-64.7-151.3-97.1
|
||||
c-2.4-1.5-4.5-3.9-8.7-3.9c0,3.7,0,7.2,0,10.7c0,187.9,0,375.8,0,563.7c0,2,0.2,4,0,6c-0.6,4.9,1.4,6.3,6.2,6.1
|
||||
c13.6-0.6,27.3-0.7,41-1c8.3-0.2,16.7-0.6,25-0.4c4.9,0.1,6.7-1.8,6.1-6.4c-0.1-1,0-2,0-3c1.2-75,4.3-149.8,13.7-224.3
|
||||
c4.6-36.3,10-72.5,19.6-107.9c1.6-5.7,0.4-9.9-3.4-14.4c-10.7-12.8-21.7-25.4-31-39.1c-19-28.1-36.8-57-49.6-88.7
|
||||
c-2.2-5.4-5.5-10.4-6.2-16.6C465,538.3,494.6,586,534.6,627.2z M2060.9,1411.1c-2.3,8.5-0.1,15.1,4,21.4
|
||||
c5.9,9.3,13.4,18.1,17.1,28.2c3.9,10.7,10.9,16.6,19.9,22.4c47.8,30.3,95.3,61.1,143,91.7c2.7,1.7,5,4.3,9.5,4.5c0-3.4,0-6.6,0-9.8
|
||||
c0-188.3,0-376.6,0-565c0-2-0.1-4,0-6c0.2-3.4-1.1-5-4.6-4.8c-2.3,0.1-4.7,0-7,0.1c-20,0.6-40,1.2-60,1.9c-6.7,0.2-6.7,0.4-7,7.4
|
||||
c0,1.3,0,2.7-0.1,4c-0.4,60-3,119.9-8.9,179.7c-5,50.1-11.3,99.9-24,148.7c-1.4,5.2,0.1,8.3,3.5,12.3c15,17.8,29.6,36.1,42.4,55.5
|
||||
c20,30.4,36.1,62.9,49.3,96.8c3.2,8.2,5.7,16.6,6.8,25.4c-33.9-90.8-82.6-145-107.2-169.7c-4.1,11-7.5,22.1-12.7,32.5
|
||||
c-4.9,9.9-10.3,19.5-18.8,26.8c-7.1,6.1-15.1,8.9-24.8,7.5C2073.2,1421.4,2068.3,1415.5,2060.9,1411.1z M2244.8,454.3
|
||||
c0.6,4.5-0.2,6.7-0.8,8.9c-6.5,24.2-17,47-28.2,69.2c-18.9,37.2-43,71.2-70.4,102.8c-2.7,3.2-3.6,5.8-2.5,9.8
|
||||
c7.9,29.6,13.1,59.8,17.3,90.2c6.4,45.6,10.2,91.3,12.8,137.3c2,35.3,2.6,70.6,3.1,105.9c0.1,7.6,0.2,7.6,7.6,7.8
|
||||
c17,0.5,34,1,51,1.4c5,0.1,10,0.2,15,0.5c3.2,0.1,5.1-0.9,4.9-4.5c-0.1-1.7,0-3.3,0-5c0-189.6,0-379.3-0.1-568.9
|
||||
c0-2.4,1.1-5.2-1.2-7.3c-0.8,0.1-1.6,0-2.1,0.4c-52.7,33.8-105.3,67.7-158,101.5c-2,1.3-3.7,2.6-4.7,4.9c-6.9,16-16.9,30.1-25.8,45
|
||||
c-3.1,5.2-3.4,10.2-2,15.7c2.6,0.1,3.5-1.6,4.6-2.6c15.7-13.9,33-10.6,45.6,3.8c6,6.8,10.4,14.6,14.4,22.7
|
||||
c5.1,10.3,8.3,21.3,12.8,32.9C2185.2,576.1,2221.7,520.5,2244.8,454.3z M553.7,662.8c-0.3,1.3-0.7,2.8-1,4.3
|
||||
c-7.1,31.2-11.7,62.8-15.4,94.5c-7.1,59.9-10.3,120.1-11.8,180.4c-0.3,13-0.2,26-0.5,39c-0.1,4.4,1.5,5.7,5.7,5.6
|
||||
c25-0.5,50-0.7,74.9-1.1c7.4-0.1,7.6-0.2,7.6-7.8c0-85.6,0-171.2,0.1-256.8c0-5.3-1.7-8.5-5.8-11.7c-16.6-12.8-32.4-26.6-46.9-41.9
|
||||
C559,665.6,557.6,663,553.7,662.8z M553.6,1319.7c2.1-1,2.8-1.2,3.2-1.6c16.4-17.1,34-32.9,52.7-47.4c3-2.4,3.8-5,3.8-8.6
|
||||
c-0.1-13.3-0.1-26.7-0.1-40c0-72.3,0-144.7,0-217c0-10.3,0.4-9-9.2-9.2c-23.3-0.4-46.7-0.8-70-1.1c-2.8,0-5.9-1-9.3,1.5
|
||||
C526.2,1104.4,531.1,1212.3,553.6,1319.7z M2119.8,1319.2c4.8-20.9,8.3-40.2,11.1-59.6c5.8-39.5,9.8-79.2,12.5-119.1
|
||||
c3.1-45.5,4.7-91.1,5.1-136.7c0.1-7.6-0.1-7.7-7.6-7.7c-24.3,0.1-48.6,0.3-72.9,0.5c-7.9,0.1-8,0.1-8,8.5c0,84.6,0,169.1-0.1,253.7
|
||||
c0,5.6,1.7,9.3,5.9,12.7c12.1,9.9,24.2,19.9,36,30.1C2107.7,1306.9,2113.1,1312.7,2119.8,1319.2z M2120.4,664.4
|
||||
c-2.1-0.6-2.9-0.1-3.6,0.7c-15.2,16.5-33.1,29.9-50.2,44.2c-4.8,4-6.8,8-6.8,14.3c0.3,32,0.2,64,0.2,96c0,52.3,0,104.6,0,157
|
||||
c0,8.1,0.1,8.2,8.2,8.2c18.3,0.2,36.7,0.2,55,0.4c6.7,0.1,13.3,0.1,20,0.3c3.7,0.1,5.7-1.1,5.4-5.1c-0.2-2.3,0-4.7-0.1-7
|
||||
c-0.5-45-2.1-89.9-5.2-134.8C2139.2,780.1,2132.7,721.9,2120.4,664.4z M878.7,1138.5c-36,15-69.8,32.9-102.3,53.1
|
||||
c-3.4,2.1-4.9,4.7-5.7,8.6c-2.8,14.4-5.7,28.8-12.2,42c-3.6,7.3-7.7,14.4-17.3,14.4c-9.6,0-13.4-7.4-17.4-14.4
|
||||
c-1.9-3.2-2-7.6-5.4-9.8c-22.3,17.4-41.6,37.2-58.8,58.9c-5.8,7.3-10.2,15.3-12,24.8c-3.8,20.3-9.1,40.2-16.7,59.5
|
||||
c-2.6,6.7-1.3,13.3-1.5,20.3c1.5-0.4,2-0.3,2.1-0.5c3.2-3.8,6.4-7.6,9.4-11.6c31.2-40.5,66-77.4,105.4-110.2
|
||||
c36.9-30.8,75.6-59.1,117.2-83.1c54.3-31.2,110.9-57.4,170.8-76.4c38.4-12.2,77.4-21.8,117-29.2c9.5-1.8,19.1-3,28.6-4.6
|
||||
c1.5-0.3,3.8,0.3,4.4-1.5c1.4-3.9,0.6-8-0.4-11.7c-1-3.6-4.5-1.7-6.9-1.5c-40.6,3.2-80.4,11-120.4,18c-7,1.2-10.4,4.1-10.9,11.2
|
||||
c-0.3,3.8-0.5,9.1-5.8,9c-4.6-0.1-4.7-5-5.3-8.5c-0.7-4.5-2.4-6-7.3-4.9c-39.3,9.1-78,19.9-115.5,34.7c-4.6,1.8-7.1,4.2-7.8,8.9
|
||||
c-0.9,5.9-2.2,11.8-5.2,17.1c-4.1,7.1-10.6,7.2-14.6,0C881.9,1147.7,880.7,1143.4,878.7,1138.5z M1032.8,891.9
|
||||
c0.9-3.8,1.6-6.7,2.4-9.5c0.7-2.3,1.7-4.6,4.4-4.9c3.4-0.3,4.4,2.3,5.3,4.9c0.4,1.3,0.8,2.6,0.8,3.9c0.1,7.7,4.5,10.4,11.7,11.6
|
||||
c32.5,5.5,64.8,11.9,97.5,15.7c7.9,0.9,15.9,1.7,23.8,2.4c1.9,0.2,4,0.7,5.3-1.6c3.5-6.5,0.3-13-7-14.2c-1-0.2-2-0.2-3-0.4
|
||||
c-42.6-5.8-84.2-15.9-125.4-28.2c-41.8-12.4-82.5-28.2-121.6-47.1c-51-24.6-99.8-53.3-144.7-87.9c-20.6-15.8-40.9-32.1-59.9-49.7
|
||||
c-29.1-26.9-56.4-55.7-80.1-87.7c-3.7-4.9-7.7-9.6-11.6-14.3c-0.8,7.6-2.4,14.7,0.5,21.8c7.4,18.2,12.4,37.2,16,56.5
|
||||
c1.7,9.1,5,17.3,10.6,24.4c9.2,11.4,18.5,22.8,28.7,33.3c9.9,10.1,21.1,19,32.2,28.9c3.8-6.5,5.6-13,9.8-18.2
|
||||
c7.7-9.4,17.7-9.5,25.3-0.1c3.6,4.5,5.7,9.7,7.8,15c4.3,10.9,6.8,22.3,8.9,33.7c0.9,5.2,3.3,8.1,7.5,10.7
|
||||
c21.9,13.4,44.1,26,67.3,37.1c10.9,5.2,22.1,10,33.5,15.2c1.8-4.8,3.1-8.6,4.7-12.2c1.6-3.5,4.2-6.3,8.3-6.1
|
||||
c3.6,0.2,5.8,2.9,7.3,6.1c2.4,5.2,3.8,10.6,4.8,16.2c0.8,4.6,2.9,7.2,7.7,9C950.6,871.5,990.8,883,1032.8,891.9z M1953.8,1233.7
|
||||
c-2.9,5.8-4.9,11.5-8.8,16.2c-7.6,9.3-17.8,9.3-25.3,0c-3.4-4.2-5.5-9-7.5-14c-4.4-11.5-7.2-23.5-9.4-35.7
|
||||
c-0.7-3.9-2.4-6.4-5.8-8.6c-30.4-19.4-62.4-35.6-95-50.8c-6.9-3.3-7-3-9.9,4.7c-0.7,1.9-1.5,3.7-2.3,5.5c-1.6,3.1-3.7,5.6-7.6,5.6
|
||||
c-3.8,0-5.9-2.5-7.6-5.6c-2.9-5.4-4.3-11.3-5.1-17.2c-0.7-5-3-7.3-7.5-9.1c-37.8-15.2-77.3-24.6-116.7-34.3c-3.8-0.9-5.4,0.2-6.2,4
|
||||
c-0.7,3.9-0.2,9.9-6.3,9.5c-5-0.3-5.1-6.1-5.3-9.6c-0.4-6.6-3.3-8.9-9.6-10c-32.8-5.6-65.4-12.2-98.6-15.7
|
||||
c-8.3-0.9-16.5-2.1-24.8-2.7c-5.4-0.4-7.1,1.2-7.5,6.6c-0.3,3.9-1.3,8,5.2,8.4c6.3,0.4,12.4,2.1,18.7,3.1
|
||||
c47.7,7.5,94.4,19.1,140.3,34.3c42.4,14,83.5,31.2,123,52c46.3,24.4,90.2,52.6,131,85.6c48.5,39.1,93.7,81.2,129.4,132.6
|
||||
c1.9,2.8,4.8,7.5,7.3,6.7c4.5-1.5,1.4-6.8,2.1-10.4c0.6-3.4-0.5-6.6-1.8-9.7c-7.8-19.5-12.6-40-17.1-60.4c-1.1-5.1-3-9.4-6-13.5
|
||||
c-17-22.8-36.7-43.1-57.6-62.2C1959.6,1237.1,1957.6,1234.7,1953.8,1233.7z M2043.6,585.1c-4.2,2.7-6.2,6.2-8.5,9.2
|
||||
c-29.9,40.3-64.6,76.1-102.5,109c-33.8,29.3-69.5,56.2-107.6,79.4c-51.8,31.6-106.3,57.7-163.9,77.3
|
||||
c-40.7,13.8-82.1,24.9-124.2,33.1c-13.7,2.7-27.6,4.7-41.3,7c-2.2,0.4-5.3-0.1-6.2,2.4c-1.4,3.8-1.1,8.2,0.3,11.7
|
||||
c1.4,3.5,5.3,1.6,8.1,1.3c40.1-4.2,79.8-10.7,119.5-18.1c6.2-1.2,10.4-2.7,10.5-10.2c0-3.5,0.3-9.3,5.4-9.5c6-0.3,5.5,5.7,6.2,9.6
|
||||
c0.7,3.9,2.3,4.9,6.1,3.9c10-2.7,20-5.1,30-7.5c28.5-6.9,56.4-15.6,83.9-25.7c6.8-2.5,10.2-6.3,11-13.3c0.5-5,2-9.8,4.5-14.2
|
||||
c4.2-7.6,10.8-7.6,15.1,0.1c1.6,2.9,2.5,6.2,3.4,9.3c0.9,2.9,2.5,3.5,5.2,2.2c2.4-1.2,5-1.9,7.3-3.1c26.6-13.1,53.2-26,78.7-41.2
|
||||
c11.9-7.1,19.6-14.7,21.3-29.3c1.4-12.4,5.5-24.8,12.8-35.4c8.1-11.7,19.5-11.7,27.5,0.1c3.1,4.6,5.4,9.7,8.2,14.9
|
||||
c1.9-1.4,3.6-2.5,5.1-3.8c22.2-19.8,42.7-41.1,60.6-64.9c2.6-3.5,4.3-7.3,5.2-11.7c4.5-21.2,9.6-42.2,17.6-62.4
|
||||
C2045.3,599.2,2044.5,592.6,2043.6,585.1z M777.2,1163.8c1.2,0.2,1.6,0.4,1.9,0.3c30.4-14.3,60.9-28.3,92.6-39.6
|
||||
c4.2-1.5,4.4-3.7,3.9-7.5c-4.5-37.4-6.3-75-6.5-112.6c0-7.8-0.2-7.9-8.5-7.9c-22,0-44,0-65.9,0c-8.6,0-8.8,0.1-8.8,8.7
|
||||
c-0.4,45.3-2.2,90.5-6.6,135.6C778.4,1148.4,776.3,1155.9,777.2,1163.8z M776.8,817.8c0.3,3.8,0.5,7.1,0.9,10.4
|
||||
c2.5,21.2,4.1,42.4,5.5,63.7c1.8,28.6,2.3,57.2,2.7,85.9c0.1,7.1,0.3,7.3,7.2,7.3c22.6,0.1,45.3,0.1,67.9,0.1
|
||||
c8.8,0,8.1-0.2,8.1-8.3c0.1-37,1.9-73.9,6.4-110.6c0.7-5.8-0.5-8.2-6-10.1c-29.5-10.3-58-23.3-86.2-36.8
|
||||
C781.5,818.4,779.9,817.2,776.8,817.8z M1896.4,1163.9c1-2.7,0.2-5.3-0.1-7.9c-2.4-19.8-3.9-39.8-5.3-59.7
|
||||
c-2.1-30.3-2.5-60.6-3.3-90.9c-0.2-8.7-0.2-8.8-8.6-8.8c-16.7-0.1-33.3,0-50-0.1c-6.3,0-12.7,0.2-19-0.1c-3.9-0.1-5.5,1.4-5.2,5.3
|
||||
c0.1,1.7,0.1,3.3,0,5c-1.1,36-2,72-6.4,107.7c-0.9,7.7-1.1,7.9,6.3,10.6c28.5,10.4,56.1,22.9,83.3,36.1
|
||||
C1890.6,1162.4,1893.1,1164.4,1896.4,1163.9z M1896.8,817.7c-2.5-0.3-4.3,0.6-6.1,1.5c-6,2.9-12,5.6-18,8.6
|
||||
c-22.1,10.9-44.9,20.1-67.9,28.7c-7.5,2.8-7.3,3-6.5,10.6c1,8.9,1.9,17.9,2.7,26.8c2.4,28.2,3.2,56.5,3.6,84.7
|
||||
c0.1,5.4,1.8,6.7,6.8,6.6c22.6-0.2,45.3-0.1,67.9-0.2c1,0,2-0.1,3,0c3.8,0.3,5.6-1.2,5.2-5.1c-0.2-1.6,0-3.3,0.1-5
|
||||
c0.7-38.6,1.7-77.2,5.2-115.6C1894.1,845.4,1895.5,831.6,1896.8,817.7z M908.3,1109.6c1.4,0,2.4,0.2,3.2,0
|
||||
c37.8-11.7,75.5-24,114.2-32.5c4.5-1,5.8-2.9,5.5-7.4c-1.3-22-2.5-43.9-2.3-65.9c0.1-7.3-0.2-7.4-8-7.4c-33.3,0-66.7,0.1-100,0.2
|
||||
c-7.7,0-7.9,0.1-7.9,7.5c0,31-1.6,61.9-4,92.9C908.8,1101,907.3,1105.2,908.3,1109.6z M1765.3,1110.4c0-2.5,0.1-3.5,0-4.4
|
||||
c-3.1-33.9-4.7-67.8-4.8-101.8c0-7.4-0.4-7.6-8.2-7.7c-33.3-0.1-66.7-0.2-100-0.2c-7.3,0-7.2,0.1-7.5,7.9c-0.7,21-1.1,42-2.3,62.9
|
||||
c-0.5,9-1.3,8.3,7.6,10.4c29.8,7.1,59.4,14.9,88.5,24.5C1747.2,1104.8,1755.7,1107.4,1765.3,1110.4z M908.8,872.2
|
||||
c-0.2,0.3-0.6,0.6-0.6,0.8c2.8,35.5,5.2,71,4.8,106.7c-0.1,4.6,1.9,5.6,6,5.6c34.7-0.1,69.3-0.1,104,0.1c4.9,0,6.2-1.8,6.1-6.4
|
||||
c-0.5-22,0.8-44,2.1-65.9c0.3-5.2-0.9-7.7-6.4-8.8c-36.9-7.6-72.4-20.1-108.5-30.5C913.8,873,911.6,871.1,908.8,872.2z
|
||||
M1765.6,872.4c-3.2-0.6-5.1,0.3-7,0.9c-22.2,6.7-44.2,14.5-66.7,20.3c-14.5,3.7-29,7.3-43.5,10.8c-4.4,1.1-6.5,2.6-6.2,8
|
||||
c1.3,21.9,1.8,43.9,2.6,65.8c0.2,7,0.2,7.1,7.1,7.1c33.6,0,67.2,0,100.8-0.1c7.5,0,7.7-0.2,7.7-7.8c0.2-24.6,0.9-49.2,2.6-73.8
|
||||
C1763.8,893.4,1764.7,883.1,1765.6,872.4z M725.8,1192.5c8.7-4.8,15.8-9.8,23.8-12.8c9.6-3.7,12.7-10.5,13-19.7
|
||||
c0.1-1.3,0.2-2.7,0.4-4c3.4-27.1,5.6-54.3,6.7-81.5c1-23.6,1.5-47.2,2-70.8c0.2-7-0.1-7-7.3-7.1c-6.7-0.1-13.3,0-20,0
|
||||
c-9,0-18,0.2-27,0c-4.2-0.1-6.2,1.3-5.9,5.7c0.2,2.3,0,4.7,0.1,7c0.8,27.6,1.2,55.2,3,82.8C717,1125.1,719.9,1158.2,725.8,1192.5z
|
||||
M1946.2,1191.7c2.9-2.2,2.4-4.7,2.8-6.9c4.5-26.6,6.9-53.5,8.9-80.4c2.5-33.9,3.2-67.9,3.7-101.8c0.1-5-1.8-6.2-6.4-6.1
|
||||
c-15.3,0.2-30.6,0-46,0.1c-7.6,0-7.8,0.2-7.7,7.8c0.7,44,2,87.9,6.7,131.6c1.1,10.3,2.2,20.5,3.5,30.7c0.4,2.8,0.2,6,3.5,7.8
|
||||
C1925.8,1180.2,1936.2,1186.1,1946.2,1191.7z M1947.7,789.1c-9.1,5-16.9,10.1-25.3,13.8c-8.2,3.6-10.5,9.5-11.5,17.8
|
||||
c-3,27.1-5.6,54.3-6.9,81.6c-1.2,25.3-1.5,50.6-2.5,75.9c-0.2,5.1,1.5,7,6.7,7c15-0.3,30-0.1,45-0.2c8.6,0,8.4,0.6,8.3-8.9
|
||||
c-0.3-30.6-1-61.3-3.1-91.9C1956.3,852.9,1953.6,821.8,1947.7,789.1z M726,789.2c-3.3,16.4-5.1,31.3-6.7,46.2
|
||||
c-4.6,42.4-6.3,85-7.3,127.7c-0.1,5.7,0,11.3-0.2,17c-0.1,3.4,1.4,4.8,4.7,4.8c17,0,34,0,51,0.1c3.6,0,4.6-1.9,4.4-5
|
||||
c-0.1-2.3-0.1-4.7-0.1-7c-0.5-37.3-1.7-74.6-5-111.8c-1.4-15.9-2.8-31.9-5.5-47.6c-0.4-2.6-0.3-5.2-3.2-6.8
|
||||
C747.6,801.1,737.2,795.4,726,789.2z M1625.1,910.5c-9.8,1.8-18,3.4-26.2,4.6c-23.3,3.5-46.2,9.9-70,10
|
||||
c-10.9,0.1-21.8,2.4-32.7,3.7c-3.5,0.4-6.2,1.1-6,6c0.5,15,0.7,30,0.7,44.9c0,4.4,1.7,5.8,5.9,5.7c8.3-0.2,16.6,0,25-0.1
|
||||
c30.6,0,61.3,0,91.9-0.1c9,0,9-0.1,9.2-9.4c0.2-13,0.2-26,0.6-38.9C1623.7,928.4,1624.5,919.8,1625.1,910.5z M1625.2,1071.1
|
||||
c-2.2-23.8-2.3-46.1-2.4-68.4c0-4.5-1-6.6-6.1-6.6c-39.9,0.1-79.9,0.1-119.8-0.1c-5.1,0-6.1,2-6.1,6.5c0,14.6-0.1,29.3-0.6,43.9
|
||||
c-0.1,4.5,1.6,5.6,5.6,6.1c12.6,1.4,25,3.8,37.7,4c8,0.1,15.9,1.1,23.8,2.5C1579.5,1063,1601.7,1066.9,1625.2,1071.1z
|
||||
M1049.3,1070.4c2.6,1,4.9-0.1,7-0.6c23-5.5,46.3-8.9,69.7-12.3c16.7-2.5,33.7-3.8,50.6-5.2c5.1-0.4,6.8-1.9,6.6-7
|
||||
c-0.4-14-0.3-27.9-0.5-41.9c-0.1-7-0.2-7.2-7.2-7.2c-39.3,0-78.5,0-117.8,0.1c-6.7,0-6.8,0.2-6.8,7.5c0,16.3-0.5,32.6-1.4,48.9
|
||||
C1049.3,1058.6,1048.4,1064.5,1049.3,1070.4z M1049.8,910.2c-0.4,1.2-0.8,1.8-0.8,2.4c0.6,22.6,2.5,45.2,1.9,67.8
|
||||
c-0.1,4.2,2.1,4.8,5.6,4.8c39.9,0,79.9-0.1,119.8,0.1c5.5,0,6.7-2.3,6.6-7.2c-0.2-11,0.1-22,0.2-32.9c0.1-15.9,0.1-15.5-16.3-17.4
|
||||
c-16.5-1.9-33.1-2.4-49.6-5.2C1094.7,918.9,1072,915.9,1049.8,910.2z M1260.5,985.4c21.6,0,43.2,0,64.9,0c9.1,0,9.1-0.1,9.1-9.4
|
||||
c0-10.3,0-20.6-0.1-30.9c-0.1-7.7-0.1-7.7-7.6-7.9c-2.3-0.1-4.7,0-7,0c-41.9,0-83.8-2.2-125.5-6.2c-4.6-0.4-5.9,0.9-6,5.1
|
||||
c-0.2,13.6-0.6,27.3-0.8,40.9c-0.1,8.3,0.1,8.4,8.1,8.4C1217.3,985.4,1238.9,985.4,1260.5,985.4z M1485,931.5
|
||||
c-3.1-1.4-5.4-0.6-7.7-0.4c-26.2,1.9-52.4,4.3-78.7,5.1c-17.3,0.5-34.6,1.3-51.9,1.1c-7.4-0.1-7.5,0.1-7.6,7.8
|
||||
c-0.1,11.7,0,23.3-0.1,35c0,3.7,1,5.8,5,5.5c2-0.1,4,0,6,0c41.6,0,83.2,0,124.9,0c2,0,4-0.1,6,0c3.3,0.1,5-1.2,4.9-4.6
|
||||
C1485.5,964.2,1485.3,947.6,1485,931.5z M1485,1051.1c0.3-17,0.5-33.6,0.9-50.1c0.1-4.2-2.2-4.8-5.7-4.8c-45,0.1-89.9,0.1-134.9,0
|
||||
c-4.7,0-6.5,1.4-6.3,6.2c0.3,12,0.4,24,0,36c-0.2,5.2,2.1,6,6.5,6c11.6-0.1,23.3,0.1,34.9,0.4
|
||||
C1415,1045.8,1449.5,1048.1,1485,1051.1z M1188.4,1050.6c47.5-3.4,93.2-6.8,139.2-6.4c0.7,0,1.3,0,2,0c3.2,0.2,4.9-0.9,4.9-4.5
|
||||
c-0.1-13,0-25.9,0-38.9c0-3.4-1.6-4.8-4.9-4.6c-1.7,0.1-3.3,0-5,0c-42.6,0-85.1,0-127.7,0c-1.7,0-3.3,0.1-5,0
|
||||
c-2.8-0.1-4.4,1.1-4.4,4.1C1187.9,1016.8,1188.1,1033.4,1188.4,1050.6z M2010.7,1230.1c0.9-7.3-0.9-13.8-1.5-20.3
|
||||
c-6.8-66-10.3-132.2-10.7-198.6c-0.1-16.1-0.3-16.3-16.8-14.7c-2.9,0.3-4.2,1.1-4.1,4.1c0.1,3,0,6,0,9
|
||||
c-0.3,41.3-2.1,82.6-6.1,123.8c-2,20.2-4.3,40.4-8.2,60.4c-0.9,4.8-0.2,7.8,4.5,10.4c10.2,5.7,19.9,12.2,29.8,18.3
|
||||
C2001.8,1225,2005.6,1228.2,2010.7,1230.1z M2011.2,752.5c-2.7-0.8-3.6,0.3-4.7,1c-13.3,8.3-26.4,16.7-39.8,24.7
|
||||
c-3.6,2.1-4.2,4.5-3.5,8.2c1.4,7.8,2.8,15.7,3.9,23.6c7.4,53.8,10.1,107.9,10.6,162.2c0.1,14.1-2.6,12.8,13.7,12.8
|
||||
c7,0,7-0.2,7.1-7.3c0.4-27.3,0.6-54.6,1.8-81.8c1.4-30.9,3.1-61.8,6.1-92.6C2008,786.3,2009.6,769.5,2011.2,752.5z M662.1,1231.5
|
||||
c3.7-2.4,5.6-3.7,7.5-4.9c11.6-7.7,22.9-16.2,35.5-22.2c5.3-2.5,6.2-5.5,5.1-11c-4.6-22.5-7-45.4-9-68.3
|
||||
c-3.5-39.9-5-79.8-5.4-119.8c-0.1-8.6-0.2-8.6-8.8-8.7c-1.3,0-2.7,0-4,0c-7.9,0.1-8,0.1-7.9,8.4c0,24.7-0.6,49.3-1.6,74
|
||||
c-1.3,32-3,63.9-6,95.8C665.8,1192.9,664.1,1211.1,662.1,1231.5z M662.9,752.1c-1,1.4-0.4,3-0.3,4.6c2,20.5,4.6,41.1,6,61.7
|
||||
c3.8,52.5,6.5,105.1,6.4,157.8c0,8.8,0.1,8.9,8.5,8.9c14.5-0.1,12.1,1.3,12.3-12.1c1.1-62.7,3.5-125.2,14.8-187
|
||||
c0.8-4.1-0.9-6-4.2-7.7c-13.4-6.8-25.6-15.5-38-23.9C666.8,753.1,665.5,751.1,662.9,752.1z M2044.3,728.6c-5.4,1.1-7.1,3.9-7.7,8.3
|
||||
c-1.7,14.9-4,29.7-5.6,44.6c-4.6,42.1-7.2,84.2-8.9,126.5c-1,25.3-1.3,50.5-1.3,76.8c6.5,0,12.4-0.1,18.3,0
|
||||
c3.8,0.1,5.5-1.4,5.2-5.3c-0.2-1.7,0-3.3,0-5c0-78.9,0-157.9,0-236.8C2044.3,734.9,2044.3,732,2044.3,728.6z M2044.3,1253
|
||||
c0-3.7,0-6.4,0-9c0-78.6,0-157.2,0-235.8c0-13,1.3-11.5-11.9-11.5c-13,0-11.5-1.8-11.5,11.8c0,59.3,3.1,118.5,8.7,177.6
|
||||
c1.9,19.5,4,39.1,7,58.5C2037.3,1248.9,2038.8,1251.8,2044.3,1253z M629.2,728.7c0,4.3,0,8,0,11.6c0,76.5,0,153.1,0,229.6
|
||||
c0,15.8,0,15.8,16,15.3c7.6-0.2,7.6-0.2,7.7-7.7c0.1-3.3,0-6.7,0-10c-0.8-55.6-3.1-111.1-8.5-166.4c-2-21.2-5-42.3-7.3-63.4
|
||||
C636.6,733.3,635,730.3,629.2,728.7z M629.2,1253.2c5.3-2.1,7.4-4.5,7.9-9.2c1.6-14.9,4-29.7,5.5-44.6
|
||||
c6.5-62.4,9.5-124.9,10.3-187.6c0.2-15.6,0.1-15.6-15-15.6c-9,0-8.8-0.6-8.7,8.6c0,2,0,4,0,6c0,75.7,0,151.3,0,227
|
||||
C629.2,1242.5,629.2,1247.2,629.2,1253.2z M2060,697.5c19-15.9,36.7-29.3,52.3-45.3c3.5-3.5,3.7-6.7,2.4-10.7
|
||||
c-2.8-8.9-5.4-17.8-8.7-26.5c-3.5-9.3-7.4-18.5-13.7-26.4c-5.3-6.6-8-6.6-13.4-0.4c-1.3,1.5-2.7,3-3.6,4.7
|
||||
c-8.9,15.7-17.3,31.5-15.3,50.5c0.4,3.6,0.1,7.3,0.1,11C2060,668,2060,681.5,2060,697.5z M613.2,1283.1
|
||||
c-20.8,13.6-36.9,29.9-53.8,45.4c-2.8,2.6-2.3,5.1-1.5,8.1c4.1,14.7,8.6,29.3,15.3,43.2c2.6,5.4,5.4,10.7,9.8,15
|
||||
c4.3,4.2,6.2,4.6,10,0.4c10.7-11.9,18.4-26.3,19.7-41.7C614.7,1330.6,613.2,1307.6,613.2,1283.1z M2060,1284.4
|
||||
c0,19.5,0.7,36.8-0.2,54c-1.1,19.1,6.3,35.2,15.8,50.5c7.4,12.1,12.1,11.7,19.8-0.6c9.7-15.4,14.6-32.7,19.7-50
|
||||
c1.1-3.7,0-6.2-2.6-8.8C2096.8,1313.6,2079.3,1299.9,2060,1284.4z M611.9,698.2c2.1-3.3,1.3-5.9,1.3-8.4c0.1-16-0.8-32,0.3-47.9
|
||||
c1.2-18.4-6.2-33.8-15.1-48.6c-7.6-12.7-12.6-12.4-20.4,0.3c-10.1,16.3-15.1,34.6-20.4,52.8c-1,3.6,0.6,5.5,3,7.7
|
||||
c8.8,8.1,17.3,16.7,26.3,24.5C594.8,685.4,603.3,691.5,611.9,698.2z M1788.2,864c-12.5,3.2-12.6,3.2-13.9,13.8
|
||||
c-0.9,7.6-1.6,15.2-2.1,22.8c-1.8,26.2-2.9,52.5-3.2,78.8c0,4,0.8,6.3,5.4,6c4.6-0.3,9.3-0.2,14,0c5.1,0.1,6.9-2.1,6.7-7.3
|
||||
c-0.5-11.3-0.2-22.6-0.7-33.9C1793.2,917.5,1792.3,890.9,1788.2,864z M885.3,1118.4c12.8-3,12.5-3,13.8-13.9
|
||||
c3.5-29.5,4.6-59.1,5-88.8c0.3-23.5,5.2-18.7-19.8-19.4c-4.4-0.1-5.7,1.5-5.7,5.8C878.8,1040.8,880.3,1079.4,885.3,1118.4z
|
||||
M885.7,863.5c-0.2,0.9-0.6,1.9-0.7,2.9c-4.3,33.7-6,67.5-6.1,101.4c0,5.5-2.5,12.6,1.3,16.2c3.8,3.6,10.8,1.1,16.3,1.2
|
||||
c7.9,0.1,8.1-0.1,7.9-8.4c-0.6-33.2-1.5-66.4-5.3-99.4C897.9,865.6,898,865.6,885.7,863.5z M1788.4,1117.5
|
||||
c4.4-34.4,5.9-68.6,6.5-102.9c0.1-5.6,2.4-12.7-0.9-16.5c-3.9-4.3-11.3-1.4-17.2-1.6c-7.6-0.2-7.7,0.1-7.7,7.6
|
||||
c0,24.7,1.3,49.3,2.8,73.9c0.6,10.3,2.2,20.5,2.9,30.8c0.3,3.6,1.7,5.5,4.9,6.5C1782.2,1116,1784.4,1117.7,1788.4,1117.5z
|
||||
M1481.1,1078.1c0-2.7,0.1-4.4,0-6c-0.3-7.8-0.3-8.1-8.2-8.7c-22.9-1.7-45.8-3.3-68.7-4.7c-19.9-1.2-39.9-0.9-59.9-1
|
||||
c-4.6,0-5.5,2-5.4,6c0.1,3.7,0.3,6.7,5.2,6.2c1.3-0.1,2.7,0,4,0c34.3-0.3,68.4,1.9,102.5,5.6
|
||||
C1460.6,1076.6,1470.4,1078.5,1481.1,1078.1z M1480,903c-15.3,1.6-30.8,3.6-46.3,4.8c-29.5,2.4-59.1,4.7-88.8,3.9
|
||||
c-5.1-0.1-6.1,2-6,6.5c0.1,4.1,1.1,6.3,5.6,5.8c1.3-0.2,2.7,0,4,0c42.3,0.3,84.5-2.6,126.7-5.9c4.2-0.3,6.2-1.6,5.8-5.9
|
||||
C1480.7,909.3,1482,906.3,1480,903z M1192.2,1078.2c3.1-0.2,5.4-0.3,7.6-0.5c37-4.6,74-7.6,111.3-7.8c6,0,12-0.1,18,0
|
||||
c3.7,0.1,5.3-1.1,5.4-5.1c0.1-6.5,0-6.9-7.3-7c-40.6-0.8-81.1,1.7-121.6,4.6C1193.1,1063.3,1193.2,1063.6,1192.2,1078.2z
|
||||
M1192.6,903.2c0.2,14.4,0.2,14.4,12.8,16c3,0.4,6,0.6,8.9,0.8c19.3,1.1,38.5,2,57.8,3.2c18.6,1.2,37.2,0.8,55.9,0.8
|
||||
c6.4,0,6.3-0.5,6.6-6.8c0.2-5.8-3.3-5.5-7.1-5.4c-28.3,0.8-56.5-1.5-84.7-3.6C1226.3,907,1209.8,905.4,1192.6,903.2z M653.6,694.6
|
||||
c2,12.3,3.8,22.4,5.1,32.6c0.5,3.7,1.7,6.1,4.8,8c14.2,8.8,27.5,18.8,42.2,26.6c2.7,1.4,6.2,6.3,8.9,2.4c2.7-3.7-2.6-6.2-5-8.5
|
||||
c-19-17.3-36.7-35.8-51.8-56.7C657,698.1,656,697.3,653.6,694.6z M654,1284.6c0.8,0,1.3,0.1,1.5,0c1.4-1.4,2.9-2.7,4-4.3
|
||||
c14.6-20.8,32.8-38.3,51.2-55.5c1.2-1.1,2.6-2.1,3.6-3.4c1.4-1.7,1.5-3.6-0.2-5.7c-2.9,0.3-5.3,2.1-7.8,3.6
|
||||
c-10.2,6.3-19.8,13.5-30.5,18.8c-12.9,6.4-19.3,15.7-19.2,30.2C656.6,1273.7,654.9,1279.1,654,1284.6z M2019.1,698.8
|
||||
c-0.5-0.3-1-0.6-1.5-0.9c-1,1.2-2,2.4-3,3.6c-6.8,9-13,18.4-21.4,26.1c-11,10.1-20.5,21.7-32.4,30.8c-1.9,1.5-3.6,3.2-2.1,5.9
|
||||
c2,3.5,4.1,0.5,5.7-0.4c15.4-9.7,30.8-19.6,46.1-29.5c2.3-1.5,4-3.1,4.4-6.2C2016.2,718.5,2017.7,708.6,2019.1,698.8z
|
||||
M2017.1,1283.9c0.7-0.2,1.4-0.5,2.1-0.7c-1.5-10.1-3-20.2-4.4-30.4c-0.4-2.6-1.9-4.1-3.9-5.4c-15.6-10-31.3-20.1-46.9-30
|
||||
c-1.5-1-3.3-2.8-5-0.5c-1.7,2.4-0.3,4.2,1.7,5.9c4,3.4,8.1,6.9,11.8,10.6C1988.4,1249.3,2005.3,1264.3,2017.1,1283.9z M1044.6,1034
|
||||
c0.2,0,0.3,0,0.5,0c0-11-0.1-21.9,0.1-32.9c0.1-4.2-2-4.8-5.5-4.9c-4.5-0.1-4.6,2.6-4.6,5.9c0.1,22.3,1.2,44.5,2.4,66.8
|
||||
c0.1,1,0.2,2,0.4,3c0.2,1.2,1.2,1.8,2.1,1.7c0.8-0.1,1.9-1,2.2-1.7c0.5-1.5,0.7-3.2,0.8-4.9C1043.5,1055.9,1044,1045,1044.6,1034z
|
||||
M1636.4,1033.8c0.7,0,1.4,0,2.1,0c0-10.6,0-21.2,0-31.8c0-2.7,0.2-5.3-3.6-5.8c-4.7-0.6-6,0.4-6,6.1c-0.1,21.9,0.2,43.7,2.1,65.6
|
||||
c0.1,1.3,0.1,2.7,0.6,3.9c0.3,0.7,1.5,1.6,2.2,1.5c0.7,0,1.7-1.1,1.9-1.8c0.5-1.9,0.7-3.9,0.7-5.9
|
||||
C1636.4,1055,1636.4,1044.4,1636.4,1033.8z M1638.5,948.1c-0.2,0-0.5,0-0.7,0c-0.5-11.6-1-23.3-1.6-34.9c-0.1-1.9,0.2-4.8-2.3-4.9
|
||||
c-3.3-0.1-2.8,3-3,5.2c-0.3,3-0.4,6-0.7,9c-1.5,19-1.4,37.9-1.4,56.9c0,5.3,2,6.8,6.4,5.8c3.8-0.8,3.3-3.6,3.3-6.2
|
||||
C1638.5,968.8,1638.5,958.4,1638.5,948.1z M1045.1,948c-0.1,0-0.3,0-0.4,0c-0.6-11.9-1.2-23.9-1.9-35.8c-0.1-1.9-0.4-4.3-3.2-3.9
|
||||
c-2.2,0.3-1.9,2.4-2,4c-1.3,22.5-2.3,45.1-2.5,67.7c0,3.7,1.2,5.4,5.1,5.4c4.1,0.1,5-1.9,5-5.5C1045,969.3,1045.1,958.7,1045.1,948
|
||||
z M741.9,740.1c-4.9,5.5-6.3,11-8.3,16.2c-0.8,2-0.8,4,1.2,5.3c6.2,4.1,11.6,9.4,19.7,13C751.6,762.1,748.8,750.9,741.9,740.1z
|
||||
M1931.7,1241.3c11.7-16.6,11.5-18.2-3-28.3c-2.7-1.9-4.9-4.5-9.1-4.9C1921.9,1219.6,1924.7,1230.7,1931.7,1241.3z M1931.6,740
|
||||
c-6.6,10.9-9.8,22-12.1,33.9c4-0.9,6.1-3.1,8.5-4.8C1943.5,758.3,1943.6,757.2,1931.6,740z M742.1,1241.3
|
||||
c6.6-11.1,9.8-21.9,12.2-34.3c-8,3.6-13.2,8.8-19.2,12.6c-2,1.3-2.3,3.1-1.6,5.1C735.6,1230.2,736.7,1236.2,742.1,1241.3z
|
||||
M597.6,1464.4c16.9-6.5,17.3-8.9,14-24.4C605.8,1447.2,601.7,1455,597.6,1464.4z M2061.5,1442.5c-5.1,14.3,5.6,17.3,12.5,22.8
|
||||
C2072.1,1457,2066.2,1451,2061.5,1442.5z M1781.6,836.9c-1,5.2-3.6,8.5-1.8,12.6C1785.7,846.2,1785.7,846.2,1781.6,836.9z
|
||||
M895.4,849.3c-1.5-4.5-1.6-7.9-3.8-11.1C887.9,846.3,887.9,846.3,895.4,849.3z M630.1,1284.6c-0.4-5.3,4.1-10.2,1.8-17
|
||||
C627.4,1273.6,629.9,1279.2,630.1,1284.6z M629.9,697.1c0.2,5.4-3,11.4,2.4,16.6C633.8,707.4,630.4,702.5,629.9,697.1z
|
||||
M2044.5,1284.5c-1-5.2,1.5-10.3-2.8-15C2040.3,1275.4,2042.4,1279.6,2044.5,1284.5z M2041.9,712.1c4.1-4.8,1.6-10,2.2-14.6
|
||||
C2043,702.1,2039.8,706.2,2041.9,712.1z M892.2,1143.9c0.4-4.2,3.5-7.3,1.6-11.5C887.6,1136,887.6,1136,892.2,1143.9z
|
||||
M1779.6,1132.2c-1,2.9-0.4,4.8,0.4,6.5c0.7,1.5,0.4,3.6,2.3,4.6C1785.7,1135.3,1785.7,1135.3,1779.6,1132.2z M729.5,774
|
||||
c1.5,1.6,2.6,4,5.8,3.2C733.9,774.7,732.2,773.5,729.5,774z M737.9,778.4c-0.2,0.2-0.5,0.4-0.7,0.6c0.2,0.2,0.4,0.6,0.6,0.6
|
||||
c0.2,0,0.5-0.3,0.7-0.5C738.4,778.8,738.1,778.6,737.9,778.4z"/>
|
||||
<path class="st1" d="M534.9,1353.5c4.5,11.6,7.4,22.2,12.2,32c4.7,9.6,9.8,18.8,17.1,26.8c10.6,11.6,28.8,15.5,42.7,2.9
|
||||
c1.4-1.3,3.1-2.4,5.5-4.2c1.4,7.2,1.7,13.2-2.5,19.1c-8.4,11.7-14.8,24.4-20.8,37.4c-1.7,3.7-3.7,6.7-7.3,9.1
|
||||
c-52.7,33.7-105.3,67.6-158,101.4c-1.1,0.7-2.1,1.7-4.1,1.1c0-195.1,0-390.4,0-585.6c2.1-1.2,4.1-0.9,6-0.8
|
||||
c21.6,0.5,43.3,1.1,64.9,1.3c5.3,0,6.7,1.8,6.7,6.9c0.5,40,1.4,79.9,3.9,119.8c2.4,38.2,5.6,76.4,10.6,114.3
|
||||
c4.6,34.7,10.1,69.1,19.2,103c1.1,4,0.6,7.1-2.3,10c-0.9,0.9-1.7,2.1-2.6,3.1c-22.1,25.1-40.6,52.7-57.5,81.5
|
||||
c-11.6,19.7-19.9,41-28.9,61.9c14.9-23.9,25.7-50.1,41.8-73.5C497.6,1397.6,514.9,1375.3,534.9,1353.5z"/>
|
||||
<path class="st1" d="M534.6,627.2c-40-41.2-69.6-89-94.2-140.1c0.7,6.2,4,11.2,6.2,16.6c12.7,31.7,30.5,60.6,49.6,88.7
|
||||
c9.3,13.7,20.4,26.3,31,39.1c3.8,4.5,5,8.7,3.4,14.4c-9.6,35.4-15,71.6-19.6,107.9c-9.4,74.5-12.5,149.3-13.7,224.3c0,1-0.1,2,0,3
|
||||
c0.6,4.7-1.2,6.5-6.1,6.4c-8.3-0.2-16.7,0.2-25,0.4c-13.7,0.3-27.3,0.4-41,1c-4.8,0.2-6.8-1.2-6.2-6.1c0.3-2,0-4,0-6
|
||||
c0-187.9,0-375.8,0-563.7c0-3.5,0-7.1,0-10.7c4.2-0.1,6.3,2.3,8.7,3.9c50.4,32.4,100.8,64.9,151.3,97.1c5,3.2,8.4,7.1,10.8,12.4
|
||||
c5.1,11.2,10.4,22.4,17.8,32.3c4.9,6.6,6.7,13.5,5.1,21.9c-3.6-0.1-5.6-3.1-7.9-5c-12-9.8-28.1-7.7-38.9,2.8
|
||||
c-12.6,12.2-19,27.9-25.2,43.7C538.8,616.5,538.2,622.2,534.6,627.2z"/>
|
||||
<path class="st1" d="M2060.9,1411.1c7.4,4.3,12.3,10.3,20.6,11.4c9.8,1.4,17.7-1.4,24.8-7.5c8.5-7.3,13.9-16.9,18.8-26.8
|
||||
c5.1-10.4,8.6-21.5,12.7-32.5c24.7,24.7,73.3,78.9,107.2,169.7c-1.1-8.8-3.5-17.2-6.8-25.4c-13.2-33.9-29.3-66.4-49.3-96.8
|
||||
c-12.8-19.5-27.4-37.7-42.4-55.5c-3.4-4-4.8-7.1-3.5-12.3c12.8-48.8,19.1-98.6,24-148.7c5.9-59.7,8.4-119.6,8.9-179.7
|
||||
c0-1.3,0-2.7,0.1-4c0.2-7,0.2-7.1,7-7.4c20-0.7,40-1.3,60-1.9c2.3-0.1,4.7,0,7-0.1c3.5-0.2,4.8,1.4,4.6,4.8c-0.1,2,0,4,0,6
|
||||
c0,188.3,0,376.6,0,565c0,3.2,0,6.4,0,9.8c-4.5-0.2-6.9-2.8-9.5-4.5c-47.7-30.6-95.2-61.4-143-91.7c-9-5.7-16-11.6-19.9-22.4
|
||||
c-3.7-10.1-11.2-18.9-17.1-28.2C2060.8,1426.2,2058.6,1419.6,2060.9,1411.1z"/>
|
||||
<path class="st1" d="M2244.8,454.3c-23,66.2-59.6,121.8-106.6,172.4c-4.5-11.6-7.7-22.6-12.8-32.9c-4-8.1-8.4-15.8-14.4-22.7
|
||||
c-12.6-14.4-29.9-17.7-45.6-3.8c-1.1,1-2,2.7-4.6,2.6c-1.4-5.5-1.1-10.5,2-15.7c8.8-14.9,18.9-29,25.8-45c1-2.3,2.7-3.7,4.7-4.9
|
||||
c52.7-33.9,105.3-67.7,158-101.5c0.5-0.3,1.3-0.2,2.1-0.4c2.3,2.1,1.2,4.9,1.2,7.3c0.1,189.6,0.1,379.3,0.1,568.9
|
||||
c0,1.7-0.1,3.3,0,5c0.2,3.6-1.7,4.6-4.9,4.5c-5-0.2-10-0.3-15-0.5c-17-0.5-34-1-51-1.4c-7.4-0.2-7.5-0.2-7.6-7.8
|
||||
c-0.5-35.3-1.1-70.6-3.1-105.9c-2.6-45.9-6.4-91.7-12.8-137.3c-4.2-30.3-9.4-60.5-17.3-90.2c-1.1-4-0.2-6.6,2.5-9.8
|
||||
c27.4-31.6,51.5-65.6,70.4-102.8c11.3-22.2,21.7-44.9,28.2-69.2C2244.6,460.9,2245.4,458.8,2244.8,454.3z"/>
|
||||
<path class="st1" d="M553.7,662.8c3.8,0.2,5.2,2.8,6.9,4.6c14.5,15.2,30.3,29,46.9,41.9c4.1,3.2,5.8,6.4,5.8,11.7
|
||||
c-0.2,85.6-0.1,171.2-0.1,256.8c0,7.5-0.1,7.7-7.6,7.8c-25,0.4-50,0.6-74.9,1.1c-4.3,0.1-5.9-1.2-5.7-5.6c0.4-13,0.2-26,0.5-39
|
||||
c1.5-60.3,4.7-120.5,11.8-180.4c3.7-31.7,8.3-63.3,15.4-94.5C553,665.6,553.4,664.1,553.7,662.8z"/>
|
||||
<path class="st1" d="M553.6,1319.7c-22.4-107.4-27.4-215.3-28.7-323.3c3.3-2.5,6.4-1.5,9.3-1.5c23.3,0.3,46.7,0.7,70,1.1
|
||||
c9.5,0.1,9.2-1.2,9.2,9.2c0,72.3,0,144.7,0,217c0,13.3-0.1,26.7,0.1,40c0,3.6-0.7,6.2-3.8,8.6c-18.7,14.5-36.3,30.3-52.7,47.4
|
||||
C556.3,1318.5,555.6,1318.7,553.6,1319.7z"/>
|
||||
<path class="st1" d="M2119.8,1319.2c-6.7-6.5-12.1-12.3-18-17.5c-11.8-10.2-23.8-20.2-36-30.1c-4.2-3.4-5.9-7.1-5.9-12.7
|
||||
c0.2-84.6,0.1-169.1,0.1-253.7c0-8.4,0.1-8.5,8-8.5c24.3-0.2,48.6-0.4,72.9-0.5c7.5,0,7.7,0,7.6,7.7c-0.3,45.6-1.9,91.2-5.1,136.7
|
||||
c-2.7,39.9-6.7,79.6-12.5,119.1C2128.1,1279,2124.6,1298.3,2119.8,1319.2z"/>
|
||||
<path class="st1" d="M2120.4,664.4c12.3,57.5,18.7,115.7,22.8,174.2c3.1,44.9,4.7,89.8,5.2,134.8c0,2.3-0.1,4.7,0.1,7
|
||||
c0.3,4-1.7,5.2-5.4,5.1c-6.7-0.2-13.3-0.3-20-0.3c-18.3-0.1-36.7-0.2-55-0.4c-8-0.1-8.2-0.1-8.2-8.2c0-52.3,0-104.6,0-157
|
||||
c0-32,0.2-64-0.2-96c-0.1-6.3,2-10.3,6.8-14.3c17.1-14.3,35.1-27.7,50.2-44.2C2117.5,664.3,2118.3,663.8,2120.4,664.4z"/>
|
||||
<path class="st1" d="M878.7,1138.5c2,4.9,3.2,9.1,5.3,12.8c4,7.2,10.5,7.1,14.6,0c3-5.3,4.3-11.2,5.2-17.1c0.8-4.7,3.2-7.1,7.8-8.9
|
||||
c37.6-14.8,76.3-25.6,115.5-34.7c5-1.1,6.6,0.4,7.3,4.9c0.6,3.5,0.7,8.4,5.3,8.5c5.3,0.1,5.5-5.2,5.8-9c0.5-7.1,3.9-10,10.9-11.2
|
||||
c40-7,79.8-14.9,120.4-18c2.4-0.2,5.9-2.1,6.9,1.5c1,3.7,1.8,7.8,0.4,11.7c-0.7,1.8-2.9,1.2-4.4,1.5c-9.5,1.6-19.1,2.8-28.6,4.6
|
||||
c-39.6,7.4-78.6,17-117,29.2c-59.9,19-116.5,45.2-170.8,76.4c-41.6,24-80.2,52.3-117.2,83.1c-39.4,32.8-74.3,69.6-105.4,110.2
|
||||
c-3,3.9-6.2,7.7-9.4,11.6c-0.1,0.2-0.6,0.2-2.1,0.5c0.2-7-1.1-13.6,1.5-20.3c7.6-19.2,12.9-39.2,16.7-59.5
|
||||
c1.8-9.6,6.2-17.5,12-24.8c17.2-21.7,36.5-41.5,58.8-58.9c3.4,2.2,3.5,6.6,5.4,9.8c4,7,7.8,14.4,17.4,14.4
|
||||
c9.6,0,13.8-7.2,17.3-14.4c6.5-13.3,9.4-27.7,12.2-42c0.8-3.9,2.2-6.5,5.7-8.6C808.9,1171.4,842.7,1153.5,878.7,1138.5z"/>
|
||||
<path class="st1" d="M1032.8,891.9c-42-9-82.2-20.4-121.3-35.7c-4.8-1.9-6.9-4.4-7.7-9c-0.9-5.6-2.3-11.1-4.8-16.2
|
||||
c-1.5-3.2-3.7-5.9-7.3-6.1c-4.1-0.2-6.8,2.5-8.3,6.1c-1.6,3.6-2.8,7.4-4.7,12.2c-11.4-5.2-22.6-10-33.5-15.2
|
||||
c-23.1-11.1-45.4-23.7-67.3-37.1c-4.3-2.6-6.6-5.5-7.5-10.7c-2.1-11.4-4.6-22.8-8.9-33.7c-2.1-5.3-4.2-10.5-7.8-15
|
||||
c-7.6-9.4-17.5-9.4-25.3,0.1c-4.2,5.2-6,11.7-9.8,18.2c-11.1-9.9-22.3-18.8-32.2-28.9c-10.2-10.4-19.5-21.8-28.7-33.3
|
||||
c-5.6-7-8.9-15.3-10.6-24.4c-3.7-19.3-8.6-38.3-16-56.5c-2.9-7.1-1.4-14.1-0.5-21.8c3.9,4.8,7.9,9.4,11.6,14.3
|
||||
c23.7,32,51,60.7,80.1,87.7c19,17.6,39.4,33.9,59.9,49.7c45,34.7,93.8,63.3,144.7,87.9c39.2,18.9,79.8,34.7,121.6,47.1
|
||||
c41.2,12.2,82.8,22.3,125.4,28.2c1,0.1,2,0.2,3,0.4c7.3,1.2,10.5,7.7,7,14.2c-1.3,2.3-3.4,1.7-5.3,1.6c-8-0.7-15.9-1.5-23.8-2.4
|
||||
c-32.8-3.7-65.1-10.2-97.5-15.7c-7.1-1.2-11.5-3.9-11.7-11.6c0-1.3-0.4-2.6-0.8-3.9c-0.8-2.6-1.9-5.3-5.3-4.9
|
||||
c-2.7,0.3-3.7,2.6-4.4,4.9C1034.4,885.3,1033.7,888.2,1032.8,891.9z"/>
|
||||
<path class="st1" d="M1953.8,1233.7c3.8,1,5.8,3.4,8,5.3c21,19.1,40.7,39.4,57.6,62.2c3,4.1,4.9,8.4,6,13.5
|
||||
c4.5,20.5,9.3,40.9,17.1,60.4c1.3,3.1,2.4,6.3,1.8,9.7c-0.6,3.6,2.4,8.9-2.1,10.4c-2.5,0.8-5.4-3.9-7.3-6.7
|
||||
c-35.7-51.5-81-93.6-129.4-132.6c-40.8-32.9-84.7-61.2-131-85.6c-39.5-20.8-80.6-37.9-123-52c-45.9-15.2-92.6-26.8-140.3-34.3
|
||||
c-6.2-1-12.4-2.7-18.7-3.1c-6.5-0.4-5.5-4.5-5.2-8.4c0.4-5.4,2.1-7.1,7.5-6.6c8.3,0.7,16.5,1.9,24.8,2.7
|
||||
c33.2,3.4,65.8,10.1,98.6,15.7c6.3,1.1,9.2,3.4,9.6,10c0.2,3.6,0.3,9.3,5.3,9.6c6,0.4,5.5-5.6,6.3-9.5c0.7-3.9,2.4-5,6.2-4
|
||||
c39.4,9.7,78.9,19.2,116.7,34.3c4.5,1.8,6.8,4.2,7.5,9.1c0.9,5.9,2.2,11.8,5.1,17.2c1.7,3.1,3.8,5.6,7.6,5.6c3.8,0,6-2.5,7.6-5.6
|
||||
c0.9-1.8,1.7-3.6,2.3-5.5c2.8-7.7,2.9-7.9,9.9-4.7c32.6,15.3,64.6,31.4,95,50.8c3.4,2.2,5.1,4.7,5.8,8.6
|
||||
c2.1,12.1,4.9,24.1,9.4,35.7c1.9,5,4.1,9.8,7.5,14c7.5,9.4,17.7,9.4,25.3,0C1948.9,1245.3,1950.9,1239.5,1953.8,1233.7z"/>
|
||||
<path class="st1" d="M2043.6,585.1c0.9,7.5,1.7,14.1-0.8,20.4c-8,20.2-13.1,41.2-17.6,62.4c-0.9,4.4-2.6,8.3-5.2,11.7
|
||||
c-17.8,23.8-38.4,45.2-60.6,64.9c-1.5,1.3-3.2,2.3-5.1,3.8c-2.9-5.3-5.1-10.4-8.2-14.9c-8-11.8-19.3-11.8-27.5-0.1
|
||||
c-7.4,10.6-11.4,23-12.8,35.4c-1.7,14.6-9.4,22.2-21.3,29.3c-25.5,15.2-52.1,28.1-78.7,41.2c-2.4,1.2-5,2-7.3,3.1
|
||||
c-2.7,1.3-4.3,0.7-5.2-2.2c-1-3.2-1.9-6.4-3.4-9.3c-4.2-7.7-10.8-7.7-15.1-0.1c-2.5,4.4-3.9,9.3-4.5,14.2c-0.8,7-4.2,10.7-11,13.3
|
||||
c-27.5,10.2-55.4,18.9-83.9,25.7c-10,2.4-20.1,4.8-30,7.5c-3.8,1-5.4,0-6.1-3.9c-0.7-4-0.2-9.9-6.2-9.6c-5.1,0.2-5.3,6-5.4,9.5
|
||||
c-0.1,7.5-4.3,9-10.5,10.2c-39.6,7.4-79.4,13.9-119.5,18.1c-2.8,0.3-6.7,2.3-8.1-1.3c-1.4-3.6-1.7-7.9-0.3-11.7
|
||||
c0.9-2.4,3.9-2,6.2-2.4c13.8-2.4,27.6-4.4,41.3-7c42.1-8.2,83.5-19.3,124.2-33.1c57.6-19.6,112.2-45.7,163.9-77.3
|
||||
c38.1-23.2,73.8-50.1,107.6-79.4c37.9-32.8,72.6-68.6,102.5-109C2037.5,591.3,2039.4,587.8,2043.6,585.1z"/>
|
||||
<path class="st1" d="M777.2,1163.8c-0.9-7.9,1.2-15.4,2-23c4.4-45.1,6.3-90.3,6.6-135.6c0.1-8.6,0.2-8.7,8.8-8.7
|
||||
c22-0.1,44-0.1,65.9,0c8.3,0,8.4,0.1,8.5,7.9c0.2,37.7,2,75.2,6.5,112.6c0.5,3.8,0.3,6-3.9,7.5c-31.7,11.3-62.2,25.3-92.6,39.6
|
||||
C778.8,1164.3,778.4,1164.1,777.2,1163.8z"/>
|
||||
<path class="st1" d="M776.8,817.8c3.1-0.6,4.7,0.6,6.5,1.4c28.2,13.5,56.6,26.5,86.2,36.8c5.4,1.9,6.7,4.3,6,10.1
|
||||
c-4.5,36.7-6.3,73.6-6.4,110.6c0,8.1,0.7,8.3-8.1,8.3c-22.6-0.1-45.3,0-67.9-0.1c-7,0-7.1-0.2-7.2-7.3c-0.4-28.6-0.9-57.3-2.7-85.9
|
||||
c-1.3-21.3-2.9-42.5-5.5-63.7C777.2,824.9,777.1,821.6,776.8,817.8z"/>
|
||||
<path class="st1" d="M1896.4,1163.9c-3.3,0.5-5.7-1.5-8.4-2.8c-27.2-13.3-54.8-25.7-83.3-36.1c-7.4-2.7-7.2-2.9-6.3-10.6
|
||||
c4.4-35.8,5.3-71.8,6.4-107.7c0.1-1.7,0.1-3.3,0-5c-0.3-3.8,1.3-5.4,5.2-5.3c6.3,0.2,12.7,0.1,19,0.1c16.7,0,33.3,0,50,0.1
|
||||
c8.4,0,8.4,0.1,8.6,8.8c0.8,30.3,1.2,60.6,3.3,90.9c1.4,19.9,2.9,39.9,5.3,59.7C1896.5,1158.6,1897.4,1161.2,1896.4,1163.9z"/>
|
||||
<path class="st1" d="M1896.8,817.7c-1.3,13.9-2.7,27.7-3.9,41.6c-3.5,38.5-4.5,77-5.2,115.6c0,1.7-0.2,3.3-0.1,5
|
||||
c0.4,4-1.5,5.5-5.2,5.1c-1-0.1-2,0-3,0c-22.6,0-45.3-0.1-67.9,0.2c-5,0-6.8-1.2-6.8-6.6c-0.4-28.3-1.2-56.5-3.6-84.7
|
||||
c-0.8-8.9-1.6-17.9-2.7-26.8c-0.9-7.6-1.1-7.8,6.5-10.6c23-8.7,45.8-17.8,67.9-28.7c6-2.9,12-5.7,18-8.6
|
||||
C1892.5,818.3,1894.3,817.4,1896.8,817.7z"/>
|
||||
<path class="st1" d="M908.3,1109.6c-1-4.4,0.4-8.6,0.8-12.8c2.5-30.9,4-61.8,4-92.9c0-7.4,0.2-7.5,7.9-7.5
|
||||
c33.3-0.1,66.7-0.2,100-0.2c7.8,0,8.1,0.1,8,7.4c-0.3,22,1,44,2.3,65.9c0.3,4.5-1,6.5-5.5,7.4c-38.7,8.5-76.4,20.9-114.2,32.5
|
||||
C910.7,1109.8,909.7,1109.6,908.3,1109.6z"/>
|
||||
<path class="st1" d="M1765.3,1110.4c-9.6-3-18.1-5.6-26.6-8.4c-29.1-9.6-58.7-17.4-88.5-24.5c-8.9-2.1-8.1-1.4-7.6-10.4
|
||||
c1.2-21,1.6-42,2.3-62.9c0.2-7.7,0.2-7.9,7.5-7.9c33.3,0,66.7,0.1,100,0.2c7.8,0,8.2,0.3,8.2,7.7c0.2,34,1.7,68,4.8,101.8
|
||||
C1765.4,1106.9,1765.3,1107.9,1765.3,1110.4z"/>
|
||||
<path class="st1" d="M908.8,872.2c2.8-1.1,5,0.8,7.5,1.5c36.1,10.3,71.6,22.9,108.5,30.5c5.5,1.1,6.7,3.6,6.4,8.8
|
||||
c-1.3,21.9-2.6,43.9-2.1,65.9c0.1,4.7-1.2,6.5-6.1,6.4c-34.7-0.2-69.3-0.2-104-0.1c-4.1,0-6.1-1-6-5.6c0.4-35.6-2-71.2-4.8-106.7
|
||||
C908.2,872.8,908.6,872.5,908.8,872.2z"/>
|
||||
<path class="st1" d="M1765.6,872.4c-0.9,10.7-1.8,20.9-2.5,31.2c-1.7,24.6-2.4,49.2-2.6,73.8c-0.1,7.7-0.2,7.8-7.7,7.8
|
||||
c-33.6,0.1-67.2,0.1-100.8,0.1c-6.9,0-6.9-0.2-7.1-7.1c-0.8-21.9-1.3-43.9-2.6-65.8c-0.3-5.4,1.8-7,6.2-8
|
||||
c14.5-3.5,29-7.1,43.5-10.8c22.5-5.7,44.5-13.5,66.7-20.3C1760.5,872.7,1762.3,871.8,1765.6,872.4z"/>
|
||||
<path class="st1" d="M725.8,1192.5c-5.9-34.3-8.8-67.4-11-100.6c-1.8-27.6-2.2-55.2-3-82.8c-0.1-2.3,0.1-4.7-0.1-7
|
||||
c-0.4-4.4,1.6-5.8,5.9-5.7c9,0.2,18,0,27,0c6.7,0,13.3-0.1,20,0c7.2,0.1,7.4,0.2,7.3,7.1c-0.5,23.6-1,47.2-2,70.8
|
||||
c-1.2,27.2-3.3,54.4-6.7,81.5c-0.2,1.3-0.3,2.6-0.4,4c-0.4,9.2-3.4,16-13,19.7C741.7,1182.7,734.5,1187.7,725.8,1192.5z"/>
|
||||
<path class="st1" d="M1946.2,1191.7c-10.1-5.6-20.4-11.5-30.8-17.2c-3.3-1.8-3.1-5-3.5-7.8c-1.4-10.2-2.4-20.5-3.5-30.7
|
||||
c-4.7-43.8-6-87.7-6.7-131.6c-0.1-7.7,0.1-7.8,7.7-7.8c15.3-0.1,30.6,0.1,46-0.1c4.6-0.1,6.5,1.1,6.4,6.1
|
||||
c-0.5,34-1.2,67.9-3.7,101.8c-2,26.9-4.5,53.7-8.9,80.4C1948.7,1187,1949.1,1189.5,1946.2,1191.7z"/>
|
||||
<path class="st1" d="M1947.7,789.1c5.8,32.7,8.6,63.8,10.8,95c2.1,30.6,2.8,61.2,3.1,91.9c0.1,9.6,0.3,8.9-8.3,8.9
|
||||
c-15,0-30-0.1-45,0.2c-5.3,0.1-6.9-1.8-6.7-7c0.9-25.3,1.3-50.6,2.5-75.9c1.3-27.3,3.9-54.4,6.9-81.6c0.9-8.3,3.3-14.2,11.5-17.8
|
||||
C1930.8,799.2,1938.6,794.1,1947.7,789.1z"/>
|
||||
<path class="st1" d="M726,789.2c11.3,6.2,21.7,11.9,32.2,17.5c2.9,1.6,2.8,4.2,3.2,6.8c2.7,15.8,4.1,31.7,5.5,47.6
|
||||
c3.3,37.2,4.5,74.5,5,111.8c0,2.3,0,4.7,0.1,7c0.1,3.1-0.8,5-4.4,5c-17-0.1-34-0.1-51-0.1c-3.3,0-4.8-1.5-4.7-4.8
|
||||
c0.2-5.7,0-11.3,0.2-17c1-42.6,2.7-85.2,7.3-127.7C720.9,820.5,722.6,805.7,726,789.2z"/>
|
||||
<path class="st1" d="M1625.1,910.5c-0.6,9.4-1.4,18-1.7,26.5c-0.4,13-0.4,26-0.6,38.9c-0.1,9.3-0.2,9.4-9.2,9.4
|
||||
c-30.6,0.1-61.3,0.1-91.9,0.1c-8.3,0-16.7-0.1-25,0.1c-4.2,0.1-5.9-1.3-5.9-5.7c0-15-0.1-30-0.7-44.9c-0.2-4.8,2.5-5.5,6-6
|
||||
c10.9-1.3,21.8-3.6,32.7-3.7c23.9-0.2,46.7-6.5,70-10C1607.1,913.9,1615.2,912.2,1625.1,910.5z"/>
|
||||
<path class="st1" d="M1625.2,1071.1c-23.5-4.2-45.7-8.1-67.9-12.1c-7.9-1.4-15.8-2.4-23.8-2.5c-12.7-0.1-25.1-2.6-37.7-4
|
||||
c-4-0.4-5.8-1.6-5.6-6.1c0.5-14.6,0.6-29.3,0.6-43.9c0-4.5,1-6.5,6.1-6.5c39.9,0.2,79.9,0.2,119.8,0.1c5,0,6.1,2.1,6.1,6.6
|
||||
C1622.9,1025,1623,1047.3,1625.2,1071.1z"/>
|
||||
<path class="st1" d="M1049.3,1070.4c-1-5.9-0.1-11.8,0.3-17.7c0.9-16.3,1.4-32.6,1.4-48.9c0-7.3,0.2-7.5,6.8-7.5
|
||||
c39.3-0.1,78.5-0.1,117.8-0.1c7,0,7.1,0.2,7.2,7.2c0.2,14,0,27.9,0.5,41.9c0.2,5.2-1.5,6.6-6.6,7c-16.9,1.3-33.8,2.7-50.6,5.2
|
||||
c-23.3,3.4-46.7,6.8-69.7,12.3C1054.2,1070.3,1051.9,1071.4,1049.3,1070.4z"/>
|
||||
<path class="st1" d="M1049.8,910.2c22.1,5.7,44.8,8.7,67.4,12.5c16.4,2.8,33.1,3.3,49.6,5.2c16.4,1.9,16.4,1.5,16.3,17.4
|
||||
c-0.1,11-0.3,22-0.2,32.9c0.1,4.9-1,7.2-6.6,7.2c-39.9-0.2-79.9-0.2-119.8-0.1c-3.4,0-5.7-0.6-5.6-4.8c0.6-22.6-1.3-45.2-1.9-67.8
|
||||
C1049,912,1049.4,911.4,1049.8,910.2z"/>
|
||||
<path class="st1" d="M1260.5,985.4c-21.6,0-43.2,0-64.9,0c-8,0-8.2-0.1-8.1-8.4c0.2-13.6,0.6-27.3,0.8-40.9c0.1-4.2,1.4-5.6,6-5.1
|
||||
c41.7,4.1,83.5,6.2,125.5,6.2c2.3,0,4.7,0,7,0c7.5,0.2,7.5,0.2,7.6,7.9c0.1,10.3,0.1,20.6,0.1,30.9c0,9.3,0,9.4-9.1,9.4
|
||||
C1303.8,985.4,1282.2,985.4,1260.5,985.4z"/>
|
||||
<path class="st1" d="M1485,931.5c0.3,16.2,0.5,32.7,0.8,49.3c0.1,3.4-1.7,4.8-4.9,4.6c-2-0.1-4,0-6,0c-41.6,0-83.2,0-124.9,0
|
||||
c-2,0-4-0.1-6,0c-4,0.3-5.1-1.8-5-5.5c0.1-11.7,0-23.3,0.1-35c0.1-7.6,0.1-7.8,7.6-7.8c17.3,0.2,34.6-0.6,51.9-1.1
|
||||
c26.3-0.7,52.5-3.1,78.7-5.1C1479.6,930.9,1481.8,930,1485,931.5z"/>
|
||||
<path class="st1" d="M1485,1051.1c-35.4-3-69.9-5.3-104.5-6.4c-11.6-0.4-23.3-0.5-34.9-0.4c-4.4,0-6.7-0.8-6.5-6
|
||||
c0.4-12,0.3-24,0-36c-0.1-4.8,1.6-6.3,6.3-6.2c45,0.1,89.9,0.1,134.9,0c3.5,0,5.8,0.7,5.7,4.8
|
||||
C1485.5,1017.5,1485.3,1034.1,1485,1051.1z"/>
|
||||
<path class="st1" d="M1188.4,1050.6c-0.3-17.2-0.5-33.7-0.8-50.3c0-2.9,1.6-4.1,4.4-4.1c1.7,0,3.3,0,5,0c42.6,0,85.1,0,127.7,0
|
||||
c1.7,0,3.3,0.1,5,0c3.3-0.2,4.9,1.2,4.9,4.6c-0.1,13-0.1,25.9,0,38.9c0,3.6-1.7,4.7-4.9,4.5c-0.7,0-1.3,0-2,0
|
||||
C1281.6,1043.7,1235.9,1047.2,1188.4,1050.6z"/>
|
||||
<path class="st1" d="M2010.7,1230.1c-5.1-1.9-9-5.1-13.1-7.7c-9.9-6.1-19.6-12.6-29.8-18.3c-4.8-2.6-5.5-5.6-4.5-10.4
|
||||
c3.9-20,6.2-40.1,8.2-60.4c4-41.2,5.8-82.4,6.1-123.8c0-3,0.1-6,0-9c-0.1-3,1.1-3.8,4.1-4.1c16.5-1.5,16.7-1.4,16.8,14.7
|
||||
c0.4,66.4,3.9,132.5,10.7,198.6C2009.9,1216.3,2011.6,1222.7,2010.7,1230.1z"/>
|
||||
<path class="st1" d="M2011.2,752.5c-1.6,16.9-3.2,33.8-4.8,50.6c-3,30.8-4.6,61.7-6.1,92.6c-1.3,27.3-1.5,54.5-1.8,81.8
|
||||
c-0.1,7.1-0.2,7.3-7.1,7.3c-16.3,0-13.6,1.3-13.7-12.8c-0.5-54.3-3.2-108.4-10.6-162.2c-1.1-7.9-2.5-15.8-3.9-23.6
|
||||
c-0.7-3.6-0.1-6,3.5-8.2c13.4-8,26.6-16.4,39.8-24.7C2007.6,752.8,2008.5,751.7,2011.2,752.5z"/>
|
||||
<path class="st1" d="M662.1,1231.5c1.9-20.4,3.7-38.6,5.4-56.8c3-31.9,4.7-63.8,6-95.8c1-24.6,1.6-49.3,1.6-74
|
||||
c0-8.3,0.1-8.3,7.9-8.4c1.3,0,2.7,0,4,0c8.6,0.1,8.7,0.1,8.8,8.7c0.4,40,1.8,80,5.4,119.8c2,22.9,4.4,45.8,9,68.3
|
||||
c1.1,5.5,0.2,8.5-5.1,11c-12.7,6-23.9,14.5-35.5,22.2C667.7,1227.9,665.8,1229.1,662.1,1231.5z"/>
|
||||
<path class="st1" d="M662.9,752.1c2.6-0.9,3.9,1,5.4,2.1c12.4,8.4,24.6,17.1,38,23.9c3.3,1.7,5,3.6,4.2,7.7
|
||||
c-11.3,61.8-13.7,124.4-14.8,187c-0.2,13.5,2.2,12.1-12.3,12.1c-8.4,0-8.5-0.1-8.5-8.9c0.1-52.7-2.6-105.3-6.4-157.8
|
||||
c-1.5-20.6-4-41.1-6-61.7C662.5,755.1,661.9,753.4,662.9,752.1z"/>
|
||||
<path class="st1" d="M2044.3,728.6c0,3.4,0,6.3,0,9.2c0,78.9,0,157.9,0,236.8c0,1.7-0.1,3.3,0,5c0.4,3.9-1.3,5.4-5.2,5.3
|
||||
c-5.9-0.2-11.9,0-18.3,0c0-26.3,0.3-51.6,1.3-76.8c1.7-42.3,4.4-84.5,8.9-126.5c1.6-14.9,3.9-29.7,5.6-44.6
|
||||
C2037.2,732.4,2038.9,729.7,2044.3,728.6z"/>
|
||||
<path class="st1" d="M2044.3,1253c-5.6-1.2-7-4.1-7.7-8.6c-3.1-19.4-5.2-38.9-7-58.5c-5.6-59-8.7-118.2-8.7-177.6
|
||||
c0-13.6-1.4-11.7,11.5-11.8c13.3,0,11.9-1.5,11.9,11.5c0,78.6,0,157.2,0,235.8C2044.3,1246.6,2044.3,1249.3,2044.3,1253z"/>
|
||||
<path class="st1" d="M629.2,728.7c5.8,1.6,7.4,4.6,7.9,9c2.3,21.2,5.3,42.2,7.3,63.4c5.3,55.3,7.7,110.8,8.5,166.4
|
||||
c0,3.3,0.1,6.7,0,10c-0.1,7.5-0.1,7.5-7.7,7.7c-16,0.5-16,0.5-16-15.3c0-76.5,0-153.1,0-229.6C629.2,736.7,629.2,733,629.2,728.7z"
|
||||
/>
|
||||
<path class="st1" d="M629.2,1253.2c0-6,0-10.6,0-15.3c0-75.7,0-151.3,0-227c0-2,0-4,0-6c-0.1-9.2-0.3-8.7,8.7-8.6
|
||||
c15.1,0,15.2,0,15,15.6c-0.7,62.7-3.8,125.2-10.3,187.6c-1.5,14.9-3.9,29.7-5.5,44.6C636.6,1248.7,634.5,1251.1,629.2,1253.2z"/>
|
||||
<path class="st1" d="M2060,697.5c0-15.9,0-29.5,0-43c0-3.7,0.3-7.3-0.1-11c-2-19,6.4-34.8,15.3-50.5c1-1.7,2.3-3.2,3.6-4.7
|
||||
c5.4-6.2,8.1-6.2,13.4,0.4c6.3,7.8,10.3,17.1,13.7,26.4c3.2,8.7,5.9,17.6,8.7,26.5c1.3,4,1,7.1-2.4,10.7
|
||||
C2096.7,668.2,2079,681.6,2060,697.5z"/>
|
||||
<path class="st1" d="M613.2,1283.1c0,24.5,1.5,47.6-0.5,70.3c-1.4,15.4-9,29.8-19.7,41.7c-3.8,4.2-5.7,3.9-10-0.4
|
||||
c-4.3-4.3-7.2-9.6-9.8-15c-6.7-13.8-11.2-28.4-15.3-43.2c-0.8-3-1.3-5.6,1.5-8.1C576.3,1313,592.4,1296.7,613.2,1283.1z"/>
|
||||
<path class="st1" d="M2060,1284.4c19.3,15.5,36.8,29.3,52.5,45.2c2.6,2.6,3.7,5.1,2.6,8.8c-5.1,17.2-10,34.5-19.7,50
|
||||
c-7.7,12.2-12.4,12.7-19.8,0.6c-9.5-15.4-16.9-31.4-15.8-50.5C2060.7,1321.2,2060,1303.9,2060,1284.4z"/>
|
||||
<path class="st1" d="M611.9,698.2c-8.5-6.7-17.1-12.8-25-19.7c-9.1-7.8-17.5-16.4-26.3-24.5c-2.3-2.2-4-4.1-3-7.7
|
||||
c5.3-18.2,10.3-36.5,20.4-52.8c7.8-12.7,12.8-13,20.4-0.3c8.9,14.9,16.3,30.2,15.1,48.6c-1,15.9-0.2,31.9-0.3,47.9
|
||||
C613.2,692.2,614,694.9,611.9,698.2z"/>
|
||||
<path class="st1" d="M1788.2,864c4.1,27,4.9,53.5,6.1,80.1c0.5,11.3,0.2,22.6,0.7,33.9c0.2,5.2-1.6,7.4-6.7,7.3
|
||||
c-4.7-0.1-9.3-0.3-14,0c-4.6,0.3-5.5-2-5.4-6c0.2-26.3,1.3-52.5,3.2-78.8c0.5-7.6,1.2-15.2,2.1-22.8
|
||||
C1775.6,867.2,1775.7,867.2,1788.2,864z"/>
|
||||
<path class="st1" d="M885.3,1118.4c-5.1-39.1-6.6-77.6-6.6-116.3c0-4.2,1.3-5.9,5.7-5.8c25,0.7,20.1-4.1,19.8,19.4
|
||||
c-0.4,29.6-1.5,59.3-5,88.8C897.9,1115.4,898.1,1115.4,885.3,1118.4z"/>
|
||||
<path class="st1" d="M885.7,863.5c12.4,2.1,12.2,2.1,13.5,13.8c3.8,33,4.6,66.2,5.3,99.4c0.2,8.3,0,8.5-7.9,8.4
|
||||
c-5.6-0.1-12.6,2.4-16.3-1.2c-3.8-3.6-1.3-10.7-1.3-16.2c0.1-33.9,1.8-67.7,6.1-101.4C885.1,865.4,885.4,864.5,885.7,863.5z"/>
|
||||
<path class="st1" d="M1788.4,1117.5c-4,0.3-6.2-1.5-8.7-2.2c-3.2-0.9-4.6-2.9-4.9-6.5c-0.7-10.3-2.2-20.5-2.9-30.8
|
||||
c-1.5-24.6-2.8-49.2-2.8-73.9c0-7.5,0.1-7.8,7.7-7.6c5.9,0.2,13.3-2.7,17.2,1.6c3.4,3.7,1,10.9,0.9,16.5
|
||||
C1794.3,1048.8,1792.8,1083.1,1788.4,1117.5z"/>
|
||||
<path class="st1" d="M1481.1,1078.1c-10.7,0.3-20.5-1.5-30.3-2.6c-34.1-3.7-68.2-5.9-102.5-5.6c-1.3,0-2.7-0.1-4,0
|
||||
c-4.9,0.5-5.1-2.5-5.2-6.2c-0.1-4,0.7-6,5.4-6c20,0.1,39.9-0.2,59.9,1c22.9,1.4,45.8,3,68.7,4.7c7.9,0.6,7.8,0.9,8.2,8.7
|
||||
C1481.2,1073.7,1481.1,1075.4,1481.1,1078.1z"/>
|
||||
<path class="st1" d="M1480,903c2,3.3,0.7,6.3,1,9.2c0.4,4.3-1.6,5.6-5.8,5.9c-42.2,3.3-84.3,6.2-126.7,5.9c-1.3,0-2.7-0.1-4,0
|
||||
c-4.5,0.5-5.5-1.7-5.6-5.8c-0.1-4.5,0.9-6.6,6-6.5c29.7,0.7,59.2-1.5,88.8-3.9C1449.3,906.5,1464.7,904.6,1480,903z"/>
|
||||
<path class="st1" d="M1192.2,1078.2c1-14.7,0.9-15,13.5-15.8c40.5-2.9,80.9-5.4,121.6-4.6c7.3,0.1,7.4,0.5,7.3,7
|
||||
c-0.1,4-1.7,5.2-5.4,5.1c-6-0.2-12-0.1-18,0c-37.3,0.2-74.3,3.2-111.3,7.8C1197.6,1078,1195.3,1078,1192.2,1078.2z"/>
|
||||
<path class="st1" d="M1192.6,903.2c17.2,2.2,33.7,3.8,50.2,5c28.2,2.1,56.4,4.4,84.7,3.6c3.8-0.1,7.3-0.4,7.1,5.4
|
||||
c-0.3,6.3-0.2,6.8-6.6,6.8c-18.6,0-37.2,0.4-55.9-0.8c-19.2-1.2-38.5-2.1-57.8-3.2c-3-0.2-6-0.4-8.9-0.8
|
||||
C1192.8,917.6,1192.8,917.6,1192.6,903.2z"/>
|
||||
<path class="st1" d="M653.6,694.6c2.5,2.7,3.4,3.5,4.2,4.6c15.1,20.9,32.8,39.4,51.8,56.7c2.5,2.3,7.7,4.8,5,8.5
|
||||
c-2.7,3.9-6.2-1-8.9-2.4c-14.7-7.8-28.1-17.9-42.2-26.6c-3.1-1.9-4.3-4.4-4.8-8C657.4,717,655.5,706.9,653.6,694.6z"/>
|
||||
<path class="st1" d="M654,1284.6c0.9-5.5,2.7-11,2.6-16.4c-0.1-14.4,6.3-23.8,19.2-30.2c10.6-5.3,20.3-12.5,30.5-18.8
|
||||
c2.5-1.5,4.9-3.3,7.8-3.6c1.6,2.1,1.5,4,0.2,5.7c-1,1.3-2.4,2.3-3.6,3.4c-18.4,17.3-36.5,34.8-51.2,55.5c-1.1,1.6-2.6,2.9-4,4.3
|
||||
C655.3,1284.8,654.8,1284.6,654,1284.6z"/>
|
||||
<path class="st1" d="M2019.1,698.8c-1.4,9.8-3,19.6-4.2,29.5c-0.4,3-2.2,4.7-4.4,6.2c-15.3,9.9-30.7,19.7-46.1,29.5
|
||||
c-1.5,1-3.7,4-5.7,0.4c-1.5-2.6,0.2-4.4,2.1-5.9c11.9-9.2,21.5-20.7,32.4-30.8c8.4-7.7,14.5-17.2,21.4-26.1c0.9-1.2,2-2.4,3-3.6
|
||||
C2018.1,698.2,2018.6,698.5,2019.1,698.8z"/>
|
||||
<path class="st1" d="M2017.1,1283.9c-11.8-19.6-28.7-34.6-44.6-50.5c-3.7-3.7-7.8-7.2-11.8-10.6c-1.9-1.7-3.4-3.4-1.7-5.9
|
||||
c1.7-2.4,3.5-0.5,5,0.5c15.7,10,31.3,20,46.9,30c2,1.3,3.5,2.9,3.9,5.4c1.4,10.1,2.9,20.2,4.4,30.4
|
||||
C2018.5,1283.4,2017.8,1283.7,2017.1,1283.9z"/>
|
||||
<path class="st1" d="M1044.6,1034c-0.6,11-1.1,21.9-1.7,32.9c-0.1,1.6-0.3,3.3-0.8,4.9c-0.3,0.8-1.4,1.6-2.2,1.7
|
||||
c-0.9,0.1-1.9-0.5-2.1-1.7c-0.1-1-0.3-2-0.4-3c-1.2-22.2-2.3-44.5-2.4-66.8c0-3.2,0-6,4.6-5.9c3.5,0.1,5.6,0.7,5.5,4.9
|
||||
c-0.2,11-0.1,21.9-0.1,32.9C1044.9,1034,1044.8,1034,1044.6,1034z"/>
|
||||
<path class="st1" d="M1636.4,1033.8c0,10.6,0,21.2,0,31.8c0,2-0.3,4-0.7,5.9c-0.2,0.8-1.2,1.8-1.9,1.8c-0.8,0-1.9-0.8-2.2-1.5
|
||||
c-0.5-1.2-0.5-2.6-0.6-3.9c-1.9-21.8-2.2-43.7-2.1-65.6c0-5.7,1.3-6.7,6-6.1c3.9,0.5,3.6,3.1,3.6,5.8c0,10.6,0,21.2,0,31.8
|
||||
C1637.8,1033.8,1637.1,1033.8,1636.4,1033.8z"/>
|
||||
<path class="st1" d="M1638.5,948.1c0,10.3,0,20.6,0,31c0,2.6,0.5,5.4-3.3,6.2c-4.4,0.9-6.4-0.5-6.4-5.8c0-19,0-38,1.4-56.9
|
||||
c0.2-3,0.4-6,0.7-9c0.2-2.2-0.3-5.4,3-5.2c2.5,0.1,2.2,3,2.3,4.9c0.6,11.6,1.1,23.3,1.6,34.9C1638,948.1,1638.2,948.1,1638.5,948.1
|
||||
z"/>
|
||||
<path class="st1" d="M1045.1,948c0,10.6-0.1,21.2,0,31.9c0,3.6-0.9,5.5-5,5.5c-3.9-0.1-5.1-1.7-5.1-5.4c0.1-22.6,1.2-45.1,2.5-67.7
|
||||
c0.1-1.6-0.1-3.7,2-4c2.8-0.4,3.1,1.9,3.2,3.9c0.7,11.9,1.3,23.9,1.9,35.8C1044.8,948,1044.9,948,1045.1,948z"/>
|
||||
<path class="st1" d="M741.9,740.1c7,10.8,9.7,22,12.6,34.5c-8.1-3.6-13.5-8.9-19.7-13c-2-1.3-2-3.3-1.2-5.3
|
||||
C735.6,751.1,737,745.6,741.9,740.1z"/>
|
||||
<path class="st1" d="M1931.7,1241.3c-6.9-10.6-9.7-21.7-12.1-33.3c4.2,0.4,6.4,3.1,9.1,4.9
|
||||
C1943.2,1223.1,1943.4,1224.7,1931.7,1241.3z"/>
|
||||
<path class="st1" d="M1931.6,740c12,17.2,11.9,18.3-3.6,29.1c-2.4,1.7-4.5,3.9-8.5,4.8C1921.8,761.9,1925,750.9,1931.6,740z"/>
|
||||
<path class="st1" d="M742.1,1241.3c-5.3-5.2-6.5-11.1-8.6-16.5c-0.8-2-0.5-3.8,1.6-5.1c6-3.9,11.2-9.1,19.2-12.6
|
||||
C751.8,1219.5,748.7,1230.3,742.1,1241.3z"/>
|
||||
<path class="st1" d="M599.6,517.5c0.6,0.1,1.4,0.1,1.9,0.4c12.6,7.1,13,7.9,10.7,24.4c-5.9-7.9-11-15.2-13.4-24.1
|
||||
C599,518,599.3,517.7,599.6,517.5z"/>
|
||||
<path class="st1" d="M597.6,1464.4c4-9.4,8.1-17.2,14-24.4C614.9,1455.5,614.5,1457.9,597.6,1464.4z"/>
|
||||
<path class="st1" d="M2072.4,516.4c2.2,0.6,2.3,2,1.3,3.7c-3.8,6.1-7.7,12.1-11.9,18.6c-2.8-10.6-2.1-12.7,5.3-17.8
|
||||
c1.9-1.3,4.2-2,5.3-4.3L2072.4,516.4z"/>
|
||||
<path class="st1" d="M2061.5,1442.5c4.7,8.5,10.5,14.5,12.5,22.8C2067.2,1459.8,2056.4,1456.8,2061.5,1442.5z"/>
|
||||
<path class="st1" d="M1781.6,836.9c4.2,9.3,4.2,9.3-1.8,12.6C1777.9,845.4,1780.6,842.1,1781.6,836.9z"/>
|
||||
<path class="st1" d="M895.4,849.3c-7.5-2.9-7.5-2.9-3.8-11.1C893.8,841.3,893.9,844.7,895.4,849.3z"/>
|
||||
<path class="st1" d="M630.1,1284.6c-0.2-5.4-2.7-11,1.8-17C634.2,1274.4,629.7,1279.3,630.1,1284.6z"/>
|
||||
<path class="st1" d="M629.9,697.1c0.5,5.4,3.9,10.3,2.4,16.6C626.9,708.5,630.1,702.5,629.9,697.1z"/>
|
||||
<path class="st1" d="M2044.5,1284.5c-2.2-5-4.3-9.2-2.8-15C2046.1,1274.3,2043.5,1279.4,2044.5,1284.5z"/>
|
||||
<path class="st1" d="M2041.9,712.1c-2.2-6,1.1-10.1,2.2-14.6C2043.5,702.2,2046,707.4,2041.9,712.1z"/>
|
||||
<path class="st1" d="M892.2,1143.9c-4.6-7.9-4.6-7.9,1.6-11.5C895.7,1136.5,892.6,1139.7,892.2,1143.9z"/>
|
||||
<path class="st1" d="M1779.6,1132.2c6.1,3.1,6.1,3.1,2.7,11.1c-1.9-1-1.7-3.1-2.3-4.6C1779.2,1137,1778.6,1135.1,1779.6,1132.2z"/>
|
||||
<path class="st1" d="M1934.8,1202.5c4.1-0.6,6.5,2.5,11.4,4.9c-6.3,0.5-8.2-3.5-11.3-5.1L1934.8,1202.5z"/>
|
||||
<path class="st1" d="M1935.8,777c3.1,0.9,4.5-3.5,7.8-2.9c-1.8,4-4.7,3.9-7.9,2.8L1935.8,777z"/>
|
||||
<path class="st1" d="M729.5,774c2.7-0.5,4.4,0.6,5.8,3.2C732.1,778,731,775.7,729.5,774z"/>
|
||||
<path class="st1" d="M734,1202.5c2.3,4.6-1.2,4.8-4.2,5.9c-0.6-3.7,3.7-3.3,4.2-5.7C734,1202.7,734,1202.5,734,1202.5z"/>
|
||||
<path class="st1" d="M1935.2,780.3c-0.4,0.2-0.8,0.5-1.2,0.7c0.3,0,0.6,0.1,0.8,0c0.2-0.1,0.4-0.3,0.6-0.5
|
||||
C1935.5,780.5,1935.2,780.3,1935.2,780.3z"/>
|
||||
<path class="st1" d="M740,1203.1c-1.9-0.8-4.2,2.9-6-0.5c0,0,0,0.2,0,0.1C736,1202.8,738,1202.9,740,1203.1L740,1203.1z"/>
|
||||
<path class="st1" d="M1935.7,777c1.4,1.3,0.9,2.4-0.5,3.3c0,0,0.3,0.3,0.3,0.3c0.1-1.2,0.3-2.3,0.4-3.5
|
||||
C1935.8,777,1935.7,777,1935.7,777z"/>
|
||||
<path class="st1" d="M737.9,778.4c0.2,0.2,0.4,0.5,0.6,0.7c-0.2,0.2-0.5,0.5-0.7,0.5c-0.2,0-0.4-0.4-0.6-0.6
|
||||
C737.4,778.8,737.7,778.6,737.9,778.4z"/>
|
||||
<path class="st1" d="M740,1203.1c0-0.3,0-0.8-0.2-1c-0.8-0.8-0.7-1.1,0.4-0.8C739.5,1201.9,739.4,1202.4,740,1203.1
|
||||
C740,1203.1,740,1203.1,740,1203.1z"/>
|
||||
<path class="st1" d="M1934.9,1202.3c0-0.2-0.1-0.3-0.1-0.5c0,0-0.1,0.1-0.1,0.1c0.1,0.2,0.1,0.4,0.2,0.6
|
||||
C1934.8,1202.5,1934.9,1202.3,1934.9,1202.3z"/>
|
||||
<path class="st1" d="M2072.4,516.6c0.7,0.4,1.4,0.8,2.1,1.2c-0.2,0.2-0.4,0.7-0.6,0.7c-1.1-0.2-1.5-1-1.4-2
|
||||
C2072.4,516.4,2072.4,516.6,2072.4,516.6z"/>
|
||||
<path class="st1" d="M598.7,518.3c-1-0.2-1.8-0.8-1-1.6c0.8-0.9,1.4,0,1.8,0.8C599.3,517.7,599,518,598.7,518.3z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 48 KiB |
|
|
@ -1,458 +0,0 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 24.2.1, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
|
||||
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
viewBox="0 0 2670 1980" style="enable-background:new 0 0 2670 1980;" xml:space="preserve">
|
||||
<style type="text/css">
|
||||
.st0{fill:#FCFCFB;}
|
||||
.st1{fill:none;}
|
||||
</style>
|
||||
<g>
|
||||
<path class="st0" d="M2275.3,1000.1c-0.1,2.3,0,4.7,0,7c0,189.3,0,378.7,0,568c0,6,0.1,12.1-0.8,17.9c-2.4,15.2-12.9,17.4-23.6,10
|
||||
c-45.4-31-91.2-61.4-136.9-91.9l-30.1-20.3c-9.8-6.3-19.2-13.2-29.2-19.3c-6.7-4.1-9.8-9.7-9.9-17.2c-0.1-9.3-0.6-18.7-0.3-28
|
||||
c0.2-7.3-1.7-13-6.5-19c-18.6-23.3-36.2-47.3-56.9-69c-18-18.8-36.7-36.8-56.4-53.8c-31.8-27.5-65.4-52.7-101-74.9
|
||||
c-32.7-20.4-66.8-38.4-102.2-54c-50.7-22.4-103.2-39.2-157.1-51.5c-36.7-8.4-73.8-14.4-111.2-18.5c-34.5-3.8-69-6.5-103.7-5.8
|
||||
c-23.1,0.5-46.2-1.3-69.3,0.4c-22.9,1.8-45.9,3-68.7,5.6c-72.7,8.4-143.7,24.6-212.1,50.6c-65.5,24.8-127.4,57-184.9,97.4
|
||||
c-46.8,32.8-90.1,69.8-129.5,111.1c-18.1,19-35.1,39.2-50.5,60.6c-3.9,5.5-5.5,11-5.3,17.6c0.4,9.3,0,18.7,0,28
|
||||
c0,9.3-3.1,16.4-11.8,21.4c-10.4,5.9-19.9,13.4-30.1,19.8l-25.4,17.2c-11.6,7.8-23.3,15.5-34.9,23.3c-34.3,23-68.6,46-102.9,69
|
||||
c-14.6,9.7-23.6,5.5-25.2-12.1c-0.5-5.3-0.5-10.7-0.5-16c0-189.7,0-379.3,0-569l-0.1-26.1c0.2-190.3,0.1-380.7,0.2-571
|
||||
c0-5.7,0.4-11.3,0.5-17c0.1-3.7,1.2-7.3,3-10.5c3.4-5.9,9.3-7.7,15.4-4.8c2.7,1.3,5.1,3.1,7.6,4.8c44.3,29.6,88.5,59.3,132.8,88.9
|
||||
l29.7,20.1c10.8,6.6,20.9,14.3,31.7,20.9c6.7,4.1,9.7,9.8,9.8,17.3c0.2,11,0.4,22,0.2,33c-0.1,5.2,1.2,9.6,4.2,13.9
|
||||
c15.4,22.2,33,42.8,51.6,62.2c53.9,56.4,114.1,105.1,181.4,144.6c38.4,22.6,78.3,42.4,119.9,58.6c40,15.6,80.9,28.3,122.8,38
|
||||
c52.7,12.1,106,19.3,160,22.6c18.3,1.1,36.6,2.1,54.9,1.5c17.7-0.6,35.5,1.1,53.2-0.1c17.6-1.1,35.2-2.3,52.8-3.6
|
||||
c35.9-2.5,71.4-8.5,106.7-15.1c53.4-9.9,105.5-25,156-45.1c53.8-21.5,105.1-48.2,153.2-80.9c43.4-29.5,83.6-62.7,121.1-99.3
|
||||
c27.3-26.6,50.3-56.8,73.9-86.6c2.9-3.6,3.6-7.6,3.6-12.1c-0.1-10.7,0.2-21.3,0.4-32c0.1-7.5,3.2-13.1,9.9-17.2
|
||||
c10.2-6.2,19.9-13.4,30-19.8l23.2-16.3c4.2-1.3,7.4-4.3,11-6.7c42.9-28.7,85.8-57.4,128.7-86.1c1.1-0.7,2.3-1.4,3.3-2.3
|
||||
c10.9-9.1,21.4-4.7,23.8,10.7c0.8,5.6,0.6,11.3,0.6,17c0,189.3,0,378.7,0,568L2275.3,1000.1z M740,1203.1c-2-0.2-4-0.3-6-0.5
|
||||
c-0.5,2.5-4.8,2-4.2,5.8c2.9-1,6.4-1.2,4.2-5.8C735.8,1206,738.1,1202.3,740,1203.1c-0.6-0.7-0.5-1.3,0.2-1.7
|
||||
c-1.1-0.3-1.1-0.1-0.4,0.8C740,1202.3,739.9,1202.8,740,1203.1z M1935.5,780.5c-0.2,0.2-0.4,0.4-0.6,0.4c-0.3,0.1-0.6,0-0.8,0
|
||||
c0.4-0.2,0.8-0.5,1.1-0.8c1.4-0.9,2-1.9,0.5-3.3c3.2,1.1,6.1,1.2,7.9-2.8c-3.3-0.7-4.6,3.8-7.7,2.9
|
||||
C1935.7,778.2,1935.6,779.4,1935.5,780.5z M1934.8,1202.5c-0.1-0.2-0.1-0.4-0.2-0.6c0,0,0.1-0.1,0.1-0.1c0,0.2,0.1,0.3,0.1,0.5
|
||||
c3.1,1.6,5,5.7,11.3,5.1C1941.3,1205,1938.9,1202,1934.8,1202.5z M2072.4,516.6c-1,2.3-3.4,3-5.3,4.3c-7.4,5.1-8.2,7.2-5.3,17.8
|
||||
c4.2-6.5,8-12.6,11.9-18.6c1.1-1.7,0.9-3.1-1.3-3.6c0,1,0.3,1.8,1.4,2c0.2,0,0.4-0.4,0.6-0.7C2073.8,517.4,2073.1,517,2072.4,516.6
|
||||
z M599.6,517.5c-0.4-0.8-1.1-1.7-1.8-0.8c-0.8,0.8,0.1,1.5,1,1.6c2.4,8.9,7.5,16.2,13.4,24.1c2.3-16.5,1.9-17.3-10.7-24.4
|
||||
C600.9,517.6,600.2,517.6,599.6,517.5z M534.9,1353.5c-20,21.8-37.3,44.1-53.4,67.4c-16.1,23.4-26.8,49.5-41.8,73.5
|
||||
c9-21,17.4-42.2,28.9-61.9c16.9-28.7,35.4-56.4,57.5-81.5c0.9-1,1.6-2.1,2.6-3.1c2.9-2.9,3.3-6,2.3-10
|
||||
c-9.1-33.8-14.6-68.3-19.2-103c-5-38-8.3-76.1-10.6-114.3c-2.5-39.9-3.4-79.9-3.9-119.8c-0.1-5.1-1.4-6.8-6.7-6.9
|
||||
c-21.6-0.2-43.3-0.8-64.9-1.3c-2,0-4-0.4-6,0.8c0,195.2,0,390.5,0,585.6c2.1,0.6,3.1-0.4,4.1-1.1c52.7-33.8,105.3-67.7,158-101.4
|
||||
c3.6-2.3,5.7-5.4,7.3-9.1c6-13,12.5-25.8,20.8-37.4c4.2-5.9,3.9-11.9,2.5-19.1c-2.4,1.8-4,2.9-5.5,4.2
|
||||
c-13.9,12.6-32.1,8.7-42.7-2.9c-7.3-8-12.4-17.2-17.1-26.8C542.3,1375.7,539.4,1365.1,534.9,1353.5z M534.6,627.2
|
||||
c3.6-5,4.3-10.7,6.3-15.9c6.2-15.8,12.6-31.5,25.2-43.7c10.8-10.5,26.8-12.6,38.9-2.8c2.3,1.9,4.3,5,7.9,5
|
||||
c1.6-8.3-0.2-15.3-5.1-21.9c-7.4-9.9-12.7-21-17.8-32.3c-2.4-5.3-5.8-9.2-10.8-12.4c-50.5-32.2-100.9-64.7-151.3-97.1
|
||||
c-2.4-1.5-4.5-3.9-8.7-3.9c0,3.7,0,7.2,0,10.7c0,187.9,0,375.8,0,563.7c0,2,0.2,4,0,6c-0.6,4.9,1.4,6.3,6.2,6.1
|
||||
c13.6-0.6,27.3-0.7,41-1c8.3-0.2,16.7-0.6,25-0.4c4.9,0.1,6.7-1.8,6.1-6.4c-0.1-1,0-2,0-3c1.2-75,4.3-149.8,13.7-224.3
|
||||
c4.6-36.3,10-72.5,19.6-107.9c1.6-5.7,0.4-9.9-3.4-14.4c-10.7-12.8-21.7-25.4-31-39.1c-19-28.1-36.8-57-49.6-88.7
|
||||
c-2.2-5.4-5.5-10.4-6.2-16.6C465,538.3,494.6,586,534.6,627.2z M2060.9,1411.1c-2.3,8.5-0.1,15.1,4,21.4
|
||||
c5.9,9.3,13.4,18.1,17.1,28.2c3.9,10.7,10.9,16.6,19.9,22.4c47.8,30.3,95.3,61.1,143,91.7c2.7,1.7,5,4.3,9.5,4.5c0-3.4,0-6.6,0-9.8
|
||||
c0-188.3,0-376.6,0-565c0-2-0.1-4,0-6c0.2-3.4-1.1-5-4.6-4.8c-2.3,0.1-4.7,0-7,0.1c-20,0.6-40,1.2-60,1.9c-6.7,0.2-6.7,0.4-7,7.4
|
||||
c0,1.3,0,2.7-0.1,4c-0.4,60-3,119.9-8.9,179.7c-5,50.1-11.3,99.9-24,148.7c-1.4,5.2,0.1,8.3,3.5,12.3c15,17.8,29.6,36.1,42.4,55.5
|
||||
c20,30.4,36.1,62.9,49.3,96.8c3.2,8.2,5.7,16.6,6.8,25.4c-33.9-90.8-82.6-145-107.2-169.7c-4.1,11-7.5,22.1-12.7,32.5
|
||||
c-4.9,9.9-10.3,19.5-18.8,26.8c-7.1,6.1-15.1,8.9-24.8,7.5C2073.2,1421.4,2068.3,1415.5,2060.9,1411.1z M2244.8,454.3
|
||||
c0.6,4.5-0.2,6.7-0.8,8.9c-6.5,24.2-17,47-28.2,69.2c-18.9,37.2-43,71.2-70.4,102.8c-2.7,3.2-3.6,5.8-2.5,9.8
|
||||
c7.9,29.6,13.1,59.8,17.3,90.2c6.4,45.6,10.2,91.3,12.8,137.3c2,35.3,2.6,70.6,3.1,105.9c0.1,7.6,0.2,7.6,7.6,7.8
|
||||
c17,0.5,34,1,51,1.4c5,0.1,10,0.2,15,0.5c3.2,0.1,5.1-0.9,4.9-4.5c-0.1-1.7,0-3.3,0-5c0-189.6,0-379.3-0.1-568.9
|
||||
c0-2.4,1.1-5.2-1.2-7.3c-0.8,0.1-1.6,0-2.1,0.4c-52.7,33.8-105.3,67.7-158,101.5c-2,1.3-3.7,2.6-4.7,4.9c-6.9,16-16.9,30.1-25.8,45
|
||||
c-3.1,5.2-3.4,10.2-2,15.7c2.6,0.1,3.5-1.6,4.6-2.6c15.7-13.9,33-10.6,45.6,3.8c6,6.8,10.4,14.6,14.4,22.7
|
||||
c5.1,10.3,8.3,21.3,12.8,32.9C2185.2,576.1,2221.7,520.5,2244.8,454.3z M553.7,662.8c-0.3,1.3-0.7,2.8-1,4.3
|
||||
c-7.1,31.2-11.7,62.8-15.4,94.5c-7.1,59.9-10.3,120.1-11.8,180.4c-0.3,13-0.2,26-0.5,39c-0.1,4.4,1.5,5.7,5.7,5.6
|
||||
c25-0.5,50-0.7,74.9-1.1c7.4-0.1,7.6-0.2,7.6-7.8c0-85.6,0-171.2,0.1-256.8c0-5.3-1.7-8.5-5.8-11.7c-16.6-12.8-32.4-26.6-46.9-41.9
|
||||
C559,665.6,557.6,663,553.7,662.8z M553.6,1319.7c2.1-1,2.8-1.2,3.2-1.6c16.4-17.1,34-32.9,52.7-47.4c3-2.4,3.8-5,3.8-8.6
|
||||
c-0.1-13.3-0.1-26.7-0.1-40c0-72.3,0-144.7,0-217c0-10.3,0.4-9-9.2-9.2c-23.3-0.4-46.7-0.8-70-1.1c-2.8,0-5.9-1-9.3,1.5
|
||||
C526.2,1104.4,531.1,1212.3,553.6,1319.7z M2119.8,1319.2c4.8-20.9,8.3-40.2,11.1-59.6c5.8-39.5,9.8-79.2,12.5-119.1
|
||||
c3.1-45.5,4.7-91.1,5.1-136.7c0.1-7.6-0.1-7.7-7.6-7.7c-24.3,0.1-48.6,0.3-72.9,0.5c-7.9,0.1-8,0.1-8,8.5c0,84.6,0,169.1-0.1,253.7
|
||||
c0,5.6,1.7,9.3,5.9,12.7c12.1,9.9,24.2,19.9,36,30.1C2107.7,1306.9,2113.1,1312.7,2119.8,1319.2z M2120.4,664.4
|
||||
c-2.1-0.6-2.9-0.1-3.6,0.7c-15.2,16.5-33.1,29.9-50.2,44.2c-4.8,4-6.8,8-6.8,14.3c0.3,32,0.2,64,0.2,96c0,52.3,0,104.6,0,157
|
||||
c0,8.1,0.1,8.2,8.2,8.2c18.3,0.2,36.7,0.2,55,0.4c6.7,0.1,13.3,0.1,20,0.3c3.7,0.1,5.7-1.1,5.4-5.1c-0.2-2.3,0-4.7-0.1-7
|
||||
c-0.5-45-2.1-89.9-5.2-134.8C2139.2,780.1,2132.7,721.9,2120.4,664.4z M878.7,1138.5c-36,15-69.8,32.9-102.3,53.1
|
||||
c-3.4,2.1-4.9,4.7-5.7,8.6c-2.8,14.4-5.7,28.8-12.2,42c-3.6,7.3-7.7,14.4-17.3,14.4c-9.6,0-13.4-7.4-17.4-14.4
|
||||
c-1.9-3.2-2-7.6-5.4-9.8c-22.3,17.4-41.6,37.2-58.8,58.9c-5.8,7.3-10.2,15.3-12,24.8c-3.8,20.3-9.1,40.2-16.7,59.5
|
||||
c-2.6,6.7-1.3,13.3-1.5,20.3c1.5-0.4,2-0.3,2.1-0.5c3.2-3.8,6.4-7.6,9.4-11.6c31.2-40.5,66-77.4,105.4-110.2
|
||||
c36.9-30.8,75.6-59.1,117.2-83.1c54.3-31.2,110.9-57.4,170.8-76.4c38.4-12.2,77.4-21.8,117-29.2c9.5-1.8,19.1-3,28.6-4.6
|
||||
c1.5-0.3,3.8,0.3,4.4-1.5c1.4-3.9,0.6-8-0.4-11.7c-1-3.6-4.5-1.7-6.9-1.5c-40.6,3.2-80.4,11-120.4,18c-7,1.2-10.4,4.1-10.9,11.2
|
||||
c-0.3,3.8-0.5,9.1-5.8,9c-4.6-0.1-4.7-5-5.3-8.5c-0.7-4.5-2.4-6-7.3-4.9c-39.3,9.1-78,19.9-115.5,34.7c-4.6,1.8-7.1,4.2-7.8,8.9
|
||||
c-0.9,5.9-2.2,11.8-5.2,17.1c-4.1,7.1-10.6,7.2-14.6,0C881.9,1147.7,880.7,1143.4,878.7,1138.5z M1032.8,891.9
|
||||
c0.9-3.8,1.6-6.7,2.4-9.5c0.7-2.3,1.7-4.6,4.4-4.9c3.4-0.3,4.4,2.3,5.3,4.9c0.4,1.3,0.8,2.6,0.8,3.9c0.1,7.7,4.5,10.4,11.7,11.6
|
||||
c32.5,5.5,64.8,11.9,97.5,15.7c7.9,0.9,15.9,1.7,23.8,2.4c1.9,0.2,4,0.7,5.3-1.6c3.5-6.5,0.3-13-7-14.2c-1-0.2-2-0.2-3-0.4
|
||||
c-42.6-5.8-84.2-15.9-125.4-28.2c-41.8-12.4-82.5-28.2-121.6-47.1c-51-24.6-99.8-53.3-144.7-87.9c-20.6-15.8-40.9-32.1-59.9-49.7
|
||||
c-29.1-26.9-56.4-55.7-80.1-87.7c-3.7-4.9-7.7-9.6-11.6-14.3c-0.8,7.6-2.4,14.7,0.5,21.8c7.4,18.2,12.4,37.2,16,56.5
|
||||
c1.7,9.1,5,17.3,10.6,24.4c9.2,11.4,18.5,22.8,28.7,33.3c9.9,10.1,21.1,19,32.2,28.9c3.8-6.5,5.6-13,9.8-18.2
|
||||
c7.7-9.4,17.7-9.5,25.3-0.1c3.6,4.5,5.7,9.7,7.8,15c4.3,10.9,6.8,22.3,8.9,33.7c0.9,5.2,3.3,8.1,7.5,10.7
|
||||
c21.9,13.4,44.1,26,67.3,37.1c10.9,5.2,22.1,10,33.5,15.2c1.8-4.8,3.1-8.6,4.7-12.2c1.6-3.5,4.2-6.3,8.3-6.1
|
||||
c3.6,0.2,5.8,2.9,7.3,6.1c2.4,5.2,3.8,10.6,4.8,16.2c0.8,4.6,2.9,7.2,7.7,9C950.6,871.5,990.8,883,1032.8,891.9z M1953.8,1233.7
|
||||
c-2.9,5.8-4.9,11.5-8.8,16.2c-7.6,9.3-17.8,9.3-25.3,0c-3.4-4.2-5.5-9-7.5-14c-4.4-11.5-7.2-23.5-9.4-35.7
|
||||
c-0.7-3.9-2.4-6.4-5.8-8.6c-30.4-19.4-62.4-35.6-95-50.8c-6.9-3.3-7-3-9.9,4.7c-0.7,1.9-1.5,3.7-2.3,5.5c-1.6,3.1-3.7,5.6-7.6,5.6
|
||||
c-3.8,0-5.9-2.5-7.6-5.6c-2.9-5.4-4.3-11.3-5.1-17.2c-0.7-5-3-7.3-7.5-9.1c-37.8-15.2-77.3-24.6-116.7-34.3c-3.8-0.9-5.4,0.2-6.2,4
|
||||
c-0.7,3.9-0.2,9.9-6.3,9.5c-5-0.3-5.1-6.1-5.3-9.6c-0.4-6.6-3.3-8.9-9.6-10c-32.8-5.6-65.4-12.2-98.6-15.7
|
||||
c-8.3-0.9-16.5-2.1-24.8-2.7c-5.4-0.4-7.1,1.2-7.5,6.6c-0.3,3.9-1.3,8,5.2,8.4c6.3,0.4,12.4,2.1,18.7,3.1
|
||||
c47.7,7.5,94.4,19.1,140.3,34.3c42.4,14,83.5,31.2,123,52c46.3,24.4,90.2,52.6,131,85.6c48.5,39.1,93.7,81.2,129.4,132.6
|
||||
c1.9,2.8,4.8,7.5,7.3,6.7c4.5-1.5,1.4-6.8,2.1-10.4c0.6-3.4-0.5-6.6-1.8-9.7c-7.8-19.5-12.6-40-17.1-60.4c-1.1-5.1-3-9.4-6-13.5
|
||||
c-17-22.8-36.7-43.1-57.6-62.2C1959.6,1237.1,1957.6,1234.7,1953.8,1233.7z M2043.6,585.1c-4.2,2.7-6.2,6.2-8.5,9.2
|
||||
c-29.9,40.3-64.6,76.1-102.5,109c-33.8,29.3-69.5,56.2-107.6,79.4c-51.8,31.6-106.3,57.7-163.9,77.3
|
||||
c-40.7,13.8-82.1,24.9-124.2,33.1c-13.7,2.7-27.6,4.7-41.3,7c-2.2,0.4-5.3-0.1-6.2,2.4c-1.4,3.8-1.1,8.2,0.3,11.7
|
||||
c1.4,3.5,5.3,1.6,8.1,1.3c40.1-4.2,79.8-10.7,119.5-18.1c6.2-1.2,10.4-2.7,10.5-10.2c0-3.5,0.3-9.3,5.4-9.5c6-0.3,5.5,5.7,6.2,9.6
|
||||
c0.7,3.9,2.3,4.9,6.1,3.9c10-2.7,20-5.1,30-7.5c28.5-6.9,56.4-15.6,83.9-25.7c6.8-2.5,10.2-6.3,11-13.3c0.5-5,2-9.8,4.5-14.2
|
||||
c4.2-7.6,10.8-7.6,15.1,0.1c1.6,2.9,2.5,6.2,3.4,9.3c0.9,2.9,2.5,3.5,5.2,2.2c2.4-1.2,5-1.9,7.3-3.1c26.6-13.1,53.2-26,78.7-41.2
|
||||
c11.9-7.1,19.6-14.7,21.3-29.3c1.4-12.4,5.5-24.8,12.8-35.4c8.1-11.7,19.5-11.7,27.5,0.1c3.1,4.6,5.4,9.7,8.2,14.9
|
||||
c1.9-1.4,3.6-2.5,5.1-3.8c22.2-19.8,42.7-41.1,60.6-64.9c2.6-3.5,4.3-7.3,5.2-11.7c4.5-21.2,9.6-42.2,17.6-62.4
|
||||
C2045.3,599.2,2044.5,592.6,2043.6,585.1z M777.2,1163.8c1.2,0.2,1.6,0.4,1.9,0.3c30.4-14.3,60.9-28.3,92.6-39.6
|
||||
c4.2-1.5,4.4-3.7,3.9-7.5c-4.5-37.4-6.3-75-6.5-112.6c0-7.8-0.2-7.9-8.5-7.9c-22,0-44,0-65.9,0c-8.6,0-8.8,0.1-8.8,8.7
|
||||
c-0.4,45.3-2.2,90.5-6.6,135.6C778.4,1148.4,776.3,1155.9,777.2,1163.8z M776.8,817.8c0.3,3.8,0.5,7.1,0.9,10.4
|
||||
c2.5,21.2,4.1,42.4,5.5,63.7c1.8,28.6,2.3,57.2,2.7,85.9c0.1,7.1,0.3,7.3,7.2,7.3c22.6,0.1,45.3,0.1,67.9,0.1
|
||||
c8.8,0,8.1-0.2,8.1-8.3c0.1-37,1.9-73.9,6.4-110.6c0.7-5.8-0.5-8.2-6-10.1c-29.5-10.3-58-23.3-86.2-36.8
|
||||
C781.5,818.4,779.9,817.2,776.8,817.8z M1896.4,1163.9c1-2.7,0.2-5.3-0.1-7.9c-2.4-19.8-3.9-39.8-5.3-59.7
|
||||
c-2.1-30.3-2.5-60.6-3.3-90.9c-0.2-8.7-0.2-8.8-8.6-8.8c-16.7-0.1-33.3,0-50-0.1c-6.3,0-12.7,0.2-19-0.1c-3.9-0.1-5.5,1.4-5.2,5.3
|
||||
c0.1,1.7,0.1,3.3,0,5c-1.1,36-2,72-6.4,107.7c-0.9,7.7-1.1,7.9,6.3,10.6c28.5,10.4,56.1,22.9,83.3,36.1
|
||||
C1890.6,1162.4,1893.1,1164.4,1896.4,1163.9z M1896.8,817.7c-2.5-0.3-4.3,0.6-6.1,1.5c-6,2.9-12,5.6-18,8.6
|
||||
c-22.1,10.9-44.9,20.1-67.9,28.7c-7.5,2.8-7.3,3-6.5,10.6c1,8.9,1.9,17.9,2.7,26.8c2.4,28.2,3.2,56.5,3.6,84.7
|
||||
c0.1,5.4,1.8,6.7,6.8,6.6c22.6-0.2,45.3-0.1,67.9-0.2c1,0,2-0.1,3,0c3.8,0.3,5.6-1.2,5.2-5.1c-0.2-1.6,0-3.3,0.1-5
|
||||
c0.7-38.6,1.7-77.2,5.2-115.6C1894.1,845.4,1895.5,831.6,1896.8,817.7z M908.3,1109.6c1.4,0,2.4,0.2,3.2,0
|
||||
c37.8-11.7,75.5-24,114.2-32.5c4.5-1,5.8-2.9,5.5-7.4c-1.3-22-2.5-43.9-2.3-65.9c0.1-7.3-0.2-7.4-8-7.4c-33.3,0-66.7,0.1-100,0.2
|
||||
c-7.7,0-7.9,0.1-7.9,7.5c0,31-1.6,61.9-4,92.9C908.8,1101,907.3,1105.2,908.3,1109.6z M1765.3,1110.4c0-2.5,0.1-3.5,0-4.4
|
||||
c-3.1-33.9-4.7-67.8-4.8-101.8c0-7.4-0.4-7.6-8.2-7.7c-33.3-0.1-66.7-0.2-100-0.2c-7.3,0-7.2,0.1-7.5,7.9c-0.7,21-1.1,42-2.3,62.9
|
||||
c-0.5,9-1.3,8.3,7.6,10.4c29.8,7.1,59.4,14.9,88.5,24.5C1747.2,1104.8,1755.7,1107.4,1765.3,1110.4z M908.8,872.2
|
||||
c-0.2,0.3-0.6,0.6-0.6,0.8c2.8,35.5,5.2,71,4.8,106.7c-0.1,4.6,1.9,5.6,6,5.6c34.7-0.1,69.3-0.1,104,0.1c4.9,0,6.2-1.8,6.1-6.4
|
||||
c-0.5-22,0.8-44,2.1-65.9c0.3-5.2-0.9-7.7-6.4-8.8c-36.9-7.6-72.4-20.1-108.5-30.5C913.8,873,911.6,871.1,908.8,872.2z
|
||||
M1765.6,872.4c-3.2-0.6-5.1,0.3-7,0.9c-22.2,6.7-44.2,14.5-66.7,20.3c-14.5,3.7-29,7.3-43.5,10.8c-4.4,1.1-6.5,2.6-6.2,8
|
||||
c1.3,21.9,1.8,43.9,2.6,65.8c0.2,7,0.2,7.1,7.1,7.1c33.6,0,67.2,0,100.8-0.1c7.5,0,7.7-0.2,7.7-7.8c0.2-24.6,0.9-49.2,2.6-73.8
|
||||
C1763.8,893.4,1764.7,883.1,1765.6,872.4z M725.8,1192.5c8.7-4.8,15.8-9.8,23.8-12.8c9.6-3.7,12.7-10.5,13-19.7
|
||||
c0.1-1.3,0.2-2.7,0.4-4c3.4-27.1,5.6-54.3,6.7-81.5c1-23.6,1.5-47.2,2-70.8c0.2-7-0.1-7-7.3-7.1c-6.7-0.1-13.3,0-20,0
|
||||
c-9,0-18,0.2-27,0c-4.2-0.1-6.2,1.3-5.9,5.7c0.2,2.3,0,4.7,0.1,7c0.8,27.6,1.2,55.2,3,82.8C717,1125.1,719.9,1158.2,725.8,1192.5z
|
||||
M1946.2,1191.7c2.9-2.2,2.4-4.7,2.8-6.9c4.5-26.6,6.9-53.5,8.9-80.4c2.5-33.9,3.2-67.9,3.7-101.8c0.1-5-1.8-6.2-6.4-6.1
|
||||
c-15.3,0.2-30.6,0-46,0.1c-7.6,0-7.8,0.2-7.7,7.8c0.7,44,2,87.9,6.7,131.6c1.1,10.3,2.2,20.5,3.5,30.7c0.4,2.8,0.2,6,3.5,7.8
|
||||
C1925.8,1180.2,1936.2,1186.1,1946.2,1191.7z M1947.7,789.1c-9.1,5-16.9,10.1-25.3,13.8c-8.2,3.6-10.5,9.5-11.5,17.8
|
||||
c-3,27.1-5.6,54.3-6.9,81.6c-1.2,25.3-1.5,50.6-2.5,75.9c-0.2,5.1,1.5,7,6.7,7c15-0.3,30-0.1,45-0.2c8.6,0,8.4,0.6,8.3-8.9
|
||||
c-0.3-30.6-1-61.3-3.1-91.9C1956.3,852.9,1953.6,821.8,1947.7,789.1z M726,789.2c-3.3,16.4-5.1,31.3-6.7,46.2
|
||||
c-4.6,42.4-6.3,85-7.3,127.7c-0.1,5.7,0,11.3-0.2,17c-0.1,3.4,1.4,4.8,4.7,4.8c17,0,34,0,51,0.1c3.6,0,4.6-1.9,4.4-5
|
||||
c-0.1-2.3-0.1-4.7-0.1-7c-0.5-37.3-1.7-74.6-5-111.8c-1.4-15.9-2.8-31.9-5.5-47.6c-0.4-2.6-0.3-5.2-3.2-6.8
|
||||
C747.6,801.1,737.2,795.4,726,789.2z M1625.1,910.5c-9.8,1.8-18,3.4-26.2,4.6c-23.3,3.5-46.2,9.9-70,10
|
||||
c-10.9,0.1-21.8,2.4-32.7,3.7c-3.5,0.4-6.2,1.1-6,6c0.5,15,0.7,30,0.7,44.9c0,4.4,1.7,5.8,5.9,5.7c8.3-0.2,16.6,0,25-0.1
|
||||
c30.6,0,61.3,0,91.9-0.1c9,0,9-0.1,9.2-9.4c0.2-13,0.2-26,0.6-38.9C1623.7,928.4,1624.5,919.8,1625.1,910.5z M1625.2,1071.1
|
||||
c-2.2-23.8-2.3-46.1-2.4-68.4c0-4.5-1-6.6-6.1-6.6c-39.9,0.1-79.9,0.1-119.8-0.1c-5.1,0-6.1,2-6.1,6.5c0,14.6-0.1,29.3-0.6,43.9
|
||||
c-0.1,4.5,1.6,5.6,5.6,6.1c12.6,1.4,25,3.8,37.7,4c8,0.1,15.9,1.1,23.8,2.5C1579.5,1063,1601.7,1066.9,1625.2,1071.1z
|
||||
M1049.3,1070.4c2.6,1,4.9-0.1,7-0.6c23-5.5,46.3-8.9,69.7-12.3c16.7-2.5,33.7-3.8,50.6-5.2c5.1-0.4,6.8-1.9,6.6-7
|
||||
c-0.4-14-0.3-27.9-0.5-41.9c-0.1-7-0.2-7.2-7.2-7.2c-39.3,0-78.5,0-117.8,0.1c-6.7,0-6.8,0.2-6.8,7.5c0,16.3-0.5,32.6-1.4,48.9
|
||||
C1049.3,1058.6,1048.4,1064.5,1049.3,1070.4z M1049.8,910.2c-0.4,1.2-0.8,1.8-0.8,2.4c0.6,22.6,2.5,45.2,1.9,67.8
|
||||
c-0.1,4.2,2.1,4.8,5.6,4.8c39.9,0,79.9-0.1,119.8,0.1c5.5,0,6.7-2.3,6.6-7.2c-0.2-11,0.1-22,0.2-32.9c0.1-15.9,0.1-15.5-16.3-17.4
|
||||
c-16.5-1.9-33.1-2.4-49.6-5.2C1094.7,918.9,1072,915.9,1049.8,910.2z M1260.5,985.4c21.6,0,43.2,0,64.9,0c9.1,0,9.1-0.1,9.1-9.4
|
||||
c0-10.3,0-20.6-0.1-30.9c-0.1-7.7-0.1-7.7-7.6-7.9c-2.3-0.1-4.7,0-7,0c-41.9,0-83.8-2.2-125.5-6.2c-4.6-0.4-5.9,0.9-6,5.1
|
||||
c-0.2,13.6-0.6,27.3-0.8,40.9c-0.1,8.3,0.1,8.4,8.1,8.4C1217.3,985.4,1238.9,985.4,1260.5,985.4z M1485,931.5
|
||||
c-3.1-1.4-5.4-0.6-7.7-0.4c-26.2,1.9-52.4,4.3-78.7,5.1c-17.3,0.5-34.6,1.3-51.9,1.1c-7.4-0.1-7.5,0.1-7.6,7.8
|
||||
c-0.1,11.7,0,23.3-0.1,35c0,3.7,1,5.8,5,5.5c2-0.1,4,0,6,0c41.6,0,83.2,0,124.9,0c2,0,4-0.1,6,0c3.3,0.1,5-1.2,4.9-4.6
|
||||
C1485.5,964.2,1485.3,947.6,1485,931.5z M1485,1051.1c0.3-17,0.5-33.6,0.9-50.1c0.1-4.2-2.2-4.8-5.7-4.8c-45,0.1-89.9,0.1-134.9,0
|
||||
c-4.7,0-6.5,1.4-6.3,6.2c0.3,12,0.4,24,0,36c-0.2,5.2,2.1,6,6.5,6c11.6-0.1,23.3,0.1,34.9,0.4
|
||||
C1415,1045.8,1449.5,1048.1,1485,1051.1z M1188.4,1050.6c47.5-3.4,93.2-6.8,139.2-6.4c0.7,0,1.3,0,2,0c3.2,0.2,4.9-0.9,4.9-4.5
|
||||
c-0.1-13,0-25.9,0-38.9c0-3.4-1.6-4.8-4.9-4.6c-1.7,0.1-3.3,0-5,0c-42.6,0-85.1,0-127.7,0c-1.7,0-3.3,0.1-5,0
|
||||
c-2.8-0.1-4.4,1.1-4.4,4.1C1187.9,1016.8,1188.1,1033.4,1188.4,1050.6z M2010.7,1230.1c0.9-7.3-0.9-13.8-1.5-20.3
|
||||
c-6.8-66-10.3-132.2-10.7-198.6c-0.1-16.1-0.3-16.3-16.8-14.7c-2.9,0.3-4.2,1.1-4.1,4.1c0.1,3,0,6,0,9
|
||||
c-0.3,41.3-2.1,82.6-6.1,123.8c-2,20.2-4.3,40.4-8.2,60.4c-0.9,4.8-0.2,7.8,4.5,10.4c10.2,5.7,19.9,12.2,29.8,18.3
|
||||
C2001.8,1225,2005.6,1228.2,2010.7,1230.1z M2011.2,752.5c-2.7-0.8-3.6,0.3-4.7,1c-13.3,8.3-26.4,16.7-39.8,24.7
|
||||
c-3.6,2.1-4.2,4.5-3.5,8.2c1.4,7.8,2.8,15.7,3.9,23.6c7.4,53.8,10.1,107.9,10.6,162.2c0.1,14.1-2.6,12.8,13.7,12.8
|
||||
c7,0,7-0.2,7.1-7.3c0.4-27.3,0.6-54.6,1.8-81.8c1.4-30.9,3.1-61.8,6.1-92.6C2008,786.3,2009.6,769.5,2011.2,752.5z M662.1,1231.5
|
||||
c3.7-2.4,5.6-3.7,7.5-4.9c11.6-7.7,22.9-16.2,35.5-22.2c5.3-2.5,6.2-5.5,5.1-11c-4.6-22.5-7-45.4-9-68.3
|
||||
c-3.5-39.9-5-79.8-5.4-119.8c-0.1-8.6-0.2-8.6-8.8-8.7c-1.3,0-2.7,0-4,0c-7.9,0.1-8,0.1-7.9,8.4c0,24.7-0.6,49.3-1.6,74
|
||||
c-1.3,32-3,63.9-6,95.8C665.8,1192.9,664.1,1211.1,662.1,1231.5z M662.9,752.1c-1,1.4-0.4,3-0.3,4.6c2,20.5,4.6,41.1,6,61.7
|
||||
c3.8,52.5,6.5,105.1,6.4,157.8c0,8.8,0.1,8.9,8.5,8.9c14.5-0.1,12.1,1.3,12.3-12.1c1.1-62.7,3.5-125.2,14.8-187
|
||||
c0.8-4.1-0.9-6-4.2-7.7c-13.4-6.8-25.6-15.5-38-23.9C666.8,753.1,665.5,751.1,662.9,752.1z M2044.3,728.6c-5.4,1.1-7.1,3.9-7.7,8.3
|
||||
c-1.7,14.9-4,29.7-5.6,44.6c-4.6,42.1-7.2,84.2-8.9,126.5c-1,25.3-1.3,50.5-1.3,76.8c6.5,0,12.4-0.1,18.3,0
|
||||
c3.8,0.1,5.5-1.4,5.2-5.3c-0.2-1.7,0-3.3,0-5c0-78.9,0-157.9,0-236.8C2044.3,734.9,2044.3,732,2044.3,728.6z M2044.3,1253
|
||||
c0-3.7,0-6.4,0-9c0-78.6,0-157.2,0-235.8c0-13,1.3-11.5-11.9-11.5c-13,0-11.5-1.8-11.5,11.8c0,59.3,3.1,118.5,8.7,177.6
|
||||
c1.9,19.5,4,39.1,7,58.5C2037.3,1248.9,2038.8,1251.8,2044.3,1253z M629.2,728.7c0,4.3,0,8,0,11.6c0,76.5,0,153.1,0,229.6
|
||||
c0,15.8,0,15.8,16,15.3c7.6-0.2,7.6-0.2,7.7-7.7c0.1-3.3,0-6.7,0-10c-0.8-55.6-3.1-111.1-8.5-166.4c-2-21.2-5-42.3-7.3-63.4
|
||||
C636.6,733.3,635,730.3,629.2,728.7z M629.2,1253.2c5.3-2.1,7.4-4.5,7.9-9.2c1.6-14.9,4-29.7,5.5-44.6
|
||||
c6.5-62.4,9.5-124.9,10.3-187.6c0.2-15.6,0.1-15.6-15-15.6c-9,0-8.8-0.6-8.7,8.6c0,2,0,4,0,6c0,75.7,0,151.3,0,227
|
||||
C629.2,1242.5,629.2,1247.2,629.2,1253.2z M2060,697.5c19-15.9,36.7-29.3,52.3-45.3c3.5-3.5,3.7-6.7,2.4-10.7
|
||||
c-2.8-8.9-5.4-17.8-8.7-26.5c-3.5-9.3-7.4-18.5-13.7-26.4c-5.3-6.6-8-6.6-13.4-0.4c-1.3,1.5-2.7,3-3.6,4.7
|
||||
c-8.9,15.7-17.3,31.5-15.3,50.5c0.4,3.6,0.1,7.3,0.1,11C2060,668,2060,681.5,2060,697.5z M613.2,1283.1
|
||||
c-20.8,13.6-36.9,29.9-53.8,45.4c-2.8,2.6-2.3,5.1-1.5,8.1c4.1,14.7,8.6,29.3,15.3,43.2c2.6,5.4,5.4,10.7,9.8,15
|
||||
c4.3,4.2,6.2,4.6,10,0.4c10.7-11.9,18.4-26.3,19.7-41.7C614.7,1330.6,613.2,1307.6,613.2,1283.1z M2060,1284.4
|
||||
c0,19.5,0.7,36.8-0.2,54c-1.1,19.1,6.3,35.2,15.8,50.5c7.4,12.1,12.1,11.7,19.8-0.6c9.7-15.4,14.6-32.7,19.7-50
|
||||
c1.1-3.7,0-6.2-2.6-8.8C2096.8,1313.6,2079.3,1299.9,2060,1284.4z M611.9,698.2c2.1-3.3,1.3-5.9,1.3-8.4c0.1-16-0.8-32,0.3-47.9
|
||||
c1.2-18.4-6.2-33.8-15.1-48.6c-7.6-12.7-12.6-12.4-20.4,0.3c-10.1,16.3-15.1,34.6-20.4,52.8c-1,3.6,0.6,5.5,3,7.7
|
||||
c8.8,8.1,17.3,16.7,26.3,24.5C594.8,685.4,603.3,691.5,611.9,698.2z M1788.2,864c-12.5,3.2-12.6,3.2-13.9,13.8
|
||||
c-0.9,7.6-1.6,15.2-2.1,22.8c-1.8,26.2-2.9,52.5-3.2,78.8c0,4,0.8,6.3,5.4,6c4.6-0.3,9.3-0.2,14,0c5.1,0.1,6.9-2.1,6.7-7.3
|
||||
c-0.5-11.3-0.2-22.6-0.7-33.9C1793.2,917.5,1792.3,890.9,1788.2,864z M885.3,1118.4c12.8-3,12.5-3,13.8-13.9
|
||||
c3.5-29.5,4.6-59.1,5-88.8c0.3-23.5,5.2-18.7-19.8-19.4c-4.4-0.1-5.7,1.5-5.7,5.8C878.8,1040.8,880.3,1079.4,885.3,1118.4z
|
||||
M885.7,863.5c-0.2,0.9-0.6,1.9-0.7,2.9c-4.3,33.7-6,67.5-6.1,101.4c0,5.5-2.5,12.6,1.3,16.2c3.8,3.6,10.8,1.1,16.3,1.2
|
||||
c7.9,0.1,8.1-0.1,7.9-8.4c-0.6-33.2-1.5-66.4-5.3-99.4C897.9,865.6,898,865.6,885.7,863.5z M1788.4,1117.5
|
||||
c4.4-34.4,5.9-68.6,6.5-102.9c0.1-5.6,2.4-12.7-0.9-16.5c-3.9-4.3-11.3-1.4-17.2-1.6c-7.6-0.2-7.7,0.1-7.7,7.6
|
||||
c0,24.7,1.3,49.3,2.8,73.9c0.6,10.3,2.2,20.5,2.9,30.8c0.3,3.6,1.7,5.5,4.9,6.5C1782.2,1116,1784.4,1117.7,1788.4,1117.5z
|
||||
M1481.1,1078.1c0-2.7,0.1-4.4,0-6c-0.3-7.8-0.3-8.1-8.2-8.7c-22.9-1.7-45.8-3.3-68.7-4.7c-19.9-1.2-39.9-0.9-59.9-1
|
||||
c-4.6,0-5.5,2-5.4,6c0.1,3.7,0.3,6.7,5.2,6.2c1.3-0.1,2.7,0,4,0c34.3-0.3,68.4,1.9,102.5,5.6
|
||||
C1460.6,1076.6,1470.4,1078.5,1481.1,1078.1z M1480,903c-15.3,1.6-30.8,3.6-46.3,4.8c-29.5,2.4-59.1,4.7-88.8,3.9
|
||||
c-5.1-0.1-6.1,2-6,6.5c0.1,4.1,1.1,6.3,5.6,5.8c1.3-0.2,2.7,0,4,0c42.3,0.3,84.5-2.6,126.7-5.9c4.2-0.3,6.2-1.6,5.8-5.9
|
||||
C1480.7,909.3,1482,906.3,1480,903z M1192.2,1078.2c3.1-0.2,5.4-0.3,7.6-0.5c37-4.6,74-7.6,111.3-7.8c6,0,12-0.1,18,0
|
||||
c3.7,0.1,5.3-1.1,5.4-5.1c0.1-6.5,0-6.9-7.3-7c-40.6-0.8-81.1,1.7-121.6,4.6C1193.1,1063.3,1193.2,1063.6,1192.2,1078.2z
|
||||
M1192.6,903.2c0.2,14.4,0.2,14.4,12.8,16c3,0.4,6,0.6,8.9,0.8c19.3,1.1,38.5,2,57.8,3.2c18.6,1.2,37.2,0.8,55.9,0.8
|
||||
c6.4,0,6.3-0.5,6.6-6.8c0.2-5.8-3.3-5.5-7.1-5.4c-28.3,0.8-56.5-1.5-84.7-3.6C1226.3,907,1209.8,905.4,1192.6,903.2z M653.6,694.6
|
||||
c2,12.3,3.8,22.4,5.1,32.6c0.5,3.7,1.7,6.1,4.8,8c14.2,8.8,27.5,18.8,42.2,26.6c2.7,1.4,6.2,6.3,8.9,2.4c2.7-3.7-2.6-6.2-5-8.5
|
||||
c-19-17.3-36.7-35.8-51.8-56.7C657,698.1,656,697.3,653.6,694.6z M654,1284.6c0.8,0,1.3,0.1,1.5,0c1.4-1.4,2.9-2.7,4-4.3
|
||||
c14.6-20.8,32.8-38.3,51.2-55.5c1.2-1.1,2.6-2.1,3.6-3.4c1.4-1.7,1.5-3.6-0.2-5.7c-2.9,0.3-5.3,2.1-7.8,3.6
|
||||
c-10.2,6.3-19.8,13.5-30.5,18.8c-12.9,6.4-19.3,15.7-19.2,30.2C656.6,1273.7,654.9,1279.1,654,1284.6z M2019.1,698.8
|
||||
c-0.5-0.3-1-0.6-1.5-0.9c-1,1.2-2,2.4-3,3.6c-6.8,9-13,18.4-21.4,26.1c-11,10.1-20.5,21.7-32.4,30.8c-1.9,1.5-3.6,3.2-2.1,5.9
|
||||
c2,3.5,4.1,0.5,5.7-0.4c15.4-9.7,30.8-19.6,46.1-29.5c2.3-1.5,4-3.1,4.4-6.2C2016.2,718.5,2017.7,708.6,2019.1,698.8z
|
||||
M2017.1,1283.9c0.7-0.2,1.4-0.5,2.1-0.7c-1.5-10.1-3-20.2-4.4-30.4c-0.4-2.6-1.9-4.1-3.9-5.4c-15.6-10-31.3-20.1-46.9-30
|
||||
c-1.5-1-3.3-2.8-5-0.5c-1.7,2.4-0.3,4.2,1.7,5.9c4,3.4,8.1,6.9,11.8,10.6C1988.4,1249.3,2005.3,1264.3,2017.1,1283.9z M1044.6,1034
|
||||
c0.2,0,0.3,0,0.5,0c0-11-0.1-21.9,0.1-32.9c0.1-4.2-2-4.8-5.5-4.9c-4.5-0.1-4.6,2.6-4.6,5.9c0.1,22.3,1.2,44.5,2.4,66.8
|
||||
c0.1,1,0.2,2,0.4,3c0.2,1.2,1.2,1.8,2.1,1.7c0.8-0.1,1.9-1,2.2-1.7c0.5-1.5,0.7-3.2,0.8-4.9C1043.5,1055.9,1044,1045,1044.6,1034z
|
||||
M1636.4,1033.8c0.7,0,1.4,0,2.1,0c0-10.6,0-21.2,0-31.8c0-2.7,0.2-5.3-3.6-5.8c-4.7-0.6-6,0.4-6,6.1c-0.1,21.9,0.2,43.7,2.1,65.6
|
||||
c0.1,1.3,0.1,2.7,0.6,3.9c0.3,0.7,1.5,1.6,2.2,1.5c0.7,0,1.7-1.1,1.9-1.8c0.5-1.9,0.7-3.9,0.7-5.9
|
||||
C1636.4,1055,1636.4,1044.4,1636.4,1033.8z M1638.5,948.1c-0.2,0-0.5,0-0.7,0c-0.5-11.6-1-23.3-1.6-34.9c-0.1-1.9,0.2-4.8-2.3-4.9
|
||||
c-3.3-0.1-2.8,3-3,5.2c-0.3,3-0.4,6-0.7,9c-1.5,19-1.4,37.9-1.4,56.9c0,5.3,2,6.8,6.4,5.8c3.8-0.8,3.3-3.6,3.3-6.2
|
||||
C1638.5,968.8,1638.5,958.4,1638.5,948.1z M1045.1,948c-0.1,0-0.3,0-0.4,0c-0.6-11.9-1.2-23.9-1.9-35.8c-0.1-1.9-0.4-4.3-3.2-3.9
|
||||
c-2.2,0.3-1.9,2.4-2,4c-1.3,22.5-2.3,45.1-2.5,67.7c0,3.7,1.2,5.4,5.1,5.4c4.1,0.1,5-1.9,5-5.5C1045,969.3,1045.1,958.7,1045.1,948
|
||||
z M741.9,740.1c-4.9,5.5-6.3,11-8.3,16.2c-0.8,2-0.8,4,1.2,5.3c6.2,4.1,11.6,9.4,19.7,13C751.6,762.1,748.8,750.9,741.9,740.1z
|
||||
M1931.7,1241.3c11.7-16.6,11.5-18.2-3-28.3c-2.7-1.9-4.9-4.5-9.1-4.9C1921.9,1219.6,1924.7,1230.7,1931.7,1241.3z M1931.6,740
|
||||
c-6.6,10.9-9.8,22-12.1,33.9c4-0.9,6.1-3.1,8.5-4.8C1943.5,758.3,1943.6,757.2,1931.6,740z M742.1,1241.3
|
||||
c6.6-11.1,9.8-21.9,12.2-34.3c-8,3.6-13.2,8.8-19.2,12.6c-2,1.3-2.3,3.1-1.6,5.1C735.6,1230.2,736.7,1236.2,742.1,1241.3z
|
||||
M597.6,1464.4c16.9-6.5,17.3-8.9,14-24.4C605.8,1447.2,601.7,1455,597.6,1464.4z M2061.5,1442.5c-5.1,14.3,5.6,17.3,12.5,22.8
|
||||
C2072.1,1457,2066.2,1451,2061.5,1442.5z M1781.6,836.9c-1,5.2-3.6,8.5-1.8,12.6C1785.7,846.2,1785.7,846.2,1781.6,836.9z
|
||||
M895.4,849.3c-1.5-4.5-1.6-7.9-3.8-11.1C887.9,846.3,887.9,846.3,895.4,849.3z M630.1,1284.6c-0.4-5.3,4.1-10.2,1.8-17
|
||||
C627.4,1273.6,629.9,1279.2,630.1,1284.6z M629.9,697.1c0.2,5.4-3,11.4,2.4,16.6C633.8,707.4,630.4,702.5,629.9,697.1z
|
||||
M2044.5,1284.5c-1-5.2,1.5-10.3-2.8-15C2040.3,1275.4,2042.4,1279.6,2044.5,1284.5z M2041.9,712.1c4.1-4.8,1.6-10,2.2-14.6
|
||||
C2043,702.1,2039.8,706.2,2041.9,712.1z M892.2,1143.9c0.4-4.2,3.5-7.3,1.6-11.5C887.6,1136,887.6,1136,892.2,1143.9z
|
||||
M1779.6,1132.2c-1,2.9-0.4,4.8,0.4,6.5c0.7,1.5,0.4,3.6,2.3,4.6C1785.7,1135.3,1785.7,1135.3,1779.6,1132.2z M729.5,774
|
||||
c1.5,1.6,2.6,4,5.8,3.2C733.9,774.7,732.2,773.5,729.5,774z M737.9,778.4c-0.2,0.2-0.5,0.4-0.7,0.6c0.2,0.2,0.4,0.6,0.6,0.6
|
||||
c0.2,0,0.5-0.3,0.7-0.5C738.4,778.8,738.1,778.6,737.9,778.4z"/>
|
||||
<path class="st1" d="M534.9,1353.5c4.5,11.6,7.4,22.2,12.2,32c4.7,9.6,9.8,18.8,17.1,26.8c10.6,11.6,28.8,15.5,42.7,2.9
|
||||
c1.4-1.3,3.1-2.4,5.5-4.2c1.4,7.2,1.7,13.2-2.5,19.1c-8.4,11.7-14.8,24.4-20.8,37.4c-1.7,3.7-3.7,6.7-7.3,9.1
|
||||
c-52.7,33.7-105.3,67.6-158,101.4c-1.1,0.7-2.1,1.7-4.1,1.1c0-195.1,0-390.4,0-585.6c2.1-1.2,4.1-0.9,6-0.8
|
||||
c21.6,0.5,43.3,1.1,64.9,1.3c5.3,0,6.7,1.8,6.7,6.9c0.5,40,1.4,79.9,3.9,119.8c2.4,38.2,5.6,76.4,10.6,114.3
|
||||
c4.6,34.7,10.1,69.1,19.2,103c1.1,4,0.6,7.1-2.3,10c-0.9,0.9-1.7,2.1-2.6,3.1c-22.1,25.1-40.6,52.7-57.5,81.5
|
||||
c-11.6,19.7-19.9,41-28.9,61.9c14.9-23.9,25.7-50.1,41.8-73.5C497.6,1397.6,514.9,1375.3,534.9,1353.5z"/>
|
||||
<path class="st1" d="M534.6,627.2c-40-41.2-69.6-89-94.2-140.1c0.7,6.2,4,11.2,6.2,16.6c12.7,31.7,30.5,60.6,49.6,88.7
|
||||
c9.3,13.7,20.4,26.3,31,39.1c3.8,4.5,5,8.7,3.4,14.4c-9.6,35.4-15,71.6-19.6,107.9c-9.4,74.5-12.5,149.3-13.7,224.3c0,1-0.1,2,0,3
|
||||
c0.6,4.7-1.2,6.5-6.1,6.4c-8.3-0.2-16.7,0.2-25,0.4c-13.7,0.3-27.3,0.4-41,1c-4.8,0.2-6.8-1.2-6.2-6.1c0.3-2,0-4,0-6
|
||||
c0-187.9,0-375.8,0-563.7c0-3.5,0-7.1,0-10.7c4.2-0.1,6.3,2.3,8.7,3.9c50.4,32.4,100.8,64.9,151.3,97.1c5,3.2,8.4,7.1,10.8,12.4
|
||||
c5.1,11.2,10.4,22.4,17.8,32.3c4.9,6.6,6.7,13.5,5.1,21.9c-3.6-0.1-5.6-3.1-7.9-5c-12-9.8-28.1-7.7-38.9,2.8
|
||||
c-12.6,12.2-19,27.9-25.2,43.7C538.8,616.5,538.2,622.2,534.6,627.2z"/>
|
||||
<path class="st1" d="M2060.9,1411.1c7.4,4.3,12.3,10.3,20.6,11.4c9.8,1.4,17.7-1.4,24.8-7.5c8.5-7.3,13.9-16.9,18.8-26.8
|
||||
c5.1-10.4,8.6-21.5,12.7-32.5c24.7,24.7,73.3,78.9,107.2,169.7c-1.1-8.8-3.5-17.2-6.8-25.4c-13.2-33.9-29.3-66.4-49.3-96.8
|
||||
c-12.8-19.5-27.4-37.7-42.4-55.5c-3.4-4-4.8-7.1-3.5-12.3c12.8-48.8,19.1-98.6,24-148.7c5.9-59.7,8.4-119.6,8.9-179.7
|
||||
c0-1.3,0-2.7,0.1-4c0.2-7,0.2-7.1,7-7.4c20-0.7,40-1.3,60-1.9c2.3-0.1,4.7,0,7-0.1c3.5-0.2,4.8,1.4,4.6,4.8c-0.1,2,0,4,0,6
|
||||
c0,188.3,0,376.6,0,565c0,3.2,0,6.4,0,9.8c-4.5-0.2-6.9-2.8-9.5-4.5c-47.7-30.6-95.2-61.4-143-91.7c-9-5.7-16-11.6-19.9-22.4
|
||||
c-3.7-10.1-11.2-18.9-17.1-28.2C2060.8,1426.2,2058.6,1419.6,2060.9,1411.1z"/>
|
||||
<path class="st1" d="M2244.8,454.3c-23,66.2-59.6,121.8-106.6,172.4c-4.5-11.6-7.7-22.6-12.8-32.9c-4-8.1-8.4-15.8-14.4-22.7
|
||||
c-12.6-14.4-29.9-17.7-45.6-3.8c-1.1,1-2,2.7-4.6,2.6c-1.4-5.5-1.1-10.5,2-15.7c8.8-14.9,18.9-29,25.8-45c1-2.3,2.7-3.7,4.7-4.9
|
||||
c52.7-33.9,105.3-67.7,158-101.5c0.5-0.3,1.3-0.2,2.1-0.4c2.3,2.1,1.2,4.9,1.2,7.3c0.1,189.6,0.1,379.3,0.1,568.9
|
||||
c0,1.7-0.1,3.3,0,5c0.2,3.6-1.7,4.6-4.9,4.5c-5-0.2-10-0.3-15-0.5c-17-0.5-34-1-51-1.4c-7.4-0.2-7.5-0.2-7.6-7.8
|
||||
c-0.5-35.3-1.1-70.6-3.1-105.9c-2.6-45.9-6.4-91.7-12.8-137.3c-4.2-30.3-9.4-60.5-17.3-90.2c-1.1-4-0.2-6.6,2.5-9.8
|
||||
c27.4-31.6,51.5-65.6,70.4-102.8c11.3-22.2,21.7-44.9,28.2-69.2C2244.6,460.9,2245.4,458.8,2244.8,454.3z"/>
|
||||
<path class="st1" d="M553.7,662.8c3.8,0.2,5.2,2.8,6.9,4.6c14.5,15.2,30.3,29,46.9,41.9c4.1,3.2,5.8,6.4,5.8,11.7
|
||||
c-0.2,85.6-0.1,171.2-0.1,256.8c0,7.5-0.1,7.7-7.6,7.8c-25,0.4-50,0.6-74.9,1.1c-4.3,0.1-5.9-1.2-5.7-5.6c0.4-13,0.2-26,0.5-39
|
||||
c1.5-60.3,4.7-120.5,11.8-180.4c3.7-31.7,8.3-63.3,15.4-94.5C553,665.6,553.4,664.1,553.7,662.8z"/>
|
||||
<path class="st1" d="M553.6,1319.7c-22.4-107.4-27.4-215.3-28.7-323.3c3.3-2.5,6.4-1.5,9.3-1.5c23.3,0.3,46.7,0.7,70,1.1
|
||||
c9.5,0.1,9.2-1.2,9.2,9.2c0,72.3,0,144.7,0,217c0,13.3-0.1,26.7,0.1,40c0,3.6-0.7,6.2-3.8,8.6c-18.7,14.5-36.3,30.3-52.7,47.4
|
||||
C556.3,1318.5,555.6,1318.7,553.6,1319.7z"/>
|
||||
<path class="st1" d="M2119.8,1319.2c-6.7-6.5-12.1-12.3-18-17.5c-11.8-10.2-23.8-20.2-36-30.1c-4.2-3.4-5.9-7.1-5.9-12.7
|
||||
c0.2-84.6,0.1-169.1,0.1-253.7c0-8.4,0.1-8.5,8-8.5c24.3-0.2,48.6-0.4,72.9-0.5c7.5,0,7.7,0,7.6,7.7c-0.3,45.6-1.9,91.2-5.1,136.7
|
||||
c-2.7,39.9-6.7,79.6-12.5,119.1C2128.1,1279,2124.6,1298.3,2119.8,1319.2z"/>
|
||||
<path class="st1" d="M2120.4,664.4c12.3,57.5,18.7,115.7,22.8,174.2c3.1,44.9,4.7,89.8,5.2,134.8c0,2.3-0.1,4.7,0.1,7
|
||||
c0.3,4-1.7,5.2-5.4,5.1c-6.7-0.2-13.3-0.3-20-0.3c-18.3-0.1-36.7-0.2-55-0.4c-8-0.1-8.2-0.1-8.2-8.2c0-52.3,0-104.6,0-157
|
||||
c0-32,0.2-64-0.2-96c-0.1-6.3,2-10.3,6.8-14.3c17.1-14.3,35.1-27.7,50.2-44.2C2117.5,664.3,2118.3,663.8,2120.4,664.4z"/>
|
||||
<path class="st1" d="M878.7,1138.5c2,4.9,3.2,9.1,5.3,12.8c4,7.2,10.5,7.1,14.6,0c3-5.3,4.3-11.2,5.2-17.1c0.8-4.7,3.2-7.1,7.8-8.9
|
||||
c37.6-14.8,76.3-25.6,115.5-34.7c5-1.1,6.6,0.4,7.3,4.9c0.6,3.5,0.7,8.4,5.3,8.5c5.3,0.1,5.5-5.2,5.8-9c0.5-7.1,3.9-10,10.9-11.2
|
||||
c40-7,79.8-14.9,120.4-18c2.4-0.2,5.9-2.1,6.9,1.5c1,3.7,1.8,7.8,0.4,11.7c-0.7,1.8-2.9,1.2-4.4,1.5c-9.5,1.6-19.1,2.8-28.6,4.6
|
||||
c-39.6,7.4-78.6,17-117,29.2c-59.9,19-116.5,45.2-170.8,76.4c-41.6,24-80.2,52.3-117.2,83.1c-39.4,32.8-74.3,69.6-105.4,110.2
|
||||
c-3,3.9-6.2,7.7-9.4,11.6c-0.1,0.2-0.6,0.2-2.1,0.5c0.2-7-1.1-13.6,1.5-20.3c7.6-19.2,12.9-39.2,16.7-59.5
|
||||
c1.8-9.6,6.2-17.5,12-24.8c17.2-21.7,36.5-41.5,58.8-58.9c3.4,2.2,3.5,6.6,5.4,9.8c4,7,7.8,14.4,17.4,14.4
|
||||
c9.6,0,13.8-7.2,17.3-14.4c6.5-13.3,9.4-27.7,12.2-42c0.8-3.9,2.2-6.5,5.7-8.6C808.9,1171.4,842.7,1153.5,878.7,1138.5z"/>
|
||||
<path class="st1" d="M1032.8,891.9c-42-9-82.2-20.4-121.3-35.7c-4.8-1.9-6.9-4.4-7.7-9c-0.9-5.6-2.3-11.1-4.8-16.2
|
||||
c-1.5-3.2-3.7-5.9-7.3-6.1c-4.1-0.2-6.8,2.5-8.3,6.1c-1.6,3.6-2.8,7.4-4.7,12.2c-11.4-5.2-22.6-10-33.5-15.2
|
||||
c-23.1-11.1-45.4-23.7-67.3-37.1c-4.3-2.6-6.6-5.5-7.5-10.7c-2.1-11.4-4.6-22.8-8.9-33.7c-2.1-5.3-4.2-10.5-7.8-15
|
||||
c-7.6-9.4-17.5-9.4-25.3,0.1c-4.2,5.2-6,11.7-9.8,18.2c-11.1-9.9-22.3-18.8-32.2-28.9c-10.2-10.4-19.5-21.8-28.7-33.3
|
||||
c-5.6-7-8.9-15.3-10.6-24.4c-3.7-19.3-8.6-38.3-16-56.5c-2.9-7.1-1.4-14.1-0.5-21.8c3.9,4.8,7.9,9.4,11.6,14.3
|
||||
c23.7,32,51,60.7,80.1,87.7c19,17.6,39.4,33.9,59.9,49.7c45,34.7,93.8,63.3,144.7,87.9c39.2,18.9,79.8,34.7,121.6,47.1
|
||||
c41.2,12.2,82.8,22.3,125.4,28.2c1,0.1,2,0.2,3,0.4c7.3,1.2,10.5,7.7,7,14.2c-1.3,2.3-3.4,1.7-5.3,1.6c-8-0.7-15.9-1.5-23.8-2.4
|
||||
c-32.8-3.7-65.1-10.2-97.5-15.7c-7.1-1.2-11.5-3.9-11.7-11.6c0-1.3-0.4-2.6-0.8-3.9c-0.8-2.6-1.9-5.3-5.3-4.9
|
||||
c-2.7,0.3-3.7,2.6-4.4,4.9C1034.4,885.3,1033.7,888.2,1032.8,891.9z"/>
|
||||
<path class="st1" d="M1953.8,1233.7c3.8,1,5.8,3.4,8,5.3c21,19.1,40.7,39.4,57.6,62.2c3,4.1,4.9,8.4,6,13.5
|
||||
c4.5,20.5,9.3,40.9,17.1,60.4c1.3,3.1,2.4,6.3,1.8,9.7c-0.6,3.6,2.4,8.9-2.1,10.4c-2.5,0.8-5.4-3.9-7.3-6.7
|
||||
c-35.7-51.5-81-93.6-129.4-132.6c-40.8-32.9-84.7-61.2-131-85.6c-39.5-20.8-80.6-37.9-123-52c-45.9-15.2-92.6-26.8-140.3-34.3
|
||||
c-6.2-1-12.4-2.7-18.7-3.1c-6.5-0.4-5.5-4.5-5.2-8.4c0.4-5.4,2.1-7.1,7.5-6.6c8.3,0.7,16.5,1.9,24.8,2.7
|
||||
c33.2,3.4,65.8,10.1,98.6,15.7c6.3,1.1,9.2,3.4,9.6,10c0.2,3.6,0.3,9.3,5.3,9.6c6,0.4,5.5-5.6,6.3-9.5c0.7-3.9,2.4-5,6.2-4
|
||||
c39.4,9.7,78.9,19.2,116.7,34.3c4.5,1.8,6.8,4.2,7.5,9.1c0.9,5.9,2.2,11.8,5.1,17.2c1.7,3.1,3.8,5.6,7.6,5.6c3.8,0,6-2.5,7.6-5.6
|
||||
c0.9-1.8,1.7-3.6,2.3-5.5c2.8-7.7,2.9-7.9,9.9-4.7c32.6,15.3,64.6,31.4,95,50.8c3.4,2.2,5.1,4.7,5.8,8.6
|
||||
c2.1,12.1,4.9,24.1,9.4,35.7c1.9,5,4.1,9.8,7.5,14c7.5,9.4,17.7,9.4,25.3,0C1948.9,1245.3,1950.9,1239.5,1953.8,1233.7z"/>
|
||||
<path class="st1" d="M2043.6,585.1c0.9,7.5,1.7,14.1-0.8,20.4c-8,20.2-13.1,41.2-17.6,62.4c-0.9,4.4-2.6,8.3-5.2,11.7
|
||||
c-17.8,23.8-38.4,45.2-60.6,64.9c-1.5,1.3-3.2,2.3-5.1,3.8c-2.9-5.3-5.1-10.4-8.2-14.9c-8-11.8-19.3-11.8-27.5-0.1
|
||||
c-7.4,10.6-11.4,23-12.8,35.4c-1.7,14.6-9.4,22.2-21.3,29.3c-25.5,15.2-52.1,28.1-78.7,41.2c-2.4,1.2-5,2-7.3,3.1
|
||||
c-2.7,1.3-4.3,0.7-5.2-2.2c-1-3.2-1.9-6.4-3.4-9.3c-4.2-7.7-10.8-7.7-15.1-0.1c-2.5,4.4-3.9,9.3-4.5,14.2c-0.8,7-4.2,10.7-11,13.3
|
||||
c-27.5,10.2-55.4,18.9-83.9,25.7c-10,2.4-20.1,4.8-30,7.5c-3.8,1-5.4,0-6.1-3.9c-0.7-4-0.2-9.9-6.2-9.6c-5.1,0.2-5.3,6-5.4,9.5
|
||||
c-0.1,7.5-4.3,9-10.5,10.2c-39.6,7.4-79.4,13.9-119.5,18.1c-2.8,0.3-6.7,2.3-8.1-1.3c-1.4-3.6-1.7-7.9-0.3-11.7
|
||||
c0.9-2.4,3.9-2,6.2-2.4c13.8-2.4,27.6-4.4,41.3-7c42.1-8.2,83.5-19.3,124.2-33.1c57.6-19.6,112.2-45.7,163.9-77.3
|
||||
c38.1-23.2,73.8-50.1,107.6-79.4c37.9-32.8,72.6-68.6,102.5-109C2037.5,591.3,2039.4,587.8,2043.6,585.1z"/>
|
||||
<path class="st1" d="M777.2,1163.8c-0.9-7.9,1.2-15.4,2-23c4.4-45.1,6.3-90.3,6.6-135.6c0.1-8.6,0.2-8.7,8.8-8.7
|
||||
c22-0.1,44-0.1,65.9,0c8.3,0,8.4,0.1,8.5,7.9c0.2,37.7,2,75.2,6.5,112.6c0.5,3.8,0.3,6-3.9,7.5c-31.7,11.3-62.2,25.3-92.6,39.6
|
||||
C778.8,1164.3,778.4,1164.1,777.2,1163.8z"/>
|
||||
<path class="st1" d="M776.8,817.8c3.1-0.6,4.7,0.6,6.5,1.4c28.2,13.5,56.6,26.5,86.2,36.8c5.4,1.9,6.7,4.3,6,10.1
|
||||
c-4.5,36.7-6.3,73.6-6.4,110.6c0,8.1,0.7,8.3-8.1,8.3c-22.6-0.1-45.3,0-67.9-0.1c-7,0-7.1-0.2-7.2-7.3c-0.4-28.6-0.9-57.3-2.7-85.9
|
||||
c-1.3-21.3-2.9-42.5-5.5-63.7C777.2,824.9,777.1,821.6,776.8,817.8z"/>
|
||||
<path class="st1" d="M1896.4,1163.9c-3.3,0.5-5.7-1.5-8.4-2.8c-27.2-13.3-54.8-25.7-83.3-36.1c-7.4-2.7-7.2-2.9-6.3-10.6
|
||||
c4.4-35.8,5.3-71.8,6.4-107.7c0.1-1.7,0.1-3.3,0-5c-0.3-3.8,1.3-5.4,5.2-5.3c6.3,0.2,12.7,0.1,19,0.1c16.7,0,33.3,0,50,0.1
|
||||
c8.4,0,8.4,0.1,8.6,8.8c0.8,30.3,1.2,60.6,3.3,90.9c1.4,19.9,2.9,39.9,5.3,59.7C1896.5,1158.6,1897.4,1161.2,1896.4,1163.9z"/>
|
||||
<path class="st1" d="M1896.8,817.7c-1.3,13.9-2.7,27.7-3.9,41.6c-3.5,38.5-4.5,77-5.2,115.6c0,1.7-0.2,3.3-0.1,5
|
||||
c0.4,4-1.5,5.5-5.2,5.1c-1-0.1-2,0-3,0c-22.6,0-45.3-0.1-67.9,0.2c-5,0-6.8-1.2-6.8-6.6c-0.4-28.3-1.2-56.5-3.6-84.7
|
||||
c-0.8-8.9-1.6-17.9-2.7-26.8c-0.9-7.6-1.1-7.8,6.5-10.6c23-8.7,45.8-17.8,67.9-28.7c6-2.9,12-5.7,18-8.6
|
||||
C1892.5,818.3,1894.3,817.4,1896.8,817.7z"/>
|
||||
<path class="st1" d="M908.3,1109.6c-1-4.4,0.4-8.6,0.8-12.8c2.5-30.9,4-61.8,4-92.9c0-7.4,0.2-7.5,7.9-7.5
|
||||
c33.3-0.1,66.7-0.2,100-0.2c7.8,0,8.1,0.1,8,7.4c-0.3,22,1,44,2.3,65.9c0.3,4.5-1,6.5-5.5,7.4c-38.7,8.5-76.4,20.9-114.2,32.5
|
||||
C910.7,1109.8,909.7,1109.6,908.3,1109.6z"/>
|
||||
<path class="st1" d="M1765.3,1110.4c-9.6-3-18.1-5.6-26.6-8.4c-29.1-9.6-58.7-17.4-88.5-24.5c-8.9-2.1-8.1-1.4-7.6-10.4
|
||||
c1.2-21,1.6-42,2.3-62.9c0.2-7.7,0.2-7.9,7.5-7.9c33.3,0,66.7,0.1,100,0.2c7.8,0,8.2,0.3,8.2,7.7c0.2,34,1.7,68,4.8,101.8
|
||||
C1765.4,1106.9,1765.3,1107.9,1765.3,1110.4z"/>
|
||||
<path class="st1" d="M908.8,872.2c2.8-1.1,5,0.8,7.5,1.5c36.1,10.3,71.6,22.9,108.5,30.5c5.5,1.1,6.7,3.6,6.4,8.8
|
||||
c-1.3,21.9-2.6,43.9-2.1,65.9c0.1,4.7-1.2,6.5-6.1,6.4c-34.7-0.2-69.3-0.2-104-0.1c-4.1,0-6.1-1-6-5.6c0.4-35.6-2-71.2-4.8-106.7
|
||||
C908.2,872.8,908.6,872.5,908.8,872.2z"/>
|
||||
<path class="st1" d="M1765.6,872.4c-0.9,10.7-1.8,20.9-2.5,31.2c-1.7,24.6-2.4,49.2-2.6,73.8c-0.1,7.7-0.2,7.8-7.7,7.8
|
||||
c-33.6,0.1-67.2,0.1-100.8,0.1c-6.9,0-6.9-0.2-7.1-7.1c-0.8-21.9-1.3-43.9-2.6-65.8c-0.3-5.4,1.8-7,6.2-8
|
||||
c14.5-3.5,29-7.1,43.5-10.8c22.5-5.7,44.5-13.5,66.7-20.3C1760.5,872.7,1762.3,871.8,1765.6,872.4z"/>
|
||||
<path class="st1" d="M725.8,1192.5c-5.9-34.3-8.8-67.4-11-100.6c-1.8-27.6-2.2-55.2-3-82.8c-0.1-2.3,0.1-4.7-0.1-7
|
||||
c-0.4-4.4,1.6-5.8,5.9-5.7c9,0.2,18,0,27,0c6.7,0,13.3-0.1,20,0c7.2,0.1,7.4,0.2,7.3,7.1c-0.5,23.6-1,47.2-2,70.8
|
||||
c-1.2,27.2-3.3,54.4-6.7,81.5c-0.2,1.3-0.3,2.6-0.4,4c-0.4,9.2-3.4,16-13,19.7C741.7,1182.7,734.5,1187.7,725.8,1192.5z"/>
|
||||
<path class="st1" d="M1946.2,1191.7c-10.1-5.6-20.4-11.5-30.8-17.2c-3.3-1.8-3.1-5-3.5-7.8c-1.4-10.2-2.4-20.5-3.5-30.7
|
||||
c-4.7-43.8-6-87.7-6.7-131.6c-0.1-7.7,0.1-7.8,7.7-7.8c15.3-0.1,30.6,0.1,46-0.1c4.6-0.1,6.5,1.1,6.4,6.1
|
||||
c-0.5,34-1.2,67.9-3.7,101.8c-2,26.9-4.5,53.7-8.9,80.4C1948.7,1187,1949.1,1189.5,1946.2,1191.7z"/>
|
||||
<path class="st1" d="M1947.7,789.1c5.8,32.7,8.6,63.8,10.8,95c2.1,30.6,2.8,61.2,3.1,91.9c0.1,9.6,0.3,8.9-8.3,8.9
|
||||
c-15,0-30-0.1-45,0.2c-5.3,0.1-6.9-1.8-6.7-7c0.9-25.3,1.3-50.6,2.5-75.9c1.3-27.3,3.9-54.4,6.9-81.6c0.9-8.3,3.3-14.2,11.5-17.8
|
||||
C1930.8,799.2,1938.6,794.1,1947.7,789.1z"/>
|
||||
<path class="st1" d="M726,789.2c11.3,6.2,21.7,11.9,32.2,17.5c2.9,1.6,2.8,4.2,3.2,6.8c2.7,15.8,4.1,31.7,5.5,47.6
|
||||
c3.3,37.2,4.5,74.5,5,111.8c0,2.3,0,4.7,0.1,7c0.1,3.1-0.8,5-4.4,5c-17-0.1-34-0.1-51-0.1c-3.3,0-4.8-1.5-4.7-4.8
|
||||
c0.2-5.7,0-11.3,0.2-17c1-42.6,2.7-85.2,7.3-127.7C720.9,820.5,722.6,805.7,726,789.2z"/>
|
||||
<path class="st1" d="M1625.1,910.5c-0.6,9.4-1.4,18-1.7,26.5c-0.4,13-0.4,26-0.6,38.9c-0.1,9.3-0.2,9.4-9.2,9.4
|
||||
c-30.6,0.1-61.3,0.1-91.9,0.1c-8.3,0-16.7-0.1-25,0.1c-4.2,0.1-5.9-1.3-5.9-5.7c0-15-0.1-30-0.7-44.9c-0.2-4.8,2.5-5.5,6-6
|
||||
c10.9-1.3,21.8-3.6,32.7-3.7c23.9-0.2,46.7-6.5,70-10C1607.1,913.9,1615.2,912.2,1625.1,910.5z"/>
|
||||
<path class="st1" d="M1625.2,1071.1c-23.5-4.2-45.7-8.1-67.9-12.1c-7.9-1.4-15.8-2.4-23.8-2.5c-12.7-0.1-25.1-2.6-37.7-4
|
||||
c-4-0.4-5.8-1.6-5.6-6.1c0.5-14.6,0.6-29.3,0.6-43.9c0-4.5,1-6.5,6.1-6.5c39.9,0.2,79.9,0.2,119.8,0.1c5,0,6.1,2.1,6.1,6.6
|
||||
C1622.9,1025,1623,1047.3,1625.2,1071.1z"/>
|
||||
<path class="st1" d="M1049.3,1070.4c-1-5.9-0.1-11.8,0.3-17.7c0.9-16.3,1.4-32.6,1.4-48.9c0-7.3,0.2-7.5,6.8-7.5
|
||||
c39.3-0.1,78.5-0.1,117.8-0.1c7,0,7.1,0.2,7.2,7.2c0.2,14,0,27.9,0.5,41.9c0.2,5.2-1.5,6.6-6.6,7c-16.9,1.3-33.8,2.7-50.6,5.2
|
||||
c-23.3,3.4-46.7,6.8-69.7,12.3C1054.2,1070.3,1051.9,1071.4,1049.3,1070.4z"/>
|
||||
<path class="st1" d="M1049.8,910.2c22.1,5.7,44.8,8.7,67.4,12.5c16.4,2.8,33.1,3.3,49.6,5.2c16.4,1.9,16.4,1.5,16.3,17.4
|
||||
c-0.1,11-0.3,22-0.2,32.9c0.1,4.9-1,7.2-6.6,7.2c-39.9-0.2-79.9-0.2-119.8-0.1c-3.4,0-5.7-0.6-5.6-4.8c0.6-22.6-1.3-45.2-1.9-67.8
|
||||
C1049,912,1049.4,911.4,1049.8,910.2z"/>
|
||||
<path class="st1" d="M1260.5,985.4c-21.6,0-43.2,0-64.9,0c-8,0-8.2-0.1-8.1-8.4c0.2-13.6,0.6-27.3,0.8-40.9c0.1-4.2,1.4-5.6,6-5.1
|
||||
c41.7,4.1,83.5,6.2,125.5,6.2c2.3,0,4.7,0,7,0c7.5,0.2,7.5,0.2,7.6,7.9c0.1,10.3,0.1,20.6,0.1,30.9c0,9.3,0,9.4-9.1,9.4
|
||||
C1303.8,985.4,1282.2,985.4,1260.5,985.4z"/>
|
||||
<path class="st1" d="M1485,931.5c0.3,16.2,0.5,32.7,0.8,49.3c0.1,3.4-1.7,4.8-4.9,4.6c-2-0.1-4,0-6,0c-41.6,0-83.2,0-124.9,0
|
||||
c-2,0-4-0.1-6,0c-4,0.3-5.1-1.8-5-5.5c0.1-11.7,0-23.3,0.1-35c0.1-7.6,0.1-7.8,7.6-7.8c17.3,0.2,34.6-0.6,51.9-1.1
|
||||
c26.3-0.7,52.5-3.1,78.7-5.1C1479.6,930.9,1481.8,930,1485,931.5z"/>
|
||||
<path class="st1" d="M1485,1051.1c-35.4-3-69.9-5.3-104.5-6.4c-11.6-0.4-23.3-0.5-34.9-0.4c-4.4,0-6.7-0.8-6.5-6
|
||||
c0.4-12,0.3-24,0-36c-0.1-4.8,1.6-6.3,6.3-6.2c45,0.1,89.9,0.1,134.9,0c3.5,0,5.8,0.7,5.7,4.8
|
||||
C1485.5,1017.5,1485.3,1034.1,1485,1051.1z"/>
|
||||
<path class="st1" d="M1188.4,1050.6c-0.3-17.2-0.5-33.7-0.8-50.3c0-2.9,1.6-4.1,4.4-4.1c1.7,0,3.3,0,5,0c42.6,0,85.1,0,127.7,0
|
||||
c1.7,0,3.3,0.1,5,0c3.3-0.2,4.9,1.2,4.9,4.6c-0.1,13-0.1,25.9,0,38.9c0,3.6-1.7,4.7-4.9,4.5c-0.7,0-1.3,0-2,0
|
||||
C1281.6,1043.7,1235.9,1047.2,1188.4,1050.6z"/>
|
||||
<path class="st1" d="M2010.7,1230.1c-5.1-1.9-9-5.1-13.1-7.7c-9.9-6.1-19.6-12.6-29.8-18.3c-4.8-2.6-5.5-5.6-4.5-10.4
|
||||
c3.9-20,6.2-40.1,8.2-60.4c4-41.2,5.8-82.4,6.1-123.8c0-3,0.1-6,0-9c-0.1-3,1.1-3.8,4.1-4.1c16.5-1.5,16.7-1.4,16.8,14.7
|
||||
c0.4,66.4,3.9,132.5,10.7,198.6C2009.9,1216.3,2011.6,1222.7,2010.7,1230.1z"/>
|
||||
<path class="st1" d="M2011.2,752.5c-1.6,16.9-3.2,33.8-4.8,50.6c-3,30.8-4.6,61.7-6.1,92.6c-1.3,27.3-1.5,54.5-1.8,81.8
|
||||
c-0.1,7.1-0.2,7.3-7.1,7.3c-16.3,0-13.6,1.3-13.7-12.8c-0.5-54.3-3.2-108.4-10.6-162.2c-1.1-7.9-2.5-15.8-3.9-23.6
|
||||
c-0.7-3.6-0.1-6,3.5-8.2c13.4-8,26.6-16.4,39.8-24.7C2007.6,752.8,2008.5,751.7,2011.2,752.5z"/>
|
||||
<path class="st1" d="M662.1,1231.5c1.9-20.4,3.7-38.6,5.4-56.8c3-31.9,4.7-63.8,6-95.8c1-24.6,1.6-49.3,1.6-74
|
||||
c0-8.3,0.1-8.3,7.9-8.4c1.3,0,2.7,0,4,0c8.6,0.1,8.7,0.1,8.8,8.7c0.4,40,1.8,80,5.4,119.8c2,22.9,4.4,45.8,9,68.3
|
||||
c1.1,5.5,0.2,8.5-5.1,11c-12.7,6-23.9,14.5-35.5,22.2C667.7,1227.9,665.8,1229.1,662.1,1231.5z"/>
|
||||
<path class="st1" d="M662.9,752.1c2.6-0.9,3.9,1,5.4,2.1c12.4,8.4,24.6,17.1,38,23.9c3.3,1.7,5,3.6,4.2,7.7
|
||||
c-11.3,61.8-13.7,124.4-14.8,187c-0.2,13.5,2.2,12.1-12.3,12.1c-8.4,0-8.5-0.1-8.5-8.9c0.1-52.7-2.6-105.3-6.4-157.8
|
||||
c-1.5-20.6-4-41.1-6-61.7C662.5,755.1,661.9,753.4,662.9,752.1z"/>
|
||||
<path class="st1" d="M2044.3,728.6c0,3.4,0,6.3,0,9.2c0,78.9,0,157.9,0,236.8c0,1.7-0.1,3.3,0,5c0.4,3.9-1.3,5.4-5.2,5.3
|
||||
c-5.9-0.2-11.9,0-18.3,0c0-26.3,0.3-51.6,1.3-76.8c1.7-42.3,4.4-84.5,8.9-126.5c1.6-14.9,3.9-29.7,5.6-44.6
|
||||
C2037.2,732.4,2038.9,729.7,2044.3,728.6z"/>
|
||||
<path class="st1" d="M2044.3,1253c-5.6-1.2-7-4.1-7.7-8.6c-3.1-19.4-5.2-38.9-7-58.5c-5.6-59-8.7-118.2-8.7-177.6
|
||||
c0-13.6-1.4-11.7,11.5-11.8c13.3,0,11.9-1.5,11.9,11.5c0,78.6,0,157.2,0,235.8C2044.3,1246.6,2044.3,1249.3,2044.3,1253z"/>
|
||||
<path class="st1" d="M629.2,728.7c5.8,1.6,7.4,4.6,7.9,9c2.3,21.2,5.3,42.2,7.3,63.4c5.3,55.3,7.7,110.8,8.5,166.4
|
||||
c0,3.3,0.1,6.7,0,10c-0.1,7.5-0.1,7.5-7.7,7.7c-16,0.5-16,0.5-16-15.3c0-76.5,0-153.1,0-229.6C629.2,736.7,629.2,733,629.2,728.7z"
|
||||
/>
|
||||
<path class="st1" d="M629.2,1253.2c0-6,0-10.6,0-15.3c0-75.7,0-151.3,0-227c0-2,0-4,0-6c-0.1-9.2-0.3-8.7,8.7-8.6
|
||||
c15.1,0,15.2,0,15,15.6c-0.7,62.7-3.8,125.2-10.3,187.6c-1.5,14.9-3.9,29.7-5.5,44.6C636.6,1248.7,634.5,1251.1,629.2,1253.2z"/>
|
||||
<path class="st1" d="M2060,697.5c0-15.9,0-29.5,0-43c0-3.7,0.3-7.3-0.1-11c-2-19,6.4-34.8,15.3-50.5c1-1.7,2.3-3.2,3.6-4.7
|
||||
c5.4-6.2,8.1-6.2,13.4,0.4c6.3,7.8,10.3,17.1,13.7,26.4c3.2,8.7,5.9,17.6,8.7,26.5c1.3,4,1,7.1-2.4,10.7
|
||||
C2096.7,668.2,2079,681.6,2060,697.5z"/>
|
||||
<path class="st1" d="M613.2,1283.1c0,24.5,1.5,47.6-0.5,70.3c-1.4,15.4-9,29.8-19.7,41.7c-3.8,4.2-5.7,3.9-10-0.4
|
||||
c-4.3-4.3-7.2-9.6-9.8-15c-6.7-13.8-11.2-28.4-15.3-43.2c-0.8-3-1.3-5.6,1.5-8.1C576.3,1313,592.4,1296.7,613.2,1283.1z"/>
|
||||
<path class="st1" d="M2060,1284.4c19.3,15.5,36.8,29.3,52.5,45.2c2.6,2.6,3.7,5.1,2.6,8.8c-5.1,17.2-10,34.5-19.7,50
|
||||
c-7.7,12.2-12.4,12.7-19.8,0.6c-9.5-15.4-16.9-31.4-15.8-50.5C2060.7,1321.2,2060,1303.9,2060,1284.4z"/>
|
||||
<path class="st1" d="M611.9,698.2c-8.5-6.7-17.1-12.8-25-19.7c-9.1-7.8-17.5-16.4-26.3-24.5c-2.3-2.2-4-4.1-3-7.7
|
||||
c5.3-18.2,10.3-36.5,20.4-52.8c7.8-12.7,12.8-13,20.4-0.3c8.9,14.9,16.3,30.2,15.1,48.6c-1,15.9-0.2,31.9-0.3,47.9
|
||||
C613.2,692.2,614,694.9,611.9,698.2z"/>
|
||||
<path class="st1" d="M1788.2,864c4.1,27,4.9,53.5,6.1,80.1c0.5,11.3,0.2,22.6,0.7,33.9c0.2,5.2-1.6,7.4-6.7,7.3
|
||||
c-4.7-0.1-9.3-0.3-14,0c-4.6,0.3-5.5-2-5.4-6c0.2-26.3,1.3-52.5,3.2-78.8c0.5-7.6,1.2-15.2,2.1-22.8
|
||||
C1775.6,867.2,1775.7,867.2,1788.2,864z"/>
|
||||
<path class="st1" d="M885.3,1118.4c-5.1-39.1-6.6-77.6-6.6-116.3c0-4.2,1.3-5.9,5.7-5.8c25,0.7,20.1-4.1,19.8,19.4
|
||||
c-0.4,29.6-1.5,59.3-5,88.8C897.9,1115.4,898.1,1115.4,885.3,1118.4z"/>
|
||||
<path class="st1" d="M885.7,863.5c12.4,2.1,12.2,2.1,13.5,13.8c3.8,33,4.6,66.2,5.3,99.4c0.2,8.3,0,8.5-7.9,8.4
|
||||
c-5.6-0.1-12.6,2.4-16.3-1.2c-3.8-3.6-1.3-10.7-1.3-16.2c0.1-33.9,1.8-67.7,6.1-101.4C885.1,865.4,885.4,864.5,885.7,863.5z"/>
|
||||
<path class="st1" d="M1788.4,1117.5c-4,0.3-6.2-1.5-8.7-2.2c-3.2-0.9-4.6-2.9-4.9-6.5c-0.7-10.3-2.2-20.5-2.9-30.8
|
||||
c-1.5-24.6-2.8-49.2-2.8-73.9c0-7.5,0.1-7.8,7.7-7.6c5.9,0.2,13.3-2.7,17.2,1.6c3.4,3.7,1,10.9,0.9,16.5
|
||||
C1794.3,1048.8,1792.8,1083.1,1788.4,1117.5z"/>
|
||||
<path class="st1" d="M1481.1,1078.1c-10.7,0.3-20.5-1.5-30.3-2.6c-34.1-3.7-68.2-5.9-102.5-5.6c-1.3,0-2.7-0.1-4,0
|
||||
c-4.9,0.5-5.1-2.5-5.2-6.2c-0.1-4,0.7-6,5.4-6c20,0.1,39.9-0.2,59.9,1c22.9,1.4,45.8,3,68.7,4.7c7.9,0.6,7.8,0.9,8.2,8.7
|
||||
C1481.2,1073.7,1481.1,1075.4,1481.1,1078.1z"/>
|
||||
<path class="st1" d="M1480,903c2,3.3,0.7,6.3,1,9.2c0.4,4.3-1.6,5.6-5.8,5.9c-42.2,3.3-84.3,6.2-126.7,5.9c-1.3,0-2.7-0.1-4,0
|
||||
c-4.5,0.5-5.5-1.7-5.6-5.8c-0.1-4.5,0.9-6.6,6-6.5c29.7,0.7,59.2-1.5,88.8-3.9C1449.3,906.5,1464.7,904.6,1480,903z"/>
|
||||
<path class="st1" d="M1192.2,1078.2c1-14.7,0.9-15,13.5-15.8c40.5-2.9,80.9-5.4,121.6-4.6c7.3,0.1,7.4,0.5,7.3,7
|
||||
c-0.1,4-1.7,5.2-5.4,5.1c-6-0.2-12-0.1-18,0c-37.3,0.2-74.3,3.2-111.3,7.8C1197.6,1078,1195.3,1078,1192.2,1078.2z"/>
|
||||
<path class="st1" d="M1192.6,903.2c17.2,2.2,33.7,3.8,50.2,5c28.2,2.1,56.4,4.4,84.7,3.6c3.8-0.1,7.3-0.4,7.1,5.4
|
||||
c-0.3,6.3-0.2,6.8-6.6,6.8c-18.6,0-37.2,0.4-55.9-0.8c-19.2-1.2-38.5-2.1-57.8-3.2c-3-0.2-6-0.4-8.9-0.8
|
||||
C1192.8,917.6,1192.8,917.6,1192.6,903.2z"/>
|
||||
<path class="st1" d="M653.6,694.6c2.5,2.7,3.4,3.5,4.2,4.6c15.1,20.9,32.8,39.4,51.8,56.7c2.5,2.3,7.7,4.8,5,8.5
|
||||
c-2.7,3.9-6.2-1-8.9-2.4c-14.7-7.8-28.1-17.9-42.2-26.6c-3.1-1.9-4.3-4.4-4.8-8C657.4,717,655.5,706.9,653.6,694.6z"/>
|
||||
<path class="st1" d="M654,1284.6c0.9-5.5,2.7-11,2.6-16.4c-0.1-14.4,6.3-23.8,19.2-30.2c10.6-5.3,20.3-12.5,30.5-18.8
|
||||
c2.5-1.5,4.9-3.3,7.8-3.6c1.6,2.1,1.5,4,0.2,5.7c-1,1.3-2.4,2.3-3.6,3.4c-18.4,17.3-36.5,34.8-51.2,55.5c-1.1,1.6-2.6,2.9-4,4.3
|
||||
C655.3,1284.8,654.8,1284.6,654,1284.6z"/>
|
||||
<path class="st1" d="M2019.1,698.8c-1.4,9.8-3,19.6-4.2,29.5c-0.4,3-2.2,4.7-4.4,6.2c-15.3,9.9-30.7,19.7-46.1,29.5
|
||||
c-1.5,1-3.7,4-5.7,0.4c-1.5-2.6,0.2-4.4,2.1-5.9c11.9-9.2,21.5-20.7,32.4-30.8c8.4-7.7,14.5-17.2,21.4-26.1c0.9-1.2,2-2.4,3-3.6
|
||||
C2018.1,698.2,2018.6,698.5,2019.1,698.8z"/>
|
||||
<path class="st1" d="M2017.1,1283.9c-11.8-19.6-28.7-34.6-44.6-50.5c-3.7-3.7-7.8-7.2-11.8-10.6c-1.9-1.7-3.4-3.4-1.7-5.9
|
||||
c1.7-2.4,3.5-0.5,5,0.5c15.7,10,31.3,20,46.9,30c2,1.3,3.5,2.9,3.9,5.4c1.4,10.1,2.9,20.2,4.4,30.4
|
||||
C2018.5,1283.4,2017.8,1283.7,2017.1,1283.9z"/>
|
||||
<path class="st1" d="M1044.6,1034c-0.6,11-1.1,21.9-1.7,32.9c-0.1,1.6-0.3,3.3-0.8,4.9c-0.3,0.8-1.4,1.6-2.2,1.7
|
||||
c-0.9,0.1-1.9-0.5-2.1-1.7c-0.1-1-0.3-2-0.4-3c-1.2-22.2-2.3-44.5-2.4-66.8c0-3.2,0-6,4.6-5.9c3.5,0.1,5.6,0.7,5.5,4.9
|
||||
c-0.2,11-0.1,21.9-0.1,32.9C1044.9,1034,1044.8,1034,1044.6,1034z"/>
|
||||
<path class="st1" d="M1636.4,1033.8c0,10.6,0,21.2,0,31.8c0,2-0.3,4-0.7,5.9c-0.2,0.8-1.2,1.8-1.9,1.8c-0.8,0-1.9-0.8-2.2-1.5
|
||||
c-0.5-1.2-0.5-2.6-0.6-3.9c-1.9-21.8-2.2-43.7-2.1-65.6c0-5.7,1.3-6.7,6-6.1c3.9,0.5,3.6,3.1,3.6,5.8c0,10.6,0,21.2,0,31.8
|
||||
C1637.8,1033.8,1637.1,1033.8,1636.4,1033.8z"/>
|
||||
<path class="st1" d="M1638.5,948.1c0,10.3,0,20.6,0,31c0,2.6,0.5,5.4-3.3,6.2c-4.4,0.9-6.4-0.5-6.4-5.8c0-19,0-38,1.4-56.9
|
||||
c0.2-3,0.4-6,0.7-9c0.2-2.2-0.3-5.4,3-5.2c2.5,0.1,2.2,3,2.3,4.9c0.6,11.6,1.1,23.3,1.6,34.9C1638,948.1,1638.2,948.1,1638.5,948.1
|
||||
z"/>
|
||||
<path class="st1" d="M1045.1,948c0,10.6-0.1,21.2,0,31.9c0,3.6-0.9,5.5-5,5.5c-3.9-0.1-5.1-1.7-5.1-5.4c0.1-22.6,1.2-45.1,2.5-67.7
|
||||
c0.1-1.6-0.1-3.7,2-4c2.8-0.4,3.1,1.9,3.2,3.9c0.7,11.9,1.3,23.9,1.9,35.8C1044.8,948,1044.9,948,1045.1,948z"/>
|
||||
<path class="st1" d="M741.9,740.1c7,10.8,9.7,22,12.6,34.5c-8.1-3.6-13.5-8.9-19.7-13c-2-1.3-2-3.3-1.2-5.3
|
||||
C735.6,751.1,737,745.6,741.9,740.1z"/>
|
||||
<path class="st1" d="M1931.7,1241.3c-6.9-10.6-9.7-21.7-12.1-33.3c4.2,0.4,6.4,3.1,9.1,4.9
|
||||
C1943.2,1223.1,1943.4,1224.7,1931.7,1241.3z"/>
|
||||
<path class="st1" d="M1931.6,740c12,17.2,11.9,18.3-3.6,29.1c-2.4,1.7-4.5,3.9-8.5,4.8C1921.8,761.9,1925,750.9,1931.6,740z"/>
|
||||
<path class="st1" d="M742.1,1241.3c-5.3-5.2-6.5-11.1-8.6-16.5c-0.8-2-0.5-3.8,1.6-5.1c6-3.9,11.2-9.1,19.2-12.6
|
||||
C751.8,1219.5,748.7,1230.3,742.1,1241.3z"/>
|
||||
<path class="st1" d="M599.6,517.5c0.6,0.1,1.4,0.1,1.9,0.4c12.6,7.1,13,7.9,10.7,24.4c-5.9-7.9-11-15.2-13.4-24.1
|
||||
C599,518,599.3,517.7,599.6,517.5z"/>
|
||||
<path class="st1" d="M597.6,1464.4c4-9.4,8.1-17.2,14-24.4C614.9,1455.5,614.5,1457.9,597.6,1464.4z"/>
|
||||
<path class="st1" d="M2072.4,516.4c2.2,0.6,2.3,2,1.3,3.7c-3.8,6.1-7.7,12.1-11.9,18.6c-2.8-10.6-2.1-12.7,5.3-17.8
|
||||
c1.9-1.3,4.2-2,5.3-4.3L2072.4,516.4z"/>
|
||||
<path class="st1" d="M2061.5,1442.5c4.7,8.5,10.5,14.5,12.5,22.8C2067.2,1459.8,2056.4,1456.8,2061.5,1442.5z"/>
|
||||
<path class="st1" d="M1781.6,836.9c4.2,9.3,4.2,9.3-1.8,12.6C1777.9,845.4,1780.6,842.1,1781.6,836.9z"/>
|
||||
<path class="st1" d="M895.4,849.3c-7.5-2.9-7.5-2.9-3.8-11.1C893.8,841.3,893.9,844.7,895.4,849.3z"/>
|
||||
<path class="st1" d="M630.1,1284.6c-0.2-5.4-2.7-11,1.8-17C634.2,1274.4,629.7,1279.3,630.1,1284.6z"/>
|
||||
<path class="st1" d="M629.9,697.1c0.5,5.4,3.9,10.3,2.4,16.6C626.9,708.5,630.1,702.5,629.9,697.1z"/>
|
||||
<path class="st1" d="M2044.5,1284.5c-2.2-5-4.3-9.2-2.8-15C2046.1,1274.3,2043.5,1279.4,2044.5,1284.5z"/>
|
||||
<path class="st1" d="M2041.9,712.1c-2.2-6,1.1-10.1,2.2-14.6C2043.5,702.2,2046,707.4,2041.9,712.1z"/>
|
||||
<path class="st1" d="M892.2,1143.9c-4.6-7.9-4.6-7.9,1.6-11.5C895.7,1136.5,892.6,1139.7,892.2,1143.9z"/>
|
||||
<path class="st1" d="M1779.6,1132.2c6.1,3.1,6.1,3.1,2.7,11.1c-1.9-1-1.7-3.1-2.3-4.6C1779.2,1137,1778.6,1135.1,1779.6,1132.2z"/>
|
||||
<path class="st1" d="M1934.8,1202.5c4.1-0.6,6.5,2.5,11.4,4.9c-6.3,0.5-8.2-3.5-11.3-5.1L1934.8,1202.5z"/>
|
||||
<path class="st1" d="M1935.8,777c3.1,0.9,4.5-3.5,7.8-2.9c-1.8,4-4.7,3.9-7.9,2.8L1935.8,777z"/>
|
||||
<path class="st1" d="M729.5,774c2.7-0.5,4.4,0.6,5.8,3.2C732.1,778,731,775.7,729.5,774z"/>
|
||||
<path class="st1" d="M734,1202.5c2.3,4.6-1.2,4.8-4.2,5.9c-0.6-3.7,3.7-3.3,4.2-5.7C734,1202.7,734,1202.5,734,1202.5z"/>
|
||||
<path class="st1" d="M1935.2,780.3c-0.4,0.2-0.8,0.5-1.2,0.7c0.3,0,0.6,0.1,0.8,0c0.2-0.1,0.4-0.3,0.6-0.5
|
||||
C1935.5,780.5,1935.2,780.3,1935.2,780.3z"/>
|
||||
<path class="st1" d="M740,1203.1c-1.9-0.8-4.2,2.9-6-0.5c0,0,0,0.2,0,0.1C736,1202.8,738,1202.9,740,1203.1L740,1203.1z"/>
|
||||
<path class="st1" d="M1935.7,777c1.4,1.3,0.9,2.4-0.5,3.3c0,0,0.3,0.3,0.3,0.3c0.1-1.2,0.3-2.3,0.4-3.5
|
||||
C1935.8,777,1935.7,777,1935.7,777z"/>
|
||||
<path class="st1" d="M737.9,778.4c0.2,0.2,0.4,0.5,0.6,0.7c-0.2,0.2-0.5,0.5-0.7,0.5c-0.2,0-0.4-0.4-0.6-0.6
|
||||
C737.4,778.8,737.7,778.6,737.9,778.4z"/>
|
||||
<path class="st1" d="M740,1203.1c0-0.3,0-0.8-0.2-1c-0.8-0.8-0.7-1.1,0.4-0.8C739.5,1201.9,739.4,1202.4,740,1203.1
|
||||
C740,1203.1,740,1203.1,740,1203.1z"/>
|
||||
<path class="st1" d="M1934.9,1202.3c0-0.2-0.1-0.3-0.1-0.5c0,0-0.1,0.1-0.1,0.1c0.1,0.2,0.1,0.4,0.2,0.6
|
||||
C1934.8,1202.5,1934.9,1202.3,1934.9,1202.3z"/>
|
||||
<path class="st1" d="M2072.4,516.6c0.7,0.4,1.4,0.8,2.1,1.2c-0.2,0.2-0.4,0.7-0.6,0.7c-1.1-0.2-1.5-1-1.4-2
|
||||
C2072.4,516.4,2072.4,516.6,2072.4,516.6z"/>
|
||||
<path class="st1" d="M598.7,518.3c-1-0.2-1.8-0.8-1-1.6c0.8-0.9,1.4,0,1.8,0.8C599.3,517.7,599,518,598.7,518.3z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 48 KiB |
|
|
@ -1,458 +0,0 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 24.2.1, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
|
||||
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
viewBox="0 0 2670 1980" style="enable-background:new 0 0 2670 1980;" xml:space="preserve">
|
||||
<style type="text/css">
|
||||
.st0{fill:#0A0A0A;}
|
||||
.st1{fill:none;}
|
||||
</style>
|
||||
<g>
|
||||
<path class="st0" d="M2275.3,1000.1c-0.1,2.3,0,4.7,0,7c0,189.3,0,378.7,0,568c0,6,0.1,12.1-0.8,17.9c-2.4,15.2-12.9,17.4-23.6,10
|
||||
c-45.4-31-91.2-61.4-136.9-91.9l-30.1-20.3c-9.8-6.3-19.2-13.2-29.2-19.3c-6.7-4.1-9.8-9.7-9.9-17.2c-0.1-9.3-0.6-18.7-0.3-28
|
||||
c0.2-7.3-1.7-13-6.5-19c-18.6-23.3-36.2-47.3-56.9-69c-18-18.8-36.7-36.8-56.4-53.8c-31.8-27.5-65.4-52.7-101-74.9
|
||||
c-32.7-20.4-66.8-38.4-102.2-54c-50.7-22.4-103.2-39.2-157.1-51.5c-36.7-8.4-73.8-14.4-111.2-18.5c-34.5-3.8-69-6.5-103.7-5.8
|
||||
c-23.1,0.5-46.2-1.3-69.3,0.4c-22.9,1.8-45.9,3-68.7,5.6c-72.7,8.4-143.7,24.6-212.1,50.6c-65.5,24.8-127.4,57-184.9,97.4
|
||||
c-46.8,32.8-90.1,69.8-129.5,111.1c-18.1,19-35.1,39.2-50.5,60.6c-3.9,5.5-5.5,11-5.3,17.6c0.4,9.3,0,18.7,0,28
|
||||
c0,9.3-3.1,16.4-11.8,21.4c-10.4,5.9-19.9,13.4-30.1,19.8l-25.4,17.2c-11.6,7.8-23.3,15.5-34.9,23.3c-34.3,23-68.6,46-102.9,69
|
||||
c-14.6,9.7-23.6,5.5-25.2-12.1c-0.5-5.3-0.5-10.7-0.5-16c0-189.7,0-379.3,0-569l-0.1-26.1c0.2-190.3,0.1-380.7,0.2-571
|
||||
c0-5.7,0.4-11.3,0.5-17c0.1-3.7,1.2-7.3,3-10.5c3.4-5.9,9.3-7.7,15.4-4.8c2.7,1.3,5.1,3.1,7.6,4.8c44.3,29.6,88.5,59.3,132.8,88.9
|
||||
l29.7,20.1c10.8,6.6,20.9,14.3,31.7,20.9c6.7,4.1,9.7,9.8,9.8,17.3c0.2,11,0.4,22,0.2,33c-0.1,5.2,1.2,9.6,4.2,13.9
|
||||
c15.4,22.2,33,42.8,51.6,62.2c53.9,56.4,114.1,105.1,181.4,144.6c38.4,22.6,78.3,42.4,119.9,58.6c40,15.6,80.9,28.3,122.8,38
|
||||
c52.7,12.1,106,19.3,160,22.6c18.3,1.1,36.6,2.1,54.9,1.5c17.7-0.6,35.5,1.1,53.2-0.1c17.6-1.1,35.2-2.3,52.8-3.6
|
||||
c35.9-2.5,71.4-8.5,106.7-15.1c53.4-9.9,105.5-25,156-45.1c53.8-21.5,105.1-48.2,153.2-80.9c43.4-29.5,83.6-62.7,121.1-99.3
|
||||
c27.3-26.6,50.3-56.8,73.9-86.6c2.9-3.6,3.6-7.6,3.6-12.1c-0.1-10.7,0.2-21.3,0.4-32c0.1-7.5,3.2-13.1,9.9-17.2
|
||||
c10.2-6.2,19.9-13.4,30-19.8l23.2-16.3c4.2-1.3,7.4-4.3,11-6.7c42.9-28.7,85.8-57.4,128.7-86.1c1.1-0.7,2.3-1.4,3.3-2.3
|
||||
c10.9-9.1,21.4-4.7,23.8,10.7c0.8,5.6,0.6,11.3,0.6,17c0,189.3,0,378.7,0,568L2275.3,1000.1z M740,1203.1c-2-0.2-4-0.3-6-0.5
|
||||
c-0.5,2.5-4.8,2-4.2,5.8c2.9-1,6.4-1.2,4.2-5.8C735.8,1206,738.1,1202.3,740,1203.1c-0.6-0.7-0.5-1.3,0.2-1.7
|
||||
c-1.1-0.3-1.1-0.1-0.4,0.8C740,1202.3,739.9,1202.8,740,1203.1z M1935.5,780.5c-0.2,0.2-0.4,0.4-0.6,0.4c-0.3,0.1-0.6,0-0.8,0
|
||||
c0.4-0.2,0.8-0.5,1.1-0.8c1.4-0.9,2-1.9,0.5-3.3c3.2,1.1,6.1,1.2,7.9-2.8c-3.3-0.7-4.6,3.8-7.7,2.9
|
||||
C1935.7,778.2,1935.6,779.4,1935.5,780.5z M1934.8,1202.5c-0.1-0.2-0.1-0.4-0.2-0.6c0,0,0.1-0.1,0.1-0.1c0,0.2,0.1,0.3,0.1,0.5
|
||||
c3.1,1.6,5,5.7,11.3,5.1C1941.3,1205,1938.9,1202,1934.8,1202.5z M2072.4,516.6c-1,2.3-3.4,3-5.3,4.3c-7.4,5.1-8.2,7.2-5.3,17.8
|
||||
c4.2-6.5,8-12.6,11.9-18.6c1.1-1.7,0.9-3.1-1.3-3.6c0,1,0.3,1.8,1.4,2c0.2,0,0.4-0.4,0.6-0.7C2073.8,517.4,2073.1,517,2072.4,516.6
|
||||
z M599.6,517.5c-0.4-0.8-1.1-1.7-1.8-0.8c-0.8,0.8,0.1,1.5,1,1.6c2.4,8.9,7.5,16.2,13.4,24.1c2.3-16.5,1.9-17.3-10.7-24.4
|
||||
C600.9,517.6,600.2,517.6,599.6,517.5z M534.9,1353.5c-20,21.8-37.3,44.1-53.4,67.4c-16.1,23.4-26.8,49.5-41.8,73.5
|
||||
c9-21,17.4-42.2,28.9-61.9c16.9-28.7,35.4-56.4,57.5-81.5c0.9-1,1.6-2.1,2.6-3.1c2.9-2.9,3.3-6,2.3-10
|
||||
c-9.1-33.8-14.6-68.3-19.2-103c-5-38-8.3-76.1-10.6-114.3c-2.5-39.9-3.4-79.9-3.9-119.8c-0.1-5.1-1.4-6.8-6.7-6.9
|
||||
c-21.6-0.2-43.3-0.8-64.9-1.3c-2,0-4-0.4-6,0.8c0,195.2,0,390.5,0,585.6c2.1,0.6,3.1-0.4,4.1-1.1c52.7-33.8,105.3-67.7,158-101.4
|
||||
c3.6-2.3,5.7-5.4,7.3-9.1c6-13,12.5-25.8,20.8-37.4c4.2-5.9,3.9-11.9,2.5-19.1c-2.4,1.8-4,2.9-5.5,4.2
|
||||
c-13.9,12.6-32.1,8.7-42.7-2.9c-7.3-8-12.4-17.2-17.1-26.8C542.3,1375.7,539.4,1365.1,534.9,1353.5z M534.6,627.2
|
||||
c3.6-5,4.3-10.7,6.3-15.9c6.2-15.8,12.6-31.5,25.2-43.7c10.8-10.5,26.8-12.6,38.9-2.8c2.3,1.9,4.3,5,7.9,5
|
||||
c1.6-8.3-0.2-15.3-5.1-21.9c-7.4-9.9-12.7-21-17.8-32.3c-2.4-5.3-5.8-9.2-10.8-12.4c-50.5-32.2-100.9-64.7-151.3-97.1
|
||||
c-2.4-1.5-4.5-3.9-8.7-3.9c0,3.7,0,7.2,0,10.7c0,187.9,0,375.8,0,563.7c0,2,0.2,4,0,6c-0.6,4.9,1.4,6.3,6.2,6.1
|
||||
c13.6-0.6,27.3-0.7,41-1c8.3-0.2,16.7-0.6,25-0.4c4.9,0.1,6.7-1.8,6.1-6.4c-0.1-1,0-2,0-3c1.2-75,4.3-149.8,13.7-224.3
|
||||
c4.6-36.3,10-72.5,19.6-107.9c1.6-5.7,0.4-9.9-3.4-14.4c-10.7-12.8-21.7-25.4-31-39.1c-19-28.1-36.8-57-49.6-88.7
|
||||
c-2.2-5.4-5.5-10.4-6.2-16.6C465,538.3,494.6,586,534.6,627.2z M2060.9,1411.1c-2.3,8.5-0.1,15.1,4,21.4
|
||||
c5.9,9.3,13.4,18.1,17.1,28.2c3.9,10.7,10.9,16.6,19.9,22.4c47.8,30.3,95.3,61.1,143,91.7c2.7,1.7,5,4.3,9.5,4.5c0-3.4,0-6.6,0-9.8
|
||||
c0-188.3,0-376.6,0-565c0-2-0.1-4,0-6c0.2-3.4-1.1-5-4.6-4.8c-2.3,0.1-4.7,0-7,0.1c-20,0.6-40,1.2-60,1.9c-6.7,0.2-6.7,0.4-7,7.4
|
||||
c0,1.3,0,2.7-0.1,4c-0.4,60-3,119.9-8.9,179.7c-5,50.1-11.3,99.9-24,148.7c-1.4,5.2,0.1,8.3,3.5,12.3c15,17.8,29.6,36.1,42.4,55.5
|
||||
c20,30.4,36.1,62.9,49.3,96.8c3.2,8.2,5.7,16.6,6.8,25.4c-33.9-90.8-82.6-145-107.2-169.7c-4.1,11-7.5,22.1-12.7,32.5
|
||||
c-4.9,9.9-10.3,19.5-18.8,26.8c-7.1,6.1-15.1,8.9-24.8,7.5C2073.2,1421.4,2068.3,1415.5,2060.9,1411.1z M2244.8,454.3
|
||||
c0.6,4.5-0.2,6.7-0.8,8.9c-6.5,24.2-17,47-28.2,69.2c-18.9,37.2-43,71.2-70.4,102.8c-2.7,3.2-3.6,5.8-2.5,9.8
|
||||
c7.9,29.6,13.1,59.8,17.3,90.2c6.4,45.6,10.2,91.3,12.8,137.3c2,35.3,2.6,70.6,3.1,105.9c0.1,7.6,0.2,7.6,7.6,7.8
|
||||
c17,0.5,34,1,51,1.4c5,0.1,10,0.2,15,0.5c3.2,0.1,5.1-0.9,4.9-4.5c-0.1-1.7,0-3.3,0-5c0-189.6,0-379.3-0.1-568.9
|
||||
c0-2.4,1.1-5.2-1.2-7.3c-0.8,0.1-1.6,0-2.1,0.4c-52.7,33.8-105.3,67.7-158,101.5c-2,1.3-3.7,2.6-4.7,4.9c-6.9,16-16.9,30.1-25.8,45
|
||||
c-3.1,5.2-3.4,10.2-2,15.7c2.6,0.1,3.5-1.6,4.6-2.6c15.7-13.9,33-10.6,45.6,3.8c6,6.8,10.4,14.6,14.4,22.7
|
||||
c5.1,10.3,8.3,21.3,12.8,32.9C2185.2,576.1,2221.7,520.5,2244.8,454.3z M553.7,662.8c-0.3,1.3-0.7,2.8-1,4.3
|
||||
c-7.1,31.2-11.7,62.8-15.4,94.5c-7.1,59.9-10.3,120.1-11.8,180.4c-0.3,13-0.2,26-0.5,39c-0.1,4.4,1.5,5.7,5.7,5.6
|
||||
c25-0.5,50-0.7,74.9-1.1c7.4-0.1,7.6-0.2,7.6-7.8c0-85.6,0-171.2,0.1-256.8c0-5.3-1.7-8.5-5.8-11.7c-16.6-12.8-32.4-26.6-46.9-41.9
|
||||
C559,665.6,557.6,663,553.7,662.8z M553.6,1319.7c2.1-1,2.8-1.2,3.2-1.6c16.4-17.1,34-32.9,52.7-47.4c3-2.4,3.8-5,3.8-8.6
|
||||
c-0.1-13.3-0.1-26.7-0.1-40c0-72.3,0-144.7,0-217c0-10.3,0.4-9-9.2-9.2c-23.3-0.4-46.7-0.8-70-1.1c-2.8,0-5.9-1-9.3,1.5
|
||||
C526.2,1104.4,531.1,1212.3,553.6,1319.7z M2119.8,1319.2c4.8-20.9,8.3-40.2,11.1-59.6c5.8-39.5,9.8-79.2,12.5-119.1
|
||||
c3.1-45.5,4.7-91.1,5.1-136.7c0.1-7.6-0.1-7.7-7.6-7.7c-24.3,0.1-48.6,0.3-72.9,0.5c-7.9,0.1-8,0.1-8,8.5c0,84.6,0,169.1-0.1,253.7
|
||||
c0,5.6,1.7,9.3,5.9,12.7c12.1,9.9,24.2,19.9,36,30.1C2107.7,1306.9,2113.1,1312.7,2119.8,1319.2z M2120.4,664.4
|
||||
c-2.1-0.6-2.9-0.1-3.6,0.7c-15.2,16.5-33.1,29.9-50.2,44.2c-4.8,4-6.8,8-6.8,14.3c0.3,32,0.2,64,0.2,96c0,52.3,0,104.6,0,157
|
||||
c0,8.1,0.1,8.2,8.2,8.2c18.3,0.2,36.7,0.2,55,0.4c6.7,0.1,13.3,0.1,20,0.3c3.7,0.1,5.7-1.1,5.4-5.1c-0.2-2.3,0-4.7-0.1-7
|
||||
c-0.5-45-2.1-89.9-5.2-134.8C2139.2,780.1,2132.7,721.9,2120.4,664.4z M878.7,1138.5c-36,15-69.8,32.9-102.3,53.1
|
||||
c-3.4,2.1-4.9,4.7-5.7,8.6c-2.8,14.4-5.7,28.8-12.2,42c-3.6,7.3-7.7,14.4-17.3,14.4c-9.6,0-13.4-7.4-17.4-14.4
|
||||
c-1.9-3.2-2-7.6-5.4-9.8c-22.3,17.4-41.6,37.2-58.8,58.9c-5.8,7.3-10.2,15.3-12,24.8c-3.8,20.3-9.1,40.2-16.7,59.5
|
||||
c-2.6,6.7-1.3,13.3-1.5,20.3c1.5-0.4,2-0.3,2.1-0.5c3.2-3.8,6.4-7.6,9.4-11.6c31.2-40.5,66-77.4,105.4-110.2
|
||||
c36.9-30.8,75.6-59.1,117.2-83.1c54.3-31.2,110.9-57.4,170.8-76.4c38.4-12.2,77.4-21.8,117-29.2c9.5-1.8,19.1-3,28.6-4.6
|
||||
c1.5-0.3,3.8,0.3,4.4-1.5c1.4-3.9,0.6-8-0.4-11.7c-1-3.6-4.5-1.7-6.9-1.5c-40.6,3.2-80.4,11-120.4,18c-7,1.2-10.4,4.1-10.9,11.2
|
||||
c-0.3,3.8-0.5,9.1-5.8,9c-4.6-0.1-4.7-5-5.3-8.5c-0.7-4.5-2.4-6-7.3-4.9c-39.3,9.1-78,19.9-115.5,34.7c-4.6,1.8-7.1,4.2-7.8,8.9
|
||||
c-0.9,5.9-2.2,11.8-5.2,17.1c-4.1,7.1-10.6,7.2-14.6,0C881.9,1147.7,880.7,1143.4,878.7,1138.5z M1032.8,891.9
|
||||
c0.9-3.8,1.6-6.7,2.4-9.5c0.7-2.3,1.7-4.6,4.4-4.9c3.4-0.3,4.4,2.3,5.3,4.9c0.4,1.3,0.8,2.6,0.8,3.9c0.1,7.7,4.5,10.4,11.7,11.6
|
||||
c32.5,5.5,64.8,11.9,97.5,15.7c7.9,0.9,15.9,1.7,23.8,2.4c1.9,0.2,4,0.7,5.3-1.6c3.5-6.5,0.3-13-7-14.2c-1-0.2-2-0.2-3-0.4
|
||||
c-42.6-5.8-84.2-15.9-125.4-28.2c-41.8-12.4-82.5-28.2-121.6-47.1c-51-24.6-99.8-53.3-144.7-87.9c-20.6-15.8-40.9-32.1-59.9-49.7
|
||||
c-29.1-26.9-56.4-55.7-80.1-87.7c-3.7-4.9-7.7-9.6-11.6-14.3c-0.8,7.6-2.4,14.7,0.5,21.8c7.4,18.2,12.4,37.2,16,56.5
|
||||
c1.7,9.1,5,17.3,10.6,24.4c9.2,11.4,18.5,22.8,28.7,33.3c9.9,10.1,21.1,19,32.2,28.9c3.8-6.5,5.6-13,9.8-18.2
|
||||
c7.7-9.4,17.7-9.5,25.3-0.1c3.6,4.5,5.7,9.7,7.8,15c4.3,10.9,6.8,22.3,8.9,33.7c0.9,5.2,3.3,8.1,7.5,10.7
|
||||
c21.9,13.4,44.1,26,67.3,37.1c10.9,5.2,22.1,10,33.5,15.2c1.8-4.8,3.1-8.6,4.7-12.2c1.6-3.5,4.2-6.3,8.3-6.1
|
||||
c3.6,0.2,5.8,2.9,7.3,6.1c2.4,5.2,3.8,10.6,4.8,16.2c0.8,4.6,2.9,7.2,7.7,9C950.6,871.5,990.8,883,1032.8,891.9z M1953.8,1233.7
|
||||
c-2.9,5.8-4.9,11.5-8.8,16.2c-7.6,9.3-17.8,9.3-25.3,0c-3.4-4.2-5.5-9-7.5-14c-4.4-11.5-7.2-23.5-9.4-35.7
|
||||
c-0.7-3.9-2.4-6.4-5.8-8.6c-30.4-19.4-62.4-35.6-95-50.8c-6.9-3.3-7-3-9.9,4.7c-0.7,1.9-1.5,3.7-2.3,5.5c-1.6,3.1-3.7,5.6-7.6,5.6
|
||||
c-3.8,0-5.9-2.5-7.6-5.6c-2.9-5.4-4.3-11.3-5.1-17.2c-0.7-5-3-7.3-7.5-9.1c-37.8-15.2-77.3-24.6-116.7-34.3c-3.8-0.9-5.4,0.2-6.2,4
|
||||
c-0.7,3.9-0.2,9.9-6.3,9.5c-5-0.3-5.1-6.1-5.3-9.6c-0.4-6.6-3.3-8.9-9.6-10c-32.8-5.6-65.4-12.2-98.6-15.7
|
||||
c-8.3-0.9-16.5-2.1-24.8-2.7c-5.4-0.4-7.1,1.2-7.5,6.6c-0.3,3.9-1.3,8,5.2,8.4c6.3,0.4,12.4,2.1,18.7,3.1
|
||||
c47.7,7.5,94.4,19.1,140.3,34.3c42.4,14,83.5,31.2,123,52c46.3,24.4,90.2,52.6,131,85.6c48.5,39.1,93.7,81.2,129.4,132.6
|
||||
c1.9,2.8,4.8,7.5,7.3,6.7c4.5-1.5,1.4-6.8,2.1-10.4c0.6-3.4-0.5-6.6-1.8-9.7c-7.8-19.5-12.6-40-17.1-60.4c-1.1-5.1-3-9.4-6-13.5
|
||||
c-17-22.8-36.7-43.1-57.6-62.2C1959.6,1237.1,1957.6,1234.7,1953.8,1233.7z M2043.6,585.1c-4.2,2.7-6.2,6.2-8.5,9.2
|
||||
c-29.9,40.3-64.6,76.1-102.5,109c-33.8,29.3-69.5,56.2-107.6,79.4c-51.8,31.6-106.3,57.7-163.9,77.3
|
||||
c-40.7,13.8-82.1,24.9-124.2,33.1c-13.7,2.7-27.6,4.7-41.3,7c-2.2,0.4-5.3-0.1-6.2,2.4c-1.4,3.8-1.1,8.2,0.3,11.7
|
||||
c1.4,3.5,5.3,1.6,8.1,1.3c40.1-4.2,79.8-10.7,119.5-18.1c6.2-1.2,10.4-2.7,10.5-10.2c0-3.5,0.3-9.3,5.4-9.5c6-0.3,5.5,5.7,6.2,9.6
|
||||
c0.7,3.9,2.3,4.9,6.1,3.9c10-2.7,20-5.1,30-7.5c28.5-6.9,56.4-15.6,83.9-25.7c6.8-2.5,10.2-6.3,11-13.3c0.5-5,2-9.8,4.5-14.2
|
||||
c4.2-7.6,10.8-7.6,15.1,0.1c1.6,2.9,2.5,6.2,3.4,9.3c0.9,2.9,2.5,3.5,5.2,2.2c2.4-1.2,5-1.9,7.3-3.1c26.6-13.1,53.2-26,78.7-41.2
|
||||
c11.9-7.1,19.6-14.7,21.3-29.3c1.4-12.4,5.5-24.8,12.8-35.4c8.1-11.7,19.5-11.7,27.5,0.1c3.1,4.6,5.4,9.7,8.2,14.9
|
||||
c1.9-1.4,3.6-2.5,5.1-3.8c22.2-19.8,42.7-41.1,60.6-64.9c2.6-3.5,4.3-7.3,5.2-11.7c4.5-21.2,9.6-42.2,17.6-62.4
|
||||
C2045.3,599.2,2044.5,592.6,2043.6,585.1z M777.2,1163.8c1.2,0.2,1.6,0.4,1.9,0.3c30.4-14.3,60.9-28.3,92.6-39.6
|
||||
c4.2-1.5,4.4-3.7,3.9-7.5c-4.5-37.4-6.3-75-6.5-112.6c0-7.8-0.2-7.9-8.5-7.9c-22,0-44,0-65.9,0c-8.6,0-8.8,0.1-8.8,8.7
|
||||
c-0.4,45.3-2.2,90.5-6.6,135.6C778.4,1148.4,776.3,1155.9,777.2,1163.8z M776.8,817.8c0.3,3.8,0.5,7.1,0.9,10.4
|
||||
c2.5,21.2,4.1,42.4,5.5,63.7c1.8,28.6,2.3,57.2,2.7,85.9c0.1,7.1,0.3,7.3,7.2,7.3c22.6,0.1,45.3,0.1,67.9,0.1
|
||||
c8.8,0,8.1-0.2,8.1-8.3c0.1-37,1.9-73.9,6.4-110.6c0.7-5.8-0.5-8.2-6-10.1c-29.5-10.3-58-23.3-86.2-36.8
|
||||
C781.5,818.4,779.9,817.2,776.8,817.8z M1896.4,1163.9c1-2.7,0.2-5.3-0.1-7.9c-2.4-19.8-3.9-39.8-5.3-59.7
|
||||
c-2.1-30.3-2.5-60.6-3.3-90.9c-0.2-8.7-0.2-8.8-8.6-8.8c-16.7-0.1-33.3,0-50-0.1c-6.3,0-12.7,0.2-19-0.1c-3.9-0.1-5.5,1.4-5.2,5.3
|
||||
c0.1,1.7,0.1,3.3,0,5c-1.1,36-2,72-6.4,107.7c-0.9,7.7-1.1,7.9,6.3,10.6c28.5,10.4,56.1,22.9,83.3,36.1
|
||||
C1890.6,1162.4,1893.1,1164.4,1896.4,1163.9z M1896.8,817.7c-2.5-0.3-4.3,0.6-6.1,1.5c-6,2.9-12,5.6-18,8.6
|
||||
c-22.1,10.9-44.9,20.1-67.9,28.7c-7.5,2.8-7.3,3-6.5,10.6c1,8.9,1.9,17.9,2.7,26.8c2.4,28.2,3.2,56.5,3.6,84.7
|
||||
c0.1,5.4,1.8,6.7,6.8,6.6c22.6-0.2,45.3-0.1,67.9-0.2c1,0,2-0.1,3,0c3.8,0.3,5.6-1.2,5.2-5.1c-0.2-1.6,0-3.3,0.1-5
|
||||
c0.7-38.6,1.7-77.2,5.2-115.6C1894.1,845.4,1895.5,831.6,1896.8,817.7z M908.3,1109.6c1.4,0,2.4,0.2,3.2,0
|
||||
c37.8-11.7,75.5-24,114.2-32.5c4.5-1,5.8-2.9,5.5-7.4c-1.3-22-2.5-43.9-2.3-65.9c0.1-7.3-0.2-7.4-8-7.4c-33.3,0-66.7,0.1-100,0.2
|
||||
c-7.7,0-7.9,0.1-7.9,7.5c0,31-1.6,61.9-4,92.9C908.8,1101,907.3,1105.2,908.3,1109.6z M1765.3,1110.4c0-2.5,0.1-3.5,0-4.4
|
||||
c-3.1-33.9-4.7-67.8-4.8-101.8c0-7.4-0.4-7.6-8.2-7.7c-33.3-0.1-66.7-0.2-100-0.2c-7.3,0-7.2,0.1-7.5,7.9c-0.7,21-1.1,42-2.3,62.9
|
||||
c-0.5,9-1.3,8.3,7.6,10.4c29.8,7.1,59.4,14.9,88.5,24.5C1747.2,1104.8,1755.7,1107.4,1765.3,1110.4z M908.8,872.2
|
||||
c-0.2,0.3-0.6,0.6-0.6,0.8c2.8,35.5,5.2,71,4.8,106.7c-0.1,4.6,1.9,5.6,6,5.6c34.7-0.1,69.3-0.1,104,0.1c4.9,0,6.2-1.8,6.1-6.4
|
||||
c-0.5-22,0.8-44,2.1-65.9c0.3-5.2-0.9-7.7-6.4-8.8c-36.9-7.6-72.4-20.1-108.5-30.5C913.8,873,911.6,871.1,908.8,872.2z
|
||||
M1765.6,872.4c-3.2-0.6-5.1,0.3-7,0.9c-22.2,6.7-44.2,14.5-66.7,20.3c-14.5,3.7-29,7.3-43.5,10.8c-4.4,1.1-6.5,2.6-6.2,8
|
||||
c1.3,21.9,1.8,43.9,2.6,65.8c0.2,7,0.2,7.1,7.1,7.1c33.6,0,67.2,0,100.8-0.1c7.5,0,7.7-0.2,7.7-7.8c0.2-24.6,0.9-49.2,2.6-73.8
|
||||
C1763.8,893.4,1764.7,883.1,1765.6,872.4z M725.8,1192.5c8.7-4.8,15.8-9.8,23.8-12.8c9.6-3.7,12.7-10.5,13-19.7
|
||||
c0.1-1.3,0.2-2.7,0.4-4c3.4-27.1,5.6-54.3,6.7-81.5c1-23.6,1.5-47.2,2-70.8c0.2-7-0.1-7-7.3-7.1c-6.7-0.1-13.3,0-20,0
|
||||
c-9,0-18,0.2-27,0c-4.2-0.1-6.2,1.3-5.9,5.7c0.2,2.3,0,4.7,0.1,7c0.8,27.6,1.2,55.2,3,82.8C717,1125.1,719.9,1158.2,725.8,1192.5z
|
||||
M1946.2,1191.7c2.9-2.2,2.4-4.7,2.8-6.9c4.5-26.6,6.9-53.5,8.9-80.4c2.5-33.9,3.2-67.9,3.7-101.8c0.1-5-1.8-6.2-6.4-6.1
|
||||
c-15.3,0.2-30.6,0-46,0.1c-7.6,0-7.8,0.2-7.7,7.8c0.7,44,2,87.9,6.7,131.6c1.1,10.3,2.2,20.5,3.5,30.7c0.4,2.8,0.2,6,3.5,7.8
|
||||
C1925.8,1180.2,1936.2,1186.1,1946.2,1191.7z M1947.7,789.1c-9.1,5-16.9,10.1-25.3,13.8c-8.2,3.6-10.5,9.5-11.5,17.8
|
||||
c-3,27.1-5.6,54.3-6.9,81.6c-1.2,25.3-1.5,50.6-2.5,75.9c-0.2,5.1,1.5,7,6.7,7c15-0.3,30-0.1,45-0.2c8.6,0,8.4,0.6,8.3-8.9
|
||||
c-0.3-30.6-1-61.3-3.1-91.9C1956.3,852.9,1953.6,821.8,1947.7,789.1z M726,789.2c-3.3,16.4-5.1,31.3-6.7,46.2
|
||||
c-4.6,42.4-6.3,85-7.3,127.7c-0.1,5.7,0,11.3-0.2,17c-0.1,3.4,1.4,4.8,4.7,4.8c17,0,34,0,51,0.1c3.6,0,4.6-1.9,4.4-5
|
||||
c-0.1-2.3-0.1-4.7-0.1-7c-0.5-37.3-1.7-74.6-5-111.8c-1.4-15.9-2.8-31.9-5.5-47.6c-0.4-2.6-0.3-5.2-3.2-6.8
|
||||
C747.6,801.1,737.2,795.4,726,789.2z M1625.1,910.5c-9.8,1.8-18,3.4-26.2,4.6c-23.3,3.5-46.2,9.9-70,10
|
||||
c-10.9,0.1-21.8,2.4-32.7,3.7c-3.5,0.4-6.2,1.1-6,6c0.5,15,0.7,30,0.7,44.9c0,4.4,1.7,5.8,5.9,5.7c8.3-0.2,16.6,0,25-0.1
|
||||
c30.6,0,61.3,0,91.9-0.1c9,0,9-0.1,9.2-9.4c0.2-13,0.2-26,0.6-38.9C1623.7,928.4,1624.5,919.8,1625.1,910.5z M1625.2,1071.1
|
||||
c-2.2-23.8-2.3-46.1-2.4-68.4c0-4.5-1-6.6-6.1-6.6c-39.9,0.1-79.9,0.1-119.8-0.1c-5.1,0-6.1,2-6.1,6.5c0,14.6-0.1,29.3-0.6,43.9
|
||||
c-0.1,4.5,1.6,5.6,5.6,6.1c12.6,1.4,25,3.8,37.7,4c8,0.1,15.9,1.1,23.8,2.5C1579.5,1063,1601.7,1066.9,1625.2,1071.1z
|
||||
M1049.3,1070.4c2.6,1,4.9-0.1,7-0.6c23-5.5,46.3-8.9,69.7-12.3c16.7-2.5,33.7-3.8,50.6-5.2c5.1-0.4,6.8-1.9,6.6-7
|
||||
c-0.4-14-0.3-27.9-0.5-41.9c-0.1-7-0.2-7.2-7.2-7.2c-39.3,0-78.5,0-117.8,0.1c-6.7,0-6.8,0.2-6.8,7.5c0,16.3-0.5,32.6-1.4,48.9
|
||||
C1049.3,1058.6,1048.4,1064.5,1049.3,1070.4z M1049.8,910.2c-0.4,1.2-0.8,1.8-0.8,2.4c0.6,22.6,2.5,45.2,1.9,67.8
|
||||
c-0.1,4.2,2.1,4.8,5.6,4.8c39.9,0,79.9-0.1,119.8,0.1c5.5,0,6.7-2.3,6.6-7.2c-0.2-11,0.1-22,0.2-32.9c0.1-15.9,0.1-15.5-16.3-17.4
|
||||
c-16.5-1.9-33.1-2.4-49.6-5.2C1094.7,918.9,1072,915.9,1049.8,910.2z M1260.5,985.4c21.6,0,43.2,0,64.9,0c9.1,0,9.1-0.1,9.1-9.4
|
||||
c0-10.3,0-20.6-0.1-30.9c-0.1-7.7-0.1-7.7-7.6-7.9c-2.3-0.1-4.7,0-7,0c-41.9,0-83.8-2.2-125.5-6.2c-4.6-0.4-5.9,0.9-6,5.1
|
||||
c-0.2,13.6-0.6,27.3-0.8,40.9c-0.1,8.3,0.1,8.4,8.1,8.4C1217.3,985.4,1238.9,985.4,1260.5,985.4z M1485,931.5
|
||||
c-3.1-1.4-5.4-0.6-7.7-0.4c-26.2,1.9-52.4,4.3-78.7,5.1c-17.3,0.5-34.6,1.3-51.9,1.1c-7.4-0.1-7.5,0.1-7.6,7.8
|
||||
c-0.1,11.7,0,23.3-0.1,35c0,3.7,1,5.8,5,5.5c2-0.1,4,0,6,0c41.6,0,83.2,0,124.9,0c2,0,4-0.1,6,0c3.3,0.1,5-1.2,4.9-4.6
|
||||
C1485.5,964.2,1485.3,947.6,1485,931.5z M1485,1051.1c0.3-17,0.5-33.6,0.9-50.1c0.1-4.2-2.2-4.8-5.7-4.8c-45,0.1-89.9,0.1-134.9,0
|
||||
c-4.7,0-6.5,1.4-6.3,6.2c0.3,12,0.4,24,0,36c-0.2,5.2,2.1,6,6.5,6c11.6-0.1,23.3,0.1,34.9,0.4
|
||||
C1415,1045.8,1449.5,1048.1,1485,1051.1z M1188.4,1050.6c47.5-3.4,93.2-6.8,139.2-6.4c0.7,0,1.3,0,2,0c3.2,0.2,4.9-0.9,4.9-4.5
|
||||
c-0.1-13,0-25.9,0-38.9c0-3.4-1.6-4.8-4.9-4.6c-1.7,0.1-3.3,0-5,0c-42.6,0-85.1,0-127.7,0c-1.7,0-3.3,0.1-5,0
|
||||
c-2.8-0.1-4.4,1.1-4.4,4.1C1187.9,1016.8,1188.1,1033.4,1188.4,1050.6z M2010.7,1230.1c0.9-7.3-0.9-13.8-1.5-20.3
|
||||
c-6.8-66-10.3-132.2-10.7-198.6c-0.1-16.1-0.3-16.3-16.8-14.7c-2.9,0.3-4.2,1.1-4.1,4.1c0.1,3,0,6,0,9
|
||||
c-0.3,41.3-2.1,82.6-6.1,123.8c-2,20.2-4.3,40.4-8.2,60.4c-0.9,4.8-0.2,7.8,4.5,10.4c10.2,5.7,19.9,12.2,29.8,18.3
|
||||
C2001.8,1225,2005.6,1228.2,2010.7,1230.1z M2011.2,752.5c-2.7-0.8-3.6,0.3-4.7,1c-13.3,8.3-26.4,16.7-39.8,24.7
|
||||
c-3.6,2.1-4.2,4.5-3.5,8.2c1.4,7.8,2.8,15.7,3.9,23.6c7.4,53.8,10.1,107.9,10.6,162.2c0.1,14.1-2.6,12.8,13.7,12.8
|
||||
c7,0,7-0.2,7.1-7.3c0.4-27.3,0.6-54.6,1.8-81.8c1.4-30.9,3.1-61.8,6.1-92.6C2008,786.3,2009.6,769.5,2011.2,752.5z M662.1,1231.5
|
||||
c3.7-2.4,5.6-3.7,7.5-4.9c11.6-7.7,22.9-16.2,35.5-22.2c5.3-2.5,6.2-5.5,5.1-11c-4.6-22.5-7-45.4-9-68.3
|
||||
c-3.5-39.9-5-79.8-5.4-119.8c-0.1-8.6-0.2-8.6-8.8-8.7c-1.3,0-2.7,0-4,0c-7.9,0.1-8,0.1-7.9,8.4c0,24.7-0.6,49.3-1.6,74
|
||||
c-1.3,32-3,63.9-6,95.8C665.8,1192.9,664.1,1211.1,662.1,1231.5z M662.9,752.1c-1,1.4-0.4,3-0.3,4.6c2,20.5,4.6,41.1,6,61.7
|
||||
c3.8,52.5,6.5,105.1,6.4,157.8c0,8.8,0.1,8.9,8.5,8.9c14.5-0.1,12.1,1.3,12.3-12.1c1.1-62.7,3.5-125.2,14.8-187
|
||||
c0.8-4.1-0.9-6-4.2-7.7c-13.4-6.8-25.6-15.5-38-23.9C666.8,753.1,665.5,751.1,662.9,752.1z M2044.3,728.6c-5.4,1.1-7.1,3.9-7.7,8.3
|
||||
c-1.7,14.9-4,29.7-5.6,44.6c-4.6,42.1-7.2,84.2-8.9,126.5c-1,25.3-1.3,50.5-1.3,76.8c6.5,0,12.4-0.1,18.3,0
|
||||
c3.8,0.1,5.5-1.4,5.2-5.3c-0.2-1.7,0-3.3,0-5c0-78.9,0-157.9,0-236.8C2044.3,734.9,2044.3,732,2044.3,728.6z M2044.3,1253
|
||||
c0-3.7,0-6.4,0-9c0-78.6,0-157.2,0-235.8c0-13,1.3-11.5-11.9-11.5c-13,0-11.5-1.8-11.5,11.8c0,59.3,3.1,118.5,8.7,177.6
|
||||
c1.9,19.5,4,39.1,7,58.5C2037.3,1248.9,2038.8,1251.8,2044.3,1253z M629.2,728.7c0,4.3,0,8,0,11.6c0,76.5,0,153.1,0,229.6
|
||||
c0,15.8,0,15.8,16,15.3c7.6-0.2,7.6-0.2,7.7-7.7c0.1-3.3,0-6.7,0-10c-0.8-55.6-3.1-111.1-8.5-166.4c-2-21.2-5-42.3-7.3-63.4
|
||||
C636.6,733.3,635,730.3,629.2,728.7z M629.2,1253.2c5.3-2.1,7.4-4.5,7.9-9.2c1.6-14.9,4-29.7,5.5-44.6
|
||||
c6.5-62.4,9.5-124.9,10.3-187.6c0.2-15.6,0.1-15.6-15-15.6c-9,0-8.8-0.6-8.7,8.6c0,2,0,4,0,6c0,75.7,0,151.3,0,227
|
||||
C629.2,1242.5,629.2,1247.2,629.2,1253.2z M2060,697.5c19-15.9,36.7-29.3,52.3-45.3c3.5-3.5,3.7-6.7,2.4-10.7
|
||||
c-2.8-8.9-5.4-17.8-8.7-26.5c-3.5-9.3-7.4-18.5-13.7-26.4c-5.3-6.6-8-6.6-13.4-0.4c-1.3,1.5-2.7,3-3.6,4.7
|
||||
c-8.9,15.7-17.3,31.5-15.3,50.5c0.4,3.6,0.1,7.3,0.1,11C2060,668,2060,681.5,2060,697.5z M613.2,1283.1
|
||||
c-20.8,13.6-36.9,29.9-53.8,45.4c-2.8,2.6-2.3,5.1-1.5,8.1c4.1,14.7,8.6,29.3,15.3,43.2c2.6,5.4,5.4,10.7,9.8,15
|
||||
c4.3,4.2,6.2,4.6,10,0.4c10.7-11.9,18.4-26.3,19.7-41.7C614.7,1330.6,613.2,1307.6,613.2,1283.1z M2060,1284.4
|
||||
c0,19.5,0.7,36.8-0.2,54c-1.1,19.1,6.3,35.2,15.8,50.5c7.4,12.1,12.1,11.7,19.8-0.6c9.7-15.4,14.6-32.7,19.7-50
|
||||
c1.1-3.7,0-6.2-2.6-8.8C2096.8,1313.6,2079.3,1299.9,2060,1284.4z M611.9,698.2c2.1-3.3,1.3-5.9,1.3-8.4c0.1-16-0.8-32,0.3-47.9
|
||||
c1.2-18.4-6.2-33.8-15.1-48.6c-7.6-12.7-12.6-12.4-20.4,0.3c-10.1,16.3-15.1,34.6-20.4,52.8c-1,3.6,0.6,5.5,3,7.7
|
||||
c8.8,8.1,17.3,16.7,26.3,24.5C594.8,685.4,603.3,691.5,611.9,698.2z M1788.2,864c-12.5,3.2-12.6,3.2-13.9,13.8
|
||||
c-0.9,7.6-1.6,15.2-2.1,22.8c-1.8,26.2-2.9,52.5-3.2,78.8c0,4,0.8,6.3,5.4,6c4.6-0.3,9.3-0.2,14,0c5.1,0.1,6.9-2.1,6.7-7.3
|
||||
c-0.5-11.3-0.2-22.6-0.7-33.9C1793.2,917.5,1792.3,890.9,1788.2,864z M885.3,1118.4c12.8-3,12.5-3,13.8-13.9
|
||||
c3.5-29.5,4.6-59.1,5-88.8c0.3-23.5,5.2-18.7-19.8-19.4c-4.4-0.1-5.7,1.5-5.7,5.8C878.8,1040.8,880.3,1079.4,885.3,1118.4z
|
||||
M885.7,863.5c-0.2,0.9-0.6,1.9-0.7,2.9c-4.3,33.7-6,67.5-6.1,101.4c0,5.5-2.5,12.6,1.3,16.2c3.8,3.6,10.8,1.1,16.3,1.2
|
||||
c7.9,0.1,8.1-0.1,7.9-8.4c-0.6-33.2-1.5-66.4-5.3-99.4C897.9,865.6,898,865.6,885.7,863.5z M1788.4,1117.5
|
||||
c4.4-34.4,5.9-68.6,6.5-102.9c0.1-5.6,2.4-12.7-0.9-16.5c-3.9-4.3-11.3-1.4-17.2-1.6c-7.6-0.2-7.7,0.1-7.7,7.6
|
||||
c0,24.7,1.3,49.3,2.8,73.9c0.6,10.3,2.2,20.5,2.9,30.8c0.3,3.6,1.7,5.5,4.9,6.5C1782.2,1116,1784.4,1117.7,1788.4,1117.5z
|
||||
M1481.1,1078.1c0-2.7,0.1-4.4,0-6c-0.3-7.8-0.3-8.1-8.2-8.7c-22.9-1.7-45.8-3.3-68.7-4.7c-19.9-1.2-39.9-0.9-59.9-1
|
||||
c-4.6,0-5.5,2-5.4,6c0.1,3.7,0.3,6.7,5.2,6.2c1.3-0.1,2.7,0,4,0c34.3-0.3,68.4,1.9,102.5,5.6
|
||||
C1460.6,1076.6,1470.4,1078.5,1481.1,1078.1z M1480,903c-15.3,1.6-30.8,3.6-46.3,4.8c-29.5,2.4-59.1,4.7-88.8,3.9
|
||||
c-5.1-0.1-6.1,2-6,6.5c0.1,4.1,1.1,6.3,5.6,5.8c1.3-0.2,2.7,0,4,0c42.3,0.3,84.5-2.6,126.7-5.9c4.2-0.3,6.2-1.6,5.8-5.9
|
||||
C1480.7,909.3,1482,906.3,1480,903z M1192.2,1078.2c3.1-0.2,5.4-0.3,7.6-0.5c37-4.6,74-7.6,111.3-7.8c6,0,12-0.1,18,0
|
||||
c3.7,0.1,5.3-1.1,5.4-5.1c0.1-6.5,0-6.9-7.3-7c-40.6-0.8-81.1,1.7-121.6,4.6C1193.1,1063.3,1193.2,1063.6,1192.2,1078.2z
|
||||
M1192.6,903.2c0.2,14.4,0.2,14.4,12.8,16c3,0.4,6,0.6,8.9,0.8c19.3,1.1,38.5,2,57.8,3.2c18.6,1.2,37.2,0.8,55.9,0.8
|
||||
c6.4,0,6.3-0.5,6.6-6.8c0.2-5.8-3.3-5.5-7.1-5.4c-28.3,0.8-56.5-1.5-84.7-3.6C1226.3,907,1209.8,905.4,1192.6,903.2z M653.6,694.6
|
||||
c2,12.3,3.8,22.4,5.1,32.6c0.5,3.7,1.7,6.1,4.8,8c14.2,8.8,27.5,18.8,42.2,26.6c2.7,1.4,6.2,6.3,8.9,2.4c2.7-3.7-2.6-6.2-5-8.5
|
||||
c-19-17.3-36.7-35.8-51.8-56.7C657,698.1,656,697.3,653.6,694.6z M654,1284.6c0.8,0,1.3,0.1,1.5,0c1.4-1.4,2.9-2.7,4-4.3
|
||||
c14.6-20.8,32.8-38.3,51.2-55.5c1.2-1.1,2.6-2.1,3.6-3.4c1.4-1.7,1.5-3.6-0.2-5.7c-2.9,0.3-5.3,2.1-7.8,3.6
|
||||
c-10.2,6.3-19.8,13.5-30.5,18.8c-12.9,6.4-19.3,15.7-19.2,30.2C656.6,1273.7,654.9,1279.1,654,1284.6z M2019.1,698.8
|
||||
c-0.5-0.3-1-0.6-1.5-0.9c-1,1.2-2,2.4-3,3.6c-6.8,9-13,18.4-21.4,26.1c-11,10.1-20.5,21.7-32.4,30.8c-1.9,1.5-3.6,3.2-2.1,5.9
|
||||
c2,3.5,4.1,0.5,5.7-0.4c15.4-9.7,30.8-19.6,46.1-29.5c2.3-1.5,4-3.1,4.4-6.2C2016.2,718.5,2017.7,708.6,2019.1,698.8z
|
||||
M2017.1,1283.9c0.7-0.2,1.4-0.5,2.1-0.7c-1.5-10.1-3-20.2-4.4-30.4c-0.4-2.6-1.9-4.1-3.9-5.4c-15.6-10-31.3-20.1-46.9-30
|
||||
c-1.5-1-3.3-2.8-5-0.5c-1.7,2.4-0.3,4.2,1.7,5.9c4,3.4,8.1,6.9,11.8,10.6C1988.4,1249.3,2005.3,1264.3,2017.1,1283.9z M1044.6,1034
|
||||
c0.2,0,0.3,0,0.5,0c0-11-0.1-21.9,0.1-32.9c0.1-4.2-2-4.8-5.5-4.9c-4.5-0.1-4.6,2.6-4.6,5.9c0.1,22.3,1.2,44.5,2.4,66.8
|
||||
c0.1,1,0.2,2,0.4,3c0.2,1.2,1.2,1.8,2.1,1.7c0.8-0.1,1.9-1,2.2-1.7c0.5-1.5,0.7-3.2,0.8-4.9C1043.5,1055.9,1044,1045,1044.6,1034z
|
||||
M1636.4,1033.8c0.7,0,1.4,0,2.1,0c0-10.6,0-21.2,0-31.8c0-2.7,0.2-5.3-3.6-5.8c-4.7-0.6-6,0.4-6,6.1c-0.1,21.9,0.2,43.7,2.1,65.6
|
||||
c0.1,1.3,0.1,2.7,0.6,3.9c0.3,0.7,1.5,1.6,2.2,1.5c0.7,0,1.7-1.1,1.9-1.8c0.5-1.9,0.7-3.9,0.7-5.9
|
||||
C1636.4,1055,1636.4,1044.4,1636.4,1033.8z M1638.5,948.1c-0.2,0-0.5,0-0.7,0c-0.5-11.6-1-23.3-1.6-34.9c-0.1-1.9,0.2-4.8-2.3-4.9
|
||||
c-3.3-0.1-2.8,3-3,5.2c-0.3,3-0.4,6-0.7,9c-1.5,19-1.4,37.9-1.4,56.9c0,5.3,2,6.8,6.4,5.8c3.8-0.8,3.3-3.6,3.3-6.2
|
||||
C1638.5,968.8,1638.5,958.4,1638.5,948.1z M1045.1,948c-0.1,0-0.3,0-0.4,0c-0.6-11.9-1.2-23.9-1.9-35.8c-0.1-1.9-0.4-4.3-3.2-3.9
|
||||
c-2.2,0.3-1.9,2.4-2,4c-1.3,22.5-2.3,45.1-2.5,67.7c0,3.7,1.2,5.4,5.1,5.4c4.1,0.1,5-1.9,5-5.5C1045,969.3,1045.1,958.7,1045.1,948
|
||||
z M741.9,740.1c-4.9,5.5-6.3,11-8.3,16.2c-0.8,2-0.8,4,1.2,5.3c6.2,4.1,11.6,9.4,19.7,13C751.6,762.1,748.8,750.9,741.9,740.1z
|
||||
M1931.7,1241.3c11.7-16.6,11.5-18.2-3-28.3c-2.7-1.9-4.9-4.5-9.1-4.9C1921.9,1219.6,1924.7,1230.7,1931.7,1241.3z M1931.6,740
|
||||
c-6.6,10.9-9.8,22-12.1,33.9c4-0.9,6.1-3.1,8.5-4.8C1943.5,758.3,1943.6,757.2,1931.6,740z M742.1,1241.3
|
||||
c6.6-11.1,9.8-21.9,12.2-34.3c-8,3.6-13.2,8.8-19.2,12.6c-2,1.3-2.3,3.1-1.6,5.1C735.6,1230.2,736.7,1236.2,742.1,1241.3z
|
||||
M597.6,1464.4c16.9-6.5,17.3-8.9,14-24.4C605.8,1447.2,601.7,1455,597.6,1464.4z M2061.5,1442.5c-5.1,14.3,5.6,17.3,12.5,22.8
|
||||
C2072.1,1457,2066.2,1451,2061.5,1442.5z M1781.6,836.9c-1,5.2-3.6,8.5-1.8,12.6C1785.7,846.2,1785.7,846.2,1781.6,836.9z
|
||||
M895.4,849.3c-1.5-4.5-1.6-7.9-3.8-11.1C887.9,846.3,887.9,846.3,895.4,849.3z M630.1,1284.6c-0.4-5.3,4.1-10.2,1.8-17
|
||||
C627.4,1273.6,629.9,1279.2,630.1,1284.6z M629.9,697.1c0.2,5.4-3,11.4,2.4,16.6C633.8,707.4,630.4,702.5,629.9,697.1z
|
||||
M2044.5,1284.5c-1-5.2,1.5-10.3-2.8-15C2040.3,1275.4,2042.4,1279.6,2044.5,1284.5z M2041.9,712.1c4.1-4.8,1.6-10,2.2-14.6
|
||||
C2043,702.1,2039.8,706.2,2041.9,712.1z M892.2,1143.9c0.4-4.2,3.5-7.3,1.6-11.5C887.6,1136,887.6,1136,892.2,1143.9z
|
||||
M1779.6,1132.2c-1,2.9-0.4,4.8,0.4,6.5c0.7,1.5,0.4,3.6,2.3,4.6C1785.7,1135.3,1785.7,1135.3,1779.6,1132.2z M729.5,774
|
||||
c1.5,1.6,2.6,4,5.8,3.2C733.9,774.7,732.2,773.5,729.5,774z M737.9,778.4c-0.2,0.2-0.5,0.4-0.7,0.6c0.2,0.2,0.4,0.6,0.6,0.6
|
||||
c0.2,0,0.5-0.3,0.7-0.5C738.4,778.8,738.1,778.6,737.9,778.4z"/>
|
||||
<path class="st1" d="M534.9,1353.5c4.5,11.6,7.4,22.2,12.2,32c4.7,9.6,9.8,18.8,17.1,26.8c10.6,11.6,28.8,15.5,42.7,2.9
|
||||
c1.4-1.3,3.1-2.4,5.5-4.2c1.4,7.2,1.7,13.2-2.5,19.1c-8.4,11.7-14.8,24.4-20.8,37.4c-1.7,3.7-3.7,6.7-7.3,9.1
|
||||
c-52.7,33.7-105.3,67.6-158,101.4c-1.1,0.7-2.1,1.7-4.1,1.1c0-195.1,0-390.4,0-585.6c2.1-1.2,4.1-0.9,6-0.8
|
||||
c21.6,0.5,43.3,1.1,64.9,1.3c5.3,0,6.7,1.8,6.7,6.9c0.5,40,1.4,79.9,3.9,119.8c2.4,38.2,5.6,76.4,10.6,114.3
|
||||
c4.6,34.7,10.1,69.1,19.2,103c1.1,4,0.6,7.1-2.3,10c-0.9,0.9-1.7,2.1-2.6,3.1c-22.1,25.1-40.6,52.7-57.5,81.5
|
||||
c-11.6,19.7-19.9,41-28.9,61.9c14.9-23.9,25.7-50.1,41.8-73.5C497.6,1397.6,514.9,1375.3,534.9,1353.5z"/>
|
||||
<path class="st1" d="M534.6,627.2c-40-41.2-69.6-89-94.2-140.1c0.7,6.2,4,11.2,6.2,16.6c12.7,31.7,30.5,60.6,49.6,88.7
|
||||
c9.3,13.7,20.4,26.3,31,39.1c3.8,4.5,5,8.7,3.4,14.4c-9.6,35.4-15,71.6-19.6,107.9c-9.4,74.5-12.5,149.3-13.7,224.3c0,1-0.1,2,0,3
|
||||
c0.6,4.7-1.2,6.5-6.1,6.4c-8.3-0.2-16.7,0.2-25,0.4c-13.7,0.3-27.3,0.4-41,1c-4.8,0.2-6.8-1.2-6.2-6.1c0.3-2,0-4,0-6
|
||||
c0-187.9,0-375.8,0-563.7c0-3.5,0-7.1,0-10.7c4.2-0.1,6.3,2.3,8.7,3.9c50.4,32.4,100.8,64.9,151.3,97.1c5,3.2,8.4,7.1,10.8,12.4
|
||||
c5.1,11.2,10.4,22.4,17.8,32.3c4.9,6.6,6.7,13.5,5.1,21.9c-3.6-0.1-5.6-3.1-7.9-5c-12-9.8-28.1-7.7-38.9,2.8
|
||||
c-12.6,12.2-19,27.9-25.2,43.7C538.8,616.5,538.2,622.2,534.6,627.2z"/>
|
||||
<path class="st1" d="M2060.9,1411.1c7.4,4.3,12.3,10.3,20.6,11.4c9.8,1.4,17.7-1.4,24.8-7.5c8.5-7.3,13.9-16.9,18.8-26.8
|
||||
c5.1-10.4,8.6-21.5,12.7-32.5c24.7,24.7,73.3,78.9,107.2,169.7c-1.1-8.8-3.5-17.2-6.8-25.4c-13.2-33.9-29.3-66.4-49.3-96.8
|
||||
c-12.8-19.5-27.4-37.7-42.4-55.5c-3.4-4-4.8-7.1-3.5-12.3c12.8-48.8,19.1-98.6,24-148.7c5.9-59.7,8.4-119.6,8.9-179.7
|
||||
c0-1.3,0-2.7,0.1-4c0.2-7,0.2-7.1,7-7.4c20-0.7,40-1.3,60-1.9c2.3-0.1,4.7,0,7-0.1c3.5-0.2,4.8,1.4,4.6,4.8c-0.1,2,0,4,0,6
|
||||
c0,188.3,0,376.6,0,565c0,3.2,0,6.4,0,9.8c-4.5-0.2-6.9-2.8-9.5-4.5c-47.7-30.6-95.2-61.4-143-91.7c-9-5.7-16-11.6-19.9-22.4
|
||||
c-3.7-10.1-11.2-18.9-17.1-28.2C2060.8,1426.2,2058.6,1419.6,2060.9,1411.1z"/>
|
||||
<path class="st1" d="M2244.8,454.3c-23,66.2-59.6,121.8-106.6,172.4c-4.5-11.6-7.7-22.6-12.8-32.9c-4-8.1-8.4-15.8-14.4-22.7
|
||||
c-12.6-14.4-29.9-17.7-45.6-3.8c-1.1,1-2,2.7-4.6,2.6c-1.4-5.5-1.1-10.5,2-15.7c8.8-14.9,18.9-29,25.8-45c1-2.3,2.7-3.7,4.7-4.9
|
||||
c52.7-33.9,105.3-67.7,158-101.5c0.5-0.3,1.3-0.2,2.1-0.4c2.3,2.1,1.2,4.9,1.2,7.3c0.1,189.6,0.1,379.3,0.1,568.9
|
||||
c0,1.7-0.1,3.3,0,5c0.2,3.6-1.7,4.6-4.9,4.5c-5-0.2-10-0.3-15-0.5c-17-0.5-34-1-51-1.4c-7.4-0.2-7.5-0.2-7.6-7.8
|
||||
c-0.5-35.3-1.1-70.6-3.1-105.9c-2.6-45.9-6.4-91.7-12.8-137.3c-4.2-30.3-9.4-60.5-17.3-90.2c-1.1-4-0.2-6.6,2.5-9.8
|
||||
c27.4-31.6,51.5-65.6,70.4-102.8c11.3-22.2,21.7-44.9,28.2-69.2C2244.6,460.9,2245.4,458.8,2244.8,454.3z"/>
|
||||
<path class="st1" d="M553.7,662.8c3.8,0.2,5.2,2.8,6.9,4.6c14.5,15.2,30.3,29,46.9,41.9c4.1,3.2,5.8,6.4,5.8,11.7
|
||||
c-0.2,85.6-0.1,171.2-0.1,256.8c0,7.5-0.1,7.7-7.6,7.8c-25,0.4-50,0.6-74.9,1.1c-4.3,0.1-5.9-1.2-5.7-5.6c0.4-13,0.2-26,0.5-39
|
||||
c1.5-60.3,4.7-120.5,11.8-180.4c3.7-31.7,8.3-63.3,15.4-94.5C553,665.6,553.4,664.1,553.7,662.8z"/>
|
||||
<path class="st1" d="M553.6,1319.7c-22.4-107.4-27.4-215.3-28.7-323.3c3.3-2.5,6.4-1.5,9.3-1.5c23.3,0.3,46.7,0.7,70,1.1
|
||||
c9.5,0.1,9.2-1.2,9.2,9.2c0,72.3,0,144.7,0,217c0,13.3-0.1,26.7,0.1,40c0,3.6-0.7,6.2-3.8,8.6c-18.7,14.5-36.3,30.3-52.7,47.4
|
||||
C556.3,1318.5,555.6,1318.7,553.6,1319.7z"/>
|
||||
<path class="st1" d="M2119.8,1319.2c-6.7-6.5-12.1-12.3-18-17.5c-11.8-10.2-23.8-20.2-36-30.1c-4.2-3.4-5.9-7.1-5.9-12.7
|
||||
c0.2-84.6,0.1-169.1,0.1-253.7c0-8.4,0.1-8.5,8-8.5c24.3-0.2,48.6-0.4,72.9-0.5c7.5,0,7.7,0,7.6,7.7c-0.3,45.6-1.9,91.2-5.1,136.7
|
||||
c-2.7,39.9-6.7,79.6-12.5,119.1C2128.1,1279,2124.6,1298.3,2119.8,1319.2z"/>
|
||||
<path class="st1" d="M2120.4,664.4c12.3,57.5,18.7,115.7,22.8,174.2c3.1,44.9,4.7,89.8,5.2,134.8c0,2.3-0.1,4.7,0.1,7
|
||||
c0.3,4-1.7,5.2-5.4,5.1c-6.7-0.2-13.3-0.3-20-0.3c-18.3-0.1-36.7-0.2-55-0.4c-8-0.1-8.2-0.1-8.2-8.2c0-52.3,0-104.6,0-157
|
||||
c0-32,0.2-64-0.2-96c-0.1-6.3,2-10.3,6.8-14.3c17.1-14.3,35.1-27.7,50.2-44.2C2117.5,664.3,2118.3,663.8,2120.4,664.4z"/>
|
||||
<path class="st1" d="M878.7,1138.5c2,4.9,3.2,9.1,5.3,12.8c4,7.2,10.5,7.1,14.6,0c3-5.3,4.3-11.2,5.2-17.1c0.8-4.7,3.2-7.1,7.8-8.9
|
||||
c37.6-14.8,76.3-25.6,115.5-34.7c5-1.1,6.6,0.4,7.3,4.9c0.6,3.5,0.7,8.4,5.3,8.5c5.3,0.1,5.5-5.2,5.8-9c0.5-7.1,3.9-10,10.9-11.2
|
||||
c40-7,79.8-14.9,120.4-18c2.4-0.2,5.9-2.1,6.9,1.5c1,3.7,1.8,7.8,0.4,11.7c-0.7,1.8-2.9,1.2-4.4,1.5c-9.5,1.6-19.1,2.8-28.6,4.6
|
||||
c-39.6,7.4-78.6,17-117,29.2c-59.9,19-116.5,45.2-170.8,76.4c-41.6,24-80.2,52.3-117.2,83.1c-39.4,32.8-74.3,69.6-105.4,110.2
|
||||
c-3,3.9-6.2,7.7-9.4,11.6c-0.1,0.2-0.6,0.2-2.1,0.5c0.2-7-1.1-13.6,1.5-20.3c7.6-19.2,12.9-39.2,16.7-59.5
|
||||
c1.8-9.6,6.2-17.5,12-24.8c17.2-21.7,36.5-41.5,58.8-58.9c3.4,2.2,3.5,6.6,5.4,9.8c4,7,7.8,14.4,17.4,14.4
|
||||
c9.6,0,13.8-7.2,17.3-14.4c6.5-13.3,9.4-27.7,12.2-42c0.8-3.9,2.2-6.5,5.7-8.6C808.9,1171.4,842.7,1153.5,878.7,1138.5z"/>
|
||||
<path class="st1" d="M1032.8,891.9c-42-9-82.2-20.4-121.3-35.7c-4.8-1.9-6.9-4.4-7.7-9c-0.9-5.6-2.3-11.1-4.8-16.2
|
||||
c-1.5-3.2-3.7-5.9-7.3-6.1c-4.1-0.2-6.8,2.5-8.3,6.1c-1.6,3.6-2.8,7.4-4.7,12.2c-11.4-5.2-22.6-10-33.5-15.2
|
||||
c-23.1-11.1-45.4-23.7-67.3-37.1c-4.3-2.6-6.6-5.5-7.5-10.7c-2.1-11.4-4.6-22.8-8.9-33.7c-2.1-5.3-4.2-10.5-7.8-15
|
||||
c-7.6-9.4-17.5-9.4-25.3,0.1c-4.2,5.2-6,11.7-9.8,18.2c-11.1-9.9-22.3-18.8-32.2-28.9c-10.2-10.4-19.5-21.8-28.7-33.3
|
||||
c-5.6-7-8.9-15.3-10.6-24.4c-3.7-19.3-8.6-38.3-16-56.5c-2.9-7.1-1.4-14.1-0.5-21.8c3.9,4.8,7.9,9.4,11.6,14.3
|
||||
c23.7,32,51,60.7,80.1,87.7c19,17.6,39.4,33.9,59.9,49.7c45,34.7,93.8,63.3,144.7,87.9c39.2,18.9,79.8,34.7,121.6,47.1
|
||||
c41.2,12.2,82.8,22.3,125.4,28.2c1,0.1,2,0.2,3,0.4c7.3,1.2,10.5,7.7,7,14.2c-1.3,2.3-3.4,1.7-5.3,1.6c-8-0.7-15.9-1.5-23.8-2.4
|
||||
c-32.8-3.7-65.1-10.2-97.5-15.7c-7.1-1.2-11.5-3.9-11.7-11.6c0-1.3-0.4-2.6-0.8-3.9c-0.8-2.6-1.9-5.3-5.3-4.9
|
||||
c-2.7,0.3-3.7,2.6-4.4,4.9C1034.4,885.3,1033.7,888.2,1032.8,891.9z"/>
|
||||
<path class="st1" d="M1953.8,1233.7c3.8,1,5.8,3.4,8,5.3c21,19.1,40.7,39.4,57.6,62.2c3,4.1,4.9,8.4,6,13.5
|
||||
c4.5,20.5,9.3,40.9,17.1,60.4c1.3,3.1,2.4,6.3,1.8,9.7c-0.6,3.6,2.4,8.9-2.1,10.4c-2.5,0.8-5.4-3.9-7.3-6.7
|
||||
c-35.7-51.5-81-93.6-129.4-132.6c-40.8-32.9-84.7-61.2-131-85.6c-39.5-20.8-80.6-37.9-123-52c-45.9-15.2-92.6-26.8-140.3-34.3
|
||||
c-6.2-1-12.4-2.7-18.7-3.1c-6.5-0.4-5.5-4.5-5.2-8.4c0.4-5.4,2.1-7.1,7.5-6.6c8.3,0.7,16.5,1.9,24.8,2.7
|
||||
c33.2,3.4,65.8,10.1,98.6,15.7c6.3,1.1,9.2,3.4,9.6,10c0.2,3.6,0.3,9.3,5.3,9.6c6,0.4,5.5-5.6,6.3-9.5c0.7-3.9,2.4-5,6.2-4
|
||||
c39.4,9.7,78.9,19.2,116.7,34.3c4.5,1.8,6.8,4.2,7.5,9.1c0.9,5.9,2.2,11.8,5.1,17.2c1.7,3.1,3.8,5.6,7.6,5.6c3.8,0,6-2.5,7.6-5.6
|
||||
c0.9-1.8,1.7-3.6,2.3-5.5c2.8-7.7,2.9-7.9,9.9-4.7c32.6,15.3,64.6,31.4,95,50.8c3.4,2.2,5.1,4.7,5.8,8.6
|
||||
c2.1,12.1,4.9,24.1,9.4,35.7c1.9,5,4.1,9.8,7.5,14c7.5,9.4,17.7,9.4,25.3,0C1948.9,1245.3,1950.9,1239.5,1953.8,1233.7z"/>
|
||||
<path class="st1" d="M2043.6,585.1c0.9,7.5,1.7,14.1-0.8,20.4c-8,20.2-13.1,41.2-17.6,62.4c-0.9,4.4-2.6,8.3-5.2,11.7
|
||||
c-17.8,23.8-38.4,45.2-60.6,64.9c-1.5,1.3-3.2,2.3-5.1,3.8c-2.9-5.3-5.1-10.4-8.2-14.9c-8-11.8-19.3-11.8-27.5-0.1
|
||||
c-7.4,10.6-11.4,23-12.8,35.4c-1.7,14.6-9.4,22.2-21.3,29.3c-25.5,15.2-52.1,28.1-78.7,41.2c-2.4,1.2-5,2-7.3,3.1
|
||||
c-2.7,1.3-4.3,0.7-5.2-2.2c-1-3.2-1.9-6.4-3.4-9.3c-4.2-7.7-10.8-7.7-15.1-0.1c-2.5,4.4-3.9,9.3-4.5,14.2c-0.8,7-4.2,10.7-11,13.3
|
||||
c-27.5,10.2-55.4,18.9-83.9,25.7c-10,2.4-20.1,4.8-30,7.5c-3.8,1-5.4,0-6.1-3.9c-0.7-4-0.2-9.9-6.2-9.6c-5.1,0.2-5.3,6-5.4,9.5
|
||||
c-0.1,7.5-4.3,9-10.5,10.2c-39.6,7.4-79.4,13.9-119.5,18.1c-2.8,0.3-6.7,2.3-8.1-1.3c-1.4-3.6-1.7-7.9-0.3-11.7
|
||||
c0.9-2.4,3.9-2,6.2-2.4c13.8-2.4,27.6-4.4,41.3-7c42.1-8.2,83.5-19.3,124.2-33.1c57.6-19.6,112.2-45.7,163.9-77.3
|
||||
c38.1-23.2,73.8-50.1,107.6-79.4c37.9-32.8,72.6-68.6,102.5-109C2037.5,591.3,2039.4,587.8,2043.6,585.1z"/>
|
||||
<path class="st1" d="M777.2,1163.8c-0.9-7.9,1.2-15.4,2-23c4.4-45.1,6.3-90.3,6.6-135.6c0.1-8.6,0.2-8.7,8.8-8.7
|
||||
c22-0.1,44-0.1,65.9,0c8.3,0,8.4,0.1,8.5,7.9c0.2,37.7,2,75.2,6.5,112.6c0.5,3.8,0.3,6-3.9,7.5c-31.7,11.3-62.2,25.3-92.6,39.6
|
||||
C778.8,1164.3,778.4,1164.1,777.2,1163.8z"/>
|
||||
<path class="st1" d="M776.8,817.8c3.1-0.6,4.7,0.6,6.5,1.4c28.2,13.5,56.6,26.5,86.2,36.8c5.4,1.9,6.7,4.3,6,10.1
|
||||
c-4.5,36.7-6.3,73.6-6.4,110.6c0,8.1,0.7,8.3-8.1,8.3c-22.6-0.1-45.3,0-67.9-0.1c-7,0-7.1-0.2-7.2-7.3c-0.4-28.6-0.9-57.3-2.7-85.9
|
||||
c-1.3-21.3-2.9-42.5-5.5-63.7C777.2,824.9,777.1,821.6,776.8,817.8z"/>
|
||||
<path class="st1" d="M1896.4,1163.9c-3.3,0.5-5.7-1.5-8.4-2.8c-27.2-13.3-54.8-25.7-83.3-36.1c-7.4-2.7-7.2-2.9-6.3-10.6
|
||||
c4.4-35.8,5.3-71.8,6.4-107.7c0.1-1.7,0.1-3.3,0-5c-0.3-3.8,1.3-5.4,5.2-5.3c6.3,0.2,12.7,0.1,19,0.1c16.7,0,33.3,0,50,0.1
|
||||
c8.4,0,8.4,0.1,8.6,8.8c0.8,30.3,1.2,60.6,3.3,90.9c1.4,19.9,2.9,39.9,5.3,59.7C1896.5,1158.6,1897.4,1161.2,1896.4,1163.9z"/>
|
||||
<path class="st1" d="M1896.8,817.7c-1.3,13.9-2.7,27.7-3.9,41.6c-3.5,38.5-4.5,77-5.2,115.6c0,1.7-0.2,3.3-0.1,5
|
||||
c0.4,4-1.5,5.5-5.2,5.1c-1-0.1-2,0-3,0c-22.6,0-45.3-0.1-67.9,0.2c-5,0-6.8-1.2-6.8-6.6c-0.4-28.3-1.2-56.5-3.6-84.7
|
||||
c-0.8-8.9-1.6-17.9-2.7-26.8c-0.9-7.6-1.1-7.8,6.5-10.6c23-8.7,45.8-17.8,67.9-28.7c6-2.9,12-5.7,18-8.6
|
||||
C1892.5,818.3,1894.3,817.4,1896.8,817.7z"/>
|
||||
<path class="st1" d="M908.3,1109.6c-1-4.4,0.4-8.6,0.8-12.8c2.5-30.9,4-61.8,4-92.9c0-7.4,0.2-7.5,7.9-7.5
|
||||
c33.3-0.1,66.7-0.2,100-0.2c7.8,0,8.1,0.1,8,7.4c-0.3,22,1,44,2.3,65.9c0.3,4.5-1,6.5-5.5,7.4c-38.7,8.5-76.4,20.9-114.2,32.5
|
||||
C910.7,1109.8,909.7,1109.6,908.3,1109.6z"/>
|
||||
<path class="st1" d="M1765.3,1110.4c-9.6-3-18.1-5.6-26.6-8.4c-29.1-9.6-58.7-17.4-88.5-24.5c-8.9-2.1-8.1-1.4-7.6-10.4
|
||||
c1.2-21,1.6-42,2.3-62.9c0.2-7.7,0.2-7.9,7.5-7.9c33.3,0,66.7,0.1,100,0.2c7.8,0,8.2,0.3,8.2,7.7c0.2,34,1.7,68,4.8,101.8
|
||||
C1765.4,1106.9,1765.3,1107.9,1765.3,1110.4z"/>
|
||||
<path class="st1" d="M908.8,872.2c2.8-1.1,5,0.8,7.5,1.5c36.1,10.3,71.6,22.9,108.5,30.5c5.5,1.1,6.7,3.6,6.4,8.8
|
||||
c-1.3,21.9-2.6,43.9-2.1,65.9c0.1,4.7-1.2,6.5-6.1,6.4c-34.7-0.2-69.3-0.2-104-0.1c-4.1,0-6.1-1-6-5.6c0.4-35.6-2-71.2-4.8-106.7
|
||||
C908.2,872.8,908.6,872.5,908.8,872.2z"/>
|
||||
<path class="st1" d="M1765.6,872.4c-0.9,10.7-1.8,20.9-2.5,31.2c-1.7,24.6-2.4,49.2-2.6,73.8c-0.1,7.7-0.2,7.8-7.7,7.8
|
||||
c-33.6,0.1-67.2,0.1-100.8,0.1c-6.9,0-6.9-0.2-7.1-7.1c-0.8-21.9-1.3-43.9-2.6-65.8c-0.3-5.4,1.8-7,6.2-8
|
||||
c14.5-3.5,29-7.1,43.5-10.8c22.5-5.7,44.5-13.5,66.7-20.3C1760.5,872.7,1762.3,871.8,1765.6,872.4z"/>
|
||||
<path class="st1" d="M725.8,1192.5c-5.9-34.3-8.8-67.4-11-100.6c-1.8-27.6-2.2-55.2-3-82.8c-0.1-2.3,0.1-4.7-0.1-7
|
||||
c-0.4-4.4,1.6-5.8,5.9-5.7c9,0.2,18,0,27,0c6.7,0,13.3-0.1,20,0c7.2,0.1,7.4,0.2,7.3,7.1c-0.5,23.6-1,47.2-2,70.8
|
||||
c-1.2,27.2-3.3,54.4-6.7,81.5c-0.2,1.3-0.3,2.6-0.4,4c-0.4,9.2-3.4,16-13,19.7C741.7,1182.7,734.5,1187.7,725.8,1192.5z"/>
|
||||
<path class="st1" d="M1946.2,1191.7c-10.1-5.6-20.4-11.5-30.8-17.2c-3.3-1.8-3.1-5-3.5-7.8c-1.4-10.2-2.4-20.5-3.5-30.7
|
||||
c-4.7-43.8-6-87.7-6.7-131.6c-0.1-7.7,0.1-7.8,7.7-7.8c15.3-0.1,30.6,0.1,46-0.1c4.6-0.1,6.5,1.1,6.4,6.1
|
||||
c-0.5,34-1.2,67.9-3.7,101.8c-2,26.9-4.5,53.7-8.9,80.4C1948.7,1187,1949.1,1189.5,1946.2,1191.7z"/>
|
||||
<path class="st1" d="M1947.7,789.1c5.8,32.7,8.6,63.8,10.8,95c2.1,30.6,2.8,61.2,3.1,91.9c0.1,9.6,0.3,8.9-8.3,8.9
|
||||
c-15,0-30-0.1-45,0.2c-5.3,0.1-6.9-1.8-6.7-7c0.9-25.3,1.3-50.6,2.5-75.9c1.3-27.3,3.9-54.4,6.9-81.6c0.9-8.3,3.3-14.2,11.5-17.8
|
||||
C1930.8,799.2,1938.6,794.1,1947.7,789.1z"/>
|
||||
<path class="st1" d="M726,789.2c11.3,6.2,21.7,11.9,32.2,17.5c2.9,1.6,2.8,4.2,3.2,6.8c2.7,15.8,4.1,31.7,5.5,47.6
|
||||
c3.3,37.2,4.5,74.5,5,111.8c0,2.3,0,4.7,0.1,7c0.1,3.1-0.8,5-4.4,5c-17-0.1-34-0.1-51-0.1c-3.3,0-4.8-1.5-4.7-4.8
|
||||
c0.2-5.7,0-11.3,0.2-17c1-42.6,2.7-85.2,7.3-127.7C720.9,820.5,722.6,805.7,726,789.2z"/>
|
||||
<path class="st1" d="M1625.1,910.5c-0.6,9.4-1.4,18-1.7,26.5c-0.4,13-0.4,26-0.6,38.9c-0.1,9.3-0.2,9.4-9.2,9.4
|
||||
c-30.6,0.1-61.3,0.1-91.9,0.1c-8.3,0-16.7-0.1-25,0.1c-4.2,0.1-5.9-1.3-5.9-5.7c0-15-0.1-30-0.7-44.9c-0.2-4.8,2.5-5.5,6-6
|
||||
c10.9-1.3,21.8-3.6,32.7-3.7c23.9-0.2,46.7-6.5,70-10C1607.1,913.9,1615.2,912.2,1625.1,910.5z"/>
|
||||
<path class="st1" d="M1625.2,1071.1c-23.5-4.2-45.7-8.1-67.9-12.1c-7.9-1.4-15.8-2.4-23.8-2.5c-12.7-0.1-25.1-2.6-37.7-4
|
||||
c-4-0.4-5.8-1.6-5.6-6.1c0.5-14.6,0.6-29.3,0.6-43.9c0-4.5,1-6.5,6.1-6.5c39.9,0.2,79.9,0.2,119.8,0.1c5,0,6.1,2.1,6.1,6.6
|
||||
C1622.9,1025,1623,1047.3,1625.2,1071.1z"/>
|
||||
<path class="st1" d="M1049.3,1070.4c-1-5.9-0.1-11.8,0.3-17.7c0.9-16.3,1.4-32.6,1.4-48.9c0-7.3,0.2-7.5,6.8-7.5
|
||||
c39.3-0.1,78.5-0.1,117.8-0.1c7,0,7.1,0.2,7.2,7.2c0.2,14,0,27.9,0.5,41.9c0.2,5.2-1.5,6.6-6.6,7c-16.9,1.3-33.8,2.7-50.6,5.2
|
||||
c-23.3,3.4-46.7,6.8-69.7,12.3C1054.2,1070.3,1051.9,1071.4,1049.3,1070.4z"/>
|
||||
<path class="st1" d="M1049.8,910.2c22.1,5.7,44.8,8.7,67.4,12.5c16.4,2.8,33.1,3.3,49.6,5.2c16.4,1.9,16.4,1.5,16.3,17.4
|
||||
c-0.1,11-0.3,22-0.2,32.9c0.1,4.9-1,7.2-6.6,7.2c-39.9-0.2-79.9-0.2-119.8-0.1c-3.4,0-5.7-0.6-5.6-4.8c0.6-22.6-1.3-45.2-1.9-67.8
|
||||
C1049,912,1049.4,911.4,1049.8,910.2z"/>
|
||||
<path class="st1" d="M1260.5,985.4c-21.6,0-43.2,0-64.9,0c-8,0-8.2-0.1-8.1-8.4c0.2-13.6,0.6-27.3,0.8-40.9c0.1-4.2,1.4-5.6,6-5.1
|
||||
c41.7,4.1,83.5,6.2,125.5,6.2c2.3,0,4.7,0,7,0c7.5,0.2,7.5,0.2,7.6,7.9c0.1,10.3,0.1,20.6,0.1,30.9c0,9.3,0,9.4-9.1,9.4
|
||||
C1303.8,985.4,1282.2,985.4,1260.5,985.4z"/>
|
||||
<path class="st1" d="M1485,931.5c0.3,16.2,0.5,32.7,0.8,49.3c0.1,3.4-1.7,4.8-4.9,4.6c-2-0.1-4,0-6,0c-41.6,0-83.2,0-124.9,0
|
||||
c-2,0-4-0.1-6,0c-4,0.3-5.1-1.8-5-5.5c0.1-11.7,0-23.3,0.1-35c0.1-7.6,0.1-7.8,7.6-7.8c17.3,0.2,34.6-0.6,51.9-1.1
|
||||
c26.3-0.7,52.5-3.1,78.7-5.1C1479.6,930.9,1481.8,930,1485,931.5z"/>
|
||||
<path class="st1" d="M1485,1051.1c-35.4-3-69.9-5.3-104.5-6.4c-11.6-0.4-23.3-0.5-34.9-0.4c-4.4,0-6.7-0.8-6.5-6
|
||||
c0.4-12,0.3-24,0-36c-0.1-4.8,1.6-6.3,6.3-6.2c45,0.1,89.9,0.1,134.9,0c3.5,0,5.8,0.7,5.7,4.8
|
||||
C1485.5,1017.5,1485.3,1034.1,1485,1051.1z"/>
|
||||
<path class="st1" d="M1188.4,1050.6c-0.3-17.2-0.5-33.7-0.8-50.3c0-2.9,1.6-4.1,4.4-4.1c1.7,0,3.3,0,5,0c42.6,0,85.1,0,127.7,0
|
||||
c1.7,0,3.3,0.1,5,0c3.3-0.2,4.9,1.2,4.9,4.6c-0.1,13-0.1,25.9,0,38.9c0,3.6-1.7,4.7-4.9,4.5c-0.7,0-1.3,0-2,0
|
||||
C1281.6,1043.7,1235.9,1047.2,1188.4,1050.6z"/>
|
||||
<path class="st1" d="M2010.7,1230.1c-5.1-1.9-9-5.1-13.1-7.7c-9.9-6.1-19.6-12.6-29.8-18.3c-4.8-2.6-5.5-5.6-4.5-10.4
|
||||
c3.9-20,6.2-40.1,8.2-60.4c4-41.2,5.8-82.4,6.1-123.8c0-3,0.1-6,0-9c-0.1-3,1.1-3.8,4.1-4.1c16.5-1.5,16.7-1.4,16.8,14.7
|
||||
c0.4,66.4,3.9,132.5,10.7,198.6C2009.9,1216.3,2011.6,1222.7,2010.7,1230.1z"/>
|
||||
<path class="st1" d="M2011.2,752.5c-1.6,16.9-3.2,33.8-4.8,50.6c-3,30.8-4.6,61.7-6.1,92.6c-1.3,27.3-1.5,54.5-1.8,81.8
|
||||
c-0.1,7.1-0.2,7.3-7.1,7.3c-16.3,0-13.6,1.3-13.7-12.8c-0.5-54.3-3.2-108.4-10.6-162.2c-1.1-7.9-2.5-15.8-3.9-23.6
|
||||
c-0.7-3.6-0.1-6,3.5-8.2c13.4-8,26.6-16.4,39.8-24.7C2007.6,752.8,2008.5,751.7,2011.2,752.5z"/>
|
||||
<path class="st1" d="M662.1,1231.5c1.9-20.4,3.7-38.6,5.4-56.8c3-31.9,4.7-63.8,6-95.8c1-24.6,1.6-49.3,1.6-74
|
||||
c0-8.3,0.1-8.3,7.9-8.4c1.3,0,2.7,0,4,0c8.6,0.1,8.7,0.1,8.8,8.7c0.4,40,1.8,80,5.4,119.8c2,22.9,4.4,45.8,9,68.3
|
||||
c1.1,5.5,0.2,8.5-5.1,11c-12.7,6-23.9,14.5-35.5,22.2C667.7,1227.9,665.8,1229.1,662.1,1231.5z"/>
|
||||
<path class="st1" d="M662.9,752.1c2.6-0.9,3.9,1,5.4,2.1c12.4,8.4,24.6,17.1,38,23.9c3.3,1.7,5,3.6,4.2,7.7
|
||||
c-11.3,61.8-13.7,124.4-14.8,187c-0.2,13.5,2.2,12.1-12.3,12.1c-8.4,0-8.5-0.1-8.5-8.9c0.1-52.7-2.6-105.3-6.4-157.8
|
||||
c-1.5-20.6-4-41.1-6-61.7C662.5,755.1,661.9,753.4,662.9,752.1z"/>
|
||||
<path class="st1" d="M2044.3,728.6c0,3.4,0,6.3,0,9.2c0,78.9,0,157.9,0,236.8c0,1.7-0.1,3.3,0,5c0.4,3.9-1.3,5.4-5.2,5.3
|
||||
c-5.9-0.2-11.9,0-18.3,0c0-26.3,0.3-51.6,1.3-76.8c1.7-42.3,4.4-84.5,8.9-126.5c1.6-14.9,3.9-29.7,5.6-44.6
|
||||
C2037.2,732.4,2038.9,729.7,2044.3,728.6z"/>
|
||||
<path class="st1" d="M2044.3,1253c-5.6-1.2-7-4.1-7.7-8.6c-3.1-19.4-5.2-38.9-7-58.5c-5.6-59-8.7-118.2-8.7-177.6
|
||||
c0-13.6-1.4-11.7,11.5-11.8c13.3,0,11.9-1.5,11.9,11.5c0,78.6,0,157.2,0,235.8C2044.3,1246.6,2044.3,1249.3,2044.3,1253z"/>
|
||||
<path class="st1" d="M629.2,728.7c5.8,1.6,7.4,4.6,7.9,9c2.3,21.2,5.3,42.2,7.3,63.4c5.3,55.3,7.7,110.8,8.5,166.4
|
||||
c0,3.3,0.1,6.7,0,10c-0.1,7.5-0.1,7.5-7.7,7.7c-16,0.5-16,0.5-16-15.3c0-76.5,0-153.1,0-229.6C629.2,736.7,629.2,733,629.2,728.7z"
|
||||
/>
|
||||
<path class="st1" d="M629.2,1253.2c0-6,0-10.6,0-15.3c0-75.7,0-151.3,0-227c0-2,0-4,0-6c-0.1-9.2-0.3-8.7,8.7-8.6
|
||||
c15.1,0,15.2,0,15,15.6c-0.7,62.7-3.8,125.2-10.3,187.6c-1.5,14.9-3.9,29.7-5.5,44.6C636.6,1248.7,634.5,1251.1,629.2,1253.2z"/>
|
||||
<path class="st1" d="M2060,697.5c0-15.9,0-29.5,0-43c0-3.7,0.3-7.3-0.1-11c-2-19,6.4-34.8,15.3-50.5c1-1.7,2.3-3.2,3.6-4.7
|
||||
c5.4-6.2,8.1-6.2,13.4,0.4c6.3,7.8,10.3,17.1,13.7,26.4c3.2,8.7,5.9,17.6,8.7,26.5c1.3,4,1,7.1-2.4,10.7
|
||||
C2096.7,668.2,2079,681.6,2060,697.5z"/>
|
||||
<path class="st1" d="M613.2,1283.1c0,24.5,1.5,47.6-0.5,70.3c-1.4,15.4-9,29.8-19.7,41.7c-3.8,4.2-5.7,3.9-10-0.4
|
||||
c-4.3-4.3-7.2-9.6-9.8-15c-6.7-13.8-11.2-28.4-15.3-43.2c-0.8-3-1.3-5.6,1.5-8.1C576.3,1313,592.4,1296.7,613.2,1283.1z"/>
|
||||
<path class="st1" d="M2060,1284.4c19.3,15.5,36.8,29.3,52.5,45.2c2.6,2.6,3.7,5.1,2.6,8.8c-5.1,17.2-10,34.5-19.7,50
|
||||
c-7.7,12.2-12.4,12.7-19.8,0.6c-9.5-15.4-16.9-31.4-15.8-50.5C2060.7,1321.2,2060,1303.9,2060,1284.4z"/>
|
||||
<path class="st1" d="M611.9,698.2c-8.5-6.7-17.1-12.8-25-19.7c-9.1-7.8-17.5-16.4-26.3-24.5c-2.3-2.2-4-4.1-3-7.7
|
||||
c5.3-18.2,10.3-36.5,20.4-52.8c7.8-12.7,12.8-13,20.4-0.3c8.9,14.9,16.3,30.2,15.1,48.6c-1,15.9-0.2,31.9-0.3,47.9
|
||||
C613.2,692.2,614,694.9,611.9,698.2z"/>
|
||||
<path class="st1" d="M1788.2,864c4.1,27,4.9,53.5,6.1,80.1c0.5,11.3,0.2,22.6,0.7,33.9c0.2,5.2-1.6,7.4-6.7,7.3
|
||||
c-4.7-0.1-9.3-0.3-14,0c-4.6,0.3-5.5-2-5.4-6c0.2-26.3,1.3-52.5,3.2-78.8c0.5-7.6,1.2-15.2,2.1-22.8
|
||||
C1775.6,867.2,1775.7,867.2,1788.2,864z"/>
|
||||
<path class="st1" d="M885.3,1118.4c-5.1-39.1-6.6-77.6-6.6-116.3c0-4.2,1.3-5.9,5.7-5.8c25,0.7,20.1-4.1,19.8,19.4
|
||||
c-0.4,29.6-1.5,59.3-5,88.8C897.9,1115.4,898.1,1115.4,885.3,1118.4z"/>
|
||||
<path class="st1" d="M885.7,863.5c12.4,2.1,12.2,2.1,13.5,13.8c3.8,33,4.6,66.2,5.3,99.4c0.2,8.3,0,8.5-7.9,8.4
|
||||
c-5.6-0.1-12.6,2.4-16.3-1.2c-3.8-3.6-1.3-10.7-1.3-16.2c0.1-33.9,1.8-67.7,6.1-101.4C885.1,865.4,885.4,864.5,885.7,863.5z"/>
|
||||
<path class="st1" d="M1788.4,1117.5c-4,0.3-6.2-1.5-8.7-2.2c-3.2-0.9-4.6-2.9-4.9-6.5c-0.7-10.3-2.2-20.5-2.9-30.8
|
||||
c-1.5-24.6-2.8-49.2-2.8-73.9c0-7.5,0.1-7.8,7.7-7.6c5.9,0.2,13.3-2.7,17.2,1.6c3.4,3.7,1,10.9,0.9,16.5
|
||||
C1794.3,1048.8,1792.8,1083.1,1788.4,1117.5z"/>
|
||||
<path class="st1" d="M1481.1,1078.1c-10.7,0.3-20.5-1.5-30.3-2.6c-34.1-3.7-68.2-5.9-102.5-5.6c-1.3,0-2.7-0.1-4,0
|
||||
c-4.9,0.5-5.1-2.5-5.2-6.2c-0.1-4,0.7-6,5.4-6c20,0.1,39.9-0.2,59.9,1c22.9,1.4,45.8,3,68.7,4.7c7.9,0.6,7.8,0.9,8.2,8.7
|
||||
C1481.2,1073.7,1481.1,1075.4,1481.1,1078.1z"/>
|
||||
<path class="st1" d="M1480,903c2,3.3,0.7,6.3,1,9.2c0.4,4.3-1.6,5.6-5.8,5.9c-42.2,3.3-84.3,6.2-126.7,5.9c-1.3,0-2.7-0.1-4,0
|
||||
c-4.5,0.5-5.5-1.7-5.6-5.8c-0.1-4.5,0.9-6.6,6-6.5c29.7,0.7,59.2-1.5,88.8-3.9C1449.3,906.5,1464.7,904.6,1480,903z"/>
|
||||
<path class="st1" d="M1192.2,1078.2c1-14.7,0.9-15,13.5-15.8c40.5-2.9,80.9-5.4,121.6-4.6c7.3,0.1,7.4,0.5,7.3,7
|
||||
c-0.1,4-1.7,5.2-5.4,5.1c-6-0.2-12-0.1-18,0c-37.3,0.2-74.3,3.2-111.3,7.8C1197.6,1078,1195.3,1078,1192.2,1078.2z"/>
|
||||
<path class="st1" d="M1192.6,903.2c17.2,2.2,33.7,3.8,50.2,5c28.2,2.1,56.4,4.4,84.7,3.6c3.8-0.1,7.3-0.4,7.1,5.4
|
||||
c-0.3,6.3-0.2,6.8-6.6,6.8c-18.6,0-37.2,0.4-55.9-0.8c-19.2-1.2-38.5-2.1-57.8-3.2c-3-0.2-6-0.4-8.9-0.8
|
||||
C1192.8,917.6,1192.8,917.6,1192.6,903.2z"/>
|
||||
<path class="st1" d="M653.6,694.6c2.5,2.7,3.4,3.5,4.2,4.6c15.1,20.9,32.8,39.4,51.8,56.7c2.5,2.3,7.7,4.8,5,8.5
|
||||
c-2.7,3.9-6.2-1-8.9-2.4c-14.7-7.8-28.1-17.9-42.2-26.6c-3.1-1.9-4.3-4.4-4.8-8C657.4,717,655.5,706.9,653.6,694.6z"/>
|
||||
<path class="st1" d="M654,1284.6c0.9-5.5,2.7-11,2.6-16.4c-0.1-14.4,6.3-23.8,19.2-30.2c10.6-5.3,20.3-12.5,30.5-18.8
|
||||
c2.5-1.5,4.9-3.3,7.8-3.6c1.6,2.1,1.5,4,0.2,5.7c-1,1.3-2.4,2.3-3.6,3.4c-18.4,17.3-36.5,34.8-51.2,55.5c-1.1,1.6-2.6,2.9-4,4.3
|
||||
C655.3,1284.8,654.8,1284.6,654,1284.6z"/>
|
||||
<path class="st1" d="M2019.1,698.8c-1.4,9.8-3,19.6-4.2,29.5c-0.4,3-2.2,4.7-4.4,6.2c-15.3,9.9-30.7,19.7-46.1,29.5
|
||||
c-1.5,1-3.7,4-5.7,0.4c-1.5-2.6,0.2-4.4,2.1-5.9c11.9-9.2,21.5-20.7,32.4-30.8c8.4-7.7,14.5-17.2,21.4-26.1c0.9-1.2,2-2.4,3-3.6
|
||||
C2018.1,698.2,2018.6,698.5,2019.1,698.8z"/>
|
||||
<path class="st1" d="M2017.1,1283.9c-11.8-19.6-28.7-34.6-44.6-50.5c-3.7-3.7-7.8-7.2-11.8-10.6c-1.9-1.7-3.4-3.4-1.7-5.9
|
||||
c1.7-2.4,3.5-0.5,5,0.5c15.7,10,31.3,20,46.9,30c2,1.3,3.5,2.9,3.9,5.4c1.4,10.1,2.9,20.2,4.4,30.4
|
||||
C2018.5,1283.4,2017.8,1283.7,2017.1,1283.9z"/>
|
||||
<path class="st1" d="M1044.6,1034c-0.6,11-1.1,21.9-1.7,32.9c-0.1,1.6-0.3,3.3-0.8,4.9c-0.3,0.8-1.4,1.6-2.2,1.7
|
||||
c-0.9,0.1-1.9-0.5-2.1-1.7c-0.1-1-0.3-2-0.4-3c-1.2-22.2-2.3-44.5-2.4-66.8c0-3.2,0-6,4.6-5.9c3.5,0.1,5.6,0.7,5.5,4.9
|
||||
c-0.2,11-0.1,21.9-0.1,32.9C1044.9,1034,1044.8,1034,1044.6,1034z"/>
|
||||
<path class="st1" d="M1636.4,1033.8c0,10.6,0,21.2,0,31.8c0,2-0.3,4-0.7,5.9c-0.2,0.8-1.2,1.8-1.9,1.8c-0.8,0-1.9-0.8-2.2-1.5
|
||||
c-0.5-1.2-0.5-2.6-0.6-3.9c-1.9-21.8-2.2-43.7-2.1-65.6c0-5.7,1.3-6.7,6-6.1c3.9,0.5,3.6,3.1,3.6,5.8c0,10.6,0,21.2,0,31.8
|
||||
C1637.8,1033.8,1637.1,1033.8,1636.4,1033.8z"/>
|
||||
<path class="st1" d="M1638.5,948.1c0,10.3,0,20.6,0,31c0,2.6,0.5,5.4-3.3,6.2c-4.4,0.9-6.4-0.5-6.4-5.8c0-19,0-38,1.4-56.9
|
||||
c0.2-3,0.4-6,0.7-9c0.2-2.2-0.3-5.4,3-5.2c2.5,0.1,2.2,3,2.3,4.9c0.6,11.6,1.1,23.3,1.6,34.9C1638,948.1,1638.2,948.1,1638.5,948.1
|
||||
z"/>
|
||||
<path class="st1" d="M1045.1,948c0,10.6-0.1,21.2,0,31.9c0,3.6-0.9,5.5-5,5.5c-3.9-0.1-5.1-1.7-5.1-5.4c0.1-22.6,1.2-45.1,2.5-67.7
|
||||
c0.1-1.6-0.1-3.7,2-4c2.8-0.4,3.1,1.9,3.2,3.9c0.7,11.9,1.3,23.9,1.9,35.8C1044.8,948,1044.9,948,1045.1,948z"/>
|
||||
<path class="st1" d="M741.9,740.1c7,10.8,9.7,22,12.6,34.5c-8.1-3.6-13.5-8.9-19.7-13c-2-1.3-2-3.3-1.2-5.3
|
||||
C735.6,751.1,737,745.6,741.9,740.1z"/>
|
||||
<path class="st1" d="M1931.7,1241.3c-6.9-10.6-9.7-21.7-12.1-33.3c4.2,0.4,6.4,3.1,9.1,4.9
|
||||
C1943.2,1223.1,1943.4,1224.7,1931.7,1241.3z"/>
|
||||
<path class="st1" d="M1931.6,740c12,17.2,11.9,18.3-3.6,29.1c-2.4,1.7-4.5,3.9-8.5,4.8C1921.8,761.9,1925,750.9,1931.6,740z"/>
|
||||
<path class="st1" d="M742.1,1241.3c-5.3-5.2-6.5-11.1-8.6-16.5c-0.8-2-0.5-3.8,1.6-5.1c6-3.9,11.2-9.1,19.2-12.6
|
||||
C751.8,1219.5,748.7,1230.3,742.1,1241.3z"/>
|
||||
<path class="st1" d="M599.6,517.5c0.6,0.1,1.4,0.1,1.9,0.4c12.6,7.1,13,7.9,10.7,24.4c-5.9-7.9-11-15.2-13.4-24.1
|
||||
C599,518,599.3,517.7,599.6,517.5z"/>
|
||||
<path class="st1" d="M597.6,1464.4c4-9.4,8.1-17.2,14-24.4C614.9,1455.5,614.5,1457.9,597.6,1464.4z"/>
|
||||
<path class="st1" d="M2072.4,516.4c2.2,0.6,2.3,2,1.3,3.7c-3.8,6.1-7.7,12.1-11.9,18.6c-2.8-10.6-2.1-12.7,5.3-17.8
|
||||
c1.9-1.3,4.2-2,5.3-4.3L2072.4,516.4z"/>
|
||||
<path class="st1" d="M2061.5,1442.5c4.7,8.5,10.5,14.5,12.5,22.8C2067.2,1459.8,2056.4,1456.8,2061.5,1442.5z"/>
|
||||
<path class="st1" d="M1781.6,836.9c4.2,9.3,4.2,9.3-1.8,12.6C1777.9,845.4,1780.6,842.1,1781.6,836.9z"/>
|
||||
<path class="st1" d="M895.4,849.3c-7.5-2.9-7.5-2.9-3.8-11.1C893.8,841.3,893.9,844.7,895.4,849.3z"/>
|
||||
<path class="st1" d="M630.1,1284.6c-0.2-5.4-2.7-11,1.8-17C634.2,1274.4,629.7,1279.3,630.1,1284.6z"/>
|
||||
<path class="st1" d="M629.9,697.1c0.5,5.4,3.9,10.3,2.4,16.6C626.9,708.5,630.1,702.5,629.9,697.1z"/>
|
||||
<path class="st1" d="M2044.5,1284.5c-2.2-5-4.3-9.2-2.8-15C2046.1,1274.3,2043.5,1279.4,2044.5,1284.5z"/>
|
||||
<path class="st1" d="M2041.9,712.1c-2.2-6,1.1-10.1,2.2-14.6C2043.5,702.2,2046,707.4,2041.9,712.1z"/>
|
||||
<path class="st1" d="M892.2,1143.9c-4.6-7.9-4.6-7.9,1.6-11.5C895.7,1136.5,892.6,1139.7,892.2,1143.9z"/>
|
||||
<path class="st1" d="M1779.6,1132.2c6.1,3.1,6.1,3.1,2.7,11.1c-1.9-1-1.7-3.1-2.3-4.6C1779.2,1137,1778.6,1135.1,1779.6,1132.2z"/>
|
||||
<path class="st1" d="M1934.8,1202.5c4.1-0.6,6.5,2.5,11.4,4.9c-6.3,0.5-8.2-3.5-11.3-5.1L1934.8,1202.5z"/>
|
||||
<path class="st1" d="M1935.8,777c3.1,0.9,4.5-3.5,7.8-2.9c-1.8,4-4.7,3.9-7.9,2.8L1935.8,777z"/>
|
||||
<path class="st1" d="M729.5,774c2.7-0.5,4.4,0.6,5.8,3.2C732.1,778,731,775.7,729.5,774z"/>
|
||||
<path class="st1" d="M734,1202.5c2.3,4.6-1.2,4.8-4.2,5.9c-0.6-3.7,3.7-3.3,4.2-5.7C734,1202.7,734,1202.5,734,1202.5z"/>
|
||||
<path class="st1" d="M1935.2,780.3c-0.4,0.2-0.8,0.5-1.2,0.7c0.3,0,0.6,0.1,0.8,0c0.2-0.1,0.4-0.3,0.6-0.5
|
||||
C1935.5,780.5,1935.2,780.3,1935.2,780.3z"/>
|
||||
<path class="st1" d="M740,1203.1c-1.9-0.8-4.2,2.9-6-0.5c0,0,0,0.2,0,0.1C736,1202.8,738,1202.9,740,1203.1L740,1203.1z"/>
|
||||
<path class="st1" d="M1935.7,777c1.4,1.3,0.9,2.4-0.5,3.3c0,0,0.3,0.3,0.3,0.3c0.1-1.2,0.3-2.3,0.4-3.5
|
||||
C1935.8,777,1935.7,777,1935.7,777z"/>
|
||||
<path class="st1" d="M737.9,778.4c0.2,0.2,0.4,0.5,0.6,0.7c-0.2,0.2-0.5,0.5-0.7,0.5c-0.2,0-0.4-0.4-0.6-0.6
|
||||
C737.4,778.8,737.7,778.6,737.9,778.4z"/>
|
||||
<path class="st1" d="M740,1203.1c0-0.3,0-0.8-0.2-1c-0.8-0.8-0.7-1.1,0.4-0.8C739.5,1201.9,739.4,1202.4,740,1203.1
|
||||
C740,1203.1,740,1203.1,740,1203.1z"/>
|
||||
<path class="st1" d="M1934.9,1202.3c0-0.2-0.1-0.3-0.1-0.5c0,0-0.1,0.1-0.1,0.1c0.1,0.2,0.1,0.4,0.2,0.6
|
||||
C1934.8,1202.5,1934.9,1202.3,1934.9,1202.3z"/>
|
||||
<path class="st1" d="M2072.4,516.6c0.7,0.4,1.4,0.8,2.1,1.2c-0.2,0.2-0.4,0.7-0.6,0.7c-1.1-0.2-1.5-1-1.4-2
|
||||
C2072.4,516.4,2072.4,516.6,2072.4,516.6z"/>
|
||||
<path class="st1" d="M598.7,518.3c-1-0.2-1.8-0.8-1-1.6c0.8-0.9,1.4,0,1.8,0.8C599.3,517.7,599,518,598.7,518.3z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 48 KiB |
|
|
@ -1,455 +0,0 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 24.2.1, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
|
||||
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
viewBox="0 0 2670 1980" style="enable-background:new 0 0 2670 1980;" xml:space="preserve">
|
||||
|
||||
<g>
|
||||
<path fill="#808080" d="M2275.3,1000.1c-0.1,2.3,0,4.7,0,7c0,189.3,0,378.7,0,568c0,6,0.1,12.1-0.8,17.9c-2.4,15.2-12.9,17.4-23.6,10
|
||||
c-45.4-31-91.2-61.4-136.9-91.9l-30.1-20.3c-9.8-6.3-19.2-13.2-29.2-19.3c-6.7-4.1-9.8-9.7-9.9-17.2c-0.1-9.3-0.6-18.7-0.3-28
|
||||
c0.2-7.3-1.7-13-6.5-19c-18.6-23.3-36.2-47.3-56.9-69c-18-18.8-36.7-36.8-56.4-53.8c-31.8-27.5-65.4-52.7-101-74.9
|
||||
c-32.7-20.4-66.8-38.4-102.2-54c-50.7-22.4-103.2-39.2-157.1-51.5c-36.7-8.4-73.8-14.4-111.2-18.5c-34.5-3.8-69-6.5-103.7-5.8
|
||||
c-23.1,0.5-46.2-1.3-69.3,0.4c-22.9,1.8-45.9,3-68.7,5.6c-72.7,8.4-143.7,24.6-212.1,50.6c-65.5,24.8-127.4,57-184.9,97.4
|
||||
c-46.8,32.8-90.1,69.8-129.5,111.1c-18.1,19-35.1,39.2-50.5,60.6c-3.9,5.5-5.5,11-5.3,17.6c0.4,9.3,0,18.7,0,28
|
||||
c0,9.3-3.1,16.4-11.8,21.4c-10.4,5.9-19.9,13.4-30.1,19.8l-25.4,17.2c-11.6,7.8-23.3,15.5-34.9,23.3c-34.3,23-68.6,46-102.9,69
|
||||
c-14.6,9.7-23.6,5.5-25.2-12.1c-0.5-5.3-0.5-10.7-0.5-16c0-189.7,0-379.3,0-569l-0.1-26.1c0.2-190.3,0.1-380.7,0.2-571
|
||||
c0-5.7,0.4-11.3,0.5-17c0.1-3.7,1.2-7.3,3-10.5c3.4-5.9,9.3-7.7,15.4-4.8c2.7,1.3,5.1,3.1,7.6,4.8c44.3,29.6,88.5,59.3,132.8,88.9
|
||||
l29.7,20.1c10.8,6.6,20.9,14.3,31.7,20.9c6.7,4.1,9.7,9.8,9.8,17.3c0.2,11,0.4,22,0.2,33c-0.1,5.2,1.2,9.6,4.2,13.9
|
||||
c15.4,22.2,33,42.8,51.6,62.2c53.9,56.4,114.1,105.1,181.4,144.6c38.4,22.6,78.3,42.4,119.9,58.6c40,15.6,80.9,28.3,122.8,38
|
||||
c52.7,12.1,106,19.3,160,22.6c18.3,1.1,36.6,2.1,54.9,1.5c17.7-0.6,35.5,1.1,53.2-0.1c17.6-1.1,35.2-2.3,52.8-3.6
|
||||
c35.9-2.5,71.4-8.5,106.7-15.1c53.4-9.9,105.5-25,156-45.1c53.8-21.5,105.1-48.2,153.2-80.9c43.4-29.5,83.6-62.7,121.1-99.3
|
||||
c27.3-26.6,50.3-56.8,73.9-86.6c2.9-3.6,3.6-7.6,3.6-12.1c-0.1-10.7,0.2-21.3,0.4-32c0.1-7.5,3.2-13.1,9.9-17.2
|
||||
c10.2-6.2,19.9-13.4,30-19.8l23.2-16.3c4.2-1.3,7.4-4.3,11-6.7c42.9-28.7,85.8-57.4,128.7-86.1c1.1-0.7,2.3-1.4,3.3-2.3
|
||||
c10.9-9.1,21.4-4.7,23.8,10.7c0.8,5.6,0.6,11.3,0.6,17c0,189.3,0,378.7,0,568L2275.3,1000.1z M740,1203.1c-2-0.2-4-0.3-6-0.5
|
||||
c-0.5,2.5-4.8,2-4.2,5.8c2.9-1,6.4-1.2,4.2-5.8C735.8,1206,738.1,1202.3,740,1203.1c-0.6-0.7-0.5-1.3,0.2-1.7
|
||||
c-1.1-0.3-1.1-0.1-0.4,0.8C740,1202.3,739.9,1202.8,740,1203.1z M1935.5,780.5c-0.2,0.2-0.4,0.4-0.6,0.4c-0.3,0.1-0.6,0-0.8,0
|
||||
c0.4-0.2,0.8-0.5,1.1-0.8c1.4-0.9,2-1.9,0.5-3.3c3.2,1.1,6.1,1.2,7.9-2.8c-3.3-0.7-4.6,3.8-7.7,2.9
|
||||
C1935.7,778.2,1935.6,779.4,1935.5,780.5z M1934.8,1202.5c-0.1-0.2-0.1-0.4-0.2-0.6c0,0,0.1-0.1,0.1-0.1c0,0.2,0.1,0.3,0.1,0.5
|
||||
c3.1,1.6,5,5.7,11.3,5.1C1941.3,1205,1938.9,1202,1934.8,1202.5z M2072.4,516.6c-1,2.3-3.4,3-5.3,4.3c-7.4,5.1-8.2,7.2-5.3,17.8
|
||||
c4.2-6.5,8-12.6,11.9-18.6c1.1-1.7,0.9-3.1-1.3-3.6c0,1,0.3,1.8,1.4,2c0.2,0,0.4-0.4,0.6-0.7C2073.8,517.4,2073.1,517,2072.4,516.6
|
||||
z M599.6,517.5c-0.4-0.8-1.1-1.7-1.8-0.8c-0.8,0.8,0.1,1.5,1,1.6c2.4,8.9,7.5,16.2,13.4,24.1c2.3-16.5,1.9-17.3-10.7-24.4
|
||||
C600.9,517.6,600.2,517.6,599.6,517.5z M534.9,1353.5c-20,21.8-37.3,44.1-53.4,67.4c-16.1,23.4-26.8,49.5-41.8,73.5
|
||||
c9-21,17.4-42.2,28.9-61.9c16.9-28.7,35.4-56.4,57.5-81.5c0.9-1,1.6-2.1,2.6-3.1c2.9-2.9,3.3-6,2.3-10
|
||||
c-9.1-33.8-14.6-68.3-19.2-103c-5-38-8.3-76.1-10.6-114.3c-2.5-39.9-3.4-79.9-3.9-119.8c-0.1-5.1-1.4-6.8-6.7-6.9
|
||||
c-21.6-0.2-43.3-0.8-64.9-1.3c-2,0-4-0.4-6,0.8c0,195.2,0,390.5,0,585.6c2.1,0.6,3.1-0.4,4.1-1.1c52.7-33.8,105.3-67.7,158-101.4
|
||||
c3.6-2.3,5.7-5.4,7.3-9.1c6-13,12.5-25.8,20.8-37.4c4.2-5.9,3.9-11.9,2.5-19.1c-2.4,1.8-4,2.9-5.5,4.2
|
||||
c-13.9,12.6-32.1,8.7-42.7-2.9c-7.3-8-12.4-17.2-17.1-26.8C542.3,1375.7,539.4,1365.1,534.9,1353.5z M534.6,627.2
|
||||
c3.6-5,4.3-10.7,6.3-15.9c6.2-15.8,12.6-31.5,25.2-43.7c10.8-10.5,26.8-12.6,38.9-2.8c2.3,1.9,4.3,5,7.9,5
|
||||
c1.6-8.3-0.2-15.3-5.1-21.9c-7.4-9.9-12.7-21-17.8-32.3c-2.4-5.3-5.8-9.2-10.8-12.4c-50.5-32.2-100.9-64.7-151.3-97.1
|
||||
c-2.4-1.5-4.5-3.9-8.7-3.9c0,3.7,0,7.2,0,10.7c0,187.9,0,375.8,0,563.7c0,2,0.2,4,0,6c-0.6,4.9,1.4,6.3,6.2,6.1
|
||||
c13.6-0.6,27.3-0.7,41-1c8.3-0.2,16.7-0.6,25-0.4c4.9,0.1,6.7-1.8,6.1-6.4c-0.1-1,0-2,0-3c1.2-75,4.3-149.8,13.7-224.3
|
||||
c4.6-36.3,10-72.5,19.6-107.9c1.6-5.7,0.4-9.9-3.4-14.4c-10.7-12.8-21.7-25.4-31-39.1c-19-28.1-36.8-57-49.6-88.7
|
||||
c-2.2-5.4-5.5-10.4-6.2-16.6C465,538.3,494.6,586,534.6,627.2z M2060.9,1411.1c-2.3,8.5-0.1,15.1,4,21.4
|
||||
c5.9,9.3,13.4,18.1,17.1,28.2c3.9,10.7,10.9,16.6,19.9,22.4c47.8,30.3,95.3,61.1,143,91.7c2.7,1.7,5,4.3,9.5,4.5c0-3.4,0-6.6,0-9.8
|
||||
c0-188.3,0-376.6,0-565c0-2-0.1-4,0-6c0.2-3.4-1.1-5-4.6-4.8c-2.3,0.1-4.7,0-7,0.1c-20,0.6-40,1.2-60,1.9c-6.7,0.2-6.7,0.4-7,7.4
|
||||
c0,1.3,0,2.7-0.1,4c-0.4,60-3,119.9-8.9,179.7c-5,50.1-11.3,99.9-24,148.7c-1.4,5.2,0.1,8.3,3.5,12.3c15,17.8,29.6,36.1,42.4,55.5
|
||||
c20,30.4,36.1,62.9,49.3,96.8c3.2,8.2,5.7,16.6,6.8,25.4c-33.9-90.8-82.6-145-107.2-169.7c-4.1,11-7.5,22.1-12.7,32.5
|
||||
c-4.9,9.9-10.3,19.5-18.8,26.8c-7.1,6.1-15.1,8.9-24.8,7.5C2073.2,1421.4,2068.3,1415.5,2060.9,1411.1z M2244.8,454.3
|
||||
c0.6,4.5-0.2,6.7-0.8,8.9c-6.5,24.2-17,47-28.2,69.2c-18.9,37.2-43,71.2-70.4,102.8c-2.7,3.2-3.6,5.8-2.5,9.8
|
||||
c7.9,29.6,13.1,59.8,17.3,90.2c6.4,45.6,10.2,91.3,12.8,137.3c2,35.3,2.6,70.6,3.1,105.9c0.1,7.6,0.2,7.6,7.6,7.8
|
||||
c17,0.5,34,1,51,1.4c5,0.1,10,0.2,15,0.5c3.2,0.1,5.1-0.9,4.9-4.5c-0.1-1.7,0-3.3,0-5c0-189.6,0-379.3-0.1-568.9
|
||||
c0-2.4,1.1-5.2-1.2-7.3c-0.8,0.1-1.6,0-2.1,0.4c-52.7,33.8-105.3,67.7-158,101.5c-2,1.3-3.7,2.6-4.7,4.9c-6.9,16-16.9,30.1-25.8,45
|
||||
c-3.1,5.2-3.4,10.2-2,15.7c2.6,0.1,3.5-1.6,4.6-2.6c15.7-13.9,33-10.6,45.6,3.8c6,6.8,10.4,14.6,14.4,22.7
|
||||
c5.1,10.3,8.3,21.3,12.8,32.9C2185.2,576.1,2221.7,520.5,2244.8,454.3z M553.7,662.8c-0.3,1.3-0.7,2.8-1,4.3
|
||||
c-7.1,31.2-11.7,62.8-15.4,94.5c-7.1,59.9-10.3,120.1-11.8,180.4c-0.3,13-0.2,26-0.5,39c-0.1,4.4,1.5,5.7,5.7,5.6
|
||||
c25-0.5,50-0.7,74.9-1.1c7.4-0.1,7.6-0.2,7.6-7.8c0-85.6,0-171.2,0.1-256.8c0-5.3-1.7-8.5-5.8-11.7c-16.6-12.8-32.4-26.6-46.9-41.9
|
||||
C559,665.6,557.6,663,553.7,662.8z M553.6,1319.7c2.1-1,2.8-1.2,3.2-1.6c16.4-17.1,34-32.9,52.7-47.4c3-2.4,3.8-5,3.8-8.6
|
||||
c-0.1-13.3-0.1-26.7-0.1-40c0-72.3,0-144.7,0-217c0-10.3,0.4-9-9.2-9.2c-23.3-0.4-46.7-0.8-70-1.1c-2.8,0-5.9-1-9.3,1.5
|
||||
C526.2,1104.4,531.1,1212.3,553.6,1319.7z M2119.8,1319.2c4.8-20.9,8.3-40.2,11.1-59.6c5.8-39.5,9.8-79.2,12.5-119.1
|
||||
c3.1-45.5,4.7-91.1,5.1-136.7c0.1-7.6-0.1-7.7-7.6-7.7c-24.3,0.1-48.6,0.3-72.9,0.5c-7.9,0.1-8,0.1-8,8.5c0,84.6,0,169.1-0.1,253.7
|
||||
c0,5.6,1.7,9.3,5.9,12.7c12.1,9.9,24.2,19.9,36,30.1C2107.7,1306.9,2113.1,1312.7,2119.8,1319.2z M2120.4,664.4
|
||||
c-2.1-0.6-2.9-0.1-3.6,0.7c-15.2,16.5-33.1,29.9-50.2,44.2c-4.8,4-6.8,8-6.8,14.3c0.3,32,0.2,64,0.2,96c0,52.3,0,104.6,0,157
|
||||
c0,8.1,0.1,8.2,8.2,8.2c18.3,0.2,36.7,0.2,55,0.4c6.7,0.1,13.3,0.1,20,0.3c3.7,0.1,5.7-1.1,5.4-5.1c-0.2-2.3,0-4.7-0.1-7
|
||||
c-0.5-45-2.1-89.9-5.2-134.8C2139.2,780.1,2132.7,721.9,2120.4,664.4z M878.7,1138.5c-36,15-69.8,32.9-102.3,53.1
|
||||
c-3.4,2.1-4.9,4.7-5.7,8.6c-2.8,14.4-5.7,28.8-12.2,42c-3.6,7.3-7.7,14.4-17.3,14.4c-9.6,0-13.4-7.4-17.4-14.4
|
||||
c-1.9-3.2-2-7.6-5.4-9.8c-22.3,17.4-41.6,37.2-58.8,58.9c-5.8,7.3-10.2,15.3-12,24.8c-3.8,20.3-9.1,40.2-16.7,59.5
|
||||
c-2.6,6.7-1.3,13.3-1.5,20.3c1.5-0.4,2-0.3,2.1-0.5c3.2-3.8,6.4-7.6,9.4-11.6c31.2-40.5,66-77.4,105.4-110.2
|
||||
c36.9-30.8,75.6-59.1,117.2-83.1c54.3-31.2,110.9-57.4,170.8-76.4c38.4-12.2,77.4-21.8,117-29.2c9.5-1.8,19.1-3,28.6-4.6
|
||||
c1.5-0.3,3.8,0.3,4.4-1.5c1.4-3.9,0.6-8-0.4-11.7c-1-3.6-4.5-1.7-6.9-1.5c-40.6,3.2-80.4,11-120.4,18c-7,1.2-10.4,4.1-10.9,11.2
|
||||
c-0.3,3.8-0.5,9.1-5.8,9c-4.6-0.1-4.7-5-5.3-8.5c-0.7-4.5-2.4-6-7.3-4.9c-39.3,9.1-78,19.9-115.5,34.7c-4.6,1.8-7.1,4.2-7.8,8.9
|
||||
c-0.9,5.9-2.2,11.8-5.2,17.1c-4.1,7.1-10.6,7.2-14.6,0C881.9,1147.7,880.7,1143.4,878.7,1138.5z M1032.8,891.9
|
||||
c0.9-3.8,1.6-6.7,2.4-9.5c0.7-2.3,1.7-4.6,4.4-4.9c3.4-0.3,4.4,2.3,5.3,4.9c0.4,1.3,0.8,2.6,0.8,3.9c0.1,7.7,4.5,10.4,11.7,11.6
|
||||
c32.5,5.5,64.8,11.9,97.5,15.7c7.9,0.9,15.9,1.7,23.8,2.4c1.9,0.2,4,0.7,5.3-1.6c3.5-6.5,0.3-13-7-14.2c-1-0.2-2-0.2-3-0.4
|
||||
c-42.6-5.8-84.2-15.9-125.4-28.2c-41.8-12.4-82.5-28.2-121.6-47.1c-51-24.6-99.8-53.3-144.7-87.9c-20.6-15.8-40.9-32.1-59.9-49.7
|
||||
c-29.1-26.9-56.4-55.7-80.1-87.7c-3.7-4.9-7.7-9.6-11.6-14.3c-0.8,7.6-2.4,14.7,0.5,21.8c7.4,18.2,12.4,37.2,16,56.5
|
||||
c1.7,9.1,5,17.3,10.6,24.4c9.2,11.4,18.5,22.8,28.7,33.3c9.9,10.1,21.1,19,32.2,28.9c3.8-6.5,5.6-13,9.8-18.2
|
||||
c7.7-9.4,17.7-9.5,25.3-0.1c3.6,4.5,5.7,9.7,7.8,15c4.3,10.9,6.8,22.3,8.9,33.7c0.9,5.2,3.3,8.1,7.5,10.7
|
||||
c21.9,13.4,44.1,26,67.3,37.1c10.9,5.2,22.1,10,33.5,15.2c1.8-4.8,3.1-8.6,4.7-12.2c1.6-3.5,4.2-6.3,8.3-6.1
|
||||
c3.6,0.2,5.8,2.9,7.3,6.1c2.4,5.2,3.8,10.6,4.8,16.2c0.8,4.6,2.9,7.2,7.7,9C950.6,871.5,990.8,883,1032.8,891.9z M1953.8,1233.7
|
||||
c-2.9,5.8-4.9,11.5-8.8,16.2c-7.6,9.3-17.8,9.3-25.3,0c-3.4-4.2-5.5-9-7.5-14c-4.4-11.5-7.2-23.5-9.4-35.7
|
||||
c-0.7-3.9-2.4-6.4-5.8-8.6c-30.4-19.4-62.4-35.6-95-50.8c-6.9-3.3-7-3-9.9,4.7c-0.7,1.9-1.5,3.7-2.3,5.5c-1.6,3.1-3.7,5.6-7.6,5.6
|
||||
c-3.8,0-5.9-2.5-7.6-5.6c-2.9-5.4-4.3-11.3-5.1-17.2c-0.7-5-3-7.3-7.5-9.1c-37.8-15.2-77.3-24.6-116.7-34.3c-3.8-0.9-5.4,0.2-6.2,4
|
||||
c-0.7,3.9-0.2,9.9-6.3,9.5c-5-0.3-5.1-6.1-5.3-9.6c-0.4-6.6-3.3-8.9-9.6-10c-32.8-5.6-65.4-12.2-98.6-15.7
|
||||
c-8.3-0.9-16.5-2.1-24.8-2.7c-5.4-0.4-7.1,1.2-7.5,6.6c-0.3,3.9-1.3,8,5.2,8.4c6.3,0.4,12.4,2.1,18.7,3.1
|
||||
c47.7,7.5,94.4,19.1,140.3,34.3c42.4,14,83.5,31.2,123,52c46.3,24.4,90.2,52.6,131,85.6c48.5,39.1,93.7,81.2,129.4,132.6
|
||||
c1.9,2.8,4.8,7.5,7.3,6.7c4.5-1.5,1.4-6.8,2.1-10.4c0.6-3.4-0.5-6.6-1.8-9.7c-7.8-19.5-12.6-40-17.1-60.4c-1.1-5.1-3-9.4-6-13.5
|
||||
c-17-22.8-36.7-43.1-57.6-62.2C1959.6,1237.1,1957.6,1234.7,1953.8,1233.7z M2043.6,585.1c-4.2,2.7-6.2,6.2-8.5,9.2
|
||||
c-29.9,40.3-64.6,76.1-102.5,109c-33.8,29.3-69.5,56.2-107.6,79.4c-51.8,31.6-106.3,57.7-163.9,77.3
|
||||
c-40.7,13.8-82.1,24.9-124.2,33.1c-13.7,2.7-27.6,4.7-41.3,7c-2.2,0.4-5.3-0.1-6.2,2.4c-1.4,3.8-1.1,8.2,0.3,11.7
|
||||
c1.4,3.5,5.3,1.6,8.1,1.3c40.1-4.2,79.8-10.7,119.5-18.1c6.2-1.2,10.4-2.7,10.5-10.2c0-3.5,0.3-9.3,5.4-9.5c6-0.3,5.5,5.7,6.2,9.6
|
||||
c0.7,3.9,2.3,4.9,6.1,3.9c10-2.7,20-5.1,30-7.5c28.5-6.9,56.4-15.6,83.9-25.7c6.8-2.5,10.2-6.3,11-13.3c0.5-5,2-9.8,4.5-14.2
|
||||
c4.2-7.6,10.8-7.6,15.1,0.1c1.6,2.9,2.5,6.2,3.4,9.3c0.9,2.9,2.5,3.5,5.2,2.2c2.4-1.2,5-1.9,7.3-3.1c26.6-13.1,53.2-26,78.7-41.2
|
||||
c11.9-7.1,19.6-14.7,21.3-29.3c1.4-12.4,5.5-24.8,12.8-35.4c8.1-11.7,19.5-11.7,27.5,0.1c3.1,4.6,5.4,9.7,8.2,14.9
|
||||
c1.9-1.4,3.6-2.5,5.1-3.8c22.2-19.8,42.7-41.1,60.6-64.9c2.6-3.5,4.3-7.3,5.2-11.7c4.5-21.2,9.6-42.2,17.6-62.4
|
||||
C2045.3,599.2,2044.5,592.6,2043.6,585.1z M777.2,1163.8c1.2,0.2,1.6,0.4,1.9,0.3c30.4-14.3,60.9-28.3,92.6-39.6
|
||||
c4.2-1.5,4.4-3.7,3.9-7.5c-4.5-37.4-6.3-75-6.5-112.6c0-7.8-0.2-7.9-8.5-7.9c-22,0-44,0-65.9,0c-8.6,0-8.8,0.1-8.8,8.7
|
||||
c-0.4,45.3-2.2,90.5-6.6,135.6C778.4,1148.4,776.3,1155.9,777.2,1163.8z M776.8,817.8c0.3,3.8,0.5,7.1,0.9,10.4
|
||||
c2.5,21.2,4.1,42.4,5.5,63.7c1.8,28.6,2.3,57.2,2.7,85.9c0.1,7.1,0.3,7.3,7.2,7.3c22.6,0.1,45.3,0.1,67.9,0.1
|
||||
c8.8,0,8.1-0.2,8.1-8.3c0.1-37,1.9-73.9,6.4-110.6c0.7-5.8-0.5-8.2-6-10.1c-29.5-10.3-58-23.3-86.2-36.8
|
||||
C781.5,818.4,779.9,817.2,776.8,817.8z M1896.4,1163.9c1-2.7,0.2-5.3-0.1-7.9c-2.4-19.8-3.9-39.8-5.3-59.7
|
||||
c-2.1-30.3-2.5-60.6-3.3-90.9c-0.2-8.7-0.2-8.8-8.6-8.8c-16.7-0.1-33.3,0-50-0.1c-6.3,0-12.7,0.2-19-0.1c-3.9-0.1-5.5,1.4-5.2,5.3
|
||||
c0.1,1.7,0.1,3.3,0,5c-1.1,36-2,72-6.4,107.7c-0.9,7.7-1.1,7.9,6.3,10.6c28.5,10.4,56.1,22.9,83.3,36.1
|
||||
C1890.6,1162.4,1893.1,1164.4,1896.4,1163.9z M1896.8,817.7c-2.5-0.3-4.3,0.6-6.1,1.5c-6,2.9-12,5.6-18,8.6
|
||||
c-22.1,10.9-44.9,20.1-67.9,28.7c-7.5,2.8-7.3,3-6.5,10.6c1,8.9,1.9,17.9,2.7,26.8c2.4,28.2,3.2,56.5,3.6,84.7
|
||||
c0.1,5.4,1.8,6.7,6.8,6.6c22.6-0.2,45.3-0.1,67.9-0.2c1,0,2-0.1,3,0c3.8,0.3,5.6-1.2,5.2-5.1c-0.2-1.6,0-3.3,0.1-5
|
||||
c0.7-38.6,1.7-77.2,5.2-115.6C1894.1,845.4,1895.5,831.6,1896.8,817.7z M908.3,1109.6c1.4,0,2.4,0.2,3.2,0
|
||||
c37.8-11.7,75.5-24,114.2-32.5c4.5-1,5.8-2.9,5.5-7.4c-1.3-22-2.5-43.9-2.3-65.9c0.1-7.3-0.2-7.4-8-7.4c-33.3,0-66.7,0.1-100,0.2
|
||||
c-7.7,0-7.9,0.1-7.9,7.5c0,31-1.6,61.9-4,92.9C908.8,1101,907.3,1105.2,908.3,1109.6z M1765.3,1110.4c0-2.5,0.1-3.5,0-4.4
|
||||
c-3.1-33.9-4.7-67.8-4.8-101.8c0-7.4-0.4-7.6-8.2-7.7c-33.3-0.1-66.7-0.2-100-0.2c-7.3,0-7.2,0.1-7.5,7.9c-0.7,21-1.1,42-2.3,62.9
|
||||
c-0.5,9-1.3,8.3,7.6,10.4c29.8,7.1,59.4,14.9,88.5,24.5C1747.2,1104.8,1755.7,1107.4,1765.3,1110.4z M908.8,872.2
|
||||
c-0.2,0.3-0.6,0.6-0.6,0.8c2.8,35.5,5.2,71,4.8,106.7c-0.1,4.6,1.9,5.6,6,5.6c34.7-0.1,69.3-0.1,104,0.1c4.9,0,6.2-1.8,6.1-6.4
|
||||
c-0.5-22,0.8-44,2.1-65.9c0.3-5.2-0.9-7.7-6.4-8.8c-36.9-7.6-72.4-20.1-108.5-30.5C913.8,873,911.6,871.1,908.8,872.2z
|
||||
M1765.6,872.4c-3.2-0.6-5.1,0.3-7,0.9c-22.2,6.7-44.2,14.5-66.7,20.3c-14.5,3.7-29,7.3-43.5,10.8c-4.4,1.1-6.5,2.6-6.2,8
|
||||
c1.3,21.9,1.8,43.9,2.6,65.8c0.2,7,0.2,7.1,7.1,7.1c33.6,0,67.2,0,100.8-0.1c7.5,0,7.7-0.2,7.7-7.8c0.2-24.6,0.9-49.2,2.6-73.8
|
||||
C1763.8,893.4,1764.7,883.1,1765.6,872.4z M725.8,1192.5c8.7-4.8,15.8-9.8,23.8-12.8c9.6-3.7,12.7-10.5,13-19.7
|
||||
c0.1-1.3,0.2-2.7,0.4-4c3.4-27.1,5.6-54.3,6.7-81.5c1-23.6,1.5-47.2,2-70.8c0.2-7-0.1-7-7.3-7.1c-6.7-0.1-13.3,0-20,0
|
||||
c-9,0-18,0.2-27,0c-4.2-0.1-6.2,1.3-5.9,5.7c0.2,2.3,0,4.7,0.1,7c0.8,27.6,1.2,55.2,3,82.8C717,1125.1,719.9,1158.2,725.8,1192.5z
|
||||
M1946.2,1191.7c2.9-2.2,2.4-4.7,2.8-6.9c4.5-26.6,6.9-53.5,8.9-80.4c2.5-33.9,3.2-67.9,3.7-101.8c0.1-5-1.8-6.2-6.4-6.1
|
||||
c-15.3,0.2-30.6,0-46,0.1c-7.6,0-7.8,0.2-7.7,7.8c0.7,44,2,87.9,6.7,131.6c1.1,10.3,2.2,20.5,3.5,30.7c0.4,2.8,0.2,6,3.5,7.8
|
||||
C1925.8,1180.2,1936.2,1186.1,1946.2,1191.7z M1947.7,789.1c-9.1,5-16.9,10.1-25.3,13.8c-8.2,3.6-10.5,9.5-11.5,17.8
|
||||
c-3,27.1-5.6,54.3-6.9,81.6c-1.2,25.3-1.5,50.6-2.5,75.9c-0.2,5.1,1.5,7,6.7,7c15-0.3,30-0.1,45-0.2c8.6,0,8.4,0.6,8.3-8.9
|
||||
c-0.3-30.6-1-61.3-3.1-91.9C1956.3,852.9,1953.6,821.8,1947.7,789.1z M726,789.2c-3.3,16.4-5.1,31.3-6.7,46.2
|
||||
c-4.6,42.4-6.3,85-7.3,127.7c-0.1,5.7,0,11.3-0.2,17c-0.1,3.4,1.4,4.8,4.7,4.8c17,0,34,0,51,0.1c3.6,0,4.6-1.9,4.4-5
|
||||
c-0.1-2.3-0.1-4.7-0.1-7c-0.5-37.3-1.7-74.6-5-111.8c-1.4-15.9-2.8-31.9-5.5-47.6c-0.4-2.6-0.3-5.2-3.2-6.8
|
||||
C747.6,801.1,737.2,795.4,726,789.2z M1625.1,910.5c-9.8,1.8-18,3.4-26.2,4.6c-23.3,3.5-46.2,9.9-70,10
|
||||
c-10.9,0.1-21.8,2.4-32.7,3.7c-3.5,0.4-6.2,1.1-6,6c0.5,15,0.7,30,0.7,44.9c0,4.4,1.7,5.8,5.9,5.7c8.3-0.2,16.6,0,25-0.1
|
||||
c30.6,0,61.3,0,91.9-0.1c9,0,9-0.1,9.2-9.4c0.2-13,0.2-26,0.6-38.9C1623.7,928.4,1624.5,919.8,1625.1,910.5z M1625.2,1071.1
|
||||
c-2.2-23.8-2.3-46.1-2.4-68.4c0-4.5-1-6.6-6.1-6.6c-39.9,0.1-79.9,0.1-119.8-0.1c-5.1,0-6.1,2-6.1,6.5c0,14.6-0.1,29.3-0.6,43.9
|
||||
c-0.1,4.5,1.6,5.6,5.6,6.1c12.6,1.4,25,3.8,37.7,4c8,0.1,15.9,1.1,23.8,2.5C1579.5,1063,1601.7,1066.9,1625.2,1071.1z
|
||||
M1049.3,1070.4c2.6,1,4.9-0.1,7-0.6c23-5.5,46.3-8.9,69.7-12.3c16.7-2.5,33.7-3.8,50.6-5.2c5.1-0.4,6.8-1.9,6.6-7
|
||||
c-0.4-14-0.3-27.9-0.5-41.9c-0.1-7-0.2-7.2-7.2-7.2c-39.3,0-78.5,0-117.8,0.1c-6.7,0-6.8,0.2-6.8,7.5c0,16.3-0.5,32.6-1.4,48.9
|
||||
C1049.3,1058.6,1048.4,1064.5,1049.3,1070.4z M1049.8,910.2c-0.4,1.2-0.8,1.8-0.8,2.4c0.6,22.6,2.5,45.2,1.9,67.8
|
||||
c-0.1,4.2,2.1,4.8,5.6,4.8c39.9,0,79.9-0.1,119.8,0.1c5.5,0,6.7-2.3,6.6-7.2c-0.2-11,0.1-22,0.2-32.9c0.1-15.9,0.1-15.5-16.3-17.4
|
||||
c-16.5-1.9-33.1-2.4-49.6-5.2C1094.7,918.9,1072,915.9,1049.8,910.2z M1260.5,985.4c21.6,0,43.2,0,64.9,0c9.1,0,9.1-0.1,9.1-9.4
|
||||
c0-10.3,0-20.6-0.1-30.9c-0.1-7.7-0.1-7.7-7.6-7.9c-2.3-0.1-4.7,0-7,0c-41.9,0-83.8-2.2-125.5-6.2c-4.6-0.4-5.9,0.9-6,5.1
|
||||
c-0.2,13.6-0.6,27.3-0.8,40.9c-0.1,8.3,0.1,8.4,8.1,8.4C1217.3,985.4,1238.9,985.4,1260.5,985.4z M1485,931.5
|
||||
c-3.1-1.4-5.4-0.6-7.7-0.4c-26.2,1.9-52.4,4.3-78.7,5.1c-17.3,0.5-34.6,1.3-51.9,1.1c-7.4-0.1-7.5,0.1-7.6,7.8
|
||||
c-0.1,11.7,0,23.3-0.1,35c0,3.7,1,5.8,5,5.5c2-0.1,4,0,6,0c41.6,0,83.2,0,124.9,0c2,0,4-0.1,6,0c3.3,0.1,5-1.2,4.9-4.6
|
||||
C1485.5,964.2,1485.3,947.6,1485,931.5z M1485,1051.1c0.3-17,0.5-33.6,0.9-50.1c0.1-4.2-2.2-4.8-5.7-4.8c-45,0.1-89.9,0.1-134.9,0
|
||||
c-4.7,0-6.5,1.4-6.3,6.2c0.3,12,0.4,24,0,36c-0.2,5.2,2.1,6,6.5,6c11.6-0.1,23.3,0.1,34.9,0.4
|
||||
C1415,1045.8,1449.5,1048.1,1485,1051.1z M1188.4,1050.6c47.5-3.4,93.2-6.8,139.2-6.4c0.7,0,1.3,0,2,0c3.2,0.2,4.9-0.9,4.9-4.5
|
||||
c-0.1-13,0-25.9,0-38.9c0-3.4-1.6-4.8-4.9-4.6c-1.7,0.1-3.3,0-5,0c-42.6,0-85.1,0-127.7,0c-1.7,0-3.3,0.1-5,0
|
||||
c-2.8-0.1-4.4,1.1-4.4,4.1C1187.9,1016.8,1188.1,1033.4,1188.4,1050.6z M2010.7,1230.1c0.9-7.3-0.9-13.8-1.5-20.3
|
||||
c-6.8-66-10.3-132.2-10.7-198.6c-0.1-16.1-0.3-16.3-16.8-14.7c-2.9,0.3-4.2,1.1-4.1,4.1c0.1,3,0,6,0,9
|
||||
c-0.3,41.3-2.1,82.6-6.1,123.8c-2,20.2-4.3,40.4-8.2,60.4c-0.9,4.8-0.2,7.8,4.5,10.4c10.2,5.7,19.9,12.2,29.8,18.3
|
||||
C2001.8,1225,2005.6,1228.2,2010.7,1230.1z M2011.2,752.5c-2.7-0.8-3.6,0.3-4.7,1c-13.3,8.3-26.4,16.7-39.8,24.7
|
||||
c-3.6,2.1-4.2,4.5-3.5,8.2c1.4,7.8,2.8,15.7,3.9,23.6c7.4,53.8,10.1,107.9,10.6,162.2c0.1,14.1-2.6,12.8,13.7,12.8
|
||||
c7,0,7-0.2,7.1-7.3c0.4-27.3,0.6-54.6,1.8-81.8c1.4-30.9,3.1-61.8,6.1-92.6C2008,786.3,2009.6,769.5,2011.2,752.5z M662.1,1231.5
|
||||
c3.7-2.4,5.6-3.7,7.5-4.9c11.6-7.7,22.9-16.2,35.5-22.2c5.3-2.5,6.2-5.5,5.1-11c-4.6-22.5-7-45.4-9-68.3
|
||||
c-3.5-39.9-5-79.8-5.4-119.8c-0.1-8.6-0.2-8.6-8.8-8.7c-1.3,0-2.7,0-4,0c-7.9,0.1-8,0.1-7.9,8.4c0,24.7-0.6,49.3-1.6,74
|
||||
c-1.3,32-3,63.9-6,95.8C665.8,1192.9,664.1,1211.1,662.1,1231.5z M662.9,752.1c-1,1.4-0.4,3-0.3,4.6c2,20.5,4.6,41.1,6,61.7
|
||||
c3.8,52.5,6.5,105.1,6.4,157.8c0,8.8,0.1,8.9,8.5,8.9c14.5-0.1,12.1,1.3,12.3-12.1c1.1-62.7,3.5-125.2,14.8-187
|
||||
c0.8-4.1-0.9-6-4.2-7.7c-13.4-6.8-25.6-15.5-38-23.9C666.8,753.1,665.5,751.1,662.9,752.1z M2044.3,728.6c-5.4,1.1-7.1,3.9-7.7,8.3
|
||||
c-1.7,14.9-4,29.7-5.6,44.6c-4.6,42.1-7.2,84.2-8.9,126.5c-1,25.3-1.3,50.5-1.3,76.8c6.5,0,12.4-0.1,18.3,0
|
||||
c3.8,0.1,5.5-1.4,5.2-5.3c-0.2-1.7,0-3.3,0-5c0-78.9,0-157.9,0-236.8C2044.3,734.9,2044.3,732,2044.3,728.6z M2044.3,1253
|
||||
c0-3.7,0-6.4,0-9c0-78.6,0-157.2,0-235.8c0-13,1.3-11.5-11.9-11.5c-13,0-11.5-1.8-11.5,11.8c0,59.3,3.1,118.5,8.7,177.6
|
||||
c1.9,19.5,4,39.1,7,58.5C2037.3,1248.9,2038.8,1251.8,2044.3,1253z M629.2,728.7c0,4.3,0,8,0,11.6c0,76.5,0,153.1,0,229.6
|
||||
c0,15.8,0,15.8,16,15.3c7.6-0.2,7.6-0.2,7.7-7.7c0.1-3.3,0-6.7,0-10c-0.8-55.6-3.1-111.1-8.5-166.4c-2-21.2-5-42.3-7.3-63.4
|
||||
C636.6,733.3,635,730.3,629.2,728.7z M629.2,1253.2c5.3-2.1,7.4-4.5,7.9-9.2c1.6-14.9,4-29.7,5.5-44.6
|
||||
c6.5-62.4,9.5-124.9,10.3-187.6c0.2-15.6,0.1-15.6-15-15.6c-9,0-8.8-0.6-8.7,8.6c0,2,0,4,0,6c0,75.7,0,151.3,0,227
|
||||
C629.2,1242.5,629.2,1247.2,629.2,1253.2z M2060,697.5c19-15.9,36.7-29.3,52.3-45.3c3.5-3.5,3.7-6.7,2.4-10.7
|
||||
c-2.8-8.9-5.4-17.8-8.7-26.5c-3.5-9.3-7.4-18.5-13.7-26.4c-5.3-6.6-8-6.6-13.4-0.4c-1.3,1.5-2.7,3-3.6,4.7
|
||||
c-8.9,15.7-17.3,31.5-15.3,50.5c0.4,3.6,0.1,7.3,0.1,11C2060,668,2060,681.5,2060,697.5z M613.2,1283.1
|
||||
c-20.8,13.6-36.9,29.9-53.8,45.4c-2.8,2.6-2.3,5.1-1.5,8.1c4.1,14.7,8.6,29.3,15.3,43.2c2.6,5.4,5.4,10.7,9.8,15
|
||||
c4.3,4.2,6.2,4.6,10,0.4c10.7-11.9,18.4-26.3,19.7-41.7C614.7,1330.6,613.2,1307.6,613.2,1283.1z M2060,1284.4
|
||||
c0,19.5,0.7,36.8-0.2,54c-1.1,19.1,6.3,35.2,15.8,50.5c7.4,12.1,12.1,11.7,19.8-0.6c9.7-15.4,14.6-32.7,19.7-50
|
||||
c1.1-3.7,0-6.2-2.6-8.8C2096.8,1313.6,2079.3,1299.9,2060,1284.4z M611.9,698.2c2.1-3.3,1.3-5.9,1.3-8.4c0.1-16-0.8-32,0.3-47.9
|
||||
c1.2-18.4-6.2-33.8-15.1-48.6c-7.6-12.7-12.6-12.4-20.4,0.3c-10.1,16.3-15.1,34.6-20.4,52.8c-1,3.6,0.6,5.5,3,7.7
|
||||
c8.8,8.1,17.3,16.7,26.3,24.5C594.8,685.4,603.3,691.5,611.9,698.2z M1788.2,864c-12.5,3.2-12.6,3.2-13.9,13.8
|
||||
c-0.9,7.6-1.6,15.2-2.1,22.8c-1.8,26.2-2.9,52.5-3.2,78.8c0,4,0.8,6.3,5.4,6c4.6-0.3,9.3-0.2,14,0c5.1,0.1,6.9-2.1,6.7-7.3
|
||||
c-0.5-11.3-0.2-22.6-0.7-33.9C1793.2,917.5,1792.3,890.9,1788.2,864z M885.3,1118.4c12.8-3,12.5-3,13.8-13.9
|
||||
c3.5-29.5,4.6-59.1,5-88.8c0.3-23.5,5.2-18.7-19.8-19.4c-4.4-0.1-5.7,1.5-5.7,5.8C878.8,1040.8,880.3,1079.4,885.3,1118.4z
|
||||
M885.7,863.5c-0.2,0.9-0.6,1.9-0.7,2.9c-4.3,33.7-6,67.5-6.1,101.4c0,5.5-2.5,12.6,1.3,16.2c3.8,3.6,10.8,1.1,16.3,1.2
|
||||
c7.9,0.1,8.1-0.1,7.9-8.4c-0.6-33.2-1.5-66.4-5.3-99.4C897.9,865.6,898,865.6,885.7,863.5z M1788.4,1117.5
|
||||
c4.4-34.4,5.9-68.6,6.5-102.9c0.1-5.6,2.4-12.7-0.9-16.5c-3.9-4.3-11.3-1.4-17.2-1.6c-7.6-0.2-7.7,0.1-7.7,7.6
|
||||
c0,24.7,1.3,49.3,2.8,73.9c0.6,10.3,2.2,20.5,2.9,30.8c0.3,3.6,1.7,5.5,4.9,6.5C1782.2,1116,1784.4,1117.7,1788.4,1117.5z
|
||||
M1481.1,1078.1c0-2.7,0.1-4.4,0-6c-0.3-7.8-0.3-8.1-8.2-8.7c-22.9-1.7-45.8-3.3-68.7-4.7c-19.9-1.2-39.9-0.9-59.9-1
|
||||
c-4.6,0-5.5,2-5.4,6c0.1,3.7,0.3,6.7,5.2,6.2c1.3-0.1,2.7,0,4,0c34.3-0.3,68.4,1.9,102.5,5.6
|
||||
C1460.6,1076.6,1470.4,1078.5,1481.1,1078.1z M1480,903c-15.3,1.6-30.8,3.6-46.3,4.8c-29.5,2.4-59.1,4.7-88.8,3.9
|
||||
c-5.1-0.1-6.1,2-6,6.5c0.1,4.1,1.1,6.3,5.6,5.8c1.3-0.2,2.7,0,4,0c42.3,0.3,84.5-2.6,126.7-5.9c4.2-0.3,6.2-1.6,5.8-5.9
|
||||
C1480.7,909.3,1482,906.3,1480,903z M1192.2,1078.2c3.1-0.2,5.4-0.3,7.6-0.5c37-4.6,74-7.6,111.3-7.8c6,0,12-0.1,18,0
|
||||
c3.7,0.1,5.3-1.1,5.4-5.1c0.1-6.5,0-6.9-7.3-7c-40.6-0.8-81.1,1.7-121.6,4.6C1193.1,1063.3,1193.2,1063.6,1192.2,1078.2z
|
||||
M1192.6,903.2c0.2,14.4,0.2,14.4,12.8,16c3,0.4,6,0.6,8.9,0.8c19.3,1.1,38.5,2,57.8,3.2c18.6,1.2,37.2,0.8,55.9,0.8
|
||||
c6.4,0,6.3-0.5,6.6-6.8c0.2-5.8-3.3-5.5-7.1-5.4c-28.3,0.8-56.5-1.5-84.7-3.6C1226.3,907,1209.8,905.4,1192.6,903.2z M653.6,694.6
|
||||
c2,12.3,3.8,22.4,5.1,32.6c0.5,3.7,1.7,6.1,4.8,8c14.2,8.8,27.5,18.8,42.2,26.6c2.7,1.4,6.2,6.3,8.9,2.4c2.7-3.7-2.6-6.2-5-8.5
|
||||
c-19-17.3-36.7-35.8-51.8-56.7C657,698.1,656,697.3,653.6,694.6z M654,1284.6c0.8,0,1.3,0.1,1.5,0c1.4-1.4,2.9-2.7,4-4.3
|
||||
c14.6-20.8,32.8-38.3,51.2-55.5c1.2-1.1,2.6-2.1,3.6-3.4c1.4-1.7,1.5-3.6-0.2-5.7c-2.9,0.3-5.3,2.1-7.8,3.6
|
||||
c-10.2,6.3-19.8,13.5-30.5,18.8c-12.9,6.4-19.3,15.7-19.2,30.2C656.6,1273.7,654.9,1279.1,654,1284.6z M2019.1,698.8
|
||||
c-0.5-0.3-1-0.6-1.5-0.9c-1,1.2-2,2.4-3,3.6c-6.8,9-13,18.4-21.4,26.1c-11,10.1-20.5,21.7-32.4,30.8c-1.9,1.5-3.6,3.2-2.1,5.9
|
||||
c2,3.5,4.1,0.5,5.7-0.4c15.4-9.7,30.8-19.6,46.1-29.5c2.3-1.5,4-3.1,4.4-6.2C2016.2,718.5,2017.7,708.6,2019.1,698.8z
|
||||
M2017.1,1283.9c0.7-0.2,1.4-0.5,2.1-0.7c-1.5-10.1-3-20.2-4.4-30.4c-0.4-2.6-1.9-4.1-3.9-5.4c-15.6-10-31.3-20.1-46.9-30
|
||||
c-1.5-1-3.3-2.8-5-0.5c-1.7,2.4-0.3,4.2,1.7,5.9c4,3.4,8.1,6.9,11.8,10.6C1988.4,1249.3,2005.3,1264.3,2017.1,1283.9z M1044.6,1034
|
||||
c0.2,0,0.3,0,0.5,0c0-11-0.1-21.9,0.1-32.9c0.1-4.2-2-4.8-5.5-4.9c-4.5-0.1-4.6,2.6-4.6,5.9c0.1,22.3,1.2,44.5,2.4,66.8
|
||||
c0.1,1,0.2,2,0.4,3c0.2,1.2,1.2,1.8,2.1,1.7c0.8-0.1,1.9-1,2.2-1.7c0.5-1.5,0.7-3.2,0.8-4.9C1043.5,1055.9,1044,1045,1044.6,1034z
|
||||
M1636.4,1033.8c0.7,0,1.4,0,2.1,0c0-10.6,0-21.2,0-31.8c0-2.7,0.2-5.3-3.6-5.8c-4.7-0.6-6,0.4-6,6.1c-0.1,21.9,0.2,43.7,2.1,65.6
|
||||
c0.1,1.3,0.1,2.7,0.6,3.9c0.3,0.7,1.5,1.6,2.2,1.5c0.7,0,1.7-1.1,1.9-1.8c0.5-1.9,0.7-3.9,0.7-5.9
|
||||
C1636.4,1055,1636.4,1044.4,1636.4,1033.8z M1638.5,948.1c-0.2,0-0.5,0-0.7,0c-0.5-11.6-1-23.3-1.6-34.9c-0.1-1.9,0.2-4.8-2.3-4.9
|
||||
c-3.3-0.1-2.8,3-3,5.2c-0.3,3-0.4,6-0.7,9c-1.5,19-1.4,37.9-1.4,56.9c0,5.3,2,6.8,6.4,5.8c3.8-0.8,3.3-3.6,3.3-6.2
|
||||
C1638.5,968.8,1638.5,958.4,1638.5,948.1z M1045.1,948c-0.1,0-0.3,0-0.4,0c-0.6-11.9-1.2-23.9-1.9-35.8c-0.1-1.9-0.4-4.3-3.2-3.9
|
||||
c-2.2,0.3-1.9,2.4-2,4c-1.3,22.5-2.3,45.1-2.5,67.7c0,3.7,1.2,5.4,5.1,5.4c4.1,0.1,5-1.9,5-5.5C1045,969.3,1045.1,958.7,1045.1,948
|
||||
z M741.9,740.1c-4.9,5.5-6.3,11-8.3,16.2c-0.8,2-0.8,4,1.2,5.3c6.2,4.1,11.6,9.4,19.7,13C751.6,762.1,748.8,750.9,741.9,740.1z
|
||||
M1931.7,1241.3c11.7-16.6,11.5-18.2-3-28.3c-2.7-1.9-4.9-4.5-9.1-4.9C1921.9,1219.6,1924.7,1230.7,1931.7,1241.3z M1931.6,740
|
||||
c-6.6,10.9-9.8,22-12.1,33.9c4-0.9,6.1-3.1,8.5-4.8C1943.5,758.3,1943.6,757.2,1931.6,740z M742.1,1241.3
|
||||
c6.6-11.1,9.8-21.9,12.2-34.3c-8,3.6-13.2,8.8-19.2,12.6c-2,1.3-2.3,3.1-1.6,5.1C735.6,1230.2,736.7,1236.2,742.1,1241.3z
|
||||
M597.6,1464.4c16.9-6.5,17.3-8.9,14-24.4C605.8,1447.2,601.7,1455,597.6,1464.4z M2061.5,1442.5c-5.1,14.3,5.6,17.3,12.5,22.8
|
||||
C2072.1,1457,2066.2,1451,2061.5,1442.5z M1781.6,836.9c-1,5.2-3.6,8.5-1.8,12.6C1785.7,846.2,1785.7,846.2,1781.6,836.9z
|
||||
M895.4,849.3c-1.5-4.5-1.6-7.9-3.8-11.1C887.9,846.3,887.9,846.3,895.4,849.3z M630.1,1284.6c-0.4-5.3,4.1-10.2,1.8-17
|
||||
C627.4,1273.6,629.9,1279.2,630.1,1284.6z M629.9,697.1c0.2,5.4-3,11.4,2.4,16.6C633.8,707.4,630.4,702.5,629.9,697.1z
|
||||
M2044.5,1284.5c-1-5.2,1.5-10.3-2.8-15C2040.3,1275.4,2042.4,1279.6,2044.5,1284.5z M2041.9,712.1c4.1-4.8,1.6-10,2.2-14.6
|
||||
C2043,702.1,2039.8,706.2,2041.9,712.1z M892.2,1143.9c0.4-4.2,3.5-7.3,1.6-11.5C887.6,1136,887.6,1136,892.2,1143.9z
|
||||
M1779.6,1132.2c-1,2.9-0.4,4.8,0.4,6.5c0.7,1.5,0.4,3.6,2.3,4.6C1785.7,1135.3,1785.7,1135.3,1779.6,1132.2z M729.5,774
|
||||
c1.5,1.6,2.6,4,5.8,3.2C733.9,774.7,732.2,773.5,729.5,774z M737.9,778.4c-0.2,0.2-0.5,0.4-0.7,0.6c0.2,0.2,0.4,0.6,0.6,0.6
|
||||
c0.2,0,0.5-0.3,0.7-0.5C738.4,778.8,738.1,778.6,737.9,778.4z"/>
|
||||
<path fill="none" d="M534.9,1353.5c4.5,11.6,7.4,22.2,12.2,32c4.7,9.6,9.8,18.8,17.1,26.8c10.6,11.6,28.8,15.5,42.7,2.9
|
||||
c1.4-1.3,3.1-2.4,5.5-4.2c1.4,7.2,1.7,13.2-2.5,19.1c-8.4,11.7-14.8,24.4-20.8,37.4c-1.7,3.7-3.7,6.7-7.3,9.1
|
||||
c-52.7,33.7-105.3,67.6-158,101.4c-1.1,0.7-2.1,1.7-4.1,1.1c0-195.1,0-390.4,0-585.6c2.1-1.2,4.1-0.9,6-0.8
|
||||
c21.6,0.5,43.3,1.1,64.9,1.3c5.3,0,6.7,1.8,6.7,6.9c0.5,40,1.4,79.9,3.9,119.8c2.4,38.2,5.6,76.4,10.6,114.3
|
||||
c4.6,34.7,10.1,69.1,19.2,103c1.1,4,0.6,7.1-2.3,10c-0.9,0.9-1.7,2.1-2.6,3.1c-22.1,25.1-40.6,52.7-57.5,81.5
|
||||
c-11.6,19.7-19.9,41-28.9,61.9c14.9-23.9,25.7-50.1,41.8-73.5C497.6,1397.6,514.9,1375.3,534.9,1353.5z"/>
|
||||
<path fill="none" d="M534.6,627.2c-40-41.2-69.6-89-94.2-140.1c0.7,6.2,4,11.2,6.2,16.6c12.7,31.7,30.5,60.6,49.6,88.7
|
||||
c9.3,13.7,20.4,26.3,31,39.1c3.8,4.5,5,8.7,3.4,14.4c-9.6,35.4-15,71.6-19.6,107.9c-9.4,74.5-12.5,149.3-13.7,224.3c0,1-0.1,2,0,3
|
||||
c0.6,4.7-1.2,6.5-6.1,6.4c-8.3-0.2-16.7,0.2-25,0.4c-13.7,0.3-27.3,0.4-41,1c-4.8,0.2-6.8-1.2-6.2-6.1c0.3-2,0-4,0-6
|
||||
c0-187.9,0-375.8,0-563.7c0-3.5,0-7.1,0-10.7c4.2-0.1,6.3,2.3,8.7,3.9c50.4,32.4,100.8,64.9,151.3,97.1c5,3.2,8.4,7.1,10.8,12.4
|
||||
c5.1,11.2,10.4,22.4,17.8,32.3c4.9,6.6,6.7,13.5,5.1,21.9c-3.6-0.1-5.6-3.1-7.9-5c-12-9.8-28.1-7.7-38.9,2.8
|
||||
c-12.6,12.2-19,27.9-25.2,43.7C538.8,616.5,538.2,622.2,534.6,627.2z"/>
|
||||
<path fill="none" d="M2060.9,1411.1c7.4,4.3,12.3,10.3,20.6,11.4c9.8,1.4,17.7-1.4,24.8-7.5c8.5-7.3,13.9-16.9,18.8-26.8
|
||||
c5.1-10.4,8.6-21.5,12.7-32.5c24.7,24.7,73.3,78.9,107.2,169.7c-1.1-8.8-3.5-17.2-6.8-25.4c-13.2-33.9-29.3-66.4-49.3-96.8
|
||||
c-12.8-19.5-27.4-37.7-42.4-55.5c-3.4-4-4.8-7.1-3.5-12.3c12.8-48.8,19.1-98.6,24-148.7c5.9-59.7,8.4-119.6,8.9-179.7
|
||||
c0-1.3,0-2.7,0.1-4c0.2-7,0.2-7.1,7-7.4c20-0.7,40-1.3,60-1.9c2.3-0.1,4.7,0,7-0.1c3.5-0.2,4.8,1.4,4.6,4.8c-0.1,2,0,4,0,6
|
||||
c0,188.3,0,376.6,0,565c0,3.2,0,6.4,0,9.8c-4.5-0.2-6.9-2.8-9.5-4.5c-47.7-30.6-95.2-61.4-143-91.7c-9-5.7-16-11.6-19.9-22.4
|
||||
c-3.7-10.1-11.2-18.9-17.1-28.2C2060.8,1426.2,2058.6,1419.6,2060.9,1411.1z"/>
|
||||
<path fill="none" d="M2244.8,454.3c-23,66.2-59.6,121.8-106.6,172.4c-4.5-11.6-7.7-22.6-12.8-32.9c-4-8.1-8.4-15.8-14.4-22.7
|
||||
c-12.6-14.4-29.9-17.7-45.6-3.8c-1.1,1-2,2.7-4.6,2.6c-1.4-5.5-1.1-10.5,2-15.7c8.8-14.9,18.9-29,25.8-45c1-2.3,2.7-3.7,4.7-4.9
|
||||
c52.7-33.9,105.3-67.7,158-101.5c0.5-0.3,1.3-0.2,2.1-0.4c2.3,2.1,1.2,4.9,1.2,7.3c0.1,189.6,0.1,379.3,0.1,568.9
|
||||
c0,1.7-0.1,3.3,0,5c0.2,3.6-1.7,4.6-4.9,4.5c-5-0.2-10-0.3-15-0.5c-17-0.5-34-1-51-1.4c-7.4-0.2-7.5-0.2-7.6-7.8
|
||||
c-0.5-35.3-1.1-70.6-3.1-105.9c-2.6-45.9-6.4-91.7-12.8-137.3c-4.2-30.3-9.4-60.5-17.3-90.2c-1.1-4-0.2-6.6,2.5-9.8
|
||||
c27.4-31.6,51.5-65.6,70.4-102.8c11.3-22.2,21.7-44.9,28.2-69.2C2244.6,460.9,2245.4,458.8,2244.8,454.3z"/>
|
||||
<path fill="none" d="M553.7,662.8c3.8,0.2,5.2,2.8,6.9,4.6c14.5,15.2,30.3,29,46.9,41.9c4.1,3.2,5.8,6.4,5.8,11.7
|
||||
c-0.2,85.6-0.1,171.2-0.1,256.8c0,7.5-0.1,7.7-7.6,7.8c-25,0.4-50,0.6-74.9,1.1c-4.3,0.1-5.9-1.2-5.7-5.6c0.4-13,0.2-26,0.5-39
|
||||
c1.5-60.3,4.7-120.5,11.8-180.4c3.7-31.7,8.3-63.3,15.4-94.5C553,665.6,553.4,664.1,553.7,662.8z"/>
|
||||
<path fill="none" d="M553.6,1319.7c-22.4-107.4-27.4-215.3-28.7-323.3c3.3-2.5,6.4-1.5,9.3-1.5c23.3,0.3,46.7,0.7,70,1.1
|
||||
c9.5,0.1,9.2-1.2,9.2,9.2c0,72.3,0,144.7,0,217c0,13.3-0.1,26.7,0.1,40c0,3.6-0.7,6.2-3.8,8.6c-18.7,14.5-36.3,30.3-52.7,47.4
|
||||
C556.3,1318.5,555.6,1318.7,553.6,1319.7z"/>
|
||||
<path fill="none" d="M2119.8,1319.2c-6.7-6.5-12.1-12.3-18-17.5c-11.8-10.2-23.8-20.2-36-30.1c-4.2-3.4-5.9-7.1-5.9-12.7
|
||||
c0.2-84.6,0.1-169.1,0.1-253.7c0-8.4,0.1-8.5,8-8.5c24.3-0.2,48.6-0.4,72.9-0.5c7.5,0,7.7,0,7.6,7.7c-0.3,45.6-1.9,91.2-5.1,136.7
|
||||
c-2.7,39.9-6.7,79.6-12.5,119.1C2128.1,1279,2124.6,1298.3,2119.8,1319.2z"/>
|
||||
<path fill="none" d="M2120.4,664.4c12.3,57.5,18.7,115.7,22.8,174.2c3.1,44.9,4.7,89.8,5.2,134.8c0,2.3-0.1,4.7,0.1,7
|
||||
c0.3,4-1.7,5.2-5.4,5.1c-6.7-0.2-13.3-0.3-20-0.3c-18.3-0.1-36.7-0.2-55-0.4c-8-0.1-8.2-0.1-8.2-8.2c0-52.3,0-104.6,0-157
|
||||
c0-32,0.2-64-0.2-96c-0.1-6.3,2-10.3,6.8-14.3c17.1-14.3,35.1-27.7,50.2-44.2C2117.5,664.3,2118.3,663.8,2120.4,664.4z"/>
|
||||
<path fill="none" d="M878.7,1138.5c2,4.9,3.2,9.1,5.3,12.8c4,7.2,10.5,7.1,14.6,0c3-5.3,4.3-11.2,5.2-17.1c0.8-4.7,3.2-7.1,7.8-8.9
|
||||
c37.6-14.8,76.3-25.6,115.5-34.7c5-1.1,6.6,0.4,7.3,4.9c0.6,3.5,0.7,8.4,5.3,8.5c5.3,0.1,5.5-5.2,5.8-9c0.5-7.1,3.9-10,10.9-11.2
|
||||
c40-7,79.8-14.9,120.4-18c2.4-0.2,5.9-2.1,6.9,1.5c1,3.7,1.8,7.8,0.4,11.7c-0.7,1.8-2.9,1.2-4.4,1.5c-9.5,1.6-19.1,2.8-28.6,4.6
|
||||
c-39.6,7.4-78.6,17-117,29.2c-59.9,19-116.5,45.2-170.8,76.4c-41.6,24-80.2,52.3-117.2,83.1c-39.4,32.8-74.3,69.6-105.4,110.2
|
||||
c-3,3.9-6.2,7.7-9.4,11.6c-0.1,0.2-0.6,0.2-2.1,0.5c0.2-7-1.1-13.6,1.5-20.3c7.6-19.2,12.9-39.2,16.7-59.5
|
||||
c1.8-9.6,6.2-17.5,12-24.8c17.2-21.7,36.5-41.5,58.8-58.9c3.4,2.2,3.5,6.6,5.4,9.8c4,7,7.8,14.4,17.4,14.4
|
||||
c9.6,0,13.8-7.2,17.3-14.4c6.5-13.3,9.4-27.7,12.2-42c0.8-3.9,2.2-6.5,5.7-8.6C808.9,1171.4,842.7,1153.5,878.7,1138.5z"/>
|
||||
<path fill="none" d="M1032.8,891.9c-42-9-82.2-20.4-121.3-35.7c-4.8-1.9-6.9-4.4-7.7-9c-0.9-5.6-2.3-11.1-4.8-16.2
|
||||
c-1.5-3.2-3.7-5.9-7.3-6.1c-4.1-0.2-6.8,2.5-8.3,6.1c-1.6,3.6-2.8,7.4-4.7,12.2c-11.4-5.2-22.6-10-33.5-15.2
|
||||
c-23.1-11.1-45.4-23.7-67.3-37.1c-4.3-2.6-6.6-5.5-7.5-10.7c-2.1-11.4-4.6-22.8-8.9-33.7c-2.1-5.3-4.2-10.5-7.8-15
|
||||
c-7.6-9.4-17.5-9.4-25.3,0.1c-4.2,5.2-6,11.7-9.8,18.2c-11.1-9.9-22.3-18.8-32.2-28.9c-10.2-10.4-19.5-21.8-28.7-33.3
|
||||
c-5.6-7-8.9-15.3-10.6-24.4c-3.7-19.3-8.6-38.3-16-56.5c-2.9-7.1-1.4-14.1-0.5-21.8c3.9,4.8,7.9,9.4,11.6,14.3
|
||||
c23.7,32,51,60.7,80.1,87.7c19,17.6,39.4,33.9,59.9,49.7c45,34.7,93.8,63.3,144.7,87.9c39.2,18.9,79.8,34.7,121.6,47.1
|
||||
c41.2,12.2,82.8,22.3,125.4,28.2c1,0.1,2,0.2,3,0.4c7.3,1.2,10.5,7.7,7,14.2c-1.3,2.3-3.4,1.7-5.3,1.6c-8-0.7-15.9-1.5-23.8-2.4
|
||||
c-32.8-3.7-65.1-10.2-97.5-15.7c-7.1-1.2-11.5-3.9-11.7-11.6c0-1.3-0.4-2.6-0.8-3.9c-0.8-2.6-1.9-5.3-5.3-4.9
|
||||
c-2.7,0.3-3.7,2.6-4.4,4.9C1034.4,885.3,1033.7,888.2,1032.8,891.9z"/>
|
||||
<path fill="none" d="M1953.8,1233.7c3.8,1,5.8,3.4,8,5.3c21,19.1,40.7,39.4,57.6,62.2c3,4.1,4.9,8.4,6,13.5
|
||||
c4.5,20.5,9.3,40.9,17.1,60.4c1.3,3.1,2.4,6.3,1.8,9.7c-0.6,3.6,2.4,8.9-2.1,10.4c-2.5,0.8-5.4-3.9-7.3-6.7
|
||||
c-35.7-51.5-81-93.6-129.4-132.6c-40.8-32.9-84.7-61.2-131-85.6c-39.5-20.8-80.6-37.9-123-52c-45.9-15.2-92.6-26.8-140.3-34.3
|
||||
c-6.2-1-12.4-2.7-18.7-3.1c-6.5-0.4-5.5-4.5-5.2-8.4c0.4-5.4,2.1-7.1,7.5-6.6c8.3,0.7,16.5,1.9,24.8,2.7
|
||||
c33.2,3.4,65.8,10.1,98.6,15.7c6.3,1.1,9.2,3.4,9.6,10c0.2,3.6,0.3,9.3,5.3,9.6c6,0.4,5.5-5.6,6.3-9.5c0.7-3.9,2.4-5,6.2-4
|
||||
c39.4,9.7,78.9,19.2,116.7,34.3c4.5,1.8,6.8,4.2,7.5,9.1c0.9,5.9,2.2,11.8,5.1,17.2c1.7,3.1,3.8,5.6,7.6,5.6c3.8,0,6-2.5,7.6-5.6
|
||||
c0.9-1.8,1.7-3.6,2.3-5.5c2.8-7.7,2.9-7.9,9.9-4.7c32.6,15.3,64.6,31.4,95,50.8c3.4,2.2,5.1,4.7,5.8,8.6
|
||||
c2.1,12.1,4.9,24.1,9.4,35.7c1.9,5,4.1,9.8,7.5,14c7.5,9.4,17.7,9.4,25.3,0C1948.9,1245.3,1950.9,1239.5,1953.8,1233.7z"/>
|
||||
<path fill="none" d="M2043.6,585.1c0.9,7.5,1.7,14.1-0.8,20.4c-8,20.2-13.1,41.2-17.6,62.4c-0.9,4.4-2.6,8.3-5.2,11.7
|
||||
c-17.8,23.8-38.4,45.2-60.6,64.9c-1.5,1.3-3.2,2.3-5.1,3.8c-2.9-5.3-5.1-10.4-8.2-14.9c-8-11.8-19.3-11.8-27.5-0.1
|
||||
c-7.4,10.6-11.4,23-12.8,35.4c-1.7,14.6-9.4,22.2-21.3,29.3c-25.5,15.2-52.1,28.1-78.7,41.2c-2.4,1.2-5,2-7.3,3.1
|
||||
c-2.7,1.3-4.3,0.7-5.2-2.2c-1-3.2-1.9-6.4-3.4-9.3c-4.2-7.7-10.8-7.7-15.1-0.1c-2.5,4.4-3.9,9.3-4.5,14.2c-0.8,7-4.2,10.7-11,13.3
|
||||
c-27.5,10.2-55.4,18.9-83.9,25.7c-10,2.4-20.1,4.8-30,7.5c-3.8,1-5.4,0-6.1-3.9c-0.7-4-0.2-9.9-6.2-9.6c-5.1,0.2-5.3,6-5.4,9.5
|
||||
c-0.1,7.5-4.3,9-10.5,10.2c-39.6,7.4-79.4,13.9-119.5,18.1c-2.8,0.3-6.7,2.3-8.1-1.3c-1.4-3.6-1.7-7.9-0.3-11.7
|
||||
c0.9-2.4,3.9-2,6.2-2.4c13.8-2.4,27.6-4.4,41.3-7c42.1-8.2,83.5-19.3,124.2-33.1c57.6-19.6,112.2-45.7,163.9-77.3
|
||||
c38.1-23.2,73.8-50.1,107.6-79.4c37.9-32.8,72.6-68.6,102.5-109C2037.5,591.3,2039.4,587.8,2043.6,585.1z"/>
|
||||
<path fill="none" d="M777.2,1163.8c-0.9-7.9,1.2-15.4,2-23c4.4-45.1,6.3-90.3,6.6-135.6c0.1-8.6,0.2-8.7,8.8-8.7
|
||||
c22-0.1,44-0.1,65.9,0c8.3,0,8.4,0.1,8.5,7.9c0.2,37.7,2,75.2,6.5,112.6c0.5,3.8,0.3,6-3.9,7.5c-31.7,11.3-62.2,25.3-92.6,39.6
|
||||
C778.8,1164.3,778.4,1164.1,777.2,1163.8z"/>
|
||||
<path fill="none" d="M776.8,817.8c3.1-0.6,4.7,0.6,6.5,1.4c28.2,13.5,56.6,26.5,86.2,36.8c5.4,1.9,6.7,4.3,6,10.1
|
||||
c-4.5,36.7-6.3,73.6-6.4,110.6c0,8.1,0.7,8.3-8.1,8.3c-22.6-0.1-45.3,0-67.9-0.1c-7,0-7.1-0.2-7.2-7.3c-0.4-28.6-0.9-57.3-2.7-85.9
|
||||
c-1.3-21.3-2.9-42.5-5.5-63.7C777.2,824.9,777.1,821.6,776.8,817.8z"/>
|
||||
<path fill="none" d="M1896.4,1163.9c-3.3,0.5-5.7-1.5-8.4-2.8c-27.2-13.3-54.8-25.7-83.3-36.1c-7.4-2.7-7.2-2.9-6.3-10.6
|
||||
c4.4-35.8,5.3-71.8,6.4-107.7c0.1-1.7,0.1-3.3,0-5c-0.3-3.8,1.3-5.4,5.2-5.3c6.3,0.2,12.7,0.1,19,0.1c16.7,0,33.3,0,50,0.1
|
||||
c8.4,0,8.4,0.1,8.6,8.8c0.8,30.3,1.2,60.6,3.3,90.9c1.4,19.9,2.9,39.9,5.3,59.7C1896.5,1158.6,1897.4,1161.2,1896.4,1163.9z"/>
|
||||
<path fill="none" d="M1896.8,817.7c-1.3,13.9-2.7,27.7-3.9,41.6c-3.5,38.5-4.5,77-5.2,115.6c0,1.7-0.2,3.3-0.1,5
|
||||
c0.4,4-1.5,5.5-5.2,5.1c-1-0.1-2,0-3,0c-22.6,0-45.3-0.1-67.9,0.2c-5,0-6.8-1.2-6.8-6.6c-0.4-28.3-1.2-56.5-3.6-84.7
|
||||
c-0.8-8.9-1.6-17.9-2.7-26.8c-0.9-7.6-1.1-7.8,6.5-10.6c23-8.7,45.8-17.8,67.9-28.7c6-2.9,12-5.7,18-8.6
|
||||
C1892.5,818.3,1894.3,817.4,1896.8,817.7z"/>
|
||||
<path fill="none" d="M908.3,1109.6c-1-4.4,0.4-8.6,0.8-12.8c2.5-30.9,4-61.8,4-92.9c0-7.4,0.2-7.5,7.9-7.5
|
||||
c33.3-0.1,66.7-0.2,100-0.2c7.8,0,8.1,0.1,8,7.4c-0.3,22,1,44,2.3,65.9c0.3,4.5-1,6.5-5.5,7.4c-38.7,8.5-76.4,20.9-114.2,32.5
|
||||
C910.7,1109.8,909.7,1109.6,908.3,1109.6z"/>
|
||||
<path fill="none" d="M1765.3,1110.4c-9.6-3-18.1-5.6-26.6-8.4c-29.1-9.6-58.7-17.4-88.5-24.5c-8.9-2.1-8.1-1.4-7.6-10.4
|
||||
c1.2-21,1.6-42,2.3-62.9c0.2-7.7,0.2-7.9,7.5-7.9c33.3,0,66.7,0.1,100,0.2c7.8,0,8.2,0.3,8.2,7.7c0.2,34,1.7,68,4.8,101.8
|
||||
C1765.4,1106.9,1765.3,1107.9,1765.3,1110.4z"/>
|
||||
<path fill="none" d="M908.8,872.2c2.8-1.1,5,0.8,7.5,1.5c36.1,10.3,71.6,22.9,108.5,30.5c5.5,1.1,6.7,3.6,6.4,8.8
|
||||
c-1.3,21.9-2.6,43.9-2.1,65.9c0.1,4.7-1.2,6.5-6.1,6.4c-34.7-0.2-69.3-0.2-104-0.1c-4.1,0-6.1-1-6-5.6c0.4-35.6-2-71.2-4.8-106.7
|
||||
C908.2,872.8,908.6,872.5,908.8,872.2z"/>
|
||||
<path fill="none" d="M1765.6,872.4c-0.9,10.7-1.8,20.9-2.5,31.2c-1.7,24.6-2.4,49.2-2.6,73.8c-0.1,7.7-0.2,7.8-7.7,7.8
|
||||
c-33.6,0.1-67.2,0.1-100.8,0.1c-6.9,0-6.9-0.2-7.1-7.1c-0.8-21.9-1.3-43.9-2.6-65.8c-0.3-5.4,1.8-7,6.2-8
|
||||
c14.5-3.5,29-7.1,43.5-10.8c22.5-5.7,44.5-13.5,66.7-20.3C1760.5,872.7,1762.3,871.8,1765.6,872.4z"/>
|
||||
<path fill="none" d="M725.8,1192.5c-5.9-34.3-8.8-67.4-11-100.6c-1.8-27.6-2.2-55.2-3-82.8c-0.1-2.3,0.1-4.7-0.1-7
|
||||
c-0.4-4.4,1.6-5.8,5.9-5.7c9,0.2,18,0,27,0c6.7,0,13.3-0.1,20,0c7.2,0.1,7.4,0.2,7.3,7.1c-0.5,23.6-1,47.2-2,70.8
|
||||
c-1.2,27.2-3.3,54.4-6.7,81.5c-0.2,1.3-0.3,2.6-0.4,4c-0.4,9.2-3.4,16-13,19.7C741.7,1182.7,734.5,1187.7,725.8,1192.5z"/>
|
||||
<path fill="none" d="M1946.2,1191.7c-10.1-5.6-20.4-11.5-30.8-17.2c-3.3-1.8-3.1-5-3.5-7.8c-1.4-10.2-2.4-20.5-3.5-30.7
|
||||
c-4.7-43.8-6-87.7-6.7-131.6c-0.1-7.7,0.1-7.8,7.7-7.8c15.3-0.1,30.6,0.1,46-0.1c4.6-0.1,6.5,1.1,6.4,6.1
|
||||
c-0.5,34-1.2,67.9-3.7,101.8c-2,26.9-4.5,53.7-8.9,80.4C1948.7,1187,1949.1,1189.5,1946.2,1191.7z"/>
|
||||
<path fill="none" d="M1947.7,789.1c5.8,32.7,8.6,63.8,10.8,95c2.1,30.6,2.8,61.2,3.1,91.9c0.1,9.6,0.3,8.9-8.3,8.9
|
||||
c-15,0-30-0.1-45,0.2c-5.3,0.1-6.9-1.8-6.7-7c0.9-25.3,1.3-50.6,2.5-75.9c1.3-27.3,3.9-54.4,6.9-81.6c0.9-8.3,3.3-14.2,11.5-17.8
|
||||
C1930.8,799.2,1938.6,794.1,1947.7,789.1z"/>
|
||||
<path fill="none" d="M726,789.2c11.3,6.2,21.7,11.9,32.2,17.5c2.9,1.6,2.8,4.2,3.2,6.8c2.7,15.8,4.1,31.7,5.5,47.6
|
||||
c3.3,37.2,4.5,74.5,5,111.8c0,2.3,0,4.7,0.1,7c0.1,3.1-0.8,5-4.4,5c-17-0.1-34-0.1-51-0.1c-3.3,0-4.8-1.5-4.7-4.8
|
||||
c0.2-5.7,0-11.3,0.2-17c1-42.6,2.7-85.2,7.3-127.7C720.9,820.5,722.6,805.7,726,789.2z"/>
|
||||
<path fill="none" d="M1625.1,910.5c-0.6,9.4-1.4,18-1.7,26.5c-0.4,13-0.4,26-0.6,38.9c-0.1,9.3-0.2,9.4-9.2,9.4
|
||||
c-30.6,0.1-61.3,0.1-91.9,0.1c-8.3,0-16.7-0.1-25,0.1c-4.2,0.1-5.9-1.3-5.9-5.7c0-15-0.1-30-0.7-44.9c-0.2-4.8,2.5-5.5,6-6
|
||||
c10.9-1.3,21.8-3.6,32.7-3.7c23.9-0.2,46.7-6.5,70-10C1607.1,913.9,1615.2,912.2,1625.1,910.5z"/>
|
||||
<path fill="none" d="M1625.2,1071.1c-23.5-4.2-45.7-8.1-67.9-12.1c-7.9-1.4-15.8-2.4-23.8-2.5c-12.7-0.1-25.1-2.6-37.7-4
|
||||
c-4-0.4-5.8-1.6-5.6-6.1c0.5-14.6,0.6-29.3,0.6-43.9c0-4.5,1-6.5,6.1-6.5c39.9,0.2,79.9,0.2,119.8,0.1c5,0,6.1,2.1,6.1,6.6
|
||||
C1622.9,1025,1623,1047.3,1625.2,1071.1z"/>
|
||||
<path fill="none" d="M1049.3,1070.4c-1-5.9-0.1-11.8,0.3-17.7c0.9-16.3,1.4-32.6,1.4-48.9c0-7.3,0.2-7.5,6.8-7.5
|
||||
c39.3-0.1,78.5-0.1,117.8-0.1c7,0,7.1,0.2,7.2,7.2c0.2,14,0,27.9,0.5,41.9c0.2,5.2-1.5,6.6-6.6,7c-16.9,1.3-33.8,2.7-50.6,5.2
|
||||
c-23.3,3.4-46.7,6.8-69.7,12.3C1054.2,1070.3,1051.9,1071.4,1049.3,1070.4z"/>
|
||||
<path fill="none" d="M1049.8,910.2c22.1,5.7,44.8,8.7,67.4,12.5c16.4,2.8,33.1,3.3,49.6,5.2c16.4,1.9,16.4,1.5,16.3,17.4
|
||||
c-0.1,11-0.3,22-0.2,32.9c0.1,4.9-1,7.2-6.6,7.2c-39.9-0.2-79.9-0.2-119.8-0.1c-3.4,0-5.7-0.6-5.6-4.8c0.6-22.6-1.3-45.2-1.9-67.8
|
||||
C1049,912,1049.4,911.4,1049.8,910.2z"/>
|
||||
<path fill="none" d="M1260.5,985.4c-21.6,0-43.2,0-64.9,0c-8,0-8.2-0.1-8.1-8.4c0.2-13.6,0.6-27.3,0.8-40.9c0.1-4.2,1.4-5.6,6-5.1
|
||||
c41.7,4.1,83.5,6.2,125.5,6.2c2.3,0,4.7,0,7,0c7.5,0.2,7.5,0.2,7.6,7.9c0.1,10.3,0.1,20.6,0.1,30.9c0,9.3,0,9.4-9.1,9.4
|
||||
C1303.8,985.4,1282.2,985.4,1260.5,985.4z"/>
|
||||
<path fill="none" d="M1485,931.5c0.3,16.2,0.5,32.7,0.8,49.3c0.1,3.4-1.7,4.8-4.9,4.6c-2-0.1-4,0-6,0c-41.6,0-83.2,0-124.9,0
|
||||
c-2,0-4-0.1-6,0c-4,0.3-5.1-1.8-5-5.5c0.1-11.7,0-23.3,0.1-35c0.1-7.6,0.1-7.8,7.6-7.8c17.3,0.2,34.6-0.6,51.9-1.1
|
||||
c26.3-0.7,52.5-3.1,78.7-5.1C1479.6,930.9,1481.8,930,1485,931.5z"/>
|
||||
<path fill="none" d="M1485,1051.1c-35.4-3-69.9-5.3-104.5-6.4c-11.6-0.4-23.3-0.5-34.9-0.4c-4.4,0-6.7-0.8-6.5-6
|
||||
c0.4-12,0.3-24,0-36c-0.1-4.8,1.6-6.3,6.3-6.2c45,0.1,89.9,0.1,134.9,0c3.5,0,5.8,0.7,5.7,4.8
|
||||
C1485.5,1017.5,1485.3,1034.1,1485,1051.1z"/>
|
||||
<path fill="none" d="M1188.4,1050.6c-0.3-17.2-0.5-33.7-0.8-50.3c0-2.9,1.6-4.1,4.4-4.1c1.7,0,3.3,0,5,0c42.6,0,85.1,0,127.7,0
|
||||
c1.7,0,3.3,0.1,5,0c3.3-0.2,4.9,1.2,4.9,4.6c-0.1,13-0.1,25.9,0,38.9c0,3.6-1.7,4.7-4.9,4.5c-0.7,0-1.3,0-2,0
|
||||
C1281.6,1043.7,1235.9,1047.2,1188.4,1050.6z"/>
|
||||
<path fill="none" d="M2010.7,1230.1c-5.1-1.9-9-5.1-13.1-7.7c-9.9-6.1-19.6-12.6-29.8-18.3c-4.8-2.6-5.5-5.6-4.5-10.4
|
||||
c3.9-20,6.2-40.1,8.2-60.4c4-41.2,5.8-82.4,6.1-123.8c0-3,0.1-6,0-9c-0.1-3,1.1-3.8,4.1-4.1c16.5-1.5,16.7-1.4,16.8,14.7
|
||||
c0.4,66.4,3.9,132.5,10.7,198.6C2009.9,1216.3,2011.6,1222.7,2010.7,1230.1z"/>
|
||||
<path fill="none" d="M2011.2,752.5c-1.6,16.9-3.2,33.8-4.8,50.6c-3,30.8-4.6,61.7-6.1,92.6c-1.3,27.3-1.5,54.5-1.8,81.8
|
||||
c-0.1,7.1-0.2,7.3-7.1,7.3c-16.3,0-13.6,1.3-13.7-12.8c-0.5-54.3-3.2-108.4-10.6-162.2c-1.1-7.9-2.5-15.8-3.9-23.6
|
||||
c-0.7-3.6-0.1-6,3.5-8.2c13.4-8,26.6-16.4,39.8-24.7C2007.6,752.8,2008.5,751.7,2011.2,752.5z"/>
|
||||
<path fill="none" d="M662.1,1231.5c1.9-20.4,3.7-38.6,5.4-56.8c3-31.9,4.7-63.8,6-95.8c1-24.6,1.6-49.3,1.6-74
|
||||
c0-8.3,0.1-8.3,7.9-8.4c1.3,0,2.7,0,4,0c8.6,0.1,8.7,0.1,8.8,8.7c0.4,40,1.8,80,5.4,119.8c2,22.9,4.4,45.8,9,68.3
|
||||
c1.1,5.5,0.2,8.5-5.1,11c-12.7,6-23.9,14.5-35.5,22.2C667.7,1227.9,665.8,1229.1,662.1,1231.5z"/>
|
||||
<path fill="none" d="M662.9,752.1c2.6-0.9,3.9,1,5.4,2.1c12.4,8.4,24.6,17.1,38,23.9c3.3,1.7,5,3.6,4.2,7.7
|
||||
c-11.3,61.8-13.7,124.4-14.8,187c-0.2,13.5,2.2,12.1-12.3,12.1c-8.4,0-8.5-0.1-8.5-8.9c0.1-52.7-2.6-105.3-6.4-157.8
|
||||
c-1.5-20.6-4-41.1-6-61.7C662.5,755.1,661.9,753.4,662.9,752.1z"/>
|
||||
<path fill="none" d="M2044.3,728.6c0,3.4,0,6.3,0,9.2c0,78.9,0,157.9,0,236.8c0,1.7-0.1,3.3,0,5c0.4,3.9-1.3,5.4-5.2,5.3
|
||||
c-5.9-0.2-11.9,0-18.3,0c0-26.3,0.3-51.6,1.3-76.8c1.7-42.3,4.4-84.5,8.9-126.5c1.6-14.9,3.9-29.7,5.6-44.6
|
||||
C2037.2,732.4,2038.9,729.7,2044.3,728.6z"/>
|
||||
<path fill="none" d="M2044.3,1253c-5.6-1.2-7-4.1-7.7-8.6c-3.1-19.4-5.2-38.9-7-58.5c-5.6-59-8.7-118.2-8.7-177.6
|
||||
c0-13.6-1.4-11.7,11.5-11.8c13.3,0,11.9-1.5,11.9,11.5c0,78.6,0,157.2,0,235.8C2044.3,1246.6,2044.3,1249.3,2044.3,1253z"/>
|
||||
<path fill="none" d="M629.2,728.7c5.8,1.6,7.4,4.6,7.9,9c2.3,21.2,5.3,42.2,7.3,63.4c5.3,55.3,7.7,110.8,8.5,166.4
|
||||
c0,3.3,0.1,6.7,0,10c-0.1,7.5-0.1,7.5-7.7,7.7c-16,0.5-16,0.5-16-15.3c0-76.5,0-153.1,0-229.6C629.2,736.7,629.2,733,629.2,728.7z"
|
||||
/>
|
||||
<path fill="none" d="M629.2,1253.2c0-6,0-10.6,0-15.3c0-75.7,0-151.3,0-227c0-2,0-4,0-6c-0.1-9.2-0.3-8.7,8.7-8.6
|
||||
c15.1,0,15.2,0,15,15.6c-0.7,62.7-3.8,125.2-10.3,187.6c-1.5,14.9-3.9,29.7-5.5,44.6C636.6,1248.7,634.5,1251.1,629.2,1253.2z"/>
|
||||
<path fill="none" d="M2060,697.5c0-15.9,0-29.5,0-43c0-3.7,0.3-7.3-0.1-11c-2-19,6.4-34.8,15.3-50.5c1-1.7,2.3-3.2,3.6-4.7
|
||||
c5.4-6.2,8.1-6.2,13.4,0.4c6.3,7.8,10.3,17.1,13.7,26.4c3.2,8.7,5.9,17.6,8.7,26.5c1.3,4,1,7.1-2.4,10.7
|
||||
C2096.7,668.2,2079,681.6,2060,697.5z"/>
|
||||
<path fill="none" d="M613.2,1283.1c0,24.5,1.5,47.6-0.5,70.3c-1.4,15.4-9,29.8-19.7,41.7c-3.8,4.2-5.7,3.9-10-0.4
|
||||
c-4.3-4.3-7.2-9.6-9.8-15c-6.7-13.8-11.2-28.4-15.3-43.2c-0.8-3-1.3-5.6,1.5-8.1C576.3,1313,592.4,1296.7,613.2,1283.1z"/>
|
||||
<path fill="none" d="M2060,1284.4c19.3,15.5,36.8,29.3,52.5,45.2c2.6,2.6,3.7,5.1,2.6,8.8c-5.1,17.2-10,34.5-19.7,50
|
||||
c-7.7,12.2-12.4,12.7-19.8,0.6c-9.5-15.4-16.9-31.4-15.8-50.5C2060.7,1321.2,2060,1303.9,2060,1284.4z"/>
|
||||
<path fill="none" d="M611.9,698.2c-8.5-6.7-17.1-12.8-25-19.7c-9.1-7.8-17.5-16.4-26.3-24.5c-2.3-2.2-4-4.1-3-7.7
|
||||
c5.3-18.2,10.3-36.5,20.4-52.8c7.8-12.7,12.8-13,20.4-0.3c8.9,14.9,16.3,30.2,15.1,48.6c-1,15.9-0.2,31.9-0.3,47.9
|
||||
C613.2,692.2,614,694.9,611.9,698.2z"/>
|
||||
<path fill="none" d="M1788.2,864c4.1,27,4.9,53.5,6.1,80.1c0.5,11.3,0.2,22.6,0.7,33.9c0.2,5.2-1.6,7.4-6.7,7.3
|
||||
c-4.7-0.1-9.3-0.3-14,0c-4.6,0.3-5.5-2-5.4-6c0.2-26.3,1.3-52.5,3.2-78.8c0.5-7.6,1.2-15.2,2.1-22.8
|
||||
C1775.6,867.2,1775.7,867.2,1788.2,864z"/>
|
||||
<path fill="none" d="M885.3,1118.4c-5.1-39.1-6.6-77.6-6.6-116.3c0-4.2,1.3-5.9,5.7-5.8c25,0.7,20.1-4.1,19.8,19.4
|
||||
c-0.4,29.6-1.5,59.3-5,88.8C897.9,1115.4,898.1,1115.4,885.3,1118.4z"/>
|
||||
<path fill="none" d="M885.7,863.5c12.4,2.1,12.2,2.1,13.5,13.8c3.8,33,4.6,66.2,5.3,99.4c0.2,8.3,0,8.5-7.9,8.4
|
||||
c-5.6-0.1-12.6,2.4-16.3-1.2c-3.8-3.6-1.3-10.7-1.3-16.2c0.1-33.9,1.8-67.7,6.1-101.4C885.1,865.4,885.4,864.5,885.7,863.5z"/>
|
||||
<path fill="none" d="M1788.4,1117.5c-4,0.3-6.2-1.5-8.7-2.2c-3.2-0.9-4.6-2.9-4.9-6.5c-0.7-10.3-2.2-20.5-2.9-30.8
|
||||
c-1.5-24.6-2.8-49.2-2.8-73.9c0-7.5,0.1-7.8,7.7-7.6c5.9,0.2,13.3-2.7,17.2,1.6c3.4,3.7,1,10.9,0.9,16.5
|
||||
C1794.3,1048.8,1792.8,1083.1,1788.4,1117.5z"/>
|
||||
<path fill="none" d="M1481.1,1078.1c-10.7,0.3-20.5-1.5-30.3-2.6c-34.1-3.7-68.2-5.9-102.5-5.6c-1.3,0-2.7-0.1-4,0
|
||||
c-4.9,0.5-5.1-2.5-5.2-6.2c-0.1-4,0.7-6,5.4-6c20,0.1,39.9-0.2,59.9,1c22.9,1.4,45.8,3,68.7,4.7c7.9,0.6,7.8,0.9,8.2,8.7
|
||||
C1481.2,1073.7,1481.1,1075.4,1481.1,1078.1z"/>
|
||||
<path fill="none" d="M1480,903c2,3.3,0.7,6.3,1,9.2c0.4,4.3-1.6,5.6-5.8,5.9c-42.2,3.3-84.3,6.2-126.7,5.9c-1.3,0-2.7-0.1-4,0
|
||||
c-4.5,0.5-5.5-1.7-5.6-5.8c-0.1-4.5,0.9-6.6,6-6.5c29.7,0.7,59.2-1.5,88.8-3.9C1449.3,906.5,1464.7,904.6,1480,903z"/>
|
||||
<path fill="none" d="M1192.2,1078.2c1-14.7,0.9-15,13.5-15.8c40.5-2.9,80.9-5.4,121.6-4.6c7.3,0.1,7.4,0.5,7.3,7
|
||||
c-0.1,4-1.7,5.2-5.4,5.1c-6-0.2-12-0.1-18,0c-37.3,0.2-74.3,3.2-111.3,7.8C1197.6,1078,1195.3,1078,1192.2,1078.2z"/>
|
||||
<path fill="none" d="M1192.6,903.2c17.2,2.2,33.7,3.8,50.2,5c28.2,2.1,56.4,4.4,84.7,3.6c3.8-0.1,7.3-0.4,7.1,5.4
|
||||
c-0.3,6.3-0.2,6.8-6.6,6.8c-18.6,0-37.2,0.4-55.9-0.8c-19.2-1.2-38.5-2.1-57.8-3.2c-3-0.2-6-0.4-8.9-0.8
|
||||
C1192.8,917.6,1192.8,917.6,1192.6,903.2z"/>
|
||||
<path fill="none" d="M653.6,694.6c2.5,2.7,3.4,3.5,4.2,4.6c15.1,20.9,32.8,39.4,51.8,56.7c2.5,2.3,7.7,4.8,5,8.5
|
||||
c-2.7,3.9-6.2-1-8.9-2.4c-14.7-7.8-28.1-17.9-42.2-26.6c-3.1-1.9-4.3-4.4-4.8-8C657.4,717,655.5,706.9,653.6,694.6z"/>
|
||||
<path fill="none" d="M654,1284.6c0.9-5.5,2.7-11,2.6-16.4c-0.1-14.4,6.3-23.8,19.2-30.2c10.6-5.3,20.3-12.5,30.5-18.8
|
||||
c2.5-1.5,4.9-3.3,7.8-3.6c1.6,2.1,1.5,4,0.2,5.7c-1,1.3-2.4,2.3-3.6,3.4c-18.4,17.3-36.5,34.8-51.2,55.5c-1.1,1.6-2.6,2.9-4,4.3
|
||||
C655.3,1284.8,654.8,1284.6,654,1284.6z"/>
|
||||
<path fill="none" d="M2019.1,698.8c-1.4,9.8-3,19.6-4.2,29.5c-0.4,3-2.2,4.7-4.4,6.2c-15.3,9.9-30.7,19.7-46.1,29.5
|
||||
c-1.5,1-3.7,4-5.7,0.4c-1.5-2.6,0.2-4.4,2.1-5.9c11.9-9.2,21.5-20.7,32.4-30.8c8.4-7.7,14.5-17.2,21.4-26.1c0.9-1.2,2-2.4,3-3.6
|
||||
C2018.1,698.2,2018.6,698.5,2019.1,698.8z"/>
|
||||
<path fill="none" d="M2017.1,1283.9c-11.8-19.6-28.7-34.6-44.6-50.5c-3.7-3.7-7.8-7.2-11.8-10.6c-1.9-1.7-3.4-3.4-1.7-5.9
|
||||
c1.7-2.4,3.5-0.5,5,0.5c15.7,10,31.3,20,46.9,30c2,1.3,3.5,2.9,3.9,5.4c1.4,10.1,2.9,20.2,4.4,30.4
|
||||
C2018.5,1283.4,2017.8,1283.7,2017.1,1283.9z"/>
|
||||
<path fill="none" d="M1044.6,1034c-0.6,11-1.1,21.9-1.7,32.9c-0.1,1.6-0.3,3.3-0.8,4.9c-0.3,0.8-1.4,1.6-2.2,1.7
|
||||
c-0.9,0.1-1.9-0.5-2.1-1.7c-0.1-1-0.3-2-0.4-3c-1.2-22.2-2.3-44.5-2.4-66.8c0-3.2,0-6,4.6-5.9c3.5,0.1,5.6,0.7,5.5,4.9
|
||||
c-0.2,11-0.1,21.9-0.1,32.9C1044.9,1034,1044.8,1034,1044.6,1034z"/>
|
||||
<path fill="none" d="M1636.4,1033.8c0,10.6,0,21.2,0,31.8c0,2-0.3,4-0.7,5.9c-0.2,0.8-1.2,1.8-1.9,1.8c-0.8,0-1.9-0.8-2.2-1.5
|
||||
c-0.5-1.2-0.5-2.6-0.6-3.9c-1.9-21.8-2.2-43.7-2.1-65.6c0-5.7,1.3-6.7,6-6.1c3.9,0.5,3.6,3.1,3.6,5.8c0,10.6,0,21.2,0,31.8
|
||||
C1637.8,1033.8,1637.1,1033.8,1636.4,1033.8z"/>
|
||||
<path fill="none" d="M1638.5,948.1c0,10.3,0,20.6,0,31c0,2.6,0.5,5.4-3.3,6.2c-4.4,0.9-6.4-0.5-6.4-5.8c0-19,0-38,1.4-56.9
|
||||
c0.2-3,0.4-6,0.7-9c0.2-2.2-0.3-5.4,3-5.2c2.5,0.1,2.2,3,2.3,4.9c0.6,11.6,1.1,23.3,1.6,34.9C1638,948.1,1638.2,948.1,1638.5,948.1
|
||||
z"/>
|
||||
<path fill="none" d="M1045.1,948c0,10.6-0.1,21.2,0,31.9c0,3.6-0.9,5.5-5,5.5c-3.9-0.1-5.1-1.7-5.1-5.4c0.1-22.6,1.2-45.1,2.5-67.7
|
||||
c0.1-1.6-0.1-3.7,2-4c2.8-0.4,3.1,1.9,3.2,3.9c0.7,11.9,1.3,23.9,1.9,35.8C1044.8,948,1044.9,948,1045.1,948z"/>
|
||||
<path fill="none" d="M741.9,740.1c7,10.8,9.7,22,12.6,34.5c-8.1-3.6-13.5-8.9-19.7-13c-2-1.3-2-3.3-1.2-5.3
|
||||
C735.6,751.1,737,745.6,741.9,740.1z"/>
|
||||
<path fill="none" d="M1931.7,1241.3c-6.9-10.6-9.7-21.7-12.1-33.3c4.2,0.4,6.4,3.1,9.1,4.9
|
||||
C1943.2,1223.1,1943.4,1224.7,1931.7,1241.3z"/>
|
||||
<path fill="none" d="M1931.6,740c12,17.2,11.9,18.3-3.6,29.1c-2.4,1.7-4.5,3.9-8.5,4.8C1921.8,761.9,1925,750.9,1931.6,740z"/>
|
||||
<path fill="none" d="M742.1,1241.3c-5.3-5.2-6.5-11.1-8.6-16.5c-0.8-2-0.5-3.8,1.6-5.1c6-3.9,11.2-9.1,19.2-12.6
|
||||
C751.8,1219.5,748.7,1230.3,742.1,1241.3z"/>
|
||||
<path fill="none" d="M599.6,517.5c0.6,0.1,1.4,0.1,1.9,0.4c12.6,7.1,13,7.9,10.7,24.4c-5.9-7.9-11-15.2-13.4-24.1
|
||||
C599,518,599.3,517.7,599.6,517.5z"/>
|
||||
<path fill="none" d="M597.6,1464.4c4-9.4,8.1-17.2,14-24.4C614.9,1455.5,614.5,1457.9,597.6,1464.4z"/>
|
||||
<path fill="none" d="M2072.4,516.4c2.2,0.6,2.3,2,1.3,3.7c-3.8,6.1-7.7,12.1-11.9,18.6c-2.8-10.6-2.1-12.7,5.3-17.8
|
||||
c1.9-1.3,4.2-2,5.3-4.3L2072.4,516.4z"/>
|
||||
<path fill="none" d="M2061.5,1442.5c4.7,8.5,10.5,14.5,12.5,22.8C2067.2,1459.8,2056.4,1456.8,2061.5,1442.5z"/>
|
||||
<path fill="none" d="M1781.6,836.9c4.2,9.3,4.2,9.3-1.8,12.6C1777.9,845.4,1780.6,842.1,1781.6,836.9z"/>
|
||||
<path fill="none" d="M895.4,849.3c-7.5-2.9-7.5-2.9-3.8-11.1C893.8,841.3,893.9,844.7,895.4,849.3z"/>
|
||||
<path fill="none" d="M630.1,1284.6c-0.2-5.4-2.7-11,1.8-17C634.2,1274.4,629.7,1279.3,630.1,1284.6z"/>
|
||||
<path fill="none" d="M629.9,697.1c0.5,5.4,3.9,10.3,2.4,16.6C626.9,708.5,630.1,702.5,629.9,697.1z"/>
|
||||
<path fill="none" d="M2044.5,1284.5c-2.2-5-4.3-9.2-2.8-15C2046.1,1274.3,2043.5,1279.4,2044.5,1284.5z"/>
|
||||
<path fill="none" d="M2041.9,712.1c-2.2-6,1.1-10.1,2.2-14.6C2043.5,702.2,2046,707.4,2041.9,712.1z"/>
|
||||
<path fill="none" d="M892.2,1143.9c-4.6-7.9-4.6-7.9,1.6-11.5C895.7,1136.5,892.6,1139.7,892.2,1143.9z"/>
|
||||
<path fill="none" d="M1779.6,1132.2c6.1,3.1,6.1,3.1,2.7,11.1c-1.9-1-1.7-3.1-2.3-4.6C1779.2,1137,1778.6,1135.1,1779.6,1132.2z"/>
|
||||
<path fill="none" d="M1934.8,1202.5c4.1-0.6,6.5,2.5,11.4,4.9c-6.3,0.5-8.2-3.5-11.3-5.1L1934.8,1202.5z"/>
|
||||
<path fill="none" d="M1935.8,777c3.1,0.9,4.5-3.5,7.8-2.9c-1.8,4-4.7,3.9-7.9,2.8L1935.8,777z"/>
|
||||
<path fill="none" d="M729.5,774c2.7-0.5,4.4,0.6,5.8,3.2C732.1,778,731,775.7,729.5,774z"/>
|
||||
<path fill="none" d="M734,1202.5c2.3,4.6-1.2,4.8-4.2,5.9c-0.6-3.7,3.7-3.3,4.2-5.7C734,1202.7,734,1202.5,734,1202.5z"/>
|
||||
<path fill="none" d="M1935.2,780.3c-0.4,0.2-0.8,0.5-1.2,0.7c0.3,0,0.6,0.1,0.8,0c0.2-0.1,0.4-0.3,0.6-0.5
|
||||
C1935.5,780.5,1935.2,780.3,1935.2,780.3z"/>
|
||||
<path fill="none" d="M740,1203.1c-1.9-0.8-4.2,2.9-6-0.5c0,0,0,0.2,0,0.1C736,1202.8,738,1202.9,740,1203.1L740,1203.1z"/>
|
||||
<path fill="none" d="M1935.7,777c1.4,1.3,0.9,2.4-0.5,3.3c0,0,0.3,0.3,0.3,0.3c0.1-1.2,0.3-2.3,0.4-3.5
|
||||
C1935.8,777,1935.7,777,1935.7,777z"/>
|
||||
<path fill="none" d="M737.9,778.4c0.2,0.2,0.4,0.5,0.6,0.7c-0.2,0.2-0.5,0.5-0.7,0.5c-0.2,0-0.4-0.4-0.6-0.6
|
||||
C737.4,778.8,737.7,778.6,737.9,778.4z"/>
|
||||
<path fill="none" d="M740,1203.1c0-0.3,0-0.8-0.2-1c-0.8-0.8-0.7-1.1,0.4-0.8C739.5,1201.9,739.4,1202.4,740,1203.1
|
||||
C740,1203.1,740,1203.1,740,1203.1z"/>
|
||||
<path fill="none" d="M1934.9,1202.3c0-0.2-0.1-0.3-0.1-0.5c0,0-0.1,0.1-0.1,0.1c0.1,0.2,0.1,0.4,0.2,0.6
|
||||
C1934.8,1202.5,1934.9,1202.3,1934.9,1202.3z"/>
|
||||
<path fill="none" d="M2072.4,516.6c0.7,0.4,1.4,0.8,2.1,1.2c-0.2,0.2-0.4,0.7-0.6,0.7c-1.1-0.2-1.5-1-1.4-2
|
||||
C2072.4,516.4,2072.4,516.6,2072.4,516.6z"/>
|
||||
<path fill="none" d="M598.7,518.3c-1-0.2-1.8-0.8-1-1.6c0.8-0.9,1.4,0,1.8,0.8C599.3,517.7,599,518,598.7,518.3z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 48 KiB |
|
|
@ -1,134 +0,0 @@
|
|||
Contexts and streaming
|
||||
======================
|
||||
|
||||
The modern core of ``tractor``: a :class:`Context` links a task in
|
||||
one actor to a task in another as a *single* structured concurrency
|
||||
(SC) scope stretched across the IPC boundary — errors, results and
|
||||
cancellation flow between the pair `exactly like trio`_ tasks under
|
||||
a common `nursery`_. Open one with ``Portal.open_context()`` (see
|
||||
:func:`tractor._context.open_context_from_portal` in
|
||||
:doc:`/api/core`), then optionally bridge a bidirectional
|
||||
:class:`MsgStream` between the two tasks.
|
||||
|
||||
.. d2:: diagrams/context_handshake.d2
|
||||
:caption: The ``open_context()`` <-> ``ctx.started()`` handshake.
|
||||
:margin:
|
||||
:alt: parent and child actor context handshake sequence
|
||||
|
||||
For the guided, example-driven tour see :doc:`/guide/context`; this
|
||||
page is the precise API surface.
|
||||
|
||||
.. currentmodule:: tractor
|
||||
|
||||
The ``@context`` decorator
|
||||
--------------------------
|
||||
|
||||
.. autofunction:: context
|
||||
|
||||
.. note::
|
||||
|
||||
The decorated function **must** declare a parameter annotated
|
||||
``tractor.Context`` (any param name works); the runtime injects
|
||||
the context instance there on each remote invocation. Pass
|
||||
``pld_spec`` to type-restrict (and validate) the payloads this
|
||||
endpoint may shuttle — violations raise
|
||||
:class:`MsgTypeError`. See ``examples/typed_payloads.py``.
|
||||
|
||||
``Context``
|
||||
-----------
|
||||
|
||||
.. autoclass:: Context
|
||||
:members: started,
|
||||
wait_for_result,
|
||||
cancel,
|
||||
cid,
|
||||
chan,
|
||||
side,
|
||||
cancel_called,
|
||||
cancelled_caught,
|
||||
cancel_acked,
|
||||
canceller,
|
||||
maybe_error,
|
||||
outcome
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
``Context.result()`` warns; use :meth:`Context.wait_for_result`.
|
||||
|
||||
.. note::
|
||||
|
||||
A :class:`Context` is **not** a :class:`trio.CancelScope`:
|
||||
:meth:`Context.cancel` requests cancellation of the *remote*
|
||||
peer task and does not cancel the local scope. If *you*
|
||||
requested the cancel, the resulting :class:`ContextCancelled`
|
||||
is absorbed at ``open_context()`` exit; a cancel originating
|
||||
anywhere else (the peer, or a third-party actor recorded in
|
||||
:attr:`ContextCancelled.canceller`) *is* raised locally. This
|
||||
self-vs-cross-cancel rule is the key to writing correct
|
||||
inter-actor teardown logic — see :doc:`/guide/context`.
|
||||
|
||||
Bidirectional streaming
|
||||
-----------------------
|
||||
|
||||
.. autofunction:: tractor._streaming.open_stream_from_ctx
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`~tractor._streaming.open_stream_from_ctx` is bound as
|
||||
the **method-alias** ``Context.open_stream()`` — call it as
|
||||
``async with ctx.open_stream() as stream:``. Both sides of the
|
||||
context must enter it for the dialog to be open.
|
||||
|
||||
.. autoclass:: MsgStream
|
||||
:members: send,
|
||||
receive,
|
||||
receive_nowait,
|
||||
aclose,
|
||||
subscribe,
|
||||
ctx,
|
||||
closed
|
||||
|
||||
.. note::
|
||||
|
||||
A :class:`MsgStream` is one-shot use: once closed it can never
|
||||
be "re-opened" — open a fresh :class:`Context` instead. Remote
|
||||
end-of-stream surfaces as :class:`StopAsyncIteration` from
|
||||
``async for``; un-consumed sends overrun the receiver and raise
|
||||
:class:`tractor._exceptions.StreamOverrun` unless the context
|
||||
was opened with ``allow_overruns=True``.
|
||||
|
||||
:meth:`MsgStream.subscribe` fans a single IPC stream out to
|
||||
multiple *local* tasks via a
|
||||
:class:`tractor.trionics.BroadcastReceiver` (see
|
||||
:doc:`/api/trionics`); the underlying allocation is idempotent and
|
||||
non-reversible for the stream's lifetime. See
|
||||
``examples/streaming_broadcast_fanout.py`` for the pattern in
|
||||
action.
|
||||
|
||||
Legacy one-way streaming
|
||||
------------------------
|
||||
|
||||
.. autofunction:: stream
|
||||
|
||||
.. warning::
|
||||
|
||||
``@tractor.stream`` and ``Portal.open_stream_from()`` are the
|
||||
*legacy* one-way streaming API kept for backward compat: a
|
||||
plain async-generator function streamed parent-ward with no
|
||||
child-side receive leg. New code should use
|
||||
``@tractor.context`` + ``ctx.open_stream()`` (bidirectional,
|
||||
SC-linked, typed). Note ``ctx`` is now a reserved param name
|
||||
for ``@context`` endpoints — ``@stream`` functions must use
|
||||
``stream`` instead, and ``ctx.send_yield()`` is deprecated in
|
||||
favor of :meth:`MsgStream.send`.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/api/errors` for :class:`ContextCancelled` /
|
||||
:class:`MsgTypeError` semantics, :doc:`/api/msg` for payload
|
||||
typing via ``pld_spec`` and codecs, :doc:`/api/trionics` for
|
||||
the broadcast fan-out machinery, and the guided tours in
|
||||
:doc:`/guide/streaming` + :doc:`/guide/cancellation`.
|
||||
|
||||
.. _exactly like trio: https://trio.readthedocs.io/en/latest/reference-core.html#cancellation-semantics
|
||||
.. _nursery: https://trio.readthedocs.io/en/latest/reference-core.html#nurseries-and-spawning
|
||||
|
|
@ -1,157 +0,0 @@
|
|||
Runtime and spawning
|
||||
====================
|
||||
|
||||
The core lifecycle API: boot the runtime in your root process,
|
||||
spawn trio-"actors" (processes running ``trio.run()`` task trees)
|
||||
under a one-cancels-all supervisor, and talk to them through
|
||||
portals. This is structured concurrency (SC) applied *transitively*:
|
||||
every spawned process is owned by a nursery block and errors
|
||||
`always propagate`_. If you can create zombies it **is a bug**.
|
||||
|
||||
.. currentmodule:: tractor
|
||||
|
||||
Booting the runtime
|
||||
-------------------
|
||||
|
||||
.. autofunction:: open_root_actor
|
||||
|
||||
.. note::
|
||||
|
||||
The env vars ``TRACTOR_LOGLEVEL`` and ``TRACTOR_SPAWN_METHOD``
|
||||
override the ``loglevel`` / ``start_method`` params so you can
|
||||
crank verbosity or swap spawn backends without touching app
|
||||
code. Exactly **one** IPC transport may be enabled per actor
|
||||
(see ``enable_transports`` and :doc:`/api/ipc`).
|
||||
|
||||
.. autofunction:: run_daemon
|
||||
|
||||
Spawning actors
|
||||
---------------
|
||||
|
||||
.. d2:: diagrams/actor_tree.d2
|
||||
:caption: A supervised actor (process) tree.
|
||||
:margin:
|
||||
:alt: root actor supervising a tree of subactors
|
||||
|
||||
.. autofunction:: open_nursery
|
||||
|
||||
.. autoclass:: ActorNursery
|
||||
:members: start_actor,
|
||||
run_in_actor,
|
||||
cancel,
|
||||
cancel_called,
|
||||
cancelled_caught
|
||||
|
||||
.. note::
|
||||
|
||||
:meth:`ActorNursery.start_actor` (daemon actor + portal) is the
|
||||
blessed spawning primitive; pair it with
|
||||
``Portal.open_context()`` for SC-linked remote tasks.
|
||||
:meth:`ActorNursery.run_in_actor` is a *convenience* one-shot —
|
||||
spawn, run a single task, auto-cancel after the result — slated
|
||||
to be rebuilt as a high-level wrapper, so don't design around
|
||||
it as the core model.
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
``ActorNursery.cancelled`` warns; use
|
||||
:attr:`ActorNursery.cancel_called` and
|
||||
:attr:`ActorNursery.cancelled_caught`. The ``rpc_module_paths``
|
||||
kwarg is likewise deprecated in favor of ``enable_modules``.
|
||||
|
||||
Portals
|
||||
-------
|
||||
|
||||
A :class:`Portal` "opens a portal" into a peer actor's memory
|
||||
domain: you call functions and start SC-linked tasks *over IPC* as
|
||||
though they were local, with results, errors and cancellation
|
||||
flowing back `exactly like trio`_.
|
||||
|
||||
.. autoclass:: Portal
|
||||
:members: run,
|
||||
run_from_ns,
|
||||
open_stream_from,
|
||||
wait_for_result,
|
||||
cancel_actor,
|
||||
chan
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
``Portal.result()`` warns; use :meth:`Portal.wait_for_result`.
|
||||
The str-form ``Portal.run('mod.path', 'fn_name')`` also warns;
|
||||
pass a function *object* whose module is listed in the target's
|
||||
``enable_modules``. ``Portal.channel`` is the legacy spelling
|
||||
of :attr:`Portal.chan`.
|
||||
|
||||
.. autofunction:: tractor._context.open_context_from_portal
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`~tractor._context.open_context_from_portal` is bound as
|
||||
the **method-alias** ``Portal.open_context()`` — that's the
|
||||
spelling you should actually call:
|
||||
``portal.open_context(fn, **kwargs)``. See :doc:`/api/context`
|
||||
for the full ``Context`` + ``MsgStream`` API it unlocks.
|
||||
|
||||
.. note::
|
||||
|
||||
:meth:`Portal.cancel_actor` cancels the *whole* remote runtime
|
||||
and process (machine-level), not a single task — use
|
||||
:meth:`Context.cancel` for task-level cancellation. Pass
|
||||
``raise_on_timeout=True`` to get an ``ActorTooSlowError`` you
|
||||
can escalate per SC discipline (see :doc:`/api/errors`).
|
||||
|
||||
Clusters
|
||||
--------
|
||||
|
||||
.. autofunction:: open_actor_cluster
|
||||
|
||||
Spawn a *flat* cluster of ``count`` worker actors (default: one
|
||||
per core) all serving the RPC ``modules`` list, yielding a
|
||||
``dict[str, Portal]`` keyed by actor name. Handy for
|
||||
embarrassingly parallel fan-out; see ``examples/quick_cluster.py``.
|
||||
|
||||
Runtime introspection
|
||||
---------------------
|
||||
|
||||
.. autofunction:: current_actor
|
||||
|
||||
.. autoclass:: Actor
|
||||
:members: aid,
|
||||
name,
|
||||
uid,
|
||||
is_registrar,
|
||||
is_infected_aio,
|
||||
cancel_soon
|
||||
|
||||
.. note::
|
||||
|
||||
:class:`Actor` is the per-process runtime singleton (msg loop,
|
||||
RPC scheduling, IPC server) — you never instantiate it yourself
|
||||
and should normally only touch the identity/introspection
|
||||
surface listed above. The canonical identity type is
|
||||
:attr:`Actor.aid` (a ``tractor.msg.Aid`` struct);
|
||||
:attr:`Actor.uid` is the legacy ``(name, uuid)`` 2-tuple which
|
||||
is still pervasive in logs and error metadata.
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
``Actor.is_arbiter`` warns; use :attr:`Actor.is_registrar`.
|
||||
The ``arbiter_addr`` constructor kwarg is deprecated for
|
||||
``registry_addrs``.
|
||||
|
||||
.. autofunction:: current_ipc_ctx
|
||||
|
||||
.. autofunction:: is_root_process
|
||||
|
||||
.. autofunction:: get_runtime_vars
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/api/context` for the SC-linked remote task API,
|
||||
:doc:`/api/discovery` for finding actors by name, and the
|
||||
guided tours in :doc:`/guide/spawning`, :doc:`/guide/rpc` and
|
||||
:doc:`/guide/context`.
|
||||
|
||||
.. _always propagate: https://trio.readthedocs.io/en/latest/design.html#exceptions-always-propagate
|
||||
.. _exactly like trio: https://trio.readthedocs.io/en/latest/reference-core.html#cancellation-semantics
|
||||
|
|
@ -1,92 +0,0 @@
|
|||
Debugging and devx: ``tractor.devx``
|
||||
====================================
|
||||
|
||||
Multi-process debugging that actually works: boot the tree with
|
||||
``open_root_actor(debug_mode=True)`` (or pass it to
|
||||
``open_nursery()``) and any crash or explicit pause in *any* actor
|
||||
acquires a tree-global TTY lock and drops you into a
|
||||
`pdbp`_-powered REPL — one actor at a time, ``SIGINT`` shielded,
|
||||
no garbled terminals. The top-level helpers below are the daily
|
||||
drivers; the rest of the toolbox lives under ``tractor.devx``.
|
||||
|
||||
.. currentmodule:: tractor
|
||||
|
||||
Pausing and post-mortems
|
||||
------------------------
|
||||
|
||||
.. autofunction:: pause
|
||||
|
||||
.. autofunction:: pause_from_sync
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`pause_from_sync` needs the `greenback`_ portal: boot
|
||||
with ``open_root_actor(maybe_enable_greenback=True)`` (mind
|
||||
the `performance implications`_). With ``debug_mode`` on, the
|
||||
built-in ``breakpoint()`` is also remapped to a
|
||||
``tractor``-safe equivalent.
|
||||
|
||||
.. autofunction:: post_mortem
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
``tractor.breakpoint()`` warns and simply calls :func:`pause`
|
||||
— use :func:`tractor.pause` (async) or
|
||||
:func:`tractor.pause_from_sync` in new code.
|
||||
|
||||
Crash handling for CLIs and sync entrypoints
|
||||
--------------------------------------------
|
||||
|
||||
.. currentmodule:: tractor.devx
|
||||
|
||||
.. autofunction:: open_crash_handler
|
||||
|
||||
.. autofunction:: maybe_open_crash_handler
|
||||
|
||||
.. note::
|
||||
|
||||
Both are *sync* context managers usable before (or without)
|
||||
``trio.run()`` — wrap your CLI ``main()`` to get a post-mortem
|
||||
REPL on any uncaught exception instead of a bare traceback.
|
||||
|
||||
Runtime hang-hunting
|
||||
--------------------
|
||||
|
||||
.. autofunction:: enable_stack_on_sig
|
||||
|
||||
With stackscope_ integration enabled (also via
|
||||
``open_root_actor(enable_stack_on_sig=True)`` or the
|
||||
``TRACTOR_ENABLE_STACKSCOPE`` env var) a ``SIGUSR1`` triggers a
|
||||
full trio task-tree dump from every actor — works on live,
|
||||
*non*-debug-mode trees too:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
pkill --signal SIGUSR1 -f <part-of-your-cmd>
|
||||
|
||||
Dumps also tee to ``/tmp/tractor-stackscope-<pid>.log`` so you
|
||||
still get output under captured/CI stdio.
|
||||
|
||||
Lower-level debug plumbing
|
||||
--------------------------
|
||||
|
||||
.. autofunction:: mk_pdb
|
||||
|
||||
.. autofunction:: maybe_wait_for_debugger
|
||||
|
||||
:func:`maybe_wait_for_debugger` is mainly useful in runtime/test
|
||||
code that must avoid tearing down a tree while a child still
|
||||
holds the global debug lock.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/api/errors` for the boxed error types you'll inspect
|
||||
from the REPL, :doc:`/api/core` for the ``debug_mode`` /
|
||||
``maybe_enable_greenback`` / ``enable_stack_on_sig`` boot
|
||||
flags on :func:`tractor.open_root_actor`, and
|
||||
:doc:`/guide/debugging` for the guided multi-actor REPL tour.
|
||||
|
||||
.. _pdbp: https://github.com/mdmintz/pdbp
|
||||
.. _greenback: https://greenback.readthedocs.io/en/latest/
|
||||
.. _performance implications: https://greenback.readthedocs.io/en/latest/principle.html#performance
|
||||
.. _stackscope: https://github.com/oremanj/stackscope
|
||||
|
|
@ -1,67 +0,0 @@
|
|||
Discovery and the registrar
|
||||
===========================
|
||||
|
||||
Every actor registers its ``(name, uuid)`` and transport addresses
|
||||
with a *registrar* actor — by default the root of the tree, or
|
||||
whichever actor serves at the ``registry_addrs`` you boot with.
|
||||
The discovery API lets any actor look up any other **by name** and
|
||||
get back a connected :class:`~tractor.Portal`, giving you
|
||||
service-discovery patterns (daemons, service trees, multi-host
|
||||
meshes) without hard-coding addresses.
|
||||
|
||||
Lookups first scan already-connected peers before RPC-ing the
|
||||
registrar, and multihomed results are ranked UDS > local TCP >
|
||||
remote TCP. See ``examples/service_daemon_discovery.py`` for the
|
||||
canonical daemon + lookup pattern.
|
||||
|
||||
.. currentmodule:: tractor
|
||||
|
||||
Lookup APIs
|
||||
-----------
|
||||
|
||||
.. autofunction:: find_actor
|
||||
|
||||
.. autofunction:: wait_for_actor
|
||||
|
||||
.. autofunction:: query_actor
|
||||
|
||||
.. autofunction:: get_registry
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`find_actor` yields ``None`` when nothing is registered
|
||||
under the name (or raises with ``raise_on_none=True``);
|
||||
:func:`wait_for_actor` blocks until the name appears;
|
||||
:func:`query_actor` only *looks up* the address without
|
||||
connecting to the target.
|
||||
|
||||
The ``Registrar``
|
||||
-----------------
|
||||
|
||||
.. autoclass:: Registrar
|
||||
:show-inheritance:
|
||||
|
||||
A :class:`Registrar` is just an :class:`~tractor.Actor` subtype
|
||||
maintaining the name -> addresses table; you rarely touch it
|
||||
directly beyond passing ``registry_addrs`` /
|
||||
``ensure_registry=True`` to :func:`~tractor.open_root_actor`.
|
||||
Check :attr:`Actor.is_registrar <tractor.Actor.is_registrar>` to
|
||||
ask "am I it?".
|
||||
|
||||
Legacy ``Arbiter`` alias
|
||||
------------------------
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
``tractor.Arbiter`` survives only as a class alias of
|
||||
:class:`Registrar` and all "arbiter" terminology is replaced
|
||||
by "registrar"/"registry" across the API: ``get_arbiter()`` is
|
||||
removed (use :func:`get_registry`) and the ``arbiter_addr``
|
||||
kwarg is replaced by ``registry_addrs``.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/api/core` for booting a registrar via
|
||||
``open_root_actor(registry_addrs=...)``, :doc:`/api/ipc` for
|
||||
the transport/address model the registry stores, and
|
||||
:doc:`/guide/discovery` for the worked walkthrough.
|
||||
|
|
@ -1,102 +0,0 @@
|
|||
Errors and cancellation types
|
||||
=============================
|
||||
|
||||
``tractor`` extends trio's "exceptions `always propagate`_" rule
|
||||
across the process boundary: a crash in any actor is serialized as
|
||||
an ``Error`` msg, shuttled over IPC, and re-raised in the linked
|
||||
parent scope as a *boxed* :class:`RemoteActorError` — preserving
|
||||
the original type, traceback text and source-actor identity, even
|
||||
across multi-hop relays (a.k.a. "inceptions").
|
||||
|
||||
The most-used types below are importable from ``tractor``
|
||||
directly; the remainder live in ``tractor._exceptions`` (not yet
|
||||
re-exported at top level).
|
||||
|
||||
.. currentmodule:: tractor
|
||||
|
||||
Boxed remote errors
|
||||
-------------------
|
||||
|
||||
.. autoexception:: RemoteActorError
|
||||
:members: boxed_type,
|
||||
src_uid,
|
||||
relay_uid,
|
||||
pformat
|
||||
|
||||
.. code:: python
|
||||
|
||||
try:
|
||||
async with portal.open_context(ep_fn) as (ctx, first):
|
||||
...
|
||||
except tractor.RemoteActorError as rae:
|
||||
if rae.boxed_type is ValueError:
|
||||
... # remote task raised a `ValueError`
|
||||
|
||||
.. autoexception:: ContextCancelled
|
||||
:show-inheritance:
|
||||
:members: canceller
|
||||
|
||||
.. note::
|
||||
|
||||
Inspect :attr:`ContextCancelled.canceller` (the requesting
|
||||
actor's uid) to distinguish a *self*-requested cancel (absorbed
|
||||
at ``open_context()`` exit) from a *cross*-actor cancel (raised
|
||||
locally) — the full rules live in :doc:`/api/context`.
|
||||
|
||||
Typed-messaging errors
|
||||
----------------------
|
||||
|
||||
.. autoexception:: MsgTypeError
|
||||
:show-inheritance:
|
||||
:members: bad_msg,
|
||||
expected_msg_type
|
||||
|
||||
An "IPC ``TypeError``": a message failed validation against the
|
||||
active msg-spec / ``pld_spec`` (see :doc:`/api/msg`). Raised
|
||||
sender-side for control msgs (``Started``/``Return``) and
|
||||
receiver-side for stream ``Yield`` payloads.
|
||||
|
||||
.. autoexception:: tractor._exceptions.StreamOverrun
|
||||
:show-inheritance:
|
||||
|
||||
The sender out-paced the receiver's buffer on a
|
||||
:class:`~tractor.MsgStream` opened without
|
||||
``allow_overruns=True``; subtypes :class:`trio.TooSlowError`.
|
||||
|
||||
Transport and runtime errors
|
||||
----------------------------
|
||||
|
||||
.. autoexception:: TransportClosed
|
||||
|
||||
.. autoexception:: ModuleNotExposed
|
||||
:show-inheritance:
|
||||
|
||||
Raised when an RPC requests a function from a module not listed
|
||||
in the target actor's ``enable_modules`` allowlist —
|
||||
capability-style access control, not an import bug on your end ;)
|
||||
|
||||
.. autoexception:: tractor._exceptions.NoRuntime
|
||||
:show-inheritance:
|
||||
|
||||
Raised by :func:`tractor.current_actor` (and friends) when no
|
||||
actor runtime is up in the current process.
|
||||
|
||||
.. autoexception:: tractor._exceptions.ActorTooSlowError
|
||||
:show-inheritance:
|
||||
|
||||
A peer actor failed to ack a cancel request within the bounded
|
||||
wait — the SC-sanctioned escalation signal from APIs like
|
||||
``Portal.cancel_actor(raise_on_timeout=True)``. Catch it to
|
||||
escalate (e.g. hard-kill via the supervising
|
||||
:class:`~tractor.ActorNursery`); never just ignore it, that's how
|
||||
zombies happen.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/api/context` for how cancellation and errors flow
|
||||
through a :class:`~tractor.Context`, :doc:`/api/devx` for
|
||||
crash-handling REPL tooling (``debug_mode``, post-mortems),
|
||||
and :doc:`/guide/cancellation` for the full SC-cancellation
|
||||
story.
|
||||
|
||||
.. _always propagate: https://trio.readthedocs.io/en/latest/design.html#exceptions-always-propagate
|
||||
|
|
@ -1,67 +0,0 @@
|
|||
API reference
|
||||
=============
|
||||
|
||||
This is the curated reference for ``tractor``'s public surface: the
|
||||
names you can import and lean on without reading runtime internals.
|
||||
Everything below is re-exported at the top level (``import
|
||||
tractor``) unless a page says otherwise; subsystems like
|
||||
``tractor.msg``, ``tractor.trionics``, ``tractor.to_asyncio``,
|
||||
``tractor.devx`` and ``tractor.log`` are importable as submodules.
|
||||
|
||||
``tractor`` is "just trio_" extended across processes: every API
|
||||
here is designed to keep the structured concurrency (SC) rules you
|
||||
already know from the `trio docs`_ intact across the process
|
||||
boundary. If a name isn't documented here it's an internal — expect
|
||||
it to change without notice B).
|
||||
|
||||
.. currentmodule:: tractor
|
||||
|
||||
Most-used names at a glance:
|
||||
|
||||
.. autosummary::
|
||||
:nosignatures:
|
||||
|
||||
open_root_actor
|
||||
open_nursery
|
||||
run_daemon
|
||||
ActorNursery
|
||||
Portal
|
||||
context
|
||||
Context
|
||||
MsgStream
|
||||
open_actor_cluster
|
||||
find_actor
|
||||
wait_for_actor
|
||||
get_registry
|
||||
current_actor
|
||||
current_ipc_ctx
|
||||
is_root_process
|
||||
get_runtime_vars
|
||||
RemoteActorError
|
||||
ContextCancelled
|
||||
MsgTypeError
|
||||
pause
|
||||
post_mortem
|
||||
Channel
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Reference pages
|
||||
|
||||
core
|
||||
context
|
||||
discovery
|
||||
errors
|
||||
msg
|
||||
trionics
|
||||
to_asyncio
|
||||
devx
|
||||
ipc
|
||||
|
||||
Where to next? If you're new, start with the runtime and spawning
|
||||
APIs in :doc:`/api/core`, then graduate to the inter-actor task
|
||||
linking model in :doc:`/api/context` — it's the heart of the whole
|
||||
system.
|
||||
|
||||
.. _trio: https://github.com/python-trio/trio
|
||||
.. _trio docs: https://trio.readthedocs.io/en/latest/
|
||||
|
|
@ -1,89 +0,0 @@
|
|||
IPC and logging
|
||||
===============
|
||||
|
||||
Under every portal, context and stream sits a per-peer
|
||||
:class:`~tractor.Channel`: a msgpack-typed messaging link wrapping
|
||||
one OS transport connection. Transports are pluggable per actor
|
||||
via ``enable_transports=['tcp' | 'uds']`` — TCP is the default,
|
||||
UDS (unix domain sockets) gives you port-less, same-host IPC with
|
||||
kernel-provided peer credentials for free — and exactly **one**
|
||||
transport may currently be enabled per actor.
|
||||
|
||||
.. d2:: diagrams/runtime_stack.d2
|
||||
:caption: Where ``Channel`` sits in the runtime stack.
|
||||
:margin:
|
||||
:alt: layered runtime stack from app code down to transports
|
||||
|
||||
Addresses are "unwrapped" tuples at the API edges:
|
||||
``('host', port)`` for TCP, filesystem-path pairs for UDS. For
|
||||
the full layering story — transport protocols, the IPC server,
|
||||
address types and the msg loop — see
|
||||
:doc:`/explain/architecture`.
|
||||
|
||||
.. currentmodule:: tractor
|
||||
|
||||
``Channel``
|
||||
-----------
|
||||
|
||||
.. autoclass:: Channel
|
||||
:members: from_addr,
|
||||
send,
|
||||
recv,
|
||||
aclose,
|
||||
connected,
|
||||
apply_codec,
|
||||
aid,
|
||||
laddr,
|
||||
raddr,
|
||||
closed
|
||||
|
||||
.. deprecated:: 0.1.0a6
|
||||
|
||||
``Channel.uid`` warns; use :attr:`Channel.aid` which carries
|
||||
richer (optional) identity fields beyond the legacy
|
||||
``(name, uuid)`` pair.
|
||||
|
||||
.. note::
|
||||
|
||||
You rarely construct a :class:`Channel` yourself — the runtime
|
||||
hands them out via :attr:`Portal.chan <tractor.Portal.chan>`
|
||||
and :attr:`Context.chan <tractor.Context.chan>`. Treat the
|
||||
send/recv surface as advanced API: normal apps should speak
|
||||
:class:`~tractor.MsgStream` instead.
|
||||
|
||||
Choosing a transport
|
||||
--------------------
|
||||
|
||||
.. literalinclude:: ../../examples/uds_transport_actor_tree.py
|
||||
:caption: examples/uds_transport_actor_tree.py
|
||||
:language: python
|
||||
|
||||
Logging
|
||||
-------
|
||||
|
||||
``tractor.log`` provides the structured, colorized console
|
||||
logging used across the runtime — with actor-name + task-aware
|
||||
record headers and extra log levels below :data:`logging.DEBUG`
|
||||
(``'transport'``, ``'runtime'``, ``'cancel'``, ``'devx'``) for
|
||||
spelunking the runtime itself. Use it for your app too: it's
|
||||
already distributed-system aware.
|
||||
|
||||
.. currentmodule:: tractor.log
|
||||
|
||||
.. autofunction:: get_logger
|
||||
|
||||
.. autofunction:: get_console_log
|
||||
|
||||
.. note::
|
||||
|
||||
The ``TRACTOR_LOGLEVEL`` env var overrides any caller-passed
|
||||
``loglevel`` (e.g. to ``open_root_actor()``) so you can crank
|
||||
console verbosity without touching code; subactors inherit
|
||||
the root's level by default.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/explain/architecture` for the transport/server
|
||||
internals, :doc:`/api/discovery` for how channel addresses
|
||||
get registered and found, and :doc:`/api/msg` for the codec
|
||||
layer every channel speaks.
|
||||
107
docs/api/msg.rst
107
docs/api/msg.rst
|
|
@ -1,107 +0,0 @@
|
|||
Typed messaging: ``tractor.msg``
|
||||
================================
|
||||
|
||||
All inter-actor communication rides a small, strictly-typed
|
||||
msgpack wire protocol built from :class:`msgspec.Struct` types —
|
||||
the "SC-shuttle" protocol that powers contexts, streams, RPC and
|
||||
cancellation. You normally never touch these msg types directly
|
||||
(the :class:`~tractor.Context` API speaks them for you) but you
|
||||
*do* use this subpackage to define **payload type contracts**: per
|
||||
endpoint via ``@tractor.context(pld_spec=...)`` or per channel via
|
||||
custom codecs.
|
||||
|
||||
Violations of an active msg-spec surface as
|
||||
:class:`~tractor.MsgTypeError` (see :doc:`/api/errors`); the full
|
||||
typed-payload workflow is shown in ``examples/typed_payloads.py``.
|
||||
|
||||
.. currentmodule:: tractor.msg
|
||||
|
||||
The protocol message set
|
||||
------------------------
|
||||
|
||||
.. autosummary::
|
||||
:nosignatures:
|
||||
|
||||
PayloadMsg
|
||||
Aid
|
||||
SpawnSpec
|
||||
Start
|
||||
StartAck
|
||||
Started
|
||||
Yield
|
||||
Stop
|
||||
Return
|
||||
CancelAck
|
||||
Error
|
||||
|
||||
``Aid`` (identity handshake) and ``SpawnSpec`` (parent -> child
|
||||
init) run at connection setup; ``Start``/``StartAck`` initiate an
|
||||
RPC task; ``Started``/``Yield``/``Stop``/``Return`` are the
|
||||
:class:`~tractor.Context` dialog phases; ``CancelAck`` and
|
||||
``Error`` close the loop on cancellation and (boxed) failure.
|
||||
``Msg`` is a legacy alias of ``PayloadMsg``. The union of all of
|
||||
the above is exported as ``MsgType`` (also ``__msg_spec__``).
|
||||
|
||||
.. automodule:: tractor.msg.types
|
||||
:no-members:
|
||||
|
||||
.. currentmodule:: tractor.msg
|
||||
|
||||
Codec construction and override
|
||||
-------------------------------
|
||||
|
||||
.. autofunction:: mk_codec
|
||||
|
||||
.. autoclass:: MsgCodec
|
||||
:members: encode,
|
||||
decode,
|
||||
msg_spec
|
||||
|
||||
.. autofunction:: mk_dec
|
||||
|
||||
.. autoclass:: MsgDec
|
||||
:members: decode,
|
||||
spec
|
||||
|
||||
.. autofunction:: apply_codec
|
||||
|
||||
.. autofunction:: current_codec
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`apply_codec` swaps the codec via a
|
||||
:class:`contextvars.ContextVar` — the override only applies to
|
||||
the *current task* (and tasks it starts), not sibling tasks
|
||||
already running in the actor. Payload-decoding is layered: the
|
||||
outer codec leaves ``.pld`` fields as ``msgspec.Raw`` and each
|
||||
context's payload-receiver decodes them against *its* spec
|
||||
(the "cheap-or-nasty" validation pattern).
|
||||
|
||||
Namespace pointers
|
||||
------------------
|
||||
|
||||
.. autoclass:: NamespacePath
|
||||
:members: from_ref,
|
||||
load_ref,
|
||||
to_tuple
|
||||
|
||||
The ``'module.path:obj_name'`` :class:`str`-subtype used to
|
||||
address every RPC target function over the wire (same format as
|
||||
:func:`pkgutil.resolve_name`).
|
||||
|
||||
Pretty structs
|
||||
--------------
|
||||
|
||||
.. autoclass:: Struct
|
||||
:show-inheritance:
|
||||
|
||||
A :class:`msgspec.Struct` subtype with a multi-line pretty
|
||||
``__repr__`` — handy as a base for your own IPC payload types so
|
||||
crash logs stay readable.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/api/context` for where ``pld_spec`` typing plugs into
|
||||
the ``@context`` decorator, :doc:`/api/errors` for
|
||||
:class:`~tractor.MsgTypeError` semantics, and
|
||||
:doc:`/guide/msging` for the guided typed-messaging tour.
|
||||
|
|
@ -1,97 +0,0 @@
|
|||
asyncio interop: ``tractor.to_asyncio``
|
||||
=======================================
|
||||
|
||||
"Infected asyncio" mode: spawn an actor with
|
||||
``start_actor(..., infect_asyncio=True)`` and its process runs
|
||||
:mod:`trio` as a `guest`_ on top of the :mod:`asyncio` loop —
|
||||
letting your trio task tree drive asyncio tasks *in the same
|
||||
process* while the rest of the actor tree stays pure trio. Each
|
||||
trio <-> asyncio task pair is linked with structured concurrency
|
||||
(SC) semantics: error or cancellation on either side tears down
|
||||
both, with the cause translated cross-loop.
|
||||
|
||||
.. d2:: diagrams/infected_aio.d2
|
||||
:caption: A trio guest driving asyncio tasks in one actor.
|
||||
:margin:
|
||||
:alt: trio guest mode inside an asyncio-infected actor
|
||||
|
||||
See ``examples/infected_asyncio_echo_server.py`` for a complete
|
||||
worked example.
|
||||
|
||||
.. currentmodule:: tractor.to_asyncio
|
||||
|
||||
Starting asyncio tasks from trio
|
||||
--------------------------------
|
||||
|
||||
.. autofunction:: open_channel_from
|
||||
|
||||
.. autofunction:: run_task
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`open_channel_from` mirrors the
|
||||
``Portal.open_context()`` handshake: the asyncio side calls
|
||||
``chan.started_nowait(value)`` and that value pops out as
|
||||
``first`` on the trio side. :func:`run_task` is the one-shot
|
||||
form — run a single asyncio-compatible coroutine fn and return
|
||||
its result to trio.
|
||||
|
||||
The inter-loop channel
|
||||
----------------------
|
||||
|
||||
.. autoclass:: LinkedTaskChannel
|
||||
:members: send,
|
||||
receive,
|
||||
wait_for_result,
|
||||
subscribe,
|
||||
started_nowait,
|
||||
send_nowait,
|
||||
get,
|
||||
cancel_asyncio_task,
|
||||
closed
|
||||
|
||||
.. note::
|
||||
|
||||
The trio side uses the async API
|
||||
(:meth:`LinkedTaskChannel.send` /
|
||||
:meth:`LinkedTaskChannel.receive`); the asyncio side uses the
|
||||
loop-safe sync/await mix
|
||||
(:meth:`LinkedTaskChannel.send_nowait` /
|
||||
:meth:`LinkedTaskChannel.get` /
|
||||
:meth:`LinkedTaskChannel.started_nowait`).
|
||||
|
||||
Translated exception types
|
||||
--------------------------
|
||||
|
||||
Cross-loop failures are re-raised on the *other* side as one of
|
||||
these explicit translation types, so you always know which loop
|
||||
actually died first:
|
||||
|
||||
.. autoexception:: tractor._exceptions.AsyncioCancelled
|
||||
|
||||
.. autoexception:: tractor._exceptions.AsyncioTaskExited
|
||||
|
||||
.. autoexception:: tractor._exceptions.TrioCancelled
|
||||
|
||||
.. autoexception:: tractor._exceptions.TrioTaskExited
|
||||
|
||||
.. autoexception:: AsyncioRuntimeTranslationError
|
||||
:show-inheritance:
|
||||
|
||||
Guest-mode entrypoint
|
||||
---------------------
|
||||
|
||||
``run_as_asyncio_guest()`` is the runtime-internal entrypoint that
|
||||
boots trio in `guest`_ mode inside an infected actor — you get it
|
||||
implicitly via ``infect_asyncio=True`` and shouldn't need to call
|
||||
it yourself.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/api/core` for the ``infect_asyncio`` spawn flag,
|
||||
:meth:`tractor.Actor.is_infected_aio` for runtime
|
||||
introspection, :doc:`/api/devx` for using the debugger from
|
||||
inside asyncio tasks, and :doc:`/guide/asyncio` for the
|
||||
guided tour.
|
||||
|
||||
.. _guest: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
|
||||
|
|
@ -1,76 +0,0 @@
|
|||
Trio patterns: ``tractor.trionics``
|
||||
===================================
|
||||
|
||||
Sugary structured concurrency (SC) patterns for plain :mod:`trio`
|
||||
code — **no actor runtime required**. These helpers grew out of
|
||||
real distributed-system needs in ``tractor`` apps but every one of
|
||||
them works in a single-process program too; import via
|
||||
``from tractor import trionics``.
|
||||
|
||||
.. currentmodule:: tractor.trionics
|
||||
|
||||
Context-manager helpers
|
||||
-----------------------
|
||||
|
||||
.. autofunction:: gather_contexts
|
||||
|
||||
.. autofunction:: maybe_open_context
|
||||
|
||||
.. autofunction:: maybe_open_nursery
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`gather_contexts` is "a nursery for async context
|
||||
managers": it enters N acms concurrently and yields their
|
||||
values in input order. :func:`maybe_open_context` is the
|
||||
actor-wide cache/multiplex layer on top — the first task pays
|
||||
the acm setup cost, later callers get ``(cache_hit=True, ...)``
|
||||
and share the same value until all users exit.
|
||||
|
||||
Broadcast fan-out
|
||||
-----------------
|
||||
|
||||
.. autofunction:: broadcast_receiver
|
||||
|
||||
.. autoclass:: BroadcastReceiver
|
||||
:members: receive,
|
||||
subscribe,
|
||||
aclose
|
||||
|
||||
.. autoexception:: Lagged
|
||||
:show-inheritance:
|
||||
|
||||
A single-producer, many-consumer broadcast layer over any
|
||||
``trio``-style receive channel: non-lossy for the *fastest*
|
||||
consumer while slower consumers raise :class:`Lagged` (a
|
||||
:class:`trio.TooSlowError` subtype) once they fall behind the
|
||||
internal ring. This is exactly the machinery behind
|
||||
:meth:`tractor.MsgStream.subscribe` — see
|
||||
``examples/streaming_broadcast_fanout.py``.
|
||||
|
||||
ExceptionGroup helpers
|
||||
----------------------
|
||||
|
||||
.. autofunction:: collapse_eg
|
||||
|
||||
.. autofunction:: maybe_raise_from_masking_exc
|
||||
|
||||
.. note::
|
||||
|
||||
:func:`collapse_eg` "un-nests" single-exception
|
||||
:class:`ExceptionGroup` wrappers from strict-eg ``trio``
|
||||
nurseries so your ``except`` clauses match the original error;
|
||||
:func:`maybe_raise_from_masking_exc` surfaces real errors that
|
||||
would otherwise be masked by :class:`trio.Cancelled` during
|
||||
teardown.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/api/context` for the IPC-stream consumer of
|
||||
:class:`BroadcastReceiver`, :doc:`/guide/streaming` for
|
||||
fan-out in a worked pipeline, and the `trio docs`_ for the
|
||||
underlying channel and `nursery`_ semantics these helpers
|
||||
compose.
|
||||
|
||||
.. _trio docs: https://trio.readthedocs.io/en/latest/
|
||||
.. _nursery: https://trio.readthedocs.io/en/latest/reference-core.html#nurseries-and-spawning
|
||||
211
docs/conf.py
211
docs/conf.py
|
|
@ -1,160 +1,105 @@
|
|||
# tractor: distributed structured concurrency.
|
||||
'''
|
||||
Sphinx config for `tractor`'s documentation.
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file only contains a selection of the most common options. For a full
|
||||
# list see the documentation:
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html
|
||||
|
||||
Theme-wise we ride the `pydata_sphinx_theme` (per the
|
||||
research + history in #157) skinned to a minimal
|
||||
black + white look via `_static/css/custom.css`; see
|
||||
the local extensions under `_ext/` for our `.. d2::`
|
||||
diagram and `.. margin::` aside directives.
|
||||
# -- Path setup --------------------------------------------------------------
|
||||
|
||||
Build locally via,
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
# import os
|
||||
# import sys
|
||||
# sys.path.insert(0, os.path.abspath('.'))
|
||||
|
||||
uv run --group docs make -C docs html
|
||||
# Warn about all references to unknown targets
|
||||
nitpicky = True
|
||||
|
||||
'''
|
||||
from importlib.metadata import version as get_version
|
||||
from pathlib import Path
|
||||
import sys
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
# local sphinx extensions live in `_ext/`:
|
||||
# - `d2diagrams`: `.. d2::` diagram rendering
|
||||
# - `marginalia`: `.. margin::` RHS prose-asides
|
||||
sys.path.insert(
|
||||
0,
|
||||
str((Path(__file__).parent / '_ext').resolve()),
|
||||
)
|
||||
|
||||
# -- project info ---------------------------------
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = 'tractor'
|
||||
copyright = '2018-2026, Tyler Goodlet'
|
||||
copyright = '2018, Tyler Goodlet'
|
||||
author = 'Tyler Goodlet'
|
||||
release: str = get_version('tractor')
|
||||
version: str = release
|
||||
|
||||
# -- general config -------------------------------
|
||||
# The full version, including alpha/beta/rc tags
|
||||
release = '0.0.0a0.dev0'
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
'sphinx.ext.autodoc',
|
||||
'sphinx.ext.autosummary',
|
||||
'sphinx.ext.intersphinx',
|
||||
'sphinx.ext.viewcode',
|
||||
'sphinx.ext.todo',
|
||||
# emit a `.nojekyll` so GitHub Pages serves the `_static/`
|
||||
# + `_images/` dirs (Jekyll would otherwise drop `_`-prefixed
|
||||
# paths and break all styling/assets).
|
||||
'sphinx.ext.githubpages',
|
||||
'sphinx_design',
|
||||
'sphinx_copybutton',
|
||||
'sphinxext.opengraph',
|
||||
'sphinx_togglebutton',
|
||||
'd2diagrams',
|
||||
'marginalia',
|
||||
]
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['_templates']
|
||||
exclude_patterns = [
|
||||
'_build',
|
||||
# the pypi/gh readme + its standalone generator
|
||||
# sub-project; NOT doc-tree pages.
|
||||
'README.rst',
|
||||
'github_readme',
|
||||
'Thumbs.db',
|
||||
'.DS_Store',
|
||||
]
|
||||
root_doc = 'index'
|
||||
# TODO: flip this on + burn down the (many) warnings
|
||||
# from our informal docstring style; see the autodoc
|
||||
# readiness notes from the revamp's recon pass.
|
||||
nitpicky = False
|
||||
|
||||
# -- autodoc/autosummary --------------------------
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path.
|
||||
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
|
||||
|
||||
autodoc_member_order = 'bysource'
|
||||
autodoc_typehints = 'description'
|
||||
autosummary_generate = True
|
||||
|
||||
# -- intersphinx ----------------------------------
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
intersphinx_mapping = {
|
||||
'python': (
|
||||
'https://docs.python.org/3',
|
||||
None,
|
||||
),
|
||||
'trio': (
|
||||
'https://trio.readthedocs.io/en/stable',
|
||||
None,
|
||||
),
|
||||
# NOTE, msgspec's site doesn't publish an
|
||||
# `objects.inv` (404s) so no intersphinx for it.
|
||||
'pytest': (
|
||||
'https://docs.pytest.org/en/stable',
|
||||
None,
|
||||
),
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
#
|
||||
html_theme = 'sphinx_book_theme'
|
||||
|
||||
pygments_style = 'algol_nu'
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
html_theme_options = {
|
||||
# 'logo': 'tractor_logo_side.svg',
|
||||
# 'description': 'Structured concurrent "actors"',
|
||||
"repository_url": "https://github.com/goodboy/tractor",
|
||||
"use_repository_button": True,
|
||||
"home_page_in_toc": False,
|
||||
"show_toc_level": 1,
|
||||
"path_to_docs": "docs",
|
||||
|
||||
}
|
||||
html_sidebars = {
|
||||
"**": [
|
||||
"sbt-sidebar-nav.html",
|
||||
# "sidebar-search-bs.html",
|
||||
# 'localtoc.html',
|
||||
],
|
||||
# 'logo.html',
|
||||
# 'github.html',
|
||||
# 'relations.html',
|
||||
# 'searchbox.html'
|
||||
# ]
|
||||
}
|
||||
|
||||
# -- html output ----------------------------------
|
||||
# doesn't seem to work?
|
||||
# extra_navbar = "<p>nextttt-gennnnn</p>"
|
||||
|
||||
html_theme = 'pydata_sphinx_theme'
|
||||
html_title = 'tractor'
|
||||
# canonical site root (GitHub Pages); drives <link rel=canonical>,
|
||||
# og:url + any future sitemap. Update if a custom domain is added.
|
||||
html_baseurl = 'https://goodboy.github.io/tractor/'
|
||||
html_title = ''
|
||||
html_logo = '_static/tractor_logo_side.svg'
|
||||
html_favicon = '_static/tractor_logo_side.svg'
|
||||
# show_navbar_depth = 1
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ['_static']
|
||||
html_css_files = ['css/custom.css']
|
||||
html_show_sourcelink = False
|
||||
html_theme_options = {
|
||||
# theme-adaptive navbar logo: faces transparent, linework
|
||||
# near-black on light / near-white on dark (pydata swaps by
|
||||
# the active theme). Matches the landing hero's wireframe.
|
||||
'logo': {
|
||||
'image_light': '_static/tractor_logo_nav_light.svg',
|
||||
'image_dark': '_static/tractor_logo_nav_dark.svg',
|
||||
'alt_text': 'tractor',
|
||||
# text shown to the right of the navbar logo (à la
|
||||
# polars).
|
||||
'text': 'tractor',
|
||||
},
|
||||
'github_url': 'https://github.com/goodboy/tractor',
|
||||
'navbar_align': 'content',
|
||||
'show_toc_level': 2,
|
||||
'secondary_sidebar_items': {
|
||||
'**': ['page-toc'],
|
||||
'index': [],
|
||||
},
|
||||
'use_edit_page_button': True,
|
||||
'footer_start': ['copyright'],
|
||||
'footer_end': ['theme-version'],
|
||||
'pygments_light_style': 'algol_nu',
|
||||
'pygments_dark_style': 'github-dark',
|
||||
|
||||
# Example configuration for intersphinx: refer to the Python standard library.
|
||||
intersphinx_mapping = {
|
||||
"python": ("https://docs.python.org/3", None),
|
||||
"pytest": ("https://docs.pytest.org/en/latest", None),
|
||||
"setuptools": ("https://setuptools.readthedocs.io/en/latest", None),
|
||||
}
|
||||
html_context = {
|
||||
'github_user': 'goodboy',
|
||||
'github_repo': 'tractor',
|
||||
'github_version': 'main',
|
||||
'doc_path': 'docs',
|
||||
}
|
||||
|
||||
# -- ext: opengraph -------------------------------
|
||||
|
||||
ogp_site_url = 'https://goodboy.github.io/tractor/'
|
||||
ogp_use_first_image = True
|
||||
|
||||
# -- ext: copybutton ------------------------------
|
||||
|
||||
copybutton_prompt_text = r'>>> |\.\.\. |\$ '
|
||||
copybutton_prompt_is_regexp = True
|
||||
|
||||
# -- ext: todo ------------------------------------
|
||||
|
||||
todo_include_todos = True
|
||||
|
||||
# -- ext: d2diagrams (local) ----------------------
|
||||
|
||||
# normally resolved from PATH or the `D2_BIN` env
|
||||
# var; when neither hits, the committed SVGs under
|
||||
# `_diagrams/` are used as-is.
|
||||
d2_bin = None
|
||||
d2_args = []
|
||||
|
|
|
|||
|
|
@ -0,0 +1,51 @@
|
|||
Hot tips for ``tractor`` hackers
|
||||
================================
|
||||
|
||||
This is a WIP guide for newcomers to the project mostly to do with
|
||||
dev, testing, CI and release gotchas, reminders and best practises.
|
||||
|
||||
``tractor`` is a fairly novel project compared to most since it is
|
||||
effectively a new way of doing distributed computing in Python and is
|
||||
much closer to working with an "application level runtime" (like erlang
|
||||
OTP or scala's akka project) then it is a traditional Python library.
|
||||
As such, having an arsenal of tools and recipes for figuring out the
|
||||
right way to debug problems when they do arise is somewhat of
|
||||
a necessity.
|
||||
|
||||
|
||||
Making a Release
|
||||
----------------
|
||||
We currently do nothing special here except the traditional
|
||||
PyPa release recipe as in `documented by twine`_. I personally
|
||||
create sub-dirs within the generated `dist/` with an explicit
|
||||
release name such as `alpha3/` when there's been a sequence of
|
||||
releases I've made, but it really is up to you how you like to
|
||||
organize generated sdists locally.
|
||||
|
||||
The resulting build cmds are approximately:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
python setup.py sdist -d ./dist/XXX.X/
|
||||
|
||||
twine upload -r testpypi dist/XXX.X/*
|
||||
|
||||
twine upload dist/XXX.X/*
|
||||
|
||||
|
||||
|
||||
.. _documented by twine: https://twine.readthedocs.io/en/latest/#using-twine
|
||||
|
||||
|
||||
Debugging and monitoring actor trees
|
||||
------------------------------------
|
||||
TODO: but there are tips in the readme for some terminal commands
|
||||
which can be used to see the process trees easily on Linux.
|
||||
|
||||
|
||||
Using the log system to trace `trio` task flow
|
||||
----------------------------------------------
|
||||
TODO: the logging system is meant to be oriented around
|
||||
stack "layers" of the runtime such that you can track
|
||||
"logical abstraction layers" in the code such as errors, cancellation,
|
||||
IPC and streaming, and the low level transport and wire protocols.
|
||||
|
|
@ -1,31 +0,0 @@
|
|||
# tractor docs diagram: the hero supervision tree.
|
||||
# A process tree of trio-task-trees; every arrow is
|
||||
# a parent which *must wait* on its children.
|
||||
vars: {
|
||||
d2-config: {
|
||||
sketch: true
|
||||
theme-id: 1
|
||||
pad: 16
|
||||
layout-engine: elk
|
||||
}
|
||||
}
|
||||
direction: down
|
||||
root: "root actor" {
|
||||
main: "main()"
|
||||
an: "ActorNursery"
|
||||
main -> an: opens
|
||||
}
|
||||
w0: "subactor\n'worker_0'" {
|
||||
tasks: "trio task tree"
|
||||
}
|
||||
w1: "subactor\n'worker_1'" {
|
||||
main: "serve()"
|
||||
an: "ActorNursery"
|
||||
main -> an: opens
|
||||
}
|
||||
gc: "subactor\n'deeper'" {
|
||||
tasks: "trio task tree"
|
||||
}
|
||||
root.an -> w0: "spawns +\nsupervises"
|
||||
root.an -> w1: "spawns +\nsupervises"
|
||||
w1.an -> gc: "spawns +\nsupervises"
|
||||
|
|
@ -1,33 +0,0 @@
|
|||
# tractor docs diagram: the inter-actor `Context`
|
||||
# dialog; tractor's SC-transitive msg protocol as
|
||||
# seen from both sides of `Portal.open_context()`.
|
||||
vars: {
|
||||
d2-config: {
|
||||
sketch: true
|
||||
theme-id: 1
|
||||
pad: 16
|
||||
layout-engine: elk
|
||||
}
|
||||
}
|
||||
shape: sequence_diagram
|
||||
parent: "parent task\n(Portal)"
|
||||
child: "child actor task\n(@tractor.context fn)"
|
||||
parent -> child: "Start: open_context(fn, **kwargs)"
|
||||
child -> parent: "StartAck"
|
||||
child -> parent: "Started: await ctx.started(value)"
|
||||
parent -> child: "Yield: stream.send()"
|
||||
child -> parent: "Yield: stream.send()"
|
||||
parent -> child: "Stop: stream closed"
|
||||
child -> parent: "Return: fn return value"
|
||||
# the non-graceful endings: errors/cancels always
|
||||
# propagate, both ways (`ContextCancelled` rides in
|
||||
# an `Error` msg, hence the `Error: ...` labels).
|
||||
"... or, instead of a graceful Return": {
|
||||
parent-initiated: {
|
||||
parent -> child: "ctx.cancel() | parent-side error"
|
||||
child -> parent: "Error: ContextCancelled"
|
||||
}
|
||||
child-initiated: {
|
||||
child -> parent: "Error: child raised | ContextCancelled"
|
||||
}
|
||||
}
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
# tractor docs diagram: multi-actor debugger REPL
|
||||
# serialization via the root actor's tty lock.
|
||||
vars: {
|
||||
d2-config: {
|
||||
sketch: true
|
||||
theme-id: 1
|
||||
pad: 16
|
||||
layout-engine: elk
|
||||
}
|
||||
}
|
||||
shape: sequence_diagram
|
||||
mary: "subactor 'mary'"
|
||||
root: "root actor\n(owns the tty)"
|
||||
bob: "subactor 'bob'"
|
||||
mary -> root: "tractor.pause(): acquire tty lock"
|
||||
root -> mary: "granted: pdb REPL is yours"
|
||||
bob -> root: "tractor.pause(): acquire tty lock"
|
||||
mary -> root: "continue: release lock"
|
||||
root -> bob: "granted: pdb REPL is yours"
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
# tractor docs diagram: one-cancels-all error
|
||||
# propagation up a (sub-)actor tree; no zombies,
|
||||
# no lost errors.
|
||||
vars: {
|
||||
d2-config: {
|
||||
sketch: true
|
||||
theme-id: 1
|
||||
pad: 16
|
||||
layout-engine: elk
|
||||
}
|
||||
}
|
||||
direction: down
|
||||
root: "root actor\n(ActorNursery)"
|
||||
gertie: "subactor 'gertie'\n(healthy, gets cancelled)"
|
||||
bobbie: "subactor 'bobbie'\nraises NameError"
|
||||
root -> gertie: supervises
|
||||
root -> bobbie: supervises
|
||||
bobbie -> root: "RemoteActorError\n(boxed NameError)" {
|
||||
style: {
|
||||
stroke-dash: 3
|
||||
}
|
||||
}
|
||||
root -> gertie: "cancel()" {
|
||||
style: {
|
||||
stroke-dash: 3
|
||||
}
|
||||
}
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
# tractor docs diagram: "infected asyncio" mode;
|
||||
# trio runs as a guest on the asyncio loop with
|
||||
# SC supervision linking tasks across both.
|
||||
vars: {
|
||||
d2-config: {
|
||||
sketch: true
|
||||
theme-id: 1
|
||||
pad: 16
|
||||
layout-engine: elk
|
||||
}
|
||||
}
|
||||
direction: down
|
||||
sub: "subactor process (infect_asyncio=True)" {
|
||||
aio: "asyncio event loop" {
|
||||
aiotask: "asyncio.Task\naio_echo_server()"
|
||||
trio: "trio (guest mode)" {
|
||||
triotask: "trio task\n@tractor.context fn"
|
||||
}
|
||||
trio.triotask <-> aiotask: "LinkedTaskChannel\n.send()/.receive()"
|
||||
}
|
||||
}
|
||||
parent: "parent actor (pure trio)"
|
||||
parent <-> sub.aio.trio.triotask: "IPC: Context + MsgStream"
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
# tractor docs diagram: the layered runtime view
|
||||
# inside any single actor process.
|
||||
vars: {
|
||||
d2-config: {
|
||||
sketch: true
|
||||
theme-id: 1
|
||||
pad: 16
|
||||
layout-engine: elk
|
||||
}
|
||||
}
|
||||
grid-rows: 4
|
||||
grid-gap: 0
|
||||
app: "your app: plain trio tasks + nurseries"
|
||||
tractor: "tractor runtime: actors, portals,\ncontexts + streams, RPC"
|
||||
ipc: "IPC Channel: msgspec-typed msgs\nover TCP | UDS transports"
|
||||
os: "OS: one process per actor"
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
# tractor docs diagram: multi-actor streaming
|
||||
# pipeline topology from
|
||||
# examples/full_fledged_streaming_service.py
|
||||
vars: {
|
||||
d2-config: {
|
||||
sketch: true
|
||||
theme-id: 1
|
||||
pad: 16
|
||||
layout-engine: elk
|
||||
}
|
||||
}
|
||||
direction: right
|
||||
s0: "actor 'streamer_0'" {
|
||||
fn: "stream_data(0)"
|
||||
}
|
||||
s1: "actor 'streamer_1'" {
|
||||
fn: "stream_data(1)"
|
||||
}
|
||||
agg: "actor 'aggregator'" {
|
||||
fn: "aggregate()"
|
||||
}
|
||||
main: "root actor" {
|
||||
fn: "main()"
|
||||
}
|
||||
s0 -> agg: "async for:\nyielded ints"
|
||||
s1 -> agg: "async for:\nyielded ints"
|
||||
agg -> main: "deduped\nstream"
|
||||
|
|
@ -1,369 +0,0 @@
|
|||
Anatomy of the runtime
|
||||
======================
|
||||
|
||||
You can get a long way with ``tractor`` by treating it as "trio_
|
||||
with nurseries that spawn processes". But once you start asking
|
||||
*where does my msg actually go?*, *which process is that?* or
|
||||
*who keeps the phonebook?*, it pays to know how the runtime hangs
|
||||
together. This page walks the stack top to bottom.
|
||||
|
||||
.. d2:: diagrams/runtime_stack.d2
|
||||
:caption: The four runtime layers inside *every* actor process.
|
||||
:alt: layer cake of app tasks, tractor runtime, IPC, OS process
|
||||
:width: 70%
|
||||
|
||||
The layer cake
|
||||
--------------
|
||||
|
||||
Every actor process is the same four-layer sandwich:
|
||||
|
||||
- **your app**: plain ``trio`` tasks, nurseries and cancel
|
||||
scopes; nothing special. ``tractor`` is a `structured
|
||||
concurrency`_ (SC) multi-processing runtime built on trio_ and
|
||||
the whole pitch is that this layer stays *just trio*: no
|
||||
callbacks, no futures, no proxy objects.
|
||||
- **the** ``tractor`` **runtime**: a per-process
|
||||
:class:`tractor.Actor` running the msg loop and RPC task
|
||||
scheduler, plus the user-facing primitives layered on it:
|
||||
:class:`tractor.ActorNursery` (spawning + supervision),
|
||||
:class:`tractor.Portal` (calling into a peer) and
|
||||
:class:`tractor.Context` + :class:`tractor.MsgStream`
|
||||
(SC-linked cross-actor task pairs and streaming).
|
||||
- **IPC channels**: one :class:`tractor.Channel` per connected
|
||||
peer, each wrapping a ``MsgTransport`` that ships
|
||||
msgspec_-typed msgs over TCP or UDS.
|
||||
- **the OS**: one process per actor, started by a swappable
|
||||
spawn backend.
|
||||
|
||||
The property that holds it all together: SC composes *through*
|
||||
the layers. A crash in a leaf actor's app task unwinds that
|
||||
actor's trio tree, ships across its IPC channel as a typed
|
||||
``Error`` msg, and unwinds the parent's trio tree in turn — the
|
||||
"SC-transitive supervision protocol" from the README's pitch.
|
||||
The whole tree cancels and errors like one big trio program; it
|
||||
just happens to be spread across processes.
|
||||
|
||||
One actor, one process, one ``trio.run()``
|
||||
------------------------------------------
|
||||
|
||||
A ``tractor`` "actor" is not a green thread, nor an object with
|
||||
a mailbox, nor a coroutine: it's one OS process running one
|
||||
:func:`trio.run` whose root task boots the runtime machinery —
|
||||
msg loop, RPC task scheduler, IPC server — all embodied by a
|
||||
single :class:`tractor.Actor` instance.
|
||||
|
||||
.. margin:: Shared nothing
|
||||
|
||||
Processes buy you a real `shared nothing architecture`_: no
|
||||
accidentally-shared mutable state, no GIL contention, and
|
||||
every actor can be inspected (or killed) like any other OS
|
||||
process.
|
||||
|
||||
You rarely construct an :class:`~tractor.Actor` yourself; the
|
||||
runtime makes exactly one per process and you grab it with
|
||||
:func:`tractor.current_actor`:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import tractor
|
||||
|
||||
actor = tractor.current_actor() # NoRuntime if none running
|
||||
print(actor.aid.name) # str name, need not be unique
|
||||
print(actor.aid.uuid) # uuid4 str, IS unique
|
||||
print(actor.aid.pid) # the OS pid
|
||||
print(actor.uid) # legacy (name, uuid) pair
|
||||
|
||||
Identity is carried by the ``Aid`` msg-struct (see
|
||||
``tractor.msg.types``): a ``name``/``uuid``/``pid`` triple
|
||||
exchanged in the very first "mailbox handshake" whenever two
|
||||
actors connect. It's what the registrar stores and what shows up
|
||||
in logs and proc-titles. The older ``.uid`` 2-tuple of
|
||||
``(name, uuid)`` predates ``Aid`` and is still pervasive across
|
||||
the codebase; treat it as the legacy spelling of the same
|
||||
identity.
|
||||
|
||||
If this smells like the `actor model`_, sure — but as the
|
||||
README warns, it probably doesn't look like what you *think* an
|
||||
actor model looks like, and that's intentional. Here an "actor"
|
||||
is purely a runtime-unit-of-abstraction: process +
|
||||
``trio.run()`` + IPC machinery.
|
||||
|
||||
IPC: channels, transports, addresses
|
||||
------------------------------------
|
||||
|
||||
Two connected actors talk through a :class:`tractor.Channel`: a
|
||||
duplex, per-peer msg pipe. Each ``Channel`` wraps a
|
||||
``MsgTransport`` instance which does the wire work: framing,
|
||||
encode/decode and the socket itself. The encoding is msgpack
|
||||
(via msgspec_) and *every* msg is an instance of one of the
|
||||
runtime's tagged-union :class:`msgspec.Struct` types: the
|
||||
``Aid`` handshake, ``Start``/``StartAck`` (RPC init),
|
||||
``Started``/``Yield``/``Stop``/``Return`` (the ctx dialog
|
||||
phases), ``Error``, etc. There is no raw-bytes mode; the
|
||||
msg-spec *is* the protocol, which is exactly what lets payloads
|
||||
be type-limited per-context (see ``pld_spec`` in
|
||||
:doc:`/guide/context`).
|
||||
|
||||
Addresses come in two spellings:
|
||||
|
||||
- *unwrapped*: the plain-tuple form you pass to user APIs —
|
||||
``('127.0.0.1', 1616)`` for tcp, or a
|
||||
``(<filedir>, <filename>)`` path-pair for uds;
|
||||
- *wrapped*: the internal ``TCPAddress``/``UDSAddress`` struct
|
||||
types (plus libp2p-style multiaddr helpers over in
|
||||
``tractor.discovery``).
|
||||
|
||||
You only ever need the tuple form; the runtime wraps and
|
||||
unwraps at the boundaries.
|
||||
|
||||
TCP: the boring default
|
||||
***********************
|
||||
|
||||
The default transport (``'tcp'``) binds each actor's IPC server
|
||||
to loopback ``('127.0.0.1', <random port>)`` unless told
|
||||
otherwise, and is the only choice when your tree spans hosts.
|
||||
Nothing exotic: ``trio`` TCP streams + length-prefixed msgpack
|
||||
framing.
|
||||
|
||||
UDS: same-host, creds included
|
||||
******************************
|
||||
|
||||
Pass ``enable_transports=['uds']`` and actors instead talk over
|
||||
unix-domain sockets, with socket files placed in the per-user
|
||||
runtime dir (``$XDG_RUNTIME_DIR/tractor/`` on linux, the
|
||||
``platformdirs`` equivalent elsewhere). Two perks over tcp on a
|
||||
single host:
|
||||
|
||||
- no ports to fight over; addrs are just file paths,
|
||||
- the kernel snitches on your peer for free: the listening side
|
||||
reads the connector's ``pid`` (plus ``uid``/``gid`` on linux)
|
||||
straight off the socket via ``SO_PEERCRED`` /
|
||||
``LOCAL_PEERPID`` — no extra handshake msgs required B)
|
||||
|
||||
.. warning::
|
||||
|
||||
Socket-file lifetime == listening actor lifetime. On
|
||||
listener teardown the runtime ``os.unlink()``\s the socket
|
||||
file immediately, so any *late* connection attempt (say, a
|
||||
sub-actor racing to deregister with a registrar that's
|
||||
already shutting down) fails with ``FileNotFoundError``.
|
||||
And ofc, UDS is same-host only.
|
||||
|
||||
Here's a full actor tree run entirely over uds:
|
||||
|
||||
.. literalinclude:: ../../examples/uds_transport_actor_tree.py
|
||||
:caption: examples/uds_transport_actor_tree.py
|
||||
:language: python
|
||||
|
||||
Picking a transport
|
||||
*******************
|
||||
|
||||
Transport choice is per-actor via the ``enable_transports``
|
||||
kwarg accepted by :func:`tractor.open_root_actor` (and proxied
|
||||
through ``open_nursery()`` when it implicitly boots the
|
||||
runtime), plus per-child via
|
||||
``ActorNursery.start_actor(enable_transports=...)``. Two rules
|
||||
the runtime enforces today:
|
||||
|
||||
- exactly ONE transport per actor: multi-transport actors are
|
||||
on the roadmap but currently raise ``RuntimeError``;
|
||||
- your ``registry_addrs`` protos must all be in
|
||||
``enable_transports``: mismatches fail fast with
|
||||
``ValueError`` instead of (as in darker times) hanging the
|
||||
registrar handshake forever.
|
||||
|
||||
Spawn backends
|
||||
--------------
|
||||
|
||||
How does an actor actually *become* a process? Via a swappable
|
||||
spawn backend, selected with the ``start_method`` kwarg to
|
||||
:func:`tractor.open_root_actor`:
|
||||
|
||||
``'trio'`` (default)
|
||||
The home-grown spawner: re-exec the child as
|
||||
``python -m tractor._child`` using ``trio``'s subprocess
|
||||
machinery, then bootstrap it over the first IPC exchange
|
||||
(the parent ships a ``SpawnSpec`` msg carrying all init
|
||||
state). Supported on all platforms and the most battle
|
||||
tested choice by far.
|
||||
|
||||
``'mp_spawn'`` / ``'mp_forkserver'``
|
||||
The stdlib :mod:`multiprocessing` start-methods of the same
|
||||
names (forkserver is posix-only). Mostly interesting for
|
||||
ecosystem compat and start-up-latency tuning.
|
||||
|
||||
``'subint'`` (in development, py3.14+)
|
||||
On the roadmap (not yet selectable via ``start_method`` on
|
||||
this release): run each actor as a `PEP 734`_
|
||||
sub-interpreter (``concurrent.interpreters``) driven on its
|
||||
own OS thread *inside the parent process* — interpreter-level
|
||||
shared-nothing isolation with much faster start-up. Yes, this
|
||||
bends the one-actor-one-process rule; the rest of the model
|
||||
is unchanged.
|
||||
|
||||
The ``TRACTOR_SPAWN_METHOD`` env-var beats any caller-passed
|
||||
``start_method``, so you can swap backends under an unmodified
|
||||
app:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
TRACTOR_SPAWN_METHOD=mp_forkserver python my_app.py
|
||||
|
||||
One current limitation worth knowing: ``debug_mode=True`` (the
|
||||
crash-to-REPL machinery) is only supported on backends whose
|
||||
child-side runtime is trio-native, e.g. the default ``'trio'``;
|
||||
see :doc:`/guide/debugging` for the deats.
|
||||
|
||||
The registrar
|
||||
-------------
|
||||
|
||||
Discovery needs a phonebook. Every actor, as part of boot,
|
||||
registers its ``Aid`` and bind-addrs with the *registrar*: an
|
||||
otherwise ordinary actor (a :class:`tractor.Registrar`, subtype
|
||||
of :class:`~tractor.Actor`) that keeps the name -> addrs table
|
||||
for the tree; on graceful exit each actor de-registers itself.
|
||||
|
||||
.. margin:: Default registry addrs
|
||||
|
||||
With no ``registry_addrs`` passed:
|
||||
|
||||
- tcp: ``('127.0.0.1', 1616)``
|
||||
- uds: ``registry@1616.sock``
|
||||
in the runtime dir
|
||||
|
||||
Who *is* the registrar? Decided at root boot, rendezvous style.
|
||||
:func:`tractor.open_root_actor` probes each addr in
|
||||
``registry_addrs`` with a quick connect-ping, then:
|
||||
|
||||
- **somebody answered**: this root is a plain actor; it
|
||||
registers with the existing registrar and binds random
|
||||
same-proto addrs for its own IPC server;
|
||||
- **nobody answered**: this root *becomes* the registrar and
|
||||
binds the registry addrs itself.
|
||||
|
||||
So single-program trees need zero config — the root quietly
|
||||
self-appoints — while multi-program setups share a registrar by
|
||||
pointing every program at the same ``registry_addrs``. Pass
|
||||
``ensure_registry=True`` to demand that *this* call create the
|
||||
registry; it raises if the addrs are already served.
|
||||
|
||||
The lookup APIs — :func:`tractor.find_actor`,
|
||||
:func:`tractor.wait_for_actor`, :func:`tractor.query_actor` and
|
||||
:func:`tractor.get_registry` — all consult it (after first
|
||||
checking already-connected peers):
|
||||
|
||||
.. literalinclude:: ../../examples/service_daemon_discovery.py
|
||||
:caption: examples/service_daemon_discovery.py
|
||||
:language: python
|
||||
|
||||
If you bump into "arbiter" in old issues or posts: that's the
|
||||
legacy name for the same thing, surviving in-code only as the
|
||||
``Arbiter = Registrar`` class alias; all current terminology is
|
||||
"registrar"/"registry". Fair warning per the README: this is
|
||||
still a **very naive** discovery sys (no re-election, no gossip
|
||||
protocol... yet) and a registrar is expected to outlive its
|
||||
registrants.
|
||||
|
||||
Runtime env vars
|
||||
----------------
|
||||
|
||||
A few env-vars let you re-tune a whole tree *without touching
|
||||
app code*; each wins over its corresponding kwarg:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 30 44 26
|
||||
|
||||
* - env-var
|
||||
- effect
|
||||
- vs. kwarg
|
||||
* - ``TRACTOR_LOGLEVEL``
|
||||
- crank (or silence) console-log verbosity for every actor
|
||||
in the tree
|
||||
- beats ``loglevel``
|
||||
* - ``TRACTOR_SPAWN_METHOD``
|
||||
- swap the process spawn backend
|
||||
- beats ``start_method``
|
||||
* - ``TRACTOR_ENABLE_STACKSCOPE``
|
||||
- install the ``SIGUSR1`` task-tree-dump handler in every
|
||||
actor, even outside ``debug_mode`` (see
|
||||
:doc:`/guide/debugging`)
|
||||
- OR'd with ``enable_stack_on_sig``
|
||||
|
||||
Spotting actors from your shell
|
||||
*******************************
|
||||
|
||||
Every sub-actor sets an OS-level proc-title of the form
|
||||
``_subactor[<name>@<pid>]`` (via ``setproctitle``, silently
|
||||
skipped when not installed) so ``ps``/``htop``/``pstree`` show
|
||||
*which actor is which* at a glance. The README's signature
|
||||
incantation — watch a tree build and self-destruct live:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$TERM -e watch -n 0.1 "pstree -a $$" \
|
||||
& python examples/nested_actor_tree.py \
|
||||
&& kill $!
|
||||
|
||||
For scripting there are two stable cmdline markers:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
pgrep -fa '_subactor\[' # live, titled sub-actors
|
||||
pgrep -fa 'tractor._child' # 'trio'-backend children not
|
||||
# yet (re)titled
|
||||
|
||||
The title also lands in the kernel ``comm`` (truncated to ~15
|
||||
bytes) which survives into zombie state — that's what
|
||||
``tractor``'s own test-harness reapers key off. To be crystal
|
||||
clear about the contract though: you should never *need* a
|
||||
reaper; if you can create zombie child processes (without using
|
||||
a system signal) it **is a bug** — please report it!
|
||||
|
||||
Logging
|
||||
-------
|
||||
|
||||
The runtime logs through a thin adapter over stdlib
|
||||
:mod:`logging` that stamps every record with actor + task info.
|
||||
Two calls get you going:
|
||||
|
||||
.. code:: python
|
||||
|
||||
from tractor.log import get_console_log, get_logger
|
||||
|
||||
log = get_logger(__name__) # actor/task-aware sub-logger
|
||||
get_console_log('info') # attach console handler @ level
|
||||
|
||||
(or just pass ``loglevel='info'`` to
|
||||
:func:`tractor.open_root_actor` and the console handler comes up
|
||||
with the runtime).
|
||||
|
||||
``tractor`` adds custom levels — and matching logger methods —
|
||||
that slot between the stdlib ones so you can dial in *which
|
||||
runtime subsystem* you want to hear from: ``.transport()`` (5),
|
||||
``.runtime()`` (15), ``.devx()`` (17), ``.cancel()`` (22), plus
|
||||
a ``PDB`` (500) level for debugger chatter. E.g.
|
||||
``loglevel='cancel'`` plays the whole cancellation chorus while
|
||||
staying quiet about transport-layer noise. Beyond that
|
||||
``tractor`` isn't opinionated about how you consume logs: it's
|
||||
all stdlib ``logging`` underneath.
|
||||
|
||||
Where to next?
|
||||
--------------
|
||||
|
||||
.. seealso::
|
||||
|
||||
- :doc:`/guide/context` — the SC-linked cross-actor task API
|
||||
that rides on every ``Channel``.
|
||||
- :doc:`/guide/debugging` — ``debug_mode``, the multi-process
|
||||
REPL and ``stackscope`` task-tree dumps.
|
||||
- :doc:`/explain/sc-distributed` — *why*
|
||||
one-actor-one-process, and what kind of "actor model" this
|
||||
is (and isn't).
|
||||
|
||||
.. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||
.. _trio: https://github.com/python-trio/trio
|
||||
.. _actor model: https://en.wikipedia.org/wiki/Actor_model
|
||||
.. _shared nothing architecture: https://en.wikipedia.org/wiki/Shared-nothing_architecture
|
||||
.. _msgspec: https://jcristharif.com/msgspec/
|
||||
.. _PEP 734: https://peps.python.org/pep-0734/
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
Big ideas
|
||||
=========
|
||||
The conceptual core of ``tractor``: what *structured
|
||||
concurrency* (SC) means once your "tasks" are whole
|
||||
processes, and how the runtime is layered to deliver
|
||||
that without (much) magic.
|
||||
|
||||
If you only read one page in these docs make it
|
||||
:doc:`sc-distributed`; if you read two, follow it with
|
||||
:doc:`architecture`.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
sc-distributed
|
||||
architecture
|
||||
|
|
@ -1,221 +0,0 @@
|
|||
Structured concurrency, across processes
|
||||
=========================================
|
||||
``tractor`` makes one bet: the discipline that made
|
||||
``trio``'s concurrency *readable* — `structured
|
||||
concurrency`_ (SC) — works just as well when the
|
||||
"tasks" are whole OS processes talking over a wire.
|
||||
This page distills what that means, from first
|
||||
principles, with as little ceremony as possible.
|
||||
|
||||
.. margin:: The canon
|
||||
|
||||
If SC is new to you, the seminal `blog post`_ is
|
||||
still the best hour you'll spend on concurrent
|
||||
programming; the `trio docs`_, wikipedia's SC_
|
||||
page and the diagrams over at libdill-docs_ round
|
||||
it out nicely.
|
||||
|
||||
SC in one breath
|
||||
----------------
|
||||
Structured concurrency is the rule that **concurrency
|
||||
gets a scope**: every task is spawned *inside* a block
|
||||
(a ``trio`` *nursery*) and that block **cannot exit
|
||||
until every task it spawned has finished** — returned,
|
||||
errored, or been cancelled.
|
||||
|
||||
That one rule buys you the properties you already
|
||||
rely on in sequential code,
|
||||
|
||||
- a function call is a *black box*: when it returns,
|
||||
everything it started is **done** — no secret
|
||||
background tasks leaking out the sides,
|
||||
- an exception **always has somewhere to go**: up the
|
||||
(task) tree to a parent which is, by construction,
|
||||
still there waiting,
|
||||
- cancellation has a well defined *shape*: cancel a
|
||||
scope and it flows down to every task inside it,
|
||||
and only those.
|
||||
|
||||
In short: your **runtime task tree matches your source
|
||||
code's indentation**. Concurrency you can read.
|
||||
|
||||
The leap: process-shaped tasks
|
||||
------------------------------
|
||||
Now swap "task" for "process".
|
||||
|
||||
A ``tractor`` *actor* is just a Python process running
|
||||
its own ``trio.run()`` — its own private task tree,
|
||||
sharing **nothing** with its siblings. You spawn
|
||||
actors from an :class:`tractor.ActorNursery`, which
|
||||
behaves exactly the way the name implies,
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with tractor.open_nursery() as an:
|
||||
portal = await an.start_actor(
|
||||
'worker',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
...
|
||||
# ^ block exit == every spawned process has
|
||||
# completed, errored or been cancelled, and
|
||||
# been **reaped**. No exceptions, no zombies.
|
||||
|
||||
so the whole program becomes a *tree of process-trees*
|
||||
— a `supervision tree`_ in erlang-speak — where every
|
||||
arrow means "spawned by, **waited on by**, and
|
||||
supervised by".
|
||||
|
||||
.. d2:: diagrams/actor_tree.d2
|
||||
:caption: A ``tractor`` program: a process tree of
|
||||
``trio`` task trees; every parent **must wait**
|
||||
on its children.
|
||||
:width: 85%
|
||||
|
||||
Causality: no process outlives its parent
|
||||
-----------------------------------------
|
||||
The stdlib's ``multiprocessing`` (and most "job
|
||||
queue" systems) treat child processes as
|
||||
fire-and-forget by default: orphans, zombies, lost
|
||||
tracebacks and ``kill -9`` cleanup scripts are *your*
|
||||
problem. ``tractor`` instead inherits ``trio``'s
|
||||
`causality`_ discipline,
|
||||
|
||||
- **no spawning willy-nilly**: every actor is born
|
||||
from a nursery block with a known parent,
|
||||
- **lifetimes nest**: a sub-actor's entire process
|
||||
tree lives strictly inside its parent's nursery
|
||||
scope,
|
||||
- **teardown is guaranteed**: when a scope exits (or
|
||||
errors, or is cancelled) the runtime SIGINTs,
|
||||
waits, and (only if it must) hard-kills + reaps
|
||||
everything underneath.
|
||||
|
||||
We take the zombie thing personally: *if you can
|
||||
create orphaned child processes without using a
|
||||
system signal, it* **is a bug** — and there's a test
|
||||
suite to back that sentence up.
|
||||
|
||||
Errors always propagate (yes, across the wire)
|
||||
----------------------------------------------
|
||||
In ``trio``, an exception in any task tears through
|
||||
its nursery to a parent that must handle it —
|
||||
`exceptions always propagate`_. ``tractor`` extends
|
||||
the same guarantee across process boundaries: an
|
||||
uncaught error in a remote task is
|
||||
|
||||
1. captured + serialized in the child,
|
||||
2. shipped home over IPC as a typed ``Error`` msg,
|
||||
3. re-raised in the parent **boxed** as a
|
||||
:class:`tractor.RemoteActorError` carrying the
|
||||
original type (``.boxed_type``), a rendered remote
|
||||
traceback, and the erroring actor's id,
|
||||
|
||||
while the supervising nursery applies its (currently
|
||||
*one-cancels-all*, just like ``trio``) strategy to any
|
||||
sibling actors. A crash three processes deep arrives
|
||||
at your shell as one coherent, causal traceback chain
|
||||
— not a silent dead worker and a stuck queue.
|
||||
|
||||
Cancellation is a request, supervision is the rule
|
||||
--------------------------------------------------
|
||||
Cancellation likewise keeps ``trio``'s semantics
|
||||
*verbatim*, just transported: cancelling an actor
|
||||
nursery (or a single :class:`tractor.Context` between
|
||||
two tasks in different processes) sends an explicit
|
||||
cancel **request** over IPC which the remote runtime
|
||||
translates into a real ``trio`` cancel-scope cancel —
|
||||
then *acks back* so the requester can await
|
||||
confirmation within a bounded time. Nothing is ever
|
||||
"just killed" first; graceful always precedes brutal.
|
||||
|
||||
Because every cross-process dialog is a pair of
|
||||
**linked tasks** — one on each side, each inside its
|
||||
own cancel scope — SC stays *transitive*: supervision
|
||||
doesn't stop at the process boundary, it tunnels
|
||||
through every hop of the tree. The wire protocol that
|
||||
enforces this (a small set of typed msgs:
|
||||
``Start``/``Started``/``Yield``/``Stop``/``Return``/
|
||||
``Error``) is detailed in :doc:`/guide/msging` and
|
||||
:doc:`/guide/context`.
|
||||
|
||||
Hold up, is this an "actor model"?
|
||||
----------------------------------
|
||||
Let's stop and ask how many canon actor model papers
|
||||
you've actually read ;)
|
||||
|
||||
From our experience many "actor systems" aren't really
|
||||
"actor models" since they **don't adhere** to the `3
|
||||
axioms`_ and pay even less attention to the problem of
|
||||
`unbounded non-determinism`_ — which was the whole
|
||||
point for creating the model in the first place.
|
||||
|
||||
From `the author's mouth`_, the **only** requirement
|
||||
is `adherence to`_ the `3 axioms`_::
|
||||
|
||||
In response to a message, an actor may:
|
||||
|
||||
- send a finite number of new messages
|
||||
- create a finite number of new actors
|
||||
- designate a new behavior to process subsequent
|
||||
messages
|
||||
|
||||
Want to debate whether that makes us a "real" actor
|
||||
model? Chime in on our chat, or on one of these issues
|
||||
*after you've read everything in them* ;)
|
||||
|
||||
- `#210 <https://github.com/goodboy/tractor/issues/210>`_
|
||||
- `#18 <https://github.com/goodboy/tractor/issues/18>`_
|
||||
|
||||
``tractor`` adheres — actors exchange msgs, spawn
|
||||
actors, and swap behaviors — **with no extra API** to
|
||||
learn. What we *don't* copy is the cultural baggage:
|
||||
no visible mailboxes, no untyped fire-and-forget
|
||||
``send()``, no "let it crash" without a supervisor
|
||||
that actually hears about it, and definitely no
|
||||
shared-reference *proxy objects* pretending the
|
||||
network isn't there. If our "actors" don't look like
|
||||
what you expected, that's **intentional**: being an
|
||||
actor model is just one property of the system; being
|
||||
*structured* is the point.
|
||||
|
||||
Why processes at all?
|
||||
---------------------
|
||||
Python has a GIL; an actor model by definition shares
|
||||
no state; so the *process* is the natural runtime
|
||||
unit — you get real multi-core parallelism and hard
|
||||
memory isolation for free. But the deeper win is
|
||||
uniformity: because actors only ever talk via msgs
|
||||
over a :class:`tractor.Channel` (TCP, UDS, more to
|
||||
come), the **same code** runs your laptop's worker
|
||||
pool and a multi-host cluster; "distributed" is a
|
||||
deployment detail, not an API.
|
||||
|
||||
It's just ``trio``
|
||||
------------------
|
||||
If you remember one framing, make it this: ``tractor``
|
||||
**is just** ``trio`` — with nurseries that can spawn
|
||||
processes and streams that can cross them. Same
|
||||
nursery discipline, same cancellation semantics, same
|
||||
"how was this not always the API?" feeling, one level
|
||||
up the process tree.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`/explain/architecture` for how the runtime
|
||||
layers deliver all of the above, and
|
||||
:doc:`/start/quickstart` to feel it in ~20 lines of
|
||||
code.
|
||||
|
||||
.. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||
.. _SC: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||
.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
|
||||
.. _trio docs: https://trio.readthedocs.io/en/latest/
|
||||
.. _libdill-docs: https://sustrik.github.io/libdill/structured-concurrency.html
|
||||
.. _supervision tree: https://www.erlang.org/doc/design_principles/des_princ.html
|
||||
.. _causality: https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#c-c-c-c-causality-breaker
|
||||
.. _exceptions always propagate: https://trio.readthedocs.io/en/latest/design.html#exceptions-always-propagate
|
||||
.. _the author's mouth: https://www.youtube.com/watch?v=7erJ1DV_Tlo&t=162s
|
||||
.. _adherence to: https://www.youtube.com/watch?v=7erJ1DV_Tlo&t=1821s
|
||||
.. _3 axioms: https://en.wikipedia.org/wiki/Actor_model#Fundamental_concepts
|
||||
.. _unbounded non-determinism: https://en.wikipedia.org/wiki/Unbounded_nondeterminism
|
||||
|
|
@ -1,309 +0,0 @@
|
|||
Infected ``asyncio``
|
||||
====================
|
||||
``tractor`` is "just trio_", but the Python world is packed with
|
||||
libraries that only speak ``asyncio``: websocket stacks, vendor
|
||||
SDKs, that one exchange client you can't route around. Rather than
|
||||
make you rewrite them, ``tractor`` lets you *quarantine* them inside
|
||||
a dedicated subactor which runs both event loops at once, with full
|
||||
`structured concurrency`_ (SC) guarantees maintained across the
|
||||
loop boundary *and* the process tree.
|
||||
|
||||
In the project's own words:
|
||||
|
||||
Yes, we spawn a python process, run ``asyncio``, start ``trio``
|
||||
on the ``asyncio`` loop, then send commands to the ``trio``
|
||||
scheduled tasks to tell ``asyncio`` tasks what to do XD
|
||||
|
||||
We call this "infected ``asyncio``" mode: the subactor's stdlib
|
||||
loop runs as the *host* with ``trio`` embedded on top in `guest`_
|
||||
mode, and your ``trio`` tasks drive ``asyncio`` tasks through
|
||||
a linked, SC-supervised, in-memory channel.
|
||||
|
||||
.. d2:: diagrams/infected_aio.d2
|
||||
:caption: One process, two schedulers: ``trio`` rides the
|
||||
``asyncio`` loop as a guest while the parent speaks plain
|
||||
``tractor`` IPC, none the wiser.
|
||||
:alt: parent actor connected over IPC to a subactor whose
|
||||
asyncio loop hosts trio in guest mode, with a
|
||||
LinkedTaskChannel pairing a trio task to an asyncio task
|
||||
|
||||
.. note::
|
||||
|
||||
Infected ``asyncio`` mode is **experimental**: it works (we
|
||||
beat on it plenty) but parts of the API surface and some
|
||||
edge-case semantics are still settling. Got opinions on the
|
||||
interop design? Feel free to sling them in `#273`_!
|
||||
|
||||
How the infection takes hold
|
||||
----------------------------
|
||||
A normal subactor boots by running the ``tractor`` runtime's task
|
||||
tree directly under ``trio.run()``. Pass ``infect_asyncio=True``
|
||||
at spawn time and the child's entrypoint changes shape entirely:
|
||||
|
||||
1. the process starts the stdlib loop via ``asyncio.run()``,
|
||||
2. the first ``asyncio`` task calls
|
||||
``trio.lowlevel.start_guest_run()``, embedding the ``trio``
|
||||
scheduler *inside* the already running ``asyncio`` loop (the
|
||||
upstream `guest`_-mode feature),
|
||||
3. the regular ``tractor`` runtime then boots on the guest
|
||||
``trio`` side and connects back to its parent like any other
|
||||
subactor.
|
||||
|
||||
.. margin:: Symptoms
|
||||
|
||||
Looks like your stdlib event loop has caught a case of "the
|
||||
trios"! Don't worry, you'll barely notice; and if anything
|
||||
gets too bad, your parents will know about it B)
|
||||
|
||||
Both schedulers interleave in a single thread, no GIL gymnastics
|
||||
required. From the rest of the actor tree the infected child is
|
||||
indistinguishable from any other actor: same IPC protocol, same
|
||||
supervision and cancellation semantics, same zombie-safety
|
||||
guarantees. The difference is purely internal: ``trio`` tasks in
|
||||
that process can start and drive ``asyncio`` tasks through the
|
||||
``tractor.to_asyncio`` API.
|
||||
|
||||
Spawning an infected subactor
|
||||
-----------------------------
|
||||
Just flip the flag on :meth:`tractor.ActorNursery.start_actor`:
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with tractor.open_nursery() as an:
|
||||
portal = await an.start_actor(
|
||||
'aio_side',
|
||||
enable_modules=[__name__],
|
||||
infect_asyncio=True,
|
||||
)
|
||||
|
||||
The one-shot convenience ``ActorNursery.run_in_actor()`` accepts
|
||||
the same flag. The ``to_asyncio`` APIs may **only** be called from
|
||||
tasks inside an infected actor; calling them anywhere else raises
|
||||
a loud ``RuntimeError``. You can introspect at runtime with
|
||||
``tractor.current_actor().is_infected_aio()``.
|
||||
|
||||
Linking tasks with ``open_channel_from()``
|
||||
------------------------------------------
|
||||
The core primitive is :func:`tractor.to_asyncio.open_channel_from`,
|
||||
an async context manager which starts your ``asyncio`` function as
|
||||
a real ``asyncio.Task`` and yields a two-way channel linking it to
|
||||
the calling ``trio`` task:
|
||||
|
||||
.. code:: python
|
||||
|
||||
from tractor import to_asyncio
|
||||
|
||||
async with to_asyncio.open_channel_from(
|
||||
aio_main, # async def aio_main(chan, **kwargs)
|
||||
period=0.5, # extra kwargs are passed through
|
||||
) as (chan, first):
|
||||
await chan.send('tick')
|
||||
|
||||
The semantics deliberately mirror the inter-actor ``Context``
|
||||
handshake from :doc:`/guide/context`:
|
||||
|
||||
- the target fn must declare a parameter literally named ``chan``;
|
||||
the runtime injects the shared
|
||||
:class:`~tractor.to_asyncio.LinkedTaskChannel` by keyword.
|
||||
- the ``trio`` side blocks at entry until the ``asyncio`` task
|
||||
calls ``chan.started_nowait(value)``; that value is delivered as
|
||||
``first``, exactly like the ``(ctx, first)`` pair you get from
|
||||
``Portal.open_context()`` after the child calls
|
||||
``ctx.started()``.
|
||||
- a first value **must** be sent from the ``asyncio`` side or the
|
||||
``trio`` side will never unblock.
|
||||
- on block exit the pair is torn down *together*; neither task can
|
||||
outlive the other (more on this below).
|
||||
|
||||
A full example: the echo server
|
||||
-------------------------------
|
||||
Here's the canonical demo, a round-trip echo service where the
|
||||
``asyncio`` task is told what to do by a ``trio`` task which is in
|
||||
turn driven over IPC by the root actor:
|
||||
|
||||
.. literalinclude:: ../../examples/infected_asyncio_echo_server.py
|
||||
:caption: examples/infected_asyncio_echo_server.py
|
||||
:language: python
|
||||
|
||||
What's going on?
|
||||
|
||||
- there are three task layers: the root actor's pure ``trio``
|
||||
task, the infected child's ``trio``-side ``@tractor.context``
|
||||
endpoint (``trio_to_aio_echo_server()``), and the child's
|
||||
``asyncio`` task (``aio_echo_server()``).
|
||||
- two ``started``-style handshakes compose: the aio task's
|
||||
``chan.started_nowait('start')`` unblocks the child's
|
||||
``open_channel_from()`` entry, then the child relays that same
|
||||
value up via ``await ctx.started(first)`` which unblocks the
|
||||
root's ``open_context()`` entry. Synchronization all the way
|
||||
down, er, up.
|
||||
- each round trip flows: root ``stream.send()`` -> IPC -> child
|
||||
``async for msg in stream`` -> ``chan.send(msg)`` -> aio
|
||||
``await chan.get()`` -> ``chan.send_nowait()`` -> child
|
||||
``chan.receive()`` -> ``stream.send(out)`` -> IPC -> root.
|
||||
- when the root breaks out of its stream loop and exits the
|
||||
context block, the child's stream ends, its channel block exits,
|
||||
and the ``asyncio`` task is reaped along with it; the final
|
||||
``portal.cancel_actor()`` then tears down the whole process. No
|
||||
orphaned ``asyncio`` tasks, no zombie procs; if you manage to
|
||||
create either it **is a bug**.
|
||||
|
||||
``LinkedTaskChannel``: one channel, two sides
|
||||
---------------------------------------------
|
||||
The same channel object is shared by both tasks; which methods you
|
||||
call depends on which loop schedules your task. The ``trio`` side
|
||||
gets a standard ``trio.abc.Channel`` interface while the
|
||||
``asyncio`` side gets queue-flavored, mostly-sync methods:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 14 36 50
|
||||
|
||||
* - side
|
||||
- call
|
||||
- what it does
|
||||
* - ``trio``
|
||||
- ``await chan.send(item)``
|
||||
- ship ``item`` to the ``asyncio`` task (enqueues onto an
|
||||
internal ``asyncio.Queue``).
|
||||
* - ``trio``
|
||||
- ``await chan.receive()``
|
||||
- wait for the next value from the ``asyncio`` side; the
|
||||
channel also supports ``async for``.
|
||||
* - ``trio``
|
||||
- ``await chan.wait_for_result()``
|
||||
- block until the ``asyncio`` task completes; return its
|
||||
final result or raise its (translated) error.
|
||||
* - ``trio``
|
||||
- ``chan.subscribe()``
|
||||
- acm yielding a ``BroadcastReceiver`` so N local tasks can
|
||||
each consume a copy of the inbound stream (see below).
|
||||
* - ``trio``
|
||||
- ``chan.cancel_asyncio_task()``
|
||||
- explicitly request cancellation of the linked ``asyncio``
|
||||
task.
|
||||
* - ``asyncio``
|
||||
- ``chan.started_nowait(value)``
|
||||
- deliver the "first" value; unblocks the ``trio`` side's
|
||||
``open_channel_from()`` entry (mirrors ``ctx.started()``).
|
||||
* - ``asyncio``
|
||||
- ``await chan.get()``
|
||||
- wait for the next value sent from the ``trio`` side.
|
||||
* - ``asyncio``
|
||||
- ``chan.send_nowait(item)``
|
||||
- push a value to the ``trio`` side without blocking.
|
||||
|
||||
Fan-out with ``.subscribe()``
|
||||
*****************************
|
||||
Just like :meth:`tractor.MsgStream.subscribe` does for IPC
|
||||
streams, ``chan.subscribe()`` lets multiple local ``trio`` tasks
|
||||
each receive *every* value sent from the single ``asyncio`` task:
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with chan.subscribe() as bcast:
|
||||
async for msg in bcast:
|
||||
...
|
||||
|
||||
The underlying broadcast machinery is lazily allocated on first
|
||||
use and is *not* reversible for the channel's remaining lifetime,
|
||||
so only reach for it when you actually want the fan-out.
|
||||
|
||||
One-shot calls with ``run_task()``
|
||||
----------------------------------
|
||||
When you just want a single ``asyncio`` result and no streaming
|
||||
dialog, skip the channel ceremony and use
|
||||
:func:`tractor.to_asyncio.run_task`:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import asyncio
|
||||
from tractor import to_asyncio
|
||||
|
||||
async def aio_fetch(url: str) -> str:
|
||||
await asyncio.sleep(0.3) # pretend-IO, aio style
|
||||
return f'<html>sup {url}</html>'
|
||||
|
||||
# from any trio task inside the infected actor:
|
||||
page = await to_asyncio.run_task(aio_fetch, url='https://x.io')
|
||||
|
||||
It schedules the fn as an ``asyncio.Task``, waits for completion
|
||||
and hands the return value back to ``trio``; think of it as the
|
||||
cross-loop sibling of ``ActorNursery.run_in_actor()``. Errors and
|
||||
cancellation are translated exactly as for channels.
|
||||
|
||||
Cross-loop errors and cancellation
|
||||
----------------------------------
|
||||
The paired tasks are *SC linked*: exception and cancel handling
|
||||
tears down **both** sides on any unexpected error or cancellation,
|
||||
in either loop. There is no fire-and-forget mode; a
|
||||
``LinkedTaskChannel`` is a supervision scope just like a
|
||||
``Context`` is across processes.
|
||||
|
||||
Because each loop has its own (incompatible) cancellation and exit
|
||||
machinery, boundary crossings are translated into dedicated
|
||||
exception types, all importable from ``tractor.to_asyncio``:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 26 22 52
|
||||
|
||||
* - exception
|
||||
- raised in
|
||||
- meaning
|
||||
* - ``AsyncioCancelled``
|
||||
- the ``trio`` task
|
||||
- the linked ``asyncio`` task was cancelled by itself or
|
||||
a 3rd party (i.e. *not* by the ``trio`` side).
|
||||
* - ``AsyncioTaskExited``
|
||||
- the ``trio`` task
|
||||
- the ``asyncio`` task returned/exited early while the
|
||||
``trio`` side still held the link open.
|
||||
* - ``TrioCancelled``
|
||||
- the ``asyncio`` task
|
||||
- the ``trio`` side was cancelled (or crashed) so the
|
||||
``asyncio`` task is being torn down per SC rules.
|
||||
* - ``TrioTaskExited``
|
||||
- the ``asyncio`` task
|
||||
- the ``trio`` side exited gracefully while the ``asyncio``
|
||||
task was still running; a "clean shutdown" signal much
|
||||
like closing a ``trio`` mem-chan.
|
||||
|
||||
By default ``open_channel_from(suppress_graceful_exits=True)``
|
||||
absorbs the two ``*TaskExited`` signals so happy-path teardown
|
||||
stays silent; pass ``False`` when your app wants to handle early
|
||||
peer-exit explicitly.
|
||||
|
||||
Past the task pair, everything composes with the normal actor
|
||||
story: an unhandled ``asyncio`` error is translated into the
|
||||
``trio`` side, propagates out of your ``@tractor.context``
|
||||
endpoint, and arrives at the parent boxed as
|
||||
a :class:`tractor.RemoteActorError`. One SC discipline,
|
||||
end-to-end, across loops *and* processes.
|
||||
|
||||
Breakpoints in ``asyncio`` tasks
|
||||
--------------------------------
|
||||
Yes, the multi-actor REPL works here too. With
|
||||
``debug_mode=True`` enabled on your tree the ``trio`` side of an
|
||||
infected actor can ``await tractor.pause()`` as usual, and with
|
||||
greenback enabled (``maybe_enable_greenback=True``) even the
|
||||
builtin ``breakpoint()`` works from *inside* ``asyncio`` tasks;
|
||||
see ``examples/debugging/asyncio_bp.py`` for the full tour. The
|
||||
root-TTY locking dance behind all this is covered in
|
||||
:doc:`/guide/debugging`.
|
||||
|
||||
Where to next?
|
||||
--------------
|
||||
.. seealso::
|
||||
|
||||
- :doc:`/guide/context` for the inter-actor handshake and
|
||||
streaming APIs which this whole interop layer mirrors.
|
||||
- :doc:`/guide/msging` for typing the payloads you shuttle
|
||||
between actors (and loops).
|
||||
- :doc:`/guide/debugging` for the multi-process REPL that
|
||||
keeps working even when your loop has "the trios".
|
||||
|
||||
.. _trio: https://github.com/python-trio/trio
|
||||
.. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||
.. _guest: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
|
||||
.. _#273: https://github.com/goodboy/tractor/issues/273
|
||||
|
|
@ -1,285 +0,0 @@
|
|||
Cancellation and error propagation
|
||||
==================================
|
||||
|
||||
``tractor`` supports ``trio``'s cancellation_ system *verbatim*,
|
||||
then extends it across process boundaries. If you know how to
|
||||
cancel a task in ``trio`` you already know how to cancel an actor —
|
||||
and its whole subtree — in ``tractor``; the runtime's job is making
|
||||
that statement hold over IPC with every structured concurrency (SC)
|
||||
guarantee intact.
|
||||
|
||||
The ground rules,
|
||||
|
||||
- a remote actor is **never** cancelled unless explicitly requested
|
||||
(by a parent or peer), unless supervision demands it (an error
|
||||
triggered one-cancels-all teardown), or unless there's a bug in
|
||||
``tractor`` itself (please report it!),
|
||||
- (remote) errors `always propagate`_ back to the parent
|
||||
supervisor; nothing is silently dropped on the floor,
|
||||
- every spawned process gets reaped no matter how it dies; if you
|
||||
can create a zombie child process (without using a system signal)
|
||||
it **is a bug**.
|
||||
|
||||
``trio`` cancellation, across the wire
|
||||
--------------------------------------
|
||||
|
||||
Locally everything is bog-standard ``trio``: nurseries, cancel
|
||||
scopes, timeouts. ``tractor`` adds exactly one twist: a cancel
|
||||
scope can't physically reach into another process, so the runtime
|
||||
*relays cancellation as messages*. Concretely,
|
||||
|
||||
- cancelling an *actor* means sending it a runtime-cancel request
|
||||
msg; the target then runs its own graceful teardown — cancelling
|
||||
RPC tasks, closing channels, exiting its :func:`trio.run` — and
|
||||
acks the request back to the canceller,
|
||||
- cancelling a single *cross-actor task* works through the
|
||||
:class:`tractor.Context` layer: each ``ctx`` task-pair is
|
||||
cancel-scope-linked over IPC such that either side erroring or
|
||||
cancelling relays an equivalent error to the other side (see
|
||||
:doc:`/guide/context` for the gory details),
|
||||
- a cancel is therefore always a *request with an ack*: the
|
||||
canceller does a **bounded wait** for confirmation and escalates
|
||||
if the peer is unresponsive (see the teardown ladder below).
|
||||
|
||||
One-cancels-all supervision
|
||||
---------------------------
|
||||
|
||||
An :class:`tractor.ActorNursery` supervises subactors `exactly like
|
||||
trio`_ nurseries supervise tasks: when one child errors, the error
|
||||
propagates to the supervising block and **all** sibling subactors
|
||||
get cancelled before the error continues bubbling up the (process)
|
||||
tree.
|
||||
|
||||
.. d2:: diagrams/error_propagation.d2
|
||||
:caption: One-cancels-all: no zombies, no lost errors.
|
||||
:alt: error propagation up a subactor tree
|
||||
:width: 80%
|
||||
|
||||
.. literalinclude:: ../../examples/remote_error_propagation.py
|
||||
:caption: examples/remote_error_propagation.py
|
||||
:language: python
|
||||
|
||||
What's going on here?
|
||||
|
||||
- three healthy actors are spawned as daemons via
|
||||
:meth:`tractor.ActorNursery.start_actor`; left alone they'd
|
||||
happily idle forever,
|
||||
- a fourth actor runs ``assert_err()`` via ``.run_in_actor()`` and
|
||||
promptly trips its ``assert 0``,
|
||||
- the resulting ``AssertionError`` ships back over IPC as a
|
||||
serialized error msg and re-raises *boxed* inside the nursery
|
||||
block as a :class:`tractor.RemoteActorError`,
|
||||
- the nursery reacts like any ``trio`` nursery would: it cancels
|
||||
the three healthy siblings (graceful runtime-cancel requests,
|
||||
acks awaited), reaps all four processes, then re-raises,
|
||||
- ``trio.run(main)`` sees that same ``RemoteActorError`` in the
|
||||
parent-most process — propagation is end-to-end or bust.
|
||||
|
||||
This one-cancels-all style is currently the *only* supervision
|
||||
strategy offered (it's the one ``trio`` gives you); more
|
||||
`erlang strategies`_ are roadmap, see the bottom of this page.
|
||||
|
||||
The boxed-error bestiary
|
||||
------------------------
|
||||
|
||||
All remote failures arrive locally as one of a small set of
|
||||
exception types, each carrying enough metadata to work out *who*
|
||||
failed, *where*, and *why*.
|
||||
|
||||
``RemoteActorError``
|
||||
********************
|
||||
|
||||
The workhorse: a "boxed" exception relayed over IPC from another
|
||||
actor. The original error's type, traceback string and msgdata are
|
||||
preserved so you can pattern-match on what actually went wrong
|
||||
remotely,
|
||||
|
||||
- ``.boxed_type``: the reconstructed **type** of the original
|
||||
remote exception (``ValueError``, ``NameError``, what have you),
|
||||
- ``.src_uid``: the ``(name, uuid)`` pair of the actor where the
|
||||
error *originated*,
|
||||
- ``.relay_uid`` / ``.relay_path``: when an error crosses more than
|
||||
one actor boundary (grandchild -> child -> root) every relaying
|
||||
actor is recorded; multi-hop boxings are lovingly referred to as
|
||||
"inceptions" in the runtime internals,
|
||||
- ``.pformat()``: a rich "tb box" rendering of the remote traceback
|
||||
for your logs or REPL.
|
||||
|
||||
.. code:: python
|
||||
|
||||
try:
|
||||
async with portal.open_context(ep) as (ctx, first):
|
||||
...
|
||||
except tractor.RemoteActorError as rae:
|
||||
if rae.boxed_type is ValueError:
|
||||
... # the remote task raised `ValueError`
|
||||
|
||||
``ContextCancelled``
|
||||
********************
|
||||
|
||||
The cancel-ack for a cross-actor task pair: raised when a
|
||||
:class:`tractor.Context` task is cancelled *by request*. Its
|
||||
``.canceller`` attr is the uid of the actor which **requested** the
|
||||
cancel, which powers the key rule,
|
||||
|
||||
- if **you** requested it (you called
|
||||
:meth:`tractor.Context.cancel`) the resulting ctxc is *absorbed*
|
||||
at ``open_context()`` exit: an expected outcome, not an error,
|
||||
- if **anyone else** did — the peer task, or some third-party actor
|
||||
— it *raises* locally so your code always hears about it.
|
||||
|
||||
The full self- vs. cross-cancel semantics are a core teaching point
|
||||
of :doc:`/guide/context`; go read them there.
|
||||
|
||||
``MsgTypeError``
|
||||
****************
|
||||
|
||||
An IPC-payload "type error": a msg violated the dialog's declared
|
||||
payload spec. See :doc:`/guide/msging` for the typed-messaging
|
||||
system which enforces it.
|
||||
|
||||
``TransportClosed``
|
||||
*******************
|
||||
|
||||
The underlying IPC transport (TCP stream, UDS socket, ...) died or
|
||||
closed out from under a channel. You'll normally only see this
|
||||
surface when a peer hard-exits without any graceful runtime
|
||||
teardown; the supervision machinery treats unexpected transport
|
||||
loss on a busy channel as a failure and tears down accordingly.
|
||||
|
||||
Pick your blast radius
|
||||
----------------------
|
||||
|
||||
Three cancel surfaces, three scopes of effect; choose the smallest
|
||||
hammer that does the job.
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 36 34 30
|
||||
|
||||
* - surface
|
||||
- cancels
|
||||
- typical use
|
||||
* - :meth:`tractor.ActorNursery.cancel`
|
||||
- every subactor in the nursery
|
||||
- whole-tree teardown
|
||||
* - :meth:`tractor.Portal.cancel_actor`
|
||||
- one actor: full runtime + proc
|
||||
- daemon teardown
|
||||
* - :meth:`tractor.Context.cancel`
|
||||
- exactly one remote task
|
||||
- surgical task cancel
|
||||
|
||||
``ActorNursery.cancel()``
|
||||
*************************
|
||||
|
||||
The big red button: gracefully cancel every subactor supervised by
|
||||
the nursery, in parallel, with the escalation discipline below
|
||||
applied per-child. It's invoked for you whenever an error hits the
|
||||
nursery block (one-cancels-all); call it yourself for an orderly
|
||||
early shutdown. Passing ``hard_kill=True`` skips the graceful phase
|
||||
and goes straight to OS-level process termination — rarely what you
|
||||
want outside tests.
|
||||
|
||||
``Portal.cancel_actor()``
|
||||
*************************
|
||||
|
||||
Cancel one **whole actor**: its entire runtime, every task it's
|
||||
scheduled, and (for subactors) the OS process, via a graceful
|
||||
runtime-cancel request,
|
||||
|
||||
.. code:: python
|
||||
|
||||
await portal.cancel_actor() # bounded wait, bool result
|
||||
await portal.cancel_actor(
|
||||
raise_on_timeout=True, # no ack in time?
|
||||
) # -> `ActorTooSlowError`
|
||||
|
||||
The wait for the peer's ack is *bounded* (default
|
||||
``Portal.cancel_timeout = 0.5`` seconds, tunable per call via
|
||||
``timeout=``). By default a missed ack just returns ``False``; with
|
||||
``raise_on_timeout=True`` you instead get an ``ActorTooSlowError``
|
||||
(from ``tractor._exceptions``) so *your* code can escalate per SC
|
||||
discipline — exactly what the nursery's own teardown does
|
||||
internally before resorting to OS-level signalling.
|
||||
|
||||
Note the granularity: this cancels an **actor**, not a task. For
|
||||
one remote task use the ``Context`` layer instead.
|
||||
|
||||
``Context.cancel()``
|
||||
********************
|
||||
|
||||
Request cancellation of exactly one remote task: the peer task of
|
||||
an open :class:`tractor.Context`. Two things to keep straight,
|
||||
|
||||
- it cancels the **remote** side only; a ``Context`` is *not* a
|
||||
:class:`trio.CancelScope` and your local task keeps running until
|
||||
you exit the ``open_context()`` block,
|
||||
- the resulting :class:`tractor.ContextCancelled` is absorbed
|
||||
locally (you asked for it, after all) per the self- vs.
|
||||
cross-cancel rule above.
|
||||
|
||||
Again, :doc:`/guide/context` covers this dance in depth.
|
||||
|
||||
Graceful first, hard as a last resort
|
||||
-------------------------------------
|
||||
|
||||
.. margin:: REPL-safe by design
|
||||
|
||||
The hard-kill path is *skipped* whenever an actor in the tree
|
||||
holds the debug-REPL lock (``debug_mode=True`` flavors):
|
||||
SIGTERM raining down on a tree mid-``pdb`` session would
|
||||
clobber your prompt. See :doc:`/guide/debugging`.
|
||||
|
||||
Every process teardown in ``tractor`` walks the same escalation
|
||||
ladder, top rung first,
|
||||
|
||||
1. **graceful cancel request**: a runtime-cancel msg over IPC; the
|
||||
target actor cancels its tasks, closes its channels and exits
|
||||
its :func:`trio.run` cleanly,
|
||||
2. **soft wait**: the parent waits (bounded) for the child process
|
||||
to exit on its own,
|
||||
3. **SIGTERM**: no ack within the bounded wait (internally an
|
||||
``ActorTooSlowError``) escalates to ``proc.terminate()``,
|
||||
4. **SIGKILL ultimatum**: still alive after the hard-kill timeout
|
||||
(~1.6s)? The runtime logs that the "T-800" has been deployed to
|
||||
collect the zombie and issues ``proc.kill()``. No survivors.
|
||||
|
||||
The result is the **no-zombies guarantee**: ``tractor`` tries to
|
||||
protect you from zombies, no matter what. Quoting the project
|
||||
manifesto,
|
||||
|
||||
If you can create zombie child processes (without using
|
||||
a system signal) it **is a bug**.
|
||||
|
||||
Run the quickstart's self-destructing process-tree demo
|
||||
(``examples/parallelism/we_are_processes.py``, walked through in
|
||||
:doc:`/start/quickstart`) under a ``pstree`` watcher and try to
|
||||
catch a
|
||||
straggler; we'll wait B)
|
||||
|
||||
Roadmap: ``erlang``-style strategies
|
||||
------------------------------------
|
||||
|
||||
One-cancels-all is ``trio``'s strategy and, for now, the only one
|
||||
``tractor`` ships. Pluggable `erlang strategies`_ — one-for-one
|
||||
restarts, rest-for-one, transient/permanent child specs and friends
|
||||
(see the `supervision strategies`_ canon) — are a long-standing
|
||||
roadmap item tracked in `#22`_. If supervisors are your jam that
|
||||
issue is the place to sling opinions.
|
||||
|
||||
.. seealso::
|
||||
|
||||
- :doc:`/guide/context` — the cross-actor task layer where
|
||||
per-task cancellation actually lives,
|
||||
- :doc:`/guide/msging` — the typed msg layer that raises
|
||||
:class:`tractor.MsgTypeError`,
|
||||
- :doc:`/guide/debugging` — what cancellation does (and very
|
||||
carefully does *not* do) while a REPL is up.
|
||||
|
||||
.. _cancellation: https://trio.readthedocs.io/en/latest/reference-core.html#cancellation-and-timeouts
|
||||
.. _exactly like trio: https://trio.readthedocs.io/en/latest/reference-core.html#cancellation-semantics
|
||||
.. _always propagate: https://trio.readthedocs.io/en/latest/design.html#exceptions-always-propagate
|
||||
.. _erlang strategies: https://learnyousomeerlang.com/supervisors
|
||||
.. _supervision strategies: https://www.erlang.org/doc/system/sup_princ.html
|
||||
.. _#22: https://github.com/goodboy/tractor/issues/22
|
||||
|
|
@ -1,127 +0,0 @@
|
|||
Higher-level cluster APIs
|
||||
=========================
|
||||
|
||||
Sometimes you don't want a hand-crafted supervision tree; you want
|
||||
"a pile of workers, one per core, now please". For that there's
|
||||
:func:`tractor.open_actor_cluster`: a convenience wrapper which
|
||||
spawns a *flat* cluster of subactors and hands you back a portal to
|
||||
each,
|
||||
|
||||
.. code:: python
|
||||
|
||||
@acm
|
||||
async def open_actor_cluster(
|
||||
modules: list[str], # RPC allowlist for workers
|
||||
count: int = cpu_count(), # one per core by default
|
||||
names: list[str]|None = None, # default: 'worker_{i}'
|
||||
hard_kill: bool = False, # fwd to `an.cancel()`
|
||||
**runtime_kwargs, # fwd to `open_root_actor()`
|
||||
) -> AsyncGenerator[dict[str, tractor.Portal], None]:
|
||||
|
||||
A cluster in one block
|
||||
----------------------
|
||||
|
||||
.. literalinclude:: ../../examples/quick_cluster.py
|
||||
:caption: examples/quick_cluster.py
|
||||
:language: python
|
||||
|
||||
Walkthrough,
|
||||
|
||||
- ``open_actor_cluster(modules=[__name__])`` concurrently spawns
|
||||
one subactor per detected core (per
|
||||
:func:`multiprocessing.cpu_count`); the ``modules`` list is the
|
||||
usual ``enable_modules``-style capability allowlist so workers
|
||||
may run functions defined in this module,
|
||||
- it yields a ``dict[str, tractor.Portal]`` mapping worker name to
|
||||
portal; note the keys get prefixed with the *spawning* actor's
|
||||
name, so from the root you'll see ``'root.worker_0'``,
|
||||
``'root.worker_1'``, etc.,
|
||||
- a plain :class:`trio.Nursery` then fans out one
|
||||
``portal.run(sleepy_jane)`` per worker; each prints its actor
|
||||
``.uid`` from inside its own process then naps forever — what
|
||||
runs *inside* each worker (and how many tasks you point at it)
|
||||
is entirely yours to compose,
|
||||
- ``tractor.trionics.collapse_eg()`` un-nests the strict
|
||||
``ExceptionGroup`` wrapping so the demo's ``KeyboardInterrupt``
|
||||
surfaces as itself instead of arriving eg-boxed,
|
||||
- on block exit the whole fleet is torn down for you via
|
||||
:meth:`tractor.ActorNursery.cancel`; pass ``hard_kill=True`` at
|
||||
open time to skip straight to OS-level termination instead of
|
||||
the graceful ladder described in :doc:`/guide/cancellation`.
|
||||
|
||||
Sizing, naming, fleet-wide options
|
||||
----------------------------------
|
||||
|
||||
``count`` doesn't have to be core-count and the auto-generated
|
||||
``'worker_{i}'`` names are just the default; pass your own (the
|
||||
length must match ``count`` or you get a ``ValueError``). Any
|
||||
extra ``**runtime_kwargs`` pass through verbatim to
|
||||
:func:`tractor.open_root_actor`, so fleet-wide runtime options are
|
||||
one kwarg away,
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with tractor.open_actor_cluster(
|
||||
modules=['mylib.workers'],
|
||||
count=4,
|
||||
names=['scout', 'miner', 'smelter', 'smith'],
|
||||
debug_mode=True, # whole-fleet crash-to-REPL
|
||||
) as portal_map:
|
||||
...
|
||||
|
||||
From here the composition patterns are the usual ``tractor`` fare:
|
||||
``portal.run()`` for one-shot calls (as in the demo), or — for a
|
||||
persistent bidirectional dialog per worker — concurrently enter N
|
||||
``portal.open_context()`` blocks with
|
||||
``tractor.trionics.gather_contexts()``; see :doc:`/guide/context`
|
||||
for that whole layer.
|
||||
|
||||
Clusters vs. nurseries
|
||||
----------------------
|
||||
|
||||
.. d2:: diagrams/actor_tree.d2
|
||||
:margin:
|
||||
:caption: The general shape: arbitrary nesting. A cluster is
|
||||
this, minus the nesting.
|
||||
:alt: a nested supervision tree of subactors
|
||||
|
||||
``open_actor_cluster()`` is sugar, not a new primitive: under the
|
||||
hood it's just :func:`tractor.open_nursery` plus N concurrent
|
||||
``start_actor()`` calls plus a ``.cancel()`` on the way out. Reach
|
||||
for it when,
|
||||
|
||||
- you want a *flat*, homogeneous fleet (classic worker-pool or
|
||||
map-style fan-out shapes),
|
||||
- "one per core" — or a fixed ``count`` — is the right sizing
|
||||
story,
|
||||
- every child can share the same spawn options.
|
||||
|
||||
Drop down to a raw :class:`tractor.ActorNursery` when the topology
|
||||
gets any fancier: nested trees, heterogeneous children, per-child
|
||||
``debug_mode``/transport/module options, daemons mixed with
|
||||
one-shot workers, and so on (see :doc:`/guide/parallelism` for a
|
||||
hand-rolled pool). Either way the supervision semantics are
|
||||
identical: one-cancels-all error propagation and the no-zombies
|
||||
guarantee from :doc:`/guide/cancellation` apply to clusters too.
|
||||
|
||||
Provisional, by design
|
||||
----------------------
|
||||
|
||||
.. note::
|
||||
|
||||
APIs in this section are considered **provisional**: the
|
||||
signature and semantics of :func:`tractor.open_actor_cluster`
|
||||
may shift as higher-level supervision machinery lands. We
|
||||
encourage you to try it and provide feedback — the
|
||||
`matrix channel`_ is the place to say hi, and `#22`_ tracks the
|
||||
broader supervisor-strategy roadmap.
|
||||
|
||||
.. seealso::
|
||||
|
||||
- :doc:`/guide/parallelism` — worker pools built "by hand" with
|
||||
plain actor nurseries (and why that's easy peasy),
|
||||
- :doc:`/guide/cancellation` — the teardown machinery a cluster
|
||||
inherits for free.
|
||||
|
||||
.. _matrix channel: https://matrix.to/#/!tractor:matrix.org
|
||||
.. _#22: https://github.com/goodboy/tractor/issues/22
|
||||
|
|
@ -1,344 +0,0 @@
|
|||
The ``Context``: a cross-actor task pair
|
||||
=========================================
|
||||
|
||||
If you've written any trio_ you already know the contract: every
|
||||
task lives in a nursery, errors always propagate, cancellation is
|
||||
scoped, and nothing leaks. ``tractor`` extends that exact contract
|
||||
*across processes* — the same guarantees from the seminal
|
||||
`blog post`_, just with the nursery split across two memory
|
||||
domains. The primitive that does it is :class:`tractor.Context`: a
|
||||
**linked pair of tasks**, one in each of two actors, supervised as
|
||||
a single `structured concurrency`_ (SC) scope over IPC.
|
||||
|
||||
.. d2:: diagrams/context_handshake.d2
|
||||
:caption: The SC-transitive supervision protocol, msg by msg.
|
||||
:alt: sequence diagram of the context handshake msg flow
|
||||
|
||||
Pretty much everything else is (or is slated to be) built on this
|
||||
one primitive: ``ActorNursery.run_in_actor()`` is a convenience
|
||||
for "spawn, open a context, await the result, tear down"; plain
|
||||
``Portal.run()`` RPC is planned to be re-implemented on top of it;
|
||||
the multi-process debugger's tree-wide REPL lock rides one. Grok
|
||||
this page and the rest of the library reads as convenience
|
||||
wrappers B)
|
||||
|
||||
The endpoint contract
|
||||
---------------------
|
||||
|
||||
A context endpoint is an async function decorated with
|
||||
:func:`tractor.context` which declares **a param annotated**
|
||||
``tractor.Context`` — any param name you like, the annotation is
|
||||
what's required:
|
||||
|
||||
.. code:: python
|
||||
|
||||
@tractor.context
|
||||
async def trainer(
|
||||
ctx: tractor.Context,
|
||||
model: str,
|
||||
) -> str:
|
||||
await ctx.started('ready')
|
||||
return f'trained {model}'
|
||||
|
||||
.. margin:: Who am I talking to?
|
||||
|
||||
Inside any context task
|
||||
:func:`tractor.current_ipc_ctx` returns the
|
||||
``Context`` bound to the current task; handy
|
||||
in helpers that don't take ``ctx`` explicitly.
|
||||
|
||||
The parent (aka "opener") side invokes it through a
|
||||
:class:`tractor.Portal` using ``Portal.open_context()``, passing
|
||||
any extra kwargs which are shipped over the wire as the remote
|
||||
task's arguments. Since the target fn is referenced by module
|
||||
path, that module must be listed in the peer actor's
|
||||
``enable_modules`` allowlist — RPC capability is always opt-in.
|
||||
|
||||
The decorator also accepts a ``pld_spec``: a type (union) which
|
||||
every payload in the dialog is validated against, upgrading your
|
||||
msgs to a typed contract enforced via :exc:`tractor.MsgTypeError`.
|
||||
Validation strictness follows the "`cheap or nasty`_"
|
||||
`(un)protocol`_ pattern: the one-shot ``Started`` payload gets the
|
||||
nasty treatment (stringently round-trip checked before it's even
|
||||
sent) while high-rate stream payloads stay cheap (checked only
|
||||
receiver side).
|
||||
|
||||
The handshake, on the wire
|
||||
--------------------------
|
||||
|
||||
Every context runs one instance of ``tractor``'s "SC-transitive
|
||||
supervision protocol": a tiny fixed grammar of msgspec_-typed msgs
|
||||
encapsulating *all* RPC dialogs between actors. *Transitive*
|
||||
because each IPC link obeys the same rules a local nursery does —
|
||||
starts are acked, completion is awaited, errors and cancels always
|
||||
relay — so chaining links across a process tree composes into one
|
||||
tree-wide SC scope.
|
||||
|
||||
The figure up top shows a full dialog; in order:
|
||||
|
||||
``Start``
|
||||
sent by ``Portal.open_context()``: "schedule a task running
|
||||
this function with these kwargs".
|
||||
|
||||
``StartAck``
|
||||
the peer runtime confirms the task is scheduled and that the
|
||||
endpoint really is a context-style fn.
|
||||
|
||||
``Started``
|
||||
emitted when the child task calls
|
||||
:meth:`tractor.Context.started`; carries the first payload
|
||||
and unblocks the parent's entry of ``open_context()``.
|
||||
|
||||
``Yield``
|
||||
one per :meth:`tractor.MsgStream.send`, flowing in *either*
|
||||
direction while a stream is open.
|
||||
|
||||
``Stop``
|
||||
graceful end-of-stream: the far side's ``async for``
|
||||
terminates cleanly.
|
||||
|
||||
``Return``
|
||||
the child fn returned; its value becomes the context's final
|
||||
result. If the child raised instead, an ``Error`` msg takes
|
||||
this slot carrying the boxed traceback.
|
||||
|
||||
``ctx.started()``: just like ``task_status.started()``
|
||||
*******************************************************
|
||||
|
||||
The startup phase is a deliberate clone of
|
||||
:meth:`trio.Nursery.start` semantics: the child decides when it's
|
||||
"up", optionally handing back a first value, and the parent stays
|
||||
blocked until that moment:
|
||||
|
||||
.. code:: python
|
||||
|
||||
# trio, in-process
|
||||
first = await nursery.start(child_fn)
|
||||
|
||||
# tractor, cross-process
|
||||
async with portal.open_context(child_fn) as (ctx, first):
|
||||
...
|
||||
|
||||
The ``as (ctx, first)`` tuple is exactly that pair: the
|
||||
:class:`tractor.Context` handle plus whatever value the child
|
||||
passed to ``await ctx.started(value)``. And readiness is not
|
||||
optional — for instance opening a stream before ``.started()``
|
||||
has been called raises a ``RuntimeError``; handshake first, then
|
||||
dialog.
|
||||
|
||||
Bidirectional streaming over a context
|
||||
--------------------------------------
|
||||
|
||||
The canonical ping-pong (design history: `#53`_, `#223`_) — a
|
||||
full-duplex msg stream between a parent and its spawned peer:
|
||||
|
||||
.. literalinclude:: ../../examples/rpc_bidir_streaming.py
|
||||
:caption: examples/rpc_bidir_streaming.py
|
||||
:language: python
|
||||
|
||||
What's going on?
|
||||
|
||||
- ``start_actor()`` spawns the daemon-style subactor
|
||||
``'rpc_server'`` with this very module in its allowlist.
|
||||
|
||||
- ``portal.open_context(simple_rpc, data=10)`` fires the
|
||||
``Start`` msg then blocks until the child task calls
|
||||
``await ctx.started(data + 1)`` — hence ``sent == 11``.
|
||||
|
||||
- both tasks enter ``ctx.open_stream()``: a stream dialog is only
|
||||
fully open once *each* side has entered its block.
|
||||
|
||||
- the parent seeds the first ``'ping'``; each side then echoes
|
||||
the other, one ``Yield`` msg per ``stream.send()``.
|
||||
|
||||
- after the 9th pong the parent ``break``\ s (10 pings sent in
|
||||
total) and exits its stream block, which sends ``Stop``; the
|
||||
child's ``async for`` completes gracefully and its ``else``
|
||||
clause asserts all 10 pings arrived.
|
||||
|
||||
- the 10th in-transit pong? Discarded by the implicit drain at
|
||||
``open_context()`` exit, which runs the dialog down to the
|
||||
child's ``Return`` (here ``None``).
|
||||
|
||||
- daemon actors live until told otherwise:
|
||||
``portal.cancel_actor()`` reaps the subactor explicitly.
|
||||
|
||||
Results: the ``Return`` leg
|
||||
---------------------------
|
||||
|
||||
Every context resolves to a final outcome. Wait on it explicitly
|
||||
from the parent side:
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with portal.open_context(ep) as (ctx, first):
|
||||
...
|
||||
result = await ctx.wait_for_result()
|
||||
|
||||
or just exit the block — ``__aexit__`` implicitly drains the msg
|
||||
flow until the ``Return`` (or ``Error``) arrives, discarding any
|
||||
in-transit ``Yield``\ s on the way. Either way the rule of
|
||||
`causality`_ holds exactly as in a local nursery: **the opener
|
||||
never unblocks before the remote task is done**.
|
||||
|
||||
For post-hoc inspection (think supervision/restart logic) the ctx
|
||||
also exposes ``Context.outcome``, ``.maybe_error`` and
|
||||
``.has_outcome`` — where a "result" might well be the error the
|
||||
dialog ended with.
|
||||
|
||||
Cancellation semantics
|
||||
----------------------
|
||||
|
||||
The part you actually came for; read it twice B)
|
||||
|
||||
A context's two tasks are **cancel-scope-linked across the IPC
|
||||
boundary**: whatever ends one side — error, cancellation, plain
|
||||
old return — is relayed such that the other side ends
|
||||
equivalently. No silent half-open dialogs, no orphaned remote
|
||||
tasks, ever.
|
||||
|
||||
``ctx.cancel()`` cancels the *remote* task
|
||||
*******************************************
|
||||
|
||||
:meth:`tractor.Context.cancel` requests cancellation of the
|
||||
**remote** task only:
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with portal.open_context(ep) as (ctx, first):
|
||||
await accomplish_things(ctx)
|
||||
await ctx.cancel() # remote task, NOT me
|
||||
|
||||
A :class:`tractor.Context` is **not** a :class:`trio.CancelScope`:
|
||||
the call doesn't (and can't) cancel your local task. It sends the
|
||||
cancel request and waits a bounded ``timeout`` for the peer
|
||||
runtime's ``CancelAck``, then your code proceeds to the block exit
|
||||
as normal.
|
||||
|
||||
Compare scopes here: ``Portal.cancel_actor()`` is the big hammer
|
||||
which cancels the peer's **entire runtime** (and thus process);
|
||||
``ctx.cancel()`` is the per-dialog scalpel.
|
||||
|
||||
``ContextCancelled`` and the absorption rule
|
||||
*********************************************
|
||||
|
||||
When a context task gets cancelled *by request* the requestee's
|
||||
runtime reports back with a :exc:`tractor.ContextCancelled`
|
||||
("ctxc") whose ``.canceller`` field holds the uid of the actor
|
||||
which asked. That one field decides what you observe:
|
||||
|
||||
**you requested it**
|
||||
i.e. ``ctxc.canceller == tractor.current_actor().uid``: the
|
||||
ctxc is **absorbed** at ``open_context()`` exit — nothing
|
||||
raises in your block. You asked for a graceful stop and got
|
||||
it; if you care, ``await ctx.wait_for_result()`` hands the
|
||||
ctxc back as a plain *value* for inspection.
|
||||
|
||||
**anyone else requested it**
|
||||
the peer cancelling itself, or some third actor cancelling it
|
||||
from the side: the ctxc **is raised** in your block. From
|
||||
your scope's perspective a task you depend on was killed out
|
||||
from under you and SC demands you hear about it — exactly
|
||||
like a sibling crash in a `nursery`_.
|
||||
|
||||
In code:
|
||||
|
||||
.. code:: python
|
||||
|
||||
try:
|
||||
async with portal.open_context(ep) as (ctx, first):
|
||||
...
|
||||
except tractor.ContextCancelled as ctxc:
|
||||
# can only be a peer- or third-party cancel;
|
||||
# self-requested cancels are absorbed at exit.
|
||||
assert ctxc.canceller != tractor.current_actor().uid
|
||||
|
||||
This self- vs cross-cancel split is what makes explicit teardown
|
||||
*composable*: a supervisor cancels its dialogs without try/except
|
||||
noise, while unexpected cancellation anywhere in the tree still
|
||||
propagates loudly like any other failure.
|
||||
|
||||
.. warning::
|
||||
|
||||
Once ``ctx.cancel()`` has been called the dialog is done: a
|
||||
subsequent ``ctx.open_stream()`` raises ``RuntimeError``.
|
||||
|
||||
For introspection the ctx exposes trio-flavored status props:
|
||||
``.cancel_called`` (this side requested), ``.cancel_acked`` (peer
|
||||
confirmed), ``.cancelled_caught`` and ``.canceller`` —
|
||||
deliberately mirroring :class:`trio.CancelScope` naming.
|
||||
|
||||
Errors propagate, both ways
|
||||
---------------------------
|
||||
|
||||
A crash on either end tears down the pair, SC style:
|
||||
|
||||
- **child raises**: the exception ships back as an ``Error`` msg
|
||||
and re-raises in the parent block boxed as a
|
||||
:exc:`tractor.RemoteActorError`; the original class rides along
|
||||
as ``.boxed_type`` with ``.src_uid`` naming the crashed actor.
|
||||
|
||||
- **parent raises** (or is cancelled) inside the block: an
|
||||
equivalent error/cancel is relayed to the child task so it can
|
||||
never outlive the dialog.
|
||||
|
||||
.. code:: python
|
||||
|
||||
try:
|
||||
async with portal.open_context(ep) as (ctx, first):
|
||||
...
|
||||
except tractor.RemoteActorError as rae:
|
||||
if rae.boxed_type is ValueError:
|
||||
... # remote ValueError, type preserved
|
||||
|
||||
Errors that hop through intermediary actors on their way up the
|
||||
tree ("inceptions" XD) keep the full relay trail in
|
||||
``.relay_uid`` / ``.relay_path``. Payloads violating your declared
|
||||
``pld_spec`` surface as the IPC analog of a ``TypeError``:
|
||||
:exc:`tractor.MsgTypeError`.
|
||||
|
||||
Overruns and backpressure
|
||||
-------------------------
|
||||
|
||||
Stream msgs land in a bounded per-context buffer on the receiver
|
||||
side. A sender that outpaces a non-consuming receiver *overruns*
|
||||
it and the runtime raises ``StreamOverrun`` (from
|
||||
``tractor._exceptions``; also a :exc:`trio.TooSlowError`) instead
|
||||
of buffering without bound — SC discipline applies to memory too.
|
||||
|
||||
Your knobs:
|
||||
|
||||
- ``msg_buffer_size`` on ``ctx.open_stream()`` sizes the buffer.
|
||||
|
||||
- ``allow_overruns=True`` (on ``Portal.open_context()`` and/or
|
||||
``ctx.open_stream()``) opts in to absorbing overflow instead of
|
||||
erroring — reasonable for bursty telemetry-ish feeds, just know
|
||||
you're trading the error for extra buffering.
|
||||
|
||||
One context, one stream
|
||||
-----------------------
|
||||
|
||||
A ``MsgStream`` is strictly **one-shot use**: once it closes —
|
||||
gracefully or not, from either side — it can never be re-opened
|
||||
on the same ctx. Want another round with the same peer? Open a
|
||||
fresh context; they're cheap. The full close-vs-cancel teardown
|
||||
story lives in :doc:`/guide/streaming`.
|
||||
|
||||
.. rubric:: Where to next?
|
||||
|
||||
:doc:`/guide/streaming` covers the rest of the msg-moving story:
|
||||
the legacy one-way API, multi-actor pipelines and in-actor
|
||||
broadcast fan-out. For exhaustive API detail see
|
||||
:class:`tractor.Context`, :class:`tractor.MsgStream` and
|
||||
:exc:`tractor.ContextCancelled`.
|
||||
|
||||
.. _trio: https://github.com/python-trio/trio
|
||||
.. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||
.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
|
||||
.. _nursery: https://trio.readthedocs.io/en/latest/reference-core.html#nurseries-and-spawning
|
||||
.. _causality: https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#c-c-c-c-causality-breaker
|
||||
.. _cheap or nasty: https://zguide.zeromq.org/docs/chapter7/#The-Cheap-or-Nasty-Pattern
|
||||
.. _(un)protocol: https://zguide.zeromq.org/docs/chapter7/#Unprotocols
|
||||
.. _msgspec: https://jcristharif.com/msgspec/
|
||||
.. _#53: https://github.com/goodboy/tractor/issues/53
|
||||
.. _#223: https://github.com/goodboy/tractor/issues/223
|
||||
|
|
@ -1,545 +0,0 @@
|
|||
.. _debugging:
|
||||
|
||||
================================
|
||||
"Native" multi-process debugging
|
||||
================================
|
||||
|
||||
``tractor`` ships the thing every ``multiprocessing`` user has
|
||||
wished for and quietly assumed was impossible: a multi-process
|
||||
debugger that *just works*.
|
||||
|
||||
Drop ``await tractor.pause()`` — or, with `greenback`_ installed,
|
||||
a plain builtin ``breakpoint()`` — anywhere in any actor: the
|
||||
root, a child, a grandchild, a sync helper function, even an
|
||||
``asyncio`` task inside an "infected" actor. A full-featured
|
||||
`pdbp`_ REPL opens *in that process*, with syntax-highlighted
|
||||
source listings, tab completion and sticky mode, attached to your
|
||||
one terminal.
|
||||
|
||||
Under the hood every REPL entry acquires a tree-global tty mutex
|
||||
via an IPC request to the root actor, so prompts from concurrent
|
||||
pauses and crashes never interleave. ``ctrl-c`` is shielded while
|
||||
any REPL is live, so a stray ``SIGINT`` can't vaporize the tree
|
||||
out from under you. And in debug mode any uncaught error drops
|
||||
you into a crash REPL *first in the failing child*, then again at
|
||||
each parent as the boxed :class:`~tractor.RemoteActorError` climbs
|
||||
the supervision tree.
|
||||
|
||||
No remote-pdb sockets, no ``set_trace()`` port juggling, no
|
||||
``ptrace`` attach dance: the debugger semantics you already know,
|
||||
transparently extended across an entire process tree. Because
|
||||
``tractor`` is a `structured concurrency`_ (SC) runtime, the
|
||||
debugger composes with supervision instead of fighting it — quit
|
||||
a REPL and errors keep propagating exactly like `trio`_ taught
|
||||
you, ending in clean, zombie-free teardown.
|
||||
|
||||
We're pretty sure it's the (first ever?) "native" debugging UX
|
||||
for multi-process Python B)
|
||||
|
||||
Enabling debug mode
|
||||
-------------------
|
||||
|
||||
Pass ``debug_mode=True`` to your runtime entrypoint, either
|
||||
:func:`tractor.open_nursery` (which forwards it to the implicitly
|
||||
opened root actor) or :func:`tractor.open_root_actor` directly:
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=True, # arm the whole actor tree
|
||||
) as an:
|
||||
...
|
||||
|
||||
This arms the debug machinery *tree-wide*:
|
||||
|
||||
- crash handling is enabled in every actor: uncaught errors enter
|
||||
a REPL before they propagate,
|
||||
- the internal tty-lock module is auto-exposed over RPC to every
|
||||
subactor (this is what makes the one-terminal handoff work),
|
||||
- console logging is bumped to include ``PDB``-level status msgs
|
||||
so you can see REPL acquire/release events as they happen.
|
||||
|
||||
You can instead flip it on for just one child, letting its
|
||||
siblings crash-and-burn the normal way:
|
||||
|
||||
.. code:: python
|
||||
|
||||
portal = await an.start_actor(
|
||||
'sketchy_worker',
|
||||
debug_mode=True, # OR-ed with the tree-wide flag
|
||||
)
|
||||
|
||||
See ``examples/debugging/per_actor_debug.py`` for a runnable
|
||||
proof of the selective style.
|
||||
|
||||
.. note::
|
||||
|
||||
Debug mode requires the child-side runtime to be
|
||||
``trio``-native so that the tty-lock IPC dialog works; it's
|
||||
currently supported on the ``'trio'`` (default) and
|
||||
``'main_thread_forkserver'`` spawn backends and raises
|
||||
``RuntimeError`` for any other ``start_method``.
|
||||
|
||||
Your first pause point
|
||||
----------------------
|
||||
|
||||
:func:`tractor.pause` is the SC-aware, multi-process spelling of
|
||||
the stdlib's ``breakpoint()``. In the root actor it looks almost
|
||||
boring:
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/root_actor_breakpoint.py
|
||||
:caption: examples/debugging/root_actor_breakpoint.py
|
||||
:language: python
|
||||
|
||||
Run it and you get a ``(Pdb+)`` prompt parked on the ``pause()``
|
||||
line; type ``c`` (continue) and the program finishes normally.
|
||||
|
||||
The exact same call works from *any* subactor, no matter how deep
|
||||
in the tree:
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/subactor_breakpoint.py
|
||||
:caption: examples/debugging/subactor_breakpoint.py
|
||||
:language: python
|
||||
|
||||
Each loop iteration the child actor requests the terminal from
|
||||
the root over IPC, REPLs you, then releases it on ``c``. Pause
|
||||
points are re-entrant-safe: repeat calls from the same task are
|
||||
no-op'd and other local tasks queue politely for the REPL.
|
||||
|
||||
When you get bored, type ``q`` (quit): the resulting
|
||||
``bdb.BdbQuit`` is boxed and shipped to the parent like any other
|
||||
remote error XD — causality is preserved even for your debugging
|
||||
mistakes.
|
||||
|
||||
Crash REPLs: errors climb the tree
|
||||
----------------------------------
|
||||
|
||||
Pause points are only half the story. With debug mode armed, any
|
||||
*uncaught* error anywhere in the tree triggers what we call crash
|
||||
handling mode:
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/subactor_error.py
|
||||
:caption: examples/debugging/subactor_error.py
|
||||
:language: python
|
||||
|
||||
What happens when the child hits that (very intentional)
|
||||
``NameError``:
|
||||
|
||||
1. a REPL opens **in the crashed child first** — you inspect the
|
||||
raising frame, its locals, the works, right inside the failed
|
||||
process,
|
||||
2. when you quit, the error is boxed into a
|
||||
:class:`~tractor.RemoteActorError` and relayed to the parent,
|
||||
3. the parent (here the root) gets *its own* crash REPL with the
|
||||
rendered remote traceback,
|
||||
4. quit again and the nursery tears the tree down — errors keep
|
||||
propagating per SC rules, no zombies left behind.
|
||||
|
||||
You debug the failure at every hop of the supervision tree, which
|
||||
for multi-hop trees means you can chase an error from the leaf
|
||||
that raised it all the way up to the root that supervises it.
|
||||
|
||||
Need to skip REPL entry for certain exceptions? Pass a predicate
|
||||
via ``open_root_actor(debug_filter=...)``; by default
|
||||
cancellation-only exception (groups) don't engage the REPL.
|
||||
|
||||
One terminal, many actors
|
||||
-------------------------
|
||||
|
||||
So how do N processes share one tty without garbling it? The root
|
||||
actor owns stdio for the whole tree and guards it with a FIFO
|
||||
mutex; every subactor REPL entry is an IPC lock request to the
|
||||
root. Exactly one actor-task in the entire tree can own the
|
||||
terminal at a time, so prompts never interleave — ever.
|
||||
|
||||
.. d2:: diagrams/debug_lock.d2
|
||||
:caption: Every REPL entry serializes through the root actor's
|
||||
tty lock; ``continue``-ing one REPL hands the terminal to
|
||||
the next waiter, FIFO style.
|
||||
:alt: sequence diagram of two subactors serializing pdb REPL
|
||||
access through the root actor's tty lock
|
||||
|
||||
The runtime's teardown paths cooperate too: a cancelling parent
|
||||
always waits for any live REPL to release before reaping
|
||||
children, so the debugger never gets yanked out from under you
|
||||
mid-keystroke.
|
||||
|
||||
.. margin:: Watch the tree live
|
||||
|
||||
Run any of these examples with a process-tree watcher in a
|
||||
second terminal and watch actors come and go::
|
||||
|
||||
watch -n 0.1 "pstree -a $$"
|
||||
|
||||
Here's the showpiece: one daemon child re-entering
|
||||
``tractor.pause()`` forever inside a stream, while its sibling
|
||||
repeatedly raises a ``NameError``:
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/multi_daemon_subactors.py
|
||||
:caption: examples/debugging/multi_daemon_subactors.py
|
||||
:language: python
|
||||
|
||||
What you'll actually see
|
||||
************************
|
||||
|
||||
Running it looks *roughly* like this (uids, tracebacks and source
|
||||
listings elided; REPL order can vary with who wins the lock
|
||||
race)::
|
||||
|
||||
$ python examples/debugging/multi_daemon_subactors.py
|
||||
|
||||
Opening a pdb REPL in paused actor: ('bp_forever', '<uuid>')
|
||||
<highlighted source around the `await tractor.pause()` line>
|
||||
(Pdb+) c
|
||||
|
||||
Opening a pdb REPL in crashed actor: ('name_error', '<uuid>')
|
||||
<live traceback: NameError: name 'doggypants' is not defined>
|
||||
(Pdb+) q
|
||||
|
||||
Opening a pdb REPL in crashed actor: ('root', '<uuid>')
|
||||
<boxed RemoteActorError traceback relayed from 'name_error'>
|
||||
(Pdb+) q
|
||||
|
||||
Two (then three) processes, one terminal, zero confusion:
|
||||
``c``-ing out of the paused daemon's REPL releases the tty lock,
|
||||
which immediately hands the prompt to the crashed sibling; quit
|
||||
that and the error propagates as a fully-rendered
|
||||
:class:`~tractor.RemoteActorError` to the parent where one final
|
||||
crash REPL catches it before clean, zombie-free teardown.
|
||||
|
||||
For maximum drama run
|
||||
``multi_nested_subactors_error_up_through_nurseries.py`` (under
|
||||
``examples/debugging/``) which pulls the same trick across a
|
||||
*three-deep* process tree — the tty lock keeps every prompt
|
||||
orderly the whole way up.
|
||||
|
||||
Post-mortem, on demand
|
||||
----------------------
|
||||
|
||||
Crash handling is automatic, but you can also enter a REPL on
|
||||
a live exception *manually* with :func:`tractor.post_mortem` —
|
||||
the actor-aware equivalent of ``pdb.post_mortem()`` — from inside
|
||||
any ``except`` block in any actor (kwargs: ``tb=`` for an
|
||||
explicit traceback, plus ``shield=`` and ``hide_tb=``):
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/pm_in_subactor.py
|
||||
:caption: examples/debugging/pm_in_subactor.py
|
||||
:language: python
|
||||
|
||||
This example demos three REPL entries from one error:
|
||||
|
||||
- the child's manual ``post_mortem()`` inside its ``except``,
|
||||
- the runtime's automatic crash handler in the same child once
|
||||
the error re-raises out of the RPC task,
|
||||
- a manual ``post_mortem()`` in the parent on the received
|
||||
:class:`~tractor.RemoteActorError`, whose ``.boxed_type``
|
||||
faithfully reports the original ``NameError``.
|
||||
|
||||
Pausing from sync code
|
||||
----------------------
|
||||
|
||||
No ``await``? No problem. :func:`tractor.pause_from_sync` brings
|
||||
the same tree-aware REPL to plain synchronous functions — handy
|
||||
when the suspect code is three helpers deep and decidedly not
|
||||
async.
|
||||
|
||||
It's powered by `greenback`_, which is optional, so you need to:
|
||||
|
||||
1. install it (it ships in ``tractor``'s ``sync_pause``
|
||||
dependency group),
|
||||
2. enable it at runtime entry:
|
||||
|
||||
.. code:: python
|
||||
|
||||
async with tractor.open_nursery(
|
||||
debug_mode=True,
|
||||
maybe_enable_greenback=True,
|
||||
) as an:
|
||||
...
|
||||
|
||||
With that armed, sync code can pause from three different caller
|
||||
environments: the main ``trio`` thread, ``trio.to_thread`` bg
|
||||
threads, and (see the next section) ``asyncio`` tasks in infected
|
||||
actors. The greenback "portal" hops back into the ``trio`` loop
|
||||
to do the lock/REPL dance on your behalf:
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/sync_bp.py
|
||||
:caption: examples/debugging/sync_bp.py (the sync fn, excerpt)
|
||||
:language: python
|
||||
:pyobject: sync_pause
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/sync_bp.py
|
||||
:caption: examples/debugging/sync_bp.py (called in a subactor,
|
||||
excerpt)
|
||||
:language: python
|
||||
:pyobject: start_n_sync_pause
|
||||
|
||||
The full script also exercises the hairier root-actor bg-thread
|
||||
cases (and documents their remaining sharp edges) if you want the
|
||||
deep lore.
|
||||
|
||||
The builtin ``breakpoint()`` override
|
||||
*************************************
|
||||
|
||||
When debug mode boots with greenback available, ``tractor`` wires
|
||||
Python's `PEP 553`_ hook so the *builtin* ``breakpoint()`` becomes
|
||||
the actor-aware sync pause, by exporting::
|
||||
|
||||
PYTHONBREAKPOINT=tractor.devx.debug._sync_pause_from_builtin
|
||||
|
||||
That means third-party and legacy code containing bare
|
||||
``breakpoint()`` calls debugs correctly inside your actor tree
|
||||
with zero edits (the override even forwards kwargs like
|
||||
``hide_tb`` to the underlying pause machinery, as shown in the
|
||||
excerpt above).
|
||||
|
||||
.. warning::
|
||||
|
||||
Without greenback (or with ``maybe_enable_greenback=False``,
|
||||
the default), ``debug_mode=True`` instead *blocks* the builtin
|
||||
``breakpoint()``: ``sys.breakpointhook`` is swapped for a
|
||||
raiser and ``PYTHONBREAKPOINT=0`` is set. A naive
|
||||
``breakpoint()`` from some random process would clobber the
|
||||
shared tty, so we'd rather hand you a loud ``RuntimeError``
|
||||
with install instructions.
|
||||
|
||||
Both the hook and the env var are restored to their prior values
|
||||
on runtime exit — see
|
||||
``examples/debugging/restore_builtin_breakpoint.py`` for the
|
||||
proof.
|
||||
|
||||
Breakpoints inside ``asyncio`` tasks
|
||||
------------------------------------
|
||||
|
||||
Yes, even "infected ``asyncio``" actors get the goods. Spawn a
|
||||
child with ``infect_asyncio=True`` (``trio`` runs as a guest on
|
||||
the ``asyncio`` loop inside it) and, with debug mode + greenback
|
||||
armed, every ``asyncio`` task started via ``tractor.to_asyncio``
|
||||
is automatically granted a greenback portal — so a plain builtin
|
||||
``breakpoint()`` (or ``tractor.pause_from_sync()``) inside an
|
||||
``asyncio.Task`` joins the same single-terminal, tree-locked REPL
|
||||
flow:
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/asyncio_bp.py
|
||||
:caption: examples/debugging/asyncio_bp.py
|
||||
:language: python
|
||||
|
||||
Note the interleave: a ``breakpoint()`` on the ``asyncio`` side,
|
||||
``tractor.pause()`` on the ``trio`` side of the same actor, and
|
||||
another pause up in the root — all serialized through the one tty
|
||||
lock with no cross-actor (or cross-event-loop!) clobbering.
|
||||
|
||||
One catch: ``asyncio`` tasks spawned *out-of-band* — i.e. not via
|
||||
``tractor.to_asyncio``, typically by some third-party aio lib —
|
||||
have no portal bestowed, so a sync pause from one raises a loud
|
||||
``RuntimeError`` telling you to ``greenback.ensure_portal()``
|
||||
first. See :ref:`the caveats <debugging-caveats>` below.
|
||||
|
||||
Teardown debugging: the shielded pause
|
||||
--------------------------------------
|
||||
|
||||
`Cancellation`_ is ``trio``'s bread and butter, which raises an
|
||||
awkward question: how do you REPL inside an *already-cancelled*
|
||||
scope, say while debugging some teardown sequence? A bare
|
||||
``pause()`` would itself be cancelled at its next checkpoint.
|
||||
|
||||
The answer is ``await tractor.pause(shield=True)``, which wraps
|
||||
the lock acquisition and REPL session in a shielded cancel scope
|
||||
(``post_mortem(shield=True)`` works the same way):
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/shielded_pause.py
|
||||
:caption: examples/debugging/shielded_pause.py
|
||||
:language: python
|
||||
|
||||
If you forget, ``tractor`` has your back: an unshielded
|
||||
``pause()`` from a cancelled scope fails fast with a hint
|
||||
suggesting ``await tractor.pause(shield=True)`` instead of
|
||||
silently never REPL-ing.
|
||||
|
||||
Go ahead, mash ctrl-c
|
||||
---------------------
|
||||
|
||||
While any REPL is live the runtime installs a custom ``SIGINT``
|
||||
handler tree-wide so that a reflexive ``ctrl-c`` (or five) can't
|
||||
nuke your debug session:
|
||||
|
||||
- the actor that owns the REPL ignores the interrupt and simply
|
||||
re-flushes the prompt — keep mashing, it's fine,
|
||||
- the root actor ignores ``SIGINT`` while a still-IPC-connected
|
||||
child holds the tty lock, so the supervisor won't tear down the
|
||||
tree out from under the debugger,
|
||||
- if the lock state has gone *stale* — the locking child died or
|
||||
its IPC channel dropped — the root cancels the stale lock scope
|
||||
and restores ``trio``'s default handler, so ``ctrl-c`` works
|
||||
again exactly when it should.
|
||||
|
||||
The handler is uninstalled and ``trio``'s own ``SIGINT``
|
||||
semantics restored every time a REPL releases (on ``continue`` /
|
||||
``quit``).
|
||||
|
||||
Live task-tree dumps
|
||||
--------------------
|
||||
|
||||
Sometimes there's no error to catch — the tree is just *hung* and
|
||||
you want to know where. For that ``tractor`` integrates
|
||||
`stackscope`_: send a signal, get a full ``trio`` task-tree dump
|
||||
from every actor in the tree.
|
||||
|
||||
Enable it any of three ways:
|
||||
|
||||
- ``open_root_actor(enable_stack_on_sig=True)`` (or via
|
||||
``open_nursery()`` which forwards it),
|
||||
- set ``TRACTOR_ENABLE_STACKSCOPE=1`` in the env — it's inherited
|
||||
through the process tree so every (sub)actor arms the handler
|
||||
at boot,
|
||||
- call ``tractor.devx.enable_stack_on_sig()`` directly.
|
||||
|
||||
It's intentionally *not* gated on ``debug_mode`` so you can leave
|
||||
it armed in plain runs. Then, when the hang strikes, signal the
|
||||
tree with ``SIGUSR1``.
|
||||
|
||||
.. tip::
|
||||
|
||||
No need to hunt down pids — pattern-match the original cmdline
|
||||
with ``pkill``::
|
||||
|
||||
$ pkill --signal SIGUSR1 -f "python example_script.py"
|
||||
|
||||
Each actor dumps its entire ``trio`` task tree (full nursery
|
||||
recursion via ``stackscope.extract()``) to its tty *and* tees it
|
||||
to ``/tmp/tractor-stackscope-<pid>.log`` — so the trace survives
|
||||
even under captured-stdio harnesses — then relays the signal on
|
||||
to its children, parent-before-child, until the whole tree has
|
||||
reported in.
|
||||
|
||||
Try it yourself with the demo script, which deliberately hangs a
|
||||
subactor in a shielded sleep:
|
||||
|
||||
.. literalinclude:: ../../examples/debugging/shield_hang_in_sub.py
|
||||
:caption: examples/debugging/shield_hang_in_sub.py
|
||||
:language: python
|
||||
|
||||
(That ``trio.CancelScope(shield=True)`` hang also shows off the
|
||||
zombie reaper: ``ctrl-c`` the root and the un-cancellable child
|
||||
still gets hard-reaped — if you can create a zombie it **is a
|
||||
bug**.)
|
||||
|
||||
Crash handling for sync and CLI code
|
||||
------------------------------------
|
||||
|
||||
All of the above rides on the actor runtime, but crashes don't
|
||||
politely wait for ``trio.run()``. For plain sync code — think
|
||||
``typer``/``click`` CLI endpoints, config parsing, anything
|
||||
pre-runtime — there's a sync context manager that wraps the same
|
||||
``pdbp`` post-mortem UX:
|
||||
|
||||
.. code:: python
|
||||
|
||||
from tractor.devx import open_crash_handler
|
||||
|
||||
def main(): # any sync code, no runtime required
|
||||
with open_crash_handler() as boxed:
|
||||
run_my_cli_thing()
|
||||
|
||||
By default any ``BaseException`` (minus an ``ignore`` set
|
||||
defaulting to ``KeyboardInterrupt`` and ``trio.Cancelled``)
|
||||
enters the REPL then re-raises on exit; pass
|
||||
``raise_on_exit=False`` to suppress instead and introspect the
|
||||
``boxed.value`` afterward. The ``catch``/``ignore`` sets and a
|
||||
``repl_fixture`` are all tweakable.
|
||||
|
||||
For the classic ``--pdb`` CLI-flag pattern use the conditional
|
||||
variant:
|
||||
|
||||
.. code:: python
|
||||
|
||||
from tractor.devx import maybe_open_crash_handler
|
||||
|
||||
@app.command() # a `typer` (or `click`) endpoint
|
||||
def cmd(pdb: bool = False):
|
||||
with maybe_open_crash_handler(pdb=pdb):
|
||||
...
|
||||
|
||||
REPL niceties and hooks
|
||||
-----------------------
|
||||
|
||||
Every REPL in this guide is a `pdbp`_ instance (the maintained
|
||||
fork-and-fix of `pdb++`_) pre-configured by ``tractor``:
|
||||
|
||||
- pygments syntax highlighting in listings and tracebacks,
|
||||
- tab completion — including an automatic fixup for
|
||||
libedit-compiled CPythons (e.g. ``uv``-distributed pythons),
|
||||
- sticky mode available via the ``sticky`` command (off by
|
||||
default),
|
||||
- no long-line truncation (terminal resizes behave),
|
||||
- the ``(Pdb+)`` prompt, ``ll``, hidden-frames support and the
|
||||
rest of the ``pdb++`` goodies you may already know.
|
||||
|
||||
Internal runtime frames are traceback-hidden so the REPL lands
|
||||
exactly on *your* ``pause()``-call or crash frame, never on
|
||||
``tractor`` guts.
|
||||
|
||||
Finally, if your app owns the terminal (TUIs, fullscreen
|
||||
dashboards) pass ``repl_fixture=<your ctx mngr>`` to ``pause()``,
|
||||
``post_mortem()`` or ``open_crash_handler()``: it's entered just
|
||||
before the REPL engages (return ``False`` to skip entry entirely)
|
||||
and exited on release — perfect for suspending and restoring your
|
||||
screen around a debug session.
|
||||
|
||||
.. _debugging-caveats:
|
||||
|
||||
Caveats and platform notes
|
||||
--------------------------
|
||||
|
||||
An honest list of the current rough edges:
|
||||
|
||||
- **Windows**: the debugger has no CI coverage on windows at all
|
||||
(the entire test module is skipped there); manual testing has
|
||||
shown it *can* work, but you're in uncharted territory —
|
||||
reports welcome!
|
||||
- **macOS**: supported but with rough edges: special-cased prompt
|
||||
re-flushing for ``bash``-on-darwin, a few tooling tests skipped
|
||||
on CI, and the AF_UNIX ~104-char socket-path limit forces some
|
||||
examples (like the stackscope demo above) to fall back from
|
||||
``'uds'`` to ``'tcp'`` transport. Wonder if all of it'll work
|
||||
on OS X? So do we.
|
||||
- **CPython 3.14**: ``greenback`` (via ``greenlet``) doesn't
|
||||
support 3.14 yet, so ``pause_from_sync()`` and the builtin
|
||||
``breakpoint()`` override are effectively 3.13-only for now.
|
||||
The async APIs — ``pause()`` and ``post_mortem()`` — need no
|
||||
greenback and work everywhere.
|
||||
- **out-of-band** ``asyncio`` **tasks**: sync pauses from aio
|
||||
tasks *not* spawned via ``tractor.to_asyncio`` raise a
|
||||
``RuntimeError`` (no greenback portal was bestowed); run
|
||||
``await greenback.ensure_portal()`` inside such a task first.
|
||||
- **nested-tree ctrl-c edges**: ``SIGINT`` relay through
|
||||
intermediary parents that aren't themselves in debug mode still
|
||||
has known rough edges — see `#320`_.
|
||||
- **captured stdio**: ``pytest``-style output capture can hang a
|
||||
``pause()``; use a real terminal (or a pty à la ``pexpect``,
|
||||
which is how ``tractor``'s own suite drives every one of these
|
||||
examples).
|
||||
|
||||
Where to next?
|
||||
--------------
|
||||
|
||||
.. seealso::
|
||||
|
||||
- :doc:`/guide/context` — the SC-linked cross-actor task API
|
||||
that all the crash-propagation semantics above ride on.
|
||||
- :func:`tractor.pause`, :func:`tractor.post_mortem` and
|
||||
:func:`tractor.pause_from_sync` in the API reference.
|
||||
- ``examples/debugging/`` — 20-odd runnable scripts, nearly
|
||||
every one exercised by the test suite through a real pty.
|
||||
|
||||
.. _structured concurrency:
|
||||
https://en.wikipedia.org/wiki/Structured_concurrency
|
||||
.. _trio: https://github.com/python-trio/trio
|
||||
.. _cancellation: https://trio.readthedocs.io/en/latest/
|
||||
reference-core.html#cancellation-and-timeouts
|
||||
.. _pdbp: https://github.com/mdmintz/pdbp
|
||||
.. _pdb++: https://github.com/pdbpp/pdbpp
|
||||
.. _greenback: https://github.com/oremanj/greenback
|
||||
.. _stackscope: https://github.com/oremanj/stackscope
|
||||
.. _PEP 553: https://peps.python.org/pep-0553/
|
||||
.. _#320: https://github.com/goodboy/tractor/issues/320
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue