Add NLNet Commons Fund grant proposal draft
Draft targeting the NGI Zero Commons Fund (€50k, deadline April 1 2026) covering all form fields: abstract, relevant experience, comparison w/ existing efforts, technical challenges, ecosystem engagement, and budget breakdown. - frame `piker` as sovereign trading infra for the commons addressing a clear market failure in proprietary fintech enclosure - highlight AGPLv3+ copyleft, structured concurrent distributed arch via `tractor`/`trio`, zero-web philosophy, and venue-agnostic design - detail 5 deliverable workstreams: runtime hardening, broker plugin API, EMS completion, packaging/onboarding, and security audit - include comparison against proprietary platforms, permissive-licensed FOSS libs, and institutional systems (this patch was generated in some part by [`claude-code`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-codengi0_submission
parent
79be47635f
commit
ccb6f1913c
|
|
@ -0,0 +1,436 @@
|
|||
# NLNet NGI Zero Commons Fund — Grant Proposal Draft
|
||||
|
||||
> **Deadline**: April 1st 2026 12:00 CEST
|
||||
> **Fund**: NGI Zero Commons Fund
|
||||
> **Amount range**: €5,000–€50,000
|
||||
|
||||
---
|
||||
|
||||
## Proposal Name
|
||||
|
||||
**piker: sovereign, structured-concurrent trading infrastructure for the commons**
|
||||
|
||||
## Website / Wiki
|
||||
|
||||
- https://pikers.dev/pikers/piker (self-hosted Gitea)
|
||||
- https://github.com/pikers/piker (eventual marketing mirror)
|
||||
- https://github.com/goodboy/tractor (core runtime dependency)
|
||||
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
<!-- "explain the whole project and its expected outcome(s)" -->
|
||||
<!-- NLNet guidance: "focus primarily on the what and how, not so much on the why" -->
|
||||
|
||||
`piker` is a broker-agnostic, fully libre (AGPLv3+) trading toolkit and
|
||||
runtime that enables individuals to participate in financial markets
|
||||
without surrendering their data, autonomy, or compute infrastructure to
|
||||
proprietary platforms or cloud services.
|
||||
|
||||
The project delivers a complete, federated trading stack — real-time
|
||||
data feeds, order management, charting, and (financial) signal
|
||||
processing — built entirely on free software and designed from first
|
||||
principles around structured concurrency, distributed actor
|
||||
supervision, and zero-copy IPC. Every component runs on the user's
|
||||
own hardware; no centralized service or account is required beyond
|
||||
the market venue itself.
|
||||
|
||||
**What piker provides today (alpha):**
|
||||
|
||||
- A multi-process actor runtime (via `tractor`, our structured
|
||||
concurrency framework built on `trio`) that supervises broker
|
||||
connections, data feeds, order execution, and UI rendering as
|
||||
a distributed process tree — locally or across hosts.
|
||||
|
||||
- A broker abstraction layer with working integrations for Binance,
|
||||
Kraken, Interactive Brokers, Deribit, and Kucoin, presenting
|
||||
a unified API for market data, order submission, and position
|
||||
tracking regardless of venue.
|
||||
|
||||
- A hyper performant real-time charting UI built on PyQt6 and our own
|
||||
extensions to the `PyQtGraph` graphic lib, all driven by
|
||||
in-shared-memory data (like OHLCV) buffers that multiple processes
|
||||
read without serialization overhead.
|
||||
|
||||
- a keyboard-driven, modal UI/X targeting tiling window manager users
|
||||
on linux.
|
||||
|
||||
- A financial signal processing (FSP) subsystem that allows traders
|
||||
to compose custom indicators, auto-strats, real-time analysis, or other real-time
|
||||
data oriented integrations all from modern async (`trio`) Python.
|
||||
Further feats include
|
||||
and hot-reload them against live data streams.
|
||||
|
||||
- A "dark clearing" engine which ensures all triggered/algorithmic
|
||||
orders can exist entirely trader-client-side without
|
||||
submitting/yielding any control to a particular broker/venue thus
|
||||
guaranteeing minimal stop/hedged-order losses as often seen when
|
||||
trading against larger/faster market participants.
|
||||
|
||||
- A full "paper trading" engine that simulates order execution
|
||||
against live market feeds, optionally with a per provider traffic
|
||||
profile config, enabling easy strategy development against live market
|
||||
data feeds without capital risk.
|
||||
|
||||
- Apache Arrow / Parquet and storage for OHLCV history and trade
|
||||
ledgers with full accounting primitives (positions, P&L, allocation
|
||||
tracking).
|
||||
|
||||
|
||||
**What we intend to deliver with this grant:**
|
||||
|
||||
1. **Stabilize the core runtime and IPC layer** — harden `tractor`'s
|
||||
supervision protocol, improve error propagation across actor
|
||||
boundaries, and complete the modular transport stack (TCP → UDS
|
||||
→ shared-memory ring buffers -> TIPC) so that the distributed
|
||||
architecture is production-reliable.
|
||||
|
||||
Some of this work in `tractor` core could include,
|
||||
- formalizing the typed IPC-msg semantics and API,
|
||||
- building out the hot-code-reload and supervision strategy API,
|
||||
- adopting and integrating more modern discovery protocol systems
|
||||
as built-ins,
|
||||
- experimenting with the (oddly not well known, from "team erlang"
|
||||
AND long time been inside the linux kernel) TIPC transport as
|
||||
an official IPC backend. Ostensibly its use would allow covering
|
||||
outstanding feats mentioned prior such as,
|
||||
- any sophisticated discovery sys requirements outstanding.
|
||||
- built-in failover/HA features that would normally require
|
||||
per-IPC-transport augmentation, at least for most
|
||||
in-use-in-internet-stack protocols.
|
||||
|
||||
- localhost-only shared-memory transport refinements with various
|
||||
improvements by extending the current implementation with,
|
||||
- `eventfd` signalling around our readers-writer-lock
|
||||
abstrations for managing posix-shm segments.
|
||||
- use of apache `arrow` buffers as an data-buffer-backend to the
|
||||
std lib's `multiprocessing.sharedmemory`,
|
||||
|
||||
2. **Extend the broker/data provider integration set** — formalize
|
||||
the provider pluggable backened APIs so that community
|
||||
contributors can more easily add new venues (DEXs, traditional
|
||||
brokerages, prediction markets) without deep framework knowledge.
|
||||
Ideally our ongoing agentic AI dev helpers should make such
|
||||
integrations very vibe-able and thus simple to get started.
|
||||
Publish integration guides and reference implementations.
|
||||
|
||||
3. **Mature the order management and execution system (EMS)** —
|
||||
complete the state machine for multi-leg and conditional orders,
|
||||
add journaling for audit trails, and ensure the paper-trading
|
||||
engine faithfully models slippage and fee structures per venue.
|
||||
|
||||
4. **Ship packaging and onboarding** — produce reproducible builds
|
||||
via Nix flakes and `uv`-based installs, write user-facing
|
||||
documentation, and establish a contributor onboarding path so the
|
||||
project can grow beyond its current core team.
|
||||
|
||||
5. **Security and privacy hardening** — integrate VPN-aware
|
||||
connection management so traders can route venue traffic through
|
||||
tunnels, preventing brokers and market makers from correlating
|
||||
order flow with identity. Audit all IPC surfaces for
|
||||
injection/escalation vectors.
|
||||
|
||||
All outcomes are released under the GNU Affero General Public License
|
||||
v3 or later. The AGPL's network-interaction clause ensures that
|
||||
anyone who deploys `piker` as a service must share their
|
||||
modifications — preventing the enclosure of community work by
|
||||
proprietary cloud platforms, which is often the dominant failure mode
|
||||
in financial technology today available to most self-funded retail
|
||||
investors and traders.
|
||||
|
||||
---
|
||||
|
||||
## Relevant Experience / Prior Involvement
|
||||
|
||||
<!-- "Have you been involved with projects or organisations relevant
|
||||
to this project before?" -->
|
||||
|
||||
@goodboy is primary creator and steward of both `piker` and
|
||||
`tractor`. He has been developing structured-concurrency (SC)
|
||||
distributed systems in Python since 2018 and is an active participant
|
||||
in the `trio` ecosystem and the broader structured concurrency
|
||||
community (which has since influenced Python's `asyncio.TaskGroup`,
|
||||
Kotlin coroutines, Swift concurrency, and Java's Project Loom).
|
||||
|
||||
`tractor` itself is a novel contribution to the field: it extends the
|
||||
structured concurrency supervision model across process and host
|
||||
boundaries via an IPC-contract-enforced, "SC supervision control
|
||||
protocol" — something no other Python runtime provides nor has
|
||||
seemingly any other project formalized. This design directly informs
|
||||
`piker`'s architecture and is maintained as a standalone library so
|
||||
that other projects can continue to benefit from its distributed
|
||||
runtime principles and primitives.
|
||||
|
||||
The `piker` community operates via Matrix chat and the core code base
|
||||
is primarily developed on a self-hosted Gitea instance at
|
||||
`pikers.dev`. Contributors include a small but high dedicated core
|
||||
group of "hacker traders" (what we call actual "pikers": a new hybrid
|
||||
type of developer who are also themselves self-funded independent
|
||||
traders) and a burgeoning community of surrounding devs, engineers,
|
||||
data scientists, traders and investors.
|
||||
|
||||
---
|
||||
|
||||
## Comparison With Existing Efforts
|
||||
|
||||
<!-- "how the project differs from existing/historical efforts" -->
|
||||
|
||||
The landscape of trading software falls into three categories, none of
|
||||
which serve the commons:
|
||||
|
||||
**1. Proprietary platforms (TradingView, MetaTrader, Bloomberg Terminal,
|
||||
Thinkorswim)**
|
||||
These are closed-source, SaaS-dependent, and extract rent from users
|
||||
via subscriptions, data fees, and order-flow selling. Users have zero
|
||||
visibility into how their data is handled, cannot audit execution
|
||||
quality, and are locked into vendor-specific ecosystems. They represent
|
||||
the canonical "market failure" that NLNet's commons fund targets: an
|
||||
essential digital infrastructure captured by extractive incumbents.
|
||||
|
||||
**2. Open-source trading libraries (Zipline, Backtrader, ccxt,
|
||||
freqtrade)**
|
||||
These provide *components* (backtesting engines, API wrappers, bot
|
||||
frameworks) but not a *complete runtime*. They are typically
|
||||
single-process, single-venue, and lack real-time supervision, IPC, or
|
||||
UI. Most use permissive licenses (MIT/Apache) which allow proprietary
|
||||
enclosure — indeed, Zipline was developed by Quantopian (now defunct)
|
||||
and its maintenance has fragmented. None provide a coherent distributed
|
||||
architecture or structured concurrency guarantees.
|
||||
|
||||
**3. Institutional systems (FIX protocol gateways, internal bank
|
||||
platforms)**
|
||||
These are inaccessible to individuals and small firms, require
|
||||
expensive connectivity, and are architecturally rooted in 1990s-era
|
||||
message bus designs (TIBCO, 29West) that predate modern concurrency
|
||||
research.
|
||||
|
||||
**`piker` is unique in combining all of the following:**
|
||||
|
||||
- A *complete, integrated stack* (data → compute → orders → UI) rather
|
||||
than isolated components.
|
||||
|
||||
- *Structured concurrent distribution* as a first-class architectural
|
||||
property, not an afterthought. Every subsystem is an actor in a
|
||||
supervised tree; failures propagate and cancel cleanly; resources
|
||||
never leak. This is a direct application of formal concurrency
|
||||
research (Dijkstra, Hoare, and the recent structured concurrency
|
||||
lineage from Trio/Nurseries) to a domain that has historically
|
||||
ignored it.
|
||||
|
||||
- *Hard copyleft licensing* (AGPLv3+) that prevents the most common
|
||||
form of open-source value extraction in fintech: wrapping a
|
||||
permissively-licensed library in a proprietary cloud service.
|
||||
|
||||
- *Zero-web architecture*: native IPC (Unix domain sockets,
|
||||
shared-memory) and native Qt UI instead of HTTP/WebSocket/Electron.
|
||||
This is not aesthetic preference — it is an engineering decision that
|
||||
eliminates entire classes of latency, security, and complexity
|
||||
problems introduced by the browser runtime.
|
||||
|
||||
- *Venue agnosticism* as a design principle: the same codebase, the
|
||||
same UI, the same order management primitives regardless of whether
|
||||
the user trades crypto on Binance, equities on Interactive Brokers,
|
||||
or derivatives on Deribit. No other open-source project attempts this
|
||||
across asset classes with a unified real-time architecture.
|
||||
|
||||
---
|
||||
|
||||
## Technical Challenges
|
||||
|
||||
<!-- "significant obstacles to overcome" -->
|
||||
|
||||
**1. Structured concurrency across host boundaries**
|
||||
|
||||
`tractor`'s supervision protocol must guarantee that if any actor in the
|
||||
distributed tree fails, the failure is propagated to all dependent
|
||||
actors and resources are cleaned up — exactly as `trio` nurseries do
|
||||
within a single process. Achieving this over network transports (which
|
||||
can partition, delay, or corrupt messages) while maintaining low latency
|
||||
is an open research problem. We are implementing a cancellation-scope
|
||||
protocol layered on our IPC message spec that handles partial failures
|
||||
gracefully without resorting to the "let it crash" philosophy that
|
||||
abandons resource cleanup.
|
||||
|
||||
**2. Zero-copy shared-memory data flow**
|
||||
|
||||
Our `ShmArray` primitive allows multiple processes (chart renderer, FSP
|
||||
engine, EMS) to read the same OHLCV buffer without serialization. This
|
||||
requires careful lock-free coordination: the sampling daemon appends
|
||||
new bars while consumers read concurrently. Extending this to ring
|
||||
buffers for tick-level data (L2 order books, trade streams) without
|
||||
introducing GC pauses or memory corruption is a systems-level challenge
|
||||
that demands expertise in memory-mapped I/O and cache-line-aware data
|
||||
structures.
|
||||
|
||||
**3. Broker API heterogeneity**
|
||||
|
||||
Each venue has a different API surface, authentication model, rate-limit
|
||||
policy, market data format, and order lifecycle model. Abstracting these
|
||||
behind a unified interface without losing venue-specific capabilities
|
||||
(e.g., Binance's sub-account system, IB's complex multi-leg orders,
|
||||
Deribit's options greeks) requires a plugin architecture that is both
|
||||
general enough to be learnable and specific enough to be useful. We must
|
||||
also handle venue-specific failure modes (exchange maintenance windows,
|
||||
API deprecations, WebSocket reconnection) within the structured
|
||||
concurrency supervision tree.
|
||||
|
||||
**4. Real-time UI rendering performance**
|
||||
|
||||
Financial charting demands smooth 60fps rendering of potentially
|
||||
millions of OHLCV bars with dynamic zoom, overlaid indicators, and
|
||||
interactive order annotations. We have extended PyQtGraph with custom
|
||||
batch-rendering paths and GPU-accelerated drawing, but further work is
|
||||
needed on level-of-detail decimation, viewport culling, and efficient
|
||||
incremental updates as new data arrives — all while the UI thread
|
||||
remains responsive to keyboard input.
|
||||
|
||||
**5. Reproducible packaging across Linux distributions**
|
||||
|
||||
`piker` depends on Qt6, system-level libraries (OpenGL, SSL), Python
|
||||
native extensions (numpy, numba, pyarrow), and our own `tractor`
|
||||
runtime. Ensuring that `uv sync` and `nix develop` both produce
|
||||
working, reproducible environments across NixOS, Arch, Debian, and
|
||||
Fedora — without resorting to containerization that undermines the
|
||||
native-performance philosophy — requires continuous integration testing
|
||||
against multiple targets.
|
||||
|
||||
---
|
||||
|
||||
## Ecosystem Engagement
|
||||
|
||||
<!-- "how you'll interact with relevant actors and promote outcomes" -->
|
||||
|
||||
**Upstream contributions:**
|
||||
|
||||
`piker`'s development directly drives improvements in its dependency
|
||||
ecosystem. Bugs and feature requests discovered through our use of
|
||||
`trio`, `pyqtgraph`, `msgspec`, `polars`, and `ib_insync`/`ib_async`
|
||||
are reported and often patched upstream. Our extended PyQtGraph fork
|
||||
contains rendering optimizations that are candidates for upstream
|
||||
merge. `tractor` itself is a standalone project that any Python
|
||||
developer can use for structured-concurrent multiprocessing, independent
|
||||
of `piker`.
|
||||
|
||||
**Community building:**
|
||||
|
||||
- We operate a Matrix-based chat community for real-time collaboration.
|
||||
- Our self-hosted Gitea instance at `pikers.dev` provides
|
||||
infrastructure independent of any corporate platform (GitHub is used
|
||||
as a mirror for discoverability).
|
||||
- We maintain integration guides for AI-assisted development workflows
|
||||
(currently Claude Code) to lower the barrier for new contributors.
|
||||
- We plan to use grant funds to produce user documentation, video
|
||||
walkthroughs, and contributor onboarding materials.
|
||||
|
||||
**Standards and interoperability:**
|
||||
|
||||
- `tractor`'s IPC message spec is being formalized with the goal of
|
||||
becoming a reusable protocol for structured-concurrent RPC, applicable
|
||||
beyond trading.
|
||||
- Our broker abstraction layer is designed so that adding a new venue
|
||||
is a matter of implementing a well-documented Python module interface,
|
||||
not forking the project.
|
||||
- We use open data formats throughout: Apache Arrow/Parquet for
|
||||
time-series, TOML for configuration, and standard Python typing
|
||||
(`msgspec.Struct`) for all message schemas.
|
||||
|
||||
**Alignment with NGI Zero Commons Fund goals:**
|
||||
|
||||
`piker` addresses a clear market failure: individuals who wish to
|
||||
participate in financial markets are forced to use proprietary,
|
||||
surveillance-laden platforms that extract value from their data and
|
||||
order flow. There is no credible libre alternative today. By building
|
||||
a complete, federated, copyleft-licensed trading stack, we create
|
||||
digital infrastructure that is:
|
||||
|
||||
- **Honest**: open-source, auditable, no hidden order-flow selling.
|
||||
- **Open**: AGPLv3+ ensures all modifications remain public goods.
|
||||
- **Inclusive**: venue-agnostic design welcomes any market participant
|
||||
regardless of which broker or asset class they prefer.
|
||||
- **Robust**: structured concurrency guarantees that the system fails
|
||||
safely rather than silently corrupting state — a property that
|
||||
proprietary platforms routinely lack and that costs retail traders
|
||||
real money.
|
||||
|
||||
Financial infrastructure is critical public infrastructure. The fact
|
||||
that it is almost entirely enclosed by proprietary interests is a
|
||||
failure of the commons that this project directly addresses.
|
||||
|
||||
---
|
||||
|
||||
## Suggested Budget Breakdown
|
||||
|
||||
<!-- Adjust the total as you see fit; NLNet range is €5k–€50k -->
|
||||
|
||||
**Requested amount: €50,000**
|
||||
|
||||
| Task | Effort | Rate | Amount |
|
||||
|------|--------|------|--------|
|
||||
| `tractor` runtime hardening: supervision protocol, transport stack (TCP/UDS/shm), error propagation | 200h | €75/h | €15,000 |
|
||||
| Broker plugin API formalization + integration guides + 2 new venue integrations | 150h | €75/h | €11,250 |
|
||||
| EMS completion: multi-leg orders, journaling, paper-engine fidelity | 120h | €75/h | €9,000 |
|
||||
| Packaging & CI: Nix flakes, uv reproducibility, multi-distro testing | 60h | €75/h | €4,500 |
|
||||
| Security audit: IPC surfaces, VPN integration, dependency review | 50h | €75/h | €3,750 |
|
||||
| Documentation & onboarding: user guides, contributor docs, video walkthroughs | 60h | €75/h | €4,500 |
|
||||
| Project management & community coordination | 27h | €75/h | €2,000 |
|
||||
| **Total** | **667h** | | **€50,000** |
|
||||
|
||||
*All work is performed by the existing core team (Tyler Goodlet,
|
||||
Guillermo Rodriguez, and community contributors). The hourly rate
|
||||
reflects senior-level systems engineering in a cost-conscious FOSS
|
||||
context.*
|
||||
|
||||
---
|
||||
|
||||
## Other Funding Sources
|
||||
|
||||
This project has not received external funding to date. All development
|
||||
has been self-funded by the core developers. There are no other pending
|
||||
grant applications at this time.
|
||||
|
||||
---
|
||||
|
||||
## Generative AI Disclosure
|
||||
|
||||
This proposal draft was composed with assistance from Claude (Anthropic,
|
||||
model: claude-opus-4-6) on 2026-03-30. The AI was used to:
|
||||
|
||||
- Fetch and summarize NLNet's proposal requirements and fund mission
|
||||
statement.
|
||||
- Explore the `piker` codebase to compile an accurate technical
|
||||
description of the architecture.
|
||||
- Draft the proposal text based on the above research and the author's
|
||||
direction regarding key arguments and positioning.
|
||||
|
||||
The author reviewed, edited, and approved all content. Unedited AI
|
||||
outputs and prompts are available on request.
|
||||
|
||||
---
|
||||
|
||||
## Notes for the Applicant
|
||||
|
||||
> **Before submitting**, review and personalize the following:
|
||||
>
|
||||
> 1. **Budget**: adjust the total amount, hourly rate, and task
|
||||
> breakdown to match your actual plan and financial needs. NLNet
|
||||
> accepts €5k–€50k; projects with strong potential can scale.
|
||||
>
|
||||
> 2. **Community links**: add the actual Matrix room URL (e.g.,
|
||||
> `#piker:matrix.org` or however it's addressed).
|
||||
>
|
||||
> 3. **Contact details**: fill in name, email, phone, org, country on
|
||||
> the form itself.
|
||||
>
|
||||
> 4. **Attachments**: consider attaching the piker README, an
|
||||
> architecture diagram, or a short demo video/screencast.
|
||||
>
|
||||
> 5. **Generative AI section**: update with the actual prompts used
|
||||
> and attach unedited outputs if you want full transparency.
|
||||
>
|
||||
> 6. **Tone**: NLNet reviewers are technical; the current draft leans
|
||||
> into engineering substance over marketing. Adjust if desired.
|
||||
>
|
||||
> 7. **Timeline**: NLNet doesn't require a fixed timeline in the form,
|
||||
> but if asked, a 12-month delivery window is reasonable for this
|
||||
> scope.
|
||||
Loading…
Reference in New Issue