swadhin / justapi public

📊 p4: record multi-worker scaling + native async DB benchmarks (ADR-093)

swadhinbiswas committed 2 months ago

8f63e8a

1 parent ed4da41

commit 8f63e8a

2 files changed

+70 -0

Files

BENCHMARKS.md +33 DECISIONS.md +37

Showing 2 changed files+70-0

BENCHMARKS.md DECISIONS.md

▼BENCHMARKS.md

+33-0

@@ -1966,3 +1966,36 @@ throughput, stay on GIL-locked CPython with the native fast path (~700k).
1966	1966
1967	1967	Wheel: `justapi-2.0.8-cp314-cp314t-...whl` (no abi3). 168/170 pytest pass on
1968	1968	3.14t.
	1969	+
	1970	+## Multi-worker prefork scaling — VERIFIED (recorded 2026-08-07, item #2)
	1971	+
	1972	+`justapi serve --static-dir <dir> --workers N` prefork scaling, keep-alive
	1973	+raw-socket client, 4KB static file, this machine (2×Xeon-class, 8 cores).
	1974	+
	1975	+\| Workers \| c=8 \| c=32 \| c=64 \| Δ vs 1 worker \|
	1976	+\|---\|---:\|---:\|---:\|---:\|
	1977	+\| 1 \| 53,010 \| 43,041 \| — \| baseline \|
	1978	+\| 4 \| 99,711 \| 97,254 \| — \| 1.88× @c=8 \|
	1979	+\| 8 \| — \| 102,835 \| 94,025 \| 1.93× @c=32 \|
	1980	+
	1981	+- Prefork machinery verified end-to-end: shared listener fd handed to N child
	1982	+ processes, ~2× scaling on 2× worker count (cores saturate at 4 on this box).
	1983	+- Auto-scale smoke (`--scale --min-workers 1 --max-workers 4`): supervisor
	1984	+ spawns workers under load and serves requests (36.5k RPS c=32 during the
	1985	+ scaling transient; steady-state equals the 4-worker line).
	1986	+- Honest scope note: the CLI serves Rust-side workloads (static, middleware,
	1987	+ compression, inference) — `justapi-cli → justapi-core` only per architecture.
	1988	+ Python apps scale via `app.run()` in-process (threads) or N OS processes
	1989	+ behind a shared socket (documented; not benchmarked here).
	1990	+
	1991	+## Native async DB awaits — query_async (recorded 2026-08-07, ADR-093)
	1992	+
	1993	+HTTP, sqlite, slow query (2M-row recursive CTE ≈ 200ms), c=16:
	1994	+
	1995	+\| Path \| RPS \| Notes \|
	1996	+\|---\|---:\|---\|
	1997	+\| `await app.db.query_async(...)` \| 320 \| parallel on DB tokio runtime, loop never blocked \|
	1998	+\| `app.db.query(...)` from async handler \| 6 \| blocks the asyncio loop thread — serialized \|
	1999	+
	2000	+53× faster on slow queries. Fast sqlite queries (~sub-ms): parity (~4-4.7k
	2001	+RPS both, loop-dispatch bound). The async path is the safe choice for real DBs.
1969	2002

▼DECISIONS.md

+37-0

@@ -3602,3 +3602,40 @@ recorded: Rust-native operation types (sse_native, native CRUD, native_async),
3602	3602	never a coroutine driver (ADR-090/091) nor multi-loop (this ADR).
3603	3603
3604	3604	Evidence: A/B runs above, identical wheel, JUSTAPI_ASYNC_LOOPS=1 vs default.
	3605	+
	3606	+## ADR-093 — 2026-08-07 — Native async DB awaits: query_async / execute_async
	3607	+
	3608	+Decision: Add `DbPool.query_async(sql, params)` / `DbPool.execute_async(...)`
	3609	+— Python-awaitable DB operations that run on the DB's own multi-threaded tokio
	3610	+runtime with the GIL released for the whole execution. This is the "native
	3611	+operation type" the async experiments (ADR-090/091) said is the only async win:
	3612	+the coroutine suspends on the asyncio loop and resumes when sqlx completes; zero
	3613	+Python stepping during the query.
	3614	+
	3615	+Why this wins (and the blocking path loses): the pre-existing `query()` /
	3616	+`execute()` are synchronous PyO3 methods using `run_blocking` (GIL released but
	3617	+the CALLING THREAD blocks). Called from an async handler, that thread is the
	3618	+asyncio loop thread — one slow query freezes the entire loop and serializes all
	3619	+async handlers. `query_async` never blocks the loop; concurrent queries run in
	3620	+parallel on the DB runtime.
	3621	+
	3622	+Measured (HTTP, sqlite, 2M-row recursive CTE ≈ 200ms/query, c=16):
	3623	+
	3624	+\| Path \| RPS \| Behavior \|
	3625	+\|---\|---:\|---\|
	3626	+\| `query_async` (steady state) \| 320 \| parallel on DB runtime, loop free \|
	3627	+\| `query()` blocking from async handler \| 6 \| loop frozen, fully serialized \|
	3628	+
	3629	+→ 53× faster on slow queries. On fast (sub-ms) sqlite queries the two are
	3630	+~parity (the `future_into_py` bridge adds ~µs; the blocking path's serialization
	3631	+is harmless when queries are instant) — but the async path is the only one that
	3632	+is safe for real-world DBs (Postgres/MySQL, network latency, slow queries).
	3633	+
	3634	+Implementation notes: the async future is `rt.spawn`'d onto the DB runtime
	3635	+handle (not pyo3-async-runtimes' own runtime) so pool affinity is preserved;
	3636	+result is converted via `Python::attach` + `.unbind()` to a Send `Py<PyAny>`.
	3637	+In-memory SQLite is per-connection (each pooled connection has its own DB), so
	3638	+tests must use file-based SQLite — documented in the test.
	3639	+
	3640	+Evidence: `test_native_async_db_query_and_execute` + `test_native_async_db_concurrent_through_http`
	3641	+(pass), HTTP benchmarks above.
3605	3642