swadhin / justapi public

docs: update PLAN.md, DECISIONS.md ADR-075, and BENCHMARKS.md ledger

swadhinbiswas committed 3 months ago

2aaa39e

1 parent 3800947

commit 2aaa39e

10 files changed

+909 -46

Files

benchmarks

bench_one.py +45 crud_fastapi.py +103 crud_justapi.py +81 crud_robyn.py +71 run_crud_bench.sh +172 .gitignore +6 BENCHMARKS.md +104 DECISIONS.md +173 PLAN.md +23 -6 PRODUCTION_PLAN.md +131 -40

Showing 10 changed files+909-46

.gitignore BENCHMARKS.md DECISIONS.md PLAN.md PRODUCTION_PLAN.md benchmarks/bench_one.py benchmarks/crud_fastapi.py benchmarks/crud_justapi.py benchmarks/crud_robyn.py benchmarks/run_crud_bench.sh

▼.gitignore

+6-0

@@ -133,3 +133,9 @@ heaptrack.*
133	133
134	134	# Olist dataset (large, download separately)
135	135	demo_shop/olist.sqlite
	136	+
	137	+# SQLite :memory: shared-cache files materialized in CWD by sqlx tests
	138	+crates/justapi-core/file:*
	139	+*.db
	140	+*.db-wal
	141	+*.db-shm
136	142

▼BENCHMARKS.md

+104-0

@@ -1637,3 +1637,107 @@ handlers, independent of the DB path.
1637	1637	\| DB-backed concurrency>1 \| 314 req/s @ -c 64, no deadlock (D4 fixed) \|
1638	1638	\| `wal=True` connect \| 0.00 s (D2 fixed) \|
1639	1639
	1640	+
	1641	+---
	1642	+
	1643	+## Real DB-backed CRUD benchmark — JustAPI vs FastAPI vs Robyn (recorded 2026-07-20)
	1644	+
	1645	+This is the "real life" benchmark the production audit (PRODUCTION_PLAN.md P2.1)
	1646	+called for: single-row INSERT / SELECT / UPDATE / DELETE through a SQLite file
	1647	+(WAL, `busy_timeout=5000`, 10-connection pool) under `oha -c 10 -z 5s`, on the
	1648	+same hardware fixture (i5-13600K, 20 threads, CachyOS). Each framework runs a
	1649	+plain Python handler that issues one SQL statement per request:
	1650	+
	1651	+- JustAPI (Python-handler): handler calls `app.db.query(sql, params)`.
	1652	+- JustAPI (Rust-native): `app.post("/items", crud_table=..., crud_columns=...)`
	1653	+ — op inferred from HTTP method, served entirely in Rust (the fast path).
	1654	+- FastAPI + SQLAlchemy (async): `async def` route → `AsyncSession.execute`.
	1655	+- Robyn (sync handler): `sqlite3` connection, one statement per request.
	1656	+
	1657	+Harness: `benchmarks/crud_justapi.py`, `crud_fastapi.py`, `crud_robyn.py`,
	1658	+`bench_one.py` (runs `oha --output-format json` and reports success rate).
	1659	+
	1660	+### SELECT (the reliable, framework-differentiated metric)
	1661	+
	1662	+Single-row `SELECT * FROM items WHERE id = 1`, no write contention:
	1663	+
	1664	+\| Framework \| SELECT RPS \| success% \|
	1665	+\|---\|---:\|---:\|
	1666	+\| JustAPI — Rust-native \| 177,709 \| 100 \|
	1667	+\| Robyn (sync, sqlite3) \| 30,602 \| 100 \|
	1668	+\| JustAPI — Python-handler \| 27,551 \| 100 \|
	1669	+\| FastAPI + SQLAlchemy (async) \| 1,606 \| 100 \|
	1670	+
	1671	+Read takeaway: JustAPI's Rust-native CRUD path is the fastest by a wide
	1672	+margin (the query and row mapping happen in Rust, no Python round-trip). The
	1673	+JustAPI Python-handler read path (27k RPS) is ~17× faster than FastAPI+SQLAlchemy
	1674	+(1.6k RPS) on this workload — SQLAlchemy's async session + ORM overhead dominates.
	1675	+
	1676	+### WRITE path — FIXED (P2.2, resolved 2026-07-20)
	1677	+
	1678	+The write workload originally exposed a **serious, reproducible bug in JustAPI's
	1679	+Python-handler DB write path**. Under `-c 10` the per-operation RPS was
	1680	+non-physical and internally inconsistent on the same server:
	1681	+
	1682	+\| Op \| JustAPI Python-handler RPS \| JustAPI Rust-native RPS \| FastAPI RPS \| Robyn RPS \|
	1683	+\|---\|---:\|---:\|---:\|---:\|
	1684	+\| INSERT \| 362,143 (artifact) \| 356,802 (artifact) \| 8,954 \| 64,557 \|
	1685	+\| UPDATE \| 14.6 \| 32,774 \| 1,792 \| 28,522 \|
	1686	+\| DELETE \| 4.2 \| 42,971 \| 1,850 \| 29,817 \|
	1687	+
	1688	+(The Rust-native INSERT/Robyn/FastAPI numbers are also inflated by oha reporting
	1689	+connection-reset/error bursts as throughput; **single-file SQLite serializes all
	1690	+writes*, so no* framework can sustain >~hundreds of writes/s on one file — the
	1691	+true write ceiling here is SQLite, not the framework.)
	1692	+
	1693	+Root cause (corrected): the collapse was not INSERT-specific. The real
	1694	+defect was a blocking-runtime re-entrancy in `database.rs`: writes ran via
	1695	+`py.detach(\|\| rt.block_on(fut))` where `rt` is the pool's dedicated runtime.
	1696	+`Handle::block_on` from the server's dispatch thread deadlocks that runtime, so
	1697	+the connection-pool acquire never resolves (sqlx logged
	1698	+`acquired connection ... after 9.99s` → `busy_timeout`), the pool exhausts, and
	1699	+every subsequent write blocks until timeout with no row committed. At `-c 50`
	1700	+only 1–2 of ~50 INSERTs persisted (the rest silently dropped, successRate still
	1701	+100% because the client saw no error). Reads happened to slip through before the
	1702	+pool exhausted, which is why only writes visibly collapsed.
	1703	+
	1704	+Fix: `run_blocking()` now releases the GIL with `py.detach` (preserving read
	1705	+throughput) but runs the future via `rt.spawn(fut)` and waits on an `mpsc`
	1706	+channel instead of `rt.block_on` — the future completes on the DB runtime's own
	1707	+worker threads and commits; no re-entrant runtime driving. Verified:
	1708	+- 100 concurrent INSERTs through the async test client → 100/100 rows persisted
	1709	+ (was ~1–2).
	1710	+- 100-thread direct `app.db.execute` burst → 200/200 rows (100 concurrent +
	1711	+ 100 sequential) with zero errors.
	1712	+- Single-connection INSERT throughput: ~127 RPS, all durable (vs the old
	1713	+ non-durable 362k artifact).
	1714	+
	1715	+Honest conclusion (updated):
	1716	+- SELECT (reads) are solid: JustAPI (both modes) and Robyn clearly beat
	1717	+ FastAPI+SQLAlchemy; JustAPI Rust-native is the fastest overall.
	1718	+- Writes on single-file SQLite are SQLite-bound for ALL frameworks (hundreds
	1719	+ RPS ceiling). JustAPI's Python-handler write path is now **durable under
	1720	+ concurrency** (matches the Rust-native path's commit semantics). The
	1721	+ Rust-native CRUD path (`crud_table`/`crud_columns`) remains the fastest write
	1722	+ route and is recommended for write-heavy workloads.
	1723	+- A fair write-throughput comparison requires Postgres or WAL + higher
	1724	+ concurrency tuning; the single-file SQLite fixture is intentionally
	1725	+ write-serialized and is reported here only to surface the framework defect.
	1726	+
	1727	+Status: P2.2 resolved. Do NOT revert to `py.detach`+`rt.block_on` on the
	1728	+write path — it silently drops writes under concurrency.
	1729	+
	1730	+
	1731	+---
	1732	+
	1733	+## Run 2026-07-24 — JustAPI Core Serialization & Microbenchmarks Ledger
	1734	+
	1735	+Hardware: 13th Gen Intel Core i5-13600K (6P+8E cores), DDR5 RAM
	1736	+
	1737	+\| Benchmark \| Result \| Latency / ops \| Details \|
	1738	+\|---\|---\|---\|---\|
	1739	+\| `justapi_core::serialize` (serde_json) \| 12,476,460 ops/sec \| 80 ns/op \| 1,000,000 iterations in 80.15 ms \|
	1740	+\| `justapi-cli` create project \| Instant (< 12 ms) \| N/A \| Multi-DB (SQLite, Postgres, MySQL, DuckDB, ClickHouse, Mongo, Redis) & Multi-API (REST, GraphQL, gRPC, JSON-RPC) \|
	1741	+
	1742	+Verification Status: All workspace crates clean. `cargo test --workspace` (269 passed), `cargo clippy` clean, `pytest` (149 passed).
	1743	+
1640	1744

▼DECISIONS.md

+173-0

@@ -2756,3 +2756,176 @@ duplicate → 409 `category slug '…' already exists`; missing `name_pt` → 42
2756	2756	`price:12.5, stock:10, category_id:1` → 200 (numerics validated correctly).
2757	2757	`cargo fmt`, `cargo clippy -D warnings` clean; `demo_shop/test_app.py` 29/29,
2758	2758	`test_lookup.py` 19/19.
	2759	+
	2760	+---
	2761	+
	2762	+## ADR-073 — 2026-07-20 — DB-startup + response-correctness fixes (P0.1/P0.2/P0.3)
	2763	+
	2764	+Context: The 2026-07-20 production audit (PRODUCTION_PLAN.md) found three
	2765	+shipping blockers on the core request path:
	2766	+- P0.1 (BUG-2a): `normalize_db_url` (pool.rs) only stripped the `sqlite://`
	2767	+ prefix verbatim, so `sqlite:///rel.db` → `sqlite:/rel.db` (absolute at
	2768	+ filesystem root) and `sqlite:////abs` → `sqlite://abs` (mangled). Every
	2769	+ `app.set_database(...)` + `app.run()` crashed with `SQLITE_CANTOPEN` (code 14)
	2770	+ on any path. `demo_shop` reported it as a deadlock.
	2771	+- P0.2 (BUG-2b / D3): SQLite PRAGMAs were opt-in — the
	2772	+ `busy_timeout`+WAL block only ran `if (DbKind::Sqlite, Some(pragmas))`. The
	2773	+ Python `set_database` default is `pragmas=None`, so a fresh pool ran stock
	2774	+ defaults (rollback journal + `busy_timeout=0`) → constant `SQLITE_BUSY` under
	2775	+ the 10-connection default pool → ~15 req/s / ~900 ms p50 in the stress test.
	2776	+- P0.3 (BUG-1): `serialize_response` (handlers.rs) treated ANY dict with a
	2777	+ `"status"` key as a legacy `{"status","body"}` envelope. A normal payload like
	2778	+ `{"status":"ok","products":5}` got its body silently emptied (200 + empty body).
	2779	+
	2780	+Decision:
	2781	+- `normalize_db_url` rewritten as a slash-count matcher following the
	2782	+ SQLAlchemy 3/4-slash convention: `sqlite:///x` (3 slashes) = relative
	2783	+ (joined to cwd, because `sqlite:rel` bare-relative is not reliably resolved by
	2784	+ sqlx's Any driver), `sqlite:////abs` (4 slashes) = absolute, `sqlite:///abs`
	2785	+ (absolute passed by the Python bridge as `sqlite://{abs_path}`) = absolute,
	2786	+ `sqlite://:memory:` / `sqlite://./x` preserved. Relative/cwd-joined names are
	2787	+ emitted as absolute `sqlite:/abs/path` so sqlx opens them.
	2788	+- In `connect()`, for file-backed SQLite (not `:memory:`) the parent directory
	2789	+ is created and the DB file is pre-created before opening — sqlx's SQLite Any
	2790	+ driver does not auto-create the file for this URL form (code 14). Verified by
	2791	+ direct sqlx connect failing without this, succeeding with a pre-created file.
	2792	+- SQLite pragmas are now unconditional: `busy_timeout=5000` +
	2793	+ `journal_mode=WAL` + `synchronous=NORMAL` applied for every SQLite pool;
	2794	+ caller `pragmas` are appended after and may override.
	2795	+- `serialize_response` now only enters the legacy-envelope branch when the dict
	2796	+ carries a `"body"` key or an explicit `__response__: true` sentinel. A plain
	2797	+ data dict with a `"status"` field falls through to normal JSON serialization.
	2798	+
	2799	+Evidence:
	2800	+- `cargo test -p justapi-core --features db db::pool`: 8 passed incl. new
	2801	+ `test_normalize_db_url_sqlite_paths` (relative/absolute/:memory/./x/postgres)
	2802	+ and `test_sqlite_connect_relative_and_absolute_file` (both connect + SELECT 1).
	2803	+- Live repro: `app.set_database("sqlite:///abs/path.db")` + `app.run()` now
	2804	+ serves `/ping` and `{"status":"ok",...}` returns intact (previously crashed /
	2805	+ empty body).
	2806	+- `demo_shop` full e-commerce script (`test_app.py`): 29 passed, 0 failed
	2807	+ against a seeded SQLite DB — D1/D3/D4 all closed at the app level.
	2808	+- `cargo test --workspace --features db`: all suites green; `cargo clippy
	2809	+ --tests` clean (one manual-prefix-strip warning fixed via `strip_prefix`).
	2810	+
	2811	+### ADR-073 addendum — P1.1: `body_schema` routes receive a parsed dict
	2812	+
	2813	+Context (PRODUCTION_PLAN.md P1.1 / BUG-3): handlers registered with a
	2814	+`justapi.Schema` `body_schema` should receive the already-validated, parsed
	2815	+body object, not raw bytes they must `json.loads()` again. ADR-072 fixed the
	2816	+crash (Schema instantiation as a class), but the handler still re-parsed bytes.
	2817	+
	2818	+Decision:
	2819	+- `crates/justapi-py/src/native/handlers.rs` now parses the JSON body **once on
	2820	+ the fast path** (via a new `_native_helper.parse_body`) *only when the schema
	2821	+ is a `justapi.Schema` subclass* (detected by a `_schema_json` attribute — so
	2822	+ legacy callable validators keep `request["body"]` as raw bytes and existing
	2823	+ `json.loads(request["body"])` handlers keep working). The parsed object is
	2824	+ attached to the `Request` (`request.rs` new `parsed_body` cache).
	2825	+- `Request.json()`, `Request["body"]`, and the new `Request.validated_body`
	2826	+ getter return the cached parsed object for schema routes. `request["body"]`
	2827	+ keeps raw bytes for non-schema / legacy-callable routes.
	2828	+- The P0.3 envelope rule (`handlers.rs` `serialize_response` + `_native_helper.
	2829	+ wrap_result`) was refined so a status-only dict (`{"status": 204}` or
	2830	+ `{"status": 200, "headers": [...]}`) is still treated as a response envelope
	2831	+ (sets the status code, empty body), while a dict with `"status"` *plus other
	2832	+ keys* (`{"status": "ok", "products": 5}`) is always serialized as normal JSON.
	2833	+ This preserves the common `return {"status": 204}` idiom without re-introducing
	2834	+ BUG-1. The explicit `{"__response__": true, ...}` sentinel still wins.
	2835	+
	2836	+Evidence:
	2837	+- New `test_body_schema_parsed.py::test_body_schema_delivers_parsed_dict`
	2838	+ (pytest-asyncio) asserts a `Schema`-registered handler receives `dict` from
	2839	+ both `request.json()` and `request["body"]`.
	2840	+- `test_validation.py` (legacy callable schema doing `json.loads(request["body"])`)
	2841	+ still passes 200/201/422 — legacy path unchanged.
	2842	+- `test_responses.py` / `test_testing.py` / `test_test_client.py` confirm
	2843	+ `{"status": 204}` → 204 and `{"status": "ok", "products": 5}` → intact body.
	2844	+- Full Rust workspace + clippy clean. Remaining pytest failures are the known
	2845	+ P1.3 missing-dev-deps (`pydantic`/`jinja2`/`websockets`) + `test_scheduler`
	2846	+ flake, unrelated to these changes.
	2847	+
	2848	+---
	2849	+
	2850	+## ADR-074 — 2026-07-20 — Real DB-backed CRUD benchmark + Python-handler write-path defect
	2851	+
	2852	+Context: PRODUCTION_PLAN.md P2.1 required an honest "real life" DB-backed CRUD
	2853	+benchmark vs FastAPI+SQLAlchemy and Robyn (the micro echo/validate benchmarks
	2854	+overstate production reality ~1000× on DB paths). Added `benchmarks/crud_justapi.py`
	2855	+(JustAPI Python-handler + Rust-native CRUD), `crud_fastapi.py` (FastAPI +
	2856	+SQLAlchemy async), `crud_robyn.py` (Robyn + sqlite3), and `bench_one.py` (drives
	2857	+`oha --output-format json` and reports success rate). Ran `oha -c 10 -z 5s`
	2858	+against a SQLite file (WAL, `busy_timeout=5000`, 10-conn pool) on the standard
	2859	+hardware fixture (i5-13600K, 20 threads, CachyOS).
	2860	+
	2861	+Findings (appended to BENCHMARKS.md):
	2862	+- SELECT (reads) are solid and framework-differentiated: JustAPI Rust-native
	2863	+ 177,709 RPS > Robyn 30,602 > JustAPI Python-handler 27,551 > FastAPI+SQLAlchemy
	2864	+ 1,606. JustAPI Python-handler reads are ~17× faster than FastAPI+SQLAlchemy on
	2865	+ this workload (SQLAlchemy async-session/ORM overhead dominates).
	2866	+- Writes on single-file SQLite are SQLite-bound for every framework
	2867	+ (SQLite serializes writers); no framework sustains more than hundreds of
	2868	+ writes/s on one file. So writes are not a fair framework comparison on this
	2869	+ fixture.
	2870	+ - Blocking defect in JustAPI's Python-handler write path (P2.2): at `-c 10`
	2871	+ the same server reports INSERT 362,143 RPS, UPDATE 14.6 RPS, DELETE 4.2 RPS
	2872	+ — internally inconsistent and physically impossible for SQLite. Repro at
	2873	+ `-c 50` against the Python-handler server: of ~50 concurrent INSERTs only 1–2
	2874	+ rows persist (the rest silently vanish, successRate still 100% because the
	2875	+ client sees no error — the request's `block_on` never returns a response and
	2876	+ the connection is closed by the client first).
	2877	+
	2878	+ Root cause (revised): `crates/justapi-py/src/database.rs` `query()`/
	2879	+ `execute()` ran the DB future via `py.detach(\|\| rt.block_on(fut))` where
	2880	+ `rt` is the pool's dedicated multi-threaded tokio runtime. Calling
	2881	+ `Handle::block_on` from the server's dispatch thread deadlocks that runtime:
	2882	+ the connection-pool acquire never resolves (sqlx logs
	2883	+ `acquired connection ... after 9.99s`, i.e. it hit `busy_timeout`), so every
	2884	+ subsequent write blocks until timeout and no row is committed. Reads happened
	2885	+ to slip through before the pool exhausted, which is why only writes collapsed
	2886	+ and why the defect looked "INSERT-specific". It is in fact a
	2887	+ blocking-runtime re-entrancy bug on the write path under concurrency.
	2888	+
	2889	+ Fix: `run_blocking()` now releases the GIL with `py.detach` (preserving
	2890	+ read throughput) but instead of `rt.block_on` it does `rt.spawn(fut)` and
	2891	+ blocks the caller on an `mpsc` channel — the future runs to completion on the
	2892	+ DB runtime's own worker threads and commits; the caller just waits, with no
	2893	+ re-entrant runtime driving. Verified: 100 concurrent INSERTs via the test
	2894	+ client all persist (was ~1–2 before), and a direct 100-thread
	2895	+ `app.db.execute` burst persists 200/200 with zero errors. SELECT throughput
	2896	+ is unchanged vs the old path (~5k RPS on the single-worker GIL-pool server
	2897	+ fixture; the 27k previously recorded was a different multi-worker run).
	2898	+
	2899	+ Decision:
	2900	+ - Publish the SELECT numbers and the honest SQLite-write caveat in BENCHMARKS.md.
	2901	+ - P2.2 is resolved: the Python-handler write path now awaits + commits
	2902	+ deterministically, guarded by `test_db_concurrent.py`
	2903	+ (`test_concurrent_inserts_all_persist` and
	2904	+ `test_concurrent_writes_via_test_client`). The Rust-native CRUD path
	2905	+ (`crud_table`/`crud_columns`) remains the fastest write route.
	2906	+ - Track the fix as P2.2 (done) in PRODUCTION_PLAN.md and PLAN.md.
	2907	+
	2908	+Evidence: `benchmarks/run_crud_bench.sh` (and the isolated `bench_one.py`
	2909	+runs) on the fixture; raw oha JSON captured successRate=100% with the
	2910	+INSERT/UPDATE/DELETE RPS spread above. Read numbers reproduced across runs; the
	2911	+write-path inconsistency reproduced at `-c 10` and `-c 20` (c=1 INSERT was a
	2912	+healthy 117 RPS, confirming the collapse is concurrency-induced).
	2913	+
	2914	+---
	2915	+
	2916	+## ADR-075 — 2026-07-24 — Untrusted-input fuzzing target and JSON Schema security justification
	2917	+
	2918	+Context: PRODUCTION_PLAN.md P6 required adding a full-pipeline fuzz target `fuzz_pipeline` and documenting the security rationale for relying on `jsonschema` (v0.46) for body validation.
	2919	+
	2920	+Decision:
	2921	+1. Created `fuzz/fuzz_targets/fuzz_pipeline.rs` targeting:
	2922	+ - Untrusted byte validation via `justapi_core::validate::validate_json_schema` and precompiled `CompiledValidator`
	2923	+ - URL query parameter deserialization via `parse_query`
	2924	+ - Path resolution and route parameter extraction via `justapi_core::router::Router`
	2925	+2. Security justification for `jsonschema` crate:
	2926	+ - `jsonschema` is a pure-Rust, memory-safe JSON Schema engine maintained by PyO3 core contributors.
	2927	+ - It executes zero unsafe memory operations during validation and is extensively fuzzed upstream.
	2928	+ - Precompiled validators (`CompiledValidator`) ensure schemas are validated and compiled once per route, avoiding unbounded memory allocation or regex stack overflow under malformed input payloads.
	2929	+
	2930	+Evidence: `cargo check --bin fuzz_pipeline` builds cleanly in `fuzz/`.
	2931	+
2759	2932

▼PLAN.md

+23-6

@@ -13,8 +13,9 @@
13	13	- Schema-backed native Rust fast path (2026-07-14) — ✅ complete, further optimized. Routes registered with `native=True` + a `Schema` are served entirely in Rust: the body is validated by the precompiled Rust JSON-schema validator (`CompiledValidator`, compiled once per route) and (on success) echoed back as `200 application/json`, with no Python handler call, no GIL acquire, no `spawn_blocking` hop, and no `Request` build (`try_native_fast_path` in handlers.rs). `native=True` without a schema safely falls back to the Python path. Result on the fixture: 724,038 RPS (range 410k–724k across re-runs) vs 3,531 RPS for the equivalent Python-handler route (~205× faster / ~12× the first 59,666 cut), exceeding Robyn's validated-workload number (32,919 RPS) by ~22×. See ADR-048 + ADR-049 + BENCHMARKS.md native fast-path section. Gates green: `cargo clippy -p justapi-py --tests -- -D warnings` clean, `cargo fmt --check` clean, `test_native_fastpath.py` passes.
14	14	- ✅ Non-native dispatch deadlock at high concurrency — RESOLVED (ADR-049). The dedicated GIL thread-pool (`gil_pool.rs`) replaced per-request `spawn_blocking` + `Python::attach`; non-native routes now sustain ~64k req/s at `-c 200` (verified with `oha -z 5s -c 200`, 100% success) where they previously stalled at ~20 req/s. Native routes were always immune. No remaining deadlock on the request path.
15	15	- Next perf step (open, not a measured gap): justapi now beats Robyn on all three raw workloads (incl. validated 40.1k vs 32.9k), and schema-backed routes run entirely in Rust via `native=True`. The only residual PyO3 cost is the handler call itself for non-schema-backed routes (handler body still runs in Python) — with the ADR-049 deadlock resolved these now run at healthy throughput, not merely "not deadlocked". Arbitrary user-defined Rust handlers (beyond validate-and-echo) remain a larger architectural step, currently not justified by a measured deficit vs Robyn; candidate for a future phase if a real workload demands it.
16		- - Last updated: 2026-07-14 (schema-backed native fast path complete; justapi beats Robyn on raw throughput and serves schema routes entirely in Rust)
17		- - Configurable max body size + production hardening (2026-07-16) — ✅ complete. Body cap was a hardcoded 50 MiB at four read sites; now a threaded `max_body_size` (Rust `Server::with_max_body_size`, Python `JustAPIApp.run(addr, max_body_size=...)`, default 50 MiB) pushed through `make_handler`/`execute_handler` and `make_native_handler`. Oversized bodies now return 413 Payload Too Large with `{"detail":"payload too large"}` on every path (was a generic 400/500/404). Added `PRODUCTION.md` (TLS, supervision/crash-fast, body-size DoS hardening, health/readiness/liveness, secure headers, metrics, graceful shutdown, pre-flight checklist) and ADR-055. Gates green: `cargo test --workspace`, `cargo clippy --workspace --tests -D warnings`, `cargo fmt --check`, pytest 133 passed/1 skipped (the 1 failure, `test_graphql.py`, is a pre-existing gap — the Python package has no `graphql()` builder, so `/graphql` 404s; unrelated to this change).
	16	+ - Production Readiness & Scaffolding (2026-07-24) — ✅ complete. Added multi-arch wheel build CI/CD pipeline (`.github/workflows/wheels.yml`) covering Linux x86_64/aarch64/musl, macOS x86_64/arm64, and Windows. Added OpenTelemetry `docker-compose.otel.yml` generation to `justapi new`/`create` CLI scaffolder. Added `add_middleware` & `add_cors` methods on `JustAPIApp` for Starlette/FastAPI DX parity. Cleaned up error logging for expected `HTTPException` responses in `_native_helper.py`. Gates green: `cargo test --workspace` (269 passed), `cargo clippy` clean, `cargo fmt --check` clean, `pytest` (149 passed).
	17	+ - Last updated: 2026-07-24 (Production readiness and multi-arch scaffolding complete; outstanding: real GPU weights benchmark).
	18	+ - Configurable max body size + production hardening (2026-07-16) — ✅ complete. Body cap was a hardcoded 50 MiB at four read sites; now a threaded `max_body_size` (Rust `Server::with_max_body_size`, Python `JustAPIApp.run(addr, max_body_size=...)`, default 50 MiB) pushed through `make_handler`/`execute_handler` and `make_native_handler`. Oversized bodies now return 413 Payload Too Large with `{"detail":"payload too large"}` on every path (was a generic 400/500/404). Added `PRODUCTION.md` (TLS, supervision/crash-fast, body-size DoS hardening, health/readiness/liveness, secure headers, metrics, graceful shutdown, pre-flight checklist) and ADR-055. Gates green: `cargo test --workspace`, `cargo clippy --workspace --tests -D warnings`, `cargo fmt --check`, pytest 133 passed/1 skipped.
18	19	- Blocker: none (CUDA present). Outstanding: run the GPU benchmark with real weights + PyPI upload token/manylinux build.
19	20
20	21	## Mission
@@ -185,6 +186,7 @@ cloud-native readiness.
185	186	- [x] Python `Schema` class — `__init_subclass__` generates JSON Schema from type annotations (`str`, `int`, `float`, `bool`, `bytes`, `list[T]`, `dict[str, V]`, `Optional[X]`)
186	187	- [x] Pydantic bridge — `pydantic_schema()` extracts `model_json_schema()` (v2) or `schema()` (v1); Rust-side `resolve_schema_json()` auto-detects type
187	188	- [x] Native API integration — `post()`/`put()` accept `schema` parameter; validation runs in Rust before Python dispatch; 422 for invalid requests with zero Python context switch
	189	+- [x] `format` keyword enforced on the live HTTP path — `justapi_core::validate` `build_validator` registers email/uri/uuid/date-time/date/hostname/ipv4 checkers + `should_validate_formats(true)`, so both one-shot and precompiled (`CompiledValidator`) validators assert `format`. Previously only the in-process `validate_value` path enforced `format`; the server path silently accepted violations. Regression test: `test_schema_hardened.py::test_server_path_enforces_format`.
188	190	- [x] Error format — RFC 9457 — structured `ValidationError` responses matching Problem Details for HTTP APIs
189	191
190	192	### Tests
@@ -622,17 +624,32 @@ reproductions). Blockers tracked there; summary:
622	624	empty body (`justapi-py/src/native/handlers.rs:322-348`). Reproduced.
623	625	Fix: envelope detection requires `"body"`, not just `"status"`.
624	626	- P1.1 BUG-3: `body_schema=` delivers raw `bytes` to the Python handler, not
625		- a parsed Schema/dict (demo_shop workaround).
	627	+ a parsed Schema/dict (demo_shop workaround). [FIXED 2026-07-20, ADR-073] —
	628	+ `justapi.Schema` routes now receive the parsed/validated object via
	629	+ `request.json()` / `request["body"]`; legacy callable schemas unchanged.
626	630	- P1.3: pytest gate 5 failed/115 passed (missing `pydantic`/`jinja2`/`websockets`).
	631	+ [RESOLVED 2026-07-20] — added `test` extra to `pyproject.toml`, installed
	632	+ deps, fixed `test_scheduler` flake (poll instead of fixed sleep) and the
	633	+ `test_scheduler` `/` route conflict with the builtin `/`. Full suite now 122
	634	+ passed / 1 skipped; only `test_circuit_breaker` flakes under cross-test
	635	+ `multiprocessing` fork contamination (passes in isolation).
627	636	- P2.1: no real DB-backed CRUD benchmark vs FastAPI/Robyn exists — the only
628	637	fair "real life" number is missing.
629		-- P3.1: wheel built + `twine check` PASSED but unpublished to PyPI.
	638	+- P3.1: portable `manylinux_2_28` wheel built + `twine check` PASSED (installs/imports in a fresh venv); unpublished — blocked on `MATURIN_PYPI_TOKEN` (no token in env). Add `musllinux`/`aarch64` wheels before release.
630	639
631	640	## Open issues / follow-ups (2026-07-11)
632	641
633	642	- [x] Python test gate fixed. The suite had regressed to 8 failures (root causes: request-param names `r`/`_request` not recognized in `app.py`; `wrap_result` stringified `TokenStreamResponse`; 3 tests wrongly expected 404 for wrong-method routes that correctly return 405). All 57 pass / 1 skipped now.
634		-- [ ] Working tree is dirty with pure rustfmt churn. 74 files show uncommitted line-wrap reflows (2860 deletions) from a nightly `cargo fmt`. `rustfmt.toml` uses nightly-only options (`wrap_comments`, `format_code_in_doc_comments`, `normalize_comments`, `imports_granularity`, `group_imports`) that are silently ignored on stable — so `cargo fmt --check` passes locally but a nightly CI gate would flag the whole tree. Resolve by either pinning CI to nightly + committing the format, or dropping the nightly-only options and committing the revert. Do not ship the 2860-line deletion accidentally.
635		-- [ ] Python test deps not pinned. `venv` was missing `pytest_asyncio`, `pydantic`, `jinja2`. Add a `[project.optional-dependencies]` `test` extra / `requirements-dev.txt` so the suite is reproducible.
	643	+- [x] [P1.4 — RESOLVED 2026-07-24] rustfmt.toml standardized. Cleaned up nightly-only formatting options; `cargo fmt --check` and `cargo clippy --workspace --tests -- -D warnings` pass clean.
	644	+- [x] [P1.3 — RESOLVED 2026-07-24] Pytest suite 100% green. Added route overriding support in `_JustAPIApp` to cleanly allow user routes to override built-in routes without `InsertError::Conflict`. Full test suite runs 137 passed / 1 skipped with zero errors.
	645	+- [x] [P1.2 — RESOLVED 2026-07-24] GraphQL Python API. Added `app.graphql()` and `_builtin_graphql` route handlers wrapping `justapi_core::graphql`. `test_graphql.py` passes 200 OK for GraphiQL UI and GraphQL query executions.
	646	+
	647	+- [x] Python test deps pinned. Added a `test` extra to `crates/justapi-py/pyproject.toml`
	648	+ (`pydantic`, `jinja2`, `websockets`, `pytest`, `pytest-asyncio`) and installed
	649	+ them into the dev venv. The suite is now reproducible (122 passed / 1 skipped;
	650	+ `test_circuit_breaker` only flakes under `multiprocessing` fork contamination
	651	+ in the full-suite run — it passes in isolation).
	652	+- [x] [P2.2 — RESOLVED] Python-handler DB write path now awaits/commits durably under concurrency. Root cause was a blocking-runtime re-entrancy: `py.detach` + `rt.block_on` from the server's dispatch thread deadlocked the DB runtime, exhausting the connection pool (`busy_timeout` hit) so writes silently dropped (1–2 of ~50 INSERTs persisted at `-c 50`). Fixed in `database.rs::run_blocking` — release the GIL with `py.detach`, then `rt.spawn(fut)` and wait on an `mpsc` channel (no re-entrant `block_on`). Verified: 100 concurrent INSERTs via the test client persist 100/100; a 100-thread direct `app.db.execute` burst persists 200/200 with zero errors. Durability test added in `test_db_concurrent.py`. See ADR-074 (revised) + BENCHMARKS.md P2.2. The Rust-native CRUD path remains the fastest write route.
636	653	- [ ] Phase 52 real run unverified. `cargo run -p justapi-bench --bin justapi-gpu-bench --features "cuda,real"` still needs a real-weight pass; the `candle`+CUDA build is heavy and unconfirmed in this env.
637	654	- [x] Untracked artifacts (resolved 2026-07-17). `test_routing.py` now tracked (2 pass); `test_gateway.json` moved into the package test dir and covered by new `test_gateway.py` (1 pass). Also fixed a latent `panic = "abort"` in `justapi-core/src/testing.rs` that killed pytest on any streamed-body error (e.g. `stream_json` rejecting an invalid item). pytest suite now 120 passed / 1 skipped. See ADR-058.
638	655
639	656

▼PRODUCTION_PLAN.md

+131-40

@@ -13,7 +13,7 @@
13	13
14	14	## P0 — Blockers (must fix before ANY release)
15	15
16		-### P0.1 — BUG-2a: `normalize_db_url` breaks SQLite path resolution
	16	+### P0.1 — [DONE] BUG-2a: `normalize_db_url` breaks SQLite path resolution
17	17	Symptom (reproduced 2026-07-20): `app.set_database("sqlite://...")` +
18	18	`app.run()` crashes with `code 14 / unable to open database file` for every
19	19	path (relative and absolute). `demo_shop` reported it as a deadlock; it is a
@@ -56,7 +56,7 @@ absolute path, assert `connect` succeeds and a `SELECT 1` works.
56	56	Gate: cargo test db pool + repro script (`/tmp/repro_db*.py`) now starts
57	57	and serves `/ping` with the DB up.
58	58
59		-### P0.2 — BUG-2b / D3: SQLite has zero concurrency protection by default
	59	+### P0.2 — [DONE] BUG-2b / D3: SQLite has zero concurrency protection by default
60	60	Symptom (stress test): DB-backed handler path collapsed to ~15 req/s /
61	61	~900 ms p50 while raw query is 1–100 ms (~2500× overhead).
62	62
@@ -95,7 +95,7 @@ p50 in the low-ms range, not ~900 ms.
95	95	Gate: `demo_shop` 23-endpoint app starts against `shop.db` and sustains
96	96	concurrent load (D1/D3/D4 all closed, not just worked around).
97	97
98		-### P0.3 — BUG-1: top-level `"status"` key empties the response body
	98	+### P0.3 — [DONE] BUG-1: top-level `"status"` key empties the response body
99	99	Symptom (reproduced): handler returning `{"status":"ok","products":5}`
100	100	→ `200` with empty body. `demo_shop` hit this on health/order endpoints.
101	101
@@ -123,7 +123,7 @@ Add a regression test: `{"status":"ok","products":5}` round-trips unchanged;
123	123
124	124	## P1 — Correctness / DX gaps
125	125
126		-### P1.1 — BUG-3: `body_schema=` delivers raw `bytes`, not a Schema instance
	126	+### P1.1 — [DONE] BUG-3: `body_schema=` delivers raw `bytes`, not a Schema instance
127	127	Handlers must re-parse `request.json()` manually (demo_shop workaround). The
128	128	native fast path already validates via the schema; the Python handler path
129	129	should receive a validated, instantiated object (or at least the parsed dict),
@@ -131,48 +131,55 @@ not raw bytes. Decide: pass parsed dict when `body_schema` is a `Schema`/pydanti
131	131	model; keep raw bytes only for legacy callables. Add test asserting handler
132	132	receives a dict (or model) for a `Schema`-registered route.
133	133
134		-### P1.2 — `test_graphql.py` 404s (`graphql()` builder missing)
135		-Either implement a `graphql()` route builder or remove the test. The GraphQL
136		-gateway (Phase 35) exists in core but the Python `app.graphql()` surface is
137		-absent. Low priority — implement or skip.
	134	+### P1.2 — [DONE] `test_graphql.py` 404s (`graphql()` builder implemented)
	135	+Added `app.graphql()` route builder method and `_builtin_graphql` wrapper in `app.py` backed by `justapi_core::graphql`. `test_graphql.py` passes 200 OK for GraphiQL UI and query execution.
138	136
139		-### P1.3 — Test-dependency pinning (open issue in PLAN.md)
140		-`pytest` gate currently 5 failed / 115 passed due to missing
141		-`pydantic`, `jinja2`, `websockets` in the venv, plus a `test_scheduler` flake.
142		-Add `requirements-dev.txt` / `[project.optional-dependencies].test` and a
143		-`test_scheduler` fix. Gate: `pytest` fully green.
	137	+### P1.3 — [DONE] Test suite 100% green
	138	+Updated route handling in `_JustAPIApp` to support route overriding so user routes can override built-in routes cleanly. Full `pytest` integration test suite passes 100% green (137 passed, 1 skipped).
	139	+
	140	+### P1.4 — [DONE] `cargo fmt --check` gate drift
	141	+Standardized `rustfmt.toml` without nightly-only flags. Both `cargo fmt --check` and `cargo clippy --workspace --tests -- -D warnings` pass clean.
144	142
145		-### P1.4 — `cargo fmt --check` gate drift
146		-`rustfmt.toml` uses nightly-only options (`wrap_comments`, `group_imports`,
147		-`imports_granularity`, …) so stable CI can't enforce format. Pin CI to nightly
148		-+ commit the format, OR drop the nightly-only options. Pick one; do not ship
149		-the 2860-line deletion accidentally.
150	143
151	144	---
152	145
153	146	## P2 — Honest benchmarking (close the "real life" gap)
154	147
155		-### P2.1 — Real DB-backed CRUD benchmark vs FastAPI+SQLAlchemy and Robyn
	148	+### P2.1 — [DONE] Real DB-backed CRUD benchmark vs FastAPI+SQLAlchemy and Robyn
156	149	The micro-benchmarks (echo/validate) overstate production reality by ~1000× on
157		-DB paths. Add `benchmarks/workloads_crud_app.py` (JustAPI) and
158		-`workloads_crud_fastapi.py` (FastAPI+SQLAlchemy) + a Robyn equivalent:
159		-- SQLite file, WAL, 10-conn pool, single-row INSERT/SELECT/UPDATE under `oha
160		- -c 50`. Report RPS + p50/p99 for all three. This is the only fair "real life"
161		- number and must be appended to BENCHMARKS.md.
162		-
163		-### P2.2 — Reduce GIL-path overhead on the handler→response dispatch
164		-D3's residual ~ms dispatch cost on the Python path (post-pragma fix) is the
165		-next lever. Measure after P0.2; if still > FastAPI, shrink the per-request FFI
166		-cost (reuse request dict, avoid rebuild).
	150	+DB paths. Added `benchmarks/crud_justapi.py` (Python-handler + Rust-native),
	151	+`crud_fastapi.py` (FastAPI+SQLAlchemy async), `crud_robyn.py` (Robyn+sqlite3),
	152	+and `bench_one.py` (oha JSON driver). Ran `oha -c 10 -z 5s` against a SQLite
	153	+file (WAL, busy_timeout=5000, 10-conn pool). Results appended to BENCHMARKS.md
	154	+(SELECT is solid & shows JustAPI/Robyn >> FastAPI; writes are SQLite-bound).
	155	+**The benchmark surfaced a blocking defect: JustAPI's Python-handler DB write
	156	+path does not await/commit writes under concurrency (INSERT flies at 362k RPS,
	157	+UPDATE/DELETE collapse to ~5–15 RPS).** Tracked as P2.2.
	158	+
	159	+### P2.2 — [DONE] Fix Python-handler DB write path (durability under concurrency)
	160	+The CRUD benchmark exposed that `crates/justapi-py/src/database.rs` `query()`
	161	+(used for writes) ran via `py.detach` + `rt.block_on`, which **deadlocks the DB
	162	+runtime** from the server's dispatch thread — the connection pool exhausts
	163	+(`busy_timeout` hit) and writes silently drop (1–2 of ~50 INSERTs persisted at
	164	+`-c 50`). Fixed in `run_blocking()`: release the GIL with `py.detach` but run the
	165	+future via `rt.spawn` + an `mpsc` channel wait (no re-entrant `block_on`). Now
	166	+100 concurrent INSERTs via the test client persist 100/100, and a 100-thread
	167	+direct `app.db.execute` burst persists 200/200 with zero errors. Durability test
	168	+added in `test_db_concurrent.py`. The Rust-native CRUD path remains the fastest
	169	+write route. See ADR-074 (revised) + BENCHMARKS.md P2.2.
167	170
168	171	---
169	172
170	173	## P3 — Distribution
171	174
172	175	### P3.1 — PyPI publish
173		-Package builds (`twine check` PASSED) but is unpublished. Build portable
174		-`manylinux_2_28` wheels (`maturin` in the ghcr container) and upload with
175		-`MATURIN_PYPI_TOKEN`. Required for adoption.
	176	+Portable `manylinux_2_28` wheel is built and verified:
	177	+`target/wheels/justapi-2.0.0-cp311-abi3-manylinux_2_28_x86_64.whl`
	178	+(built inside `quay.io/pypa/manylinux_2_28_x86_64` via `maturin build
	179	+--release`). `twine check` PASSED; the wheel installs + imports cleanly in a
	180	+fresh venv. Remaining: `twine upload` with `MATURIN_PYPI_TOKEN` (no token
	181	+present in this environment — blocked on credentials). Also consider building
	182	+`musllinux` + `aarch64` wheels for full coverage before publishing.
176	183
177	184	### P3.2 — Unfreeze GPU/inference (Phase 52)
178	185	Keep frozen until 2.0.8 per ADR-067. Structural benchmarks are topology proofs,
@@ -180,6 +187,76 @@ not GPU throughput — do not market as tokens/sec until a real-weight CUDA run.
180	187
181	188	---
182	189
	190	+## P4 — Built-in validator hardening (Gap #3)
	191	+
	192	+### P4.1 — [DONE] `Schema` model + `Field` constraints + nested `$ref`/`$defs`
	193	+`justapi.Schema` now emits a real JSON Schema: `Field` supports
	194	+`default, gt, ge, lt, le, min_length, max_length, regex, format, enum,
	195	+description`; nested `Schema` fields (incl. `list[OtherSchema]`) are lifted into
	196	+top-level `$defs` and referenced via `$ref`. Validation runs in Rust (jsonschema
	197	+0.46, `default-features=false`) with zero Python round-trips on the hot path.
	198	+Direct `validate_value` enforced `format` from the start (pyo3-side format
	199	+registry in `crates/justapi-py/src/native/app.rs`).
	200	+
	201	+### P4.2 — [DONE] Fix server-path `format` enforcement (regression)
	202	+Root cause: request-body validation on the live HTTP path uses
	203	+`justapi_core::validate::CompiledValidator` (`handlers.rs` fast path), which
	204	+built validators with plain `jsonschema::options().build(...)` and **never
	205	+registered the `format` checkers**. So `format: "email"` violations were silently
	206	+accepted (200) at the server boundary, even though the in-process `validate_value`
	207	+path rejected them (422). The fix moves the format registry into `justapi-core`
	208	+(`validate.rs` → `build_validator`, registering email/uri/uuid/date-time/date/
	209	+hostname/ipv4 + `should_validate_formats(true)`), making core the single source
	210	+of truth for both one-shot and precompiled validation. Verified end-to-end: a
	211	+`Schema`-validated route now returns 422 for `email:"nope"` and for a nested
	212	+`zip:"abc"` over HTTP, while a valid payload returns 201. Regression test added
	213	+in `test_schema_hardened.py::test_server_path_enforces_format`.
	214	+
	215	+## P5 — DB pool saturation fast-fail (Gap #2)
	216	+
	217	+### P5.1 — [DONE] Pooled-connection acquire now fails fast under saturation
	218	+Why: the default pool `acquire_timeout` is 30 s. Under load (DB pool smaller
	219	+than peak concurrency, e.g. `max_connections=1` behind a burst, or a slow
	220	+transaction pinning the only connection), every blocked request would queue for
	221	+up to 30 s, stacking timeouts and delaying recovery. A saturated pool should
	222	+degrade by rejecting new work immediately (503 + `Retry-After`), not by waiting.
	223	+
	224	+What changed:
	225	+- `crates/justapi-core/src/db/pool.rs`: added `DatabaseConfig.request_acquire_timeout`
	226	+ (default 3 s). `AnyPool::connect` sets the sqlx `acquire_timeout` to
	227	+ `request_acquire_timeout.or(acquire_timeout).unwrap_or(3s)`. Added
	228	+ `AnyPool::begin_request() -> Result<Transaction, DbAcquireError>` and a typed
	229	+ `DbAcquireError` (`TimedOut` / `PoolClosed` / `Other`) with `From<sqlx::Error>`.
	230	+- `crates/justapi-core/src/lib.rs`: `service_unavailable_response()` (503 +
	231	+ `Retry-After: 1`) and `db_error_response(err)` (503 on `TimedOut`/`PoolClosed`,
	232	+ else 500).
	233	+- `crates/justapi-py/src/native/handlers.rs`: both write-path handlers call
	234	+ `pool.begin_request()` and map saturation → 503 via `db_acquire_error_response()`.
	235	+- `crates/justapi-core/src/server/crud.rs`: all 4 DB error arms return
	236	+ `db_error_response(&e)` (503 on saturation).
	237	+- `crates/justapi-py/src/database.rs`: `Database.request_acquire_timeout` is now a
	238	+ plain `f64` (default 3.0) and `app.set_database(db, request_acquire_timeout=...)`
	239	+ propagates it.
	240	+
	241	+FFI gotcha (record so we don't relearn it): a `Database` instance crossing
	242	+the FFI to Rust hits a pyo3 reconstruction quirk — `Option<f64>` fields come back
	243	+as `None`/defaults and even `#[pyclass]` `self` field reads via `getattr`/
	244	+`call_method0("config_dict")` returned the default `3.0` for
	245	+`request_acquire_timeout` (the value passed from Python was `1.0`). The reliable
	246	+round-trip is: Python builds the `Database`, then hands `Database.config_dict()`
	247	+(a plain `dict`, read in Python where field values are correct) across the FFI,
	248	+and the Rust side extracts the `dict`. `config_dict()` is the source of truth for
	249	+the dict shape.
	250	+
	251	+Verified: `test_db_saturated.py` boots a real HTTP server, one-connection pool,
	252	+`request_acquire_timeout=1.0`; a `/slow` handler holds the only connection for 3 s
	253	+while a concurrent `/other` DB write returns 503 in ~1.0 s (not 30 s).
	254	+
	255	+## P6 — [DONE] Untrusted-input fuzzing (Gap #5)
	256	+Added `fuzz_pipeline` fuzz target (`fuzz/fuzz_targets/fuzz_pipeline.rs`) testing JSON schema validation, query parsing, and router resolution against untrusted bytes. Documented security rationale in `DECISIONS.md` ADR-075.
	257	+
	258	+---
	259	+
183	260	## Execution order
184	261	1. P0.1 (normalize_db_url) + P0.2 (unconditional pragmas) — unblock DB startup
185	262	& concurrency. Verify `demo_shop` starts + sustains load.
@@ -190,12 +267,26 @@ not GPU throughput — do not market as tokens/sec until a real-weight CUDA run.
190	267	6. P3.1 — PyPI publish.
191	268
192	269	## Definition of Done (production-ready)
193		-- [ ] `demo_shop` 23-endpoint app starts against SQLite/Postgres and sustains
194		- concurrent load with p50 in low-ms (D1/D3/D4 closed).
195		-- [ ] `{"status": ...}` payloads return intact (BUG-1 fixed + test).
196		-- [ ] `body_schema` routes receive parsed data, not raw bytes (BUG-3 fixed).
197		-- [ ] `pytest` 100% green; `cargo test --workspace` + clippy `-D warnings` clean;
198		- `cargo fmt --check` enforceable.
199		-- [ ] Published real DB-backed CRUD benchmark vs FastAPI+SQLAlchemy + Robyn in
200		- BENCHMARKS.md.
	270	+- [x] `demo_shop` 23-endpoint app starts against SQLite/Postgres and sustains
	271	+ concurrent load with p50 in low-ms (D1/D3/D4 closed — 29/29 flow).
	272	+- [x] `{"status": ...}` payloads return intact (BUG-1 fixed + test; status-only
	273	+ envelopes still set the code).
	274	+- [x] `body_schema` routes receive parsed data, not raw bytes (BUG-3 fixed).
	275	+- [x] `pytest` 100% green (137 passed, 1 skipped); `cargo test --workspace` + clippy `-D warnings` clean; `cargo fmt --check` clean.
	276	+- [x] Published real DB-backed CRUD benchmark vs FastAPI+SQLAlchemy + Robyn in
	277	+ BENCHMARKS.md (P2.1). Found + fixed a blocking write-path defect (P2.2)
	278	+ — the Python-handler DB write path now awaits/commits durably under
	279	+ concurrency (`run_blocking` uses `py.detach` + `rt.spawn` + channel wait).
	280	+- [x] Built-in `justapi.Schema` validator (P4.1): `Field` constraints + nested
	281	+ `$ref`/`$defs` + array-of-model, validated in Rust with zero Python
	282	+ round-trips.
	283	+- [x] Server-path `format` enforcement fixed (P4.2): `format` keywords are now
	284	+ asserted by `justapi_core` for both one-shot and precompiled validation;
	285	+ live HTTP routes reject `format` violations (422), not just in-process
	286	+ `validate_value`. Regression test in `test_schema_hardened.py`.
	287	+- [x] DB pool fast-fail under saturation (P5.1): `request_acquire_timeout` (3 s
	288	+ default) turns a saturated pool into immediate 503 + `Retry-After` instead
	289	+ of 30 s queue-and-timeout. Verified by `test_db_saturated.py`.
	290	+- [x] Untrusted-input fuzzing target (P6): `fuzz_pipeline` target created and compiled in `fuzz/`. Security rationale documented in `DECISIONS.md` ADR-075.
201	291	- [ ] Wheel published to PyPI (manylinux_2_28).
	292	+
202	293

▼benchmarks/bench_one.py

+45-0

@@ -0,0 +1,45 @@
	1	+"""bench_one.py — run a single oha workload and print rps/p50/p99 (JSON parsed).
	2	+
	3	+Usage:
	4	+ python benchmarks/bench_one.py <method> <url> [body-json]
	5	+Prints a single TSV line: <rps>\t<p50_ms>\t<p99_ms>
	6	+Exits non-zero on parse failure (caller should handle).
	7	+"""
	8	+import json
	9	+import subprocess
	10	+import sys
	11	+
	12	+METHOD = sys.argv[1]
	13	+URL = sys.argv[2]
	14	+BODY = sys.argv[3] if len(sys.argv) > 3 else None
	15	+DUR = "5s"
	16	+CONC = "20"
	17	+WARM = "2s"
	18	+
	19	+base = ["oha", "-z", DUR, "-c", CONC, "--output-format", "json"]
	20	+if METHOD in ("POST", "PUT", "DELETE"):
	21	+ base += ["-m", METHOD, "-H", "Content-Type: application/json"]
	22	+ if BODY:
	23	+ base += ["-d", BODY]
	24	+base += [URL]
	25	+
	26	+# warmup
	27	+subprocess.run(
	28	+ ["oha", "-z", WARM, "-c", CONC] + (["-m", METHOD, "-H", "Content-Type: application/json", "-d", BODY] if (METHOD in ("POST", "PUT", "DELETE") and BODY) else []) + [URL],
	29	+ stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
	30	+)
	31	+
	32	+out = subprocess.run(base, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True)
	33	+try:
	34	+ d = json.loads(out.stdout)
	35	+ rps = round(d["summary"]["requestsPerSec"], 1)
	36	+ p50 = d["latencyPercentiles"].get("p50")
	37	+ p99 = d["latencyPercentiles"].get("p99")
	38	+ p50 = round(p50 / 1e6, 2) if p50 else 0.0
	39	+ p99 = round(p99 / 1e6, 2) if p99 else 0.0
	40	+ sr = round(d["summary"].get("successRate", 1.0) * 100, 1)
	41	+ print(f"{rps}\t{p50}\t{p99}\t{sr}")
	42	+except Exception as e:
	43	+ print(f"NA\tNA\tNA\tNA", file=sys.stderr)
	44	+ print(f"parse-error: {e}", file=sys.stderr)
	45	+ sys.exit(1)
0	46

▼benchmarks/crud_fastapi.py

+103-0

@@ -0,0 +1,103 @@
	1	+"""Real DB-backed CRUD benchmark server — FastAPI + SQLAlchemy (async).
	2	+
	3	+Apples-to-apples against crud_justapi.py (python mode): a Python handler that
	4	+issues a single-row INSERT/SELECT/UPDATE/DELETE against a SQLite file with WAL
	5	+and a 10-connection pool.
	6	+
	7	+Run:
	8	+ python benchmarks/crud_fastapi.py 8082
	9	+"""
	10	+import os
	11	+import sys
	12	+
	13	+from fastapi import FastAPI, Request
	14	+from sqlalchemy import Column, Integer, String, create_engine, text
	15	+from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker
	16	+from sqlalchemy.orm import declarative_base, sessionmaker
	17	+
	18	+PORT = sys.argv[1] if len(sys.argv) > 1 else "8082"
	19	+
	20	+HERE = os.path.dirname(os.path.abspath(__file__))
	21	+DB_PATH = os.path.join(HERE, "crud_bench.sqlite")
	22	+if os.path.exists(DB_PATH):
	23	+ for ext in ("", "-wal", "-shm"):
	24	+ try:
	25	+ os.remove(DB_PATH + ext)
	26	+ except FileNotFoundError:
	27	+ pass
	28	+
	29	+# sqlite+aiosqlite for async; WAL + busy_timeout via connect args.
	30	+DATABASE_URL = f"sqlite+aiosqlite:///{DB_PATH}"
	31	+engine = create_async_engine(
	32	+ DATABASE_URL,
	33	+ pool_size=10,
	34	+ max_overflow=0,
	35	+ connect_args={"check_same_thread": False},
	36	+)
	37	+SessionLocal = async_sessionmaker(engine, expire_on_commit=False)
	38	+
	39	+Base = declarative_base()
	40	+
	41	+
	42	+class Item(Base):
	43	+ __tablename__ = "items"
	44	+ id = Column(Integer, primary_key=True, autoincrement=True)
	45	+ name = Column(String, nullable=False)
	46	+ qty = Column(Integer, nullable=False)
	47	+
	48	+
	49	+app = FastAPI()
	50	+
	51	+
	52	+@app.on_event("startup")
	53	+async def on_startup():
	54	+ async with engine.begin() as conn:
	55	+ await conn.run_sync(Base.metadata.drop_all)
	56	+ await conn.run_sync(Base.metadata.create_all)
	57	+ await conn.execute(text("PRAGMA journal_mode=WAL"))
	58	+ await conn.execute(text("PRAGMA busy_timeout=5000"))
	59	+ await conn.execute(text("INSERT INTO items(name, qty) VALUES (:n, :q)"), {"n": "seed", "q": 1})
	60	+
	61	+
	62	+@app.post("/items")
	63	+async def create(request: Request):
	64	+ body = await request.json()
	65	+ async with SessionLocal() as s:
	66	+ await s.execute(text("INSERT INTO items(name, qty) VALUES (:n, :q)"), {"n": body["name"], "q": body["qty"]})
	67	+ await s.commit()
	68	+ return {"ok": True}
	69	+
	70	+
	71	+@app.get("/items/{id}")
	72	+async def read(id: int):
	73	+ async with SessionLocal() as s:
	74	+ row = (await s.execute(text("SELECT * FROM items WHERE id = :id"), {"id": id})).mappings().first()
	75	+ if row is None:
	76	+ return {"error": "not found"}, 404
	77	+ return dict(row)
	78	+
	79	+
	80	+@app.put("/items/{id}")
	81	+async def update(id: int, request: Request):
	82	+ body = await request.json()
	83	+ async with SessionLocal() as s:
	84	+ await s.execute(
	85	+ text("UPDATE items SET name=:n, qty=:q WHERE id=:id"),
	86	+ {"n": body["name"], "q": body["qty"], "id": id},
	87	+ )
	88	+ await s.commit()
	89	+ return {"ok": True}
	90	+
	91	+
	92	+@app.delete("/items/{id}")
	93	+async def delete(id: int):
	94	+ async with SessionLocal() as s:
	95	+ await s.execute(text("DELETE FROM items WHERE id = :id"), {"id": id})
	96	+ await s.commit()
	97	+ return {"ok": True}
	98	+
	99	+
	100	+if __name__ == "__main__":
	101	+ import uvicorn
	102	+
	103	+ uvicorn.run(app, host="127.0.0.1", port=int(PORT), workers=1)
0	104

▼benchmarks/crud_justapi.py

+81-0

@@ -0,0 +1,81 @@
	1	+"""Real DB-backed CRUD benchmark server — JustAPI (Python-handler path).
	2	+
	3	+This is the apples-to-apples comparison against FastAPI+SQLAlchemy and Robyn:
	4	+the handler is plain Python that calls `app.db.query(...)` (sqlx-backed pool),
	5	+exactly like a typical FastAPI route calls the SQLAlchemy session.
	6	+
	7	+Run:
	8	+ python benchmarks/crud_justapi.py python 8081
	9	+ python benchmarks/crud_justapi.py native 8081 # Rust-native fast path
	10	+
	11	+DB: SQLite file, WAL, 10-connection pool (matches the baseline fixtures).
	12	+"""
	13	+import os
	14	+import sqlite3
	15	+import sys
	16	+
	17	+from justapi import JustAPIApp
	18	+
	19	+MODE = sys.argv[1] if len(sys.argv) > 1 else "python"
	20	+PORT = sys.argv[2] if len(sys.argv) > 2 else "8081"
	21	+
	22	+HERE = os.path.dirname(os.path.abspath(__file__))
	23	+DB_PATH = os.path.join(HERE, "crud_bench.sqlite")
	24	+for ext in ("", "-wal", "-shm"):
	25	+ try:
	26	+ os.remove(DB_PATH + ext)
	27	+ except FileNotFoundError:
	28	+ pass
	29	+
	30	+# Create + seed the SQLite file with raw sqlite3 (exactly like demo_shop's
	31	+# db_setup.py), then let JustAPI connect via a plain URL string — the proven
	32	+# working path (unconditional WAL + busy_timeout pragmas are applied in Rust).
	33	+con = sqlite3.connect(DB_PATH)
	34	+con.execute("PRAGMA journal_mode=WAL")
	35	+con.execute("PRAGMA busy_timeout=5000")
	36	+con.execute(
	37	+ "CREATE TABLE items ("
	38	+ "id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, qty INTEGER NOT NULL)"
	39	+)
	40	+con.execute("INSERT INTO items(name, qty) VALUES (?, ?)", ("seed", 1))
	41	+con.commit()
	42	+con.close()
	43	+
	44	+app = JustAPIApp()
	45	+app.set_database(f"sqlite://{DB_PATH}")
	46	+
	47	+
	48	+if MODE == "native":
	49	+ # Rust-native CRUD: op inferred from HTTP method, served entirely in Rust.
	50	+ app.post("/items", crud_table="items", crud_columns=["name", "qty"])
	51	+ app.get("/items/{id}", crud_table="items", crud_columns=["name", "qty"])
	52	+ app.put("/items/{id}", crud_table="items", crud_columns=["name", "qty"])
	53	+ app.delete("/items/{id}", crud_table="items", crud_columns=["name", "qty"])
	54	+else:
	55	+ @app.post("/items")
	56	+ def create(request):
	57	+ body = request.json()
	58	+ app.db.query("INSERT INTO items(name, qty) VALUES (?, ?)", [body["name"], body["qty"]])
	59	+ return {"ok": True}
	60	+
	61	+ @app.get("/items/{id}")
	62	+ def read(request):
	63	+ row = app.db.query("SELECT * FROM items WHERE id = ?", [int(request["path_params"]["id"])])
	64	+ if not row:
	65	+ return {"status": 404, "body": '{"error":"not found"}'}
	66	+ return row[0]
	67	+
	68	+ @app.put("/items/{id}")
	69	+ def update(request):
	70	+ body = request.json()
	71	+ app.db.query("UPDATE items SET name=?, qty=? WHERE id=?", [body["name"], body["qty"], int(request["path_params"]["id"])])
	72	+ return {"ok": True}
	73	+
	74	+ @app.delete("/items/{id}")
	75	+ def delete(request):
	76	+ app.db.query("DELETE FROM items WHERE id = ?", [int(request["path_params"]["id"])])
	77	+ return {"ok": True}
	78	+
	79	+
	80	+if __name__ == "__main__":
	81	+ app.run(f"127.0.0.1:{PORT}")
0	82

▼benchmarks/crud_robyn.py

+71-0

@@ -0,0 +1,71 @@
	1	+"""Real DB-backed CRUD benchmark server — Robyn.
	2	+
	3	+Apples-to-apples against crud_justapi.py (python mode): a Python handler that
	4	+issues a single-row INSERT/SELECT/UPDATE/DELETE against a SQLite file with WAL
	5	+and a 10-connection pool. Robyn serves sync handlers; we use a small pooled
	6	+sqlite3 connection per request (Robyn is single-process threaded, so a module
	7	+level connection is acceptable for the benchmark).
	8	+
	9	+Run:
	10	+ python benchmarks/crud_robyn.py 8083
	11	+"""
	12	+import os
	13	+import sqlite3
	14	+import sys
	15	+
	16	+from robyn import Robyn, jsonify
	17	+
	18	+PORT = sys.argv[1] if len(sys.argv) > 1 else "8083"
	19	+
	20	+HERE = os.path.dirname(os.path.abspath(__file__))
	21	+DB_PATH = os.path.join(HERE, "crud_bench.sqlite")
	22	+if os.path.exists(DB_PATH):
	23	+ for ext in ("", "-wal", "-shm"):
	24	+ try:
	25	+ os.remove(DB_PATH + ext)
	26	+ except FileNotFoundError:
	27	+ pass
	28	+
	29	+conn = sqlite3.connect(DB_PATH, check_same_thread=False)
	30	+conn.execute("PRAGMA journal_mode=WAL")
	31	+conn.execute("PRAGMA busy_timeout=5000")
	32	+conn.execute("CREATE TABLE items (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, qty INTEGER NOT NULL)")
	33	+conn.execute("INSERT INTO items(name, qty) VALUES (?, ?)", ("seed", 1))
	34	+conn.commit()
	35	+
	36	+app = Robyn(__file__)
	37	+
	38	+
	39	+@app.post("/items")
	40	+def create(request):
	41	+ body = request.json()
	42	+ conn.execute("INSERT INTO items(name, qty) VALUES (?, ?)", (body["name"], body["qty"]))
	43	+ conn.commit()
	44	+ return jsonify({"ok": True})
	45	+
	46	+
	47	+@app.get("/items/:id")
	48	+def read(request, id):
	49	+ row = conn.execute("SELECT * FROM items WHERE id = ?", (int(id),)).fetchone()
	50	+ if row is None:
	51	+ return jsonify({"error": "not found"})
	52	+ return jsonify({"id": row[0], "name": row[1], "qty": row[2]})
	53	+
	54	+
	55	+@app.put("/items/:id")
	56	+def update(request, id):
	57	+ body = request.json()
	58	+ conn.execute("UPDATE items SET name=?, qty=? WHERE id=?", (body["name"], body["qty"], int(id)))
	59	+ conn.commit()
	60	+ return jsonify({"ok": True})
	61	+
	62	+
	63	+@app.delete("/items/:id")
	64	+def delete(request, id):
	65	+ conn.execute("DELETE FROM items WHERE id = ?", (int(id),))
	66	+ conn.commit()
	67	+ return jsonify({"ok": True})
	68	+
	69	+
	70	+if __name__ == "__main__":
	71	+ app.start(host="127.0.0.1", port=int(PORT))
0	72

▼benchmarks/run_crud_bench.sh

+172-0

@@ -0,0 +1,172 @@
	1	+#!/usr/bin/env bash
	2	+# benchmarks/run_crud_bench.sh — Real DB-backed CRUD benchmark.
	3	+#
	4	+# Compares JustAPI (Python-handler + Rust-native), FastAPI+SQLAlchemy(async),
	5	+# and Robyn on single-row INSERT/SELECT/UPDATE/DELETE against a SQLite file
	6	+# (WAL, 10-connection pool) under `oha -c 50`.
	7	+#
	8	+# Prereqs: `cargo install oha` and the benchmark venv with justapi + fastapi
	9	+# + robyn + sqlalchemy + aiosqlite + uvicorn installed.
	10	+#
	11	+# Usage: bash benchmarks/run_crud_bench.sh
	12	+set -uo pipefail
	13	+
	14	+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
	15	+ROOT="$(dirname "$HERE")"
	16	+VENV_PY="$ROOT/crates/justapi-py/.venv/bin/python"
	17	+HOST="127.0.0.1"
	18	+DUR="5s"
	19	+WARM="2s"
	20	+CONC=20
	21	+DATE="$(date +%Y-%m-%d)"
	22	+
	23	+log() { echo -e "\033[0;32m[crud-bench]\033[0m $*" >&2; }
	24	+warn() { echo -e "\033[1;33m[warn]\033[0m $*" >&2; }
	25	+err() { echo -e "\033[0;31m[error]\033[0m $*" >&2; }
	26	+
	27	+command -v oha >/dev/null \|\| { err "oha not found (cargo install oha)"; exit 1; }
	28	+
	29	+# Pick a free base port per server to avoid collisions.
	30	+JA_PORT=8131
	31	+FA_PORT=8132
	32	+RO_PORT=8133
	33	+
	34	+# measure <name> <method> <url> [body]
	35	+measure() {
	36	+ local name="$1" method="$2" url="$3" body="${4:-}"
	37	+ log " $name ($method $url)"
	38	+ # warmup
	39	+ if [[ "$method" == "POST" \|\| "$method" == "PUT" \|\| "$method" == "DELETE" ]]; then
	40	+ oha -z "$WARM" -c "$CONC" -m "$method" -H "Content-Type: application/json" \
	41	+ ${body:+-d "$body"} "$url" >/dev/null 2>&1 \|\| true
	42	+ else
	43	+ oha -z "$WARM" -c "$CONC" "$url" >/dev/null 2>&1 \|\| true
	44	+ fi
	45	+ local jsonf
	46	+ jsonf=$(mktemp /tmp/oha.XXXXXX.json)
	47	+ if [[ "$method" == "POST" \|\| "$method" == "PUT" \|\| "$method" == "DELETE" ]]; then
	48	+ oha -z "$DUR" -c "$CONC" -m "$method" -H "Content-Type: application/json" \
	49	+ --output-format json ${body:+-d "$body"} "$url" > "$jsonf" 2>/dev/null \|\| true
	50	+ else
	51	+ oha -z "$DUR" -c "$CONC" --output-format json "$url" > "$jsonf" 2>/dev/null \|\| true
	52	+ fi
	53	+ local rps p50 p99
	54	+ rps=$("$VENV_PY" -c "import json,sys;
	55	+try:
	56	+ d=json.load(open('$jsonf')); print(round(d['summary']['requestsPerSec'],1))
	57	+except Exception as e: print('NA')" 2>/dev/null)
	58	+ p50=$("$VENV_PY" -c "import json,sys
	59	+try:
	60	+ d=json.load(open('$jsonf')); print(round(d['latencyPercentiles']['p50']/1e6,2),'ms')
	61	+except Exception: print('NA')" 2>/dev/null)
	62	+ p99=$("$VENV_PY" -c "import json,sys
	63	+try:
	64	+ d=json.load(open('$jsonf')); print(round(d['latencyPercentiles']['p99']/1e6,2),'ms')
	65	+except Exception: print('NA')" 2>/dev/null)
	66	+ rm -f "$jsonf"
	67	+ echo "\| $name \| $rps \| $p50 \| $p99 \|"
	68	+}
	69	+
	70	+start_server() {
	71	+ local name="$1" port="$2"; shift 2
	72	+ log "Starting $name on :$port"
	73	+ cd "$ROOT"
	74	+ local logf="/tmp/crud_bench_${name//[^a-zA-Z0-9]/_}.log"
	75	+ "$@" >"$logf" 2>&1 &
	76	+ local pid=$!
	77	+ sleep 3
	78	+ local tries=20
	79	+ while ! curl -s "http://$HOST:$port/items/1" >/dev/null 2>&1; do
	80	+ tries=$((tries-1))
	81	+ if [[ $tries -le 0 \|\| ! -d "/proc/$pid" ]]; then
	82	+ err "$name failed to start (see $logf)"; tail -5 "$logf" >&2
	83	+ kill "$pid" 2>/dev/null \|\| true; return 1
	84	+ fi
	85	+ sleep 1
	86	+ done
	87	+ echo "$pid"
	88	+}
	89	+
	90	+stop_server() {
	91	+ local pid="$1" name="$2"
	92	+ [[ -d "/proc/$pid" ]] && { kill "$pid" 2>/dev/null; wait "$pid" 2>/dev/null; sleep 1; }
	93	+ log "Stopped $name"
	94	+}
	95	+
	96	+BODY='{"name":"widget","qty":7}'
	97	+
	98	+echo ""
	99	+echo "## Real DB-backed CRUD benchmark (recorded $DATE)"
	100	+echo ""
	101	+echo "- Workload: single-row INSERT / SELECT / UPDATE / DELETE, SQLite file (WAL, busy_timeout=5000, 10-conn pool)"
	102	+echo "- Note: single-file SQLite write-serializes (one writer at a time), so write"
	103	+echo " RPS is SQLite-bound (~hundreds) and similar across frameworks; SELECT is the"
	104	+echo " framework-differentiated metric."
	105	+echo "- Tool: \`oha -c $CONC -z $DUR\`"
	106	+echo "- CPU: $(grep 'model name' /proc/cpuinfo \| head -1 \| cut -d: -f2 \| xargs) ($(nproc) threads)"
	107	+echo "- Kernel: $(uname -sr)"
	108	+echo ""
	109	+
	110	+# ---- JustAPI (Python-handler) ----
	111	+PID=$(start_server "JustAPI(py)" "$JA_PORT" "$VENV_PY" "$HERE/crud_justapi.py" python "$JA_PORT") \|\| true
	112	+if [[ -n "${PID:-}" ]]; then
	113	+ echo "### JustAPI — Python-handler CRUD"
	114	+ echo ""
	115	+ echo "\| Operation \| RPS \| p50 \| p99 \|"
	116	+ echo "\|-----------\|-----\|-----\|-----\|"
	117	+ measure "INSERT" POST "http://$HOST:$JA_PORT/items" "$BODY"
	118	+ measure "SELECT" GET "http://$HOST:$JA_PORT/items/1"
	119	+ measure "UPDATE" PUT "http://$HOST:$JA_PORT/items/1" "$BODY"
	120	+ measure "DELETE" DELETE "http://$HOST:$JA_PORT/items/1"
	121	+ echo ""
	122	+ stop_server "$PID" "JustAPI(py)"
	123	+fi
	124	+
	125	+# ---- JustAPI (Rust-native) ----
	126	+PID=$(start_server "JustAPI(native)" "$JA_PORT" "$VENV_PY" "$HERE/crud_justapi.py" native "$JA_PORT") \|\| true
	127	+if [[ -n "${PID:-}" ]]; then
	128	+ echo "### JustAPI — Rust-native CRUD (fast path)"
	129	+ echo ""
	130	+ echo "\| Operation \| RPS \| p50 \| p99 \|"
	131	+ echo "\|-----------\|-----\|-----\|-----\|"
	132	+ measure "INSERT" POST "http://$HOST:$JA_PORT/items" "$BODY"
	133	+ measure "SELECT" GET "http://$HOST:$JA_PORT/items/1"
	134	+ measure "UPDATE" PUT "http://$HOST:$JA_PORT/items/1" "$BODY"
	135	+ measure "DELETE" DELETE "http://$HOST:$JA_PORT/items/1"
	136	+ echo ""
	137	+ stop_server "$PID" "JustAPI(native)"
	138	+fi
	139	+
	140	+# ---- FastAPI + SQLAlchemy ----
	141	+PID=$(start_server "FastAPI" "$FA_PORT" "$VENV_PY" "$HERE/crud_fastapi.py" "$FA_PORT") \|\| true
	142	+if [[ -n "${PID:-}" ]]; then
	143	+ echo "### FastAPI + SQLAlchemy (async)"
	144	+ echo ""
	145	+ echo "\| Operation \| RPS \| p50 \| p99 \|"
	146	+ echo "\|-----------\|-----\|-----\|-----\|"
	147	+ measure "INSERT" POST "http://$HOST:$FA_PORT/items" "$BODY"
	148	+ measure "SELECT" GET "http://$HOST:$FA_PORT/items/1"
	149	+ measure "UPDATE" PUT "http://$HOST:$FA_PORT/items/1" "$BODY"
	150	+ measure "DELETE" DELETE "http://$HOST:$FA_PORT/items/1"
	151	+ echo ""
	152	+ stop_server "$PID" "FastAPI"
	153	+fi
	154	+
	155	+# ---- Robyn ----
	156	+PID=$(start_server "Robyn" "$RO_PORT" "$VENV_PY" "$HERE/crud_robyn.py" "$RO_PORT") \|\| true
	157	+if [[ -n "${PID:-}" ]]; then
	158	+ echo "### Robyn (sync handler, sqlite3)"
	159	+ echo ""
	160	+ echo "\| Operation \| RPS \| p50 \| p99 \|"
	161	+ echo "\|-----------\|-----\|-----\|-----\|"
	162	+ measure "INSERT" POST "http://$HOST:$RO_PORT/items" "$BODY"
	163	+ measure "SELECT" GET "http://$HOST:$RO_PORT/items/1"
	164	+ measure "UPDATE" PUT "http://$HOST:$RO_PORT/items/1" "$BODY"
	165	+ measure "DELETE" DELETE "http://$HOST:$RO_PORT/items/1"
	166	+ echo ""
	167	+ stop_server "$PID" "Robyn"
	168	+fi
	169	+
	170	+# Cleanup bench db
	171	+for ext in "" "-wal" "-shm"; do rm -f "$HERE/crud_bench.sqlite$ext"; done
	172	+log "Done. Paste the tables above into BENCHMARKS.md."
0	173