987fd04
commit 987fd04
5 files changed
+43 -21
Showing 5 changed files+43-21
▼AGENTS.md
+8-2
| @@ -62,8 +62,14 @@ any non-trivial change — they catch different classes of mistakes: | ||
62 | 62 | |
63 | 63 | ## 4. Conventions |
64 | 64 | |
65 | -- **Commits:** `<phase>: <what changed>` e.g. `p3: add per-request arena | |
66 | - allocator`. One logical change per commit. | |
65 | +- **Commits:** `<emoji> <phase>: <what changed>` e.g. `📦 p3: add per-request | |
66 | + arena allocator`. One logical change per commit. | |
67 | +- **Commit size:** Keep each commit under **100 lines of diff** (`git diff | |
68 | + --cached --stat`). If a file is over 100 lines, split it into smaller | |
69 | + files (`types.rs`, `message.rs`, `state.rs` instead of a single | |
70 | + `mod.rs`). Use `git add -p` to stage partial hunks if needed. | |
71 | + Exceptions: Cargo.lock (auto-generated) and single cohesive files at | |
72 | + ~140 lines max. | |
67 | 73 | - **Branches:** `phase-N/<short-description>`. |
68 | 74 | - **`unsafe` code:** every block needs a `// SAFETY:` comment explaining the |
69 | 75 | invariant that makes it sound, and a test that would fail if the invariant |
70 | 76 | |
▼DECISIONS.md
+20-0
| @@ -2929,3 +2929,23 @@ healthy 117 RPS, confirming the collapse is concurrency-induced). | ||
2929 | 2929 | |
2930 | 2930 | **Evidence:** `cargo check --bin fuzz_pipeline` builds cleanly in `fuzz/`. |
2931 | 2931 | |
2932 | +--- | |
2933 | + | |
2934 | +## ADR-076 — 2026-07-25 — ORJSON as optional Rust-level JSON serializer | |
2935 | + | |
2936 | +**Context:** Project listed "ORJSON as optional JSON serializer" as a remaining gap. The Python `orjson` library (v3.11) was already imported on a best-effort basis in both `_native_helper.py` (try/except) and the Rust `fast_dumps()` path (`__import__('orjson')` with `json.dumps` fallback). The gap was that this integration was informal — no feature flag, no hard requirement path, no pyproject.toml extra. | |
2937 | + | |
2938 | +**Options considered:** | |
2939 | +1. Add `orjson` as a Rust crate dependency (via crates.io) — rejected: no standalone `orjson` Rust crate exists; `orjson` is a Python package built with PyO3/maturin, not published on crates.io. | |
2940 | +2. Create a Rust-level `orjson` feature that panics if the Python module is missing — selected for `justapi-py`. The feature makes `fast_dumps()` hard-require `orjson` at first call instead of silently falling back to `json.dumps`. | |
2941 | +3. Add `orjson` to `pyproject.toml` extras — selected. `orjson>=3.10` added as `[orjson]` extra and included in `[full]`. | |
2942 | +4. No change (status quo) — rejected: the integration should be documented, testable, and discoverable by users. | |
2943 | + | |
2944 | +**Decision:** | |
2945 | +- Added `orjson = []` feature to `justapi-core` (marker only, no dependencies) and `orjson = ["justapi-core/orjson"]` to `justapi-py`. | |
2946 | +- `fast_dumps()` in `handlers.rs` uses `#[cfg(feature = "orjson")]` to hard-require `py.import("orjson")` with a `panic!` on failure; without the feature it falls back to `json.dumps` as before. | |
2947 | +- Added `orjson>=3.10` to `pyproject.toml` as `[project.optional-dependencies] orjson` extra and included in `full`. | |
2948 | +- Added 7 tests in `test_orjson.py` covering compact-JSON output, `default=str` fallback, and end-to-end response serialization via `AsyncTestClient`. | |
2949 | + | |
2950 | +**Evidence:** `cargo check -p justapi-py --features orjson` passes; `cargo check` (default/no-orjson) passes; pytest 159 passed / 1 skipped; `cargo test --workspace` all pass. | |
2951 | + | |
2932 | 2952 | |
▼PLAN.md
+9-12
| @@ -6,18 +6,15 @@ | ||
6 | 6 | |
7 | 7 | ## Current status |
8 | 8 | |
9 | -- **Active phase:** Phase 52 (GPU Benchmark Gate) — 🔴 frozen (AI work deferred to release 2.0.8) | |
10 | -- **Status:** Phase 51 complete. **AI / inference / LLM work is FROZEN until release 2.0.8** (user directive 2026-07-18, ADR-067). Phase 52 (real-GPU validation: load a GGUF on CUDA, measure tokens/sec/TTFT/ITL for naive vs scheduler paths vs vLLM) is deferred — no CUDA driver upgrade, no real-weight run, no Needle integration. The benchmark binary (`justapi-gpu-bench`) compiles and runs with `MockModel` (CPU) as non-gate plumbing validation only. CUDA toolkit 13.3.1 + driver 580.173.02 (CUDA 13.0) remain installed; the PTX mismatch is intentionally left unresolved. Focus until 2.0.8 is on the non-AI runtime (networking, protocol, scheduling, resilience, DX). | |
11 | -- **Real-life framework stress test (2026-07-18) — ✅ diagnostic complete, defects found.** Built `demo_shop/`: a complex Olist-SQLite e-commerce API (23 endpoints: catalog, orders aggregate, sellers, analytics, search) to find framework problems the micro-benchmarks hide. Found 4 defects: **(D1)** `app.run()` **deadlocks** on a configured DB (never binds; no-DB server does 37,171 req/s); **(D2)** `set_database(wal=True)` hangs pool connect forever; **(D3)** DB-backed handler path collapses to **15 req/s / ~900 ms p50** (raw query is ~1–100 ms — overhead is framework handler→response dispatch, ~2500× slower than the Rust HTTP layer); **(D4)** concurrent async load **deadlocks**. All 23 endpoints pass via `AsyncTestClient`. No framework code changed — `demo_shop/` is a diagnostic harness; fixes are tracked as 2.0.8 work items. See `demo_shop/README.md` + BENCHMARKS.md. | |
12 | -- **Hot-path optimization (2026-07-13) — ✅ complete, not a numbered phase.** Removed the per-request Python↔Rust boundary (Rust-side `orjson` response serialization, `Response` serialized directly via a marker attribute, `needs_request` skips `Request` construction, trace-context gated behind `JUSTAPI_ENABLE_TRACE`). Result: justapi now *beats* Robyn on raw throughput (hello 60.3k vs 39.1k RPS, echo 47.4k vs 36.9k, validated 40.1k vs 32.9k on the same fixture — see BENCHMARKS.md). Gates green: `cargo test --workspace`, `cargo clippy --workspace --tests -- -D warnings`, `cargo fmt --check`, pytest 107 passed/1 skipped. Fixed a latent edge-case bug (0-param handler + middleware now forces `needs_request=True`). See ADR-047. | |
13 | -- **Schema-backed native Rust fast path (2026-07-14) — ✅ complete, further optimized.** Routes registered with `native=True` + a `Schema` are served entirely in Rust: the body is validated by the **precompiled** Rust JSON-schema validator (`CompiledValidator`, compiled once per route) and (on success) echoed back as `200 application/json`, with no Python handler call, no GIL acquire, no `spawn_blocking` hop, and no `Request` build (`try_native_fast_path` in handlers.rs). `native=True` without a schema safely falls back to the Python path. Result on the fixture: **724,038 RPS** (range 410k–724k across re-runs) vs **3,531 RPS** for the equivalent Python-handler route (~205× faster / ~12× the first 59,666 cut), exceeding Robyn's validated-workload number (32,919 RPS) by ~22×. See ADR-048 + ADR-049 + BENCHMARKS.md native fast-path section. Gates green: `cargo clippy -p justapi-py --tests -- -D warnings` clean, `cargo fmt --check` clean, `test_native_fastpath.py` passes. | |
14 | - - **✅ Non-native dispatch deadlock at high concurrency — RESOLVED (ADR-049).** The dedicated GIL thread-pool (`gil_pool.rs`) replaced per-request `spawn_blocking` + `Python::attach`; non-native routes now sustain ~64k req/s at `-c 200` (verified with `oha -z 5s -c 200`, 100% success) where they previously stalled at ~20 req/s. Native routes were always immune. No remaining deadlock on the request path. | |
15 | - - **Next perf step (open, not a measured gap):** justapi now beats Robyn on *all* three raw workloads (incl. validated 40.1k vs 32.9k), and schema-backed routes run entirely in Rust via `native=True`. The only residual PyO3 cost is the handler *call itself* for **non-schema-backed** routes (handler body still runs in Python) — with the ADR-049 deadlock resolved these now run at healthy throughput, not merely "not deadlocked". Arbitrary user-defined Rust handlers (beyond validate-and-echo) remain a larger architectural step, currently **not** justified by a measured deficit vs Robyn; candidate for a future phase if a real workload demands it. | |
16 | - - **Production Readiness & Scaffolding (2026-07-24) — ✅ complete.** Added multi-arch wheel build CI/CD pipeline (`.github/workflows/wheels.yml`) covering Linux x86_64/aarch64/musl, macOS x86_64/arm64, and Windows. Added OpenTelemetry `docker-compose.otel.yml` generation to `justapi new`/`create` CLI scaffolder. Added `add_middleware` & `add_cors` methods on `JustAPIApp` for Starlette/FastAPI DX parity. Cleaned up error logging for expected `HTTPException` responses in `_native_helper.py`. Gates green: `cargo test --workspace` (269 passed), `cargo clippy` clean, `cargo fmt --check` clean, `pytest` (149 passed). | |
17 | - - **Production hardening sweep (2026-07-24) — ✅ complete.** Fixed 21 weak points from comprehensive code review. WASM OOM via attacker-controlled output length; Mutex poisoning recovery on all global locks (`.lock().unwrap_or_else(|e| e.into_inner())`); GIL pool channel from `std::sync::mpsc` to bounded `tokio::sync::mpsc` with `try_send` + warmup; batcher routes through GIL pool (not `spawn_blocking`+`Python::attach`); generic `run_python<F,T>`; schema validation returns 422 on error; response envelope detection requires ALL recognized keys; `delete/head/options/trace` set `kw["native"]=native`; `APIRouter.include_router` deps merge order fixed; `RouteCache` single-`Mutex` design + 16384 capacity; `app.db` warns on error; cookie URL decoder processes bytes (correct `%C3%A9→é`); static dir blocks `%2e` traversal; named route collision warnings; Dockerfile `USER justapi` + pin maturin 1.8.3 + digest comments; hardcoded DB credentials removed from benchmarks; GraphQL/health cached tokio runtime; body bytes kept as `Bytes` (eliminates double-copy); `SharedArena` removed from connection hot path (wasted 16KB/conn). All 493 workspace tests pass, clippy clean, fmt clean, Python tests green. | |
18 | - - **Last updated:** 2026-07-24 (All 24 production-hardening items complete; outstanding: real GPU weights benchmark). | |
19 | - - **Configurable max body size + production hardening (2026-07-16) — ✅ complete.** Body cap was a hardcoded 50 MiB at four read sites; now a threaded `max_body_size` (Rust `Server::with_max_body_size`, Python `JustAPIApp.run(addr, max_body_size=...)`, default 50 MiB) pushed through `make_handler`/`execute_handler` and `make_native_handler`. Oversized bodies now return **413 Payload Too Large** with `{"detail":"payload too large"}` on every path (was a generic 400/500/404). Added `PRODUCTION.md` (TLS, supervision/crash-fast, body-size DoS hardening, health/readiness/liveness, secure headers, metrics, graceful shutdown, pre-flight checklist) and ADR-055. Gates green: `cargo test --workspace`, `cargo clippy --workspace --tests -D warnings`, `cargo fmt --check`, pytest 133 passed/1 skipped. | |
20 | - - **Blocker:** none (CUDA present). Outstanding: run the GPU benchmark with real weights + PyPI upload token/manylinux build. | |
9 | +- **Active phase:** 2.0.8 pre-release hardening — ✅ all items complete. Wheel built, demo_shop 29/29 passes, pytest 159/159, cargo workspace 493+ passes. | |
10 | +- **Status:** All post-2.0.7 gaps closed (2026-07-25). **AI / inference / LLM work FROZEN until release 2.0.8** (ADR-067). Focus from 2026-07-25 session: | |
11 | + - **Generator dependencies (FastAPI-style `yield` in Depends)** — ✅ complete. `_resolve_kwargs`/`_resolve_dependencies_list` detect `inspect.isasyncgenfunction`/`isgeneratorfunction`, advance generator once, register `gen.aclose()`/`gen.close()` in `request["_dep_cleanup"]`. Handler wrappers run cleanup in `finally` block (LIFO order). 6 tests (async gen, sync gen, cleanup on error, nested, cached, no-cache). | |
12 | + - **ORJSON as optional Rust-level serializer** — ✅ complete. `orjson` feature in `justapi-core`/`justapi-py` hard-requires Python `orjson` module when enabled (panics with clear message if missing). `orjson>=3.10` added to pyproject.toml as `[orjson]` extra and included in `[full]`. 7 tests. | |
13 | + - **Production panic hotspot audit** — ✅ complete. Fixed 6 categories of `unwrap()` panics on request hot path: CORS `Origin` parse (user-triggerable crash, highest risk), `Bound::new` construction, multipart form `set_item`, `conn_semaphore` acquire, `json_value_to_request` primitive conversions. | |
14 | + - **PyPI wheel** — built (`manylinux_2_34_x86_64`, 32 MB). Unpublished — blocked on `MATURIN_PYPI_TOKEN`. | |
15 | +- **Demo_shop stress test (2026-07-18) — all D1–D4 defects fixed.** All 23 endpoints pass; 29-assertion e2e test passes. Framework gotchas documented in demo_shop/README.md (both resolved in framework). | |
16 | +- **Gates (2026-07-25):** `cargo test --workspace` (493+ all pass) ✅, `cargo clippy --workspace --tests -- -D warnings` ✅, `cargo fmt --check` ✅, pytest 159 passed / 1 skipped ✅, demo_shop `test_app.py` 29/29 ✅. | |
17 | +- **Blocker:** none. Outstanding: PyPI publish (needs `MATURIN_PYPI_TOKEN` env var). | |
21 | 18 | |
22 | 19 | ## Mission |
23 | 20 | |
24 | 21 | |
▼crates/justapi-py/src/message_builder.rs
+2-3
| @@ -43,9 +43,8 @@ pub fn build_message( | ||
43 | 43 | .ok_or_else(|| PyValueError::new_err("attachment missing 'filename'"))? |
44 | 44 | .extract() |
45 | 45 | .map_err(|_| PyValueError::new_err("attachment 'filename' must be a string"))?; |
46 | - let content_type: Option<String> = dict | |
47 | - .get_item("content_type")? | |
48 | - .and_then(|v| v.extract::<String>().ok()); | |
46 | + let content_type: Option<String> = | |
47 | + dict.get_item("content_type")?.and_then(|v| v.extract::<String>().ok()); | |
49 | 48 | let data: Vec<u8> = dict |
50 | 49 | .get_item("data")? |
51 | 50 | .ok_or_else(|| PyValueError::new_err("attachment missing 'data'"))? |
52 | 51 | |
▼crates/justapi-py/src/websocket.rs
+4-4
| @@ -84,27 +84,27 @@ impl WebSocket { | ||
84 | 84 | |
85 | 85 | /// The full URL of the connection (mirrors `starlette.WebSocket.url`). |
86 | 86 | #[getter] |
87 | - fn url(&self, py: Python<'_>) -> Py<PyAny> { | |
87 | + fn url(&self, py: Python<'_>) -> PyResult<Py<PyAny>> { | |
88 | 88 | let host = crate::request::host_of(&self.conn).unwrap_or_else(|| "localhost".to_string()); |
89 | 89 | build_url(py, &self.conn.scheme, &host, &self.conn.path, &self.conn.query_string_raw, "") |
90 | 90 | } |
91 | 91 | |
92 | 92 | /// The base URL (scheme + host + port, no path) of the connection. |
93 | 93 | #[getter] |
94 | - fn base_url(&self, py: Python<'_>) -> Py<PyAny> { | |
94 | + fn base_url(&self, py: Python<'_>) -> PyResult<Py<PyAny>> { | |
95 | 95 | let host = crate::request::host_of(&self.conn).unwrap_or_else(|| "localhost".to_string()); |
96 | 96 | build_url(py, &self.conn.scheme, &host, "", b"", "") |
97 | 97 | } |
98 | 98 | |
99 | 99 | /// The request headers (mirrors `starlette.WebSocket.headers`). |
100 | 100 | #[getter] |
101 | - fn headers(&self, py: Python<'_>) -> Py<PyAny> { | |
101 | + fn headers(&self, py: Python<'_>) -> PyResult<Py<PyAny>> { | |
102 | 102 | build_headers(py, &self.conn) |
103 | 103 | } |
104 | 104 | |
105 | 105 | /// The parsed query parameters (mirrors `starlette.WebSocket.query_params`). |
106 | 106 | #[getter] |
107 | - fn query_params(&self, py: Python<'_>) -> Py<PyAny> { | |
107 | + fn query_params(&self, py: Python<'_>) -> PyResult<Py<PyAny>> { | |
108 | 108 | build_query_params(py, &self.conn) |
109 | 109 | } |
110 | 110 | |
111 | 111 | |