PRODUCTION-READINESS PLAN (2.0.8 pre-release hardening)

Goal: make JustAPI safe for real DB-backed production use and close the gaps
vs FastAPI/Robyn that the micro-benchmarks hide. This plan is derived from the
2026-07-20 audit (REPORT.md) + live reproductions. Work top-down; each section
is independently testable and gated by cargo test --workspace + pytest.

Conventions (AGENTS.md): Rust-first, typed thiserror errors at crate
boundaries, // SAFETY: on any unsafe, clippy -D warnings, fmt clean,
miri on core if touched.

P0 — Blockers (must fix before ANY release)

P0.1 — [DONE] BUG-2a: normalize_db_url breaks SQLite path resolution

Symptom (reproduced 2026-07-20): app.set_database("sqlite://...") +
app.run() crashes with code 14 / unable to open database file for every
path (relative and absolute). demo_shop reported it as a deadlock; it is a
path-resolution bug.

Root cause: crates/justapi-core/src/db/pool.rs:93 normalize_db_url
only strips the 2-char sqlite:// prefix and passes the remainder verbatim.

• sqlite:///demo_shop.db (3 slashes, relative per SQLAlchemy convention)
  → sqlite:/demo_shop.db → driver reads absolute /demo_shop.db (root).
• sqlite:////tmp/foo.db (4 slashes, absolute) → sqlite://tmp/foo.db
  → mangled double-slash path, also fails near root.

No unprivileged process can write to /demo_shop.db, hence SQLITE_CANTOPEN.

Fix: match the SQLAlchemy/FastAPI convention exactly.

fn normalize_db_url(url: &str) -> String {
    let Some(rest) = url.strip_prefix("sqlite://") else {
        return url.to_string();
    };
    if let Some(abs) = rest.strip_prefix("//") {
        format!("sqlite:/{abs}")          // sqlite:////abs -> sqlite:/abs
    } else if let Some(rel) = rest.strip_prefix('/') {
        format!("sqlite:{rel}")           // sqlite:///rel -> sqlite:rel
    } else {
        format!("sqlite:{rest}")          // sqlite://:memory:, sqlite://./x
    }
}

Preserves existing :memory: and ./explicit cases (fall through else).

Tests: add to pool.rs unit tests next to test_db_kind_from_url:

• sqlite:///foo.db → sqlite:foo.db (relative)
• sqlite:////tmp/foo.db → sqlite:/tmp/foo.db (absolute)
• sqlite://:memory: → sqlite::memory: (unchanged)
• sqlite://./x.db → sqlite:./x.db (unchanged)
  Plus an integration test: spin AnyPool::connect on a temp relative + temp
  absolute path, assert connect succeeds and a SELECT 1 works.

Gate: cargo test db pool + repro script (/tmp/repro_db*.py) now starts
and serves /ping with the DB up.

P0.2 — [DONE] BUG-2b / D3: SQLite has zero concurrency protection by default

Symptom (stress test): DB-backed handler path collapsed to ~15 req/s /
~900 ms p50 while raw query is 1–100 ms (~2500× overhead).

Root cause (confirmed in code): pragmas are opt-in. In
pool.rs:252, the busy_timeout+WAL block only runs if (DbKind::Sqlite, Some(pragmas)) — i.e. only when the caller passes pragmas. The Python
app.set_database(db) default is pragmas=None, wal=False, so a fresh
SQLite pool runs stock defaults: rollback-journal (single writer blocks all
readers/writers) + busy_timeout=0 (immediate SQLITE_BUSY on contention, no
wait). Under the default max_connections=10 pool all hammering one file,
that is constant SQLITE_BUSY. (No retry/backoff loop exists in
justapi-py/justapi-core today, so the slowdown is the driver's own
contention behavior, not a bolted-on retry loop — but the fix below removes the
cause regardless.)

Fix: make busy_timeout + WAL + synchronous=NORMAL unconditional for
SQLite, with user pragmas appended/overriding:

if kind == DbKind::Sqlite {
    let mut all = vec!["PRAGMA busy_timeout=5000".to_string()];
    all.extend(config.pragmas.clone().unwrap_or_else(|| vec![
        "journal_mode=WAL".to_string(),
        "synchronous=NORMAL".to_string(),
    ]));
    opts = opts.after_connect(move |conn, _meta| { /* unchanged body */ });
}

Keep wal=True Python flag working (it already appends journal_mode=WAL;
the unconditional default makes it redundant but harmless).

Tests: integration test asserting a 10-connection concurrent INSERT/SELECT
load on a SQLite file sustains > some floor (e.g. > 1k RPS @ -c10) without
SQLITE_BUSY. Re-run demo_shop checkout flow under oha -c 50 and confirm
p50 in the low-ms range, not ~900 ms.

Gate: demo_shop 23-endpoint app starts against shop.db and sustains
concurrent load (D1/D3/D4 all closed, not just worked around).

P0.3 — [DONE] BUG-1: top-level "status" key empties the response body

Symptom (reproduced): handler returning {"status":"ok","products":5}
→ 200 with empty body. demo_shop hit this on health/order endpoints.

Root cause: crates/justapi-py/src/native/handlers.rs:322-348
serialize_response branch 2 treats any dict containing a "status" key as
a legacy {"status","body","headers"} envelope. With no "body" key, body
becomes Vec::new(). A normal business payload with a "status" field is
silently dropped.

Fix: only enter the legacy-envelope branch when the dict actually looks
like an envelope — require an explicit sentinel, e.g. a "__response__": true
key, OR require both status and body/headers. Simplest correct rule:
treat as envelope only if it has "body" (the thing that distinguishes a
response envelope from a data dict). A plain {"status": ...} data dict falls
through to normal JSON serialization.

let is_envelope = has_body; // drop `|| has_status`

Add a regression test: {"status":"ok","products":5} round-trips unchanged;
{"status":201,"body":"x"} still works as an envelope.

Gate: repro script shows with_status body: b'{"status":"ok","products":5}'.

P1 — Correctness / DX gaps

P1.1 — [DONE] BUG-3: body_schema= delivers raw bytes, not a Schema instance

Handlers must re-parse request.json() manually (demo_shop workaround). The
native fast path already validates via the schema; the Python handler path
should receive a validated, instantiated object (or at least the parsed dict),
not raw bytes. Decide: pass parsed dict when body_schema is a Schema/pydantic
model; keep raw bytes only for legacy callables. Add test asserting handler
receives a dict (or model) for a Schema-registered route.

P1.2 — [DONE] test_graphql.py 404s (graphql() builder implemented)

Added app.graphql() route builder method and _builtin_graphql wrapper in app.py backed by justapi_core::graphql. test_graphql.py passes 200 OK for GraphiQL UI and query execution.

P1.3 — [DONE] Test suite 100% green

Updated route handling in _JustAPIApp to support route overriding so user routes can override built-in routes cleanly. Full pytest integration test suite passes 100% green (137 passed, 1 skipped).

P1.4 — [DONE] cargo fmt --check gate drift

Standardized rustfmt.toml without nightly-only flags. Both cargo fmt --check and cargo clippy --workspace --tests -- -D warnings pass clean.

P2 — Honest benchmarking (close the "real life" gap)

P2.1 — [DONE] Real DB-backed CRUD benchmark vs FastAPI+SQLAlchemy and Robyn

The micro-benchmarks (echo/validate) overstate production reality by ~1000× on
DB paths. Added benchmarks/crud_justapi.py (Python-handler + Rust-native),
crud_fastapi.py (FastAPI+SQLAlchemy async), crud_robyn.py (Robyn+sqlite3),
and bench_one.py (oha JSON driver). Ran oha -c 10 -z 5s against a SQLite
file (WAL, busy_timeout=5000, 10-conn pool). Results appended to BENCHMARKS.md
(SELECT is solid & shows JustAPI/Robyn >> FastAPI; writes are SQLite-bound).
The benchmark surfaced a blocking defect: JustAPI's Python-handler DB write
path does not await/commit writes under concurrency (INSERT flies at 362k RPS,
UPDATE/DELETE collapse to ~5–15 RPS). Tracked as P2.2.

P2.2 — [DONE] Fix Python-handler DB write path (durability under concurrency)

The CRUD benchmark exposed that crates/justapi-py/src/database.rs query()
(used for writes) ran via py.detach + rt.block_on, which deadlocks the DB
runtime from the server's dispatch thread — the connection pool exhausts
(busy_timeout hit) and writes silently drop (1–2 of ~50 INSERTs persisted at
-c 50). Fixed in run_blocking(): release the GIL with py.detach but run the
future via rt.spawn + an mpsc channel wait (no re-entrant block_on). Now
100 concurrent INSERTs via the test client persist 100/100, and a 100-thread
direct app.db.execute burst persists 200/200 with zero errors. Durability test
added in test_db_concurrent.py. The Rust-native CRUD path remains the fastest
write route. See ADR-074 (revised) + BENCHMARKS.md P2.2.

P3 — Distribution

P3.1 — PyPI publish

Portable manylinux_2_28 wheel is built and verified:
target/wheels/justapi-2.0.0-cp311-abi3-manylinux_2_28_x86_64.whl
(built inside quay.io/pypa/manylinux_2_28_x86_64 via maturin build --release). twine check PASSED; the wheel installs + imports cleanly in a
fresh venv. Remaining: twine upload with MATURIN_PYPI_TOKEN (no token
present in this environment — blocked on credentials). Also consider building
musllinux + aarch64 wheels for full coverage before publishing.

P3.2 — Unfreeze GPU/inference (Phase 52)

Keep frozen until 2.0.8 per ADR-067. Structural benchmarks are topology proofs,
not GPU throughput — do not market as tokens/sec until a real-weight CUDA run.

P4 — Built-in validator hardening (Gap #3)

P4.1 — [DONE] Schema model + Field constraints + nested $ref/$defs

justapi.Schema now emits a real JSON Schema: Field supports
default, gt, ge, lt, le, min_length, max_length, regex, format, enum, description; nested Schema fields (incl. list[OtherSchema]) are lifted into
top-level $defs and referenced via $ref. Validation runs in Rust (jsonschema
0.46, default-features=false) with zero Python round-trips on the hot path.
Direct validate_value enforced format from the start (pyo3-side format
registry in crates/justapi-py/src/native/app.rs).

P4.2 — [DONE] Fix server-path format enforcement (regression)

Root cause: request-body validation on the live HTTP path uses
justapi_core::validate::CompiledValidator (handlers.rs fast path), which
built validators with plain jsonschema::options().build(...) and never
registered the format checkers. So format: "email" violations were silently
accepted (200) at the server boundary, even though the in-process validate_value
path rejected them (422). The fix moves the format registry into justapi-core
(validate.rs → build_validator, registering email/uri/uuid/date-time/date/
hostname/ipv4 + should_validate_formats(true)), making core the single source
of truth for both one-shot and precompiled validation. Verified end-to-end: a
Schema-validated route now returns 422 for email:"nope" and for a nested
zip:"abc" over HTTP, while a valid payload returns 201. Regression test added
in test_schema_hardened.py::test_server_path_enforces_format.

P5 — DB pool saturation fast-fail (Gap #2)

P5.1 — [DONE] Pooled-connection acquire now fails fast under saturation

Why: the default pool acquire_timeout is 30 s. Under load (DB pool smaller
than peak concurrency, e.g. max_connections=1 behind a burst, or a slow
transaction pinning the only connection), every blocked request would queue for
up to 30 s, stacking timeouts and delaying recovery. A saturated pool should
degrade by rejecting new work immediately (503 + Retry-After), not by waiting.

What changed:

• crates/justapi-core/src/db/pool.rs: added DatabaseConfig.request_acquire_timeout
  (default 3 s). AnyPool::connect sets the sqlx acquire_timeout to
  request_acquire_timeout.or(acquire_timeout).unwrap_or(3s). Added
  AnyPool::begin_request() -> Result<Transaction, DbAcquireError> and a typed
  DbAcquireError (TimedOut / PoolClosed / Other) with From<sqlx::Error>.
• crates/justapi-core/src/lib.rs: service_unavailable_response() (503 +
  Retry-After: 1) and db_error_response(err) (503 on TimedOut/PoolClosed,
  else 500).
• crates/justapi-py/src/native/handlers.rs: both write-path handlers call
  pool.begin_request() and map saturation → 503 via db_acquire_error_response().
• crates/justapi-core/src/server/crud.rs: all 4 DB error arms return
  db_error_response(&e) (503 on saturation).
• crates/justapi-py/src/database.rs: Database.request_acquire_timeout is now a
  plain f64 (default 3.0) and app.set_database(db, request_acquire_timeout=...)
  propagates it.

FFI gotcha (record so we don't relearn it): a Database instance crossing
the FFI to Rust hits a pyo3 reconstruction quirk — Option<f64> fields come back
as None/defaults and even #[pyclass] self field reads via getattr/
call_method0("config_dict") returned the default 3.0 for
request_acquire_timeout (the value passed from Python was 1.0). The reliable
round-trip is: Python builds the Database, then hands Database.config_dict()
(a plain dict, read in Python where field values are correct) across the FFI,
and the Rust side extracts the dict. config_dict() is the source of truth for
the dict shape.

Verified: test_db_saturated.py boots a real HTTP server, one-connection pool,
request_acquire_timeout=1.0; a /slow handler holds the only connection for 3 s
while a concurrent /other DB write returns 503 in ~1.0 s (not 30 s).

P6 — [DONE] Untrusted-input fuzzing (Gap #5)

Added fuzz_pipeline fuzz target (fuzz/fuzz_targets/fuzz_pipeline.rs) testing JSON schema validation, query parsing, and router resolution against untrusted bytes. Documented security rationale in DECISIONS.md ADR-075.

Execution order

1. P0.1 (normalize_db_url) + P0.2 (unconditional pragmas) — unblock DB startup
   & concurrency. Verify demo_shop starts + sustains load.
2. P0.3 ("status" envelope) — correctness blocker.
3. P1.1 (body_schema) — DX correctness.
4. P1.2–P1.4 — test/dev hygiene.
5. P2.1–P2.2 — honest benchmark vs FastAPI/Robyn.
6. P3.1 — PyPI publish.

Definition of Done (production-ready)

• demo_shop 23-endpoint app starts against SQLite/Postgres and sustains
  concurrent load with p50 in low-ms (D1/D3/D4 closed — 29/29 flow).
• {"status": ...} payloads return intact (BUG-1 fixed + test; status-only
  envelopes still set the code).
• body_schema routes receive parsed data, not raw bytes (BUG-3 fixed).
• pytest 100% green (137 passed, 1 skipped); cargo test --workspace + clippy -D warnings clean; cargo fmt --check clean.
• Published real DB-backed CRUD benchmark vs FastAPI+SQLAlchemy + Robyn in
  BENCHMARKS.md (P2.1). Found + fixed a blocking write-path defect (P2.2)
  — the Python-handler DB write path now awaits/commits durably under
  concurrency (run_blocking uses py.detach + rt.spawn + channel wait).
• Built-in justapi.Schema validator (P4.1): Field constraints + nested
  $ref/$defs + array-of-model, validated in Rust with zero Python
  round-trips.
• Server-path format enforcement fixed (P4.2): format keywords are now
  asserted by justapi_core for both one-shot and precompiled validation;
  live HTTP routes reject format violations (422), not just in-process
  validate_value. Regression test in test_schema_hardened.py.
• DB pool fast-fail under saturation (P5.1): request_acquire_timeout (3 s
  default) turns a saturated pool into immediate 503 + Retry-After instead
  of 30 s queue-and-timeout. Verified by test_db_saturated.py.
• Untrusted-input fuzzing target (P6): fuzz_pipeline target created and compiled in fuzz/. Security rationale documented in DECISIONS.md ADR-075.
• Wheel published to PyPI (manylinux_2_28).