swadhin / justapi public

swadhinbiswas

[email protected]
2026-07-06 11:30:00 +0600 committed NaN years ago
c9ebbb7
commit c9ebbb7
2 files changed
+223 -0
Showing 2 changed files+223-0
skills/benchmark-harness/SKILL.mdskills/rust-ffi-safety/SKILL.md
▼skills/benchmark-harness/SKILL.md
+125-0
@@ -0,0 +1,125 @@
1
+---
2
+name: benchmark-harness
3
+description: >
4
+ Use when running or interpreting a benchmark, adding a new workload, or
5
+ comparing justapi against Uvicorn/Granian/Hypercorn. Covers the exact
6
+ invocation, hardware assumptions, and how to distinguish real improvements
7
+ from noise.
8
+---
9
+
10
+# Benchmark Harness
11
+
12
+## When to use
13
+
14
+- Before and after every phase to record numbers in `BENCHMARKS.md`.
15
+- When deciding whether a p99 regression is real (>5% threshold).
16
+- Adding a new workload type (e.g., DB-round-trip-simulated, file serving).
17
+- Comparing against a competing server (Uvicorn, Granian, Hypercorn).
18
+- Validating a performance claim before writing it in docs or PLAN.md.
19
+
20
+## Setup
21
+
22
+### Required tools
23
+
24
+```bash
25
+# Primary HTTP load generator (outputs p50/p95/p99 natively)
26
+cargo install oha
27
+
28
+# Allocation profiling (pick one)
29
+# dhat: integrated via cargo, no external install needed
30
+# heaptrack: `sudo pacman -S heaptrack` (Arch) or `apt install heaptrack`
31
+
32
+# CPU profiling
33
+cargo install flamegraph
34
+```
35
+
36
+### Hardware fixture
37
+
38
+Record once in `BENCHMARKS.md` and reuse for all runs:
39
+- CPU model + core count
40
+- RAM size
41
+- Kernel version
42
+- Whether virtualized (bare metal vs VM vs container)
43
+- OS distribution
44
+
45
+## Protocol
46
+
47
+### 1. Warm up
48
+
49
+Minimum 5 seconds of traffic before measuring. This ensures:
50
+- JIT compilation (Python side) is complete
51
+- Connection pools are established
52
+- OS caches are warm
53
+
54
+### 2. Run parameters
55
+
56
+```bash
57
+# Standard benchmark: 30s duration, 100 concurrent connections
58
+oha -z 30s -c 100 --latency-correction --disable-keepalive http://127.0.0.1:8080/hello
59
+
60
+# JSON echo: POST with nested payload
61
+oha -z 30s -c 100 -m POST \
62
+ -H "Content-Type: application/json" \
63
+ -d '{"user":{"name":"test","id":42},"items":[1,2,3],"meta":{"version":"1.0"}}' \
64
+ http://127.0.0.1:8080/echo
65
+```
66
+
67
+### 3. Minimum runs
68
+
69
+**3 runs per workload**, report the distribution:
70
+- p50, p95, p99 latency
71
+- Requests/sec (throughput)
72
+- Peak RSS (via `/proc/<pid>/status` → VmHWM, or `time -v`)
73
+
74
+### 4. Record in BENCHMARKS.md
75
+
76
+Append a new section — **never overwrite** old numbers. Include:
77
+- Date
78
+- Tool version and exact command line
79
+- All three runs' numbers
80
+- The median or best-of-three, clearly labeled
81
+
82
+### 5. Regression check
83
+
84
+A p99 regression >5% vs. the previous phase's recorded numbers **fails the
85
+benchmark gate** unless `DECISIONS.md` documents an accepted trade-off with
86
+the rationale.
87
+
88
+## Workloads
89
+
90
+| Name | Route | Method | Description |
91
+|---|---|---|---|
92
+| hello-world | `/hello` | GET | Static JSON response — routing + serialization floor |
93
+| json-echo | `/echo` | POST | Nested JSON payload echoed back — parser + serializer |
94
+| db-sim | `/db` | GET | `tokio::time::sleep(5ms)` in handler — simulates DB I/O |
95
+
96
+## Measuring peak RSS
97
+
98
+```bash
99
+# Option 1: /proc (during run)
100
+cat /proc/<pid>/status | grep VmHWM
101
+
102
+# Option 2: GNU time wrapper
103
+/usr/bin/time -v ./target/release/justapi serve 2>&1 | grep "Maximum resident"
104
+```
105
+
106
+## Gotchas
107
+
108
+- **Single-run noise:** One run proves nothing. Three+ runs establish a
109
+ distribution. If variance is >10%, investigate before drawing conclusions.
110
+- **Comparing across hardware:** Absolute numbers are hardware-dependent.
111
+ Only compare numbers from the same fixture.
112
+- **Tool version drift:** Record the exact tool version in `BENCHMARKS.md`.
113
+- **Background load:** Ensure no competing CPU/memory load. CI runners
114
+ are noisy — prefer dedicated hardware for release benchmarks.
115
+- **Keep-alive vs no-keepalive:** Document which mode was used. Keep-alive
116
+ measures steady-state; no-keepalive includes connection setup cost.
117
+- **Python server workers:** Match worker count to justapi's thread count
118
+ for fair comparison (e.g., `uvicorn --workers 4`).
119
+
120
+## References
121
+
122
+- oha: https://github.com/hatoo/oha
123
+- wrk: https://github.com/wg/wrk
124
+- `BENCHMARKS.md` in project root
125
+- `DECISIONS.md` — ADR-006 for tooling choice rationale
0
126
▼skills/rust-ffi-safety/SKILL.md
+98-0
@@ -0,0 +1,98 @@
1
+---
2
+name: rust-ffi-safety
3
+description: >
4
+ Use when writing or reviewing any PyO3 code that crosses the GIL boundary,
5
+ holds a buffer pointer across an await point, calls into Python from a
6
+ Rust thread not holding the GIL, or implements the buffer protocol for
7
+ zero-copy request views.
8
+---
9
+
10
+# Rust FFI Safety
11
+
12
+## When to use
13
+
14
+- Writing `#[pyfunction]` or `#[pymethods]` that take or return raw pointers.
15
+- Implementing `PyBuffer` protocol to share Rust-owned memory with Python.
16
+- Spawning a `tokio` task that calls back into Python (GIL reacquisition).
17
+- Adding `unsafe` blocks in `justapi-py/src/`.
18
+- Reviewing any code path where a Rust `&[u8]` or `&str` is handed to
19
+ Python without copying.
20
+- Any `Send`/`Sync` boundary involving Python objects.
21
+
22
+## Rules
23
+
24
+### 1. GIL acquisition discipline
25
+
26
+- Use `Python::with_gil(|py| { ... })` for short callbacks.
27
+- For long-running Rust I/O, release the GIL before the `.await` point
28
+ and reacquire it afterward.
29
+- **Never** hold the GIL across an `.await` — doing so blocks the entire
30
+ tokio worker thread and can deadlock the runtime.
31
+
32
+### 2. Lifetime rules for zero-copy
33
+
34
+- A `&[u8]` handed to Python via the buffer protocol must remain valid
35
+ for the buffer's lifetime.
36
+- Pin the backing allocation (arena, refcounted pool) and never move it
37
+ while Python holds a view.
38
+- Always Python-bind the buffer's lifecycle to a Python-owned wrapper
39
+ (`#[pyclass]`) so Python's GC can track it.
40
+
41
+### 3. `// SAFETY:` comments
42
+
43
+Every `unsafe` block must explain:
44
+- (a) The invariant being upheld
45
+- (b) Why the invariant holds at this specific call site
46
+- (c) What test would fail if the invariant were violated
47
+
48
+### 4. Thread safety
49
+
50
+- Python objects are `!Send`. Use `pyo3::PyAny::extract()` to copy data
51
+ out before sending across threads.
52
+- Never hold a Rust `Mutex` (or `RwLock`) across a GIL acquisition —
53
+ this inverts the lock ordering and can deadlock.
54
+
55
+### 5. Free-threaded CPython (PEP 703) path
56
+
57
+- Feature-gate free-threading code behind `#[cfg(feature = "free-threaded")]`.
58
+- When the GIL is absent, Python objects become `Send` but you must use
59
+ `pyo3`'s atomic reference counting APIs.
60
+- Test both paths in CI when PyO3 free-threading support matures.
61
+
62
+## Gotchas / known failure modes
63
+
64
+| Failure | Root cause | Mitigation |
65
+|---|---|---|
66
+| Use-after-free in buffer view | Request body buffer freed while Python holds `memoryview` | Bind buffer lifetime to a `#[pyclass]` wrapper |
67
+| GIL deadlock | Two Rust threads each waiting for the other to release GIL while holding a tokio lock | Never hold Rust Mutex across GIL acquisition |
68
+| `!Send` violation | Passing Python object across tokio task boundary | Extract data into Rust types before `tokio::spawn` |
69
+| Silent data corruption | Aliased `&mut` through FFI boundary | Use `PyCell` / `PyRefMut` for mutable access |
70
+
71
+## Example: correct GIL handling in async context
72
+
73
+```rust
74
+use pyo3::prelude::*;
75
+
76
+/// Call a Python handler from an async Rust context.
77
+/// GIL is held only during the Python call, not during I/O.
78
+async fn call_python_handler(body: Vec<u8>) -> anyhow::Result<Vec<u8>> {
79
+ // Do async I/O without the GIL
80
+ let processed = do_rust_io(&body).await?;
81
+
82
+ // Acquire GIL only for the Python call
83
+ let result = Python::with_gil(|py| -> PyResult<Vec<u8>> {
84
+ let handler = py.import("app")?.getattr("handle")?;
85
+ let py_bytes = pyo3::types::PyBytes::new(py, &processed);
86
+ let result = handler.call1((py_bytes,))?;
87
+ result.extract::<Vec<u8>>()
88
+ })?;
89
+
90
+ Ok(result)
91
+}
92
+```
93
+
94
+## References
95
+
96
+- PyO3 user guide: https://pyo3.rs/latest/
97
+- PEP 703 (free-threaded CPython): https://peps.python.org/pep-0703/
98
+- `DECISIONS.md` — GIL/concurrency strategy (to be written in Phase 4)
0
99