bench: record JustAPI vs Robyn 0.88 head-to-head on same hardware
swadhinbiswas committed 3 months ago
0593129
1 parent f43d79f
commit 0593129
1 file changed
+31 -1
Showing 1 changed file+31-1
▼BENCHMARKS.md
+31-1
| @@ -51,7 +51,8 @@ Selective benchmarking is a credibility risk, so we state the losses too: | ||
51 | 51 | pure throughput micro-benchmarks we expect to *lose* to purpose-built minimal |
52 | 52 | runtimes. |
53 | 53 | |
54 | -We will add a Robyn row to this ledger once measured on the same fixture. | |
54 | +A Robyn row is now recorded on this hardware (see the 2026-07-16 head-to-head | |
55 | +below); the earlier fixture numbers are also kept for reference. | |
55 | 56 | |
56 | 57 | --- |
57 | 58 | |
| @@ -193,6 +194,35 @@ actually takes the Rust fast path**. | ||
193 | 194 | `native=True` + a `Schema`. Plain Python handlers are correct and stable but |
194 | 195 | GIL-bound. |
195 | 196 | |
197 | +### Head-to-head: JustAPI vs Robyn 0.88 (2026-07-16, same hardware) | |
198 | + | |
199 | +Measured on the **same dev box** as the re-run above, identical workload | |
200 | +(JSON `POST` echo, `{"ok":true}`), release build, `oha -z 10s`, single-process. | |
201 | +Robyn 0.88.0 (its `actix` runtime + Python handler). This is the apples-to-apples | |
202 | +number that was previously only recorded on the i5-13600K fixture — here Robyn | |
203 | +comes in materially lower (~22k) than the fixture's 33–40k, which widens | |
204 | +JustAPI's margin on this machine. | |
205 | + | |
206 | +| Framework | Route mode | req/sec @ -c 100 | req/sec @ -c 200 | | |
207 | +|---|---|---:|---:| | |
208 | +| **JustAPI** | `native=True` + `Schema` (Rust fast path) | **485,076** | **564,909** | | |
209 | +| **JustAPI** | plain Python handler (GIL pool) | 120,111 | 119,619 | | |
210 | +| **Robyn 0.88** | Python handler (actix) | 22,300 | 22,066 | | |
211 | + | |
212 | +**Conclusion: JustAPI is faster than Robyn on this hardware on every path.** | |
213 | +- The Rust native fast path is **~21× Robyn at -c 100** and **~25× at -c 200**. | |
214 | +- Even JustAPI's *plain Python* handler path (GIL-bound) is **~5.4× Robyn**, | |
215 | + because the dedicated GIL thread-pool dispatches Python work without the | |
216 | + actix-threadpool round-trip Robyn uses for async Python handlers. | |
217 | +- The earlier "Robyn ×1.5–2× faster" conclusion (recorded 2026-07-13 on a debug | |
218 | + build, pre-fast-path) is **fully superseded**: that run crossed the PyO3 | |
219 | + boundary into a Python handler on every route; the native fast path and GIL | |
220 | + pool rewrite remove that cost. | |
221 | + | |
222 | +> Caveat: these are micro-benchmarks (echo). Robyn may close the gap on | |
223 | +> handler-heavy workloads with large Python logic, and the native fast path only | |
224 | +> applies to schema-backed validate-and-echo routes. Measure your real workload. | |
225 | + | |
196 | 226 | --- |
197 | 227 | |
198 | 228 | ## Baseline: Uvicorn+FastAPI (recorded 2026-07-03) |
199 | 229 | |