I measured Pingora 0.9.0 six times slower than nginx. The culprit was one line of my code
I wrote a minimal reverse proxy with Pingora 0.9.0 and in a 2 vCPU container it measured 21 thousand requests per second against nginx's 126 thousand: six times slower. The culprit was not Pingora but a blocking getaddrinfo my upstream_peer ran on every request. Fixed, the real gap is 1.49x, and perf quantifies it: 1.49x in cycles per request, 111.7 thousand against 74.9 thousand.
Pingora is the Rust library Cloudflare used to replace nginx at its edge, and it moves more than a trillion requests a day. Version 0.9.0, tagged on September 4th, brings reworked connection pooling and a thread pool to offload TLS handshakes. I wanted to measure it against nginx on the only thing that matters to me: a small pod, with CPU limits, the way anything runs in production.
I wrote the simplest possible reverse proxy with Pingora, put it in a container with two cores and hit it. It measured 21 thousand requests per second. nginx, in another container with the same limits, did 126 thousand on that first run. Six times slower.
That number was a lie, and the fault was mine. This post is how I found it, what it was, and what Pingora 0.9.0 really measures when the proxy is written well.
Fixing a single line, the real gap in a 2 vCPU pod is 1.49x under moderate load, and perf quantifies it: 1.49x in cycles per request, 111.7 thousand against 74.9 thousand for nginx.
How I measured it
The main results come from containers with pod-type limits, not the free host: a number measured with sixteen cores and thirty gigs free says nothing about how something runs inside a bounded pod. There were a few measurements on the host, and I flag them as such.
- Fedora 43, Docker 29.8, cgroup v2.
- Each piece in its container: backend, proxy and load generator separate.
- Backend:
nginx:alpinereturning a fixed 1000-byte response, isolated in its own process. - Both proxies with identical limits:
cpus=2,mem=256m,pids=300. - Pingora: crate 0.9.0, minimal reverse proxy, configurable threads.
- nginx:
worker_processes 2,keepalive 128to the upstream. - Load:
oha1.16 in its own container, hitting by service name over the Docker network. The measured path never touches the host.
Identical limits on pingora and nginx: the comparison is fair by construction, not by promise.
The 6x that did not add up
Six times is too much. Cloudflare would not have replaced nginx with something six times slower. Before writing a single figure, the first rule is not to trust the number you least want to understand.
I isolated the system piece by piece. It was not CPU throttling: the cgroup’s nr_throttled counter was at zero. It was not Pingora opening too many threads: with threads=2 it had exactly five threads, two of them workers. It was not a lack of connection reuse: both proxies held around 130 open connections to the backend, neither opened a new one per request.
The clue was in the latency. In the container, Pingora’s median was 4.78 ms; on the host, with the backend at 127.0.0.1, the same binary gave 0.45 ms. Ten times. Several things change between the two environments (network, isolation, limits), but the contrast pointed at how I named the backend: backend (a name) in the container, 127.0.0.1 (an IP) on the host. getaddrinfo confirmed the cause.
My upstream_peer built the destination like this, on every request:
async fn upstream_peer(&self, _s: &mut Session, _c: &mut ()) -> Result<Box<HttpPeer>> {
// (&str, u16): this resolves the name ON EVERY REQUEST
let peer = HttpPeer::new(("backend", 80), false, String::new());
Ok(Box::new(peer))
}HttpPeer::new over a (&str, u16) calls to_socket_addrs(), and that is getaddrinfo: a synchronous and blocking name resolution, run inside the tokio worker thread. With a name, every request fired a query to Docker’s DNS resolver (127.0.0.11) and blocked the thread while it waited. With an IP, to_socket_addrs() is a text parse with no syscalls, which is why the host barely noticed.
The DNS cloud is the same line of code. With the bug every request visits it and waits; fixed, it was resolved once and the thread never stops.
The fix is one line: resolve the name once at startup, keep the SocketAddr, and pass that (not the name) on every request.
// once, in main:
let addr = ("backend", 80).to_socket_addrs()?.next().unwrap();
// and in upstream_peer, no resolution on the hot path:
Ok(Box::new(HttpPeer::new(self.addr, false, String::new())))With that line, Pingora went from 21 thousand to 89 thousand requests per second in the same pod. The 6x was the cost of a getaddrinfo per request, not Pingora.
What a well-written Pingora 0.9.0 measures
With the proxy fixed, I ran the full matrix: three concurrency levels, three repetitions each, and the CPU use and throttling of every run.
Moderate load: nginx 1.49x, and neither uses more than 1.56 of the 2 cores.
Pingora returned 502 in 2 of 3 runs (7.5k and 1.4k); cause not isolated. nginx stays flat.
Both collapse: the load generator runs out of file descriptors and both return 502, not the proxies.
At c=100 the gap is pure CPU cost per request: neither saturates the two cores.
At a hundred connections, nginx delivers 1.49 times more than Pingora, and both have room to spare: neither passes 1.56 of the two cores available, with 99th percentiles under two milliseconds. At four hundred, nginx stays flat; Pingora, on the other hand, in two of three runs started returning 502 errors to the backend: one run stayed clean at 39 thousand and the other two almost all 502, with a median of 7.5 thousand. I did not isolate the cause. At a thousand connections both collapse, but that is already the load generator running out of file descriptors (not ephemeral ports) and both returning 502: that point measures nothing about Pingora or nginx.
Why nginx spends less: the profile
At a hundred connections neither saturates the CPU, so the throughput difference is a difference in cost per request. perf measures it without ambiguity.
The measured cause: Pingora runs 1.7x more instructions per request (114k vs 67k); its better IPC (1.03 vs 0.90) amortizes that to 1.49x in cycles, the throughput ratio. Context switches and futex exist but add up to less than 1% of the gap: blaming the multithread model is hypothesis, not measurement.
Pingora spends 111.7 thousand cycles per request; nginx, 74.9 thousand. The ratio is 1.49, the same number as the throughput difference. No mystery: at equal CPU available, whoever spends fewer cycles per request serves more requests.
Where do those extra cycles go? The honest answer is that Pingora runs more code per request: perf counts 1.7 times more instructions (114.6 thousand against 67.6 thousand), and only its better use of the processor (1.03 instructions per cycle against 0.90) amortizes that difference down to the 1.49 in cycles. Where that extra code runs, this profile does not resolve: 44% of the samples carry no stack. What it does rule out is epoll, which both use (epoll_wait in Pingora, epoll_pwait in nginx). And there are two signals of the multithread model: Pingora made 35,719 context switches against nginx’s 765, and 44 thousand futex calls, thirteen thousand with an error, where nginx made none. But they are too few to explain the cycle gap: the overhead is somewhere else. That the overhead comes from the shared-state thread model is a reasonable hypothesis, not something these numbers prove.
The flamegraph confirms it in the coarse: more than half the samples (55%) touch the Docker bridge’s network kernel (netfilter, conntrack, veth), which nginx crosses the same way by construction; the remaining 44% carry no stack, so the fine split of the rest stays open.
The default that costs a core
A stumble worth telling: Pingora’s default thread count is one. ServerConf::default() ships threads: 1 and each service inherits it if untouched. The flag exists and is in the configuration guide, but the default value does not appear in the quick start, and with one thread the proxy uses a single core no matter how many the machine has. The first time I measured on the host without configuring it, Pingora gave 46 thousand; with threads matched to cores, 206 thousand. A 4.5x that was not the framework’s but one line of configuration nobody forces you to write.
But careful going the other way: in a pod with a CPU quota, setting more threads than cores is counterproductive.
Not tokio: the CFS. Four threads burn the 200 ms quota in fifty and wait out the rest of the 100 ms period. The pod rule: threads = CPU quota.
With two vCPU of quota, going from two to four threads barely moves throughput (6%) and the 99th percentile jumps from 1.7 to 35 milliseconds; at eight threads, to 68, and there throughput even drops 21%. It is not tokio: it is the kernel’s CFS scheduler. With two threads the throttling counter stayed at zero of 120 periods; with four, it throttled 119 of 120; with eight, all 120: the extra threads burn the CPU quota before the period ends and wait, and that wait is the latency tail. The rule for this kind of load: threads equal to the CPU quota, not one more.
Honest opinion
The easy headline, “nginx is six times faster than Pingora,” was false, and I wrote it without meaning to. The lesson I take is not about Pingora. It is that a microbenchmark punishes without mercy any clumsiness of whoever writes it, and that a getaddrinfo hidden on the hot path disguises itself as “the framework is slow.”
With the proxy written well, Pingora 0.9.0 in a 2 vCPU pod spends 49% more cycles per request than nginx in this perf run. It is not little, and for an edge where every core is paid for, it matters. But Pingora does not compete on cycles per request: it competes on what you can program on top (routing logic, authentication, rewriting) in Rust and with memory safety, instead of C or Lua modules. My minimal proxy exercises none of that; it measures the floor, not the ceiling.
When yes and when no
- nginx if what you need is a proxy or balancer with configuration rules and every core counts: it is cheaper per request and its latency tail is flatter under pressure.
- Pingora if you are going to program the edge (logic that in nginx would end up in Lua or a C module) and you want to do it in Rust. The CPU cost is real; whether it is worth it depends on how much you program on top.
- Either way, in a pod: take name resolution off the hot path (by IP, or cached respecting the TTL), and set threads equal to the CPU quota. Those two lines are worth more than the choice of framework.
Sources
- Pingora 0.9.0, release notes and tag, September 4th, 2026.
HttpPeer::newandto_socket_addrs()on the hot path:pingora-core0.9.0,src/upstreams/peer.rs.ServerConf::default()withthreads: 1, and the flag documented in the configuration guide.- CFS throttling by CPU quota, Linux kernel documentation.
Comments
No comments yet. The first one is yours.