Cross-Language Benchmark: A2A Layer Cost

What this measures, and what it does not

This page answers one narrow question: what does the A2A protocol layer itself cost to run? It compares this Rust SDK's server against the official Python a2a-sdk server on an identical echo workload.

It is not a recommendation to change SDKs, and it is not evidence that an agent built on the official Python SDK is slow. The workload here is an echo with no application work at all, which is the case that shows protocol overhead most starkly, because there is nothing else in the request to dilute it. A real agent turn is dominated by model inference.

Put in proportion: the measured difference is 2.36 ms of CPU per request. Against an agent turn spending 1–30 s in a model, that is 0.008 %–0.24 % of the turn. No single agent will notice it. Nothing here argues otherwise.

Where it does matter is aggregation. Protocol cost is paid per request regardless of what the agent does, so it scales with traffic rather than with model latency. The same throughput costs materially different amounts of CPU:

Each row is the pinned CPU per request multiplied by the rate. No server was driven at these rates: the official SDK's single worker sustained 166 req/s with 50 connections in the concurrency test, so its 1,000 and 10,000 req/s rows describe several workers' worth of CPU, not the one worker measured. Unpinned, the 10,000 req/s row is 1.272 cores for this SDK against 24.21 for the official one.

That is the shape of the result: irrelevant inside one agent, and the whole cost model for anything that terminates A2A traffic on behalf of many.

Headline: CPU consumed per request

The primary figure, because it is what a capacity argument actually rests on. Latency cannot distinguish work from waiting — a server blocked on a poll and one burning CPU can post the same round-trip time. This is the server's own utime + stime from /proc, divided by requests served.

32.1× the CPU per request, or 2,357 µs more, on the pinned configuration described below. Unpinned, each server as its own documentation runs it, the same probe gives 127.2 µs for this SDK (19,741 requests, 0.40 % quantisation error) and 2,421.5 µs for the official SDK (1,210 requests, 0.34 %): 19.0×, or 2,294 µs more. Pinning is the more favourable configuration for this SDK; the 32.1× is the upper figure.

Latency and throughput

Both configurations are shown because either alone invites a misreading. pinned confines each server to a single distinct core, which answers the objection that a multi-threaded Rust server was given four cores while one uvicorn worker got one. default runs each server exactly as its own documentation says to.

Every latency column is the median over the 3 trials of that trial's figure, p50, p95 and p99 alike.

Note that the Rust server is slower unpinned than pinned, and the Python server is almost unaffected by pinning — it is single-threaded and CPU-bound: its p50 is 2,304.7 µs pinned and 2,218.6 µs unpinned (3.7 % lower) and its CPU per request 2,432.2 and 2,421.5 µs (0.4 % lower). The pinned configuration is the more favourable of the two for this SDK: lower latency, higher throughput, less CPU per request and a larger ratio over the Python server. It is shown because it is the like-for-like comparison, not because it is the harsher one.

pinned

pinned: rust='taskset -c 0' python='taskset -c 1' client='taskset -c 2,3'

default

No CPU pinning: every process free to run on any core, as each server's own documentation describes running it.

The floor row

floor is a server that returns a pre-baked constant and does no A2A work. It exists to separate SDK cost from the cost of the client, the loopback stack and the kernel, which sit in every number on this page. Because that shared cost is present in both real servers, the ratio of two end-to-end latencies understates the ratio of the servers' own handling cost.

The floor server runs as a thread inside the client process (a Python interpreter sharing the client's cores), not as a separate process like the two real servers, each on its own core in the pinned configuration, so floor-subtracted figures are indicative, not exact.

Subtracting the floor from the pinned p50 figures leaves 27.2 µs against 2,261.3 µs — a 83× ratio, against 32.1× on CPU. The two measure different things: floor-subtracted latency isolates protocol handling above a bare HTTP round trip, while CPU per request includes the server's share of kernel and network work. The CPU figure is the conservative one, and it is the one quoted above.

The floor's throughput is deliberately omitted. Under concurrency it measures its own crude thread-per-connection ceiling, not a floor, so it bounds nothing and must not be compared against the real servers.

Method

One shared raw-socket client sends byte-identical, pre-serialized HTTP over a warm keep-alive loopback connection to every target. Only the server differs. Driving each server with its own SDK's client would measure that SDK on both sides of the comparison and make the two legs incomparable.

Both servers already existed for interoperability testing and share an echo contract — Echo: <text> returned as a completed task artifact. Neither was written for this benchmark:

• this SDK: examples/echo-agent
• official SDK: itk/agents/python-sdk/agent.py, on a2a-sdk

The Python server runs uvicorn's fast path: uvicorn[standard] supplies uvloop and httptools, and uvicorn's default loop/http setting of auto selects them. The runner asserts both are installed and refuses to produce a result without them, because measuring plain asyncio with h11 would understate the official SDK and make this a strawman.

What remains uncontrolled

Stated so that a reader can discount the result appropriately rather than discover the caveats later.

• Response sizes still differ: 259 bytes from this SDK against 338 from the official one. The request sends historyLength: 0, which removes the largest difference — the Python agent otherwise echoes the inbound message back in history — but a timestamp and artifact naming remain. Some small part of the gap is payload size, not protocol handling.
• Loopback only. No real network, so this isolates SDK cost rather than reproducing deployment latency. Over a real link, both numbers move toward the network's.
• One uvicorn worker. A production Python deployment would run several behind a process manager, multiplying its throughput roughly by worker count at a proportional cost in cores and memory. The per-request CPU cost, which is the headline figure, does not improve with more workers.
• Echo workload. No model call, no I/O, no business logic. This maximises the visible share of protocol overhead by design.
• Shared container. Absolute figures are specific to the host below. Trial-to-trial spread is reported so run-to-run noise is visible.
• In-memory task stores on both sides, growing for the run's duration. Trial medians are stable across trials, so no drift was observed at this scale, but neither server was measured against a persistent store.

Environment

Python packages (full resolved set of 42 recorded in the result file):

a2a-sdk==1.1.4
httptools==0.8.0
httpx==0.28.1
pydantic==2.13.5
starlette==1.6.0
uvicorn==0.53.0
uvloop==0.22.1

The requirements file has since moved to a2a-sdk 1.2.3 and uvicorn 0.54.0; the figures above were measured with the versions listed.

Reproducing this

Needs Linux with taskset (util-linux), at least 4 logical CPUs, python3 -m venv and network access to install the pinned Python packages.

benches/scripts/cross_language_python.sh
benches/scripts/generate_cross_language_page.py

The runner builds both servers, pins the Python environment from benches/requirements-cross-language.txt, runs both CPU configurations, and writes benches/results/cross-language-{pinned,default}.json. Those files carry the full raw distributions and complete provenance, including the resolved dependency set and the repository commit. All measured figures on this page are generated from them; the 1–30 s agent-turn bounds and the request rates in the first table are assumptions.