Rate Limiting Federated GraphQL Operations

Rate limiting a federated graph is harder than rate limiting a REST API for one reason: requests are not comparable. One operation touches a cached field in a single subgraph; another fans out to five services and resolves two hundred entities. Counting both as “one request” produces a limit that is simultaneously too tight for cheap traffic and far too loose for expensive traffic. This page covers where limits can be applied, what each layer can and cannot see, and how to make the numbers mean something — part of Securing the Federated Gateway.

When to use this pattern

  • Use it on any endpoint you do not fully control the clients of, which includes public and partner APIs and, in practice, most internal graphs with more than a handful of consumers.
  • Use it when one subgraph is markedly more fragile than the rest, because per-subgraph shaping protects it without throttling operations that never touch it.
  • Skip a global request-per-second limit as your only control. It is trivially defeated by sending fewer, larger operations, which is exactly what an expensive client already does.

Prerequisites

Three places a limit can live

Each layer sees something the others do not, and the differences decide what each can usefully protect.

At the edge — load balancer or CDN. Sees IP addresses, headers and request size; knows nothing about operations. Its job is stopping raw volume before it costs you anything, and it is the only layer that can shed load without the request touching your application at all.

At the router — global and per-subgraph. Sees the operation, its name, the caller’s validated identity, and which subgraphs the plan will touch. This is the only layer that can say “this caller may run twenty expensive operations per minute but two thousand cheap ones”, and the only one that can protect an individual subgraph from a fan-out pattern.

At the subgraph. Sees its own load and nothing about the operation that caused it. Useful as a last line of defence, useless for attribution — a subgraph cannot tell you which client is responsible, only that it is overwhelmed.

What each layer can see, and therefore protect The edge sees addresses and volume. The router sees operations, identity and the query plan. The subgraph sees only its own load. Only the router can relate cost to caller. Only one layer can relate cost to caller layer what it can see what it should protect edge / load balancer addresses, volume, size the router's capacity router — global operation, identity, plan the graph as a whole router — per subgraph which fetches a plan makes one fragile service subgraph itself its own load only itself, without attribution A limit at the subgraph tells you it is overloaded; it can never tell you who did it.

Implementation walkthrough

The router’s own limits are the middle two rows above, and they are configured in the same block. Note the comment on the global limit — it is the detail most often misread.

# router.yaml
traffic_shaping:
  router:
    global_rate_limit:
      # PER ROUTER PROCESS. With 10 replicas this admits 10,000/s in aggregate,
      # not 1,000/s. Divide your intended cluster limit by replica count, or
      # accept that this is a per-process safety valve rather than a quota.
      capacity: 1000
      interval: 1s
    timeout: 30s

  # Per-subgraph shaping is where a fragile service gets real protection.
  subgraphs:
    legacy_pricing:
      global_rate_limit:
        capacity: 50        # this service genuinely cannot take more
        interval: 1s
      timeout: 2s
      experimental_retry:
        min_per_sec: 5
        retry_percent: 0.1  # bounded, so a struggling service is not hammered
    catalog:
      timeout: 5s
      deduplicate_query: true   # collapse identical in-flight fetches

Per-caller quotas need more than static configuration, because the identity to key on comes from a validated token. A Rhai script reads the claim and puts it in context, where a coprocessor or the metrics pipeline can act on it:

// router.rhai — attribute every request to a caller for quota and metrics
fn supergraph_service(service) {
  service.map_request(|request| {
    // apollo_authentication::JWT::claims is populated by the router's JWT
    // validation — reading a header directly would trust the client.
    let claims = request.context["apollo_authentication::JWT::claims"];
    if claims != () {
      request.context["client_id"] = claims["azp"];
      request.context["plan_tier"] = claims["tier"];
    }
    request
  });
}

Keying on a validated claim rather than a header is the whole point. A quota keyed on x-client-id is a quota any client can escape by changing one header; a quota keyed on a signed claim cannot be forged without forging the token.

Verification steps

Confirm the global limit engages and returns the right status:

# Fire past the configured capacity and count the rejections.
for i in $(seq 1 200); do
  curl -s -o /dev/null -w '%{http_code}\n' localhost:4000/graphql \
    -H 'content-type: application/json' -d '{"query":"{ __typename }"}' &
done | sort | uniq -c
# expect a mix of 200 and 429

Then confirm the per-subgraph limit protects the service it names without affecting others. Send operations that touch only the protected subgraph and watch its fetch rate plateau at the configured capacity; send operations that avoid it and confirm they are unaffected. If both slow together, the limit is being applied globally rather than per subgraph.

Finally — and this is the test people skip — verify the arithmetic under real replica counts. Scale the router to its production replica count and measure aggregate accepted throughput. A per-process limit that looked correct on one replica is off by an order of magnitude on ten, and finding that out during an incident is expensive.

A per-process limit is multiplied by replica count With a per-process capacity of one thousand per second, aggregate admitted throughput rises linearly with replicas, reaching ten thousand per second at ten replicas. A shared-state limiter holds the total flat regardless of replica count. Aggregate admitted requests per second, capacity 1,000 per-process limit — grows with replicas shared-state limit — flat 1 router replicas 10 Neither line is wrong — but only one matches what people usually mean by "a limit of 1,000/s". Autoscaling makes the pink line worse: the limit rises exactly when load is already high.

Common mistakes & gotchas

Treating the router’s limit as cluster-wide. It is per process. With autoscaling this is doubly awkward, because the effective limit rises precisely when traffic is heaviest. Use it as a safety valve and put real quotas where shared state exists.

Rate limiting by IP behind a proxy. Without correct forwarded-header handling every request appears to come from the load balancer, so one shared bucket throttles everyone at once. Verify the address the limiter actually sees before trusting any per-client number.

Counting requests rather than cost. A client that batches ten expensive operations into one document consumes ten times the resources of one cheap request and counts identically. Pair rate limits with depth, height and alias caps so the request being counted has a bounded size.

Four strategies, and the gap each leaves Requests per second bounds volume but not size. Cost limits bound size but not volume. Per-subgraph shaping protects one service. Concurrency limits bound simultaneous work. Each leaves a gap the others cover. No single strategy is sufficient alone strategy bounds misses requests per second volume one enormous operation depth / alias caps size of one document many small operations per-subgraph shaping load on one service router-side exhaustion concurrency limits simultaneous work sustained steady overload

Frequently Asked Questions

Should limits be per operation name?

For a graph with persisted operations, yes — it is the most precise control available, because the set of operation names is finite and each one has a measurable cost. Give the expensive handful their own budgets and leave the rest under a generous shared limit. Without persisted operations the name is client-supplied and therefore not trustworthy as a quota key.

What status code should a rejection use?

HTTP 429 with a Retry-After header, and a GraphQL error only if you must return a body. Returning 200 with an error inside is technically valid GraphQL and operationally awful: every client library, proxy and dashboard treats it as success, so throttling becomes invisible in exactly the graphs where it matters most.

How do I set the numbers initially?

From observed traffic, not from a target. Measure the p99 request rate per caller over a normal week, set the limit at two to three times that, and watch rejections for a month. A limit set below real peak traffic will be discovered by your users; a limit set from measurement will only be reached by genuine anomalies.

Does rate limiting help against a distributed attack?

Per-caller limits do not, because there is no single caller. What helps is the combination of a global limit at the edge, cost limits that bound what any accepted request can do, and safelisting so unknown operations never execute. Rate limiting is one layer of that, and on its own it is the weakest of the three.

Where should retries be configured?

At the router, bounded by a retry budget, and never in the client on a 429. Router-level retries with retry_percent cap the extra load a struggling subgraph receives; a client that retries on rejection converts a throttle into a load amplifier, which is the exact opposite of the intent.

How should limits interact with a queue or retry system?

Carefully, because the two can amplify each other. A background job that retries on 429 without backoff converts a throttle into sustained overload, and a queue that drains at full speed after an incident recreates the incident. Give any automated consumer its own generous-but-separate quota and require exponential backoff with jitter, so a limit reached by a batch job never competes with interactive traffic for the same budget.