Software Architecture
API Management and Gateways: Policies, Rate Limiting, Plans and Portals
Intermediate10 min readUpdated
An API gateway is the policy layer between your consumers and your services; API management is everything around it — plans, keys, quotas, portals and analytics. This guide separates the gateway from the load balancer and the service mesh, explains the rate-limiting algorithms in practical terms, and sets out the deployment topologies and product categories you will actually choose between.
What you will learn
- Distinguish an API gateway from a load balancer, a reverse proxy and a service mesh
- Choose and configure a rate-limiting algorithm appropriate to the traffic you must protect
- Design API products, plans and quotas that match how consumers actually use the API
- Place authentication and key management at the right layer of the request path
- Compare gateway deployment topologies and product categories against real requirements
Once an API has consumers you do not control, a set of concerns appears that has nothing to do with the API's business behaviour. Who is calling? Are they allowed to? How often may they call? How do we onboard the next hundred developers without a meeting each? Which endpoints are actually used, by whom, and how well are they performing? Building answers into every service means building them repeatedly and inconsistently. An API gateway concentrates them at the edge; an API management platform adds the commercial and organisational layer around it.
The distinction matters when choosing tools, because "gateway" is used for products spanning three orders of magnitude in complexity — from a hundred lines of proxy configuration to a suite with a portal, a billing integration and a governance workflow. This guide separates the technical function from the management function, and is deliberate about which problems justify which spend.
What sits in the request path
Four things commonly appear between a client and a service, and they are routinely confused.
| Component | Operates on | Primary job |
|---|---|---|
| Load balancer | Connections and requests | Spread traffic across healthy instances |
| Reverse proxy | HTTP requests | Terminate TLS, route by host and path, cache, compress |
| API gateway | API operations | Authenticate, authorise, limit, transform, meter, route |
| Service mesh | Service-to-service calls | mTLS, retries, traffic shifting and telemetry between services |
A load balancer knows about instances, not about APIs; it cannot tell that POST /transfers deserves a different rate limit from GET /rates. A reverse proxy adds path awareness but no notion of a consumer identity or a quota. An API gateway is the first component that understands who is calling which operation under which plan, which is what makes policy possible.
The mesh sits on a different axis. Gateways handle north–south traffic — entering and leaving the system — while a mesh handles east–west traffic between internal services. They overlap on retries and mTLS and are complementary rather than competing; most platforms that run a mesh still put a gateway at the edge, and plenty of systems need a gateway and no mesh at all. The mesh trade-off is discussed further in microservice design patterns.
Where the gateway differs from the enterprise service bus is intent. Both mediate; the bus was designed to hold integration logic and grew until it held the company's business rules, whereas a gateway is deliberately kept thin. SOA and the ESB covers how that went wrong; the lesson to carry forward is that any business decision inside the gateway is a decision your service's tests cannot see.
Policies the gateway applies
A gateway's behaviour is expressed as a chain of policies evaluated per request. The exact syntax varies; the categories do not.
- Traffic control — rate limits, quotas, concurrency caps, request size limits, IP allow and deny lists, circuit breaking toward a failing upstream.
- Security — TLS termination, API key validation, OAuth 2.0 token introspection or JWT signature verification, mutual TLS for partners, CORS handling, injection and payload validation against the OpenAPI schema.
- Routing — path, host and header-based routing, weighted splits for canary releases, versioned upstreams, retries and timeouts.
- Mediation — header manipulation, protocol adaptation, response caching, compression.
- Observability — access logs, latency and error metrics per operation and per consumer, trace propagation, correlation identifier injection.
# Declarative gateway configuration: one route, several policies.
routes:
- name: payments-v2
match:
host: api.example.com
path: /v2/payments
upstream: payments-service.internal:8080
policies:
- jwt:
issuer: "https://login.example.com/"
audiences: ["payments"]
requiredScopes: ["payments:write"]
- rateLimit:
identifier: consumerId # per API consumer, not per IP
limit: 100
window: 1m
policy: sliding-window
- requestSizeLimit: { maxBodyBytes: 65536 }
- timeout: { upstream: 2s }
- headers:
remove: ["X-Internal-Debug"]
add: { "X-Correlation-Id": "${trace_id}" }
Keep this configuration in version control and deploy it through the same pipeline as code. A gateway configured by clicking in a console is a production system with no history, no review and no rollback.
Rate limiting in practice
Rate limiting protects the upstream from overload and protects consumers from each other. Four algorithms cover almost every deployment, and the differences show up exactly when traffic is bursty.
Fixed window. Count requests per calendar minute; reset at the boundary. Trivial to implement and cheap to store, with one serious flaw: a client can send its full allowance in the last second of one window and again in the first second of the next, producing double the intended rate across that boundary.
Sliding window. Approximate the last 60 seconds by weighting the current and previous fixed windows, or keep a log of timestamps for exactness. Removes the boundary burst at modest extra cost, and is the sensible default.
Token bucket. A bucket holds up to B tokens and refills at R tokens per second; each request takes one, and a request with no token available is rejected. This permits bursts up to the bucket size while enforcing an average rate, which matches how real clients behave — a page load firing eight parallel requests should not be punished if the client is otherwise quiet.
Leaky bucket. Requests queue and drain at a constant rate, smoothing output completely. Useful in front of something that genuinely cannot absorb bursts, such as a legacy backend or a partner API with a hard limit, at the cost of added latency for queued requests.
| Algorithm | Allows bursts | State per client | Best for |
|---|---|---|---|
| Fixed window | Yes, at boundaries | One counter | Rough protection, high volume |
| Sliding window | Limited | Two counters or a log | General-purpose defaults |
| Token bucket | Yes, up to bucket size | Tokens plus timestamp | Interactive clients with bursty traffic |
| Leaky bucket | No | Queue | Protecting a fragile upstream |
Three details decide whether limiting helps or infuriates. First, what you key on: limiting by IP address punishes everyone behind a corporate NAT and does nothing against a distributed caller, so key on the authenticated consumer wherever one exists and fall back to IP only for anonymous traffic. Second, what you return: a 429 with Retry-After and headers reporting the limit, the remainder and the reset time lets a well-behaved client back off correctly; a bare 429 teaches it to retry immediately. Third, shared state: in a clustered gateway, counters must be shared or the effective limit is multiplied by the node count.
Identity, keys and credentials
Three mechanisms coexist, and each answers a different question.
API keys identify the calling application. They are a shared secret in a header, simple to issue and easy to leak, and they carry no user identity and no expiry unless you add one. They remain appropriate for server-to-server integrations, for read-only public data and for metering, provided they are transmitted only over TLS, hashed at rest, scoped to specific APIs and rotatable without downtime — which means supporting two active keys per consumer during a rotation.
OAuth 2.0 access tokens carry delegated authorisation: which application, acting for which user, with which scopes, until when. The gateway validates the token — signature, issuer, audience, expiry — and passes verified claims downstream, so services never handle raw credentials. This is the right default for anything involving end users.
Mutual TLS authenticates the client by certificate at the transport layer, and is common for partner and financial integrations where a certificate authority already governs who may connect.
The layering that works: the gateway rejects anything unauthenticated or malformed, and forwards verified identity; the service enforces authorization, because only the service knows whether this customer may read this account. A gateway cannot make record-level decisions, and pretending otherwise creates a system where anyone who reaches a service is trusted absolutely — a single misconfigured route away from a breach.
Never let credentials into places they persist: strip authorization headers from access logs, and reject tokens or keys passed in query strings, since those end up in proxy logs and browser history.
Products, plans and the developer portal
Management platforms add a layer of abstraction above the technical routes, and it is the part that makes an API programme scale organisationally.
An API product bundles one or more APIs into something a consumer subscribes to — "Payments" might include the payments API, the mandates API and the webhook feed. A plan attaches terms to the product: rate limits, monthly quotas, available endpoints, support level and price. Consumers register an application, subscribe it to a plan, and receive credentials scoped accordingly.
That structure gives you the vocabulary for differentiated access — a free tier at 1 000 calls a month for evaluation, a standard tier for production, a partner tier with higher limits and a support contract — without any of it appearing in service code.
The developer portal is the self-service front end: catalogue of products, interactive OpenAPI reference, getting-started guides, credential management, subscription requests, usage dashboards and changelogs. Its value is measured by time-to-first-successful-call. A portal where a new developer can register, get a sandbox key and receive a real response in ten minutes removes an enormous amount of support work; one where every credential needs a ticket removes none of it. Sandbox environments with realistic test data matter here more than any gateway feature.
Analytics closes the loop: calls per operation, per consumer and per version; error rates and their causes; latency percentiles at the edge; quota consumption approaching limits. This is how you learn that version 1 still carries 40 % of traffic from two consumers, which is the evidence a retirement decision needs. It is also, where relevant, the basis for usage-based billing.
Lifecycle governance ties it together: design review against standards before publication, a documented deprecation process with the Sunset header and portal notices, and a linter enforcing your API guidelines on every OpenAPI change. Automated checks in the pipeline scale; review boards do not.
Deployment topologies
Where the gateway runs shapes latency, blast radius and who can change it.
Centralised edge gateway. One cluster in front of everything. Simplest to operate and to reason about, uniform policy, one place to look. It is also a single point of failure and a shared resource that every team queues behind, so it needs real capacity planning and a self-service configuration path.
Gateway per domain or per zone. Separate gateways for public, partner and internal traffic, or per business domain. Isolates blast radius, allows different policy postures — internal traffic rarely needs the same throttling as public — and keeps configuration ownership closer to teams.
Micro-gateway or sidecar. A lightweight gateway deployed with each service or team, configured centrally. Removes the shared bottleneck and adds a hop's worth of consistency problems; useful in large organisations that have outgrown a single edge.
Managed cloud gateway. The cloud provider's own service, integrated with its identity, logging and certificate management. Least operational work and least portability, and typically metered per million calls, which becomes material at high volume.
A hybrid is common and reasonable: a managed gateway for public traffic, and an ingress controller inside the cluster for internal north–south routing. What matters is that each gateway has an owner, a configuration pipeline and a documented failure behaviour — specifically, whether a gateway outage fails open or closed, which is a decision to make deliberately rather than discover.
Choosing a product category
The market divides into recognisable groups, and identifying which one you need saves a great deal of evaluation effort.
- Proxy-based open source — a fast data plane with a plugin model, self-hosted, with management features available as add-ons or a paid tier. Good when you have platform engineers and want control over cost and placement.
- Full lifecycle management suites — gateway, portal, analytics, monetisation and governance in one product, often with an enterprise licence. Justified when you have many external consumers, commercial terms and audit obligations.
- Cloud provider gateways — deeply integrated, quick to start, priced per call, weakest on portability and on portal sophistication.
- Ingress controllers and the Kubernetes Gateway API — route management expressed as Kubernetes resources. Excellent for internal routing and increasingly capable at the edge; not an API management platform.
Evaluate against requirements you can demonstrate, not feature matrices. The questions that discriminate: how is configuration versioned and promoted between environments; what is added latency at your p99 under your traffic shape; how are rate-limit counters shared across nodes; what happens to traffic when the control plane is down; can the portal be branded and can documentation be published from your OpenAPI documents automatically; and what does it cost at ten times today's volume. Design the API well first — see REST API design — because no gateway compensates for a contract consumers cannot use.
Key takeaways
- A gateway is the first component in the path that understands consumers, operations and plans; load balancers and proxies do not, and a mesh solves the internal half of the problem.
- Keep the gateway thin: policy, routing and mediation belong there, business logic does not, or you rebuild the enterprise service bus.
- Choose the rate-limiting algorithm to match traffic shape, key on the authenticated consumer rather than the IP address, and return
429withRetry-Afterand limit headers. - Let the gateway authenticate and the service authorise; a gateway cannot make record-level access decisions.
- Products, plans and a self-service portal with a sandbox are what let an API programme scale beyond a handful of consumers, and analytics per version is what makes retirement decisions defensible.
- Pick the topology and product category from demonstrable requirements — configuration as code, added latency, shared counters, control-plane failure behaviour and cost at scale.
Sources and further reading
Frequently asked questions
- Do I need an API gateway if I only have a handful of internal services?
- Probably not a full management platform, but a single entry point is still useful the moment more than one client type calls more than one service. It gives you one place for TLS termination, authentication and routing, and it makes replacing a service transparent to callers. A lightweight reverse proxy or an ingress controller covers that; buy management features when you have external consumers to onboard, meter and support.
- Can the gateway do authentication for my services?
- It should do the expensive, repetitive part — validating the token signature, checking expiry and issuer, rejecting anything malformed — so that services never see unauthenticated traffic. Services must still enforce authorization, because only they know whether this user may see this record. Treat gateway authentication as a filter, not as a reason to trust every request that reaches you.
- What happens to rate limits when the gateway is clustered?
- Each node sees only its own traffic unless counters are shared, so a limit of 100 per minute across five nodes becomes 500 unless you configure a shared store such as Redis. Shared counters add a network round trip per request, so many gateways offer an approximate mode that divides the limit per node or synchronises periodically. Decide explicitly which inaccuracy you can tolerate.
- Should a gateway transform requests and responses?
- Sparingly. Stripping internal headers, adding correlation identifiers and adapting a legacy protocol at the edge are all reasonable. Rewriting business payloads is not: the logic ends up in infrastructure owned by a different team, invisible to the service's tests, and every change becomes a gateway deployment. That is exactly how the enterprise service bus became a bottleneck.