API Gateway in System Design: Authentication, Routing & Microservices

Fundamentals

What is an API Gateway?

An API Gateway is a controlled entry point through which clients communicate with backend services.

Web / Mobile / Partner

Clients

HTTPS

API Gateway

Route • Authenticate • Limit • Observe

User

Service

Order

Service

Payment

Service

Booking

Service

Think of it as the front door to backend APIs

Instead of exposing every internal service directly to public clients, the gateway provides one controlled boundary.

Microservices

Why Do We Need an API Gateway?

Without a gateway, public clients may need to understand the addresses and contracts of many internal services.

Single Entry Point

Clients communicate with one public API boundary instead of discovering many backend services.

Central Authentication

Validate credentials before requests reach internal services.

Authorization Policies

Reject requests that do not meet gateway-level access rules.

Rate Limiting

Protect backend services from excessive traffic.

Routing

Map public endpoints to internal services.

Observability

Generate common logs, tracing metadata and request metrics.

Request Transformation

Modify headers, paths or payload shape when appropriate.

API Composition

Aggregate selected backend calls into one client response.

Request Flow

Typical API Gateway Architecture

A real request usually passes through several infrastructure layers before reaching the target service.

Client

DNS / CDN / WAF

Edge protection

Load Balancer

Distribute gateway traffic

API Gateway Cluster

Policies + routing

Service A

Service B

Service C

Common Interview Question

API Gateway vs Load Balancer vs Reverse Proxy

These components overlap in modern infrastructure, but their primary responsibilities are different.

Component	Primary Role	Typical Features
Load Balancer	Distribute traffic	Health checks, balancing algorithms, failover
Reverse Proxy	Represent backend servers	TLS, routing, caching, compression, proxying
API Gateway	API policy + routing boundary	Auth, rate limits, transformations, aggregation, observability

One product may perform several roles

Modern proxies, managed gateways and service-mesh components can overlap heavily. Discuss the logical responsibility rather than assuming there must always be a separate physical product for each role.

Core Responsibility

Request Routing

The gateway maps an incoming request to the appropriate backend service.

Path-Based Routing

GET /users/42
        ↓
User Service


GET /orders/501
        ↓
Order Service


POST /payments
        ↓
Payment Service

Routing Rules

Path Based

/users/* → User Service

Host Based

payments.api.com → Payment Service

Header Based

X-Client-Version=v2 → new service

Weighted Routing

10% → v2, 90% → v1 during canary rollout

Geo Routing

Route selected traffic to regional service deployments.

Tenant Routing

Enterprise tenants may route to dedicated infrastructure.

Security

Authentication at the Gateway

Authentication answers: who is making this request?

Client

Authorization: Bearer JWT

API Gateway

Validate credential

Valid identity

Backend

Trusted identity context

JWT

Validate signature, expiry, issuer, audience and required claims.

API Key

Identify calling applications or developers.

OAuth Access Token

Validate externally issued access tokens and scopes.

mTLS

Authenticate clients or services using certificates.

JWT Claims

{
  "sub": "user_42",
  "role": "USER",
  "scope": [
    "booking:read",
    "booking:create"
  ],
  "iss": "auth.targettrek.in",
  "exp": 1790678000
}

Authentication is not authorization

Knowing that the caller is user 42 does not automatically mean user 42 can access every booking, payment or admin API.

Access Control

Authorization

Authorization determines what the authenticated caller is allowed to do.

Role-Based Access

ADMIN, USER, SUPPORT or other roles.

Scope-Based Access

booking:create, payment:read and similar OAuth-style scopes.

Route Policies

Only administrators may access /admin/* routes.

Tenant Policies

One organization should not access another organization's APIs.

Backend services should still protect business resources

A gateway can enforce broad policies, but domain-specific authorization such as “does this booking belong to this user?” usually remains the responsibility of the owning service.

Traffic Protection

Rate Limiting at the Gateway

The gateway is a natural place to enforce API quotas before expensive backend work begins.

Free Plan:
100 requests / minute


Pro Plan:
5,000 requests / minute


Payment Route:
20 requests / minute / user


OTP Route:
1 request / 30 seconds / phone

Client

Too many requests

Gateway Limiter

Token Bucket / Sliding Window

Allowed only

Backend

Protected

Input Protection

Request Validation

A gateway can reject clearly invalid requests before they consume backend resources.

Payload Size

Reject unexpectedly huge bodies before forwarding them.

Required Headers

Require content type, API version or authentication metadata.

Schema Validation

Validate basic request shape where gateway tooling supports it.

Method Restrictions

Reject unsupported HTTP methods.

Content Type

Accept only supported request encodings.

Basic Sanitization

Normalize or reject obviously malformed traffic.

Compatibility

Request & Response Transformation

Gateways can translate between a public API contract and internal service contracts.

Public Request

POST /api/orders

{
  "productId": "p1",
  "quantity": 2
}

Internal Request

POST /internal/order-service/v2/orders

Headers:

X-User-ID: user_42
X-Request-ID: req_991


{
  "sku": "p1",
  "units": 2
}

Header Injection

Add request ID, authenticated user context or tracing metadata.

Path Rewrite

Map public route names to internal service routes.

Response Filtering

Remove selected internal metadata from public responses.

Protocol Adaptation

In some architectures, gateways bridge client-facing and internal protocols.

Avoid turning the gateway into a giant business application

Heavy domain logic inside the gateway creates coupling and makes every service deployment dependent on a central component.

API Composition

Response Aggregation

One client API may need data from several backend services.

Mobile App

GET /home

API Gateway

Aggregate response

Profile

Orders

Recommendations

Combine

One Client Response

Aggregation creates latency coupling

If the gateway waits for five services, the slowest service can dominate response latency. Use parallelism, timeouts, partial responses or dedicated composition services where appropriate.

API Design

Backend for Frontend — BFF

Different clients can have very different data and latency requirements.

Web

Mobile

Partner

Web BFF

Mobile BFF

Partner API

Shared Microservices

A mobile client may prefer compact responses and fewer network calls, while a web interface may require richer payloads. BFFs allow those experiences to evolve independently.

Performance

Gateway Caching

Selected read-only responses can sometimes be cached before reaching backend services.

GET /movies/popular

API Gateway Cache

TTL = 30 sec

Cache miss only

Movie Service

Good Candidates

Public, read-heavy, relatively stable responses.

Bad Candidates

Highly personalized, authorization-sensitive or frequently changing data without a strong cache key design.

Resilience

Timeouts

Never allow gateway requests to wait indefinitely for downstream services.

Gateway timeout:

Movie Service:
500 ms

Booking Service:
2 seconds

Payment Service:
5 seconds


If timeout expires:

return / fallback / retry
depending on operation

Timeouts must reflect business behavior

A 300 ms timeout may be reasonable for a recommendation API but completely wrong for a payment provider that legitimately requires several seconds.

Resilience

Retries

Gateways can retry selected transient failures, but retries are dangerous when used blindly.

Attempt 1
   ↓ failure

wait 100 ms + jitter

Attempt 2
   ↓ failure

wait 200 ms + jitter

Attempt 3

then stop

Safer Retry

Idempotent GET requests after transient network failures can often be retried with bounded backoff.

Dangerous Retry

Retrying a payment creation request without an idempotency key could duplicate side effects.

Failure Isolation

Circuit Breaker

A circuit breaker stops repeatedly calling a dependency that is already failing.

CLOSED

Requests pass normally

Failure threshold reached

OPEN

Reject immediately

Cooldown

HALF OPEN

Allow test requests

Fail fast

If the payment service is clearly unavailable, quickly failing requests can be better than allowing thousands of connections to wait until timeout.

Dynamic Infrastructure

API Gateway + Service Discovery

Backend service instances can appear, disappear and move as deployments autoscale.

API Gateway

Need Booking Service

Service Discovery

Find healthy instances

Booking 1

Healthy

Booking 2

Healthy

Booking 3

Starting

The gateway should not need permanently hard-coded IP addresses for every backend instance. Service discovery, DNS or infrastructure routing can resolve healthy destinations.

API Evolution

API Versioning & Canary Routing

Gateways can help migrate clients and backend versions gradually.

Versioned Routes

/api/v1/orders
        ↓
Order Service v1


/api/v2/orders
        ↓
Order Service v2

Canary Routing

Order Service v1:
90% traffic


Order Service v2:
10% traffic


Observe:

error rate
latency
business metrics


Then:

increase v2 gradually

Operations

Observability at the Gateway

Because almost every external request passes through the gateway, it is a valuable place to generate consistent telemetry.

Request Metadata

X-Request-ID:
req_18271


Trace-ID:
trace_9921


User:
user_42


Route:
/api/bookings


Backend:
booking-service


Latency:
128 ms


Status:
200

Access Logs

Method, route, status, latency and request identifiers.

Distributed Tracing

Propagate trace context through downstream calls.

Metrics

Request rate, latency, errors and throttling.

Audit Logs

Record security-sensitive gateway actions where required.

Do not log secrets

Authorization tokens, passwords, card data and sensitive PII should not be dumped into gateway logs.

Security Boundary

API Gateway Security

The gateway is exposed to untrusted external traffic and should be hardened accordingly.

TLS

Encrypt public API traffic.

Authentication

Reject invalid credentials early.

Authorization

Apply route and scope policies.

Rate Limiting

Limit abuse and expensive workloads.

Payload Limits

Prevent extremely large requests from consuming resources.

WAF Integration

Filter known malicious request patterns.

CORS

Control allowed browser origins for relevant APIs.

Private Backends

Avoid exposing internal services directly when not required.

Reliability

Avoiding an API Gateway Single Point of Failure

If every request uses the gateway, the gateway itself must scale horizontally and survive instance failures.

Users

Load Balancer

Gateway traffic

Gateway 1

Gateway 2

Gateway N

Backend Services

Stateless Gateway

Keep gateway instances easy to add and replace where possible.

Multiple Instances

Do not depend on one gateway process.

Health Checks

Remove unhealthy instances from routing.

Autoscaling

Scale gateways with request volume, CPU or other relevant metrics.

External State

Shared limits or policy state may live in dedicated distributed infrastructure.

Graceful Drain

Stop sending new requests before terminating gateway instances.

Global Architecture

Multi-Region API Gateway

Global systems can route users to gateways deployed close to them or in healthy regions.

Global Users

Global Traffic Manager

Latency + health routing

India Gateway

US Gateway

EU Gateway

Global policies create coordination questions

Authentication keys, global rate limits, API configuration and policy changes need a defined replication and consistency strategy across regions.

Case Study

BookMyShow-Style API Gateway Architecture

A ticket-booking platform can use an API Gateway to protect and organize its public API surface.

Interview-style architecture

This is a conceptual ticket-booking architecture for interview preparation. It is not a claim about BookMyShow's private production implementation.

Web / Mobile

Ticket buyers

CDN / WAF

Edge protection

Load Balancer

API Gateway

Auth • Rate limit • Route • Trace

Movies

Booking

Seat booking

Payment

Transactions

User

Profiles

Example Public Routing

GET /api/movies/*
        ↓
Search Service


GET /api/shows/*
        ↓
Show Service


POST /api/bookings
        ↓
Booking Service


POST /api/payments
        ↓
Payment Service


GET /api/users/me
        ↓
User Service

Flash Sale Request Flow

Request reaches edge infrastructure.

Gateway validates authentication where required.

Rate limiter checks IP, user and booking quotas.

Gateway validates request format.

Request receives correlation and trace IDs.

Gateway routes booking request to a healthy booking service.

Booking service manages seat locking and durable booking logic.

Payment service handles payment using idempotent operations.

Gateway returns the final API response.

Gateway does not solve double booking

The gateway protects and routes the API. Booking correctness still belongs to the booking service, Redis seat holds, database transactions, locking and idempotency.

Production Engineering

API Gateway Failure Scenarios

Because the gateway sits in the critical request path, failure handling is extremely important.

Gateway Instance Crashes

Load balancer should stop routing traffic to the unhealthy instance.

Backend Service Fails

Use timeouts, circuit breakers and appropriate fallback behavior.

Backend Becomes Slow

Bound request duration so gateway resources are not consumed indefinitely.

Authentication Provider Is Down

Define whether cached verification data or another safe fallback is possible.

Redis Rate Limiter Is Down

Choose fail-open or fail-closed behavior according to endpoint risk.

Gateway CPU Saturates

Autoscale, optimize policies and avoid heavy business computation at the gateway.

Retry Storm

Bound retries and add exponential backoff plus jitter.

Bad Gateway Configuration

Use validation, staged rollout and rapid rollback for routing/policy changes.

Region Failure

Global traffic routing can redirect requests to another healthy region when architecture allows it.

Certificate Expiry

Automate certificate monitoring and renewal.

Observability

API Gateway Metrics to Monitor

Gateway metrics provide a broad view of client traffic and backend health.

Requests/sec

P50 Latency

P95 Latency

P99 Latency

2xx Rate

4xx Rate

5xx Rate

429 Rate

Auth Failures

Timeout Rate

Retry Rate

Circuit Opens

Gateway CPU

Gateway Memory

Active Connections

Backend Latency

Backend Errors

Route Traffic

Cache Hit Ratio

TLS Errors

Interview Preparation

API Gateway Interview Questions

You should be comfortable discussing all of these trade-offs in an HLD interview.

What is an API Gateway?

Why do microservices use an API Gateway?

API Gateway vs Load Balancer?

API Gateway vs Reverse Proxy?

Where would you place a gateway in the request path?

How does path-based routing work?

How would you implement host-based routing?

How would the gateway perform authentication?

How would JWT validation work at the gateway?

Authentication vs authorization?

Should services trust the gateway completely?

Where should rate limiting happen?

How would free and premium users get different API quotas?

How can the gateway validate requests?

What is request transformation?

Should business logic live in the gateway?

What is API aggregation?

What are the problems with excessive API aggregation?

What is the Backend for Frontend pattern?

When should gateway responses be cached?

How should gateway timeouts be configured?

When is retrying safe?

Why can gateway retries duplicate payments?

What is a circuit breaker?

Explain CLOSED, OPEN and HALF-OPEN circuit states.

How does the gateway discover backend service instances?

How would you implement API versioning?

How would you perform canary routing?

How would you generate distributed trace IDs?

What should never be logged by the gateway?

How do you prevent the gateway from becoming a single point of failure?

Should gateway instances be stateless?

How would you scale the API Gateway?

How do you handle millions of requests per second?

How would a multi-region gateway architecture work?

How would you handle a gateway configuration failure?

How would you protect a BookMyShow flash sale at the gateway?

What should happen when Redis-based rate limiting fails?

What gateway metrics would you monitor?

Quick Revision

Frequently Asked Questions

Short answers to common API Gateway system-design questions.

What is an API Gateway in system design?

An API Gateway is an entry point placed in front of backend services that can route requests and handle cross-cutting concerns such as authentication, authorization, rate limiting, observability, request transformation and policy enforcement.

What is the difference between an API Gateway and a load balancer?

A load balancer primarily distributes traffic across backend instances, while an API Gateway commonly performs application-level routing and policies such as authentication, authorization, throttling, transformations and API composition.

Can an API Gateway become a single point of failure?

Yes. A production gateway should therefore run as multiple instances or use highly available managed infrastructure behind redundant traffic routing.

Should business logic be placed inside an API Gateway?

Usually the gateway should focus on cross-cutting concerns and lightweight composition. Core domain business logic generally belongs in backend services.

Should API Gateway perform authentication?

It is common for gateways to validate tokens, API keys or other credentials before forwarding requests, while backend services should still enforce authorization appropriate to their own resources.

Mastering System Design — HLD

Learn how the API Gateway fits into a complete scalable architecture.

Combine API Gateway, caching, load balancing, databases, queues, rate limiting, locking, concurrency and failure handling in complete HLD interview designs.

API GatewayCachingLoad BalancingDatabasesQueuesRate LimitingBookMyShowScalability

Continue Learning