API Design
EE 547 - Unit 6
Process Isolation - Separate Failure Domains
Moving from modules to separate processes
Same code, different execution model:
# User service (separate process)
# Listens on port 8001
@app.route('/users', methods=['POST'])
def create_user():
email = request.json['email']
password = request.json['password']
user_id = generate_id()
hash_pwd = hash_password(password)
store_user(user_id, email, hash_pwd)
return {'user_id': user_id}
# Booking service (separate process)
# Listens on port 8002
@app.route('/bookings', methods=['POST'])
def create_booking():
user_id = request.json['user_id']
flight_id = request.json['flight_id']
# HTTP request instead of function call
response = requests.get(f'http://localhost:8001/users/{user_id}')
user = response.json()
if user['is_active']:
return store_booking(user_id, flight_id)Why separate processes:
- Failure isolation: User service crash doesn’t terminate booking service
- Resource isolation: CPU-intensive user validation doesn’t block booking requests
- Independent deployment: Update user service without restarting booking service
- Technology flexibility: User service in Python, booking service in Go
Process boundaries isolate failures
HTTP Request Structure - Client to Server
HTTP request anatomy:
GET /users/123 HTTP/1.1
Host: user-service.airline.com
Authorization: Bearer eyJhbGc...
Accept: application/json
User-Agent: booking-service/2.1.0Request line components:
- Method:
GET- what operation to perform - Path:
/users/123- which resource to access - Protocol:
HTTP/1.1- version of HTTP
Request headers (metadata):
Host: Which server to route to (required in HTTP/1.1)Authorization: Credentials for authenticationAccept: What response format client understandsUser-Agent: Identifies client making request
Headers are key-value pairs: Header-Name: value
Empty line separates headers from body
Requests without body (GET, DELETE) end after headers
Measured request size:
- Typical GET request: 250-400 bytes
- Headers: 200-350 bytes
- Request line: 50 bytes
Request sent as plain text over TCP
HTTP Response Structure - Server to Client
HTTP response anatomy:
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 145
Cache-Control: max-age=300
Date: Mon, 15 Jan 2025 14:30:00 GMT{
"user_id": 123,
"email": "alice@example.com",
"is_active": true,
"created_at": "2024-01-10T08:00:00Z"
}Status line components:
- Protocol:
HTTP/1.1 - Status code:
200- numeric result indicator - Reason phrase:
OK- human-readable description
Response headers:
Content-Type: Format of response body (JSON, HTML, etc)Content-Length: Body size in bytesCache-Control: How long response can be cachedDate: When response was generated
Response body:
- Actual data returned by server
- Format specified by
Content-Typeheader - In this case: JSON with user data
Empty line separates headers from body (same as request)
Response mirrors request structure
4xx vs 5xx: Client Problem vs Server Problem
4xx = Your request has a problem
Response: 400 Bad Request
{
"errors": [
{"field": "email", "message": "Invalid email format"},
{"field": "age", "message": "Must be integer"}
]
}Client must fix the request:
- Validate input before sending
- Check required fields
- Use correct data types
5xx = Server has a problem
# Server code with bug
@app.route('/users/<id>')
def get_user(id):
user = db.query(f"SELECT * FROM users WHERE id = {id}")
return user.to_dict() # Crashes if user is NoneResponse: 500 Internal Server Error
Client should retry (server might recover):
- Use exponential backoff
- Have circuit breaker
- Log for debugging
Retry strategies differ:
4xx errors: Don’t retry same request
- Fix the problem first
- 401: Get new token
- 429: Wait for rate limit reset
5xx errors: Retry might work
- Server might recover
- Different server might work
- Use exponential backoff
Safe and Unsafe Methods: Retry Implications
Safe methods can be called without side effects
# Safe to retry, cache, prefetch
GET /users/123
GET /users/123 # Same result
GET /users/123 # Same result
# Browser/proxy can cache
Cache-Control: max-age=300Unsafe methods change server state
# DELETE is idempotent but unsafe
DELETE /users/123 # Returns 204 No Content
DELETE /users/123 # Returns 404 Not Found
DELETE /users/123 # Returns 404 Not Found
# Final state same, but state did change
# POST is neither safe nor idempotent
POST /orders # Creates order 1
POST /orders # Creates order 2 (duplicate!)
POST /orders # Creates order 3 (duplicate!)Network failure handling:
Retry safety:
Always safe: GET
Safe if idempotent: PUT, DELETE
Dangerous: POST, PATCH
Need idempotency keys for POST/PATCH
Request with Body - POST Example
Creating new booking via POST:
POST /bookings HTTP/1.1
Host: booking-service.airline.com
Content-Type: application/json
Content-Length: 215
Authorization: Bearer eyJhbGc...{
"user_id": 123,
"flight_id": 456,
"seat": "12A",
"payment": {
"method": "credit_card",
"amount": 450.00,
"currency": "USD"
},
"notifications": {
"email": true,
"sms": false
}
}Additional headers for body:
Content-Type: Specifies body format (JSON, XML, form data)Content-Length: Exact size in bytes (required by HTTP/1.1)
Server response:
201 Created status indicates:
- New resource successfully created
Locationheader provides URL to access new resource- Response body contains resource details
POST request includes data in body
Connection Lifecycle - TCP Under HTTP
HTTP runs over TCP connection:
1. TCP handshake (connection establishment):
Client Server
| |
|--- SYN -------->| (50ms)
|<-- SYN-ACK -----| (50ms)
|--- ACK -------->| (50ms)
| |
[TCP established]- 3-way handshake establishes connection
- Total latency: 150ms (client-server round trip)
- Required before any HTTP data sent
2. HTTP request/response over established connection:
- Request sent over TCP connection
- Response returned on same connection
- Total: 100ms for request/response
3. Connection close:
Total measured latency for single request:
- TCP handshake: 150ms
- HTTP request/response: 100ms
- Total: 250ms
Geographic impact (measurements):
- Same datacenter: 1-2ms per round trip
- Cross-coast US: 60-80ms per round trip
- Transpacific: 150-200ms per round trip
3-way handshake before HTTP request
Connection Pooling: Managing Concurrent Requests
Problem: Service needs to handle many concurrent requests
Single connection serves requests sequentially:
Connection pool serves requests in parallel:
Connection 1: [Req1]->[Resp1] [Req4]->[Resp4]
Connection 2: [Req2]->[Resp2] [Req5]->[Resp5]
Connection 3: [Req3]->[Resp3]Connection pool implementation:
from urllib3 import PoolManager
# Create pool with size limits
pool = PoolManager(
num_pools=10, # Max 10 different hosts
maxsize=20, # Max 20 connections per host
block=True # Wait if pool exhausted
)
# Connections managed automatically
response = pool.request('GET', 'http://api/users/123')
# Connection returned to pool after response readPool sizing considerations:
- Too small: Requests wait for available connection
- Too large: Memory overhead, server connection limits
- Typical: 10-50 connections per host
Pool exhaustion behavior:
# Pool size: 2, but 3 concurrent requests
pool = PoolManager(maxsize=2)
# Thread 1: Gets connection
# Thread 2: Gets connection
# Thread 3: Blocks waiting for available connection
# Thread 1 completes: Connection returned to pool
# Thread 3: Gets recycled connectionReal scenarios requiring pools:
- Web server → Database (10-20 connections)
- API Gateway → Backend services (50-100 per service)
- Microservice → Other microservices (10-30 per service)
Request Tracing: Following Failures Across Services
Problem: Request fails somewhere in chain of services
User reports “booking failed” - what actually happened?
# Three services, thousands of concurrent requests
[14:23:01.123] booking-service: Processing request
[14:23:01.234] user-service: Database query failed
[14:23:01.345] payment-service: Processing payment
[14:23:01.456] booking-service: Request failed
# Which events are related to the user's failure?Solution: Thread request ID through all services
# Generate ID at entry point
@app.before_request
def assign_request_id():
request_id = request.headers.get('X-Request-ID',
str(uuid.uuid4()))
g.request_id = request_id
# Forward to downstream services
headers = {
'X-Request-ID': g.request_id,
'Authorization': get_token()
}
response = requests.get(user_service_url, headers=headers)
# Include in every log message
logger.info(f"[{g.request_id}] Processing user {user_id}")
REST - Architectural Style for APIs
REST: Representational State Transfer
Architectural style, not a protocol or standard
Coined by Roy Fielding (2000 dissertation) based on HTTP design principles
Core idea: Resources identified by URLs, manipulated via standard HTTP methods
What REST is NOT:
- Not a specification with compliance tests
- Not a protocol like HTTP or SOAP
- Not limited to JSON (can use XML, HTML, etc)
- Not the only way to design APIs
What REST provides:
- Set of design principles for building APIs
- Conventions for mapping operations to HTTP methods
- Guidelines for URL structure
- Constraints that enable scalability and simplicity
REST vs other approaches:
- RPC-style:
/createUser,/getUser,/deleteUser(verbs in URLs) - REST-style:
POST /users,GET /users/123,DELETE /users/123(resources + methods)
REST treats everything as a resource accessible via URL
REST uses resource URLs + HTTP methods
Framework Choice: Flask
Python web frameworks for APIs:
- Flask: Minimal, explicit routing
- FastAPI: Modern, type-safe
- Django REST Framework: Full-featured
EE 547 uses Flask
Minimal abstractions make core concepts visible. Patterns transfer to FastAPI and Django REST.
Framework-agnostic concepts covered:
- Request/response flow
- URL routing and parameters
- Request data extraction
- Production deployment
Web Framework Architecture
Framework sits between HTTP server and handler code
Client
↓ HTTP Request
HTTP Server (gunicorn)
↓ WSGI
Flask Framework
↓ Calls
Handler Function
↓ Returns
Flask Framework
↓ WSGI
HTTP Server
↓ HTTP Response
ClientWhat Flask does:
- Match URL to function
- Parse incoming data
- Call handler function
- Build HTTP response
Handler implementation:
- Functions that process requests
- Business logic
- Return data
Simple Route Example
Connecting a URL to a function
from flask import Flask
app = Flask(__name__)
@app.route('/health')
def health_check():
return {'status': 'healthy'}What happens:
@app.route('/health')registers the route- Client sends:
GET /health - Flask sees
/healthmatches registered route - Flask calls
health_check()function - Function returns dict
- Flask converts to JSON response
Response:
HTTP/1.1 200 OK
Content-Type: application/json
{"status": "healthy"}Flask automatically:
- Sets status code to 200
- Sets Content-Type header
- Converts dict to JSON
HTTP Methods in Routes
Restricting which HTTP methods a route accepts
@app.route('/models', methods=['GET'])
def list_models():
return {'models': [...]}
@app.route('/models', methods=['POST'])
def create_model():
return {'id': 123}, 201Same URL, different methods:
GET /models→ callslist_models()POST /models→ callscreate_model()PUT /models→ 405 Method Not Allowed
Why separate by method:
- GET: Read data (list models)
- POST: Create data (new model)
- Different operations, different functions
- Clear separation of concerns
Default is GET only:
URL Parameters
Capturing values from the URL
URL: GET /models/42 Result: model_id = "42" (string)
Type conversion:
URL: GET /models/42 Result: model_id = 42 (integer)
URL: GET /models/abc Result: 404 Not Found (can’t convert to int)
Multiple parameters:
@app.route('/models/<int:model_id>/predictions/<pred_id>')
def get_prediction(model_id, pred_id):
return {'model': model_id, 'prediction': pred_id}URL: GET /models/42/predictions/xyz Result: model_id = 42, pred_id = "xyz"
Accessing Request Data: JSON Body
Reading JSON from request body
from flask import request
@app.route('/predict', methods=['POST'])
def predict():
data = request.json
# data is dict: {'features': [1, 2, 3]}
features = data['features']
result = model.predict(features)
return {'prediction': float(result)}Client sends:
POST /predict
Content-Type: application/json
{"features": [1.2, 3.4, 5.6]}Flask automatically:
- Checks Content-Type header
- Parses JSON string
- Creates Python dict
- Makes available as
request.json
Safe access with get():
Accessing Request Data: Query Parameters
Reading parameters from URL query string
@app.route('/models')
def list_models():
# GET /models?limit=10&status=trained
limit = request.args.get('limit', 100, type=int)
# limit = 10 (converted to int)
status = request.args.get('status')
# status = "trained"
models = fetch_models(limit=limit, status=status)
return {'models': models}Query string after ? in URL:
- Key-value pairs:
key=value - Multiple params:
&separator /models?limit=10&status=trained
request.args.get() parameters:
- First arg: parameter name
- Second arg: default value if missing
type=int: convert to integer
Without default:
With default:
Accessing Request Data: Headers
Reading HTTP headers
@app.route('/predict', methods=['POST'])
def predict():
# Authorization header
auth = request.headers.get('Authorization')
# "Bearer eyJhbGci..."
# Custom headers
request_id = request.headers.get('X-Request-ID')
# Content type
content_type = request.headers.get('Content-Type')
# Validate token
if not auth:
return {'error': 'Missing authorization'}, 401
if not validate_token(auth):
return {'error': 'Invalid token'}, 401
# Process request
return {'prediction': 0.87}Common headers:
Authorization: Auth tokensContent-Type: Body formatX-Request-ID: Request trackingUser-Agent: Client information
Headers case-insensitive:
Building Responses: Simple Return
Return dict → Flask converts to JSON
@app.route('/predict', methods=['POST'])
def predict():
result = model.predict(request.json['features'])
return {'prediction': float(result)}Response Flask generates:
HTTP/1.1 200 OK
Content-Type: application/json
{"prediction": 0.87}Flask automatically:
- Sets status code to 200 (success)
- Sets Content-Type to application/json
- Converts Python dict to JSON string
- Returns properly formatted HTTP response
This is the most common pattern:
- Simple and clean
- Works for most GET/PUT requests
- Default 200 status appropriate for success
Building Responses: Custom Status Code
Return tuple: (data, status_code)
@app.route('/models', methods=['POST'])
def create_model():
model_id = save_model(request.json)
return {'id': model_id}, 201Response:
HTTP/1.1 201 Created
Content-Type: application/json
{"id": 42}When to use different status codes:
201 Created - Resource successfully created (POST)
204 No Content - Success but no data to return (DELETE)
404 Not Found - Resource doesn’t exist
422 Unprocessable Entity - Validation failed
Building Responses: Adding Headers
Return tuple: (data, status, headers)
@app.route('/models', methods=['POST'])
def create_model():
model_id = save_model(request.json)
return {'id': model_id}, 201, {
'Location': f'/models/{model_id}',
'X-Request-ID': request.headers.get('X-Request-ID')
}Response:
HTTP/1.1 201 Created
Content-Type: application/json
Location: /models/42
X-Request-ID: abc-123
{"id": 42}Common response headers:
Location - URL of newly created resource
X-Request-ID - Echo back for tracking
Cache-Control - Control caching
Production: Why Not flask run
Development server not for production
Problems:
- Single process, single thread
- Handles one request at a time
- No crash recovery
- Debug mode exposes code
- Poor performance under load
Example:
@app.route('/predict')
def predict():
time.sleep(2) # Prediction takes 2 seconds
return {'result': 0.87}With flask run:
- Request 1: 0-2 seconds
- Request 2: 2-4 seconds (waits)
- Request 3: 4-6 seconds (waits)
- All sequential, no concurrency
Production needs:
- Multiple worker processes
- Concurrent request handling
- Automatic crash recovery
- Process management
Single process means:
- One request blocks others
- No parallelism
- Poor resource usage
- Unacceptable for production
Production: Gunicorn with Workers
Gunicorn - Production WSGI server
What this does:
- Starts 4 separate worker processes
- Each worker handles one request at a time
- 4 concurrent requests possible
- Load balanced across workers
Worker calculation:
workers = (CPU cores × 2) + 12-core machine → 5 workers 4-core machine → 9 workers
Same 2-second prediction with 4 workers:
- Requests 1-4: All start at 0s, finish at 2s
- Request 5: Starts at 2s when worker frees
4× improvement for concurrent requests
Configuration file:
Multiple workers = concurrent processing
Each worker is independent process
Static Files: Production Strategies
Problem: Flask serves files synchronously - blocks workers
What happens:
- Flask worker reads 500MB file
- Sends to client over network
- Worker blocked 30+ seconds
- 4 concurrent downloads = all workers blocked
Solution 1: Nginx serves static files
Nginx handles /static/* directly
Flask never sees these requests
Workers free for API callsSolution 2: S3 redirect pattern
@app.route('/models/<model_id>/download')
def download_model(model_id):
# Generate temporary S3 URL (expires in 1 hour)
s3_url = generate_presigned_url(
bucket='models',
key=f'{model_id}.pkl',
expires_in=3600
)
return redirect(s3_url)Flow:
- Client requests file from Flask
- Flask returns 302 redirect to S3
- Client downloads directly from S3
- Flask worker free in <1ms
Use S3 redirect for: Large files (>10MB), model weights, datasets, user uploads
Authentication Breach: LinkedIn 2012
June 2012: 6.5 million LinkedIn password hashes stolen
What LinkedIn did:
-- Stored password hashes (not plaintext) ✓
SELECT user_id, email, password_hash FROM users;
-- But used SHA-1 without salt ✗
password_hash = SHA1(password)What attackers did:
- Built password dictionary (10 million common passwords)
- Computed hashes once:
common_hashes = {
SHA1("password123"): "password123",
SHA1("123456"): "123456",
# ... 10 million entries
}- Searched stolen database:
Result: 90% of passwords cracked within 72 hours
Why it failed:
- SHA-1 designed for speed: GPU computes 10 billion hashes/second
- No salt: Same password → same hash
- One computation → thousands of accounts compromised
Same password “123456” compromised 753,000 accounts simultaneously
LinkedIn’s failure shows why hashing alone isn’t enough.
Identity in Distributed Systems
LinkedIn’s breach shows authentication failures cascade across systems
ML API requires authentication to prevent unauthorized access:
Every API request needs to answer two questions:
- Who is making this request? (Authentication)
- Can they perform this action? (Authorization)
In a single process, identity is implicit:
def delete_file(filepath):
# Running as OS user 'alice'
# OS checks if alice can delete filepath
os.remove(filepath)In distributed systems, identity must be explicit:
def handle_delete_request(request):
# Who sent this HTTP request?
user = authenticate(request) # Extract identity
# Can they delete this file?
if not authorize(user, 'delete', filepath):
return 403
delete_file(filepath)HTTP is stateless - no memory between requests:
- No persistent connection to maintain identity
- Each request independent
- Must prove identity every time
Three approaches to maintaining identity across requests:
- Include credentials every request (HTTP Basic Auth)
- Create server-side session (Cookie-based)
- Issue cryptographic proof (Token-based)
Each approach makes different trade-offs between security, scalability, and complexity.
Identity must be explicitly established in every HTTP request.
Password Authentication: Converting Secrets to Identity
Authentication transforms a secret into verified identity
Step 1: User provides credentials
Step 2: Server verifies against stored credentials
def authenticate(email, password):
user = db.query("SELECT * FROM users WHERE email = ?", email)
if verify_password(password, user.password_hash):
return user.id # Identity established
return NoneStep 3: Server issues proof of authentication
# Option A: Server-side session
session_id = generate_random_id()
sessions[session_id] = user_id
return {"session_id": session_id}
# Option B: Cryptographic token
token = jwt.encode({"user_id": user_id, "exp": time() + 3600})
return {"token": token}Password storage determines breach impact:
Never store plaintext passwords:
-- CATASTROPHIC: Database breach exposes all passwords
SELECT * FROM users WHERE password = 'secret123'Store cryptographic hashes instead:
Authentication converts credentials to identity proof.
Hash Functions: Time as a Defense Mechanism
LinkedIn used SHA-1 hashing - why wasn’t that enough?
First, understand why plaintext is catastrophic:
Database breach with plaintext passwords:
SELECT email, password FROM users LIMIT 3;
-- alice@example.com | secret123
-- bob@example.com | secret123
-- carol@example.com | password1All accounts immediately compromised.
Hash functions provide one-way transformation:
Cannot reverse: hash → original password (computationally infeasible)
Asymmetry favors attackers:
Legitimate use: Verify one password for one user
Attack: Try millions of passwords against all users
# Attacker with stolen hash database
common_passwords = ["password", "123456", "secret123", ...]
for password in common_passwords: # 10 million
test_hash = hash(password)
for stored_hash in database: # 100,000 users
if test_hash == stored_hash:
compromised.append(...)Solution: Make hashing deliberately slow
- SHA-1 (LinkedIn’s mistake): Designed for speed → 10 billion/second on GPU
- bcrypt: Designed for passwords → 10/second on GPU
- Time difference: 1 billion× slower
This is why LinkedIn’s passwords fell in 72 hours - SHA-1 allowed rapid dictionary attacks.
This asymmetry favors defenders over attackers.
Time cost makes brute force attacks impractical.
Salt: Preventing Parallel Attacks
LinkedIn’s second mistake: No salt
Even with slow hashing, common passwords create identical hashes:
Without salt, all users with “password123” have same hash:
hash("password123") → "ef92b778bafe771e89245b89ecb..."
# Database search finds 1,847 users with this hash
# All compromised with single hash computationSalt: Random value unique to each user
def create_user(email, password):
salt = generate_random_bytes(16) # Unique per user
password_hash = hash(salt + password)
db.store(email, salt, password_hash)Now identical passwords produce different hashes:
# User 1
salt1 = "a1b2c3d4..."
hash("a1b2c3d4..." + "password123") → "7f3c6b2a..."
# User 2
salt2 = "e5f6g7h8..."
hash("e5f6g7h8..." + "password123") → "92a8b7c6..."
# Different hashes despite same passwordImpact on attack strategy:
Without salt: One computation compromises all instances
target_hash = "ef92b778..."
if computed_hash == target_hash:
# Found password for ALL users with this hashWith salt: Must attack each user individually
for user in users:
for password in dictionary:
if hash(user.salt + password) == user.hash:
# Found password for ONE user onlySalt is not secret - stored with hash, prevents mass attacks not targeted ones
With salt, LinkedIn’s 753,000 “123456” users would each need individual attacks
Salt forces individual attacks per user.
Work Factors: Adaptive Security
Combining defenses: Slow hashing + Salt + Adaptive work factor
bcrypt’s configurable work factor scales with hardware improvements:
# Work factor determines iteration count: 2^factor
bcrypt.gensalt(10) # 2^10 = 1,024 iterations (2010)
bcrypt.gensalt(12) # 2^12 = 4,096 iterations (2020)
bcrypt.gensalt(14) # 2^14 = 16,384 iterations (2030)Each increment doubles computation time:
| Factor | Iterations | Time/Hash | Passwords/Day |
|---|---|---|---|
| 10 | 1,024 | 50ms | 1.7M |
| 11 | 2,048 | 100ms | 864K |
| 12 | 4,096 | 200ms | 432K |
| 13 | 8,192 | 400ms | 216K |
| 14 | 16,384 | 800ms | 108K |
Balancing security and usability:
def choose_work_factor():
# Target: 250ms computation time
test_password = b"benchmark"
for factor in range(10, 15):
start = time.time()
bcrypt.hashpw(test_password, bcrypt.gensalt(factor))
duration = time.time() - start
if duration > 0.250: # 250ms target
return factor
return 14 # Maximum reasonable factorMoore’s Law compensation:
- Computing power doubles every 2 years
- Increase work factor by 1 every 2 years
- Maintains constant security margin
Security parameter improves over time without code changes
Work factor increases maintain security despite hardware improvements.
Sessions vs Tokens: State Management Trade-offs
Server sessions: Centralized state
# Login creates session in shared store
session_id = generate_uuid()
redis.set(f"session:{session_id}", {
"user_id": 123,
"created": timestamp,
"permissions": ["read", "write"]
})
response.set_cookie("session_id", session_id)
# Every request requires lookup
def handle_request(request):
session_id = request.cookies.get("session_id")
session = redis.get(f"session:{session_id}") # Network call
if not session:
return 401Tokens: Distributed state
# Login creates self-contained token
payload = {
"user_id": 123,
"exp": timestamp + 3600,
"permissions": ["read", "write"]
}
token = jwt.encode(payload, SECRET_KEY)
return {"token": token}
# Every request validates locally
def handle_request(request):
token = request.headers["Authorization"].split(" ")[1]
payload = jwt.decode(token, SECRET_KEY) # CPU only
# No network call requiredTrade-offs in practice:
| Aspect | Sessions | Tokens |
|---|---|---|
| Revocation | Immediate | At expiration |
| Scaling | Requires shared store | Linear |
| Network calls | Every request | None |
| State size | Server: O(users) | Server: O(1) |
| Client complexity | Simple cookie | Header management |
Sessions require coordination; tokens are independent.
Authorization: From Identity to Permissions
Authentication establishes identity; authorization determines capabilities
def process_request(request):
# Step 1: Who are you? (Authentication)
user_id = validate_token(request.headers['Authorization'])
if not user_id:
return 401 # Unauthorized - don't know who you are
# Step 2: What can you do? (Authorization)
resource = request.path # e.g., /models/123
action = request.method # e.g., DELETE
if not has_permission(user_id, resource, action):
return 403 # Forbidden - know who you are, can't do this
# Step 3: Execute
return perform_action(resource, action)Three authorization models:
1. Role-Based (RBAC): Users have roles, roles have permissions
user.roles = ["developer", "viewer"]
role_permissions = {
"developer": ["read", "write", "deploy"],
"viewer": ["read"],
"admin": ["read", "write", "deploy", "delete"]
}
# Can user deploy? Check if any role has permission2. Attribute-Based (ABAC): Decisions based on attributes
can_access = (
user.department == resource.department and
user.clearance_level >= resource.sensitivity and
current_time in user.work_hours
)3. Resource-Based: Users own resources
Authorization determines what authenticated users can do.
Token Expiration and Revocation Trade-offs
Tokens can’t be recalled after issuing:
Once issued, JWT remains valid until expiration:
token_payload = {
"user_id": 123,
"exp": time() + 3600, # Valid for 1 hour
"scopes": ["read:data", "write:data", "delete:data"]
}Employee terminated at 2:00 PM:
- Token issued: 1:30 PM
- Token expires: 2:30 PM
- Problem: 30 minutes of unauthorized access
Three approaches to bounded revocation:
1. Short-lived access tokens (15 minutes)
access_token = create_token(expires_in=15*60)
refresh_token = create_token(expires_in=30*24*60*60)
# After termination, refresh fails
def refresh():
if user_terminated(refresh_token.user_id):
return 401 # No new access token
return create_access_token()2. Blacklist critical tokens
# Maintain revoked token list (small subset)
revoked_tokens = redis.set() # Only for terminated users
def validate_token(token):
if token.jti in revoked_tokens: # Quick check
return None
return decode_token(token)3. Version-based invalidation
# User has token_version in database
user.token_version = 2 # Increment on revocation
# Token includes version
token.version = 1
# Validation checks version
if token.version < user.token_version:
return 401 # Token outdatedTrade-off: Security (short expiry) vs Performance (fewer refreshes)
Shorter tokens increase security but require more refreshes.
Stateless Scaling: The Operational Advantage
Session-based scaling requires coordination
Adding servers with sessions:
# Server 1 has session for User A
# Server 2 has session for User B
# Load balancer must remember routing (sticky sessions)
# OR share sessions via Redis (single point of failure)Measured impact with 1000 requests/second:
- Sticky sessions: Imbalanced load (Server 1: 89%, Server 2: 11%)
- Redis sessions: 2ms added latency per request
- Redis failure: All users logged out
Token-based scaling is trivial
Adding servers with tokens:
Measured impact with 1000 requests/second:
- Round-robin load balancing: Even distribution (25% each for 4 servers)
- Token validation: 0.1ms CPU time
- Server failure: Requests rerouted, no user impact
Deployment advantages:
| Operation | Sessions | Tokens |
|---|---|---|
| Add server | Update session store | Add server |
| Remove server | Migrate sessions | Remove server |
| Deploy update | Coordinate session drain | Rolling update |
| Region failover | Replicate sessions | No change |
Cost at scale (10K concurrent users):
- Sessions: Redis cluster ($200/month) + Complexity
- Tokens: No infrastructure + 0.1% CPU overhead
Stateless architecture enables linear scaling without coordination overhead.
Tokens enable horizontal scaling without coordination.
JWT: Self-Contained Identity Tokens
JSON Web Tokens encode identity without server state
JWT structure: Three Base64-encoded parts separated by dots
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
eyJ1c2VyX2lkIjoxMjMsImVtYWlsIjoiYWxpY2VAZXhhbXBsZS5jb20iLCJleHAiOjE3MDUzMjQ4MDB9.
SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5cPart 1: Header (Algorithm and type)
Part 2: Payload (Claims about user)
{
"user_id": 123,
"email": "alice@example.com",
"exp": 1705324800, // Expires: Unix timestamp
"iat": 1705321200, // Issued at
"scopes": ["read", "write"]
}Part 3: Signature (Prevents tampering)
HMACSHA256(
base64(header) + "." + base64(payload),
server_secret_key
)Critical properties:
- Self-contained: All data in token, no database lookup
- Tamper-proof: Invalid signature = rejected token
- Stateless: Server only needs secret key
- Not encrypted: Payload readable by anyone (Base64 decode)
JWT structure enables stateless authentication.
JWT Validation: Cryptographic Trust
Signature prevents token forgery
Server creates token with secret:
import jwt
secret_key = "server-secret-abc123" # Only server knows
payload = {
"user_id": 123,
"email": "alice@example.com",
"exp": time.time() + 3600
}
token = jwt.encode(payload, secret_key, algorithm="HS256")
# Result: eyJhbGciOiJIUzI1NiIs...Client cannot modify token:
# Attacker tries to change user_id
decoded = base64.decode(token.split('.')[1])
decoded['user_id'] = 999 # Change to admin
fake_payload = base64.encode(decoded)
# But cannot generate valid signature without secret
fake_token = header + "." + fake_payload + "." + random_signature
# Server will reject: Invalid signatureServer validates with same secret:
def validate_token(token):
try:
payload = jwt.decode(token, secret_key, algorithms=["HS256"])
# Signature valid, token not expired
return payload
except jwt.InvalidSignatureError:
return None # Tampered token
except jwt.ExpiredSignatureError:
return None # Token too oldSymmetric (HS256) vs Asymmetric (RS256):
- HS256: Same secret for signing and verifying (simple, fast)
- RS256: Private key signs, public key verifies (allows third-party validation)
Only server with secret can create valid tokens.
JWT Claims: Standard Fields and Custom Data
Standard claims provide common functionality
Registered claims (predefined meanings):
{
"iss": "https://auth.company.com", // Issuer
"sub": "user:123", // Subject (user)
"aud": "https://api.company.com", // Audience (recipient)
"exp": 1705324800, // Expiration time
"nbf": 1705321200, // Not before
"iat": 1705321200, // Issued at
"jti": "a1b2c3d4" // JWT ID (unique)
}Time-based validation:
current_time = 1705323000 # Unix timestamp
# Token not yet valid (nbf = not before)
if current_time < token['nbf']:
return "Token not yet valid"
# Token expired
if current_time > token['exp']:
return "Token expired"
# Valid time window: nbf <= current_time <= expCustom claims for application data:
{
// Standard claims
"exp": 1705324800,
"iat": 1705321200,
// Custom application claims
"user_id": 123,
"email": "alice@example.com",
"roles": ["developer", "reviewer"],
"department": "engineering",
"permissions": {
"models": ["read", "write"],
"data": ["read"]
}
}Token size considerations:
- Each claim adds bytes to every request
- HTTP header limit: ~8KB
- Typical JWT: 200-500 bytes
- With permissions: 500-2000 bytes
Claims carry both metadata and application data.
Refresh Tokens: Balancing Security and Usability
Short access tokens + long refresh tokens minimize risk
Dual token pattern:
def login(email, password):
if authenticate(email, password):
# Short-lived for API calls
access_token = create_jwt(
user_id=123,
expires_in=15*60 # 15 minutes
)
# Long-lived for obtaining new access tokens
refresh_token = create_jwt(
user_id=123,
token_type="refresh",
expires_in=30*24*60*60 # 30 days
)
# Store refresh token for revocation
db.store_refresh_token(refresh_token)
return {
"access_token": access_token,
"refresh_token": refresh_token,
"expires_in": 900
}Token refresh flow:
def refresh_access_token(refresh_token):
# Validate refresh token
payload = jwt.decode(refresh_token, secret_key)
# Check if revoked (requires DB check)
if is_revoked(refresh_token):
return 401 # Revoked
# Issue new access token
new_access = create_jwt(
user_id=payload['user_id'],
expires_in=15*60
)
return {"access_token": new_access}Security boundaries:
- Compromise window: Maximum 15 minutes (access token lifetime)
- Refresh check: Database lookup only on refresh (every 15 min)
- Immediate revocation: Possible via refresh token blacklist
- Performance: 1 DB check per 15 minutes vs every request
Refresh tokens enable short access tokens without constant re-authentication.
OAuth 2.0: Delegated Authorization
OAuth allows third-party access without sharing passwords
OAuth solves password sharing with third parties:
# WITHOUT OAuth (dangerous):
# GitHub analyzer needs your Google Drive files
analyzer.login(
google_email="alice@gmail.com",
google_password="secret123" # Giving password to third party!
)OAuth authorization flow:
Step 1: User authorizes at provider
Browser → https://accounts.google.com/oauth/authorize?
client_id=github-analyzer&
redirect_uri=https://analyzer.com/callback&
scope=drive.readonly&
response_type=codeStep 2: Provider redirects with authorization code
Browser ← https://analyzer.com/callback?code=abc123Step 3: Exchange code for token (backend)
# Server-to-server, not visible to browser
response = requests.post('https://oauth2.googleapis.com/token', {
'code': 'abc123',
'client_id': 'github-analyzer',
'client_secret': 'secret-key-xyz', # Proves identity
'grant_type': 'authorization_code'
})
tokens = response.json()
# {
# "access_token": "ya29.a0ARrdaM...",
# "token_type": "Bearer",
# "expires_in": 3600,
# "scope": "drive.readonly"
# }Key principles:
- User never gives password to third party
- Provider controls exactly what access is granted
- Access can be revoked without changing password
OAuth enables access without sharing credentials.
OAuth Scopes: Granular Permissions
Scopes limit what applications can access
Requesting specific permissions:
# Application requests only what it needs
auth_url = "https://github.com/login/oauth/authorize?" + urlencode({
"client_id": "ml-trainer-app",
"scope": "repo:read user:email", # Specific permissions
"redirect_uri": "https://mlapp.com/callback"
})User sees requested permissions:
ML Trainer App wants to access your GitHub account:
✓ Read access to repositories
- View code, issues, pull requests
- View repository metadata
✓ Read user email addresses
- View primary email
- View verified status
✗ Will NOT be able to:
- Write to repositories
- Delete anything
- Access billing information
[Authorize] [Deny]Token contains granted scopes:
{
"access_token": "gho_16C7e42F292c6912E7710c838347Ae178B4a",
"token_type": "bearer",
"scope": "repo:read user:email", // What was actually granted
"expires_in": 28800
}Common scope patterns:
| Provider | Scope | Permission |
|---|---|---|
| GitHub | repo |
Full repository access |
| GitHub | repo:status |
Only commit status |
drive.readonly |
Read files only | |
drive.file |
Only files created by app | |
| Slack | chat:write |
Post messages |
| Slack | users:read |
View user information |
Principle of least privilege: Request minimum necessary scope
Scopes provide granular access control.
OAuth Grant Types: Different Flows for Different Needs
OAuth defines multiple flows for different scenarios
1. Authorization Code (web apps with backend)
# Most secure: Code exchanged server-to-server
# Frontend never sees client_secret
flow = "user → provider → code → backend → token"2. Client Credentials (service-to-service)
# No user involved, service authenticates directly
response = requests.post('https://oauth2.provider.com/token', {
'grant_type': 'client_credentials',
'client_id': 'batch-processor',
'client_secret': 'secret-xyz',
'scope': 'data.process'
})
# Used for: Cron jobs, backend services, APIs calling APIs3. Implicit Flow (deprecated, was for SPAs)
// Token returned directly in URL fragment
// Insecure: Token visible in browser history
// Replaced by: Authorization Code + PKCE4. Password Grant (deprecated, legacy systems)
# User gives password to application directly
# Defeats purpose of OAuth
# Only use: Migrating legacy systemsModern standard: Authorization Code + PKCE
# PKCE (Proof Key for Code Exchange) adds security
code_verifier = generate_random_string(128)
code_challenge = sha256(code_verifier)
# Include challenge in authorization request
# Include verifier in token exchange
# Prevents code interception attacksGrant type selection:
- User-facing web app → Authorization Code
- Backend service → Client Credentials
- Mobile/SPA → Authorization Code + PKCE
- Never → Password or Implicit
Different flows optimize for security and usability.
Security Considerations: Token Storage and Transmission
Where and how to store tokens determines security
Browser storage options:
// localStorage - Persistent but vulnerable to XSS
localStorage.setItem('token', jwt_token);
// ⚠️ Any JavaScript can read: <script>alert(localStorage.token)</script>
// sessionStorage - Per-tab, still XSS vulnerable
sessionStorage.setItem('token', jwt_token);
// httpOnly cookie - Not accessible to JavaScript
// ✓ XSS protected, ✗ CSRF vulnerable
Set-Cookie: token=jwt_token; HttpOnly; Secure; SameSite=Strict
// Memory only - Most secure but lost on refresh
const token = jwt_token; // JavaScript variableMobile app storage:
# iOS Keychain / Android Keystore (encrypted)
keychain.set('access_token', token, accessible=WHEN_UNLOCKED)
# SharedPreferences / UserDefaults (not encrypted)
# ⚠️ Accessible if device rooted/jailbrokenToken transmission:
# Always use Authorization header
headers = {'Authorization': f'Bearer {token}'}
# Never in URL parameters (logged, cached, shared)
# ✗ GET /api/data?token=jwt_token # Appears in logs!
# Never in request body for GET (non-standard)
# ✗ GET /api/data {"token": "jwt_token"}Security checklist:
- ✓ HTTPS only (never HTTP)
- ✓ Short expiration times
- ✓ Refresh token rotation
- ✓ Validate on every request
- ✓ Log anomalies (geographic changes)
- ✗ Don’t log token values
- ✗ Don’t store in git
Choose storage based on threat model.
Authorization Models: From Simple to Sophisticated
Evolution of authorization complexity
Level 1: Binary access (all or nothing)
if authenticated:
return FULL_ACCESS
else:
return NO_ACCESS
# Problem: Every authenticated user can do everythingLevel 2: Resource ownership
def can_access(user_id, resource):
if resource.owner_id == user_id:
return True
return False
# Problem: No sharing, no admin accessLevel 3: Role-based (RBAC)
user_roles = ["developer"]
role_permissions = {
"viewer": ["read"],
"developer": ["read", "write"],
"admin": ["read", "write", "delete"]
}
# Problem: Roles are coarse-grainedLevel 4: Attribute-based (ABAC)
def can_access(user, resource, action, context):
return (
user.department == resource.department and
action in user.permissions and
resource.sensitivity <= user.clearance_level and
context.time in user.work_hours and
context.location in user.allowed_locations
)
# Fine-grained but complexReal systems use hybrid approaches:
- Ownership for user-created resources
- Roles for system-wide permissions
- Attributes for special cases
More complex models provide finer control.
Resource Ownership: The Foundation Pattern
Users control resources they create
Database schema enforces ownership:
CREATE TABLE models (
id INTEGER PRIMARY KEY,
owner_id INTEGER NOT NULL,
name VARCHAR(255),
created_at TIMESTAMP,
is_public BOOLEAN DEFAULT FALSE,
FOREIGN KEY (owner_id) REFERENCES users(id)
);
CREATE TABLE model_shares (
model_id INTEGER,
user_id INTEGER,
permission VARCHAR(20), -- 'read', 'write'
PRIMARY KEY (model_id, user_id)
);Authorization logic:
def get_permission(user_id, model_id):
model = db.query("SELECT * FROM models WHERE id = ?", model_id)
# Owner has full control
if model.owner_id == user_id:
return ["read", "write", "delete", "share"]
# Check explicit shares
share = db.query("""
SELECT permission FROM model_shares
WHERE model_id = ? AND user_id = ?
""", model_id, user_id)
if share:
return share.permission.split(",")
# Public resources allow read
if model.is_public:
return ["read"]
return [] # No accessCommon patterns:
- Private by default: New resources only accessible to owner
- Explicit sharing: Owner grants specific permissions to specific users
- Public option: Owner can make resource world-readable
- Transfer ownership: Special operation with audit trail
Ownership provides natural access boundaries.
Role-Based Access Control (RBAC)
Users have roles, roles have permissions
Three-level hierarchy:
# 1. Users are assigned roles
user_roles = {
123: ["developer", "reviewer"],
456: ["viewer"],
789: ["admin", "developer"]
}
# 2. Roles define permissions
role_permissions = {
"viewer": {
"models": ["read"],
"data": ["read"]
},
"developer": {
"models": ["read", "write", "execute"],
"data": ["read", "write"],
"compute": ["submit"]
},
"reviewer": {
"models": ["read", "approve"],
"audit": ["read"]
},
"admin": {
"models": ["read", "write", "delete"],
"data": ["read", "write", "delete"],
"compute": ["submit", "cancel"],
"users": ["read", "write"]
}
}
# 3. Check if any role grants permission
def has_permission(user_id, resource_type, action):
user_role_list = user_roles.get(user_id, [])
for role in user_role_list:
permissions = role_permissions.get(role, {})
allowed_actions = permissions.get(resource_type, [])
if action in allowed_actions:
return True
return FalseRBAC advantages:
- Simple to understand and audit
- Easy to onboard users (assign role)
- Consistent permissions across resources
- Well-supported by frameworks
RBAC limitations:
- Roles proliferate over time
- Exceptions require new roles
- No context awareness
RBAC separates users from permissions via roles.
Attribute-Based Access Control (ABAC)
Access decisions based on multiple attributes
Attributes from multiple sources:
# User attributes
user = {
"id": 123,
"department": "ml_research",
"clearance_level": 3,
"location": "us-west",
"employee_type": "full_time",
"projects": ["alpha", "beta"]
}
# Resource attributes
resource = {
"id": 456,
"type": "model",
"classification": "confidential",
"department": "ml_research",
"project": "alpha",
"created_date": "2024-01-15",
"tags": ["experimental", "gpu_required"]
}
# Environment attributes
context = {
"time": "14:30",
"day": "weekday",
"ip_address": "10.0.1.5",
"network": "corporate",
"request_type": "api"
}
# Action being requested
action = "write"Policy evaluation:
def evaluate_access(user, resource, action, context):
# Rule 1: Department match required
if user["department"] != resource["department"]:
return False
# Rule 2: Clearance level check
clearance_required = {
"public": 1,
"internal": 2,
"confidential": 3,
"secret": 4
}
if user["clearance_level"] < clearance_required[resource["classification"]]:
return False
# Rule 3: Time-based access
if resource["classification"] == "secret":
hour = int(context["time"].split(":")[0])
if not (9 <= hour <= 17) or context["day"] == "weekend":
return False
# Rule 4: Project membership for write
if action == "write":
if resource["project"] not in user["projects"]:
return False
return True
ABAC evaluates multiple attributes for decisions.
Hierarchical Resources and Inheritance
Permissions flow down resource hierarchies
Resource hierarchy:
Organization
├── Projects
│ ├── Models
│ │ ├── Versions
│ │ └── Deployments
│ └── Datasets
│ ├── Training
│ └── Validation
└── Teams
└── MembersPermission inheritance:
class ResourceHierarchy:
def __init__(self):
self.permissions = {} # resource_id -> {user_id: permissions}
self.parents = {} # resource_id -> parent_id
def get_effective_permissions(self, user_id, resource_id):
"""Get permissions including inherited from parents"""
permissions = set()
# Walk up the hierarchy
current = resource_id
while current:
# Get direct permissions at this level
if current in self.permissions:
user_perms = self.permissions[current].get(user_id, [])
permissions.update(user_perms)
# Move to parent
current = self.parents.get(current)
return list(permissions)
def check_permission(self, user_id, resource_id, action):
perms = self.get_effective_permissions(user_id, resource_id)
# Check for explicit deny (overrides allow)
if f"deny:{action}" in perms:
return False
# Check for allow
return action in perms or "*" in permsExample scenario:
Permissions cascade down hierarchy unless overridden.
Delegation and Impersonation
Allowing users to act on behalf of others
Delegation: Temporary permission transfer
class DelegationManager:
def create_delegation(self, from_user, to_user, resource,
permissions, expires_at):
"""User explicitly grants subset of their permissions"""
# Verify delegator has permissions to delegate
delegator_perms = get_permissions(from_user, resource)
if not all(p in delegator_perms for p in permissions):
raise Error("Cannot delegate permissions you don't have")
delegation = {
"id": generate_id(),
"from_user": from_user,
"to_user": to_user,
"resource": resource,
"permissions": permissions,
"expires_at": expires_at,
"created_at": now()
}
db.store_delegation(delegation)
audit_log("DELEGATION_CREATED", delegation)
def check_permission(self, user_id, resource, action):
# Check direct permissions
if has_direct_permission(user_id, resource, action):
return True
# Check delegated permissions
delegations = db.query("""
SELECT * FROM delegations
WHERE to_user = ? AND resource = ?
AND expires_at > NOW()
""", user_id, resource)
for delegation in delegations:
if action in delegation.permissions:
audit_log("DELEGATED_ACCESS", {
"user": user_id,
"delegator": delegation.from_user,
"action": action
})
return True
return FalseService impersonation:
# Service account acts as user for background tasks
def run_as_user(user_id, task):
"""Service executes task with user's permissions"""
# Verify service account is authorized
if not is_service_account(current_identity):
raise Error("Only service accounts can impersonate")
# Create impersonation context
with impersonate(user_id) as context:
# All authorization checks use user_id's permissions
# But audit logs show both identities
result = execute_task(task, context)
return result
Delegation and impersonation enable flexible access.
Policy as Code: Declarative Authorization
Express authorization rules as policies, not procedures
Traditional procedural approach:
def can_access(user, resource, action):
if user.role == "admin":
return True
if resource.owner == user.id:
return True
if user.department == resource.department:
if action == "read":
return True
if action == "write" and user.seniority > 2:
return True
# Complex nested logic continues...
return FalsePolicy as code (declarative):
# policies.yaml
policies:
- id: admin-full-access
description: "Admins can do anything"
effect: allow
subjects: ["role:admin"]
actions: ["*"]
resources: ["*"]
- id: owner-full-access
description: "Owners control their resources"
effect: allow
subjects: ["user:*"]
actions: ["*"]
resources: ["*"]
condition: "resource.owner == subject.id"
- id: department-read-access
description: "Same department can read"
effect: allow
subjects: ["user:*"]
actions: ["read"]
resources: ["*"]
condition: "resource.department == subject.department"
- id: senior-write-access
description: "Senior staff can write in department"
effect: allow
subjects: ["user:*"]
actions: ["write"]
resources: ["*"]
condition: |
resource.department == subject.department AND
subject.seniority > 2Policy evaluation engine:
class PolicyEngine:
def evaluate(self, subject, action, resource, context):
applicable_policies = self.find_matching_policies(
subject, action, resource
)
# Explicit deny overrides allow
for policy in applicable_policies:
if policy.effect == "deny":
return False, f"Denied by {policy.id}"
# Any allow grants access
for policy in applicable_policies:
if policy.effect == "allow":
return True, f"Allowed by {policy.id}"
# Default deny
return False, "No matching allow policy"
Declarative policies separate logic from implementation.
Cloud Provider Authorization Models
AWS IAM: Policy-based with principals and resources
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::123456:user/alice"},
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": "arn:aws:s3:::ml-models/*",
"Condition": {
"IpAddress": {"aws:SourceIp": "10.0.0.0/8"},
"DateGreaterThan": {"aws:CurrentTime": "2024-01-01"}
}
}]
}GCP IAM: Role bindings at resource level
# Binding roles to identities on resources
bindings:
- role: roles/storage.objectViewer
members:
- user:alice@example.com
- serviceAccount:ml-trainer@project.iam
resource: projects/my-project/buckets/ml-models
- role: roles/ml.modelUser
members:
- group:ml-team@example.com
condition:
expression: request.time > timestamp("2024-01-01")Azure RBAC: Scope-based assignments
# Role assignment at different scopes
assignment = {
"roleDefinitionId": "/subscriptions/sub123/providers/Microsoft.Authorization/roleDefinitions/contributor",
"principalId": "user-guid-123",
"scope": "/subscriptions/sub123/resourceGroups/ml-resources"
}
# Permissions inherit down: Subscription → Resource Group → ResourceCommon patterns across providers:
- Principle of least privilege - Start with no access
- Deny overrides allow - Explicit deny wins
- Inheritance down hierarchy - Higher level grants flow down
- Conditions/constraints - Time, IP, attributes
Cloud providers use variations of policy-based access.
The Multiple Client Problem
Different clients need different data from same resources
REST endpoint returns fixed structure:
GET /api/users/123
{
"user_id": 123,
"email": "alice@example.com",
"name": "Alice Chen",
"profile_image": "base64...[2MB]",
"preferences": {...50 fields...},
"activity_history": [...200 entries...],
"connected_devices": [...},
"subscription": {...},
"recommendations": [...}
}Each client uses different subset:
Mobile app needs:
name,profile_image(thumbnail)- Downloads 3MB, uses 50KB
Admin dashboard needs:
email,subscription,activity_history- Downloads 3MB, uses 500KB
Analytics service needs:
user_id,preferences.language- Downloads 3MB, uses 1KB
REST over-fetches:
- 95% of transferred data unused
- Mobile battery drain
- Network bandwidth cost
- Server CPU for serialization
REST solutions are inadequate:
- Sparse fieldsets:
/users/123?fields=name,email(non-standard) - Multiple endpoints:
/users/123/mobile,/users/123/admin(proliferation) - API versioning: v1, v2, v3… (maintenance burden)
Different clients require different subsets of data.
GraphQL: Query Language for APIs
GraphQL lets clients specify exactly what data they need
Instead of multiple REST calls:
# REST: 3 requests, 3 round trips
user = GET('/users/123')
posts = GET('/users/123/posts?limit=5')
for post in posts:
comments = GET(f'/posts/{post.id}/comments?limit=2')Single GraphQL query:
query GetUserWithPosts {
user(id: 123) {
name
email
posts(limit: 5) {
title
createdAt
comments(limit: 2) {
text
author {
name
}
}
}
}
}Response matches query structure exactly:
{
"data": {
"user": {
"name": "Alice Chen",
"email": "alice@example.com",
"posts": [
{
"title": "GraphQL Benefits",
"createdAt": "2024-01-15",
"comments": [
{
"text": "Great post!",
"author": {"name": "Bob"}
}
]
}
]
}
}
}Key differences from REST:
- Single endpoint:
POST /graphqlfor everything - Client controls response shape
- Nested data in one request
- No versioning needed (fields are added/deprecated)
GraphQL fetches related data in single request.
GraphQL Type System
Everything in GraphQL has a type
Schema definition:
type User {
id: ID! # ! means non-null
name: String!
email: String!
posts: [Post!]! # Array of Posts (never null)
friendCount: Int
accountType: AccountType! # Enum type
}
type Post {
id: ID!
title: String!
content: String
author: User! # Relationship to User
comments: [Comment!]!
likes: Int!
}
enum AccountType {
FREE
PREMIUM
ENTERPRISE
}
type Query {
user(id: ID!): User # Can return null if not found
users(limit: Int = 10): [User!]!
}
type Mutation {
createUser(input: CreateUserInput!): User!
deleteUser(id: ID!): Boolean!
}Type system provides:
- Contract enforcement: Server must return correct types
- Query validation: Invalid queries rejected before execution
- Auto-documentation: Tools can introspect schema
- Code generation: Type-safe clients in any language
Query validation example:
Type system provides safety and tooling.
Query vs Mutation: Read vs Write Separation
GraphQL separates reads from writes explicitly
Query: Read operations (no side effects)
- Can be cached
- Executed in parallel
- Safe to retry
- No state changes
Mutation: Write operations (changes state)
mutation CreatePost {
createPost(input: {
title: "GraphQL Benefits"
content: "..."
authorId: 123
}) {
id # Return created post
title
publishedAt
author {
name
}
}
}- Never cached
- Executed serially (in order)
- Not safe to retry
- Returns modified data
Serial execution prevents race conditions:
mutation TransferFunds {
# These execute in order, not parallel
withdraw(account: "A", amount: 100) { balance }
deposit(account: "B", amount: 100) { balance }
}Convention: Mutations return the modified object so client can update its cache without refetching.
Queries parallelize; mutations serialize.
The N+1 Query Problem
GraphQL’s flexibility creates performance challenges
Query requests users and their posts:
Naive resolver implementation:
def resolve_users(limit):
# 1 query
return db.query("SELECT * FROM users LIMIT ?", limit)
def resolve_posts(user):
# Called for EACH user (N queries)
return db.query("SELECT * FROM posts WHERE user_id = ?", user.id)
# Total: 1 + 100 = 101 database queries!Problem scales with nesting:
Solution: DataLoader pattern (batching)
# Collects all user IDs, makes single query
post_loader = DataLoader(batch_load_posts)
def batch_load_posts(user_ids):
# Single query for all users
posts = db.query(
"SELECT * FROM posts WHERE user_id IN (?)",
user_ids
)
# Group by user_id and return in order
return group_by_user(posts)
# Now: 1 + 1 = 2 queries totalMeasured impact:
- Without DataLoader: 2.3 seconds (101 queries)
- With DataLoader: 45ms (2 queries)
- 51× faster
DataLoader batches queries to prevent N+1 problem.
Query Complexity and DOS Attacks
GraphQL’s flexibility enables malicious queries
Innocent-looking query with exponential cost:
query MaliciousQuery {
users(limit: 100) {
posts {
comments {
author {
posts {
comments {
author {
posts {
title
}
}
}
}
}
}
}
}
}Query analysis:
- Level 1: 100 users
- Level 2: 100 × 10 posts = 1,000
- Level 3: 1,000 × 20 comments = 20,000
- Level 4: 20,000 × 1 author = 20,000
- Level 5: 20,000 × 10 posts = 200,000
- Level 6: 200,000 × 20 comments = 4,000,000
- Total operations: 4,221,100
Single query can overwhelm server.
Protection mechanisms:
- Query depth limiting:
- Complexity scoring:
# Assign cost to each field
complexity = users(100) * 10 + posts * 5 + comments * 2
if complexity > 1000:
return Error("Query too complex")- Timeout protection:
Nested queries can create exponential load.
Caching Challenges with GraphQL
REST caching is straightforward
- URL identifies resource
- HTTP caching works (CDN, browser)
- Cache invalidation by URL
GraphQL breaks traditional caching
All queries go to single endpoint:
POST /graphql
{"query": "{ user(id: 123) { name } }"}
POST /graphql
{"query": "{ user(id: 123) { name email } }"}Same user, different queries, same URL.
Why POST breaks caching:
- POST requests typically not cached
- Request body contains query
- CDNs can’t cache POST
- Browser won’t cache POST
GraphQL caching strategies:
- Application-level caching:
- Field-level caching:
- Client-side normalized cache:
- Apollo Client stores by ID
- Updates all queries using that object
- Complex but powerful
POST requests and dynamic queries prevent HTTP caching.
GraphQL Architectural Trade-offs
GraphQL changes fundamental assumptions about APIs
Unified query interface:
# Single endpoint handles all queries
POST /graphql
query GetDashboardData {
user(id: 123) {
name
recentPosts(limit: 3) {
title
comments(limit: 1) {
text
}
}
}
}Contrast with REST equivalent:
GET /users/123 # User data
GET /users/123/posts # User's posts
GET /posts/456/comments # Comments for each post
GET /posts/789/comments
GET /posts/012/commentsPerformance characteristics:
GraphQL advantages:
- Fewer network round trips
- Precise data fetching (no over-fetching)
- Strong typing prevents runtime errors
GraphQL costs:
- Query parsing overhead
- Complex caching (can’t use HTTP cache)
- Potential for expensive queries
- N+1 query problems without careful design
Error handling differences:
REST: HTTP status codes indicate error types
GraphQL: Always returns 200 with error details
Each approach optimizes different aspects of API interaction.
Complexity Management: When Simple Becomes Hard
GraphQL’s flexibility creates new complexity
Simple REST endpoint:
@app.route('/users/<int:user_id>')
def get_user(user_id):
user = User.query.get_or_404(user_id)
return jsonify(user.to_dict())Equivalent GraphQL implementation:
# Schema definition
type_defs = """
type User {
id: ID!
name: String!
posts: [Post!]!
}
type Query {
user(id: ID!): User
}
"""
# Resolver with N+1 protection
def resolve_user(obj, info, id):
return User.query.get(id)
def resolve_posts(user, info):
# Without DataLoader: N+1 problem
# With DataLoader: Complex batching logic
return post_loader.load(user.id)
# Query complexity analysis
def analyze_query_complexity(query_ast):
complexity = 0
for field in query_ast.selections:
complexity += calculate_field_cost(field)
if complexity > MAX_QUERY_COST:
raise GraphQLError("Query too complex")
return complexityOperational complexity increases:
Monitoring REST:
- HTTP status codes indicate health
- URL patterns for different resources
- Standard load balancer routing
Monitoring GraphQL:
- All queries return 200 (errors in body)
- Query analysis needed for performance
- Single endpoint for all traffic
- Custom metrics for query complexity
Flexibility introduces operational complexity.
The Distributed Timeout Problem
Network calls introduce unpredictable delays
Single process function call:
Distributed service call:
def calculate_score(data):
response = requests.post('http://ml-service/predict',
json=data) # ??? ms, unpredictable
return response.json()Sources of unpredictability:
- Network latency: 1-100ms base cost
- Server load: Queue time varies
- Geographic distance: Speed of light limits
- Network congestion: Shared bandwidth
- Service cold starts: 1-10 second delays
Timeouts cascade through service chains:
Service A calls Service B calls Service C:
# Service A: 30 second timeout
response_b = requests.get(url_b, timeout=30)
# Service B: 30 second timeout
response_c = requests.get(url_c, timeout=30)
# Service C: Takes 25 seconds to respondWhat happens:
- C takes 25 seconds (within its timeout)
- B waits 25 seconds, then processes (29 total)
- A waits 29 seconds, gets response just in time
- Any additional delay breaks everything
Timeout strategies must coordinate across service boundaries
Hierarchical timeouts:
# Service A: Generous timeout for user-facing request
timeout_a = 10.0 # 10 seconds
# Service B: Leaves buffer for processing
timeout_b = 8.0 # 8 seconds
# Service C: Tightest timeout for backend
timeout_c = 6.0 # 6 secondsEach layer reserves time for its own processing.
Coordinated timeouts prevent cascade failures.
Connection vs Request Timeouts
Different phases of network communication have different failure modes
Connection timeout: Establishing TCP connection
import socket
import requests
# Connection timeout: How long to wait for TCP handshake
requests.get('http://api.service.com/data',
timeout=(3, 30)) # (connect, read)
# ↑
# 3 seconds to establish connectionConnection establishment steps:
- DNS resolution: 10-100ms
- TCP handshake: 1-3 round trips
- TLS handshake: 2-3 round trips (HTTPS)
Typical connection timeout: 3-10 seconds
Read timeout: Waiting for response
# Read timeout: How long to wait for response after connection
requests.get('http://api.service.com/data',
timeout=(3, 30)) # (connect, read)
# ↑
# 30 seconds for complete responseWhy separate timeouts matter:
Connection timeout failures indicate:
- Service completely down
- Network infrastructure problems
- DNS resolution issues
- Firewall blocking connections
Read timeout failures indicate:
- Service overwhelmed (high queue time)
- Long-running operation
- Partial network failure
- Service processing issues
Retry strategy depends on timeout type:
def call_service(url, data, max_retries=3):
for attempt in range(max_retries):
try:
response = requests.post(url, json=data,
timeout=(3, 30))
return response.json()
except requests.ConnectTimeout:
# Connection failed - service likely down
# Retry immediately (fail fast)
continue
except requests.ReadTimeout:
# Request sent but no response
# Longer backoff (service may be overloaded)
time.sleep(2 ** attempt)
continue
raise ServiceUnavailableError()
Different timeout types indicate different failure modes.
Retry Strategies: When and How
Not all failures should trigger retries
Immediate retry (no backoff):
def immediate_retry(func, max_attempts=3):
for attempt in range(max_attempts):
try:
return func()
except ConnectionError:
# Network connectivity issue - retry immediately
if attempt == max_attempts - 1:
raise
continue
# Use for: Connection failures, DNS timeoutsExponential backoff with jitter:
import random
import time
def exponential_backoff_retry(func, max_attempts=5):
for attempt in range(max_attempts):
try:
return func()
except (ReadTimeout, ServerError) as e:
if attempt == max_attempts - 1:
raise
# Base delay: 2^attempt seconds
delay = 2 ** attempt
# Add jitter to prevent thundering herd
jitter = random.uniform(0, 0.1 * delay)
total_delay = delay + jitter
time.sleep(total_delay)
continue
# Retry sequence: 1s, 2s, 4s, 8s, 16s (with jitter)Fixed interval retry:
def fixed_interval_retry(func, interval=5, max_attempts=3):
for attempt in range(max_attempts):
try:
return func()
except ServiceUnavailableError:
if attempt == max_attempts - 1:
raise
time.sleep(interval) # Always wait 5 seconds
# Use for: Known service restart windowsWhen NOT to retry:
def should_retry(exception, response=None):
# Never retry these conditions
if isinstance(exception, AuthenticationError):
return False # 401 - bad credentials
if isinstance(exception, AuthorizationError):
return False # 403 - insufficient permissions
if response and response.status_code == 400:
return False # Bad request - won't improve
if response and response.status_code == 404:
return False # Not found - resource doesn't exist
# Retry these conditions
if isinstance(exception, (ConnectionError, ReadTimeout)):
return True # Transient network issues
if response and response.status_code in [500, 502, 503, 504]:
return True # Server errors - may recover
return False
Choose retry strategy based on failure type and system constraints.
Idempotency: Safe Retry Foundation
Retries are only safe when operations are idempotent
Problem: Non-idempotent operations
# Dangerous to retry - could double-charge customer
def charge_credit_card(customer_id, amount):
response = requests.post('https://payments.api/charge', {
'customer_id': customer_id,
'amount': amount,
'currency': 'USD'
})
# Network timeout after sending request
# Did the charge succeed? Unknown - timeout occurred before response
return response.json()
# Retry could result in:
charge_credit_card(123, 50.00) # $50 charged
# Timeout, retry...
charge_credit_card(123, 50.00) # Another $50 charged!Solution: Idempotency keys
import uuid
def charge_credit_card_safe(customer_id, amount, idempotency_key=None):
if not idempotency_key:
idempotency_key = str(uuid.uuid4())
response = requests.post('https://payments.api/charge', {
'customer_id': customer_id,
'amount': amount,
'currency': 'USD',
'idempotency_key': idempotency_key # Unique per logical operation
})
return response.json()
# Server implementation tracks processed keys:
def process_payment(request):
key = request.get('idempotency_key')
# Check if already processed
existing = db.query("SELECT * FROM payments WHERE idempotency_key = ?", key)
if existing:
return existing.response # Return same result as before
# Process payment
result = charge_card(request)
# Store result with key
db.execute("INSERT INTO payments (idempotency_key, response) VALUES (?, ?)",
key, result)
return resultIdempotency patterns:
- Natural idempotency (safe by design):
# GET requests
user = get_user(123) # Always safe to repeat
# PUT requests (full replacement)
update_user(123, {"name": "Alice", "email": "alice@example.com"})- Conditional updates (compare-and-swap):
def update_counter(counter_id, expected_value, new_value):
result = db.execute("""
UPDATE counters
SET value = ?
WHERE id = ? AND value = ?
""", new_value, counter_id, expected_value)
if result.rowcount == 0:
raise ConflictError("Counter was modified")
return new_value- Idempotency key tracking:
Idempotency keys enable safe retries of financial operations.
Circuit Breaker Pattern
Stop calling failing services to prevent resource exhaustion
Problem: Cascading failures
# Service A keeps trying to call failing Service B
def get_user_recommendations(user_id):
for attempt in range(5): # Keep retrying
try:
# Service B is down - this will always fail
response = requests.get(f'http://ml-service/recommend/{user_id}',
timeout=30)
return response.json()
except Exception:
time.sleep(2) # Wasting time and resources
continue
raise ServiceUnavailableError()
# Results in:
# - 5 × 30 second timeouts = 2.5 minutes per user
# - Thread pool exhaustion
# - Memory leak from pending requests
# - Service A becomes unavailable tooCircuit breaker solution:
import time
from enum import Enum
class CircuitState(Enum):
CLOSED = "closed" # Normal operation
OPEN = "open" # Failing fast
HALF_OPEN = "half_open" # Testing recovery
class CircuitBreaker:
def __init__(self, failure_threshold=5, recovery_timeout=60):
self.failure_threshold = failure_threshold
self.recovery_timeout = recovery_timeout
self.failure_count = 0
self.last_failure_time = None
self.state = CircuitState.CLOSED
def call(self, func, *args, **kwargs):
if self.state == CircuitState.OPEN:
if time.time() - self.last_failure_time < self.recovery_timeout:
raise CircuitBreakerOpenError("Service unavailable")
else:
self.state = CircuitState.HALF_OPEN
try:
result = func(*args, **kwargs)
self._on_success()
return result
except Exception as e:
self._on_failure()
raise
def _on_success(self):
self.failure_count = 0
self.state = CircuitState.CLOSED
def _on_failure(self):
self.failure_count += 1
self.last_failure_time = time.time()
if self.failure_count >= self.failure_threshold:
self.state = CircuitState.OPEN
# Usage with circuit breaker
ml_service_breaker = CircuitBreaker(failure_threshold=3, recovery_timeout=30)
def get_user_recommendations_safe(user_id):
try:
return ml_service_breaker.call(
lambda: requests.get(f'http://ml-service/recommend/{user_id}',
timeout=5).json()
)
except CircuitBreakerOpenError:
# Return cached recommendations or default
return get_default_recommendations(user_id)Circuit breaker states:
- Closed (normal): All requests pass through
- Open (failing): All requests fail immediately
- Half-open (testing): Single test request allowed
Metrics for circuit breaker tuning:
- Failure threshold: 3-10 failures (depends on service criticality)
- Recovery timeout: 30-300 seconds (based on typical recovery time)
- Success threshold: 1-3 successes to close circuit
Circuit breaker prevents cascade failures by failing fast.
Timeout and Retry Integration Strategy
Combining timeouts, retries, and circuit breakers
Layered resilience strategy:
import asyncio
from typing import Optional, Callable, Any
class ResilientServiceClient:
def __init__(self, base_url: str):
self.base_url = base_url
self.circuit_breaker = CircuitBreaker(
failure_threshold=3,
recovery_timeout=30
)
self.session = requests.Session()
# Connection pooling for efficiency
self.session.mount('http://', requests.adapters.HTTPAdapter(
pool_connections=10, pool_maxsize=20
))
async def call_service(self,
endpoint: str,
data: Optional[dict] = None,
max_retries: int = 3) -> dict:
"""
Resilient service call with integrated patterns:
- Hierarchical timeouts
- Exponential backoff retries
- Circuit breaker protection
- Request tracing
"""
request_id = generate_request_id()
for attempt in range(max_retries + 1):
try:
# Circuit breaker check
if self.circuit_breaker.state == CircuitState.OPEN:
raise CircuitBreakerOpenError(
f"Circuit breaker open for {self.base_url}"
)
# Calculate timeout (shorter on retries)
connect_timeout = 3.0
read_timeout = max(10.0 - (attempt * 2), 5.0)
start_time = time.time()
response = await self._make_request(
endpoint, data, request_id,
timeout=(connect_timeout, read_timeout)
)
# Success - reset circuit breaker
self.circuit_breaker._on_success()
# Log success metrics
duration = time.time() - start_time
self._log_request(request_id, endpoint, attempt, duration, "success")
return response.json()
except requests.exceptions.ConnectTimeout:
# Connection timeout - retry immediately
self._log_request(request_id, endpoint, attempt, None, "connect_timeout")
if attempt < max_retries:
continue
raise ServiceUnavailableError("Connection timeout")
except requests.exceptions.ReadTimeout:
# Read timeout - exponential backoff
self.circuit_breaker._on_failure()
self._log_request(request_id, endpoint, attempt, None, "read_timeout")
if attempt < max_retries:
backoff_time = (2 ** attempt) + random.uniform(0, 1)
await asyncio.sleep(backoff_time)
continue
raise ServiceUnavailableError("Read timeout")
except requests.exceptions.HTTPError as e:
if e.response.status_code >= 500:
# Server error - retry with backoff
self.circuit_breaker._on_failure()
if attempt < max_retries:
backoff_time = (2 ** attempt) + random.uniform(0, 1)
await asyncio.sleep(backoff_time)
continue
else:
# Client error - don't retry
raise
raise ServiceUnavailableError(f"Max retries exceeded for {endpoint}")
def _log_request(self, request_id: str, endpoint: str,
attempt: int, duration: Optional[float], status: str):
"""Structured logging for debugging and monitoring"""
logger.info({
"request_id": request_id,
"service": self.base_url,
"endpoint": endpoint,
"attempt": attempt,
"duration_ms": duration * 1000 if duration else None,
"status": status,
"circuit_breaker_state": self.circuit_breaker.state.value
})Real-world timeout hierarchy example:
Integrated patterns provide comprehensive resilience.
The Long Operation Problem and Async Solutions
Some operations take too long for synchronous HTTP
Typical HTTP request/response works for fast operations:
# Fast operation: 50ms
response = requests.get('https://api.service.com/users/123')
user = response.json() # Works fineLong-running operations break this model:
# Video transcoding: 5 minutes
response = requests.post('https://api.service.com/transcode',
json={'video_url': 'input.mp4'},
timeout=300) # Wait 5 minutes?
# Problems:
# - Client connection held open entire time
# - Network interruption loses everything
# - No progress visibility
# - Client can't do anything elseCore problem: Need to decouple submission from completion
Three solutions exist, each with different trade-offs:
- Polling: Client repeatedly checks “are you done yet?”
- Webhooks: Server calls client when done
- WebSockets: Persistent bidirectional connection
All three share the same pattern: Submit job → get job_id → retrieve result later. They differ in how the result is retrieved.
Pattern comparison at a glance:
Polling - Simple but wasteful:
job_id = submit_job()
while not done:
status = check_status(job_id) # Repeated HTTP requests
time.sleep(5) # Most return "not done yet"Webhooks - Efficient but complex setup:
job_id = submit_job(callback_url='https://my-app.com/done')
# Server POSTs result to callback_url when complete
# No wasted requests, but client needs public endpointWebSockets - Real-time but resource-intensive:
All three patterns decouple submission from completion.
Polling and Webhooks: Two Retrieval Strategies
Polling: Client-driven status checks
Submit once, check repeatedly:
# Submit → get job_id immediately
job_id = submit_job({'operation': 'transcode', 'input': 'video.mp4'})
# Poll until complete
while True:
status = check_status(job_id)
if status['complete']:
return status['result']
time.sleep(5) # Wait and try againServer tracks job state:
jobs["abc-123"] = {
"status": "processing", # pending → processing → completed/failed
"progress": 45,
"result": None
}Webhooks: Server-driven notifications
Submit with callback URL:
# Client submits with callback URL
job_id = submit_job({
'operation': 'transcode',
'input': 'video.mp4',
'callback_url': 'https://my-app.com/webhooks/transcode'
})
# Client provides endpoint - server calls this when done
@app.post('/webhooks/transcode')
def handle_complete(request):
data = request.json() # {job_id, status, result}
update_database(data['job_id'], data['result'])Server notifies client:
# When job completes, POST to client's callback_url
requests.post(callback_url, json={'job_id': job_id, 'result': result})Trade-offs comparison:
| Aspect | Polling | Webhooks |
|---|---|---|
| Efficiency | Wasteful (most checks return “not ready”) | Efficient (one notification) |
| Latency | poll_interval/2 average | Immediate |
| Client requirements | Simple HTTP client | Public endpoint required |
| Firewall-friendly | Yes (outbound only) | No (needs inbound) |
| Reliability | Client controls retry | Server must retry failed deliveries |
When to use:
- Polling: Mobile apps, browsers, firewall-restricted clients
- Webhooks: Server-to-server, CI/CD pipelines, payment processors
Polling is simple but wasteful; webhooks are efficient but require public endpoints.
WebSockets: Continuous vs Discrete Updates
Polling and webhooks handle discrete operations
Submit job → wait → get result. One submission, one result.
WebSockets handle continuous streams
# Connection stays open, updates flow continuously
ws.connect("wss://api.service.com/live")
ws.send({"subscribe": "job_updates"})
while True:
update = ws.recv() # Server pushes whenever state changes
# Progress: 25%, 50%, 75%, 100%
The connection itself is the communication channel, not individual HTTP requests.
Video transcoding (5 minutes)
Discrete: submit → wait → result
- Polling: 30 checks, 29 return “not ready”
- Webhook: 2 requests (submit + notification)
- WebSocket: Unnecessary overhead
Live dashboard (updates every second)
Continuous: constant stream of values
- Polling: 3600 requests/hour per client
- Webhook: Doesn’t fit (not discrete events)
- WebSocket: Push updates as they occur
Mobile app vs Backend service
Mobile can’t receive webhooks (no public endpoint):
- Must use polling or WebSocket
- Polling simpler for discrete operations
Backend can expose endpoints:
- Webhooks for discrete events (payments)
- WebSocket for continuous streams
Combining approaches for reliability:
# Webhook with polling fallback
job_id = submit_job(callback='https://my-app.com/webhook')
result = wait_for_webhook(timeout=300) or poll_until_done(job_id)Webhook efficiency when network is reliable, polling safety when it isn’t.
CORS - Browser Same-Origin Security
Problem: API works in Postman, fails in browser
// JavaScript in browser at http://localhost:3000
fetch('http://localhost:5000/predict', {
method: 'POST',
body: JSON.stringify({features: [1, 2, 3]})
})
// Error: CORS policy: No 'Access-Control-Allow-Origin' headerSame-origin policy - Browser security restriction:
- Requests allowed to same protocol + domain + port
- Requests blocked to different origins
Examples:
http://localhost:3000 → http://localhost:5000 Blocked - Different ports
https://app.example.com → https://api.example.com Blocked - Different subdomains
https://app.example.com → https://app.example.com Allowed - Same origin
Not an API problem - browser enforces this
Postman bypasses CORS (not a browser) curl bypasses CORS (not a browser) Browser JavaScript cannot bypass CORS
CORS - Server Response Grants Permission
Browser sends preflight OPTIONS request before actual request
OPTIONS /predict HTTP/1.1
Host: localhost:5000
Origin: http://localhost:3000
Access-Control-Request-Method: POST
Access-Control-Request-Headers: Content-TypeServer must respond with permission headers:
HTTP/1.1 200 OK
Access-Control-Allow-Origin: http://localhost:3000
Access-Control-Allow-Methods: POST, GET, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization
Access-Control-Max-Age: 3600Then browser sends actual request:
POST /predict HTTP/1.1
Host: localhost:5000
Origin: http://localhost:3000
Content-Type: application/json
{"features": [1, 2, 3]}Flask implementation:
from flask_cors import CORS
app = Flask(__name__)
CORS(app, origins=['http://localhost:3000'])
# Or manual headers
@app.after_request
def add_cors_headers(response):
response.headers['Access-Control-Allow-Origin'] = 'http://localhost:3000'
response.headers['Access-Control-Allow-Headers'] = 'Content-Type'
return response
Correlation IDs for Request Tracing
Tracing requests across multiple services requires unique identifiers
Three services generating thousands of log entries:
# Gateway logs (10,000 entries)
[14:23:01.123] Processing request
[14:23:01.134] Processing request
[14:23:01.145] Processing request
# User Service logs (5,000 entries)
[14:23:01.234] Database query
[14:23:01.245] Database query
[14:23:01.256] Database query failed
# Payment Service logs (8,000 entries)
[14:23:01.345] Processing payment
[14:23:01.356] Processing paymentWithout correlation: Cannot identify which entries belong to same request
With correlation ID: Thread unique identifier through all services
# Generate at API entry point
@app.before_request
def assign_request_id():
request_id = request.headers.get('X-Request-ID', str(uuid.uuid4()))
g.request_id = request_id
# Forward to downstream services
headers = {
'X-Request-ID': g.request_id,
'Authorization': get_token()
}
response = requests.post(user_service_url, headers=headers)
# Include in every log message
logger.info(f"[{g.request_id}] User {user_id} query failed")
Logging Best Practices - Structured Format
Structured logging: JSON format, not text strings
# Bad: Text logs hard to parse
logger.info(f"User {user_id} made prediction, took {duration}ms")
# Good: Structured JSON logs
logger.info(json.dumps({
"timestamp": "2024-01-15T10:30:45Z",
"level": "INFO",
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"user_id": 123,
"endpoint": "POST /predict",
"duration_ms": 247,
"status_code": 200
}))Why JSON:
- Easy to parse programmatically
- Query with tools like
jq - Aggregate metrics from logs
- Filter by any field
What to log:
request_id- Correlation across servicesuser_id- Which user affectedendpoint- What operationduration_ms- How long it tookstatus_code- Success or failureerror_message- What went wrong (if failed)
What NOT to log:
- Passwords or API tokens
- Credit card numbers
- Request bodies with PII (personally identifiable information)
- Full JWTs (contains user data)
API Gateway - Central Control Point
API Gateway sits between clients and backend services
Why gateway: Implement cross-cutting concerns once, not in every service
Six core functions:
1. Authentication/Authorization
- Validate API keys or JWT tokens
- Check permissions before routing
- Single point for auth logic
2. Rate Limiting
- Prevent abuse (100 requests/minute per client)
- Protect backend services from overload
- Return 429 when limit exceeded
3. Request Routing
- Route to service versions (v1 vs v2)
- A/B testing and canary deployments
- Load balance across instances
4. Response Caching
- Cache GET responses to reduce backend load
- Configurable TTL per endpoint
- Invalidation on mutations
5. Monitoring/Analytics
- Track request counts, latencies, error rates
- Per-client usage metrics
- Identify problem clients
6. CORS Headers
- Add Access-Control-Allow-Origin centrally
- Don’t implement in every backend service
Without gateway:
- Each service implements auth
- Each service adds CORS
- Each service does rate limiting
- Each service monitors itself
With gateway:
- Auth once at edge
- CORS once at edge
- Rate limiting once at edge
- Centralized monitoring
AWS API Gateway Configuration
AWS API Gateway - Managed service, no servers to run
Endpoint structure:
https://{api-id}.execute-api.{region}.amazonaws.com/{stage}/{resource}
https://abc123.execute-api.us-east-1.amazonaws.com/prod/predict
↑ ↑ ↑ ↑
API ID Region Stage ResourceConfiguration components:
Resources - URL paths
/users/predict/models/{id}
Methods - HTTP operations per resource
- GET /users
- POST /predict
- DELETE /models/{id}
Integration - Backend target
- Lambda function (serverless)
- HTTP endpoint (existing API)
- AWS service (DynamoDB, S3, etc.)
Stages - Environment separation
prod- Production trafficstaging- Pre-production testingdev- Development environment
Each stage has independent configuration
Usage plans - Rate limits per API key:
# Create usage plan
{
"name": "Basic Plan",
"throttle": {
"rateLimit": 100, # requests/second
"burstLimit": 200 # burst capacity
},
"quota": {
"limit": 10000, # requests
"period": "DAY" # per day
}
}Pricing:
- $3.50 per million requests
- $0.09 per GB data transfer
- Caching: $0.02/hour per GB cache
API Gateway Request Lifecycle
Complete request flow through AWS API Gateway
1. Client makes request
curl -X POST \
https://abc123.execute-api.us-east-1.amazonaws.com/prod/predict \
-H 'x-api-key: 8fk3jsl9dkfj3k4j' \
-H 'Content-Type: application/json' \
-d '{"features": [1.2, 3.4, 5.6]}'2. API Gateway validates API key
- Checks if key exists and is valid
- Returns 403 if invalid
3. API Gateway checks usage plan quota
- Checks rate limit (100 req/sec)
- Checks daily quota (10,000 req/day)
- Returns 429 if exceeded
4. API Gateway routes to backend
- Invokes Lambda function, or
- Forwards to HTTP endpoint, or
- Calls AWS service directly
5. Backend processes request
def lambda_handler(event, context):
features = json.loads(event['body'])['features']
prediction = model.predict(features)
return {
'statusCode': 200,
'body': json.dumps({'prediction': float(prediction)})
}6. API Gateway logs to CloudWatch
- Request ID, latency, status code
- Enables debugging and monitoring
