Cloud Patterns: Storage, Serverless, and Coordination

Check-in waits only for the intermediary to accept the message
The boarding-pass service takes the message when it runs
Decoupled in time: sender and receiver need not run at the same moment
Check-in now depends on the intermediary, so the intermediary must be more available than the services it separates

Managed Services Move Infrastructure Operation to the Provider

Managed service: infrastructure the provider operates and the application uses through API calls

The provider runs the machines, replicates across data centers, replaces failures, and adds capacity
The application creates a resource with a configuration, calls it with credentials, and pays per use
Storage, queues, topics, functions, and gateways are each offered this way

Events Record That Something Occurred

Three services act on every check-in

Commands: check-in asks each service to act

send("boarding-pass-jobs",
     {"passenger": 42, "flight": 7})
send("gate-list-updates",
     {"passenger": 42, "flight": 7})
send("notifications",
     {"passenger": 42, "text": "Checked in"})

Each message names its receiver and the action
A fourth service means a fourth send in check-in

Event: check-in records what occurred

publish("passenger-events",
        {"type": "passenger.checked_in",
         "passenger": 42, "flight": 7,
         "at": "2026-10-27T14:05:12Z"})

The message records that something occurred, in the past tense
Each service that needs the fact subscribes to it
A fourth service subscribes, and check-in does not change

Event-driven: services publish events and react to the events they subscribe to

Queues Reach One Receiver, Topics Reach Every Subscriber

Point-to-point: a queue

Each message goes to one receiver
Several workers of one service divide the messages
Fits work done once: generate one boarding pass

Publish-subscribe: a topic

Each subscriber gets its own copy of every event
A new subscriber needs no change to the publisher
Fits a fact several services act on: a passenger checked in

Managed Services Produce, Carry, and Process Events

Produce: S3 when an object is written, API Gateway when a request arrives, EventBridge Scheduler at a set time

Carry: SQS queues, SNS topics, Kinesis Data Streams logs

Process: Lambda runs a function for each delivered event or batch, with nothing running or billed between events

Run by the provider: machines, failures, and capacity, so each service is configured and called, not operated

Services Show Old State Until They Process the Event

After the reply: each service shows the check-in only once it has processed the event

A page that reads the gate list right after check-in shows “not checked in”
The passenger already holds a confirmation
Eventually consistent: the services agree once every event is processed, and disagree until then
It matters only where something reads a service’s state: a queued confirmation email has no reader to see it late

Failures Partway Through Leave Work Half Done

One database

BEGIN;
UPDATE bookings SET status = 'checked_in'
  WHERE passenger_id = 42 AND flight_id = 7;
INSERT INTO gate_list (flight_id, passenger_id)
  VALUES (7, 42);
COMMIT;

Both writes commit, or neither does

Two services, one event

gate-list service     passenger 42 added      done
notification service  "checked in" message    failed
check-in              replied "checked in"    done

The gate-list change stays after the notification fails
Nothing rolls back: the application retries the failed step
Or it runs a compensating action, a step that reverses the completed one
A retried step can run twice, so repeating it must not change the result

One Request Spreads Across Several Managed Services

Check-in, one request

The passport scan in storage, because other services read it later by name
The boarding pass as a queued message, because one worker makes it once
“Checked in” on a topic, because several services act on it

After the reply

The scan is stored and the message is queued, and the pass may not exist yet
No service records whether all three finished

Shared File Systems

File Systems Turn File Names Into Disk Blocks

Disk: numbered blocks of a fixed size, read and written by number

File system: the software in the operating system that gives blocks names

Directory tree: which names exist, and where
File record: a file’s size and its block numbers
Free-block list: which blocks are unused

Opening a file: a lookup in the tree, then the record, then the blocks

Cache: the tree, records, and free list are kept in memory, so most lookups never reach the disk

Block Volumes Give an Instance a Disk, Not a File System

Block volume: a disk reached over the network (EBS)

Stores and returns numbered blocks
Outlives the instance and attaches to a replacement
Instance store, by contrast, is a disk in the host machine: fast, and lost when the instance stops or terminates

File system on the instance

The instance’s operating system formats and mounts the volume
The tree, records, and free list live in that instance’s memory

Files belong to the instance that mounts the volume

Check-in on instance A writes /srv/passes/42-7.pdf
Behind a load balancer, the next request can land on B, where the file does not exist

Shared Block Volumes Corrupt Ordinary File Systems

Two instances, one volume

Each instance runs its own file system on the same blocks
Each caches its own free-block list in its own memory
Both find block 1000 free, and one file’s bytes overwrite the other’s

Ordinary file systems (ext4, XFS) assume one machine manages the volume

Network File Systems Serve Files From a Remote Server

Network file system: the file system runs on a server, and the instances send it their file operations

NFS is the standard protocol, built into Linux
One directory tree, one set of records, one free list, on the server
Every instance sees the same files at the same paths

Mounted like a disk

sudo mount -t nfs4 server:/ /srv/shared

open("/srv/shared/passes/42-7.pdf") is unchanged
The operating system turns it into an NFS request

Amazon EFS Serves One File System Through a Mount Target in Each Zone

AWS runs

The NFS servers and their failures
Storage replicated across availability zones
Capacity and throughput as files and requests grow

The application sets

A file system, one per shared directory tree
A mount target in each zone’s subnet, with a security group that admits port 2049
The mount on each instance, to the target in its own zone

EFS Storage Grows on Demand

Capacity

No size is set in advance: the file system grows as files are written and shrinks as they are deleted
A block volume, by contrast, is created at a size and billed for that size, used or not

Billing

Storage: about $0.30 per GB-month for files read often
Data read and written: billed per GB under Elastic throughput
Lifecycle rules can move files not read for a set period to a cheaper storage class, at the same path

Throughput

Scales with the requests the instances send, with no server to resize

Created and mounted

import boto3

efs = boto3.client("efs")
fs = efs.create_file_system(
    CreationToken="boarding-passes",
    ThroughputMode="elastic")
for subnet in ["subnet-a1", "subnet-b2"]:
    efs.create_mount_target(
        FileSystemId=fs["FileSystemId"],
        SubnetId=subnet,
        SecurityGroups=["sg-nfs"])

# on each instance
sudo mount -t nfs4 \
  fs-0a12.efs.us-east-1.amazonaws.com:/ \
  /srv/shared

No server, disk size, or replica appears in either step

Small-File Workloads Run Slowly on NFS

Round trip per operation: open, close, list, and every read or write not served from the client’s cache

About 1 ms or more each on EFS, against microseconds for a local file system with its metadata in memory
10,000 small files: three operations each, about 30 s

Large files

One open, then reads of large blocks streamed in sequence
Throughput, not round trips, sets the time

Workloads it serves poorly

Many small files: build trees, package installs, one file per record
Many small writes that must each reach the server: a database’s data files

NFS Changes Become Visible When the Writer Closes the File

Close-to-open consistency: the NFS guarantee

Instance A’s writes reach the server by the time A closes the file
Instance B sees them in any open that starts after that close
B, with the file already open, can keep reading its cached older bytes

Locks

Advisory: they work between programs that ask for them
Two writers that both lock the file take turns, and a writer that does not lock is not stopped

In practice: write, close, then open on another instance, and locks where two instances write the same file

Block Volumes and Network File Systems Mount Side by Side

/var/data on EBS

Seen by one instance, at local speed
Billed for the volume’s size
A database’s data directory, an instance’s own scratch files

/srv/shared on EFS

Seen by every instance that mounts it, one round trip per operation
Billed for the files stored
Boarding-pass templates and generated passes, read on any instance

Both: paths, rename, append, in-place writes, and locks, reached by a mount from inside one network

Object Storage

Object Storage Replaces File Operations With Whole-Object Requests

Object: bytes and metadata stored under a key in a bucket

Written whole by one PUT, replaced whole by the next
Read whole or by byte range with GET
No open, append, rename in place, or lock

What the trade gives

No file system state shared between clients: no tree, no free-block list
Any client with credentials reaches it over HTTPS, inside AWS or not
Stored in several availability zones, with no size set in advance

Conditional Writes Stop Two Writers From Replacing Each Other’s Object

Two writers, one key

A GET or LIST after a PUT returns the new object
Two plain PUTs both succeed, and the last one received replaces the first, with neither writer told

Conditional write: the PUT states what must be true at the key

s3.put_object(Bucket="airline-passes",
    Key="flights/7/2026-10-27/42.pdf",
    Body=pdf, IfNoneMatch="*")

IfMatch=<ETag> replaces the object only if it is still the version read

Designing Object Keys Around the Most Frequent Listing

Sorted keys, listed by prefix

S3 keeps keys in sorted order and returns a prefix’s keys together, 1,000 per request
Keys that share their leading part are listed in one run
Any other grouping is spread across the bucket

The key is the only index

The same rule as a partition key: designed for the most frequent query
At 1,080,000 passes, listing one flight reads 180 keys by flight and every key by passenger

Multipart Uploads Split Large Objects Into Independent Parts

Multipart upload: start, send numbered parts, complete

Parts of 5 MB to 5 GB, up to 10,000
The key shows nothing until the upload is completed
Parts go in parallel, and a failed part is resent alone

Also from independence

Parts sent as they are produced, before the total size is known
A restarted uploader lists the stored parts and sends only the missing ones
Required above 5 GB, the single-PUT limit

Cost: parts of an unfinished upload are billed until it is aborted

Presigned URLs Grant Time-Limited Access to One Object

Presigned URL: a request signed in advance, locally, with the application’s credentials

url = s3.generate_presigned_url(
    "put_object",
    Params={"Bucket": "airline-scans",
            "Key": "scans/42/passport.jpg"},
    ExpiresIn=300)

The signature binds

The method, the bucket, and the key
The expiry, and the credentials that signed it

Upload and download

PUT: the passport scan goes from the browser to S3, not through the API
GET: the boarding-pass PDF is linked with a URL that expires in minutes, from a private bucket

S3 Notifies Other Services When Objects Change

Event notification: S3 publishes an event when an object is created, overwritten, copied, deleted, or restored

Filtered by key prefix and suffix: scans/, .jpg
Sent to a queue, a topic, a function, or EventBridge
Carries the bucket, key, size, and ETag, not the bytes

Delivery

Asynchronous, usually within seconds, sometimes a minute or more
At least once: the same event can arrive twice
The passport check records the key and ETag it processed, so a repeat changes nothing
What runs the passport check, an instance or a function, is a separate choice

Queues

Slow Work Holds Up the Reply

Work the passenger does not need before the reply

Passport check: seconds to minutes, waiting on an outside document service
Boarding-pass generation: after the check passes
Confirmation email, gate-list update

Run inside the request

The reply waits for the slowest step
Instance terminated or restarted mid-request: no reply, and the work is lost with no record of where it stopped

Job Tables Keep Work Past the Instance That Accepted It

Thread on the instance: the handler starts the work in a thread and replies

Fast reply
Lost when the instance stops: scale-in, a release, a crash
One instance’s threads only, with no limit when check-ins spike

Job table: the handler inserts a row and replies

The row survives any instance
Any worker on any instance can take it

A worker claims one job

UPDATE jobs SET state = 'running',
                claimed_at = now()
WHERE id = (
  SELECT id FROM jobs
  WHERE state = 'queued'
  ORDER BY id
  FOR UPDATE SKIP LOCKED
  LIMIT 1)
RETURNING id, scan_key;

Two workers never claim the same row
A running job with an old claimed_at belonged to a worker that died, and is claimed again
Done: the worker sets state = 'done' and stores the result

Properties the work needs: kept until finished, taken by one worker, returned if that worker dies, processed by as many workers as are running

Message Queues Pass Work From Producers to Consumers in Arrival Order

Message queue: a service that stores messages from producers until consumers finish them

Send: the producer adds a message at the back and continues
Receive: a consumer takes the oldest message, hidden from other consumers while it works
Delete: the consumer removes it once the work is done

The job table’s properties, without the table

Producer and consumer never address each other
A message whose consumer dies becomes visible again
Consumers are added or removed without changing the producer

Amazon SQS Stores Messages Across Zones With No Capacity to Size

Amazon Simple Queue Service (SQS): message queues that applications call over HTTPS

AWS runs

Storage of every message across several availability zones
Capacity for any rate of sends and receives, with nothing to size
Hiding received messages and returning unfinished ones

Billing: about $0.40 per million requests, nothing for an idle queue

The application sets

A queue, named by a URL
How long received messages stay hidden, and how long unreceived ones are kept
Which services may send and receive, by IAM policy

Already in use: S3 sends each passport-scan event to an SQS queue, and the passport-check workers receive from it

SQS Keeps Each Message Until a Consumer Deletes It

Message: up to 1 MiB of text, kept for 4 days by default and up to 14

A larger payload is stored in S3, and the message carries its key
The S3 event for a passport scan is already such a message: bucket and key, not the image

Three calls

sqs.send_message(QueueUrl=q,
    MessageBody=json.dumps(event))

r = sqs.receive_message(QueueUrl=q,
    MaxNumberOfMessages=10)
for m in r.get("Messages", []):
    check_passport(m["Body"])
    sqs.delete_message(QueueUrl=q,
        ReceiptHandle=m["ReceiptHandle"])

Receive hands out a message and hides it
Delete is the consumer’s statement that the work is done
A consumer that stops between the two leaves the message in the queue

Unfinished Messages Return to the Queue After a Timeout

Visibility timeout: how long a received message stays hidden, 30 s by default

Consumer crashes: the message returns after the timeout, and another consumer finishes it
The timeout is the longest a failure delays the work

Shorter than the work

The message reappears while the first consumer is still running
Two consumers process it at once
A consumer with long work extends the timeout as it goes, up to 12 hours from the first receive

De-duplicating Messages Keeps Redelivery From Repeating Work

At-least-once delivery: every message is delivered, some more than once

A crash between the work and the delete is enough
So is a timeout shorter than the work

Idempotent consumer: the second delivery has no further effect

result = check_passport(scan)   # no side effects
table.put_item(
    Item={"pk": f"{key}#{etag}",
          "result": result},
    ConditionExpression=
        "attribute_not_exists(pk)")

The result and the message’s identity, the scan’s key and ETag, are written together, once
A failed condition means done already: delete the message and stop
A crash before the write leaves nothing recorded, so the redelivery does the check again

Failed Messages Move to a Dead-Letter Queue

Poison message: one that fails on every attempt, such as a scan that is not an image

Without a limit it is received, fails, and returns for as long as it is kept
Each attempt costs a receive and the work done before the failure
A delay queue holds a resent message for up to 15 minutes before it is visible, so a retry is spaced out

Redrive policy: a receive count after which SQS moves the message to a second queue

Kept there with its body and attributes, for inspection
Sent back to the main queue once the consumer is fixed
A dead-letter queue that is filling up is the signal that consumers are failing

Slow Consumers Build a Backlog

Backlog: grows at arrivals minus the processing rate, drains at the processing rate minus arrivals

Wave of 400/min for 30 min against 200/min: a backlog of 6,000 messages
Drained at 100/min afterward: another hour before the queue is empty

Competing consumers: workers on one queue take different messages

Four workers finish 400/min, and no backlog builds
Check-in sends at the same rate either way
The oldest message’s age is the passenger’s delay

FIFO Queues Keep Order Within a Message Group

Standard queue

Order is best effort: two messages can arrive in the other order
At-least-once delivery, as before
Nearly unlimited throughput

Booking 9001, two messages

1  seat changed to 14C
2  seat changed to 21A

Applied out of order, the booking ends with 14C

FIFO queue

Order kept within a message group: MessageGroupId="booking-9001"
The next message in a group is held until the one before it is deleted
Different groups proceed in parallel

Deduplication: a second send with the same deduplication ID within 5 minutes is dropped

The ID is fixed once per event, so a producer’s retry repeats it
Throughput is limited per group, so groups are chosen as fine as the order requires

Consumers Poll the Queue for Messages

Short poll: a receive returns at once, empty if nothing is visible, and every request is billed

Long poll: a receive waits up to 20 s for a message

r = sqs.receive_message(QueueUrl=q,
    MaxNumberOfMessages=10,
    WaitTimeSeconds=20)

Returns as soon as a message arrives
Up to 10 messages per receive

The worker loop: receive, process, delete, repeat

Runs on an instance or a container, whether or not messages arrive
Polling, scaling the number of workers, and restarting a crashed worker are the application’s

Functions: code that runs only when a message arrives, with the platform doing the polling

Delivering Events

Queues Do Not Solve Event Notification

One fact, several services: “passenger 42 checked in” matters to the gate list, the notification service, and baggage

One queue gives each message to one consumer
Three services on one queue each see a third of the check-ins

A queue per service

Check-in sends three messages, one to each queue
A fourth service means a change to check-in
Check-in, the producer, holds the list of everyone who cares

A Topic Sends Each Event to Every Subscriber

Publish-subscribe

Publisher: sends an event to a topic, and knows nothing about who receives it
Subscription: a link from the topic to one receiver
Every subscription receives every event published after it is created

Compared with a queue

Queue: each message to one of its consumers, for work done once
Topic: each event to all of its subscribers, for a fact several services act on
A new subscriber changes the topic’s subscriptions, not the publisher

Amazon SNS Delivers One Publish to Queues, Functions, and HTTPS Endpoints

Amazon Simple Notification Service (SNS): topics that applications publish to over HTTPS

AWS runs

Storage of each published event until it is delivered
Delivery to every subscription, retried when a receiver does not accept it
Capacity for any publish rate, with nothing to size

Billing: about $0.50 per million publishes, plus deliveries to HTTPS, email, and SMS

The application sets

A topic, passenger-events
Subscriptions: an SQS queue, a Lambda function, an HTTPS endpoint, email or SMS
Which services may publish and subscribe, by IAM policy

sns.publish(TopicArn=topic,
    Message=json.dumps(
        {"type": "passenger.checked_in",
         "passenger": 42, "flight": 7}))

A Queue per Subscriber Keeps Its Events Until That Service Is Ready

Topic to queues: each subscriber’s subscription is its own SQS queue

The topic copies each event into every queue
Each service receives from its queue at its own rate
A service that is down finds its events waiting when it returns

What each queue adds

The queue mechanics per subscriber: visibility timeout, redelivery, dead-letter queue
A slow service builds a backlog in its own queue, not anyone else’s
Delivery is at least once and unordered, as with a standard queue, so each service de-duplicates

Subscribers Can Select the Events They Receive With Filters

Filter policy: a condition on the event body, set on one subscription with FilterPolicyScope: MessageBody

{"type": ["passenger.checked_in"],
 "fare_class": ["F", "J"]}

The lounge-invitation service receives first and business check-ins
Other subscribers still receive every event
The publisher is unchanged: it publishes everything once

Without a filter

Every subscriber receives every event and discards the ones it does not use
Each discarded event is still a delivery: a receive, a function run, a charge

Routing by rules: EventBridge applies filters like these across events from many sources, AWS services included

Subscribers Only Receive Events Published After They Join

A topic keeps nothing once delivered

A fraud check added today starts with the next event
A subscriber that processed events wrongly cannot read them again after the fix

When a delivery to a subscriber fails

Retried for a limited time: up to 23 days for queues and functions, a few attempts for HTTPS by default
Then dropped, or sent to a dead-letter queue

Uses that need past events

A new service built from past events
Reprocessing after a bug
A report over the last week, read in order
FIFO topics can archive events for replay, standard topics cannot

Logs Retain Ordered Events for Replay

Log: events appended in order and kept for a set period

Reading an event does not remove it
Each reader keeps its own position and moves at its own rate
A reader added later starts at any position still kept

Compared with a queue and a topic

A queue removes a message once one consumer deletes it
A topic hands copies to current subscribers and keeps none
A log keeps every event for the retention period, for any reader

Kinesis Replays Events From Any Starting Time

Amazon Kinesis Data Streams: a managed log, split into shards

Partition key: hashed to pick the shard, Kinesis’s name for a partition
Each flight’s events land in one shard, in the order written, with no order between shards
Kept 24 hours by default, up to 365 days

Where a reader starts, per shard

kinesis.get_shard_iterator(
    StreamName="passenger-events",
    ShardId=shard,
    ShardIteratorType="AT_TIMESTAMP",
    Timestamp=datetime(2026, 10, 27, 8, 0))

Or at the oldest kept event, a recorded position, or new events only

Functions

Instances Run Whether or Not Work Arrives

A server sized for the peak

An API server, a queue worker, a scheduled job: each runs all day on an instance or a container
Billed for every hour it runs, busy or idle

Also the application’s

Adding capacity before a wave and removing it after
Replacing a server that crashes
Patching the operating system it runs on

A Function Runs Code Only When an Event Arrives

Function service: runs a handler once per event, on environments it starts and stops

No process waits for work between events
More events at once: more environments, each running one event
No events: nothing running, nothing billed

One way to run the same patterns

Queues, topics, and storage events work the same with any consumer
Instances or containers that scale with a queue’s depth do the same work, with the scaling and idle hours the application’s
AWS Lambda is AWS’s function service, and other providers offer their own

AWS Lambda Needs Only a Handler and Its Settings

AWS Lambda: the function service that receives events and runs handlers for them

AWS runs

The machines and the execution environments
Starting, reusing, and stopping environments as events arrive
Polling queues and streams on the function’s behalf

Billing: $0.20 per million invocations, plus run time times memory, about $0.0000167 per GB-second

The application sets

The handler code and its runtime, such as Python 3.12
Memory, from 128 MB to 10 GB, and a timeout of up to 15 minutes
The event sources, and the execution role the code’s AWS calls use

def handler(event, context):
    for record in event["Records"]:
        check_passport(json.loads(record["body"]))

Event Sources Define Their Own Event Formats

From S3: one record per object changed

{
  "Records": [{
    "eventName": "ObjectCreated:Put",
    "s3": {
      "bucket": {"name": "airline-scans"},
      "object": {
        "key": "scans/42/passport.jpg",
        "size": 812344
      }
    }
  }]
}

From SQS: a batch of messages

{
  "Records": [
    {"messageId": "5f1c...", "body": "{...}"},
    {"messageId": "9a2d...", "body": "{...}"}
  ]
}

From an HTTP request: through a gateway or function URL

{
  "rawPath": "/checkins",
  "requestContext": {
    "http": {"method": "POST"}
  },
  "body": "{\"passenger\": 42, \"flight\": 7}"
}

The handler’s two arguments

event: the source’s document, as a Python dict
context: the request ID and the time left before the timeout
A function reads the fields its sources send, so it is written for those sources
print output and errors go to CloudWatch Logs, tagged with the request ID

A Function Is Code, Settings, and Attached Permissions

Package: the handler and its libraries

A zip of up to 250 MB unpacked, shared libraries in layers
Or a container image of up to 10 GB
A larger package lengthens every cold start

Attached permissions

Resource policy: which services and accounts may invoke the function
Execution role: what the function’s code may call, through temporary credentials Lambda supplies
An S3 trigger that never runs the function is usually a missing resource policy, a function that fails on GetObject a missing role permission

Large Libraries and Models Need a Container Image or Separate Storage

The 250 MB limit: zip and layers together, unpacked

NumPy, pandas, and scikit-learn fill most of it
PyTorch alone exceeds it
Layers share libraries between functions without raising the limit

Beyond it

A container image, up to 10 GB
A model in S3, loaded into /tmp outside the handler
A model on EFS, mounted into a VPC-attached function
Each adds bytes to load in every cold start

New Execution Environments Add a Cold-Start Delay

Cold start: the first event in a new environment waits for it to be set up

Create the environment and load the code
Start the runtime
Run the code outside the handler

When it happens

The first events after a deploy or a quiet period
Every environment added when concurrency rises
From about 100 ms for a small function to seconds with large libraries

Cold Starts Can Be Shortened or Avoided, at a Cost

Shorter

A smaller package and fewer imports
Less work outside the handler: a client created, not a model downloaded, when the model can live elsewhere

Avoided, for a price

SnapStart: an initialized environment snapshotted at deploy, then restored (Java, Python, .NET)
Provisioned concurrency: environments kept initialized, billed whether or not events arrive
Both apply to a published version

Code Outside the Handler Runs Once per Environment

import boto3, json

# once per environment
s3 = boto3.client("s3")
rules = load_rules("visa-rules.json")

# once per event
def handler(event, context):
    for record in event["Records"]:
        msg = json.loads(record["body"])
        scan = s3.get_object(Bucket=msg["bucket"],
                             Key=msg["key"])
        check(scan, rules)

The client and the rules are created once, then reused by every event the environment runs

Kept between events in one environment

Module variables, open connections, files in /tmp (512 MB to 10 GB)

Not kept, not shared

Another environment has its own copy of everything
An environment is discarded when idle, at a time Lambda chooses
Anything a later event or another environment needs goes to storage, a queue, or a database

Deploys Switch Events to New Code in New Environments

Configuration outside the code

Environment variables: the table name, the queue URL, the stage
Secrets from a secrets store, read at startup with the execution role, not written into the package

A deploy

New code uploaded: events after the deploy start new environments with it
Warm environments on the old code stop receiving events and are discarded

Versions and aliases

Publishing a version fixes the code and settings under a number: 7, 8
An alias such as live names one version, and the triggers point at the alias
Moving live from 8 back to 7 rolls back without a new deploy
An alias can split events between two versions, such as 10% to the new one

Lambda Allocates CPU in Proportion to Memory

One setting: memory, from 128 MB to 10,240 MB

CPU share follows it: one vCPU at 1,769 MB, six at the top
A CPU-bound handler at 512 MB runs about 3.5 times longer than at 1,769 MB

Cost per run: memory times run time

Up to one vCPU, a faster run at a higher setting costs about the same
Past it, a single-threaded handler runs no faster and costs more

Lambda Stops a Handler Mid-Work When Its Timeout Expires

At the timeout

The handler is stopped where it is: no finally, no cleanup
Writes made before that point stay, and the rest never happen
The run counts as a failure, retried or not as its invocation type decides

Stopping cleanly instead

for step in steps:
    left = context.get_remaining_time_in_millis()
    if left < 5000:
        raise Retryable("out of time")
    step()

Each step idempotent, so a retry repeats nothing harmful

Synchronous Invocations Return the Handler’s Result to the Caller

Synchronous: the caller waits for the run to finish

API Gateway, a function URL, or a direct Invoke call
The return value is the reply: status, headers, body
A cold start and the whole run are in the caller’s wait

Failure is the caller’s

An error or a timeout returns to the caller, and Lambda does not retry
A gateway in front has its own limit: 30 s for an API Gateway HTTP API
Retrying is the client’s decision, safe for idempotent requests

Asynchronous Invocations Wait in a Queue Inside Lambda

Asynchronous: the source hands over the event and continues

S3 notifications, SNS, EventBridge, EventBridge Scheduler at a set time, or Invoke with InvocationType="Event"
Lambda queues the event and returns at once
The handler’s return value goes nowhere

Lambda’s retries

Two retries after a failure, about one and then two minutes apart
An event older than the maximum age, 6 hours by default, is dropped
Then the failure destination: an SQS queue, SNS topic, or another function
An event can also run twice with no failure, so the handler de-duplicates

Lambda Polls SQS and Deletes the Messages a Handler Finishes

Event source mapping: Lambda’s poller for a queue

Long-polls the queue and calls the handler with a batch of up to 10,000 messages, gathered for up to 5 minutes
Adds pollers and environments as the queue’s backlog grows
Handler returns: the batch’s messages are deleted
Handler raises: the whole batch returns to the queue after its visibility timeout

Partial batch failure: report the messages that failed

def handler(event, context):
    failed = []
    for r in event["Records"]:
        try:
            check_passport(json.loads(r["body"]))
        except Exception:
            failed.append(
                {"itemIdentifier": r["messageId"]})
    return {"batchItemFailures": failed}

With ReportBatchItemFailures set on the mapping, the rest are deleted, and the failed ones return and count toward the dead-letter limit
The queue’s visibility timeout is set to at least six times the function’s timeout

A Failing Record Holds Up Its Kinesis Shard

Polled from a stream: one batch per shard at a time, in order

A batch that fails is retried as it is, so the shard does not move
Every record behind it in that shard waits: the cost of keeping order
Other shards continue

Limits on the wait

The record expires with the stream’s retention
A maximum retry count, then the record goes to a failure destination
Splitting a failing batch in half until the bad record is alone

Concurrency Equals Event Rate × Run Time

Concurrency: environments running at once

300 events per second at 0.2 s each: 60 environments
Each function adds up to 1,000 environments every 10 seconds
1,000 per account per Region by default, shared by every function

Over a limit: throttling

Synchronous callers receive an error at once
Asynchronous events wait in Lambda’s queue and are retried
Polled sources slow down: messages stay in the queue
Reserved concurrency caps one function and guarantees it that share

Functions Run in an Isolated VPC Unless Attached to Another

Default: an isolated VPC that Lambda owns, not connected to the account’s VPCs

Public endpoints only: S3, SQS, SNS, DynamoDB

Attached: an interface in a private subnet

Database: direct, as an instance there reaches it
Other services: a VPC endpoint or the NAT gateway, else calls wait until the timeout

Execution Environments Maintain Their Own Database Connections

Connections follow environments

A connection opened outside the handler is reused by one environment, not shared
300 environments in a check-in wave: 300 connections, against a limit near 100 on a small database instance
Over the limit, new connections are refused and handlers fail

Bounds

Reserved concurrency below the database’s limit
A connection pooler between the functions and the database: RDS Proxy holds a few connections and lends them per transaction
A database reached over HTTP with no connection to hold: DynamoDB

Functions Cost Less Than Instances Below a Break-Even Request Rate

Cost

Functions: paid per request and per GB-second, nothing when idle
Instances: paid per hour, used or not
Below break-even the function costs less, above it the instances do

Limits that decide it before cost

Runs over 15 minutes, GPUs, long-lived connections: instances or containers
Latency that cannot include a cold start
Requests from outside reach a function through a gateway or a function URL

API Gateways and Edge Caches

Browsers Cannot Invoke Functions Directly

Invoking a function: an AWS API call, signed with credentials the function’s resource policy allows

A browser speaks HTTPS and holds no AWS credentials, and must not
Something AWS runs receives the HTTPS request and invokes the function for it

Three such endpoints

Function URL: an HTTPS address for one function, nothing else
Load balancer: spreads requests over targets, which can be functions or instances
API gateway: routes many paths to many backends, and checks callers before any code runs

Gateways Turn HTTP Requests Into Calls to Backend Services

Gateway: a server in front of the backends that receives every request first

Holds the public address and the TLS certificate
Checks identity and rate, then matches a route

Backends

Receive only requests that passed the checks
Are functions, containers, or other HTTP services, and can change behind one address
Refused requests cost no backend run

Amazon API Gateway Runs Checks and Routing at a Public HTTPS Endpoint

AWS runs

The endpoint, its TLS, and capacity for any request rate
Authorization checks, throttling, and routing
Invoking the function for each request that passes

Billing: about $1.00 per million requests for HTTP APIs

The application sets

Routes: a method and a path, each with its backend
An authorizer, and throttling limits
A custom domain, such as api.airline.example

Simpler: a function URL gives one function its own HTTPS address, with no routes and no authorizer beyond IAM

Routes Map Each Method and Path to a Backend

Routes

POST /checkins                check-in function
GET  /airports/{code}/flights  board function
GET  /passengers/{id}/trips    trips function

{code} captures one path segment and passes it in the event
No match: 404 from the gateway, no function runs

The event the function receives

{
  "routeKey": "GET /airports/{code}/flights",
  "pathParameters": {"code": "LAX"},
  "queryStringParameters": {"limit": "20"}
}

The reply the function returns

def handler(event, context):
    airport = event["pathParameters"]["code"]
    board = read_board(airport)
    return {
        "statusCode": 200,
        "headers": {"Cache-Control": "max-age=30"},
        "body": json.dumps(board)}

The gateway turns the returned dict into the HTTP reply: status, headers, body
One function per route, or one function for several routes that reads routeKey

Authorizers Reject Requests Before Dispatching to a Function

JWT authorizer: the gateway verifies a signed token on every request

Signature, issuer, audience, and expiry, against the identity provider’s published keys
Failure: 401, before any function runs

What the function receives

The token’s claims in the event: requestContext.authorizer.jwt.claims
Who is calling, already checked
Whether that caller may check in this booking stays the function’s decision

Gateways Can Throttle Requests Before They Reach Functions

Throttling: a rate limit per route, per client, and per account

Over the limit: 429 Too Many Requests from the gateway
10,000 requests per second per account and Region by default, before any route limit

What it protects

Function concurrency: 500 requests per second at 0.2 s is at most 100 environments
The database behind the function, which holds a fixed number of connections
The bill: a refused request runs no function

Long Requests Can Return 202 Quickly and Finish Asynchronously

The gateway’s limit: 30 s at most for the whole request

A passport check that waits on an outside document service can take longer
A cold start, then the check, all inside the passenger’s wait

Accepted, then continued

The check-in function queues the check and replies 202 with a job address
The passport check runs from the queue, with its retries and dead-letter queue
The page reads /jobs/318, a short request, until the result is there

Minimum Latency Grows With Distance to the Region

Round-trip floor: distance over the speed of light in fiber, there and back

Tokyo to Virginia: about 110 ms, before any server work
Measured round trips are longer: routes are not straight

A page is many requests

The check-in page: HTML, scripts, styles, images, the departures data
Each new connection adds round trips for TCP and TLS before the first byte
Only a response from nearby lowers the floor

Edge Caches Deliver Copies Close to Clients

Content delivery network: caches in many cities, in front of one origin

A request goes to the nearest edge location
Hit: the stored copy is returned from nearby
Miss: the edge fetches from the origin, returns it, and stores it

Who benefits

Every client after the first at each edge, for the copy’s lifetime
The origin: one fetch per edge per lifetime instead of one per client
Content many clients request alike: scripts, images, a departures board

Amazon CloudFront Caches Content at Edge Locations in Over 100 Cities

AWS runs

The edge locations, their storage, and TLS at each
Routing each client to a nearby edge
Capacity for any number of clients

Billing: per GB sent to clients and per request, cheaper per GB than sending from the origin directly

The application sets

A distribution, with a name such as www.airline.example
Origins: the S3 bucket for the page, API Gateway for /api/*
Behaviors per path: which origin, what is cached, for how long
The certificate for its domain

S3 Buckets Can Serve Static Sites Through CloudFront

Static site: files the browser runs as they are, with no server code per request

HTML, scripts, styles, images, stored as S3 objects
Served through CloudFront, with HTTPS on the site’s own domain

Private bucket

The bucket policy allows reads by this distribution only, through origin access control
Clients cannot bypass the cache or read files that were never meant to be listed
/api/* on the same domain goes to API Gateway

Longer Cache Lifetimes Raise Hit Rates and Delay Updates

Lifetime: set by the origin’s Cache-Control: max-age, or by the distribution’s default

One miss per lifetime per edge, every other request a hit
Departures board at 30 s: a client sees a gate change up to 30 s late

The trade

Longer: higher hit rate, less load on the origin, older copies
Shorter: fresher copies, more trips to the origin
Invalidating a path removes copies early, at every edge, and takes minutes

File Names With a Content Hash Prevent Stale Copies

Content in the name: the build names each file by a hash of its bytes

A changed script gets a new name, so a cached copy under an old name is never out of date
Cache-Control: max-age=31536000 on hashed files, max-age=60 on index.html

After a deploy

index.html is refreshed within a minute and names the new script
Old copies stay cached, named by no page
The one short lifetime is on the one file whose name cannot change

Coordinating Parallel Work

One Process Coordinates Multi-Step Work Without Extra Code

The document check in one program

def document_check(booking):
    with db.transaction():
        passport = check_passport(booking)
        watch = check_watch_list(booking)
        visa = check_visa(booking)
        if passport and watch and visa:
            issue_boarding_pass(booking)
            notify(booking, "checked in")
        else:
            refer_to_agent(booking)

Provided by the language and the database

Order: the lines say which step waits for which
State: local variables hold where the check stands
Completion: the function returns
Failure: an exception stops the remaining steps, and the transaction undoes what was written

None of it written by the application: it comes with running every step in one process against one database

A Workflow’s Dependencies Decide Which Steps Run at Once

Order as dependencies: which step needs which other step’s result

The three checks need nothing from each other and start together
The total time is the longest path, here through the passport check

Join: a step that waits for several others

Starting the three checks is one event to a topic, or three messages
Nothing in a queue or a topic knows that all three have finished
The join is where the application’s own record of the workflow is needed first

A Job Record Keeps the Workflow’s State Outside Every Service

One record per document check, created when the check starts

check 5120
  status    waiting
  passport  pass
  watchlist (missing)
  visa      pass
  deadline  14:20:00

Each step writes its own field and nothing else
Read by every step, and by the check-in page asking for the outcome

A check finishing

r = table.update_item(
    Key={"check": "5120"},
    UpdateExpression="SET visa = :r",
    ExpressionAttributeValues={":r": "pass"},
    ReturnValues="ALL_NEW")
rec = r["Attributes"]
done = all(k in rec for k in
           ("passport", "watchlist", "visa"))

The write returns the record as it now stands
The check that sees all three present runs the decision

Steps Finishing Together Can Both Run the Join

Write, then look: two separate requests for each check

Both writes land before either check reads the record
Both see three results, and both act

The same shape as a double booking

Two writers each read a state that allows them to act
The fix is the same: make the decision part of one atomic write

A Conditional Write Lets One Step Claim the Join

Claiming the finish: change waiting to deciding, only if all three results are present and no one has claimed it

try:
    table.update_item(
        Key={"check": "5120"},
        UpdateExpression="SET #s = :deciding",
        ConditionExpression=(
            "#s = :waiting AND "
            "attribute_exists(passport) AND "
            "attribute_exists(watchlist) AND "
            "attribute_exists(visa)"),
        ExpressionAttributeNames={"#s": "status"},
        ExpressionAttributeValues={
            ":waiting": "waiting",
            ":deciding": "deciding"})
    act_on(read_results("5120"))
except ConditionalCheckFailedException:
    pass  # claimed already, or a result missing

One write, one winner

The store applies the condition and the change together, for one record
The first check to claim changes the status, and the second’s condition fails
A check whose condition fails does nothing more

Every check tries

No check needs to know whether it was last
The one whose write meets the condition acts

Recording Which Steps Finished Makes Redelivery Harmless

A counter: each step adds one

A redelivered message adds one again
The count reaches three with a check missing

A field per step: each step sets its own

Setting the same field twice leaves the record as it was
Complete means every named field present, whatever was delivered twice

A Deadline in the Record Ends Workflows Whose Steps Never Finish

A step that never reports

The outside document service is down for hours
The step’s messages end in its dead-letter queue
The record waits forever unless something checks it

A deadline, written when the check starts

A delayed message, or a scheduled task, arrives at the deadline
A conditional write: waiting to timed_out, then refer to an agent
A late result finds the status changed and does nothing

Compensating Actions Undo Steps That Already Took Effect

A step that acts before the check finishes

Check-in issues the boarding pass while the documents are checked, so the passenger is not kept waiting
The visa check then fails
No transaction covers check-in and the visa service: the boarding pass already exists

Compensating action: a later step that undoes an earlier one’s effect

Revoke the boarding pass, return the booking to confirmed, notify the passenger
Written for each step that can need undoing, as the application’s code
Idempotent like any other step: a revocation delivered twice revokes once

Saga: the standard name for a sequence of steps, each with its compensating action

Choreography and Orchestration Differ in Where the Workflow Is Written

Choreography: services react to events, and the job record holds the state

No component holds the whole order
A new step is another subscriber
A stuck check is found by reading the record and each service’s logs

Orchestration: one workflow calls each step

The order, the waits, the retries, and the timeouts are written in one definition
Each service does its step and returns
The workflow is one more component that every run depends on

AWS Step Functions Holds a Whole Workflow in One Definition

The document check as a state machine (abridged)

{"StartAt": "Checks",
 "States": {
  "Checks": {
   "Type": "Parallel",
   "Branches": [
    {"StartAt": "Passport",
     "States": {"Passport": {
       "Type": "Task",
       "Resource": "...passport-fn",
       "TimeoutSeconds": 600,
       "Retry": [{"ErrorEquals":
         ["States.ALL"],
         "MaxAttempts": 3}],
       "End": true}}},
    "... watch list, visa ..."],
   "Catch": [{"ErrorEquals":
     ["States.ALL"], "Next": "Refer"}],
   "Next": "Decide"},
  "Decide": {"Type": "Task", "End": true},
  "Refer":  {"Type": "Task", "End": true}}}

The same four, in one definition

Order: Parallel for the three checks, Next for what follows
State: kept by Step Functions for each run, for up to a year
Completion: the parallel state ends when all three branches end, so the join is built in
Failure: Retry and TimeoutSeconds per step, Catch for the path that compensates or refers

What Step Functions runs: each run’s steps, and a history of every step, billed per state transition

Each Workflow Chooses Where Its State and Completion Live

In a job record, with choreography

Steps react to events and write their own fields
A conditional write claims the join, or closes the record at its deadline
Fits a few steps, among services the application already runs

In an orchestrator

The workflow definition holds order, state, completion, and failure paths
Each run’s history is kept in one place
Fits longer workflows with many steps, waits, and branches

Either way

Steps can run more than once, so each step is idempotent
A step that took effect is undone by a compensating action, not by a transaction
What one process provided without code, the application now places on purpose

Cloud Patterns: Storage, Serverless, and Coordination

Outline

Holding Data and Work

Running and Connecting Work

Managed Services

Tightly Coupled Services Fail Together

Intermediaries Let Senders Continue Without Waiting for Receivers

Managed Services Move Infrastructure Operation to the Provider

Events Record That Something Occurred

Queues Reach One Receiver, Topics Reach Every Subscriber

Managed Services Produce, Carry, and Process Events

Services Show Old State Until They Process the Event

Failures Partway Through Leave Work Half Done

One Request Spreads Across Several Managed Services

Shared File Systems

File Systems Turn File Names Into Disk Blocks

Block Volumes Give an Instance a Disk, Not a File System

Shared Block Volumes Corrupt Ordinary File Systems

Network File Systems Serve Files From a Remote Server

Amazon EFS Serves One File System Through a Mount Target in Each Zone

EFS Storage Grows on Demand

Small-File Workloads Run Slowly on NFS

NFS Changes Become Visible When the Writer Closes the File

Block Volumes and Network File Systems Mount Side by Side

Object Storage

Object Storage Replaces File Operations With Whole-Object Requests

Conditional Writes Stop Two Writers From Replacing Each Other’s Object

Designing Object Keys Around the Most Frequent Listing

Other Listings Read Every Key Unless Keys Are Copied or Indexed

Multipart Uploads Split Large Objects Into Independent Parts

Presigned URLs Grant Time-Limited Access to One Object

S3 Notifies Other Services When Objects Change

Queues

Slow Work Holds Up the Reply

Job Tables Keep Work Past the Instance That Accepted It

Message Queues Pass Work From Producers to Consumers in Arrival Order

Amazon SQS Stores Messages Across Zones With No Capacity to Size

SQS Keeps Each Message Until a Consumer Deletes It

Unfinished Messages Return to the Queue After a Timeout

De-duplicating Messages Keeps Redelivery From Repeating Work

Failed Messages Move to a Dead-Letter Queue

Slow Consumers Build a Backlog

FIFO Queues Keep Order Within a Message Group

Consumers Poll the Queue for Messages

Delivering Events

Queues Do Not Solve Event Notification

A Topic Sends Each Event to Every Subscriber

Amazon SNS Delivers One Publish to Queues, Functions, and HTTPS Endpoints

A Queue per Subscriber Keeps Its Events Until That Service Is Ready

Subscribers Can Select the Events They Receive With Filters

Subscribers Only Receive Events Published After They Join

Logs Retain Ordered Events for Replay

Kinesis Replays Events From Any Starting Time

Functions

Instances Run Whether or Not Work Arrives

A Function Runs Code Only When an Event Arrives

AWS Lambda Needs Only a Handler and Its Settings

Event Sources Define Their Own Event Formats

A Function Is Code, Settings, and Attached Permissions

Large Libraries and Models Need a Container Image or Separate Storage

New Execution Environments Add a Cold-Start Delay

Cold Starts Can Be Shortened or Avoided, at a Cost

Code Outside the Handler Runs Once per Environment

Deploys Switch Events to New Code in New Environments

Lambda Allocates CPU in Proportion to Memory

Lambda Stops a Handler Mid-Work When Its Timeout Expires

Synchronous Invocations Return the Handler’s Result to the Caller

Asynchronous Invocations Wait in a Queue Inside Lambda

Lambda Polls SQS and Deletes the Messages a Handler Finishes

A Failing Record Holds Up Its Kinesis Shard

Concurrency Equals Event Rate × Run Time

Functions Run in an Isolated VPC Unless Attached to Another

Execution Environments Maintain Their Own Database Connections

Functions Cost Less Than Instances Below a Break-Even Request Rate

API Gateways and Edge Caches

Browsers Cannot Invoke Functions Directly

Gateways Turn HTTP Requests Into Calls to Backend Services

Amazon API Gateway Runs Checks and Routing at a Public HTTPS Endpoint

Routes Map Each Method and Path to a Backend

Authorizers Reject Requests Before Dispatching to a Function