• October 21, 2026
• 14 min read

Amazon SQS — Core Concepts¶

Standard vs FIFO¶

Choose by the failure you cannot tolerate. Standard is the default for independent jobs. FIFO costs throughput and parallelism: while one message of a group is in flight, no other message of that group is delivered, so one hot MessageGroupId is a single-threaded lane. Use a fine-grained group key (customer ID, order ID), not a constant.

FIFO details¶

• MessageGroupId decides ordering and parallelism: consumers can process different groups concurrently, never two messages of one group at once.
• Deduplication: if MessageDeduplicationId was already accepted in the last 5 minutes, the send succeeds but the message is not delivered. SQS keeps tracking the ID even after the message was received and deleted.
• Content-based deduplication (ContentBasedDeduplication=true) derives the ID from a SHA-256 hash of the body only — message attributes are not included. Send an explicit MessageDeduplicationId when bodies are identical but must stay distinct, or when the same logical event may have different bodies.
• Within one SendMessageBatch, messages are enqueued in the order they appear in the request.
• ReceiveRequestAttemptId lets you retry a ReceiveMessage whose response was lost, and get the same messages back, for 5 minutes, as long as none of them was deleted or had its visibility changed.
• A dead-letter queue breaks strict order for a group: do not use one when order is more important than progress. Without a DLQ, a poison message blocks its group until it is deleted or expires.

High-throughput FIFO¶

Enable it with two queue attributes (both required; any other combination gives normal throughput):

sqs.create_queue(QueueName="events.fifo", Attributes={
    "FifoQueue": "true",
    "DeduplicationScope": "messageGroup",          # deduplicate per group instead of per queue
    "FifoThroughputLimit": "perMessageGroupId",    # throughput quota applies per group, not per queue
})

FIFO limits are in API requests per second per action (SendMessage, ReceiveMessage, DeleteMessage), and batching multiplies the message rate by up to 10. Documented defaults in October 2026:

These move over time; read the current numbers on the AWS "Quotas related to messages" page and in Service Quotas before sizing. Throughput scales with the number of distinct message groups (a group ID is hashed to a partition), and each partition tops out at 300 requests/s or 3,000 batched messages/s.

Message Lifecycle¶

flowchart LR
  S(["SendMessage"]) -->|"DelaySeconds > 0"| D["Delayed<br/>(hidden)"]
  S --> V["Visible<br/>(stored)"]
  D -->|"delay elapsed"| V
  V -->|"ReceiveMessage"| F["In flight<br/>(hidden for the visibility timeout)"]
  F -->|"DeleteMessage"| X(["Deleted"])
  F -->|"timeout expires, or<br/>ChangeMessageVisibility = 0"| V
  F -->|"receive count exceeds<br/>maxReceiveCount"| DL["Dead-letter queue"]
  V -->|"retention period passes"| E(["Expired and removed"])

Three facts drive most consumer bugs:

1. Receiving does not remove the message. It only hides it. The message is gone only after DeleteMessage with a valid receipt handle.
2. Every receive returns a new receipt handle. The handle belongs to the receive, not to the message. Delete or extend with the most recently received handle; an older handle might not delete the message.
3. The message ID never changes, so it is a poor idempotency key if the producer can retry (a retry creates a new message with a new ID). Use a business key from the body or an attribute.

Visibility Timeout¶

Rules of thumb:

• Set the queue timeout to the longest normal processing time plus margin. Too short means duplicates; too long delays the retry after a crash.
• For jobs of unpredictable length, keep the queue default small and run a heartbeat that extends the timeout while the work runs (pattern in 03).
• Lambda: set the queue timeout to at least six times the function timeout (plus the batching window); the function timeout must not exceed the queue's timeout.
• A message that is not deleted before the timeout expires counts as a failed receive and moves you towards maxReceiveCount.

Short and Long Polling¶

• The maximum wait is 20 seconds. The call's WaitTimeSeconds wins over the queue default.
• With fewer than about 1,000 messages in the queue, a short poll often returns nothing even though messages exist; repeating the call eventually returns them. Tests written with short polling are flaky for this reason on real SQS.
• The HTTP client's read timeout must be longer than WaitTimeSeconds. botocore's default read timeout is 60 seconds, so the 20-second maximum is safe unless you lowered it.
• An empty response has no Messages key in boto3.

Delay Queues and Message Timers¶

• Delay queue: DelaySeconds on the queue hides every new message for 0–900 seconds (15 minutes).
• Message timer: DelaySeconds on SendMessage or on a batch entry overrides the queue value for that message. Not supported on FIFO queues.
• Changing a queue's delay is not retroactive for standard queues and is retroactive for FIFO queues.
• Delay differs from visibility timeout: delay hides a message when it arrives, visibility timeout hides it after it is received.
• Longer schedules than 15 minutes: use EventBridge Scheduler, which AWS recommends for advanced scheduling.

Retention¶

Dead-letter queue retention: on a standard queue, expiry always counts from the original enqueue time, and moving a message to a DLQ does not reset it. A message that spent 1 day in the source queue has 3 days left in a DLQ with 4-day retention. On a FIFO queue the timestamp resets when the message moves. Always give the DLQ a longer retention than the source (set it to 14 days).

Message Size, Content and Attributes¶

On receive, the response has MD5OfBody and MD5OfMessageAttributes; the SDK checks them, so a corrupt response raises instead of returning bad data. Attributes you want back must be requested: MessageAttributeNames=["All"] (or names, or a prefix.*) and MessageSystemAttributeNames for ApproximateReceiveCount, SentTimestamp, MessageGroupId, SequenceNumber, AWSTraceHeader, DeadLetterQueueSourceArn and others. (The older AttributeNames parameter is deprecated.)

Dead-Letter Queues and Redrive¶

A redrive policy on the source queue names the DLQ and maxReceiveCount:

import json
sqs.set_queue_attributes(QueueUrl=source_url, Attributes={
    "RedrivePolicy": json.dumps({"deadLetterTargetArn": dlq_arn, "maxReceiveCount": 5}),   # a JSON string, not a dict
})

Redrive from the DLQ¶

After fixing the bug, move messages back with the message-move APIs: StartMessageMoveTask, ListMessageMoveTasks, CancelMessageMoveTask.

Code and a test strategy for redrive are in 03. moto does not implement these three APIs.

Fair Queues¶

On a standard queue, set MessageGroupId to a tenant identifier when sending. SQS watches which groups have a disproportionate number of in-flight messages, calls them noisy, and returns messages of the other groups first. Consumers need no change, there is no throughput limit, and unlike FIFO the ID does not order or serialize anything. Use it when one queue serves many tenants, volume is high, and dwell time (time from arrival to processing) matters. Extra CloudWatch metrics: ApproximateNumberOfNoisyGroups and the ...InQuietGroups family. Billing: requests that involve a message with a group ID are charged at both the fair-queue and standard rates.

Encryption¶

Only the message body is encrypted; queue metadata, message IDs, timestamps and attributes are not. All requests to an encrypted queue must use HTTPS and Signature Version 4. A message is encrypted only if it was sent after encryption was enabled, and moving it to a DLQ does not change its encryption state. An encrypted queue using the default AWS managed key cannot invoke a Lambda function in another account.

Queue Policies and IAM¶

A queue can carry a resource-based queue policy (attribute Policy) in addition to identity-based IAM policies. It is how another account, or an AWS service such as SNS or EventBridge, gets permission to call sqs:SendMessage. Limits: 8,192 bytes, 20 statements, 50 principals, 10 conditions. Always scope service principals with aws:SourceArn or aws:SourceAccount to avoid the confused-deputy problem. A policy example for SNS is in 04.

Attribute-based access control works on queue tags (aws:ResourceTag/...). Give a test role access to queues by name prefix or tag rather than *. Foundations covers least-privilege IAM.

Quotas¶

Pricing Dimensions¶

Prices differ by Region and change; check the SQS pricing page. The things that drive the bill:

At-Least-Once Delivery in Practice¶

See also¶

• Amazon SQS — Overview
• Amazon SQS — boto3 in Python
• Amazon SQS — Consumer Patterns
• Amazon SNS — Topics, Subscriptions & Fan-out
• AWS Foundations — boto3, Credentials & Local Testing
• Queues vs Streams: Message Delivery, Ordering & Reliability