- Newest
- Most votes
- Most comments
AWS does not officially document or guarantee support for Celery with ElastiCache Serverless for Redis (cluster‑mode enabled). While ElastiCache Serverless is API‑compatible with Redis OSS, its restrictions (disabled commands, cluster‑mode sharding, CROSSSLOT errors) mean the Celery Redis broker implementation is not fully compatible out of the box. The official AWS documentation only confirms Redis OSS API compatibility, but does not list Celery as a supported integration.
https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/WhatIs.deployment.html
ElastiCache Serverless for Redis runs in cluster‑mode and enforces the same command and key‑slot constraints as standard cluster configurations. Celery’s Redis broker implementation is not cluster‑aware—it assumes a single‑node topology and performs multi‑key list and pub/sub operations that can trigger CROSSSLOT errors or silently hang when keys land on different hash slots. That explains why your workers connect and write queue data but never consume tasks. AWS documentation for ElastiCache Serverless notes that commands like CLIENT LIST, MONITOR, and some blocking operations are restricted, which further breaks Celery’s polling loop.
—Taz
answered 9 months ago
Relevant content
asked 2 years ago
