How do I troubleshoot the “Create-failed” or “Permission denied” error that occurs when I try to restore my ElastiCache cluster from S3?

Short description

The following are common reasons that a restore for an ElastiCache backup from Amazon S3 fails:

• You attempted to restore a backup outside the backup constraints.
• ElastiCache couldn't retrieve the file from Amazon S3.
• The ElastiCache backup file is located in an Amazon S3 bucket in another AWS Region.
• You restore an rdb file that contains multiple databases to an ElastiCache cluster that has cluster mode turned on.

Resolution

You attempt to restore a backup outside the backup constraints

When you restore an ElastiCache for Redis backup, make sure that you understand the backup constraints:

• You can't restore from a backup that used a Redis cluster with cluster mode turned on to a Redis cluster with cluster mode turned off.
• When you restore a backup from an ElastiCache cluster with cluster mode turned on, you can't turn off cluster mode in the ElastiCache console. The option to turn on cluster mode is only available.
• When you export a backup with an ElastiCache cluster that has cluster mode turned on to Amazon S3, multiple rdb files are created. If you seed the backup from Amazon S3 with one backup rdb file, then only a single shard's keys are seeded to the cluster. If you try to work around this with a wildcard *rdb file, then the following error appears.
  "Error: Object or bucket does not exist for S3 object: examplebucket/cluster-mode-enabled-*.rdb."
• You can't restore a backup from a cluster that uses data tiering. For example, you can't restore a r6gd node into a cluster that doesn't use data tiering.
• You can't export the data tiering backup to Amazon S3. Also, you can't restore an ElastiCache data tiering backup from Amazon S3. For more information, see Limitations.
• For rdb files with multiple databases, you can't restore from a Redis (cluster mode turned off) cluster to a Redis (cluster mode turned on) cluster. When you try to do this, the following error appears:
  "Error: To restore a snapshot in cluster mode, all keys in the RDB file should reside in DB 0."

ElastiCache couldn't retrieve the file from Amazon S3

If ElastiCache doesn't have the necessary permissions to access the ElastiCache backup that's stored in the Amazon S3 bucket, then the following error appears:

"Restore from snapshot failed for node group 0001 in replication group test. Failed to retrieve file from S3."

To confirm the permissions issue, review your ElastiCache events. When you identify the cause of the error, check that your Region is one of the following:

• An opt-in Region
• China (Beijing) and China (Ningxia)
• AWS GovCloud (US-West)
• A default Region

For an opt-in Region, you must have a bucket policy that allows ElastiCache to retrieve the backup file from Amazon S3.
If your Amazon S3 bucket is located in one of the following Regions, you must allow the ElastiCache service access to the backup file in S3:

• China (Beijing) and China (Ningxia)
• AWS GovCloud (US-West)
• A default Region

Note: The canonical ID for the China (Beijing), China (Ningxia), and AWS GovCloud (US-West) Regions are different from the default AWS Regions:

• China (Beijing) and China (Ningxia) - b14d6a125bdf69854ed8ef2e71d8a20b7c490f252229b806e514966e490b8d83
• AWS GovCloud (US-West) - 40fa568277ad703bd160f66ae4f83fc9dfdfd06c2f1b5060ca22442ac3ef8be6
• AWS default Regions - 540804c33a284a299d2547575ce1010f2312ef3da9b3a053c8bc45bf233e4353

When you use a canonical ID or bucket policy to grant ElastiCache access to the rdb backup object, restore the Redis cluster.

The ElastiCache backup file is located in an Amazon S3 bucket in another Region

If you try to restore an ElastiCache backup that's located in an Amazon S3 bucket in another Region, then the following error appears:

"Permission denied to access S3 object. Please use the S3 object in the same region."

To resolve this error, complete the following steps:

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshoot AWS CLI errors. Also, make sure that you're using the most recent AWS CLI version.

1. Use the following command to copy the rdb backup file from the Amazon S3 bucket. Make sure that the bucket contains the backup in the Region where the Redis cluster is being restored. For more information, see What are best practices for migrating my ElastiCache for Redis cluster?

   Note: Replace example-source-bucket, example-destination-bucket, example-backup, example-source-region, and example destination-region with the appropriate name values.

   aws s3 cp s3://example-source-bucket/example-backup.rdb s3://example-destination-bucket/example-backup.rdb --acl bucket-owner-full-control --source-region example-source-region --region example-destination-region

2. Confirm that the rdb backup file has the correct permissions assigned to it (canonical ID or bucket policy).

3. Continue to restore the cluster.

You restore an rdb file that contains multiple databases to an ElastiCache cluster that has cluster mode turned on

If you restore a rdb file that contains multiple databases to an ElastiCache cluster with cluster mode turned on, then the following error appears:

"Restore from snapshot failed for node group 0001 in replication group test. To restore a snapshot in cluster mode, all keys in the RDB file should reside in DB 0. Snapshot ID: arn:aws:s3:::example-bucket/multidb.rdb"

To check the cause of the restore failure, review your ElastiCache events.

To resolve this error, complete the following steps:

1. Make sure that all the keys are migrated to a single database. For more information, see Migrate on the Redis website.
   Note: If the source database is located on ElastiCache Redis, then the migrate command isn't supported. To work around this issue, use other Redis tools such as RIOT. For more information, see riot on the Redis website.
2. Create a local backup of your Redis database.
3. Upload the backup to Amazon S3.
4. Continue to restore to an ElastiCache (cluster mode turned on) cluster.