How does elastic resize work in Amazon Redshift?

6 minute read

I want to perform an elastic resize on my Amazon Redshift cluster. I want to know how elastic resize works and what best practices I can use.

Short description

An elastic resize doesn't sort tables or reclaim disk space. Run VACUUM to sort tables and reclaim disk space. Elastic resize is available only for Amazon Redshift clusters that use the EC2-VPC platform.

Elastic resize often requires less time to complete than a classic resize. During a classic resize operation, a new cluster is provisioned when Amazon Redshift is copying the data from your source cluster. The classic resize operation first distributes the data to the new nodes according to distribution style. Then, it runs the ANALYZE command to update table statistics.

To resize a single-node cluster, a classic resize might be the better option. An elastic resize allows you to add or remove nodes from the cluster. However, it can introduce performance variation. To have your node slices match the number of slices in your target node type, use a classic resize.

The following configurations support single-node elastic resize:

1 node dc2.large ↔ 1 node ra3.xlplus, 2 node ra3.xlplus, 2 node ra3.4xlarge
1 node ra3.xlplus ↔ 2 node ra3.xlplus, 2 node ra3.4xlarge

Resolution

How elastic resize works

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Elastic resize can work differently depending on the target node type. Check whether the target node type is the same as the source node type.

To check your node type, open the Amazon Redshift console. In the navigation pane, choose Clusters. The Clusters page shows the node type under each cluster name.

You can also run the describe-clusters AWS CLI command to get more information about your Amazon Redshift cluster:

aws redshift describe-clusters --region cluster_region

Note: Replace cluster_region with your cluster's AWS Region.

The target node type is the same as the existing node type

Amazon Redshift automatically redistributes data to new nodes when you use elastic resize to resize a cluster.

Elastic resize typically completes the operation within a few minutes. You can expect a small increase in your query runtime when elastic resize is completing data redistribution in the background.

Note: Your Amazon Redshift cluster is temporarily unavailable for a few minutes during the metadata migration.

The target node type is different from the existing node type

If your node type changed, then Amazon Redshift first creates a snapshot. A new target cluster is then provisioned with the latest data from the snapshot, and data is transferred to the new cluster in the background. During the data transfer, your Amazon Redshift cluster operates in read-only mode and all writes are blocked. When the resize is almost completed, Amazon Redshift automatically updates the new cluster's endpoint to match your existing cluster's endpoint. All connections to the original cluster are then closed.

DC2 and DS2 node count limitations

Note the following limitations for DC2 and DS2 node types:

For dc2.large or ds2.xlarge node types, use half or double the current number of nodes. For example, you can resize a cluster with 6 nodes to either 12 nodes or 3 nodes.
For dc2.8xlarge, ds2.8xlarge, or ra3.xlplus node types, use half or up to double the number of nodes. For example, you can resize a cluster with 6 nodes to 3, 4, 5, 7, 8, 9, 10, 11, 12 nodes.
For ra3.16xlarge or ra3.4xlarge node types, use quarter or up to four times the current number of nodes. For example, you can resize a ra3 cluster with 8 nodes to 2, 3, 4, 5, 6, 7, 9 up to 32 nodes.

Elastic resize best practices

When you're performing an elastic resize your cluster, use the following best practices:

Before you resize your cluster, activate automated snapshots or create a manual snapshot.
Note: By default, manual snapshots are retained indefinitely, even after you delete your cluster. However, Amazon Redshift deletes automated snapshots at the end of the retention period.

Run the describe-node-configuration-options AWS CLI command to get node configuration options for a resize operation:

aws redshift describe-node-configuration-options --action-type resize-cluster --cluster-identifier cluster_name --region cluster_region

Note: Replace cluster_name with your cluster's name and cluster_region with your cluster's Region.

Run VACUUM on the cluster before you resize it. Elastic resize doesn't automatically delete rows that are marked for deletion.

Run the resize-cluster command to specify all node configuration changes:

aws redshift resize-cluster --cluster-identifier  cluster_name --cluster-type multi-node --node-type target_node_type --number-of-nodes number_of_target_nodes --no-classic --region cluster_region

Note: Replace cluster_name with your cluster's name, target_node_type with your target node type, number_of_target_nodes with your number of target nodes, and cluster_region with your cluster's Region.

Additional actions to take

When you perform an elastic resize on your Amazon Redshift cluster, take the following actions:

You must create a snapshot of your cluster.
After you initiate an elastic resize operation in Amazon Redshift, you can't cancel the operation. Wait until the resize operation is completed before you perform another resize operation or cluster reboot.
Make sure that the new node configuration has enough storage for existing data. Even when you add nodes, your new configuration might not have enough storage because of the way that the data is redistributed. For more information about storage space, see Why does a table in an Amazon Redshift provisioned cluster consume more or less disk storage space than expected?
An elastic resize operation can cause a data skew between nodes from an uneven distribution of data slices. If you experience data skew in your cluster, then perform a classic resize instead.
If the elastic resize fails, then check whether the snapshot is completed. The resize operation can fail when the snapshot doesn't complete within a few minutes.
You can upgrade to RA3 reserved nodes when you perform a resize.