How do I resize an Amazon Redshift cluster?

Short description

To resize your cluster, use either elastic resize or classic resize. It's a best practice to use elastic resize if your cluster is eligible. If your cluster doesn't qualify, then use classic resize.

Resolution

Determine your cluster eligibility

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

To verify whether your cluster is eligible for elastic resize, run the describe-node-configuration-options AWS CLI command:

aws redshift describe-node-configuration-options --cluster-identifier example-cluster-id --action-type resize-cluster

Note: Replace example-cluster-id with your cluster ID.

If the cluster is eligible for elastic resize, then the output looks similar to the following:

{
  "NodeConfigurationOptionList": [
    {
      "NodeType": "dc2.large",
      "NumberOfNodes": 2,
      "EstimatedDiskUtilizationPercent": 0.01
    },
    {
      "NodeType": "ra3.16xlarge",
      "NumberOfNodes": 2,
      "EstimatedDiskUtilizationPercent": 0.01
    }
  ]
}

If the cluster isn't eligible for elastic resize, then the output looks similar to the following:

{
  "NodeConfigurationOptionList": []
}

Reduce operation time

When you use elastic resize for a cluster with the same node type, the operation doesn't create a new cluster and completes quickly.

When you use classic resize, the operation time is based on the following factors:

• The workload on the source cluster.
• The number and size of the tables that you transfer from the source to the target cluster.
• The distribution of data across the compute nodes and slices.
• The node configuration in the source and target clusters.

To reduce the duration of a classic resize, take the following actions:

• Migrate to an RA3 node type.
• Identify and fix skewed tables. To identify skewed tables, run the AWS Labs table inspector script. For more information, see table_inspector on the GitHub website. To fix skewed tables, choose an appropriate distribution key.
• Remove unused tables. To identify unused tables, run the AWS Labs unscanned table summary script. For more information, see unscanned_table_summary on the GitHub website.
  Note: The unscanned table summary only shows recent history from the last 2-5 days. To capture usage data over a longer period, use the system object persistence utility. For more information, see SystemTablePersistence on the GitHub website.
• Identify missing table data. To identify missing table data, use the AWS Labs missing_table_stats script. For more information, see missing_table_stats on the GitHub website. Then, run the ANALYZE command on those tables.

For more information on resize performance optimization, see Top 10 performance tuning techniques for Amazon Redshift. To use the Amazon Redshift console to check the status of your resize operation, choose the Status tab on the cluster details page. The Status tab shows the average rate of transfer, the elapsed time, and the remaining time.

Resize your cluster

Use elastic resize to change your cluster node type, number of nodes, or both.

Use classic resize when elastic resize doesn't support the change in cluster size or node.

Troubleshoot your resize operation

During a resize operation, your table can increase or decrease in size. For more information, see Why does a table in an Amazon Redshift provisioned cluster consume more or less disk storage space than expected?

If your cluster has a status of NONE in the AWS CLI, then Amazon Redshift is still provisioning the target cluster. Your target cluster doesn't copy over until the provisioning stage is complete. After Amazon Redshift provisions your target cluster, the status changes to IN_PROGRESS.

If your AWS CloudFormation StackSets don't resize, and you receive an internal error message, then check if the cluster is eligible for elastic resize. The error message is similar to: "An internal error has occurred. Please try your query again at a later time." This occurs because a CloudFormation stack uses elastic resize, and Classic:false is set by default.

If you receive the error message "Please choose a larger target cluster," then your data doesn't fit into the target cluster. Resize your Amazon Redshift cluster with more nodes or a different node type.

To cancel a classic resize operation before the operation completes, choose Cancel resize from the cluster list in the Amazon Redshift console.

Billing for resized clusters

During the resize operation, AWS bills you for the clusters that are available to you. For example, during the resize operation, AWS bills you for the source configuration. After the resize is complete, AWS no longer bills you for the source configuration. Billing starts for the target configuration as soon as the cluster status changes to Available.

When you resize smaller node types, such as large or xlarge, to larger node types, such as 8xlarge, your cluster requires more storage per node. The more storage that you have per node, the more metadata that's written when you run a COMMIT operation. Therefore, the base cost for a single COMMIT operation is higher for larger nodes. If you run several small COMMIT operations concurrently, you might see a decrease in performance. For improved performance, group multiple changes into a single COMMIT operation.

If you purchased Reserved Instances, then your billing depends on the resized cluster configuration, reserved node types, and reserved node count. For more information, see How reserved nodes work.

Related information

Troubleshooting connection issues in Amazon Redshift

Building high-quality benchmark tests for Amazon Redshift using SQLWorkbench and psql

Cluster operations