How do I restore data from an OpenSearch Service domain in another AWS account?

Short description

To restore data from an OpenSearch Service domain in another account, you must set up cross-account access for the following domains:

• The source domain that you want to restore data from (Account A)
• Your OpenSearch Service destination domain (Account B)

You must also set up permissions for your domain to access the Amazon Simple Storage Service (Amazon S3) bucket that stores your data.

Note: You don't need to create an Amazon S3 bucket in the destination (Account B). OpenSearch Service uses a single Amazon S3 bucket to restore the data across your accounts.

Resolution

Note: OpenSearch Service uses the source domain in Account A for cluster snapshots.

First, create an Amazon S3 bucket in Account A in the same AWS Region as the OpenSearch Service destination domain. Then, complete the following steps to set up cross-account access. You must complete the steps for both Account A and Account B.

Set up access to the S3 bucket in both accounts

Complete the following steps:

1. Create an AWS Identity and Access Management (IAM) role that grants permission to OpenSearch Service to access the S3 bucket. Then, select Amazon Elastic Compute Cloud (Amazon EC2) as your service.
2. Create an IAM policy that grants S3 bucket access permission. Add the following policy to the IAM role:

   {     "Version": "2012-10-17",
        "Statement": [{
             "Action": [
                  "s3:ListBucket"
             ],
             "Effect": "Allow",
             "Resource": [
                  "arn:aws:s3:::AccountA_S3_Bucket_Name"
             ]
        },
        {
             "Action": [
                  "s3:GetObject",
                  "s3:PutObject",
                  "s3:DeleteObject",
                  "iam:PassRole"
             ],
             "Effect": "Allow",
             "Resource": [
                  "arn:aws:s3:::AccountA_S3_Bucket_Name/*"
             ]
        }]
   }

   Note: Replace arn:aws:s3:::AccountA_S3_Bucket_Name with your bucket's ARN.
3. Update the role's trust policy to include the following trust relationship:

   {     "Version": "2012-10-17",
        "Statement": [{
             "Effect": "Allow",
             "Principal": {
                  "Service": "es.amazonaws.com"
             },
             "Action": "sts:AssumeRole"
        }]
   }

   Note: Copy the role's ARN to a text file to use in later steps.
4. To grant OpenSearch Service write access to your S3 bucket, take one of the following actions:
   To group all your permissions under one IAM role, include the iam:PassRole permission in your existing IAM policy.
   -or-
   To split permissions, create a new IAM policy with iam:PassRole permission.
   Note: You must attach the IAM policy to the IAM role that you use to sign the HTTP request.
   The following example policy contains the required IAM permissions:

   {     "Version": "2012-10-17",
        "Statement": [{
             "Effect": "Allow",
             "Action": "iam:PassRole",
             "Resource": "arn:aws:iam::Account:snapshot role"
        },
        {
             "Effect": "Allow",
             "Action": "es:ESHttpPut",
             "Resource": "arn:aws:es:us-east-1:Account:domain/srestore/*"
        }]
   }

   Note: Replace arn:aws:iam::Account:snapshot-role with your IAM role. Also, replace arn:aws:es:us-east-1:Account:domain/restore/* with the OpenSearch Service domain in Account A or Account B.
5. If you turned on fine-grained access control for your domain, then you must map the IAM role to OpenSearch Dashboards. The IAM role uses fine-grained access control to provide access to the read or write API.
6. (Optional) If you used the Python client in Account A to register the bucket to OpenSearch Service, then launch an EC2 instance in Account A. Then, attach your IAM role.

Note: Make sure that your security group can access the OpenSearch Service domain.

Register the S3 bucket as a snapshot repository in both accounts

Important: For Account B, you must use Account B's IAM user authentication credentials and select OpenSearch Service as your destination.

To register the bucket, use the Postman application to make an HTTP PUT request. To install the application, see Download Postman on the Postman website.

Complete the following steps:

1. For PUT, enter a URL that includes the OpenSearch Service domain endpoint and S3 bucket name, for example: https://domainendpoint.amazonaws.com/_snapshot/my-snapshot-repo-name.
2. Choose the Authorization tab.
3. Under Postman, for Type, select AWS Signature. For AccessKey and SecretKey, enter the IAM user's access keys.
4. For AWS Region, enter your Region.
5. For Service Name, enter es.
6. Choose Save.
7. Choose the Headers tab.
8. For Key type, select Content-Type.
9. For Key value, select Application/JSON.
10. Choose Save.
11. Choose the Body tab.
12. Register the S3 bucket as a snapshot repository:

    {     "type": "s3",
         "settings": {
              "bucket": "snapshot",
              "region": "us-east-1",
              "role_arn": "arn:aws:iam::Account:role/cross"
         }
    }

    Note: Replace snapshot with your S3 bucket name, us-east-1 with your Region, and arn:aws:iam::Account:role/cross with the IAM role from Account A or Account B.
13. To submit the query, choose Send.

After the registration completes, you receive a Status Code: 200 OK message.

Verify that your snapshot repository is configured correctly

In Account A, complete the following steps:

1. Open OpenSearch Dashboards, and then check the available data on the bucket.
2. Run the following command to take a new snapshot:

   PUT /_snapshot/my-snapshot-repo-name/snapshot_name

   Note: Replace my-snapshot-repo-name with your S3 bucket name and snapshot name with your snapshot name.
   The following example output verifies a complete S3 bucket registration to the OpenSearch Service domain in Account A:

   GET _cat/snapshots/my-snapshot-repo-name
   
   SUCCESS 1585190280 02:38:00 1585190284 02:38:04 3.9s 4 4 0 4

In Account B, complete the following steps:

1. Open OpenSearch Dashboards.

2. Run the following command to view the snapshots from Account A that are available in the S3 bucket:

   GET _cat/snapshots/my-snapshot-repo-name

   Note: Replace my-snapshot-repo-name with your snapshot repository.
   The following example output confirms that you set up cross-account access in Account B:

   today SUCCESS 1585190280 02:38:00 1585190284 02:38:04 3.9s 4 4 0 4

3. To restore the data, run the following command:

   POST /_snapshot/my-snapshot-repo-name/snapshotID/_restore 

   Note: Replace my-snapshot-repo-name with your snapshot repository and snapshotID with your snapshot name.

4. To restore specific indices, run the following command:

   POST /_snapshot/my-snapshot-repo-name/snapshot ID/_restore 
   { 
      "indices": "index name1,index name2" 
   }

   Note: Replace my-snapshot-repo-name with your snapshot repository and snapshotID with your snapshot name. Also, replace index name1 and indexname2 with your indices.

5. To verify the restored indices, run the following command:

   GET _cat/indices?v

Related information

How can I migrate data from one OpenSearch Service domain to another?

Step 1.3: Attach a bucket policy to grant cross-account permissions to Account B

Migrating Amazon OpenSearch Service indices using remote reindex

IAM role management