How do I set up cross-account streaming from Kinesis Data Firehose to Amazon OpenSearch Service (vpc based access)?

0

I have a Opensearch inside the VPC running in account A.

I want to deliver logs from Firehose in Account B to the Opensearch in Account A.

Is it possible? (i saw the documentation here for publicly accessibly opensearch https://repost.aws/knowledge-center/kinesis-firehose-cross-account-streaming )

When I try to create delivery stream from AWS CLI or terraform I am getting below exception,

An error occurred : InvalidArgumentException: Verify that the IAM role has access to the ElasticSearch domain. The same IAM role, and same input.json works when modified to the Opensearch in Account B. I have Transit gateway connectivity enabled between the AWS accounts and I can connect telnet to the Opensearch in Account A from EC2 instance in Account B.

  • Security group for Opensearch domain, allows the Account B VPC (443) access rule. (I'm not using data transformation feature)
Mounika
asked 3 months ago204 views
1 Answer
0

Check your OpenSearch resource-based policy. It should allow actions from your Kinesis role

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html#ac-types-resource

profile picture
EXPERT
answered 3 months ago
profile picture
EXPERT
reviewed 3 months ago
    • I already included the Resource based policy on Openserach domain in Account-A like this, but same error
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "AWS": "*"
          },
          "Action": "es:*",
          "Resource": "<ES Domain ARN in Account-A>/*",
        },
        {
          "Effect": "Allow",
          "Principal": {
            "AWS": "<Firehose IAM Role ARN in Account B>"
          },
          "Action": [
            "es:ESHttpPost",
            "es:ESHttpPut"
          ],
          "Resource": [
            "<ES Domain ARN in Account-A>",
            "<ES Domain ARN in Account-A>/*"
          ]
        },
        {
          "Effect": "Allow",
          "Principal": {
            "AWS": "<Firehose IAM Role ARN in Account-B>"
          },
          "Action": "es:ESHttpGet",
          "Resource": [
            "<ES Domain ARN in Account-A>/_all/_settings",
            "<ES Domain ARN in Account-A>/_cluster/stats",
            "<ES Domain ARN in Account-A>/index-name*/_mapping/type-name",
            "<ES Domain ARN in Account-A>/roletest*/_mapping/roletest",
            "<ES Domain ARN in Account-A>/_nodes",
            "<ES Domain ARN in Account-A>/_nodes/stats",
            "<ES Domain ARN in Account-A>/_nodes/*/stats",
            "<ES Domain ARN in Account-A>/_stats",
            "<ES Domain ARN in Account-A>/index-name*/_stats",
            "<ES Domain ARN in Account-A>/roletest*/_stats"
          ]
        }
      ]
    }
    ````
    

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions