Can't enable pipeline logs in OpenSearch Ingestion.


Using OpenSearch Ingestion, I have now fully migrated around 800 indexes from an OpenSearch domain to a Serverless collection. However, I am struggling to migrate the final 76 indexes from that domain (when activated, the pipeline stays 'paused': it reads from the source but doesn't index any new data) and I can't seem to enable the pipeline logs to debug this issue as it's throwing a strange error "Policy can't be created it reached limit". I ensured that the include index-name-regex section in the pipeline configuration is correct, that the pipeline role has IAM permissions to create logs, that the log group is 'empty' and that it has the correct format as explained in the 'Monitoring pipeline logs' AWS documentation but still no luck. Do you have any idea of where I could be going wrong? Thanks in advance.

1 Answer

Have you looked at the OpenSearch Ingestion quotas section and made sure you did not breach any limit?

profile pictureAWS
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions