For Comprehend async Analysis Jobs, is there an equivalent to the async Textract Analysis Job NotificationChannel?

0

The async Textract Analysis Job NotificationChannel is a very helpful mechanism to listen for results. I can't find an equivalent to the async Comprehend Analysis Jobs. Is there a plan to implement it? If not, what are the alternative mechanisms? Is it a good enough solution to add Event Notifications to the the async Comprehend Analysis Job output bucket?

Joen
asked 4 months ago227 views
1 Answer
1
Accepted Answer

Hello, While asynchronous Textract API's have a NotificationChannel parameter to receive notifications about the job status, unfortunately this option is not available with Comprehend API's. Further, I will not be able to comment regarding future plans and availability of this feature. You can keep an eye on below link for future updates/releases. [+]https://aws.amazon.com/new/

Available options:

1)The status of the jobs can be tracked using the corresponding Describe* operation. For example, the "DescribeDocumentClassificationJob" operation can be used to track the status of a job initiated with "StartDocumentClassificationJob". You can have a custom script where the Describe* operation is made after regular intervals to check the status of the job.

Please note that we can use the S3 trigger to notify and trigger the lambda function whenever the output file is created in the given location in S3. For more details regarding this approach, please refer to [1].

  1. In addition to the method of regularly executing a Lambda function in order to execute an API corresponding to the Describe* operation, a format where the API is executed with StepFunctions can also be considered. StepFunctions makes it possible to define tasks that execute AWS service APIs [2]. By combining this with the wait state [3],automatic regular execution is possible.

For example, the following two implementation methods can be considered.

1. Execute the StepFunctions state machine with the Lambda function that executed "StartDocumentClassificationJob", and include the target Job ARN as an argument during execution [4,5].

2. Create a state machine containing both a Lambda function to execute "StartDocumentClassificationJob" and an operation to execute the Describe API. When the Lambda function is executed, the target Job ARN is included as a return value, and the Lambda function response value is used in the next Describe API execution step [4,5]

You could consider implementing the above processing, detecting job completion, and sending notifications to SNS.

3)Another alternative would be to setup S3 event notifications at the destination S3 bucket. If you have a dedicated path in the S3 bucket where the output files are located, whenever a new object is placed under the specified prefix and suffix, a notification can be sent to a destination SNS topic.

4)We can also use another approach and use Amazon EventBridge to route events dynamically. EventBridge consumes S3 events via AWS CloudTrail. A single trail can log events for one or more S3 buckets, and you can configure which data events are recorded. Please refer to [6] for more details regarding this approach.

I would request you to kindly check the provided options and implement the one which fits your use case.

In case you have further queries or concern regarding the same, I'd recommend you to reachout to AWS Support by creating a support case with Comprehend Technical support team[7] so that an engineer can help you in addressing your questions.

References:

[1]Using an Amazon S3 trigger to invoke a Lambda function: https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html [2] AWS SDK service integrations : https://docs.aws.amazon.com/step-functions/latest/dg/supported-services-awssdk.html [3] Wait: https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-wait-state.html [4] inputPath, Parameters and ResultSelector: https://docs.aws.amazon.com/step-functions/latest/dg/input-output-inputpath-params.html [5] https://dev.classmethod.jp/articles/stepfunctions-parameters-inter-states/ [6] Using dynamic Amazon S3 event handling with Amazon EventBridge: https://aws.amazon.com/blogs/compute/using-dynamic-amazon-s3-event-handling-with-amazon-eventbridge/ [7]Open a support case with AWS using the link: https://console.aws.amazon.com/support/home?#/case/create

AWS
Radhika
answered 4 months ago
  • Thank you for your thorough answer. I have accepted your answer because in the absence of having an actual feature similar to Textract's NotificationChannel, I acknowledge that those are my available workaround options. I have tried the Polling option [1] and the option using S3 Event Notification against the Comprehend output bucket [2]. Both are not ideal solutions. The polling unnecessarily adds extra Describe* API calls. It also could possibly lengthen the time from request to result. With the S3 Event Notification, an S3 Event is thrown (and the output file is available) while the Comprehend job was still in IN_PROGRESS status.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions