AWS Bedrock batch inference stuck on SCHEDULED

Question

Hello,

I’m trying to use Bedrock batch inference to process call transcripts at scale and have encountered an issue. 
In a small scale test with 100 records, each requiring a brief summary, the job status reaches "SCHEDULED" and remains there without progressing.

Since this is the only job currently running, I don’t believe it’s a quota issue. I'm using anthropic.claude-3-sonnet-20240229-v1:0  in us-east-1 with the following message format:

python

message = {
    "role": "user",
    "content": f"Summarize the following call transcript: {content}",
}
Additionally, one job I previously submitted completed with an error(the status was COMPLETED with errors in the output jsonl), suggesting that permissions may not be the issue either.

Could you provide guidance on what might be causing this delay or recommend steps to troubleshoot and resolve it?

Thank you!

re:Post Agent · Answer

I understand you're experiencing an issue with your AWS Bedrock batch inference job getting stuck in the "SCHEDULED" status while processing call transcripts. This can be frustrating, but there are a few things we can consider to troubleshoot the problem.

First, even though you're not running multiple jobs concurrently, it's still worth verifying that you haven't hit any service quotas. Sometimes, quotas can be lower than expected, especially if you're using a new account or haven't requested increases.

Since a previous job completed (albeit with errors), it suggests that your basic permissions and setup are likely correct. However, the fact that it completed with errors indicates there might be an issue with the input data or the way the model is processing it.

Here are some steps you can take to troubleshoot:

1. Check the job details in the AWS Management Console. Navigate to the Batch inference section and select your job to view its details. This might provide more information about why the job is stuck.

2. Verify that your input data is correctly formatted. Ensure that each record in your JSONL file is properly structured with a unique identifier (recordId) and the modelInput in the format required by the Claude model.

3. Consider reducing the batch size further for testing. Try with just a few records to see if the job progresses.

4. Review the S3 bucket permissions to ensure that Bedrock has the necessary access to read the input and write the output.

5. Check if there are any error messages or logs available in CloudWatch that might provide more insight into the issue.

6. If possible, try running the same prompts through the Bedrock console or API directly (not as a batch job) to see if they process successfully. This can help isolate whether the issue is with the batch processing or the prompts themselves.

7. Ensure that your account has the necessary permissions to use the Claude 3 Sonnet model in batch inference mode.

If none of these steps resolve the issue, it may be worth contacting AWS support for further assistance. They can look into the specifics of your account and job configuration to identify any underlying issues.

Remember, batch inference is designed to handle large-scale processing, so once you resolve this issue, it should provide an efficient way to summarize your call transcripts at scale.  
**Sources**  
[Enhance call center efficiency using batch inference for transcript summarization with Amazon Bedrock | AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/enhance-call-center-efficiency-using-batch-inference-for-transcript-summarization-with-amazon-bedrock/)  
[Process multiple prompts with batch inference - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html)  
[Monitor batch inference jobs - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-monitor.html)

AWS Bedrock batch inference stuck on SCHEDULED

Relevant content