Bedrock Converse API - Consistent Timeout Issue with Larger models like Anthropic Sonnet 3.5 and Meta Llama 405B

0

And yes it couldn't be any sort of rate-limit because it was my first message in last 48hrs.

Region Name - us-west-2

WARNING:bedrock_converse.utils:Retrying core.llama.bedrock_converse.utils.converse_with_retry_async._retry in 4.0 seconds as it raised ReadTimeoutError: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/meta.llama3-1-405b-instruct-v1%3A0/converse".
Retrying bedrock_converse.utils.converse_with_retry_async._retry in 4.0 seconds as it raised ReadTimeoutError: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/meta.llama3-1-405b-instruct-v1%3A0/converse"
已提問 2 個月前檢視次數 154 次
1 個回答
0

Try to add this. Note that it is client side timeout and not server-side.

 config = Config(
     connect_timeout=300,
     read_timeout=300,
 )
 bedrock_client = boto3.client(service_name='bedrock-runtime', config=config)
AWS
已回答 2 個月前
profile picture
專家
已審閱 2 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南