Bedrock Converse API - Consistent Timeout Issue with Larger models like Anthropic Sonnet 3.5 and Meta Llama 405B

0

And yes it couldn't be any sort of rate-limit because it was my first message in last 48hrs.

Region Name - us-west-2

WARNING:bedrock_converse.utils:Retrying core.llama.bedrock_converse.utils.converse_with_retry_async._retry in 4.0 seconds as it raised ReadTimeoutError: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/meta.llama3-1-405b-instruct-v1%3A0/converse".
Retrying bedrock_converse.utils.converse_with_retry_async._retry in 4.0 seconds as it raised ReadTimeoutError: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/meta.llama3-1-405b-instruct-v1%3A0/converse"
질문됨 2달 전157회 조회
1개 답변
0

Try to add this. Note that it is client side timeout and not server-side.

 config = Config(
     connect_timeout=300,
     read_timeout=300,
 )
 bedrock_client = boto3.client(service_name='bedrock-runtime', config=config)
AWS
답변함 2달 전
profile picture
전문가
검토됨 2달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠