Bedrock Converse API - Consistent Timeout Issue with Larger models like Anthropic Sonnet 3.5 and Meta Llama 405B

0

And yes it couldn't be any sort of rate-limit because it was my first message in last 48hrs.

Region Name - us-west-2

WARNING:bedrock_converse.utils:Retrying core.llama.bedrock_converse.utils.converse_with_retry_async._retry in 4.0 seconds as it raised ReadTimeoutError: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/meta.llama3-1-405b-instruct-v1%3A0/converse".
Retrying bedrock_converse.utils.converse_with_retry_async._retry in 4.0 seconds as it raised ReadTimeoutError: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/meta.llama3-1-405b-instruct-v1%3A0/converse"
preguntada hace 2 meses161 visualizaciones
1 Respuesta
0

Try to add this. Note that it is client side timeout and not server-side.

 config = Config(
     connect_timeout=300,
     read_timeout=300,
 )
 bedrock_client = boto3.client(service_name='bedrock-runtime', config=config)
AWS
respondido hace 2 meses
profile picture
EXPERTO
revisado hace 2 meses

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas