En utilisant AWS re:Post, vous acceptez les AWS re:Post Conditions d’utilisation
AWS re:Postre:Post
À propos de re:Post

Bedrock Converse API - Consistent Timeout Issue with Larger models like Anthropic Sonnet 3.5 and Meta Llama 405B

0

And yes it couldn't be any sort of rate-limit because it was my first message in last 48hrs.

Region Name - us-west-2

WARNING:bedrock_converse.utils:Retrying core.llama.bedrock_converse.utils.converse_with_retry_async._retry in 4.0 seconds as it raised ReadTimeoutError: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/meta.llama3-1-405b-instruct-v1%3A0/converse".
Retrying bedrock_converse.utils.converse_with_retry_async._retry in 4.0 seconds as it raised ReadTimeoutError: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/meta.llama3-1-405b-instruct-v1%3A0/converse"
Sujets
Machine Learning et AIL’IA générative sur AWS
Balises
Amazon Bedrock
Langue
English
Nath
demandé il y a 9 mois871 vues
1 réponse
0

Try to add this. Note that it is client side timeout and not server-side.

 config = Config(
     connect_timeout=300,
     read_timeout=300,
 )
 bedrock_client = boto3.client(service_name='bedrock-runtime', config=config)
AWS
Mi_Sha
répondu il y a 9 mois
profile picture
EXPERT
Giovanni Lauria
vérifié il y a 8 mois
  • Joe
    il y a 4 mois

    I tried this and it only extends the amount of time I must wait for the request to be retried. Instead of waiting 60 seconds, this setting makes you wait 5 minutes for a request that normally takes 5 - 15 seconds. (Note that I'm testing from EC2 instances in the same region as the Bedrock endpoint.)

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions

Contenus pertinents