AWS bedrock Claude v2 API - go sdk v2

1

I am programatically calling claude v2 api using Go sdk

import 	"github.com/aws/aws-sdk-go-v2/service/bedrockruntime"

	output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-v2"),
		ContentType: aws.String("application/json"),
	})

Most of the times, it takes more than 5 seconds which is significantly slower than the responsiveness of bedrock playground. Is anything wrong with the code, or any suggestions for improving response time?

Arvi
preguntada hace 3 meses260 visualizaciones
1 Respuesta
0

Hi,

As you say "v2", did you try v2.1 ? See https://aws.amazon.com/about-aws/whats-new/2023/11/claude-2-1-foundation-model-anthropic-amazon-bedrock/

The announcement above doesn't much resp time enhancement. But, as it is the new favored model, it may receive more computing resources than v2.

Best,

Didier

profile pictureAWS
EXPERTO
respondido hace 3 meses
  • Hey, thanks for the response. I tried v2.1 but issue still persists. updated code looks like this

    	output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
    		Body:        payloadBytes,
    		ModelId:     aws.String("anthropic.claude-v2:1"),
    		ContentType: aws.String("application/json"),
    	})
    

    region : us-east-1

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas