AWS bedrock Claude v2 API - go sdk v2

1

I am programatically calling claude v2 api using Go sdk

import 	"github.com/aws/aws-sdk-go-v2/service/bedrockruntime"

	output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-v2"),
		ContentType: aws.String("application/json"),
	})

Most of the times, it takes more than 5 seconds which is significantly slower than the responsiveness of bedrock playground. Is anything wrong with the code, or any suggestions for improving response time?

Arvi
gefragt vor 3 Monaten258 Aufrufe
1 Antwort
0

Hi,

As you say "v2", did you try v2.1 ? See https://aws.amazon.com/about-aws/whats-new/2023/11/claude-2-1-foundation-model-anthropic-amazon-bedrock/

The announcement above doesn't much resp time enhancement. But, as it is the new favored model, it may receive more computing resources than v2.

Best,

Didier

profile pictureAWS
EXPERTE
beantwortet vor 3 Monaten
  • Hey, thanks for the response. I tried v2.1 but issue still persists. updated code looks like this

    	output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
    		Body:        payloadBytes,
    		ModelId:     aws.String("anthropic.claude-v2:1"),
    		ContentType: aws.String("application/json"),
    	})
    

    region : us-east-1

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen