Bedrock FM API chunk token streaming

0

I notice when I use titan and claude with streaming. The results come out in chunk, compared to one token at a time. Is there a way to control the chunk size, or it is determined in real-time on your side.

Thanks!

Topics

Machine Learning & AI Generative AI on AWS

Tags

Language

English

asked 5 months ago273 views

1 Answer

Newest
Most votes
Most comments

1

Currently there is no way to control the length of the chunking responses. More details on what parameters you can control are available in the documentation at InvokeModelWithResponseStream and Inference parameters for foundation models

EXPERT

Francis Flannery

answered 5 months ago

Relevant content

Bedrock Titan Model Streaming /invoke-with-response-stream
ane
asked a month ago
Does AWS API Gateway support http chunk (transfer-encoding) ?
2smin
asked 3 months ago
Remaining chunks after stopping a live stream in HLS
Cyborg
asked 5 years ago
why streaming is not supported for titan and cohere model on bedrock
JCJJ
asked 5 months ago
How do I create a task in AWS DMS and turn on control tables?
AWS OFFICIALUpdated 2 years ago
How can I get custom scopes in the access token when I make an InitiateAuth or AdminInitiateAuth API call?
AWS OFFICIALUpdated 3 months ago
Why do I get a “The provided token is malformed or otherwise invalid” error when I launch an Amazon EMR cluster using Hive and Presto in the AWS China (Beijing) Region?
AWS OFFICIALUpdated 2 years ago
How can I troubleshoot 403 "missing authentication token" errors when invoking API Gateway REST or HTTP APIs with a custom domain name?
AWS OFFICIALUpdated a year ago
Bringing Generative AI to the data warehouse with Amazon Bedrock and Amazon Redshift
EXPERT
Rick Fraser
published 3 months ago
Troubleshooting "The Access Key ID or Security Token is Invalid" Error after Upgrading DynamoDB Local to Version 2.0.0 / 1.23.0 or Greater
EXPERT
Leeroy Hannigan
published 10 months ago