botocore.errorfactory.ModelTimeoutException: An error occurred (ModelTimeoutException) when calling the InvokeModel operation

Question

- I'm importing, `from langchain_community.embeddings import BedrockEmbeddings` package.
- I'm using `embeddings = BedrockEmbeddings(
    credentials_profile_name="default",
    region_name="us-east-1",
    model_id="amazon.titan-embed-text-v2:0"
)` to embed documents
- I have a function that embeds an array of documents (called `batch`) using `embeddings.embed_documents(batch)`. This works.
- My batch size is 250. I.e., an array of 250 strings
- I'm facing this error, every now and then:

`botocore.errorfactory.ModelTimeoutException: An error occurred (ModelTimeoutException) when calling the InvokeModel operation: Model has timed out in processing the request. Try your request again.`

`File "...\site-packages\langchain_community\embeddings\bedrock.py", line 150, in _embedding_func
    raise ValueError(f"Error raised by inference endpoint: {e}")
ValueError: Error raised by inference endpoint: An error occurred (ModelTimeoutException) when calling the InvokeModel operation: Model has timed out in processing the request. Try your request again.
`

Giovanni Lauria · Answer

The `ModelTimeoutException` error you're encountering when embedding documents using the `BedrockEmbeddings` package from `langchain_community.embeddings` in AWS could be due to the model taking too long to process a request, which might be resolved by reducing the batch size, implementing a retry mechanism for failed requests or monitoring the model's performance.

Joseph · Answer

Figured it! Apparently, embedding functions do not like long spaces, new lines (
) in the text. So, if they are present in your documents, they will negatively impact the embedding or simply stall. I removed the spaces in the text, and it worked!

On a side note, it was VS Code that split the sentences to make it easier for reading and programming when I prepped my text. And that's how the `
`s got introduced within my texts.

botocore.errorfactory.ModelTimeoutException: An error occurred (ModelTimeoutException) when calling the InvokeModel operation

Relevant content