I am using Bedrock to access Mistral 2 Large, with converse_stream API. I am using tools too. 
I get this message back: 

BedrockManager - ERROR - get_model_response_stream failed: An error occurred (ValidationException) when calling the ConverseStream operation: This model doesn't support tool use in streaming mode.

The call looks like this: 

         final_response = self.bedrock.converse_stream(
                modelId=self.config.model_id,
                system=system,
                messages=messages,
                inferenceConfig=inferenceConfig,
                toolConfig=tool_config_tool
            )

tool_config is 

 tool_config_tool = {
    "tools": [
        {
            "toolSpec": {
                "name": "search_document",
                "description": "Search through the uploaded document for relevant passages",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {"query": {"type": "string"}},
                        "required": ["query"],
                    }
                },
            }
        }
    ],
    "toolChoice": {"auto": {}},
}

My question is two-fold: 

 1. When will AWS support this for Mistral 2? 
 2. How do I change the tool_config_tool to make sure the API is not trying to use tools in this instance to allow for streaming.

All Content tagged with Mistral AI in Amazon Bedrock

Using Bedrock with Mistral 2 Large, converse API with tools ...