AWS re:Postを使用することにより、以下に同意したことになります AWS re:Post 利用規約

Llama 3.1 Converse calls not properly escaping quotes in tool calls

0

It looks like tool calls with Llama 3.1 models are not being escaped properly. When the model uses quotes, the tool call params are truncated. This doesn't seem to happen with another model, like Mistral Large.

$ aws bedrock-runtime converse \
  --model-id 'meta.llama3-1-405b-instruct-v1:0' \
  --messages='[
    {
      "role": "user",
      "content": [
        {
          "text": "run this query: select * from users where status=\"active\""
        }
      ]
    }
  ]' \
  --tool-config='{
    "tools": [
      {
        "toolSpec": {
          "name": "run_query",
          "description": "run a SQL query against a DB",
          "inputSchema": {
            "json": {
              "type": "object",
              "properties": {
                "query": {
                  "type": "string",
                  "description": "sql query"
                }
              }
            }
          }
        }}]}'

Llama 3.1 output:

{
  "output": {
      "message": {
          "role": "assistant",
          "content": [
              {
                  "toolUse": {
                      "toolUseId": "tooluse_S4ox6qc9QcCS-iK6AKW3Cw",
                      "name": "run_query",
                      "input": {
                          "query": "select * from users where status="
                      }
                  }
              }
          ]
      }
  },
  "stopReason": "tool_use",
  "usage": {
      "inputTokens": 70,
      "outputTokens": 25,
      "totalTokens": 95
  },
  "metrics": {
      "latencyMs": 1194
  }
}

Notice how the query is truncated. If I change the prompt to

run this query: select * from users where status=\"active\". Replace quotes with \\u0027

I sometimes correctly get

"query": "select * from users where status='active'"

But switching to a Mistral model mistral.mistral-large-2407-v1:0 gives the correct query without prompting to escape.

aws bedrock-runtime converse \
  --model-id 'mistral.mistral-large-2407-v1:0' \
  --messages='[
    {
      "role": "user",
      "content": [
        {
          "text": "run this query: select * from users where status=\"active\""
        }
      ]
    }
  ]' \
  --tool-config='{
    "tools": [
      {
        "toolSpec": {
          "name": "run_query",
          "description": "run a SQL query against a DB",
          "inputSchema": {
            "json": {
              "type": "object",
              "properties": {
                "query": {
                  "type": "string",
                  "description": "sql query"
                }
              }
            }
          }
        }}]}'

Mistral output:

{
  "output": {
      "message": {
          "role": "assistant",
          "content": [
              {
                  "toolUse": {
                      "toolUseId": "tooluse_WUcMjafeRRGtAfGUmPNWRA",
                      "name": "run_query",
                      "input": {
                          "query": "select * from users where status=\"active\""
                      }
                  }
              }
          ]
      }
  },
  "stopReason": "tool_use",
  "usage": {
      "inputTokens": 79,
      "outputTokens": 30,
      "totalTokens": 109
  },
  "metrics": {
      "latencyMs": 1924
  }
}
質問済み 4ヶ月前329ビュー
1回答
0

Hi,

What you describe really looks like a bug. Service teams do not monitor re:Post for bug reports.

So, the best is to open a support ticket with your precise description above via the console of your AWS account.

Best,

Didier

profile pictureAWS
エキスパート
回答済み 4ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ