AWS re:Post을(를) 사용하면 다음에 동의하게 됩니다. AWS re:Post 이용 약관

Llama 3.1 Converse calls not properly escaping quotes in tool calls

0

It looks like tool calls with Llama 3.1 models are not being escaped properly. When the model uses quotes, the tool call params are truncated. This doesn't seem to happen with another model, like Mistral Large.

$ aws bedrock-runtime converse \
  --model-id 'meta.llama3-1-405b-instruct-v1:0' \
  --messages='[
    {
      "role": "user",
      "content": [
        {
          "text": "run this query: select * from users where status=\"active\""
        }
      ]
    }
  ]' \
  --tool-config='{
    "tools": [
      {
        "toolSpec": {
          "name": "run_query",
          "description": "run a SQL query against a DB",
          "inputSchema": {
            "json": {
              "type": "object",
              "properties": {
                "query": {
                  "type": "string",
                  "description": "sql query"
                }
              }
            }
          }
        }}]}'

Llama 3.1 output:

{
  "output": {
      "message": {
          "role": "assistant",
          "content": [
              {
                  "toolUse": {
                      "toolUseId": "tooluse_S4ox6qc9QcCS-iK6AKW3Cw",
                      "name": "run_query",
                      "input": {
                          "query": "select * from users where status="
                      }
                  }
              }
          ]
      }
  },
  "stopReason": "tool_use",
  "usage": {
      "inputTokens": 70,
      "outputTokens": 25,
      "totalTokens": 95
  },
  "metrics": {
      "latencyMs": 1194
  }
}

Notice how the query is truncated. If I change the prompt to

run this query: select * from users where status=\"active\". Replace quotes with \\u0027

I sometimes correctly get

"query": "select * from users where status='active'"

But switching to a Mistral model mistral.mistral-large-2407-v1:0 gives the correct query without prompting to escape.

aws bedrock-runtime converse \
  --model-id 'mistral.mistral-large-2407-v1:0' \
  --messages='[
    {
      "role": "user",
      "content": [
        {
          "text": "run this query: select * from users where status=\"active\""
        }
      ]
    }
  ]' \
  --tool-config='{
    "tools": [
      {
        "toolSpec": {
          "name": "run_query",
          "description": "run a SQL query against a DB",
          "inputSchema": {
            "json": {
              "type": "object",
              "properties": {
                "query": {
                  "type": "string",
                  "description": "sql query"
                }
              }
            }
          }
        }}]}'

Mistral output:

{
  "output": {
      "message": {
          "role": "assistant",
          "content": [
              {
                  "toolUse": {
                      "toolUseId": "tooluse_WUcMjafeRRGtAfGUmPNWRA",
                      "name": "run_query",
                      "input": {
                          "query": "select * from users where status=\"active\""
                      }
                  }
              }
          ]
      }
  },
  "stopReason": "tool_use",
  "usage": {
      "inputTokens": 79,
      "outputTokens": 30,
      "totalTokens": 109
  },
  "metrics": {
      "latencyMs": 1924
  }
}
질문됨 4달 전332회 조회
1개 답변
0

Hi,

What you describe really looks like a bug. Service teams do not monitor re:Post for bug reports.

So, the best is to open a support ticket with your precise description above via the console of your AWS account.

Best,

Didier

profile pictureAWS
전문가
답변함 4달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠