why doesn't the LLM of Novasonic understand the conversation of words?

Question

Here, in the conversation of words, why doesn't the LLM of Novasonic understand?

**logs-1:
Conversation History**
```
Assistant: Hello! This is Sarah calling from ABC Company. Is this a good time to chat for just a minute?
Customer: yes
Assistant: Perfect! I'm calling to help you schedule an appointment with us.
Assistant: When would work best for you?
Customer: can i book the date on eighteenth of december
Customer: around three o'clock
Assistant: I'm sorry, but that time is outside our business hours.
Assistant: Our business hours are 9 AM to 5 PM. Could you please choose a time within these hours?
Customer: okay can you
Customer: book the appointment date on three pm
Assistant: Wonderful! So I have you down for Thursday December 18 2025 at 03:00 PM.
Assistant: Is that correct?
Assistant: Perfect! Your appointment is confirmed for Thursday December 18 2025 at 03:00 PM.
Assistant: You'll receive a confirmation email shortly. Is there anything else I can help you with?
Customer: nope thanks
Assistant: Thank you so much for your time! We look forward to seeing you.
Assistant: Have a wonderful day!
```
**log-2:
Conversation History**
```
Assistant:Hello! This is Sarah calling from ABC Company. Is this a good time to chat for just a minute?
Customer:yeah sure
Assistant:Perfect! I'm calling to help you schedule an appointment with us.
Assistant:When would work best for you?
Customer:um sometime next month
Assistant:I'm sorry, I didn't quite catch the date. Could you please say it again?
Customer:i want an appointment next month
Assistant:I'm sorry, but that date is too far in advance.
Assistant:We can only schedule appointments within the next 15 business days.
Assistant:The latest available date is January 02 2026.
Assistant:Could you please choose a date within this timeframe?
Customer:or can i have an appointment on second of january twenty
Customer:twenty six
Assistant:I'd be happy to find an available time. Which date would work best for you?
Customer:um second of january twenty twenty six
Assistant:I'd be happy to find an available time. Which date would work best for you?
Customer:um no thank you you may end the call now
Assistant:I understand. Thank you for your time. If you change your mind, feel free to call us anytime.
Assistant:Have a great day!
```
***What is the problem?***
1. The problem is in logs-1: The customer says "sometime next month," but the assistant can't catch the date.
2. The problem is in logs-1: The customer says "January twenty twenty-six," but the assistant replies, "I'd be happy to find an available time. Which date would work best for you?"
3. The problem is in logs-2: the customer says, "Can I book the date on the 18th of December around 3 o'clock?" but the assistant replies, "I'm sorry, but that time is outside our business hours."

re:Post Agent · Answer

The problem appears to be in log-2 (not log-1 as labeled in your question), where the customer says "um second of january twenty twenty six" but the assistant responds with "I'd be happy to find an available time. Which date would work best for you?" This indicates that Nova Sonic is not properly understanding the spoken date information.

Amazon Nova Sonic is designed for speech-to-speech conversational tasks, but it has limitations in how it processes certain types of information in spoken form. When dealing with dates, numbers, and complex information, the speech model may struggle more than text-based models because:

1. Speech comprehension differs from text comprehension - information that's clear in text may need to be reformulated for speech contexts.

2. Memory constraints in spoken interactions are different from text - spoken dates and numbers can be particularly challenging to process accurately.

For better results with Nova Sonic when handling dates and appointments, the system prompt should be optimized for speech interactions. For example, it should be designed to:
- Request one piece of information at a time
- Confirm understanding by repeating back information
- Break down complex information (like dates) into smaller chunks
- Use natural conversational flow rather than formal structures

The model is likely struggling with the spoken date format, which is a common challenge in speech-to-speech AI systems.  
**Sources**  
[Amazon Nova Sonic - AWS AI Service Cards](https://docs.aws.amazon.com/ai/responsible-ai/nova-sonic/overview.html)  
[Amazon Nova Sonic prompting best practices - Amazon Nova](https://docs.aws.amazon.com/nova/latest/userguide/prompting-speech.html)  
[Best practices for the Amazon Nova Sonic system prompt - Amazon Nova](https://docs.aws.amazon.com/nova/latest/userguide/prompting-speech-best-practices.html)

why doesn't the LLM of Novasonic understand the conversation of words?

Add your answer

Relevant content