2 Respostas
- Mais recentes
- Mais votos
- Mais comentários
0
The error you're seeing indicates that you may be exceeding the throttling limits for one of the AWS services, in this case Amazon Athena. Athena has quotas on the number of concurrent queries and amount of data scanned per second to prevent any single user from dominating shared resources.
A few things you could try:
- Add delays or backoff between Athena queries in your tasks to smooth out the request rate over time.
- Check your IAM roles and policies to ensure the correct service quotas and limits are in place.
- Consider using Athena concurrency scaling to automatically increase the number of workgroups as needed.
- For very high throughput needs, look into AWS Glue ETL jobs which may have higher limits than Athena alone.
- The key is to distribute the load over time rather than having all tasks attempt Athena operations at the exact same time. Some throttling is expected with serverless services, so adding retries and backoff is a best practice. Let me know if any of those suggestions help or if you have additional questions!
respondido há 7 meses
0
That is helpful but I still am left uncertain if the wrangler function will retry again or if this means it has failed and will not retry?
respondido há 7 meses
Conteúdo relevante
- AWS OFICIALAtualizada há 7 meses
- AWS OFICIALAtualizada há 3 anos
- AWS OFICIALAtualizada há 2 anos