1 Answer
- Newest
- Most votes
- Most comments
1
Hello, based on the documentation, what parameter did you configure for create_sample_dynamic_frame_from_catalog function ? num is the one that defines the maximum number of records to be fetched.
Overall, regarding the job performance, there are a couple of strategies, like changing WorkerType and NumberOfWorkers parameter on the Job.
This blog post is also handy: https://aws.amazon.com/blogs/big-data/best-practices-to-scale-apache-spark-jobs-and-partition-data-with-aws-glue/
answered 2 years ago
Relevant content
- asked 5 months ago
- Accepted Answer
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- How can I use a Lambda function to automatically start an AWS Glue job when a crawler run completes?AWS OFFICIALUpdated 2 years ago