How can I find IP addresses that attempt to connect to Amazon RDS (Aurora MySQL)?

One way is to use "Performance Insights" on the RDS console. It appears on left pane if the feature is turned on. On the insights page, choose your db from dropdown and scroll down to "Database Load". Select "Host" for "Slice By" selection. You will see a chart of all IPs connections to DB within the monitoring time window.
For a programatic solution, you will have to explore Mysql performance tables and run queries over them.

I recommend you to use VPC Flow Logs.

VPC Flow Logs is a feature that enables you to capture information about the IP traffic going to and from network interfaces in your VPC. Flow log data can be published to Amazon CloudWatch Logs or Amazon S3. After you create a flow log, you can retrieve and view its data in the chosen destination.

In your case, with VPC Flow Logs, you will be able to identify the source IP addresses that are connecting to your Amazon Aurora cluster and then create an allow list.

Here's the documentation: https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html