Make Lambda Function Urls to be accessible within the VPC only.

Lambda does not support what you are asking for. As you suggested, you could use IAM to allow access only to a holder of the appropriate IAM role.

Poor implementation. 100% should have been behind a security group, the lambda itself is...

There are many times this would be so much better just using NONE instead of IAM for authentication, and limiting to a set of ec2 or vpc etc. For example, I want to have Oracle RDS talk to IAM. In order to call the lambda, it requires sgv4 to be used with the call. In order for that to occur, you need to obtain IAM access key and secret. We don't use IAM users, we use roles, as we should. So how do we do that? we have to create a user just for the purpose of this, then we have to worry about rotating the keys from time to time, and getting them into the database.

Once that is done, we still need to write something which will generate the correct signature and then attach it to the standard request stuff.

So annoying. It's fine if you have access to AWS CLI tools or python etc, where the signature can just be handled via boto3 etc, but not so great rolling your own.

I wish I could just have set my SG on the lambda to only accept connections from my RDS oracle SG, and it would be secure.