How to create an S3 policy that allows a user to only connect and upload files?

Hello.

First, I don't think it's possible to connect to S3 directly via SFTP or similar.
I think you need to set up Transfer Family to access S3 with SFTP.
https://docs.aws.amazon.com/transfer/latest/userguide/what-is-aws-transfer-family.html

If you use WinSCP etc., you can connect like SFTP. (Actually, WinSCP handles access using AWS API rather than SFTP)
https://winscp.net/eng/docs/guide_amazon_s3

Looking at the bucket policy, it doesn't include "Deleteobject", so I think the policy does not allow you to delete objects.
Therefore, I think that the problem with not being able to connect is not the bucket policy, but the IAM user's IAM policy or connection method.

I believe that's the identity-based policy attached to the IAM user group that you described. You will need to separate the action s3:ListAllMyBuckets into its own policy statement with "Resource": "*" as its target. That permission doesn't apply to any individual bucket but rather allows the user to obtain a list of all the S3 buckets in the local account. Currently, you're restricting the permission to the individual bucket, which will never get hit.

As Riku Kobayashi pointed out, your policy already doesn't include the s3:DeleteObject, s3:DeleteObjectVersion, or s3:PutLifecycleConfiguration permissions, so the user won't be able to delete anything in the bucket, unless the bucket policy (which can add to the permissions assigned here) grants that right.