S3 Presigned URL Expiring Earlier Than Expected

I assume you are using a Role to create the pre-signed URL and here is the reason why its expiring

https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html#PresignedUrl-Expiration

A presigned URL remains valid for the period of time specified when the URL is generated. If you create a presigned URL with the Amazon S3 console, the expiration time can be set between 1 minute and 12 hours. If you use the AWS CLI or AWS SDKs, the expiration time can be set as high as 7 days.

If you created a presigned URL by using a temporary token, then the URL expires when the token expires. In general, a presigned URL expires when the credential you used to create it is revoked, deleted, or deactivated. This is true even if the URL was created with a later expiration time. For more information about how the credentials you use affect the expiration time, see Who can create a presigned URL.

AWS Security Token Service – Valid up to maximum 36 hours when signed with long-term security credentials or the duration of the temporary credential, whichever ends first.

A role uses a temporary token

If you need the Pre-signed URL to be valid longer than a roles temporary credentials, then you will have to create an IAM user and use its credentials to have a URL last up to 7 days. https://repost.aws/questions/QUd6IcnCLtQ6aN7Nd43TBUag/generate-s3-presigned-url-with-7-day-expiry-via-lambda

Hey..

This issue could be happening for a few reasons:

Clock Drift: Ensure that the machine generating the URL has the correct time. Even small clock differences can cause the URL to expire earlier.

IAM Permissions: Double-check the IAM permissions for the role generating the URL. Missing permissions can cause the presigned URL to fail.

CodeBuild Timeout: Make sure there are no timeouts or job failures in the CodeBuild stage that could be invalidating the URL sooner than expected.

For more details on troubleshooting presigned URLs, you can refer to AWS S3 Presigned URLs Documentation https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html

Hope this helps

Hey nmos,

Please go through the below steps

System Clock Skew:

• What is it? Presigned URLs are time-bound and rely on accurate system time to calculate expiration. If the system clock where the URL is generated has skew (i.e., is out of sync with standard time), it can cause the URLs to expire prematurely or even be invalid at the time of creation.

• Fix: Ensure the CodeBuild environment is synchronized with NTP (Network Time Protocol). Most AWS-managed services use NTP to keep time in sync, but custom build environments might not.

• Check:

• Confirm that the system time in the CodeBuild container matches NTP time.

• Check if there's any noticeable time drift, especially in long-running builds.

Expiration Parameter and Code Implementation:

• What to look for? The expiration time for presigned URLs is determined by the Expires parameter passed during the URL creation. In some cases, if this value is not correctly set or converted (e.g., seconds vs. milliseconds), it can lead to early expiration.

• Fix: Ensure that the Expires value in your code is set correctly to match the desired time frame. For a 24-hour expiration, the value should be 86400 seconds (24 hours).

• Check:

• Verify that you are specifying the expiration time correctly in the SDK you’re using (e.g., Boto3, AWS CLI, etc.).

• Ensure that the expiration time is not inadvertently overridden by an environment variable or a different function in your pipeline.

IAM Role Permissions:

• Impact: The IAM role that generates the presigned URL must have the correct permissions for s3:GetObject, s3:PutObject, and other relevant actions. If permissions are restrictive, they can interfere with the proper functioning of the URL.

• Fix: Check the IAM role attached to CodeBuild to ensure that it has sufficient permissions for the S3 operations.

• Check:

• Inspect the IAM policy for permissions related to S3 and the objects you’re trying to access.

• Look for conditions in the policy that might limit access based on s3:RequestObjectTag, aws:RequestTag, or other parameters.

S3 Bucket Policy or Object Lifecycle Rules:

• What happens? Sometimes S3 bucket policies or lifecycle rules can affect object accessibility and URL validity. For example, an object could be transitioned to a different storage class (such as Glacier) or deleted after a certain time frame.

• Fix: Review the bucket’s lifecycle policies and object transition rules to ensure they aren’t interfering with the object’s availability during the URL's lifetime.

• Check:

• Ensure that no conflicting lifecycle rules are configured for the bucket or the specific objects the presigned URL is referencing.

• Review the bucket’s access policy to confirm that there are no conditions preventing long-lived presigned URLs from being used.

CodeBuild Environment Configuration:

• Potential impact: The environment where the presigned URLs are generated (such as CodeBuild) could have settings or limitations that affect the presigned URL generation process. Custom Docker images or specific runtime environments may introduce edge cases.

• Fix: If you are using a custom Docker image in CodeBuild, verify that the system libraries and environment variables are correctly configured to support AWS SDK operations and time synchronization.

• Check:

• Review the CodeBuild logs to see if there are any warnings or errors when generating the URL.

• Validate the environment variables used during the build stage, especially any that affect URL generation.

Session Token Expiration (AssumeRole):

• What it means: If your CodeBuild environment is assuming an IAM role (via sts:AssumeRole), the session token associated with that role may expire before the presigned URL. This can invalidate the URL prematurely.

• Fix: Ensure that the session token or role credentials have a valid session duration matching or exceeding the expiration time of the presigned URL.

• Check:

• Review the role session expiration settings to confirm that they align with the presigned URL’s lifetime.

• Extend the session duration in the sts:AssumeRole API call if needed.

Practical Tips for Troubleshooting

1. Generate URL Locally: Try generating the presigned URL from a local machine or an EC2 instance to see if the issue is tied specifically to the CodeBuild environment. If the URL behaves as expected locally but fails in CodeBuild, the issue may lie with the build environment.

2. Increase Logging in CodeBuild: Add logging to the CodeBuild stage to capture the exact expiration time being set and any anomalies during URL generation. This can help pinpoint where the discrepancy occurs.

3. Test with Shorter Expiry: Temporarily set a shorter expiration time (e.g., 1 hour) and see if the URL expires as expected. This helps isolate whether the issue is with time calculation, environmental factors, or something else.

4. Use AWS CLI to Debug: Run the AWS CLI in the CodeBuild environment to manually generate a presigned URL and test if it behaves differently when generated directly via CLI versus in code. For example:

aws s3 presign s3://bucketname/objectname --expires-in 86400

5. Custom Error Handling: Implement error handling in the client application to gracefully handle cases where the presigned URL expires earlier than expected. For example, catch 403 errors (Access Denied) and regenerate the URL if needed.