Storing output of AWS Batch job in file

Hi JRK - I would recommend saving your output data to s3, then you can download them at will at any time in the future. There are multiple storage options (read: cheaper), and lifecycle options so you auto-delete them after X amount of days.

If you are new to AWS here are some basic steps:

1. create s3 bucket. https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html

2. If you don't have an IAM role already for your containers, this link shows you how to do so. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html 2a) if you already have a role for your containers, add permissions to the role to enable them to access your s3 bucket https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_s3_rw-bucket.html Note the example in 2a is a bit permissive for many company environments as it gives * access. You can start with "PutObject", that's the bare minimum to upload data to s3.

3. The next step is to gather your data and upload it, this will depend on your scripting/programming of your job. It could be as simple as "aws s3 cp filename.txt s3://mybucket/folder/" if you use the CLI Documentation link to upload to s3: https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html CLI installers https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

4. Once uploaded you can go to the console and browse the data, use the CLI (on your laptop) to view (use "aws s3 ls s3://mybucket"), or SDK if you desire.

Notes: a) for low quantity text data s3 will be very inexpensive, you might even be in the free tier. Standard storage, or intelligent tiering are good options when uploading https://aws.amazon.com/s3/storage-classes/ b) Lifecycle rules can help you manage your data, say autodelete after 90 days, or change to a different (cheaper) storage tier. https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html