First 3 logs show and then cloudwatch stops showing logs

0

Hi,
I am using AWS Batch to run a task which essentially consists of a Python script, which uses "print" statements to log messages. The first three of these (which are fired right after the script starts executing) show up in Cloudwatch. Almost immediately after these three print statements, there should be a fourth, which never shows up in Cloudwatch. Nor do any subsequent print statements. The script is still executing, so the logs should show up, but don't...

Any help is very very much appreciated. Thanks!

asked 4 years ago885 views
1 Answer
1

Ok I figured out the problem... Turns out that Python was buffering print statements inside of child processes (I am using parallel processing using the multiprocessing module), instead of flushed directly. This behavior of Python is not very obvious, since it does not buffer the print statements when running in a tty.

The fix was to simply supply a flag (-u) to the python command. E.g. python3 -u script.py

In case anyone is interested, more info can be found here: https://stackoverflow.com/questions/107705/disable-output-buffering

answered 4 years ago
  • I can verify that python -u script.py inside my fargate container enabled my batch job to log the output of print() statements inside script.py

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions