1 Answer
- Newest
- Most votes
- Most comments
1
Ok I figured out the problem... Turns out that Python was buffering print statements inside of child processes (I am using parallel processing using the multiprocessing module), instead of flushed directly. This behavior of Python is not very obvious, since it does not buffer the print statements when running in a tty.
The fix was to simply supply a flag (-u) to the python command. E.g. python3 -u script.py
In case anyone is interested, more info can be found here: https://stackoverflow.com/questions/107705/disable-output-buffering
answered 4 years ago
Relevant content
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 3 months ago
- AWS OFFICIALUpdated 10 months ago
I can verify that
python -u script.py
inside my fargate container enabled my batch job to log the output ofprint()
statements inside script.py