How to set up / access the Spark UI for AWS Glue jobs and notebooks


The checkbox in a Glue job is select to enable the Spark UI, and an s3 path is specified as the location for those logs. How can I access the Spark UI URL?

profile picture
asked a year ago956 views
1 Answer
Accepted Answer

You need to deploy you own history server to visualize those longs, you can use a CloudFormation template, a docker container (you can also run locally a history server from a Spark distribution but adding s3 support requires a bit of work). See steps here:

profile pictureAWS
answered a year ago
profile picture
reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions