By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

AWS Glue Spark UI Parsing error

0

Hi Experts, I have been experiencing issues with my AWS Glue Pyspark Job and hence enabled SPARK UI logging feature. However, when the job completed, I clicked on the Spark UI tab on Job run and it is failing with the below error information. I am not sure how to take it forward from here. Please share your inputs if any.

We encountered an issue trying to parse your Spark logs. Spark UI log parsing has failed for an unknown reason. Error details ErrorType: RequestLogParsingStatus ErrorCode: N/A RequestID: N/A StatusCode: FileLocation: <s2 3 path details>>

Topics
Analytics
Tags
AWS Glue
Language
English
AWS-User-8821482
asked a month ago148 views
1 Answer
0

Hello. I've recorded this 6 minutes video showing how to get Spark UI working.

In the video I cover all the steps needed including:

  • VPC setup
  • S3 buckets creation
  • Glue connection creation (to ensure no public traffic is used)
  • Glue job creation (and association with Glue connection)
  • Spark UI usage

Let me know if you want to access the recording.

Please accept this answer. Otherwise, reply this question with more information about your glue job for us to troubleshoot.

References: https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-jobs.html https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui.html

Regards, Pablo Silva

profile pictureAWS
Pablo Silva
answered a month ago
profile pictureAWS
EXPERT
Didier_Durand
reviewed a month ago
  • AWS-User-8821482
    a month ago

    @Pablo Silva - Could you please share the video. I see the logs have been created in S3 bucket, however, its failing in the parsing step.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content