Handling Json Data [AWS Glue]

0

Hi folks!

right now i'm working on project to do ETL Process using AWS Glue. I already create the query script to process the data, but my source data is in Json format. I need help to transform the json data into table form.

I need solution in Visual ETL Job

Note: right now i use s3 but need solution if using DynamoDB as the source DB

Ivan
demandé il y a 4 mois237 vues
1 réponse
0

In a visual job, you can use an S3 source and specify JSON format, with an optional JsonPath, or due the same reading from a JSON table that you can build using a crawler.
Once the source reads the data as you need, then you can store it on DynamoDB to use as the source of other jobs.

profile pictureAWS
EXPERT
répondu il y a 4 mois
profile picture
EXPERT
vérifié il y a un mois
  • Thanks for your respon! but sorry i think i will give you my example flow if using table form (Structured data)

    S3 (csv) [Sources] -> Select fields [Transforms] -> SQL Query [Transforms] -> S3 (csv) [targets]

    but now the source data is using Json format, which is i need too transform it into table/structured data so i can do data transformation using my SQL Query.

    I'm new with AWS, usually i just use ETL tools like pentaho/talend. so i prefer use visual job than the script one.

  • the "source" transfoms the data into a structured in memory table, which you can see in the "Data preview" panel and from then transform, run SQL or whatever you need

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions