Handling Json Data [AWS Glue]

0

Hi folks!

right now i'm working on project to do ETL Process using AWS Glue. I already create the query script to process the data, but my source data is in Json format. I need help to transform the json data into table form.

I need solution in Visual ETL Job

Note: right now i use s3 but need solution if using DynamoDB as the source DB

Ivan
gefragt vor 4 Monaten238 Aufrufe
1 Antwort
0

In a visual job, you can use an S3 source and specify JSON format, with an optional JsonPath, or due the same reading from a JSON table that you can build using a crawler.
Once the source reads the data as you need, then you can store it on DynamoDB to use as the source of other jobs.

profile pictureAWS
EXPERTE
beantwortet vor 4 Monaten
profile picture
EXPERTE
überprüft vor einem Monat
  • Thanks for your respon! but sorry i think i will give you my example flow if using table form (Structured data)

    S3 (csv) [Sources] -> Select fields [Transforms] -> SQL Query [Transforms] -> S3 (csv) [targets]

    but now the source data is using Json format, which is i need too transform it into table/structured data so i can do data transformation using my SQL Query.

    I'm new with AWS, usually i just use ETL tools like pentaho/talend. so i prefer use visual job than the script one.

  • the "source" transfoms the data into a structured in memory table, which you can see in the "Data preview" panel and from then transform, run SQL or whatever you need

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen