[Pandas] [Glue] ResolveChoice cannot cast to JSON

Question

Hi,
I'm trying to cast a dataframe that contains JSON strong to a json format to write it to DB. I'm using an example from here https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-crawler-pyspark-transforms-ResolveChoice.html

But the following cast to JSON is not working
```
df_events.resolveChoice(specs = [('detail','cast:json')])
```
and I get the following error 
> IllegalArgumentException: 'Invalid type name json'

Any suggestion how to cast to JSON?
it's required to write a Data Frame to PostgresSQL

Answer

You should be able to query this data with Athena. When you query this table you can use the JSON functions to query the JSON columns, for example:
```
SELECT json_extract_scalar(attributes, '$.tag1') ...
```

glue : https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-common.html

athena : https://docs.aws.amazon.com/athena/latest/ug/data-types.html

[Pandas] [Glue] ResolveChoice cannot cast to JSON

相關內容