【以下的问题经过翻译处理】 当从S3存储桶中查询数据时,我发现尝试从Athena查询时会出现问题。如果从SELECT
语句中排除时间戳列,仍然可以进行查询。有人能给出解决这个问题的建议吗?由于任务是从RDS迁移日志数据,更改日志记录中的access_at
字段可能很困难。
HIVE_BAD_DATA: Error parsing field value '2022-12-14T06:51:14.710Z' for field 13: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
表格式如下:
CREATE EXTERNAL TABLE `user_report`(
`type` int COMMENT 'from deserializer',
`system_id` int COMMENT 'from deserializer',
`id` string COMMENT 'from deserializer',
`company_id` int COMMENT 'from deserializer',
`user_id` int COMMENT 'from deserializer',
`token` char(255) COMMENT 'from deserializer',
`device_type` tinyint COMMENT 'from deserializer',
`app_version` char(255) COMMENT 'from deserializer',
`session_cnt` int COMMENT 'from deserializer',
`requested_cnt` int COMMENT 'from deserializer',
`scheduled_cnt` int COMMENT 'from deserializer',
`rescheduled_cnt` int COMMENT 'from deserializer',
`canceled_cnt` int COMMENT 'from deserializer',
`access_at` timestamp COMMENT 'from deserializer')
PARTITIONED BY (
`created_hour` string)
ROW FORMAT SERDE
'org.openx.data.jsonserde.JsonSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat'
LOCATION
's3://demo-kinesis-athena/'
TBLPROPERTIES (
'has_encrypted_data'='false',
'projection.created_hour.format'='yyyy/MM/dd/HH',
'projection.created_hour.interval'='1',
'projection.created_hour.interval.unit'='HOURS',
'projection.created_hour.range'='2018/01/01/00,NOW',
'projection.created_hour.type'='date',
'projection.enabled'='true',
'storage.location.template'='s3://demo-kinesis-athena/${created_hour}',
'transient_lastDdlTime'='1671014054')