Creating the table does not always pull the logs from S3? https://repost.aws/knowledge-center/analyze-logs-athena

0

If creating a table does not work with the above provided schema, you can try using the schema below:

CREATE EXTERNAL TABLE mybucket_logs(

bucketowner string COMMENT '',

bucket_name string COMMENT '',

requestdatetime string COMMENT '',

remoteip string COMMENT '',

requester string COMMENT '',

requestid string COMMENT '',

operation string COMMENT '',

key string COMMENT '',

request_uri string COMMENT '',

httpstatus string COMMENT '',

errorcode string COMMENT '',

bytessent bigint COMMENT '',

objectsize bigint COMMENT '',

totaltime string COMMENT '',

turnaroundtime string COMMENT '',

referrer string COMMENT '',

useragent string COMMENT '',

versionid string COMMENT '',

hostid string COMMENT '',

sigv string COMMENT '',

ciphersuite string COMMENT '',

authtype string COMMENT '',

endpoint string COMMENT '',

tlsversion string COMMENT '')

ROW FORMAT SERDE

'org.apache.hadoop.hive.serde2.RegexSerDe'

WITH SERDEPROPERTIES (

'input.regex'='([^ ]) ([^ ]) \[(.?)\] ([^ ]) ([^ ]) ([^ ]) ([^ ]) ([^ ]) ("[^"]"|-) (-|[0-9]) ([^ ]) ([^ ]) ([^ ]) ([^ ]) ([^ ]) ([^ ]) ("[^"]"|-) ([^ ])(?: ([^ ]) ([^ ]) ([^ ]) ([^ ]) ([^ ]) ([^ ]))?.*$')

STORED AS INPUTFORMAT

'org.apache.hadoop.mapred.TextInputFormat'

OUTPUTFORMAT

'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'

LOCATION

's3://bucket-name/prefix-name/account-id/region/source-bucket-name/'

已提問 2 個月前檢視次數 71 次
沒有答案

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南