1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
Given the limitations of S3 Select, I would suggest that you use Athena/Glue to CREATE TABLE - This can be done using boto3 Glue methods if the schema is know or using a crawler as shown below.
import boto3
# Create an AWS Glue client
glue_client = boto3.client('glue')
# Define the crawler settings
crawler_name = 'my-crawler'
s3_target_path = 's3://my-bucket/my-folder/'
database_name = 'my-database'
table_prefix = 'my-table-prefix'
# Create the database
glue_client.create_database(
DatabaseInput={
'Name': database_name
}
)
# Create the crawler
glue_client.create_crawler(
Name=crawler_name,
Role='arn:aws:iam::123456789012:role/service-role/AWSGlueServiceRoleDefault',
DatabaseName=database_name,
Targets={
'S3Targets': [
{
'Path': s3_target_path
}
]
},
SchemaChangePolicy={
'UpdateBehavior': 'UPDATE_IN_DATABASE',
'DeleteBehavior': 'LOG'
},
TablePrefix=table_prefix
)
# Start the crawler
glue_client.start_crawler(Name=crawler_name)
답변함 일 년 전