Trying below yml script to transfer AWS resource (amazon S3), using cloud-formation from one environment to another.
- The source environment has list of directories (Folder). for example: <BucketName>/RawZone/BatchData/Delta
- The bucket directory also enabled SNS notification.
Challenge:
- How to create default list of directories using cloud formation.
- Below script fail, because expected list of directories not found at target AWS account, to enable the notification.
Please note, to simplicity code the respective SQS and SNS resource, I have already migrate, so I remove it from below script.
AWSTemplateFormatVersion: "2010-09-09"
Description: AWS CloudFormation which creates the resources on aws for data lake
Parameters for the cloudformation template
Parameters:
S3BucketName:
Type: String
Description: Bucket name for the aws data lake (Bucket name shoule be in small characters)
AllowedPattern: "[a-zA-Z][a-zA-Z0-9_-]"
Resources:
S3Bucket:
Type: "AWS::S3::Bucket"
DeletionPolicy: Retain
Properties:
BucketName: !Ref S3BucketName
BucketEncryption:
ServerSideEncryptionConfiguration:
-
ServerSideEncryptionByDefault:
SSEAlgorithm: "AES256"
BucketKeyEnabled: true
NotificationConfiguration:
TopicConfigurations:
-
Event: "s3:ObjectCreated:"
Filter:
S3Key:
Rules:
-
Name: "Prefix"
Value: "RawZone/BatchData/Delta/"
-
Name: "Suffix"
Value: ""
Topic: !Sub "arn:aws:sns:${AWS::Region}:${AWS::AccountId}:${S3BucketName}_event_notifications_topic"
VersioningConfiguration:
Status: "Enabled"
OwnershipControls:
Rules:
-
ObjectOwnership: "BucketOwnerEnforced"
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true