Creating a DynamoDB Table with CloudFormation and Adding Items at Creation Time

4 minuto de leitura
Nível de conteúdo: Intermediário
1

How to create a DynamoDB table using CloudFormation while adding items to it at creation time.

Amazon DynamoDB is a fully managed NoSQL database service provided by AWS, known for its scalability, low latency, and seamless integration with other AWS services. CloudFormation, AWS's infrastructure-as-code service, allows you to define and provision resources in a declarative manner. In this article, we'll explore how to create a DynamoDB table using CloudFormation and add items to it at the time of creation.

Prerequisites

Before we begin, make sure you have the following prerequisites in place:

  • An AWS account with appropriate permissions to create resources using CloudFormation.
  • Familiarity with AWS CloudFormation and basic knowledge of AWS services, including DynamoDB and Lambda.
  • An understanding of YAML or JSON, as CloudFormation templates can be written in either format.

Step 1: Writing the CloudFormation Template

Let's start by creating the CloudFormation template to define our DynamoDB table and a Lambda function that will add items to the table. The CloudFormation template is written in YAML format for simplicity.

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  MyDynamoDBTable:
    Type: 'AWS::DynamoDB::Table'
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: pk
          AttributeType: S
        - AttributeName: sk
          AttributeType: S
      KeySchema:
        - AttributeName: pk
          KeyType: HASH
        - AttributeName: sk
          KeyType: RANGE
  MyCustomResourceLambdaFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      Runtime: nodejs14.x
      Handler: index.handler
      Role: !GetAtt MyLambdaExecutionRole.Arn
      Environment:
        Variables:
          tableName: !Ref MyDynamoDBTable
      Code:
        ZipFile: |
          const AWS = require('aws-sdk');
          const response = require('cfn-response');
          const client = new AWS.DynamoDB();
          const dynamodb = new AWS.DynamoDB.DocumentClient();
     
          exports.handler = async (event, context) => {
            try {
              const tableName = process.env.tableName
              console.log(tableName)
              var params = {
                'TableName': tableName
              };
            
              client.waitFor('tableExists', params, function(err, data) {
                if (err) console.log(err, err.stack); // an error occurred
                else     console.log(data);           // successful response
              });

              const itemsToAdd = [
                { pk: 'item1', sk: 'sortKey1', otherAttribute: 'value1' },
                { pk: 'item2', sk: 'sortKey2', otherAttribute: 'value2' },
                { pk: 'item3', sk: 'sortKey3', otherAttribute: 'value3' },
                { pk: 'item4', sk: 'sortKey4', otherAttribute: 'value4' },
                { pk: 'item5', sk: 'sortKey5', otherAttribute: 'value5' },
              ];
              
              const putItemPromises = itemsToAdd.map((item) => {
                const params = {
                  TableName: tableName,
                  Item: item,
                };
                return dynamodb.put(params).promise();
              });
              
              await Promise.all(putItemPromises).then(res=>console.log(res)).catch(err=>console.log(err))
              
              const responseData = { Result: 'Items added successfully' };
              await response.send(event, context, response.SUCCESS, responseData);
            } catch (error) {
              console.log(error)
              const responseData = { Error: 'Something went wrong' };
              await response.send(event, context, response.FAILED, responseData);
            }
          };
      Timeout: 30
  MyLambdaExecutionRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
        - 'arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess'
  MyCustomResource:
    Type: 'Custom::MyCustomResource'
    Properties:
      ServiceToken: !GetAtt MyCustomResourceLambdaFunction.Arn

Step 2: Understanding the CloudFormation Template

In this CloudFormation template, we define two main resources:

  • MyDynamoDBTable: This resource creates the DynamoDB table with the required schema. It sets the billing mode to PAY_PER_REQUEST, which means that you pay only for the read and write capacity that your application consumes.
  • MyCustomResourceLambdaFunction: This resource creates a custom Lambda function backed by a CloudFormation Custom Resource. The Lambda function is responsible for adding five sample items to the DynamoDB table. We use waitFor('tableExists') to ensure that the Lambda function waits until the DynamoDB table is fully created or updated before adding the items.

Step 3: Deploying the CloudFormation Stack

To deploy the CloudFormation stack, follow these steps:

  1. Save the CloudFormation template in a YAML file (e.g., dynamodb-stack.yaml).
  2. Open the AWS Management Console, navigate to the CloudFormation service, and click on "Create stack."
  3. Select "Upload a template file," and then upload the dynamodb-stack.yaml file you saved in the previous step.
  4. Provide a stack name and any additional parameters required by your template.
  5. Review the configuration and click "Create stack" to initiate the deployment.

Using the CLI:

aws cloudformation create-stack \
--stack-name <stack-name> \
--template-body file://myfile.yaml \
--capabilities CAPABILITY_NAMED_IAM

Step 4: Observing the Result

After the CloudFormation stack is created, you can navigate to the DynamoDB console and find the table which has a name that begins with the name of your CloudFormation stack. You will notice that the table has been created and contains five items with the specified partition key pk and sort key sk.

Conclusion

In this article, we explored how to use AWS CloudFormation to create a DynamoDB table with a custom schema and provision sample items at the time of table creation. By leveraging the power of CloudFormation and custom Lambda resources, you can seamlessly manage your AWS infrastructure and automate the process of adding initial data to your DynamoDB tables. This approach ensures that your DynamoDB tables are always initialized with the required data, providing a solid foundation for your applications to build upon.

profile pictureAWS
ESPECIALISTA
publicada há um ano7234 visualizações
5 Comentários

Hello Lee,

It works but actually after stack creation , during creation of MyCustomResource it was taking so much of time as it is not getting the response back and then create for custom resource failed with the reason "CloudFormation did not receive a response from your Custom Resource" and then due to rollback everything got deleted. Could you please suggest something and help me to resolve this issue as well ?

Thanks

respondeu há um ano

Thanks for the update, there was an await missing for the response.send() request.

profile pictureAWS
ESPECIALISTA
respondeu há um ano

Ingestion of data during the table creation is a classic example, but by doing it via Lambda, we are keeping an one-off Lambda in our Account, We need to write another script to decommission the Lambda, if decision is to leave the Lambda, we need to take care, if some one accidentally runs lambda , don't ingest duplicate data into Dynamo Table. My opinion, from where the CLI for Cloud-formation is being run, excute node.js code (or some other tech) to ingest data into DynamoDB table

Pavan
respondeu há um ano

DynamoDB putitem creates a new item, or replaces an old item (having same primary key) with a new item. So, even if the lambda function is executed multiple times, it would not result in any data duplication. But, to avoid unnecessary code execution, we can have conditional check on event.RequestType and below are the details-

Conditional check on event.RequestType - When the lambda function is being invoked through Custom Resource, event.RequestType would represent the Cloud Formation Stack operation (i.e. "Create", "Update" and "Delete" ) So, it would be best practice to have conditional checks on event.RequestType to execute the appropriate functionality based on Stack operation. In this case, conditional check on "event.RequestType" as "Create" would avoid unnecessary execution of updating data into DynamoDB table (unless if someone intentionally specify event.RequestType as "Create" while executing Lambda function manually). Also, it would be better to handle appropriately in your lambda function code for other event.RequestType operations (i.e. Update and Delete)

respondeu há 3 meses

Alternative approach for waitFor('tableExists') - The alternative approach for "waitFor('tableExists') " is to specify "DependsOn: MyDynamoDBTable" for the Custom Resource "MyCustomResource", so that the Lambda function is invoked only after the DynamoDB table is created during Stack creation.

Thanks a lot for detailing out on how to use Custom Resources in Cloud Formation templates

respondeu há 3 meses