How to e-mail OpenSearch reports and dashboards via opensearch-reporting-cli

Introduction

You can use OpenSearch Dashboards to create PNG, PDF, and CSV reports on demand or on a schedule. Customers have been looking for a way to send those reports to e-mail addresses in an automated way. This article shows you how to use a Lambda function to download reports from OpenSearch Dashboards and e-mail them via Amazon Simple Email Service (SES).

Prerequisites

• AWS Command Line Interface CLI
• Docker
• AWS Elastic Container Registry (ECR)
• Amazon Simple Email Service (SES)

Solution

Create and push docker image to a private repository

For this section I'm going to use AWS CloudShell but you can do these steps through your local machine or an Amazon EC2 instance as well.

1. Login to your AWS account.

2. Open CloudShell.

   • Click on the CloudShell icon on the top of your AWS console
   • 

3. Update the installed packages and package cache on your instance. Run:

   sudo yum update -y

4. Install the most recent Docker Community Edition package. Run:

   sudo yum install docker

5. Create a new file on your local machine and paste the following code in it:

# Define function directory
ARG FUNCTION_DIR="/function"

# Base image of the docker container
FROM node:lts-slim as build-image

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# AWS Lambda runtime dependencies
RUN apt-get update && \
    apt-get install -y \
        g++ \
        make \
        unzip \
        libcurl4-openssl-dev \
        autoconf \
        automake \
        libtool \
        cmake \
        python3 \
        libkrb5-dev \
        curl

# Copy function code
WORKDIR ${FUNCTION_DIR}
RUN npm install @opensearch-project/reporting-cli && npm install aws-lambda-ric

# Build Stage 2: Copy Build Stage 1 files in to Stage 2. Install chrome, then remove chrome to keep the dependencies.
FROM node:lts-slim
# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}
# Copy in the build image dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}

# Install latest chrome dev package and fonts to support major char sets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
# Note: this installs the necessary libs to make the bundled version of Chromium that Puppeteer installs, work.
RUN apt-get update \
    && apt-get install -y wget gnupg \
    && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
      --no-install-recommends \
    && apt-get remove -y google-chrome-stable \
    && rm -rf /var/lib/apt/lists/*

ENTRYPOINT ["/usr/local/bin/npx", "aws-lambda-ric"]

ENV HOME="/tmp"
CMD [ "/function/node_modules/@opensearch-project/reporting-cli/src/index.handler" ]

NOTE: The docker file above downloads the artifacts required to use the OpenSearch Reporting CLI.

6. Save the file as "Dockerfile".

7. Upload the Dockerfile to CloudShell.

   1. Navigate back to CloudShell in your browser. Click on the Actions dropdown → Upload File
   2. 

8. Create a Amazon ECR private Repository by running the following command in CloudShell

   aws ecr create-repository --repository-name opensearch-reporting-cli

9. Run push commands for your repository.

   1. Search for ECR in AWS console and click on Elastic Container Registry.

      • 

   2. Click on the opensearch-reporting-cli repository

      • 

   3. Click on View push commands

      • 

   4. Run the 4 push commands listed in Cloudshell.

      • 

Verify email address in SES

1. Search for and navigate to Amazon Simple Email Service

   1. 

2. Click on Identities and click on Create identity

   1. 

   1.

3. Select Email address → type in the sender’s email address and click create identity. Repeat for recipients address.

   1. This will send a verification email to that address with a link to click to verify.
   2. Once verified you can move to the next step.

Create Lambda function

1. Navigate to Lambda from the AWS console search bar.
2. Create function → from Container Image→ for container image URI, Browse Images, select your repository and your image. Select x86_64 for architecture, and Create function.
3. Go to Configuration → General configuration → Edit
   1. Increase memory to 1024MB or more
   2. Increase timeout to 5 Mins or more.
4. Go to permissions → click on Execution role name to add permissions to send email via SES:
   1. Add permissions → Create inline policy

{
        "Effect": "Allow",
        "Action": 
            [
             "ses:SendEmail",
             "ses:SendRawEmail"
            ],
        "Resource": "*"
 }

5. Test lambda function with a public dashboard url (no auth needed):

{
     "url": "https://playground.opensearch.org/app/visualize#/edit/104396f0-22a4-11ee-b88b-47a93b5c527c",
     "transport": "ses",
     "from": "sender@email.com",
     "to": "recepient@email.com",
     "subject": "Test lambda docker image"
}

You should now get an e-mail sent to the recipient e-mail address in a few minutes with the attached report.

Summary

This article showed you how you can email dashboards and reports in OpenSearch Dashboards on demand or on a schedule. You can add additional steps to trigger the Lambda function based on an event or on a schedule. Let us know what you think in the comments below and have fun!

Additional resources: https://opensearch.org/docs/latest/reporting/rep-cli-lambda/