How do I use API Gateway as a proxy for another AWS service?

11 minute read

I want to use Amazon API Gateway as a proxy for another AWS service and integrate other services with API Gateway.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshoot AWS CLI errors. Also, make sure that you use the most recent AWS CLI version.

AWS service APIs are REST APIs that you can make an HTTPS request to. To integrate another service with API Gateway, build an HTTPS request from API Gateway to the service API. When you use this method, all request parameters are correctly mapped.

This resolution describes an example setup when you integrate with the Amazon Simple Notification Service (Amazon SNS) Publish API with API Gateway. Use the following example as an outline for methods to use when you integrate API Gateway with other services.

Set up the required tools and resources

Configure your environment and create all the AWS resources required for your use case. For the Amazon SNS example setup, complete the following steps:

Install the AWS CLI.
Create an Amazon SNS topic. Note the topic's Amazon Resource Name (ARN).
Create a subscription to the topic.
Open the AWS Identity and Access Management (IAM) console and then create an AWS service proxy execution role. For the Amazon SNS example setup, allow the sns:Publish action. For more information, see Control and manage access to REST APIs in API Gateway.

To create an API Gateway REST API, use a test resource. For more information and examples, see Amazon API Gateway tutorials and workshops.

Note: To import the REST API, use the following example OpenAPI 2.0 (Swagger) definition. For more information, see What is OpenAPI? on the Swagger website.

Example definition:

{
  "swagger": "2.0",
  "info": {
    "version": "2019-10-09T14:10:24Z",
    "title": "aws-service-integration"
  },
  "basePath": "/dev",
  "schemes": [
    "https"
  ],
  "paths": {
    "/test": {
      "post": {
        "produces": [
          "application/json"
        ],
        "parameters": [
          {
            "name": "Message",
            "in": "query",
            "required": true,
            "type": "string"
          },
          {
            "name": "TopicArn",
            "in": "query",
            "required": true,
            "type": "string"
          }
        ],
        "responses": {
          "200": {
            "description": "200 response",
            "schema": {
              "$ref": "#/definitions/Empty"
            }
          }
        },
        "x-amazon-apigateway-integration": {
          "credentials": "arn:aws:iam::account-id:role/apigateway-sns-role",
          "uri": "arn:aws:apigateway:region:sns:action/Publish",
          "responses": {
            "default": {
              "statusCode": "200"
            }
          },
          "requestParameters": {
            "integration.request.header.Content-Type": "'application/x-www-form-urlencoded'"
          },
          "requestTemplates": {
            "application/json": {
              "Fn::Sub": "Action=Publish&TopicArn=$util.urlEncode('${**<TOPIC-ARN>**}')&Message=$util.urlEncode($input.body)##"
            }
          },
          "passthroughBehavior": "when_no_match",
          "httpMethod": "POST",
          "type": "aws"
        }
      }
    }
  },
  "definitions": {
    "Empty": {
      "type": "object",
      "title": "Empty Schema"
    }
  }
}

The preceding option preconfigures the settings for the Amazon SNS example setup. Make sure to replace arn:aws:iam::account-id:role/apigateway-sns-role with your IAM role's ARN. To create your REST API, replace region with the AWS Region that you want.

Get an example HTTPS request

An example HTTPS request from the service API that you integrate with can help you map the request parameters in API Gateway.

For the Amazon SNS Publish API, see the service's API reference for an example request.

To get an example HTTPS request, run the following example request:

https://sns.us-west-2.amazonaws.com/?Action=Publish&TargetArn=arn%3Aaws%3Asns%3Aus-west-2%3A803981987763%3Aendpoint%2FAPNS\_SANDBOX%2Fpushapp%2F98e9ced9-f136-3893-9d60-776547eafebb
&Message=%7B%22default%22%3A%22This+is+the+default+Message%22%2C%22APNS\_SANDBOX%22%3A%22%7B+%5C%22aps%5C%22+%3A+%7B+%5C%22alert%5C%22+%3A+%5C%22You+have+got+email.%5C%22%2C+%5C%22badge%5C%22+%3A+9%2C%5C%22sound%5C%22+%3A%5C%22default%5C%22%7D%7D%22%7D
&Version=2010-03-31
&AUTHPARAMS

- or -

Generate the example from an API call. Use the AWS CLI to call the service API, and then analyze the output. Determine the associated AWS CLI command for the service API that you integrate with, and then run a test request with the --debug option. For more information, see AWS CLI Command Reference.

To generate an example from an API call, run the following AWS CLI publish command:

$ aws sns publish --topic-arn arn:aws:sns:us-east-1:123456789012:test --message "hi" --debug

Note: Replace arn:aws:sns:us-east-1:123456789012:test with your Amazon SNS topic's ARN.

The command output contains the HTTPS request and the headers that are passed. The following command output is an example of the HTTPS request:

2018-11-22 11:56:39,647 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: sns2018-11-22 11:56:39,648 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.sns.Publish: calling handler <function generate\_idempotent\_uuid at 0x11093d320>
2018-11-22 11:56:39,649 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=Publish) (verify\_ssl=True) with params: {'body': {'Action': u'Publish', u'Message': u'hello', 'Version': u'2010-03-31', u'TopicArn': u'arn:aws:sns:us-east-1:123456789012:test'}, 'url': u'https://sns.us-east-1.amazonaws.com/', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'aws-cli/1.15.74 Python/2.7.14 Darwin/16.7.0 botocore/1.9.23'}, 'context': {'auth\_type': None, 'client\_region': 'us-east-1', 'has\_streaming\_input': False, 'client\_config': <botocore.config.Config object at 0x1118437d0>}, 'query\_string': '', 'url\_path': '/', 'method': u'POST'}
2018-11-22 11:56:39,650 - MainThread - botocore.hooks - DEBUG - Event request-created.sns.Publish: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x111843750>>
2018-11-22 11:56:39,650 - MainThread - botocore.hooks - DEBUG - Event choose-signer.sns.Publish: calling handler <function set\_operation\_specific\_signer at 0x11093d230>
2018-11-22 11:56:39,650 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2018-11-22 11:56:39,651 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
POST
/

content-type:application/x-www-form-urlencoded; charset=utf-8
host:sns.us-east-1.amazonaws.com
x-amz-date:20181122T062639Z
content-type;host;x-amz-date

In this example, the request is sent as a POST HTTP method.

Create a method for your API Gateway API

Complete the following steps:

Open the API Gateway console.
In the APIs pane, choose the name of your API.
In the Resources pane, choose a resource. For the Amazon SNS example setup, choose the test resource that you created.
Choose Create Method.
In the new window:

Choose the method that your service API uses in the example HTTPS request.

For the Amazon SNS example setup, choose POST.
For Integration type, choose AWS Service.
For AWS Region, choose the AWS Region of the resource associated with the service API that you integrate with.
For the Amazon SNS example setup, choose the Region of your SNS topic.
For AWS Service, choose the service that you integrate with API Gateway. For example, use the Simple Notification Service (SNS) service.
For AWS Subdomain, enter the subdomain that the AWS service uses. Check the service's documentation to confirm the availability of a subdomain.
For the Amazon SNS example setup, leave the field blank.
For HTTP method, choose the method that corresponds to the AWS service API that you integrate with.
For the Amazon SNS example setup, choose POST.
For Action Type, if the service API that you integrate with is a supported action, then choose Use action name. Check the service's API reference for a list of supported actions.
For Amazon SNS, see Actions.
For Action, enter the name of the service API. For the Amazon SNS example setup, enter Publish.
-or-
For Action Type, if the AWS service API expects a resource path in your request, then choose Use path override. For example, for the Amazon Polly ListLexicons API, enter /v1/lexicons for Path override (optional).
For Execution role, enter the ARN of the IAM role that you created.
(Optional) For Use Default Timeout, make changes as needed for your use case. For the Amazon SNS example setup, don't change the default timeout settings.
Choose Create Method.

Create parameters for the method request

Determine the required and optional request parameters for the service API that you integrate with. To identify these parameters, refer to the example HTTPS request that you got earlier, or refer to the API reference for the service API. For example, see Publish.

Complete the following steps:

Open the API Gateway console.
In the Method Execution pane for your API Gateway API's method, choose Method Request.
(Optional) To validate the query string parameters, navigate to the Method Request pane.
For Request Validator, choose a request validator, body, and headers.
Choose Edit.
Expand URL Query String Parameters.
Choose Add query string.
For Name, enter the name of a request parameter for the service API that you integrate with.
If the parameter is required, then select the check box under Required.

For the Amazon SNS example setup, create a parameter named TopicArn and another parameter named Message.

For more information, see Set up a method using the API Gateway console.

Note: For some service APIs, you must send required headers and a body in the integration request in addition to the required parameters. You can create the headers and body on the Integration Request pane under HTTP Request Headers and Request Body.

For example, if you integrate with the Amazon Rekognition ListCollections API, then create the header X-Amz-Target: RekognitionService.ListCollections.

The request appears similar to the following code example:

POST https://rekognition.us-west-2.amazonaws.com/ HTTP/1.1    Host: rekognition.us-west-2.amazonaws.com
    Accept-Encoding: identity
    Content-Length: 2
    X-Amz-Target: RekognitionService.ListCollections
    X-Amz-Date: 20170105T155800Z
    User-Agent: aws-cli/1.11.25 Python/2.7.9 Windows/8 botocore/1.4.82
    Content-Type: application/x-amz-json-1.1
    Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXX/20170105/us-west-2/rekognition/aws4_request,
      SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=XXXXXXXX

    {}

If you integrate with the Amazon Simple Queue Service (Amazon SQS) SendMessage API, then map the request body that uses the mapping expression method.request.body.JSONPath_EXPRESSION. (Replace JSONPath_EXPRESSION with a JSONPath expression for a JSON field of the body of the request.) In this example, a request appears similar to the following code example:

{'url_path': '/', 'query_string': '', 'method': 'POST','headers': {'Content-Type': 'application/x-www-form-urlencoded; 
charset=utf-8', 'User-Agent': 'aws-cli/1.16.81 Python/3.6.5 Darwin/18.7.0 botocore/1.12.183'}, 
'body': {'Action': 'SendMessage', 'Version': '2012-11-05', 'QueueUrl': 'https://sqs.ap-southeast-2.amazonaws.com/123456789012/test01', 'MessageBody': 'Hello'}, 
'url': 'https://ap-southeast-2.queue.amazonaws.com/', 'context': {'client_region': 'ap-southeast-2', 'client_config': <botocore.config.Config object at 0x106862da0>, 'has_streaming_input': False, 'auth_type': None}}

Create parameters for the integration request

Map the parameters that you created for the method request to parameters for the integration request.

Complete the following steps:

Open the API Gateway console.
Choose your API.
In the Method Execution pane for your API Gateway API's method, choose Integration Request.
In the Integration Request pane, expand URL Query String Parameters.
Choose Add query string.
For Name, enter the name of a request parameter for the service API that you integrate with.

Note: The name is case-sensitive and must appear exactly as the service API requires.
For Mapped from, enter method.request.querystring.param_name. Replace param_name with the name of the corresponding parameter that you created for the method request. For example, method.request.querystring.TopicArn.
Choose the check mark icon (Create).
Create parameters for the integration request that correspond to each parameter that you created for the method request.

Note: If you created required headers and a body for the method request, then map them to the integration request. To create required headers and a body for the method request, navigate to the Integration Request pane, under HTTP Headers and Mapping Templates.

For more information, see Set up an API integration request using the API Gateway console.

(Optional) Check your integration configuration

To verify that your integration setup is configured correctly, run the following AWS CLI get-integration command:

$ aws apigateway get-integration --rest-api-id 1234123412 --resource-id y9h6rt --http-method POST

For the Amazon SNS example setup, the output appears similar to the following code example:

{    "integrationResponses": {
        "200": {
            "responseTemplates": {
                "application/json": null
            },
            "statusCode": "200"
        }
    },
    "passthroughBehavior": "WHEN_NO_MATCH",
    "timeoutInMillis": 29000,
    "uri": "arn:aws:apigateway:us-east-2:sns:action/Publish",
    "httpMethod": "POST",
    "cacheNamespace": "y9h6rt",
    "credentials": "arn:aws:iam::1234567890:role/apigateway-sns-role",
    "type": "AWS",
    "requestParameters": {
        "integration.request.querystring.TopicArn": "method.request.querystring.TopicArn",
        "integration.request.querystring.Message": "method.request.querystring.Message"
    },
    "cacheKeyParameters": []
}

For larger message payloads, run the following Amazon SNS integration on your API:

"requestParameters": {
       "integration.request.header.Content-Type": "'application/x-www-form-urlencoded'"
},
"requestTemplates" : {
    "application/json" : "Action=Publish&TopicArn=$util.urlEncode('<ourTopicArn>')&Message=$util.urlEncode($input.body)"
}

Note: Amazon SNS supports the maximum header size of 16KB. Before you send the request to your integration, use a request mapping template to convert an incoming HTTP request into a different format. Because the Amazon SNS topic ARN is hardcoded into the mapping template, clients don't pass through the SNS topic ARN in their request. To check your integration configuration, complete the following steps:

Open the API Gateway console.
Choose your API.
Navigate to the Method Execution pane for your API Gateway API's method.
Choose TEST.
In the Method Test pane, complete the following steps:

For Query Strings integration, enter a query string that includes request parameters and values for the query strings.
For Amazon SNS integration, enter TopicArn= arn:aws:sns:us-east-1:123456789012:test&Message="Hello". Replace arn:aws:sns:us-east-1:123456789012:test with your Amazon SNS topic's ARN.
For Amazon SNS integration that uses the mapping template, enter message payload JSON data into the Request Body, and then choose Test.

Note: The payload varies depending on your configuration.
When you receive a successful response, deploy your REST API.
To test your API outside AWS console, use the invoke URL with any 3rd party tool.