Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Streaming limitations of API Gateway REST APIs for long-running MCP workloads

0

TL;DR For long-running, streaming MCP workloads, Lambda Function URLs currently appear more reliable than API Gateway REST APIs due to timeout and response size constraints.

I’m implementing an MCP (Model Context Protocol) server on AWS and evaluating whether to expose it via Lambda Function URLs or API Gateway REST APIs, specifically for long-running streaming workloads.

After reviewing the current behavior and documentation, here’s a summary of my findings.

Observed differences

Lambda Function URLs

Pros

Native Node.js streaming behavior Lower latency Minimal infrastructure setup Reliable support for incremental MCP tool results Better suited for long-running streaming sessions

Cons

Simpler routing and stage management Authorization must be handled manually No built-in caching

API Gateway REST APIs (with Lambda streaming)

Pros

Standard AWS stack Built-in routing and stage management Native IAM / authorizer support Less custom Lambda adapter code

Cons

Streaming constraints, including: Idle timeout behavior for long-lived connections Response size / throughput limits observed in practice Additional latency Incremental streaming for long-running tools is more complex and fragile

Question

For MCP servers that rely on long-running tools (e.g., tools that stream incremental results or notifications over extended periods), Lambda Function URLs currently appear to avoid several structural limitations of API Gateway REST APIs.

Is this an accurate assessment of the current streaming limitations?

Are there recommended patterns—or upcoming improvements—that would make API Gateway REST APIs a better fit for long-running streaming workloads?

Topics
ServerlessFront-End Web & MobileMachine Learning & AINetworking & Content DeliveryDeveloper ToolsGenerative AI on AWS
Tags
Amazon API GatewayServerlessAWS MCP Server
Language
English

asked 8 months ago590 views

1 Answer
1

Your assessment is largely accurate based on the current capabilities of API Gateway REST APIs with response streaming.

API Gateway REST APIs do have specific constraints for streaming workloads that are important to consider:

Streaming Duration and Timeout Limits: API Gateway supports streaming responses for up to 15 minutes maximum. Additionally, streams are subject to idle timeouts—5 minutes for Regional or private endpoints, and 30 seconds for edge-optimized endpoints. If your MCP workloads involve long-running tools that may have periods of inactivity or need to maintain connections beyond these timeframes, these timeouts could interrupt your streams.

Throughput Constraints: While the first 10MB of response payload streams without bandwidth restrictions, any data exceeding 10MB is throttled to 2MB/s. For MCP servers streaming incremental results over extended periods, this could impact performance depending on your data volume.

Connection Behavior: When connections close due to timeout (either between client and API Gateway, or between API Gateway and Lambda), the Lambda function may continue executing. This could lead to resource inefficiency or unexpected behavior in long-running scenarios.

Unsupported Features: Response streaming doesn't support endpoint caching, content encoding (compression must be handled in your integration), or response transformation with VTL. These limitations add complexity if you need these capabilities.

For long-running MCP workloads with extended streaming sessions, Lambda Function URLs would indeed avoid these structural constraints, offering more flexibility for duration and idle behavior. API Gateway REST APIs with streaming are better suited for scenarios where responses complete within 15 minutes and don't have extended idle periods, such as generative AI responses, large file transfers, or operations that stream continuously without long pauses.

There are no publicly announced upcoming improvements to these specific limitations at this time.
Sources
Building responsive APIs with Amazon API Gateway response streaming | AWS Compute Blog
Stream the integration response for your proxy integrations in API Gateway - Amazon API Gateway

answered 8 months ago

EXPERT

reviewed 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content