Recommendation to use multiple or single AWS VPC

Question

**Project Plan Overview**
We have a plan to extract data from around 60 different data sources and store it in an S3 bucket. So far, we’ve implemented the solution for 2 data sources.

**Initial Implementation Details**
VPCs Setup: We created 4 different VPCs—2 for production and 2 for non-production environments—to handle the 2 data sources.
**Application Hosting**: We hosted 2 different applications to extract data from these sources.

**Suggestions for Future Implementation**
**Scaling Concern**s: As we add more data sources, the number of VPCs required may increase, leading to complexity and potential management overhead.
**Using a Single VPC**: We could consider using one VPC for all data sources, with different subnets allocated to each source. This could simplify our network architecture and reduce the need for multiple VPCs.
**Internet Gateway Concern:** If we choose to use a single VPC, there will be one internet gateway for all traffic. The concern is whether a single internet gateway can efficiently handle the traffic from all data sources.

Would we like to explore any of these suggestions further , any other suggestions are welcome?

Brettski-AWS · Answer

Adding to the other answer(s) here:

Definitely go with multiple VPCs. It will make your networking much easier later. Make sure you're deploying the VPCs using some sort of infrastructure-as-code (such as CloudFormation).

If you did go with a single VPC, Internet Gateway is a horizontally scalable, virtual service. You don't need to worry about bandwidth. It isn't a router or any other networking device in the traditional sense.

I'd also recommend that you use multiple accounts - one for production, one non-production. That way it is far simpler for you to manage permissions. Yes, you can do tag-based access control for everything in a single account but one tiny slip will see production resources accidentally changed or deleted. It's much harder to do that when resources are in a different account that you have to log into.

NARRAVULA MUNISAI TEJA · Answer

Hey Hiii...!

>  **Multiple VPCs:** Isolated environments but increased overhead.

> **Single VPC:** Simpler management but potential performance and security risks.

> **Hybrid:** Combine both for flexibility.

**My choice is Hybrid Approach:**

* Start with a single VPC: This can streamline the initial setup and management for the first few data sources.
* Implement subnets: Divide the VPC into subnets based on data source groups, environments (dev, test, prod), or other logical criteria.
* Monitor and evaluate: Continuously assess the performance and resource utilization of the single VPC.
* Consider additional VPCs: If specific requirements for isolation, security, or performance necessitate it, create additional VPCs for specific data sources or groups.

Steve_M · Answer

Along with these other good responses, another consideration is the best use of S3 endpoints to transfer data between S3 and your data sources https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html

As the number of VPCs grows, which sounds likely from how your question describe your future plans, it may start to make sense to use a centralised VPC endpoint architecture, where the interface endpoints are all managed in a central hub VPC for accessing the service from multiple spoke VPCs https://aws.amazon.com/blogs/networking-and-content-delivery/centralize-access-using-vpc-interface-endpoints/

Recommendation to use multiple or single AWS VPC

Relevant content