How do I connect to an Amazon RDS DB instance from an Amazon SageMaker notebook instance that's in a different VPC?

Resolution

To connect a SageMaker notebook instance to an Amazon RDS DB instance that's in a different VPC, complete these steps:

1. Create or delete a VPC peering connection.

2. After the VPC peering connection is active, update the route tables. The Amazon RDS DB instance subnet and the SageMaker notebook instance subnet must have a route to each other.

   For example, for the following CIDR blocks:

   SageMaker VPC CIDR block: 192.168.0.0/16
   Amazon RDS DB instance VPC CIDR block: 10.0.0.0/24

   The Amazon RDS DB instance subnet route table looks like this:

   Destination	Target
   10.0.0.0/24	local
   192.168.0.0/16	Select the VPC peering connection from the dropdown list

   The SageMaker notebook instance subnet route table looks like this:

   Destination	Target
   10.0.0.0/24	Select the VPC peering connection from the dropdown list
   192.168.0.0/16	local

    

3. To allow traffic from the notebook instance to the Amazon RDS DB instance, confirm that both security groups are configured correctly.
   On the notebook instance security group, confirm that there's an outbound rule that allows traffic to the Amazon RDS DB instance.
   On the DB instance security group, confirm that there's an inbound rule that allows the notebook instance's security group, VPC CIDR block, or subnet CIDR block.

4. To test the connection, run the following command in a terminal on the SageMaker notebook instance.
   Note: Replace mydatabase.c5y9vfc8igjj.ap-southeast-2.rds.amazonaws.com in the example with your DB instance endpoint. Replace 3306 with your DB instance port.

   curl -v mydatabase.c5y9vfc8igjj.ap-southeast-2.rds.amazonaws.com:3306

Troubleshoot

If the notebook instance has direct internet access activated, then the test command fails with a Connection timed out error. If this failure happens, then run the following command to check the notebook instance's routing table at the operating system level:

route -n

When direct internet access is activated, the routing table looks similar to the following content:

Kernel IP routing tableDestination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.16.0.1      0.0.0.0         UG    0      0        0 eth0
10.0.32.0       0.0.0.0         255.255.224.0   U     0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.255.0   U     0      0        0 veth\_def\_agent
169.254.169.254 0.0.0.0         255.255.255.255 UH    0      0        0 eth0
172.16.0.0      0.0.0.0         255.255.224.0   U     0      0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-5a785eca34a3
192.168.0.0     0.0.0.0         255.255.128.0   U     0      0        0 eth2
192.168.0.0     192.168.0.1     255.255.0.0     UG    0      0        0 eth2

In this example, the DB instance's VPC CIDR block is 10.0.0.0/24. The operating system's routing table doesn't have a route to 10.0.0.0/24. Instead, traffic to the Amazon RDS VPC CIDR block uses the primary network interface (eth0) that handles public traffic.

To resolve the connection issue, modify the routing table to use eth2, the VPC elastic network interface for the notebook instance:

1. Note the SageMaker VPC router address. In this example, 192.168.0.0/16 (the VPC CIDR block for the notebook instance) is routed to 192.168.0.1. This means that 192.168.0.1 is the VPC router address.

2. Add the route to the routing table in the notebook instance terminal.
   Note: In the following example, replace 10.0.0.0/24 with the VPC CIDR block for your Amazon RDS DB instance. Replace 192.168.0.1 with the VPC router address for your SageMaker notebook instance.

   sudo ip route add 10.0.0.0/24 via 192.168.0.1 dev eth2

3. Confirm that the new route is in the routing table:

   Kernel IP routing tableDestination     Gateway         Genmask         Flags Metric Ref    Use Iface
   0.0.0.0         172.16.0.1      0.0.0.0         UG    0      0        0 eth0
   10.0.0.0        192.168.0.1     255.255.255.0   UG    0      0        0 eth2
   10.0.32.0       0.0.0.0         255.255.224.0   U     0      0        0 eth1
   169.254.0.0     0.0.0.0         255.255.255.0   U     0      0        0 veth\_def\_agent
   169.254.169.254 0.0.0.0         255.255.255.255 UH    0      0        0 eth0
   172.16.0.0      0.0.0.0         255.255.224.0   U     0      0        0 eth0
   172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
   172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-5a785eca34a3
   192.168.0.0     0.0.0.0         255.255.128.0   U     0      0        0 eth2
   192.168.0.0     192.168.0.1     255.255.0.0     UG    0      0        0 eth2

4. Test the connection again.

Note: Changes to the operating system routing table don't persist between notebook instance sessions. This lack of persistence means that you lose the changes each time that you stop and start the SageMaker notebook instance. To mitigate this situation, use a lifecycle configuration to add the route every time that you start the notebook instance.

Related information

Connect a notebook instance in a VPC to external resources

Connect to SageMaker within your VPC