How do I make my secondary network interface work in my Ubuntu EC2 instance?

7 minute read
0

I want to make my secondary network interface work in my Ubuntu Amazon Elastic Compute Cloud (Amazon EC2) instance.

Short description

Warning: The use of a second elastic network interface is intended for advanced users. Use this solution only for the following use case: You can't use a single network interface. You must attach two network interfaces from the same subnet to one instance. To avoid asymmetric routing issues, use a single elastic network interface, or place duplicate elastic network interfaces into non-overlapping subnets. Or, use Ubuntu 24.04 LTS or later.

Adding a secondary network interface to an instance that isn't an Amazon Linux EC2 instance causes traffic flow issues. These issues occur because the primary and secondary network interfaces are in the same subnet, and there's one routing table with one gateway. Traffic that comes into the secondary network interface uses the primary network interface to leave the instance. Because the secondary IP address doesn't belong to the MAC address of the primary network interface, the secondary interface doesn't work.

Note: The following steps aren't necessary for Ubuntu 24.04. Ubuntu 24.04 automatically handles this advanced configuration.

To make the secondary interface work after you create it, complete the following steps:

  1. Configure the routing table.
  2. Set up rules in the custom routing table policy database so that traffic for the secondary interface uses the new routing table.

Before you start, note that Amazon EC2 instances are in the AWS Cloud, and so not all use cases benefit from having multiple interfaces. The following examples show when you might not need to use a secondary network interface:

  • To increase network throughput: Because limits are set based on the instance type and size, network throughput doesn't increase. For more information, see Amazon EC2 instance types.
  • To increase Elastic IP addresses: You don't need to add Elastic IP addresses to your instance because most applications work well with the DNS. For example, Apache can use name-based virtual hosts. For more information, see Name-based virtual host support on the Apache website.

Note: The steps to add a secondary network interface are different for each of the following Ubuntu versions:

  • Ubuntu 14.04
  • Ubuntu 16.04
  • Ubuntu 18.04
  • Ubuntu 20.04
  • Ubuntu 22.04

Resolution

Note: Although the steps in the following resolution are tested on the listed Linux versions, you might need to modify your configuration's custom settings. All procedures must be run with root user permissions. Either become the root user with sudo -i, or run all commands with sudo.

Configure Ubuntu 14.04 or 16.04

You can use a single file to create the secondary interface configuration file, configure the routing table, and then set routing policy rules for Ubuntu.

To configure Ubuntu 14.04 or 16.04, complete the following steps:

  1. To get the name of the primary network interface, run the following command:

    ip a | grep ^[[:digit:]]

    You receive an output that's similar to the following message:

    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
    3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

    Important: In the preceding example, the primary interfaces are named eth0, eth1, and so on. However, for instances that support enhanced networking, such as the m4 and m5 family type, you might see a naming inconsistency. For example, the primary might be named ens3 if the secondary is named eth0. This naming inconsistency happens when you add the secondary interface when the instance is running. To avoid a naming inconsistency, add the interface at launch time or reboot the instance. Or, if the interface is running, then use the following command to change the name:

    ip link set eth0 name ens4 
    ip link set ens4 up
    ip link show ens4 --->verify
  2. Create a configuration file for the secondary interface:

    vi /etc/network/interfaces.d/51-eth1.cfg

    Note: Change eth1 to match your secondary interface name.

    The following command is an example for a single IP address of 172.31.21.115 with a gateway of 172.31.16.1 on the secondary interface:

    auto eth1  
    iface eth1 inet static address 172.31.21.115  
    netmask 255.255.240.0
    
    # Gateway configuration
    up ip route add default via 172.31.16.1 dev eth1 table 1000
    
    # Routes and rules
    up ip route add 172.31.21.115 dev eth1 table 1000
    up ip rule add from 172.31.21.115 lookup 1000

    Note: Replace the example IP address and gateway with your own information. Your gateway must be the first valid IP address in your subnet.

    The following command is an example for multiple IP addresses. In this example, the IP addresses are 172.31.21.115 and 172.31.18.46, and the gateway is 172.31.16.1:

    auto eth1
    # Enter one or more IP settings
    iface eth1 inet static
    address 172.31.21.115
    netmask 255.255.240.0
    
    iface eth1 inet static
    address 172.31.18.46
    netmask 255.255.240.0
    
    # Default gateway for eth1
    up ip route add default via 172.31.16.1 dev eth1 table 1000
    
    # A route for every IP
    up ip route add 172.31.21.115 dev eth1 table 1000
    up ip route add 172.31.18.46 dev eth1 table 1000
    
    # A policy rule for every IP
    up ip rule add from 172.31.21.115 lookup 1000
    up ip rule add from 172.31.18.46 lookup 1000

    Note: Replace the example IP addresses and gateway with your own information.

  3. Create the restrict-default-gw file to prevent the default gateway from being overwritten on the main table:

    vi /etc/dhcp/dhclient-enter-hooks.d/restrict-default-gw
  4. Add the following lines to the restrict-default-gw file:

    case ${interface} in  eth0)    ;;  *)
        unset new_routers
        ;;
    esac

    Note: Replace eth0 with your primary interface name.

  5. Restart the network.

    For Ubuntu 14.04, run the following command:

    (ifdown eth1 && ifup eth1)

    For Ubuntu 16.04, run the following command:

    systemctl restart networking

Configure Ubuntu 18.04, 20.04, and 22.04

Ubuntu 18.04, 20.04, and 22.04 use the Netplan networking configuration. Note that Netplan uses the YAML format. This YAML format means that indentation is critical. The following Netplan example uses two-space indentation.

To configure Ubuntu 18.04, 20.04, and 22.04, complete the following steps:

  1. Create a configuration file for the secondary interface:

    vi /etc/netplan/51-eth1.yaml
  2. Add the following lines to the 51-eth1.yaml file:

    network:  version: 2  renderer: networkd    
     ethernets:
        eth1:
          addresses:
           - 172.31.24.153/20
           - 172.31.28.195/20
          dhcp4: no
          routes:
           - to: 0.0.0.0/0
             via: 172.31.16.1 # Default gateway
             table: 1000
           - to: 172.31.24.153
             via: 0.0.0.0
             scope: link
             table: 1000
           - to: 172.31.28.195
             via: 0.0.0.0
             scope: link
             table: 1000
          routing-policy:
            - from: 172.31.24.153
              table: 1000
            - from: 172.31.28.195
              table: 1000

    Note: Replace the preceding example information with information specific to your use case. The preceding example YAML file configures two IP addresses on eth1, the secondary interface.

To find the CIDR range to use in the YAML file, complete the following steps:

  1. Open the Amazon EC2 console, select Instances, and then select the instance.
  2. On the Networking tab, scroll to Network interfaces, and then note the Subnet ID of your secondary network interface.
  3. Open the Amazon Virtual Private Cloud (Amazon VPC) console, select Subnets, and then note the IPv4 CIDR range listed for the subnet ID.
  4. To apply the network configuration, run the following command:
    netplan --debug apply

Related information

Configure route tables

AWS OFFICIAL
AWS OFFICIALUpdated a month ago
12 Comments

what is the default gateway for ipv6 , used ::1 but its not working [ubuntu 22.04], it worked for ipv4

I used this netplan:

network: version: 2 renderer: networkd ethernets: ens4: addresses: - 2600:1f13:65f:fabf:a051:32d:525e:8cb0/128 dhcp4: no routes: - to: ::/0 via: 2600:1f13:65f:fabf::1 table: 1000 - to: 2600:1f13:65f:fabf:a051:32d:525e:8cb0 via: ::/0 scope: link table: 1000 routing-policy: - from: 2600:1f13:65f:fabf:a051:32d:525e:8cb0 table: 1000

I have 2 subnets (different) on each ec2 instance. Ipv4 connection is working well but ipv6 does not. I am unable to ping the gateway 2600:1f13:65f:fabf::1 too.

Ping is working among the subnets without netplan but when trying to connect to a server running on 80 it does not so I tried to create a netplan above or add a static route using ::1. I have my ports open in the security group and the connection to server works with in same subnet.

replied a year ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied a year ago

Hi, I try to configure two ENIs on an Ubuntu 20.04.6 LTS instance, the primary ENI and a second one with an Elastic IP.

Ip a displays the two ip addresses correctly so I do not understand why I should modify the netplan file even if the IPs are not defined in it.

Could you explain why it is needed to modify the netplan file ?

My point is that if i try to reach a tomcat server using the Elastic IP, the request does not reach tomcat. I check the security group which allows all traffic, the route table forward all requests for the outside world to the internet gateway and the NACL allows all traffic (in and out). The tomcat is not configured with a bind address so it listens on all server IPs.

So I do not understand why the tomcat server is not reachable.

Note : if i attach the elastic ip to the primary ENI, I can reach the tomcat server

replied a year ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied a year ago

Thanks for your answer "moderator", how can i know if the knowledge center will be updated and if it is updated, how can i know when it is updated ?

Sincerly,

Bernard

replied a year ago

I describe my configuration here

replied a year ago

The answer didn't help at all with IPv6 setup

replied 7 months ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied 7 months ago

Thank you for the answer. I followed the instruction, it seems not working:

  1. Create a file for ens4: sudo vi /etc/network/interfaces.d/51-ens4.cfg Contents:

auto ens4

Enter one or more IP settings

iface ens4 inet static address 10.1.75.211 netmask 255.255.0.0

iface ens4 inet static address 10.1.115.59 netmask 255.255.0.0

Default gateway for ens4

up ip route add default via 10.1.0.1 dev ens4 table 1000

A route for every IP

up ip route add 10.1.75.211 dev ens4 table 1000 up ip route add 10.1.115.59 dev ens4 table 1000

A policy rule for every IP

up ip rule add from 10.1.75.211 lookup 1000 up ip rule add from 10.1.115.59 lookup 1000

  1. Create another file for ens4: sudo vi /etc/dhcp/dhclient-enter-hooks.d/restrict-default-gw Contents:

case ${interface} in ens3) ;; *) unset new_routers ;; esac

  1. sudo systemctl restart networking systemctl status networking.service

May 29 04:24:14 localhost.localdomain systemd[1]: Starting Raise network interfaces... May 29 04:24:14 localhost.localdomain ifup[6727]: Cannot find device "ens4" May 29 04:24:14 localhost.localdomain ifup[6727]: Failed to bring up ens4. May 29 04:24:14 localhost.localdomain systemd[1]: networking.service: Main process exited, code=exited, status=1/FAILURE

Any ideas? Thank you, -Jeff

replied 5 months ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied 5 months ago

Hi @instaview,

By looking into your conf, I think you are using an ubuntu 16 OS, hence I replicated the configuration that has been mentioned in the rePost and was getting the same error as below:

Aug 28 15:17:03 ip-172-31-16-157 systemd[1]: Failed to start Raise network interfaces.
-- Subject: Unit networking.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit networking.service has failed.

I noticed that the code snippet for /etc/network/interfaces.d/51-eth1.cfg has a another arguments after auto eth1 which led to the above error when restarting the network service, After putting the comment on the next new line, I was able to restart the service without errors and can connect to the instance using the secondary interface's IPs.

Could you please check your Secondary interface configuration and make sure the first line only contains: auto ens4

AWS
replied 2 months ago

my conf for the file: /etc/network/interfaces.d/51-ens6.cfg where the errors disappears and the configuration went with success:

auto ens6
# Enter one or more IP settings
iface ens6 inet static
address 172.31.29.122
netmask 255.255.240.0

iface ens6 inet static
address 172.31.25.217
netmask 255.255.240.0

# Default gateway for ens6
up ip route add default via 172.31.16.1 dev ens6 table 1000

# A route for every IP
up ip route add 172.31.29.122 dev ens6 table 1000
up ip route add 172.31.25.217 dev ens6 table 1000

# A policy rule for every IP
up ip rule add from 172.31.29.122 lookup 1000
up ip rule add from 172.31.25.217 lookup 1000
AWS
replied 2 months ago