How do I install and activate the latest ENA driver for enhanced network support on my Amazon EC2 instance that runs RHEL?

Short description

AMIs that run on RHEL 7.4 and later include the ENA driver by default. However, you must manually install the ENA driver version if you migrate your EC2 instance type from Xen to the AWS Nitro System. Or, you can manually upgrade the ENA driver if you require a later version than the default version in your AMI.

Note: If you migrated instance types from one that didn't have ENA to one that requires ENA, then you must stop and start the instance. It's a best practice to create a snapshot of your instance's Amazon Elastic Block Store (Amazon EBS) volume. If you experience issues, then you can use the snapshot to restore your instance.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Configure your instance for a stop and start

Note: When you stop and start an instance, the instance's public IP address changes. It's a best practice to use an Elastic IP address to route external traffic to your instance instead of a public IP address. If you use Amazon Route 53, then you might need to update the Route 53 DNS records when the public IP address changes. A stop and start is different from an instance reboot. For more information, see How EC2 instance stop and start works.

Before you stop and start your instance, take the following actions:

• If your instance uses an instance store, then save the volume data on the instance store to persistent storage. For example, you can save the data to Amazon EBS volumes or Amazon Simple Storage Service (Amazon S3) buckets.
  Important: Amazon EC2 deletes instance store data when you stop the instance.
• Temporarily remove the instance from its Amazon EC2 Auto Scaling group so that you don't accidentally terminate the instance when you stop it.
  Note: EC2 Auto Scaling might terminate stopped instances in an Auto Scaling group based on your scale-in protection settings. Instances that you launch with Amazon EMR, AWS CloudFormation, or AWS Elastic Beanstalk might be in an Auto Scaling group.
• Set the instance shutdown behavior to Stop to make sure that the instance doesn't terminate when you stop it.

Use the ENA driver on RHEL 7.4, 8, and 9

AWS provided AMIs that run RHEL 7.4, 8, and 9 include the ENA driver by default. As a result, you don't need to install the driver. Instead, verify that the driver is loaded and that you activated enaSupport on your instance.

Complete the following steps:

1. To list all elastic network interfaces on the instance and identify your primary network interface, run the following command:

   ip link show

   The interface name depends on your operating system (OS) version and instance type. For example, the interface name might be eth0, ens5, or enX0.
   To automatically identify the interface that's used for the default route, run the following command:

   PRIMARY_IFACE=$(ip -o -4 route show to default | awk '{print $5}')
   echo "Primary interface: $PRIMARY_IFACE"

2. To verify that the ENA driver is loaded and check the driver version, run the following command:

   ethtool -i $PRIMARY_IFACE

   Example output:

   driver: ena
   version: 2.16.1g
   firmware-version:
   bus-info: 0000:00:05.0

   To compare your installed version against the latest release, see ENA Linux kernel driver release notes on the GitHub website.

3. Verify that you activated enaSupport.

4. If enaSupport isn't activated, then stop the instance. Then, run the following modify-instance-attribute AWS CLI command to activate enaSupport:

   aws ec2 modify-instance-attribute \
     --instance-id instance-id \
     --ena-support \
     --region region-example

   Note: Replace instance-id with your instance ID and region-example with your AWS Region. If you migrated the instance to the Nitro instance type, then you must activate enaSupport.

5. Start the instance, and then repeat step 2 to verify that the driver is active.

(Optional) Install or upgrade the ENA driver from the source

Important: You can install or upgrade from the source for only RHEL 7.4 and later.

If your workload requires a newer ENA driver version than what's included in your AMI, then build and install the driver from the source.

Complete the following steps:

1. Run the following command to install the build prerequisites:

   sudo yum update
   sudo reboot

2. Wait for the instance to reboot, and then run the following command to install the development packages:

   sudo yum install -y gcc kernel-devel-$(uname -r) git make

3. Run the following command to clone and build the driver source:

   git clone https://github.com/amzn/amzn-drivers.git
   cd amzn-drivers/kernel/linux/ena
   make

4. Run the following command to install the module:

   sudo cp ena.ko /lib/modules/$(uname -r)/
   sudo depmod

   Note: If you plan to use Dynamic Kernel Module Support (DKMS) to automatically rebuild the module after kernel upgrades, then skip this step. For more information, see the (Optional) Keep the same driver version across kernel upgrades section.

5. Run the following command to check the installed module information:

   modinfo ena

   In the output, the filename must point to the /extra/ directory where you installed the new module. The version must match the version that you built from the source. The version shows the ENA driver version, not the kernel version.
   Example output:

     filename:  /lib/modules/5.14.0-611.49.1.el9_7.x86_64/extra/ena.ko
     version:  2.16.1g
     license:  GPL
     description: Elastic Network Adapter (ENA)
     author:   Amazon.com, Inc. or its affiliates

   Note: If the filename points to a path that's not the /extra/ directory, such as /kernel/drivers/, then the module might load from the kernel's in-tree driver. In this scenario, make sure that the new module is in the /lib/modules/$(uname -r)/extra/ location. Then, rerun the following command:

   depmod -a

6. Run one of the following commands to activate the driver version.
   Run the following command to reload the driver:

   sudo modprobe -r ena && sudo modprobe ena

   Or, run the following command to reboot the instance:

   sudo reboot

7. Run the following command to check the driver version on the interface:

   ethtool -i $PRIMARY_IFACE
   

(Optional) Keep the same driver version across kernel upgrades

If your application depends on a specific ENA driver version, then use DKMS to automatically rebuild the module after kernel upgrades. To use this solution, you must clone the driver source. However, you don't need to install the module. DKMS automatically manages module installation.

To keep the same driver version across kernel upgrades, complete the following steps:

1. Run the following command to install Extra Packages for Linux (EPEL) and DKMS to your RHEL instance:

   RHEL_MAJOR=$(rpm -E %{rhel})
   sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-${RHEL_MAJOR}.noarch.rpm
   sudo yum install -y dkms

2. Navigate to the parent directory of the cloned amzn-drivers repository.
   Note: If you completed the steps in (Optional) Install or upgrade the ENA driver from the source, then you're in amzn-drivers/kernel/linux/ena.

3. Run the following command to move the source into /usr/src/:

   VER=$(grep ^VERSION amzn-drivers/kernel/linux/rpm/Makefile | cut -d' ' -f2)
   sudo mv amzn-drivers /usr/src/amzn-drivers-${VER}

4. Run the following command to create the DKMS configuration file:

   sudo tee /usr/src/amzn-drivers-${VER}/dkms.conf > /dev/null <<EOM
   PACKAGE_NAME="ena"
   PACKAGE_VERSION="${VER}"
   CLEAN="make -C kernel/linux/ena clean"
   MAKE="make -C kernel/linux/ena/ BUILD_KERNEL=\${kernelver}"
   BUILT_MODULE_NAME[0]="ena"
   BUILT_MODULE_LOCATION="kernel/linux/ena"
   DEST_MODULE_LOCATION[0]="/updates"
   DEST_MODULE_NAME[0]="ena"
   REMAKE_INITRD="yes"
   AUTOINSTALL="yes"
   EOM

5. Run the following command to register, build, and install the module with DKMS:

   sudo dkms add -m amzn-drivers -v ${VER}
   sudo dkms build -m amzn-drivers -v ${VER}
   sudo dkms install -m amzn-drivers -v ${VER}

6. Run the following command to regenerate the initramfs so that the module is available at boot:

   sudo dracut -f -v
   

Related information

Enable enhanced networking with ENA on your EC2 instances

Linux kernel driver for Elastic Network Adapter (ENA) family on the GitHub website

ENA Linux driver best practices and performance optimization guide on the GitHub website