How can I avoid DNS resolution failures with an Amazon EC2 Linux instance?

Short description

To decrease CPU and network usage and avoid DNS resolution failures, apply a DNS cache.

If you use a DNS cache to query external DNS resources, then the cache answers most of the recurring DNS queries locally. When the cache does this, it doesn't interact with the DNS resolver over the network. You can query external DNS resources such as the following examples:

• Amazon Relational Database Service (Amazon RDS)
• Amazon ElastiCache
• Amazon Simple Storage Service (Amazon S3)

The following procedure applies to all versions of Amazon Linux. If you use another distribution, then select the documentation for your distribution from the following list:

• For information on Debian, see Local caching on the debian.org website.
• For information on Ubuntu, see Local DNS cache on the Ubuntu website.
• For information on Red Hat Enterprise Linux (RHEL), see How to configure DNS caching server with dnsmasq in RHEL on the Red Hat customer portal.

Resolution

Set up a local DNS cache with dnsmasq

To set up a local DNS cache, use dnsmasq. For more information, see dnsmasq on the thekelleys.org.uk website.

Complete the following steps:

1. To install the dnsmasq server, run the following command:

   sudo yum install -y dnsmasq

2. To create a dedicated system user to run dnsmasq, run the following command:

   sudo groupadd -r dnsmasq  
   sudo useradd -r -g dnsmasq dnsmasq

   Note: dnsmasq typically runs as the root user, but it changes to another user after startup. By default, the user is nobody. When dnsmasq changes to another user, the root privileges are dropped.

3. To create a copy of the dnsmasq.conf file, run the following command:

   sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.ori

   To open the configuration file with a text editor, run the following command:

   sudo vim /etc/dnsmasq.conf

4. To edit the /etc/dnsmasq.conf file, run the following example command:

   # Server Configurationlisten-address=127.0.0.1
   port=53
   bind-interfaces
   user=dnsmasq
   group=dnsmasq
   pid-file=/var/run/dnsmasq.pid
   
   # Name resolution options
   resolv-file=/etc/resolv.dnsmasq
   cache-size=500
   neg-ttl=60
   domain-needed
   bogus-priv

   Note: The bogus-priv option in dnsmasq causes a reverse DNS lookup to fail for private IP ranges that aren't in /etc/hosts. This option can also cause the a reverse DNS lookup to fail for the Dynamic Host Configuration Protocol (DHCP) leases file. If you want to perform a successful reverse lookup, then comment out or remove bogus-priv.

5. Create the /etc/resolv.dnsmasq file. Then, to set the Amazon DNS server or the custom domain-name-servers that you specified on DHCP options sets, run the following command:

   sudo bash -c "echo 'nameserver 169.254.169.253' > /etc/resolv.dnsmasq"

   Note: For more information about DNS server locations, see What is DHCP? In some cases, you must adjust the file /etc/resolv.dnsmasq to use the name server for that network. Adjust the file when you create an AMI from an instance with the dnsmasq cache to launch in another VPC with a different CIDR. Or, adjust the file when you specify a custom DNS server in your DHCP options.

6. To restart the dnsmasq server, and set the service to start up on boot, take one of the following actions.
   To start up Amazon Linux 1 on boot, run the following command:

   sudo service dnsmasq restart
   sudo chkconfig dnsmasq on

   To start up Amazon Linux 2 and Amazon Linux 2023 on reboot, run the following command:

   sudo systemctl restart dnsmasq.service
   sudo systemctl enable dnsmasq.service

7. To verify that dnsmasq works correctly, use the following dig command:

   dig aws.amazon.com @127.0.0.1

   If the response is similar to the following example response, then the dnsmasq cache correctly works:

   ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.62.rc1.56.amzn1 <<>> aws.amazon.com @127.0.0.1;; global options: +cmd
   ;; Got answer:
   ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25122
   ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
   
   ;; QUESTION SECTION:
   ;aws.amazon.com.            IN    A
   
   ;; ANSWER SECTION:
   aws.amazon.com.        41    IN    A    54.239.31.69
   
   ;; Query time: 1 msec
   ;; SERVER: 127.0.0.1#53(127.0.0.1)
   ...

8. Set the dnsmasq DNS cache as the default DNS resolver.
   Note: You must suppress the default DNS resolver that DHCP provides. To do this, change or create the /etc/dhcp/dhclient.conf file. For more information, see How do I assign a static DNS server to an Amazon EC2 instance that persists when I reboot?

   Then, to configure the default DNS resolver as a fallback option, run the following command:

   sudo bash -c "echo 'supersede domain-name-servers 127.0.0.1, 169.254.169.253;' >> /etc/dhcp/dhclient.conf"

9. To apply the change, take one of the following actions. Run the following dhclient command:

   sudo dhclient

   -or-
   To restart the network service, run the following command:

   sudo systemctl restart network

   -or-
   To reboot your instance, run the following command:

   sudo reboot

   Note: For Amazon Linux 2023, run sudo systemctl restart systemd-networkd.service instead of dhclient.
   To verify that your instance uses the DNS cache, run the following dig command:

   dig aws.amazon.com

   If the response indicates that the replying server is 127.0.0.1, then the DNS cache correctly works.
   Example response:

   ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.62.rc1.56.amzn1 <<>> aws.amazon.com;; global options: +cmd
   ;; Got answer:
   ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1028
   ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
   
   ;; QUESTION SECTION:
   ;aws.amazon.com.            IN    A
   
   ;; ANSWER SECTION:
   aws.amazon.com.        55    IN    A    54.239.31.69
   
   ;; Query time: 0 msec
   ;; SERVER: 127.0.0.1#53(127.0.0.1) <<<-------
   ...

Automate dnsmasq

To automate the installation and configuration of dnsmasq as a DNS resolver on Amazon Linux, use one of the following options:

• AutomateDnsmasq.sh bash script
• AutomateDnsmasq.cloudinit directives

For more information on the AutomateDnsmasq.sh bash script, see #!/bin/bash on the GithHub website.

For more information on the AutomateDnsmasq.cloudinit directives, see #cloud-config on the GithHub website.

If you want to automate the dnsmasq installation on other Linux distributions, then use either file to make the necessary customization.

When both files use the Amazon DNS server alternative address of 169.254.169.253, the files can run on VPC instances.

To run either file at launch time, pass the contents of the file in the user data field. To perform the actions on an instance, you can run the Bash script as a standalone script or use an AWS Systems Manager run command.

To run the Bash script as a standalone script, complete the following steps:

1. To download the script on your instance, and make it runnable, run the following command:

   wget https://raw.githubusercontent.com/awslabs/aws-support-tools/master/EC2/AutomateDnsmasq/AutomateDnsmasq.shchmod +x AutomateDnsmasq.sh

2. Run the following command as a root user, or use sudo:

   sudo ./AutomateDnsmasq.sh

Related information

Amazon EC2 instance IP addressing