How do I troubleshoot NTP synchronization issues on my Amazon EC2 Linux servers?

Resolution

To troubleshoot NTP synchronization issues, first check the NTP server that your Linux server uses. Then, configure the NTP synchronization, and check the connection to the NTP servers.

To confirm that your instance is synchronized to an NTP server, run the following command to check the time:

timedatectl

Check the NTP server

On Amazon Linux servers, use chrony or the NTP daemon (ntpd) to synchronize with NTP servers. For more information about NTP server configurations, see Configuring NTP using the chrony suite, or Configuring NTP using ntpd on the Red Hat Linux Enterprise (RHEL) website.

Note: It's a best practice to use chrony for time synchronization because chrony synchronizes the system clock faster and with better accuracy than ntpd. For more information, see the differences between chrony and ntpd on the RHEL website.

To check whether your instance is configured to use chrony or the ntpd daemon, run the following commands:

chrony:

sudo systemctl status chronyd
chronyc tracking

ntpd:

sudo systemctl status ntpd
ntpstat

To remove ntpd and install chrony on your Amazon Linux 2 or Amazon Linux 2023 instance, run the following commands:

sudo yum erase ntp*
sudo yum install chrony

To start and activate chrony, run the following commands:

sudo systemctl start chronyd
sudo systemctl enable chronyd

Configure NTP sources

Check the configuration file for the list of configured NTP servers. The default location for the configuration file is /etc/chrony.conf for chrony and /etc/ntp.conf for ntpd.

Amazon Linux 2

Configure chrony to use sources from the /etc/chrony.d/ directory instead of the main /etc/chrony.conf file.

Amazon Linux 2023

Configure chrony to use sources from the /run/chrony.d or /etc/chrony.d/ directories instead of the main /etc/chrony.conf file.

Check the /etc/chrony.conf configuration file for the location of the source directories (sourcedir), or check the README file in /etc/chrony.d/.

Create your own chrony source file in one of the source directories, for example /etc/chrony.d/your-source-file.sources.

Note where the sources are configured. Then, check that you use the correct servers or pools.

You can configure NTP synchronization to either the local Amazon Time Sync Service or the public Amazon Time Sync Service. It's a best practice to use the local Amazon Time Sync Service on your instances. Use the public Amazon Time Sync Service as a backup, or for devices that are located outside of Amazon Virtual Private Cloud (Amazon VPC). You can also configure NTP synchronization to the NTP server that's internal to your organization.

After you configure chrony and create the source file, restart the service. Then, run the following commands on Amazon Linux 2 or Amazon Linux 2023 to verify that your instance is running:

chrony:

sudo systemctl restart chronyd.service
sudo systemctl status chronyd.service

ntpd:

sudo systemctl restart ntpd.service
sudo systemctl status ntpd.service

Configure NTP synchronization to local Amazon Time Sync Service

Use the link-local IPv4 address 169.254.169.123 within your VPC to access Amazon Time Sync Service.

To add the Amazon Time Sync Service link-local IP address as an NTP source, open the configuration file where NTP sources are specified. For example, open the /etc/chrony.conf file for chrony. Then, add the following line:

server 169.254.169.123 prefer iburst minpoll 4 maxpoll 4

If you use the IPv6 local address that's accessible to instances built on the AWS Nitro System, then add the following line to the configuration file:

server fd00:ec2::123 prefer iburst minpoll 4 maxpoll 4

For the changes to take effect, run the following command to restart chrony:

sudo systemctl restart chronyd.service

To restart ntpd, run the following command:

sudo systemctl restart ntpd.service

Note: It's a best practice to use the IPv4 and IPv6 endpoint entries separately. The IPv4 and IPv6 NTP packets come from the same local server for your instance. If you configure both IPv4 and IPv6 endpoints, then the accuracy of the time on your instance doesn't improve.

Configure NTP synchronization to public Amazon Time Sync Service

A Linux instance or external internet-connected device can use the public endpoint time.aws.com to access the Amazon Time Sync Service.

To add the Amazon Time Sync Service public endpoint as an NTP source, open the ntpd or chrony configuration file where the sources are located. Then, run the following command:

pool time.aws.com iburst

For the changes to take effect, run the following command to restart chrony:

sudo systemctl restart chronyd.service

To restart ntpd, run the following command:

sudo systemctl restart ntpd.service

Configure NTP synchronization to your own NTP server internal to your organization

You can use an NTP server that's internal to your organization. Or, you can use an NTP server that's accessible through the public internet.

Note: It's a best practice to use only trusted NTP servers.

To add your own NTP source, open the ntpd or chrony configuration file where the sources are located. Then, add the following line:

server internal-ntp-server-hostname-or-ip-address

Note: Replace internal-ntp-server-hostname-or-ip-address with the name of your NTP server hostname or IP address.

Make sure that your Linux instance can reach the IP address and port of the new NTP server over the network.

By default, Chrony uses Amazon Time Sync time as sources through link-local and remote endpoints.

Amazon Linux 2023

To override the default configuration in Amazon Linux 2023, complete the following steps:

1. Modify the /etc/sysconfig/chronyd file to include USE_AMAZON_NTP_POOL="no".

2. Run the following command to restart chrony:

   sudo systemctl restart chrony.service

   To restart ntpd, run the following command:

   sudo systemctl restart ntpd.service

Amazon Linux 2

To override the default configuration in Amazon Linux 2, complete the following steps:

1. Modify the /etc/chrony.d/link-local.sources file. Add a # at the beginning of the line to comment out the entry for Amazon Time Sync Service:

    #server 169.254.169.123 prefer iburst minpoll 4 maxpoll 4

2. Run the following command to restart the chrony:

    sudo systemctl restart chronyd.service

   To restart ntpd, run the following command:

   sudo systemctl restart ntpd.service

Note: The /etc/chrony.d/README file provides information about other ways to override the default configuration.

Confirm that your instance is synchronized to the correct time

To verify that your instance is synchronized to the correct time, run the following commands:

chrony:

chronyc tracking
timedatectl

ntpd:

ntpstat
timedatectl

To list all the configured sources, run the following commands:

chrony:

chronyc sources -v

ntpd:

ntpq -p

Additional troubleshooting steps

If you continue to experience NTP synchronization issues, then take the following actions:

• Check the ntpd or chrony log files for error messages or warnings that can help you determine the issue.
• If the time on your system is different from the NTP server time, then manually set your system time to match the NTP server time. You can use a tool such as, timedatectl.
• Make sure that your Linux server can resolve the NTP server hostnames.
• Make sure that operating systems (OS) firewalls, security groups, network access control lists (network ACLs), or route tables don't block external NTP servers. The instance requires outbound traffic to UDP port 123. For stateless firewalls and network ACLs, make sure to allow return traffic through ephemeral ports 1024 to 65535.
• If you use a custom DHCP option set in your VPC, then make sure that you correctly configure the NTP server addresses.

Related information

Precision clock and time synchronization on your EC2 instance