RHEL 9 US-EAST-1 ARM64 Repos having issues again 2/28/2023...

0

Make a copy of your redhat-rhui.repo file, and do a search and replace with REGION with us-east-2 and you'll be able to grab updates again.

Change: mirrorlist=https://rhui.REGION.aws.ce.redhat.com/pulp/mirror/content/dist/rhel9/rhui/$releasever/$basearch/appstream/os to mirrorlist=https://rhui.us-east-2.aws.ce.redhat.com/pulp/mirror/content/dist/rhel9/rhui/$releasever/$basearch/appstream/os

It appears that the us-east-1 aarch64 RHEL9 repos are having issues again. But at least we're able to point to another region this time for updates. I've notified RHEL of the issue again.

Both us-east-2 and us-west-2 work fine for me. us-east-1 has no updates, going on 24 hrs+ now.

asked 2 years ago501 views
1 Answer
0

I tested this without success using the public RHEL 9 ARM AMI ami-0373218dcae329f21 in us-east-1. I found that updates were working normally. However I have a few ideas:

First, I spotted on https://status.redhat.com/ that there was an outage today for newly registered entitlements, and this was taking place from 13:39 - 15:55 EST. However, there is no regional component to this issue. If this was the cause then the updates should be working normally at this time.

Next, in the repository configuration there is a variable "REGION" which you mentioned substituting for a fixed string such as us-east-2. This variable is handled by a plugin called "amazon-id" which must be enabled for this variable to be converted to the proper region. If the plugin is not working normally yum/dnf operations will fail with "[Could not resolve host: rhui.REGION.aws.ce.redhat.com]".

The amazon-id plugin receives the region from the Instance Metadata Service. It's possible that the service could have an issue or that a rate limit could be reached (1024 packets per second). You can check for any limit being reached by running[1]:

[root@ip-172-31-1-190 ~]# ethtool -S eth0 | grep linklocal_allowance_exceeded

linklocal_allowance_exceeded: 0

This output should be 0, and any non-zero output is a count of dropped packets. As well, I would be sure the region is present in the output of a curl directly to the metadata service:

[root@ip-172-31-1-190 ~]# curl 169.254.169.254/latest/meta-data/placement/availability-zone

us-east-1d

Any issues with the Instance Metadata Service can be worked around by stopping and starting an instance to move it to new underlying hardware. Another interesting test for this scenario is to try substituting REGION with us-east-1. This would be a an apples-to-apples comparison with substituting the variable with us-east-2, while using REGION requires the plugin and metadata.

[1] https://aws.amazon.com/blogs/networking-and-content-delivery/amazon-ec2-instance-level-network-performance-metrics-uncover-new-insights/

profile pictureAWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions