After AWS tunnel replacement activity, our connection to our on prem device stops working. The replaced tunnel shows up and available but not working properly.

Hey, Hope you're keeping well. When AWS replaces a Site-to-Site VPN tunnel endpoint, the new AWS-side IP address changes while the connection configuration remains logically the same. Even if the tunnel shows as “UP” in the AWS console, your Palo Alto device may still be encrypting traffic toward the old endpoint until its IPsec and IKE sessions are fully renegotiated. In this state, the AWS VPN control plane detects the BGP or static configuration as available, but data plane packets are dropped because the security association does not match. On a static VPN, you must update the peer IP to the new AWS outside IP address, clear any existing phase 1 and phase 2 SAs, and reinitiate the tunnel from the Palo Alto side. Thanks and regards, Taz

When AWS performs a tunnel replacement activity on a Site-to-Site VPN connection, it's not uncommon to experience connectivity issues even though the AWS dashboard shows both tunnels as up and available. This discrepancy can occur for several reasons.

After a tunnel endpoint replacement, the tunnel may appear to be up but traffic might not be flowing properly. This could be due to the VPN connection needing to re-establish proper IPsec and IKE associations. Since your Palo Alto device is the customer gateway, you might need to "bounce" (disable and re-enable) the tunnel interface on your device to resolve the connectivity issue.

Several factors could be causing the replaced tunnel to not work properly:

1. The IPsec/IKE negotiation might not have completed successfully after the replacement
2. There could be routing issues preventing traffic from flowing through the new tunnel
3. Security policies or access lists might need to be updated on your Palo Alto device

Since you mentioned that the other tunnel works fine during maintenance periods, this confirms that your basic connectivity and routing are working correctly. The issue is specific to the replaced tunnel.

To resolve this issue, you could try:

1. Restarting the VPN tunnel on your Palo Alto device
2. Verifying that the IPsec and IKE configurations match the AWS requirements
3. Checking for any traffic routing issues between your on-premises network and AWS
4. Ensuring there are no packet loss issues over the connection

It's also worth noting that AWS performs tunnel replacements one at a time specifically to maintain connectivity through the second tunnel, which aligns with your observation that the non-replaced tunnel continues to work properly.
Sources
AWS Site-to-Site VPN tunnel endpoint replacements - AWS Site-to-Site VPN
Troubleshoot AWS Site-to-Site VPN connectivity with a Cisco ASA customer gateway device - AWS Site-to-Site VPN
Troublehsoot connectivity between SIte-to-Site VPN and VPC | AWS re:Post