Skip to content

AWS FSX Lustre Mounting from EC2 in another VPC Using AWS Private link

0

Problem: FSx Lustre mount fails with EIO (-5) when accessed cross-account via NLB + PrivateLink

Setup:

FSx Lustre (PERSISTENT_2, v2.15) in Account A EC2 hosts in Account B, different VPC, Lustre client v2.12.8 on RHEL 8.10 Cross-account access via: FSx ENIs → Internal NLB (TCP) → VPC Endpoint Service → VPC Endpoint in Account B NLB forwards ports 988 (LNET) and 1018-1023 (data), preserve_client_ip = true What works:

TCP connectivity through the VPCE is fine — nc -zv to port 988 succeeds Hosts in Account A (same VPC as FSx) mount directly without issues The VPCE is in "available" state, DNS resolves correctly What fails: sudo mount -t lustre <vpce-dns>@tcp:/<mount-path> /x01 mount.lustre: ... failed: Input/output error Is the MGS running?

dmesg output: LNet: Added LNI <client-ip>@tcp LNet: Accept secure, port 988 MGC<vpce-ip-1>@tcp: failed processing log, type 1: rc = -5 Error -104 reading HELLO from <vpce-ip-1> Connection to <vpce-ip-1>@tcp at host <vpce-ip-1> on port 988 was reset: is it running a compatible version of Lustre and is <vpce-ip-1>@tcp one of its NIDs? Error -104 reading HELLO from <vpce-ip-2> Connection to <vpce-ip-2>@tcp at host <vpce-ip-2> on port 988 was reset: is it running a compatible version of Lustre and is <vpce-ip-2>@tcp one of its NIDs? MGC<vpce-ip-2>@tcp: failed processing log, type 1: rc = -5 Unable to mount (-5)

Our analysis: The Lustre client connects to the VPCE IPs which route through the NLB to FSx. During the LNET HELLO handshake, FSx responds with its real NID (its actual private IP in Account A's VPC). The client detects a NID mismatch — the peer's NID doesn't match the IP it connected to — and resets the connection with ECONNRESET (-104).

How to solve/fix this

1 Answer
0

Hello.

While this isn't a solution to the PrivateLink issue, is it difficult to access FSX Lustre via VPC peering in your environment?
https://docs.aws.amazon.com/fsx/latest/LustreGuide/mounting-on-premises.html

As you understand, PrivateLink uses the IP address of the ENI as the connection destination IP address visible to the client. Therefore, it is likely that the error is occurring because the NID that responded to Lustre's HELLO is different.
To my understanding, there is no setting on the FSx Lustre side to rewrite the NID, and furthermore, it is not possible to overwrite the NID via NLB or PrivateLink, so I think it is currently not possible to access it via PrivateLink.

EXPERT

answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.