Why can't I mount my FSx for ONTAP file system on my EC2 Linux instance?

Short description

The following are common mount errors:

• mount.nfs: No such device
• mount.nfs: mount point directory_name does not exist
• mount.nfs: Invalid argument
• mount.nfs: an incorrect mount option was specified
• mount.nfs: Connection timed out
• mount.nfs: requested NFS version or transport protocol is not supported
• mount: /mnt: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.type helper program.
• mount.nfs: access denied by server while mounting filesystem_DNS_name

Resolution

Error: mount.nfs: No such device

This error occurs when the Network File System (NFS) module isn't mounted on the client system. To mount the NFS, load the nfs module.

1.    To check the availability of the nfs module on your machine, run the following command:

$ modinfo nfs
$ sudo lsmod | grep nfs

Example output:

$ modinfo nfs

filename: /lib/modules/5.10.184-175.749.amzn2.x86_64/kernel/fs/nfsd/nfsd.ko
license: GPL
author: Example
alias: fs-nfsd
srcversion: 4FD81F662CDF9FE3FDB9970
depends: auth_rpcgss,sunrpc,grace,lockd,nfs_ssc,nfs_acl
retpoline: Y
intree: Y
name: nfsd

$ ls -lR /usr/lib/modules/$(uname -r)/kernel/fs/ | grep -E "(nfs|nfsd).ko"

-rw-r—r-- 1 root root 912854 Jul 12 18:48 nfs.ko
-rw-r—r-- 1 root root 1273502 Jul 12 18:48 nfsd.ko

2.    To check if the modules are loaded, run the following command:

$ sudo lsmod | grep nfs
nfsv3 61440 0
nfs_acl 16384 1 nfsv3
nfsv4 950272 0
dns_resolver 16384 1 nfsv4
nfs 442368 2 nfsv4,nfsv3
lockd 131072 2 nfsv3,nfs
nfs_ssc 16384 2 nfsv4,nfs
fscache 389120 2 nfsv4,nfs
sunrpc 679936 8 nfsv4,auth_rpcgss,lockd,nfsv3,rpcsec_gss_krb5,nfs_acl,nfs

3.    If there's no output from the lsmod | grep nfs command, then run the modprobe command to load the module:

$ sudo modprobe nfs

Error: mount.nfs: mount point directory_name does not exist

This error occurs when you try to mount the file system on a directory that doesn't exist.

To check whether the directory exists, and create a directory, complete the following steps:

Note: In the following commands, replace mount_point with your mount point.

1.     To verify that the directory exists, run the following command:

$ ls -l mount_point

2.     To create the mount point directory, run the following command:

$ sudo mkdir mount_point

Error: mount.nfs: Invalid argument or mount.nfs: an incorrect mount option was specified

To correct these errors, check the mount options in the mount command or in the options. Remove or replace invalid arguments with the correct arguments.

In the following example, the command contains the incorrect mount option, foo=bar:

# mount -t nfs -o foo=bar svm-xxx.fs-xxx.fsx.eu-north-1.amazonaws.com:/vol1 /mnt -v
mount.nfs: mount(2): Invalid argument
mount.nfs: an incorrect mount option was specified

Error: mount.nfs: Connection timed out

The Connection timed out error indicates that the file system endpoint IP address isn't reachable from the client machine.

To resolve this error, complete the following steps:

1.    Check the firewall and security group rules of the client machine. For information on the egress and ingress rules and a list of the port numbers that different protocols use, see Creating a VPC security group.

2.    Open the Amazon FSx console, and then select your FSx for ONTAP file system.

3.    Choose Network & security. Check the route tables to make sure that there's a route from the client machine to the storage virtual machine (SVM) endpoint IPv4 address.

4.    Check the SVM endpoint IPv4 address. From the Endpoints tab, choose FSx, Storage virtual machines.

Or, use the verbose -v option with your mount command. The command output lists the endpoint address as the addr parameter value:

$ sudo mount -t nfs -o nfsvers=4 svm-xxx.fs-xxx.fsx.eu-north-1.amazonaws.com:/vol1 /mnt -v
mount.nfs: timeout set for Tue Aug 8 19:56:30 2023
mount.nfs: trying text-based options 'vers=4.1,addr=172.31.255.242,clientaddr=172.31.16.19'
mount.nfs: mount(2): Connection timed out
mount.nfs: Connection timed out

Note: In the preceding example output, addr=172.31.255.242 is the SVM endpoint IPv4 address for the NFS.

5.    Establish a telnet connection from the client to the file system endpoint's DNS name. Or, use the IPv4 address for different ports. Replace filesystem_management_FQDN and port with your file system's fully qualified domain name (FQDN) and your port:

$ telnet filesystem_management_FQDN port

6.    Use the NFS protocol to check the connection:

$ telnet svm-xxx.fs-xxx.fsx.eu-north-1.amazonaws.com 2049
$ telnet svm-xxx.fs-xxx.fsx.eu-north-1.amazonaws.com 635

Note: Make sure that you allow and test all the ports that are listed for a protocol in Step 1.

Error: mount.nfs: requested NFS version or transport protocol is not supported

Sometimes, file system administrators force clients to use only specific protocols or protocol versions to mount. For example, the client can use NFS v4.1 to mount, but not v3.

$ sudo mount -t nfs -o vers=3 svm-xxx.fs-xxx.fsx.eu-north-1.amazonaws.com:/vol1 /mnt
mount.nfs: requested NFS version or transport protocol is not supported

When the NFS client tries to mount a file system with NFS v3, the requested NFS version or transport protocol is not supported error occurs.

To resolve this error, complete the following steps:

1.     At the client end, use a different version of NFS. For example, if you previously tried -o vers=3, then try -o vers=4.1:

$ sudo mount -t nfs -o vers=4.1 svm-xxx.fs-xxx.fsx.eu-north-1.amazonaws.com:/vol1 /mnt

2.    To get the list of supported protocols, connect to the NetApp ONTAP CLI, and then run the following command. Replace server_name with your Vserver's name.

::>  vserver show-protocols -vserver vserver_name
Vserver Name Protocols
--------------- ---------------
fsx nfs, cifs, fcp, iscsi, ndmp, s3
::> vserver nfs show

Vserver: vserver_name

General Access: true
v3: disabled
v4.0: enabled
4.1: enabled
UDP: enabled
TCP: enabled
RDMA: enabled
Default Windows User: -
Default Windows Group: -

Note: In the preceding example output, NFS v3 is disabled.

3.    To turn on a particular version, use the following command syntax:

::> vserver nfs modify -v3 enabled
::> vserver nfs show

For more information, see vserver show-protocols on the NetApp website.

Error: mount: /mnt: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.type helper program

This error occurs when an NFS client package isn't installed on the client machine. For Amazon Linux, CentOS, and Red Hat Enterprise Linux (RHEL) operating systems, the package name is nfs-utils. For Ubuntu and Debian operating systems, the package name is nfs-common.

To resolve this error, complete the following steps:

1.     To confirm that the package doesn't exist, run the following commands:

Amazon Linux, CentOS, and RHEL OS:

$ rpm -qa | grep nfs  
$ systemctl list-unit-files | grep -i nfs

Ubuntu and Debian OS:

$ dpkg -l | grep -i nfs  
$ systemctl list-unit-files | grep -i nfs

2.     If there's no output, then there's no package. Run the following command to install the package:

Amazon Linux, CentOS, and RHEL OS:

$ sudo yum install nfs-utils -y

Ubuntu and Debian OS:

`$ sudo apt update`  
`$ sudo apt install nfs-commo`

3.     To verify that the packages are installed, repeat the commands from Step 1.

Error: mount.nfs: access denied by server while mounting filesystem_DNS_name

When you create an NFS share, export policies and rules are used. These policies and rules are a mechanism to share and grant access to share NFS clients. 

The client that you use to mount the file system must match the access criteria that are specified in the export policy rule. If the client doesn't match the access criteria, then the client receives an access denied error.

In the following example, a client receives the access denied error because the file system's export policy allows only clients with a specified IPv4 address CIDR.

Client side:

$ sudo mount -t nfs -o nfsvers=4 svm-xxx.fs-xxx.fsx.eu-north-1.amazonaws.com:/vol1 /mnt
mount.nfs: access denied by server while mounting svm-xxx.fs-xxx.fsx.eu-north-1.amazonaws.com:/vol1

$ ip -4 a show eth0
inet xxx.xxx.xxx.xxx/20 brd xxx.xxx.xxx.xxx scope global dynamic eth0

$ sudo showmount -e xxx.xxx.xxx.xxx
Export list for xxx.xxx.xxx.xxx:
/ (everyone)
/vol1 (everyone)

To correct the error, complete the following steps on FSx server:

1.    Use SSH to connect to the NetApp ONTAP CLI.

2.     List the available volumes under the SVM and their corresponding policy name:

::> vol show -fields junction-path,policy

vserver volume policy junction-path
------- -------- ---------------------- -------------
fsx fsx_root fsx-root-volume-policy /
fsx vol1 default /vol1
2 entries were displayed.

Note: In the preceding example, there are two volumes, fsx_root and vol1, under the fsx SVM. The NFS share vol1 uses the default policy.

3.    To check the default export policy for fsx, run the following command. Replace vserver_name with your Vserver's name:

::> export-policy rule show -vserver vserver_name -policyname default

             Policy          Rule    Access   Client                RO
Vserver      Name            Index   Protocol Match                 Rule
------------ --------------- ------  -------- --------------------- ---------
unix         default         1       any      1.2.3.4/32            any

Note: The policy allows any protocol access from the client only with the 1.2.3.4/32 IPv4 address.

Or, provide the client's IPv4 address to check access to the client. Replace verserver_name and volume_name with the names of your Vserver and volume:

::> export-policy check-access -vserver vserver_name -vol volume_name -client-ip 000.00.00.00 -protocol nfs4 -authentication-method sys -access-type read-write

Path    Policy   Policy Owner   Policy Owner Type   Rule Index   Access   Security Style
-------------------- ---------- --------- ---------- ---- ---------- ----------
/ fsx-root-volume-policy fsx_root volume 1 read unix
/vol1 default vol1 volume 0 denied unix
2 entries were displayed.

Note: The preceding output shows denied under the Access column for the /vol1 share.

To resolve this issue, update the export policy rules for the preceding example. Run the following command. Replace vserver_name with your Vserver's  name:

::> export-policy rule modify -vserver vserver_name -policyname default -ruleindex 1 -clientmatch 0.0.0.0/0

Note: The preceding command allows all the client machines.

Your environment might have different rules. For more information on how to modify rules for different parameters, see vserver export-policy rule modify on the NetApp website.