How does Fsx for Lustre handle supplementary POSIX groups without upcall to IDP?

Question

I would like to use FSx for Lustre as a shared filesystem where files and folders are secured with varying group (gid) and user (uid) permissions. E.g. one department folder will require a specific gid related to that department, and another department folder will require another gid. Some of my users will be member of 50 or more groups (gid's). Latest versions of Lustre only seem to supply two or three groups per user (primary group and one or two supplementary groups), and then Lustre file server is supposed to do an upcall to an identity provider such as LDAP to retrieve and cache additional supplementary groups for the user.

How does this work on FSx for Lustre, where no IDP is configured? Will the Lustre client be asked to provide relevant groups, or will the server sometimes deny access, because the client provided another of the user's supplementary groups than the one which is needed to create a file in a specific directory?

It seems that work is underway in Open Source Lustre v. 2.16 where the server is able to ask the client to retry and supply a specific group for the user (if the user is a member) when creating a new file. It also seems this method is already implemented in Lustre 2.16 when trying to open an existing file. But perhaps a solution is already in place in current FSx for Lustre v. 2.15?

For background I visited these Lustre issues and patches

https://review.whamcloud.com/c/fs/lustre-release/+/55474/13
https://jira.whamcloud.com/browse/LU-18158

In real life will use the Fsx lustre client which is part of the FSx Lustre CSI driver for EKS.
Note: I have tested a scenario with two EC2 Amazon Linux 2023 instances connected to same Fsx for Lustre file system and cannot produce any problems with supplementary groups when creating or reading files or folders secured with various groups, and users with membership of 5 supplementary groups.

Jan E · Accepted Answer

Hi Sarneet C
Thanks for your answer. It took me some time to test it myself to understand your answer. What I conclude from your answer and my tests are that POSIX permissions (chown user:groups) works with Fsx for Lustre, even if a user is only member of the group in his list of supplementary groups. ACL's does **not **work for a users' supplementary group memberships. ACL only works for a user's primary group.

So this test shows that users supplementary groups works with POSIX group permission.  If a directory has a group set with posix chmod, it works even for a users 6th supplemenatary group.
 
On a pristine EC2 instance that had not opened or listed /fsx/group6 directory before I tried this:
 
```
[root]# useradd demo1
[root]# groupadd -g 1011 -U demo1 group1
[root]# groupadd -g 1012 -U demo1 group2
[root]# groupadd -g 1013 -U demo1 group3
[root]# groupadd -g 1014 -U demo1 group4
[root]# groupadd -g 1015 -U demo1 group5
[root]# groupadd -g 1016 -U demo1 group6
[root]# sudo su demo1
[demo1]$ touch /fsx/group6/testfile2.txt
[demo1]$ ll /fsx/group6
total 1
-rw-rw-r--. 1 demo1 demo1 0 Nov 20 15:54 testfile.txt
-rw-rw-r--. 1 demo1 demo1 0 Nov 20 15:57 testfile2.txt
[demo1]$ ll /fsx/
total 25
drwxrwx---+ 2 root group6 25600 Nov 20 15:57 group6
```
As it shows, works with posix groups.

Sarneet_C · Answer

Hello,

Since FSx for Lustre doesn't currently support an LDAP integration for upcall, supplementary POSIX groups should be used to control file access. Here's an example where Supplementary groups and ACL do not work and based on ACL rule, access should have been granted:

```
$ sudo mkdir /mnt/fsx/demo
$ sudo chmod 750 /mnt/fsx/demo
$ sudo setfacl -m g:demo2:rwx /mnt/fsx/demo
$ ls -ld /mnt/fsx/demo
drwxrwxr-x+ 2 root root 14848 Jan 22 14:27 /mnt/fsx/demo

$ getfacl /mnt/fsx/demo
getfacl: Removing leading '/' from absolute path names
# file: mnt/fsx/demo
# owner: root
# group: root
user::rwx
group::rwx
group:demo2:rwx
mask::rwx
other::r-x

$ sudo su - demousr -c "touch /mnt/fsx/demo/testfile"
touch: cannot touch ‘/mnt/fsx/demo/testfile’: Permission denied
```
In this case, the Access is denied because MDT was not able to discover if `demousr` is a member of `demo2` when MDT validates ACL permissions, because identity_upcall is not working.

How does Fsx for Lustre handle supplementary POSIX groups without upcall to IDP?

Add your answer

Relevant content