How in large organization different groups login to EC2 instance

You should use Systems Manager, as it doesn't require any SSH keys and the only thing required, apart from attaching the appropriate instance role to our EC2 instance is to open outbound port 443.

You can find more info here

Generally, it is a good idea to use SSO with IAM Identity Center to issue temporary authentication information and then connect to the server using Systems Manager Session Manager.

By using SSO, authentication and authorization can be centrally controlled. Session Manager frees you from managing private keys and distributing public keys to EC2 instances.

It depends on your organization's requirements. If you do not have your requirements organized, you can try to organize them using the AWS Well-Architected Framework.

I think you would better off with a central Directory service.

In a previous Enterprise organisation where I worked, we have over 3000 Linux instances around the world. Its impossible to mange PEM files for each user of course. Session Manager doesnt scale and only logs you in as ec2-user which prevents controlling different access for different users.

We joined our Linux servers to an Active Directory domain. Here users would authenticate with Linux instances with an AD account. This negates the need to touch Linux machines every time there was an add move or change..

There is documenation here how to do this with EC2s https://docs.aws.amazon.com/directoryservice/latest/admin-guide/join_linux_instance.html

In large organizations, there are several standardized methods for logging into Amazon Elastic Compute Cloud (EC2) instances, each with its own use case and considerations:

1. Using SSH Keys (PEM files): This traditional method is suitable for scenarios where strict access control and auditability are required. It is recommended for securing critical systems or sensitive workloads, as the private keys are kept confidential and known only to authorized individuals or groups. However, managing and distributing SSH keys can become cumbersome, especially in dynamic environments where users or groups change frequently.

2. AWS Systems Manager Session Manager: This approach is recommended for scenarios where you need to establish secure connections to EC2 instances without exposing them to the internet or managing SSH keys. It is particularly useful for organizations with stringent security requirements or those that need to comply with regulatory standards. Session Manager requires minimal configuration and leverages IAM credentials for authentication, simplifying access management.

3. AWS Directory Service (e.g., AWS Managed Microsoft AD): Organizations that already have an existing Active Directory infrastructure can benefit from this method, as it allows users to leverage their existing AD credentials for single sign-on access to EC2 instances. This approach is suitable for organizations that prioritize user experience and want to minimize the overhead of managing additional credentials. However, it requires additional setup and configuration to integrate EC2 instances with the existing AD domain.

4. Password-based authentication: While not recommended for internet-facing instances due to security risks, password-based authentication can be a viable option for internal use cases within a Virtual Private Cloud (VPC). This method may be suitable for organizations with less stringent security requirements or those that prefer simplicity over advanced access controls. However, it is crucial to implement proper access management, monitoring, and follow best practices to mitigate potential security risks.

When choosing a method, organizations should consider factors such as security requirements, existing infrastructure, compliance needs, user experience, and operational overhead. It is also recommended to review and align with industry best practices and organizational policies to ensure a secure and compliant implementation.