Emr serverless network configs

2

Hi šŸ‘‹, I have a couple of questions in emr serverless networking configuration.

  1. I know public subnet is not recommended in backend, but it is not allowed in emr serverless. Any reason why only private subnetting?
  2. If my spark only needs s3 to interact, then dedicated vpc required to be configured?

Thanks in advance

Mark
posta 8 mesi fa610 visualizzazioni
1 Risposta
3
Risposta accettata

Hello,

  1. EMR serverless creates private ENI for each worker which will consume an IP address on the private subnet where it is launched by design. So, it is recommended only to private subnet in EMR serverless world. Refer here for more.
  2. Without VPC connectivity, a job can access some AWS service endpoints in the same AWS Region. These services include Amazon S3, AWS Glue, Amazon CloudWatch Logs, AWS KMS, AWS Security Token Service, Amazon DynamoDB, and AWS Secrets Manager. You can enable VPC connectivity to access other AWS services through AWS PrivateLink, but you aren't required to do this. To access external services, you can create your application with a VPC. Refer here
AWS
TECNICO DI SUPPORTO
con risposta 8 mesi fa
profile picture
ESPERTO
verificato 7 mesi fa
profile pictureAWS
ESPERTO
verificato 8 mesi fa
  • Thanks, and if I go with vpc and use s3 endpoint to interact, will there be any performance implications like the traffic hops might be not optimized ?

  • No performance impact. Basically the traffics of VPC endpoints goes within AWS networks which is indeed secure and do not bring latency in traffic.

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande