Durch die Nutzung von AWS re:Post stimmt du den AWS re:Post Nutzungsbedingungen
AWS re:Postre:Post

Running Spark jobs on ECS

0

Is running Spark job on ECS (Fargate) recommended ? K8s has support for running spark drivers .. is there something similar for ECS ? I understand there are other services (Glue, EMR ) for Spark on AWS, but would like to understand the feasibility / complexity for running Spark on ECS Fargate.

Themen
ServerlosBehälter
Tags
AWS FargateAmazon Elastischer Container-Service
Sprache
English
JackSparrow
gefragt vor 8 Monaten1240 Aufrufe
1 Antwort
1

Hello.

Running Apache Spark on ECS (especially Fargate) is not a standard approach and isn't as streamlined as using EMR or Kubernetes. ECS is primarily designed for container orchestration of microservices, but with some effort, you can run Spark jobs on it.(check this article https://datachef.co/blog/run-spark-applications-on-aws-fargate/)

Regards, Andrii

profile picture
EXPERTE
Andrii S
beantwortet vor 8 Monaten
profile picture
EXPERTE
Oleksii Bebych
überprüft vor 9 Tagen
  • JackSparrow
    vor 8 Monaten

    Thanks for the response. I understand it's not as streamlined as EMR or k8s. Having said that, I would specifically like to understand any known limitation / consideration to run Spark on ECS .

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen

Relevanter Inhalt