Durch die Nutzung von AWS re:Post stimmt du den AWS re:Post Nutzungsbedingungen
AWS re:Postre:Post

cannot Launch instance (kernel) with custom Sagemaker Image

1

The custom image has been used for a long time.

For some reason, when I created new instance by using custom image from sagemaker studio, I keep get "Failed to start kernel" error.

Same image could work fine in the past.

Also I noticed that if failed first, and try with other instance, it could work. Some times if you kept trying on same instance there was a chance worked as well.

I hope sagemaker team aware of this issue.

Themen
Maschinelles Lernen & KI
Tags
Amazon SageMaker Studio Lab
Sprache
English
coolcoolji
gefragt vor 9 Monaten332 Aufrufe
4 Antworten
1

I'm experiencing the same error. My image has also been used for months without any problems. It started yesterday.

Also, cloudwatch logs don't show any errors.

As a temporary fix, you can start an instance with a default kernel (data science 3.0) and then switch to your custom image.

Gonzalo
beantwortet vor 9 Monaten
  • coolcoolji
    vor 9 Monaten

    Thanks, this works :)

    But for some instance type (by default, only allow 1 running app per domain), I cannot use this way to switch because after I created one by default kernel, it won't let you create second one with custom image.

    Hope Sagemaker team notice this issue and implement the fix asap.

1

Enter image description here

coolcoolji
beantwortet vor 9 Monaten
0
Akzeptierte Antwort

I have been worked with Sagemaker support and they implemented the changes, the instance should be created correctly with custom images.

during the issue, @Gonzalo's temp solution was working perfectly

coolcoolji
beantwortet vor 9 Monaten
0

I also tested running the notebook from "notebook job". Looks like the kernel from job scheduler can start kernel without custom image without error.

coolcoolji
beantwortet vor 9 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen

Relevanter Inhalt