SageMaker CreateModelPackage input argument semantics and SageMaker SDK AlgorithmEstimator

0

In SageMaker, creating a model package from an algorithm looks like this:

boto3.Session().client('sagemaker').create_model_package(
    ModelPackageName = ''name",
    ModelPackageDescription = ''desc",
    SourceAlgorithmSpecification = {
        'SourceAlgorithms': [{
            'AlgorithmName': 'AlgorithmName',
            'ModelDataUrl': 's3://path/to/model/data'
        }]
    }
)

My problem is around what to provide as AlgorithmName. Looking at the API docs here shows the argument regex accepts both an ARN and a Name. The problem is when to use what

If I were to use an algorithm from the marketplace then passing over the algorithm ARN succeeds, but passing over the algorithm name fails with:

ClientError: An error occurred (ValidationException) when calling the CreateModelPackage operation: Algorithm arn:aws:sagemaker:eu-central-1:123456789012:algorithm/scikit-decision-trees-15423055-57b73412d2e93e9239e4e16f83298b8f does not exist.

However, if I have a custom algorithm I created myself (meaning not from the marketplace), then it is the other way around - using its name fails while using the ARN succeeds.

Had it been just this mixup I would have solved it by using the right one in the right context. However, when using the SageMaker SDK I have no control over what is passed to the API call. Specifically, when I try to use a custom algorithm with the AlgorithmEstimator().fit().transform() paradigm a create_model_package call is made with the custom algorithm ARN - which fails. This prevents me from using SageMaker SDK with custom algorithms. I don't know whether the solution is to have the API accept both options in both scenarios or for the SageMaker SDK to use the algorithm's name instead of the ARN.

asked a year ago101 views
1 Answer
0

The issue you’re facing seems to be a common one when using AWS SageMaker’s SDK with custom algorithms. The AlgorithmName parameter in the create_model_package function can accept both an ARN (Amazon Resource Name) and a name. However, the behavior differs based on whether the algorithm is from the marketplace or a custom one.

For marketplace algorithms, the ARN works, but the name doesn’t. For custom algorithms, the name works, but the ARN doesn’t. This discrepancy can cause confusion and issues when using the SageMaker SDK, as it automatically passes the ARN, which fails for custom algorithms.

Unfortunately, there isn’t a straightforward solution to this issue as it seems to be a limitation with the SageMaker SDK itself. One potential workaround could be to manually create the model package using the AWS CLI or the AWS Management Console, specifying the algorithm’s name instead of the ARN for custom algorithms.

profile picture
EXPERT
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions