In SageMaker, how to get the value from Http Header?

0

I have a machine learning classification model that was trained outside of SageMaker. The model is in Scikit-learn format. To run this model, the preprocessing step requires the binary content of a file and its metadata, such as the filename, author, and creation date to generate the feature vector. Since the payload is used to send the binary content, I utilize the Header to send additional information to SageMaker.

Referring to this post, I created a function called input_handler(data, context) in the inference.py file to retrieve the custom header value. However, this function was not invoked during the process. So I am not sure if the function input_handler works for SKLearnModel or not. Do we have other alternatives to get values from the header?

I used the class sagemaker.sklearn.SKLearnModel for deploying the model."

This is how I deployed the model

from sagemaker.sklearn import SKLearnModel
from sagemaker import get_execution_role


role = get_execution_role()
model_path = f's3://{bucket_name}/{model_name}.tar.gz'
entry_point = 'inference_ransomware.py'
dependencies = ...

model = SKLearnModel(
    model_data=model_path,
    role=role,
    entry_point=entry_point,
    dependencies=dependencies,
    framework_version='1.2-1'
)

predictor = model.deploy(
    initial_instance_count=1,
    instance_type='ml.m4.xlarge',
)

filename = ...

with open(filename, mode='rb') as file:
    byte_data = file.read()
    predictor.predict(byte_data, initial_args={'CustomAttributes': json.dumps({"filename": filename})})
Ken
질문됨 일 년 전445회 조회
2개 답변
0

Hey Ken,

As you said since, the input_header does not have access to the HTTP headers, its context parameter only contains information about the model and the system environment. A potential workaround could be to include the metadata in the body of the request along with the binary content, and then parse it in the input_handler or input_fn function. However, I realize this would require changes to both the client sending the request and the inference script so not sure if thats workable in your case, but just a thought.

-Zac

profile picture
Zac Dan
답변함 일 년 전
0

Hi Ken,

If you would like to add custom values from your HTTP header to invoke your SageMaker Endpoint, you can have a look into using CustomAttributes when invoking your Endpoint as in [1,2]. CustomAttributes can be used to provide additional information about an inference request or response. Amazon SageMaker strips all POST headers except those supported by the InvokeEndpoint API action and you can use the CustomAttributes header to pass metadata to the inference request or response.

I hope this helps and should you still face issues, you can reach out to AWS Premium Support for further assistance. Kindly note that AWS Support can provide guidance but wouldn't be capable of re-coding your function as it's out of scope.

References

[1] https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html [2] https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-runtime-now-supports-the-customattribute-header/

답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠