How do I pass images to Sagemaker inference endpoint

0

I have successfully deployed my Sagemaker inference endpoint (using the bring your own container method). The invocations endpoint is as follows:

@app.route("/invocations", methods=["POST"])
def transformation():
    """Do an inference on a single batch of data. In this sample server, we take data as CSV, convert
    it to a pandas data frame for internal use and then convert the predictions back to CSV (which really
    just means one prediction per line, since there's a single column).
    """

    # Get the file from the request
    file_list = list(flask.request.files.values())
    if not file_list:
        return flask.Response(response=json.dumps({'error': 'no file found'}), status=400, mimetype='application/json')

    # Get the first file in the list (assuming there is only one)
    img = file_list[0].read()

    # Process the image as needed...
    im2 = cv2.imdecode(np.frombuffer(img, np.uint8), cv2.IMREAD_UNCHANGED)

    results = ScoringService.predict(im2)

    xyxy = results.pandas().xyxy[0].values
    preds = xyxy.tolist()

    response_body = json.dumps({'predictions': preds})
    return flask.Response(response=response_body, status=200, mimetype='application/json')

I am having issues with calling the endpoint, is multipart/form-data accepted? Or do I need to pass my image as base64 and decode it in my Sagemaker endpoint?

2 Answers
1

Afraid I've so far failed to dig out the exact code & details, but I have played with form/multipart data with SageMaker endpoints last year when looking for an architecture to send extended metadata alongside video chunks or images (e.g. for stateful tasks like object tracking).

To my recollection there should be no fundamental blocker for using this content type with SageMaker, but I did find that Python itself (I was using standard library tools like email) was causing me problems when trying to serialize and extract multipart data containing non-base64-encoded binary parts. If I remember the thread correctly, I was trying to set the Content Type Encoding (see SO post 1, post 2) to stop email insisting on base64, but ran in to a bug (this one?) in handling of binary CTE messages.

I was doing request generation (a custom Serializer for the SM Python SDK) manually in Python and request parsing manually (via input_fn in a framework-based container) - rather than trying to build a fully-custom container... So for you the stack will be a bit different because I think Flask is trying to do the body processing for you already?

Either way my suggestion would be to verify locally (on notebook or wherever, just without SageMaker endpoint involved) whether you can construct and decode your multipart payload successfully.

For me at the time I couldn't get binary/UTF CTE working so was left with choice to either surrender to base64 in multipart, or switch to an alternative format. In the end I think I actually went with tar/zip archive as the request format treating each part as a "file" compressed and extracted entirely in memory... (This sample even returns a numpy npz compressed array where each item is a PNG byte string for an image - yuck!)

...But would be great if binary multipart is actually possible somehow. Sorry not exactly a clear answer, but hopefully useful 😓


Edit after comment - Only passing an image:

So I answered the above thinking you might have a use-case for sending multipart data of other types alongside the image... But per the first comment sounds like this wasn't the case! To clarify for anybody that only wants to send/receive an image as a request/response, you can just use your preferred efficient image format that the file is probably already stored in:

with open("myimg.png", "rb") as f:
    payload = f.read()  # Actual bytes of the PNG file

# Send with e.g. 'image/png' Content-Type

On the receiving end cv2.imdecode(...) should be able to load the image from the bytes.

One thing I would particularly avoid (which you will see in some online examples) is passing an image around as an uncompressed pixel array e.g. with numpy/etc: It's very bandwidth-inefficient and you'll be running in to the 6MB real-time endpoint payload size limit at much lower resolutions than with a proper compressed image format like PNG/JPEG/etc

AWS
EXPERT
Alex_T
answered a year ago
  • That was very helpful thank you. Got me thinking more about passing through flask...etc At the end this worked:

    Sending the image from client:

    with open(image_path, 'rb') as image_file: payload = image_file.read()
    response = requests.post(url, data=payload)

    And the server side (sagemaker container) would receive it like so:

    image_data = flask.request.data image_array = np.frombuffer(image_data, dtype=np.uint8) img = cv2.imdecode(image_array, cv2.IMREAD_COLOR)

    Essentially had to pass it this way.

  • Ahh gotcha - yes for only passing an image by itself that's a good pattern! Ideally you might set your request Content-Type header accordingly to whatever your image file is, but I don't think it'll actually be used by unless you start accepting multiple formats beyond what OpenCV can auto-detect - so no real impact... Have updated my answer to clarify for anybody else that's also not really needing multipart data. Glad you got it working!

0

If you wanna use AWS CLI here is how to pass image to SageMaker Endpoint:

aws sagemaker-runtime invoke-endpoint --endpoint-name myendpoint_name --body fileb://myimage.jpg --content-type=application/x-image outfile.txt

profile picture
EXPERT
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions