By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Rekognition custom labels "No predictions for this image" instead of "False Negative"

0

I'm training a Rekognition custom labels model. After finish the training I see the legend "No predictions for this image" in some files. All files are labeled with bounding boxes so my expectation was find a "False Negative" message for elements not detected by the model, instead of "No predictions for this image"

What means "No predictions for this image" and there is any difference with "False Negative" message?

Topics
Machine Learning & AI
Tags
Amazon Rekognition
Language
English
Carlos Garces
asked 3 years ago547 views
2 Answers
1

Hello, if you can open a customer support ticket and provide us with:

  1. your account ID
  2. your Project Version ARN
  3. the TestingDataResult/Input GT manifest
  4. the TestingDataResult/Output GT manifest
  5. the TestingDataResult/Validation GT manifest

then we can take a closer look at your model and give you more specialized information. Thank you.

AWS
reinaman
answered 3 years ago
profile picture
EXPERT
Giovanni Lauria
reviewed 10 months ago
0
Accepted Answer

Hi CGarces,

Thanks for reaching out for this problem. DetectCustomLabels does not return any labels/predictions with a confidence value that is lower than the MinConfidence value given in DetectCustomLabels API call. If you specify a value of 0, DetectCustomLabels returns all labels, regardless of the assumed threshold applied to each label. If you don't specify a value for MinConfidence, DetectCustomLabels returns labels based on the assumed threshold of each label. The assumed threshold is calculated based on the best F1 score achieved on the test dataset during model training for each label. You are probably not specifying a MinConfidence/threshold value.

Please specify MinConfidence parameter as 0, or a lower value to get some or all of predictions. Please also refer to the following resources for more information:

Please let us know if this problem continues for you. Thanks.

AWS
aws-yasarka
answered 3 years ago
profile picture
EXPERT
Giovanni Lauria
reviewed 10 months ago
  • Carlos Garces
    3 years ago

    I'm training the model, the messaged is raise by the AWS web interface. There is no option to set the confidence during training.

  • aws-yasarka
    3 years ago

    can you provide details about your training time and region ?

  • Carlos Garces
    3 years ago

    Training was done in eu-west-1 last one was 2022-07-06T17.41.23

  • aws-yasarka
    3 years ago

    Hi, If this happens in training, this means that your training or test dataset have not much enough samples. I understand that your training succeeded but possibly performance of trained model is not good. Please try again with a larger dataset. Thanks.

  • Carlos Garces
    3 years ago

    In my opinion is a bug. If the test dataset has a item labeled and the model can't detect it (due enough samples as you said), it must be set as "False Negative", according to the AWS documentation. "False negative – The Amazon Rekognition Custom Labels model doesn't predict that a custom label is present in the image, but the "ground truth" for that image includes this label" https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/im-metrics-use.html

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content