Comprehend Medical Entity Detection Classification Accuracy

We encourage you to try it and see how well it works for your case.
There are some accuracy results described in the papers published by the team in peer reviewed conferences.

Improving Hospital Mortality Prediction with Medical Named Entities and Multimodal Learning
End-to-end Joint Entity Extraction and Negation Detection for Clinical Text
Dynamic Transfer Learning for Named Entity Recognition

For more information, here is our Re:Invent Presentation.
https://www.youtube.com/watch?v=cJ3eUPOXV4Q&t=474s

Hey,

Dealing with medical entity detection accuracy is no small feat. I've been down this road myself, and it's a rollercoaster of emotions and learnings.

Mj12, when it comes to Comprehend Medical, I totally understand your situation. Unfortunately, Amazon Comprehend Medical's precise training/test/CV error statistics aren't publicly available...(as I know). This can be a real challenge when dealing with client inquiries about model accuracy.

What I've found helpful is to create your own evaluation process. You could set up a representative dataset, manually annotate the entities, and then compare Comprehend Medical's predictions with your manual annotations. This could give you a better understanding of its performance specific to your use case.

Remember, communication with your client is key. Be transparent about the limitations and potential inaccuracies. You might want to consider alternative solutions or a hybrid approach combining Comprehend Medical with other tools.