I built a web app with ReactJs as front end and FastAPI as back end. With the below-mentioned FastAPI code, I was able to access to webcam and capture users' facial expressions and run some ML algo on the recorded video. The code was working fine on my local machine on Mac.
cap = cv2.VideoCapture(1)
with mp_holistic.Holistic(
min_detection_confidence=0.5,
min_tracking_confidence=0.5) as holistic:
while cap.isOpened() and not stop_event.is_set():
success, image = cap.read()
if not success:
continue
image.flags.writeable = False
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
results = holistic.process(image)
image.flags.writeable = True
image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
mp_drawing.draw_landmarks(
image,
results.face_landmarks,
mp_holistic.FACEMESH_CONTOURS,
landmark_drawing_spec=None,
connection_drawing_spec=mp_drawing_styles
.get_default_face_mesh_contours_style())
if(results.face_landmarks == None):
number_of_frames_with_no_face += 1
else:
number_of_frames_with_face += 1
number_of_frames_with_face_periodic += 1
cap.release()
cv2.destroyAllWindows()
Now when I am running the same code on an EC2 Ubuntu instance(Instance type - g4dn.xlarge as I was told that it is required for GPU), I am getting the below error.
Aug 12 17:38:45 ip docker[769]: back_end-web-1 | INFO: - "GET /analysis/start?param=presentation&user_id=54 HTTP/1.1" 200 OK
Aug 12 17:38:45 ip docker[769]: back_end-web-1 | [ WARN:0@132.006] global cap_v4l.cpp:982 open VIDEOIO(V4L2:/dev/video1): can't open camera by index
Aug 12 17:38:45 ip docker[769]: back_end-web-1 | [ERROR:0@132.006] global obsensor_uvc_stream_channel.cpp:156 getStreamChannelGroup Camera index out of range
Aug 12 17:38:46 ip docker[769]: back_end-web-1 | INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Aug 12 17:38:46 ip docker[769]: back_end-web-1 | [ WARN:0@132.241] global cap_v4l.cpp:982 open VIDEOIO(V4L2:/dev/video1): can't open camera by index
Aug 12 17:38:46 ip docker[769]: back_end-web-1 | [ERROR:0@132.241] global obsensor_uvc_stream_channel.cpp:156 getStreamChannelGroup Camera index out of range
Aug 12 17:38:46 ip docker[769]: back_end-web-1 | ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
When I checked the machine with v4l2-ctl --list-devices
command, I got the response Cannot open device /dev/video0, exiting.
.
Questions -
- Would The logic(of accessing webcam of user using webapp) even work on an AWS cloud?
- If yes, then what configuration changes do I need to make to me EC2 instances so that the backend can access user's webcam and start analysis?
Thanks a lot @Riku , this is exactly what I thought. I think I will have to use Websosckets to achieve the result, sending frame-by-frame data to backend from frontend and then do the analysis there.
Yes, WebSockets can be used to send images from the front end to the back end FastAPI. https://fastapi.tiangolo.com/advanced/websockets/