Face tracking to drive a Pan/Tilt

This was an ad hoc project using OpenCV to detect faces in real-time and drive a simple pan/tilt platform with a camera attached.

A built-in laptop webcam ran the face detection and controlled the pan/tilt servos. A separate webcam attached to the pan/tilt platform did the actual recording.

Adobe Media Encoder and YouTube Live (in beta at the time) were used to stream the webcam while simultaneously recording it. The delay between live and streaming was 20+ seconds, but a benefit was that it was watchable on all devices, including iOS.

Behind the Scenes

To improve on this, servo movements could be smoothed and averaging applied to the face detection results. Also, these particular servos were really noisy, especially when working to hold a position. In a real world scenario, a Dynamixel based pan/tilt platform might be quieter.

Additional Face Tracking Tests

These videos show experiments using OpenCV to track a face in a typical single performer stage performance. The green circles are from using a haar frontal classifier and the blue circles are from using a haar profile classifier.

A near best case scenario:
opencv-test-008-good

A near worst case scenario due to over exposed lighting, glasses, mic in front face, and frequent 3/4 face profile presentation:
opencv-test-011-bad