Real-Time Face Tracking

In this project, I’d like to share a real-time face tracking system using the DJI Tello drone. The setup utilizes Python and the djitellopy library to control the drone, while OpenCV is employed for face detection. Let’s dive into the implementation and the results!

Project Overview

The goal of this project was to create a drone capable of following and tracking a face using its onboard camera. The drone is programmed to take off, maintain a stable altitude, and use computer vision to detect and follow a face within its camera’s view.

Code Walkthrough

Here’s a breakdown of the code used to achieve face tracking:

import time
import numpy as np
from djitellopy import Tello
import cv2

# Initialize the drone and connect
uav = Tello()
uav.connect()
print(f"Battery: {uav.get_battery()}%")

# Start video stream and take off
uav.streamon()
uav.takeoff()
uav.send_rc_control(0, 0, 25, 0)
time.sleep(2.2)

# Frame dimensions
w, h = 640, 480

# Face detection range and PID controller parameters
fbRange = [10000, 35000]  # Experimental range for face area
udRange = [-50, 50]  # Experimental range for face vertical position
pid = [0.4, 0, 0.4]
pError_fb = 0
pError_ud = 0


def findFace(img):
    """
    Detects faces in the provided image using Haar Cascade.

    Args:
    - img: Image in which to detect faces.

    Returns:
    - img: Image with rectangles and circles drawn around detected faces.
    - info: List containing the center coordinates and area of the largest detected face.
    """
    faceCascade = cv2.CascadeClassifier("Resources/haarcascade_frontalface_default.xml")
    imgGray = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    faces = faceCascade.detectMultiScale(imgGray, 1.2, 8)

    myFaceListC = []
    myFaceListArea = []

    for (x, y, w, h) in faces:
        cv2.rectangle(img, (x, y), (x + w, y + h), (0, 0, 255), 2)
        cx = x + w // 2
        cy = y + h // 2
        area = w * h
        cv2.circle(img, (cx, cy), 5, (0, 255, 0), cv2.FILLED)
        myFaceListC.append([cx, cy])
        myFaceListArea.append(area)
    if len(myFaceListArea) != 0:
        i = myFaceListArea.index(max(myFaceListArea))
        return img, [myFaceListC[i], myFaceListArea[i]]
    else:
        return img, [[0, 0], 0]


def trackFace(uav, info, w, h, pid, pError_fb):
    """
    Controls the drone to track a face based on its position and area.

    Args:
    - uav: Tello drone object.
    - info: List containing the center coordinates and area of the largest detected face.
    - w: Width of the video frame.
    - h: Height of the video frame.
    - pid_fb: PID controller parameters for forward-backward movement.
    - pid_ud: PID controller parameters for up-down movement.
    - pError_fb: Previous error value for forward-backward PID controller.
    - pError_ud: Previous error value for up-down PID controller.

    Returns:
    - pError_fb: Current error value for forward-backward PID controller.
    - pError_ud: Current error value for up-down PID controller.
    """
    area = info[1]
    x, y = info[0]
    fb = 0
    ud = 0

    # Forward-backward control
    error_fb = x - w // 2
    speed_fb = pid[0] * error_fb + pid[2] * (error_fb - pError_fb)
    speed_fb = int(np.clip(speed_fb, -100, 100))
    print(area)

    if fbRange[0] < area < fbRange[1]:
        fb = 0
    elif area > fbRange[1]:
        fb = -20
    elif area < fbRange[0] and area != 0:
        fb = 20

    if x == 0:
        speed_fb = 0
        error_fb = 0

    # Up-down control
    error_ud = y - h // 2

    if udRange[0] < error_ud < udRange[1]:
        ud = 0
    elif error_ud > udRange[1]:
        ud = -20
    elif error_ud < udRange[0] and error_ud != 0:
        ud = 20

    if y == 0:
        error_ud = 0

    uav.send_rc_control(0, fb, ud, speed_fb)
    return error_fb


# Main loop for capturing video and tracking the face
while True:
    img = uav.get_frame_read().frame
    img = cv2.resize(img, (w, h))
    img, info = findFace(img)
    pError_fb = trackFace(uav, info, w, h, pid, pError_fb)
    cv2.imshow("Output", img)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        uav.land()
        break

Setting Up Haar Cascade for Face Detection

To use the face detection functionality, you need the Haar Cascade XML file. Follow these steps to download and set it up:

1. Download the XML File: You can download the haarcascade_frontalface_default.xml file from the OpenCV GitHub repository. Here is the direct link: Haar Cascade XML File.

2. Save the File: After downloading the XML file, create a folder named Resources in your project directory if it doesn’t already exist.

3. Place the File: Move the downloaded haarcascade_frontalface_default.xml file into the Resources folder.

Your project directory should look something like this:

/your-project
    /Resources
        haarcascade_frontalface_default.xml
    your_script.py

Key Components

1. Drone Initialization: The Tello object connects to the drone and starts video streaming.
2. Face Detection: Using OpenCV’s Haar Cascade Classifier, the script detects faces and determines their position and size.
3. Face Tracking: The drone adjusts its position based on the detected face, using a PID controller to correct its movements.
4. Real-Time Display: The live video feed with detected faces is displayed using OpenCV’s imshow function.

Results

The face tracking works effectively in real-time, allowing the drone to follow the face within its camera’s field of view. The PID controller ensures smooth adjustments and keeps the face centered in the frame. The system also maintains a suitable distance from the face based on its size.