Keyboard Navigation, Live Video Streaming, and Image Capture

In this post, I want to share a project that involves controlling a DJI Tello drone, displaying a real-time video feed, and capturing images using Python. We’ll leverage the djitellopy library for drone control, opencv for video processing, and a custom keypress module for handling keyboard inputs. By the end of this guide, you’ll have a functional system to control your drone with your keyboard and interact with it through live video.

Handling Keyboard Inputs

To control the drone with keyboard inputs, we need a custom module to handle key presses. Here’s how to create it:

# keypress.py
import pygame


def init():
    pygame.init()
    pygame.display.set_mode((400, 400))
    pygame.display.set_caption('Key Press Detection')


def get_key(key_name):
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            pygame.quit()
            exit()

    keys = pygame.key.get_pressed()
    key = getattr(pygame, f'K_{key_name}')
    return keys[key]


def main():
    if get_key("LEFT"):
        print("Left key pressed")
    if get_key("RIGHT"):
        print("Right key pressed")


if __name__ == '__main__':
    init()
    clock = pygame.time.Clock()
    while True:
        main()
        pygame.display.update()
        clock.tick(60)  # Limit the loop to 60 iterations per second

Explanation:

• pygame.init(): Initializes all imported Pygame modules.
• pygame.display.set_mode((400, 400)): Creates a display window with dimensions of 400x400 pixels.
• pygame.display.set_caption('Key Press Detection'): Sets the window title.
• get_key(key_name): Checks if a specified key is pressed and returns True or False.

The main function tests the custom keypress handling functionality by continuously polling for specific key presses, such as the left and right arrow keys, and printing corresponding messages. When you run this module as the main script, this function executes and demonstrates the keypress functionality.

Keyboard Navigation and Image Capture

Here’s how to integrate keyboard controls with drone movement and image capture:

from djitellopy import Tello
import keypress as kp
import cv2
import time

# Initialize keyboard module and drone
kp.init()
uav = Tello()
uav.connect()
print(f"Battery: {uav.get_battery()}%")
uav.streamon()


def get_keyboard_input():
    lr, fb, ud, yv = 0, 0, 0, 0
    speed = 50

    if kp.get_key("LEFT"):
        lr = -speed
    elif kp.get_key("RIGHT"):
        lr = speed

    if kp.get_key("UP"):
        fb = speed
    elif kp.get_key("DOWN"):
        fb = -speed

    if kp.get_key("w"):
        ud = speed
    elif kp.get_key("s"):
        ud = -speed

    if kp.get_key("a"):
        yv = -speed
    elif kp.get_key("d"):
        yv = speed

    if kp.get_key("e"):
        uav.takeoff()
    if kp.get_key("q"):
        uav.land()

    if kp.get_key('z'):
        cv2.imwrite(f'Resources/Images/{time.time()}.jpg', img)
        time.sleep(0.3)

    return [lr, fb, ud, yv]


def main():
    global img
    while True:
        vals = get_keyboard_input()
        uav.send_rc_control(vals[0], vals[1], vals[2], vals[3])

        img = uav.get_frame_read().frame
        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
        img = cv2.resize(img, (360, 240))
        cv2.imshow("Image", img)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    cv2.destroyAllWindows()


if __name__ == '__main__':
    main()

get_keyboard_input() Explanation

• lr, fb, ud, yv: Variables to store the drone’s movement commands for left-right, forward-backward, up-down, and yaw (rotation) respectively.
• speed: Defines the speed of the drone’s movements.
• kp.get_key("LEFT"): Checks if the left arrow key is pressed and adjusts the left-right movement (lr) accordingly.
• kp.get_key("UP"): Checks if the up arrow key is pressed and adjusts the forward-backward movement (fb).
• kp.get_key("W"): Checks if the ‘W’ key is pressed and adjusts the up-down movement (ud).
• kp.get_key("A"): Checks if the ‘A’ key is pressed and adjusts the yaw (yv).
• kp.get_key("E"): Checks if the ‘E’ key is pressed to make the drone take off.
• kp.get_key("Q"): Checks if the ‘Q’ key is pressed to make the drone land.
• kp.get_key('Z'): Checks if the ‘Z’ key is pressed to capture an image and save it with a timestamp.

Main Loop Explanation

• global img: Allows img to be accessed globally within the main() function. Making img global ensures that it can be accessed in the main() function after it has been updated in the get_keyboard_input() function.
• vals = get_keyboard_input(): Retrieves the current drone control values based on keyboard input.
• uav.send_rc_control(vals[0], vals[1], vals[2], vals[3]): Sends control commands to the drone for movement.
• img = uav.get_frame_read().frame: Retrieves the current frame from the drone’s video stream.
• img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB): Converts the image from BGR (default color format in OpenCV) to RGB (which displays colors correctly). You may not need this line.
• img = cv2.resize(img, (360, 240)): Resizes the image for better display and performance.
• cv2.imshow("Image", img): Displays the processed image in a window titled “Image”.
• if cv2.waitKey(1) & 0xFF == ord('q'):: Checks if the ‘q’ key is pressed to exit the loop and close the video window.
• cv2.destroyAllWindows(): Closes all OpenCV windows when exiting the loop.