Built using IMU 6DOF Angular positioning data to wirelessly control the movement of a vehicle car.
Implemented with remote Flask Web Server to view live feed as well as OpenCV object detection.
May - June 2021
INTRO
Create a vehicle capable of being controlled by intuitive hand movements in 6DOF with wireless communication via WIFI and a live remote feed to a flask server.
The overall end to end system architecture is illustrated in the block diagram below. On the hand control system, a raspberry pi zero W is powered by a USB battery pack. It reads from the I2C bus on the SDA and SCL lines to receive data from the MPU 6050. The data from the MPU 6050 is then processed through a series of thresholds that determine the motor speed for the motors. A UDP socket is used to transfer the data between the raspberry pi zero W and the raspberry pi 3. The raspberry pi 3 communicates with the motor hat using I2C to regulate the current, voltage polarity and strength sent to each of the respective motors.
The camera acts as an input into raspberry pi utilizing the CSI port on the raspberry pi along with a CCD to convert analog readings of photons from the camera to digital images that can be processed by the CPU. The feed from the camera is then sent to a webserver created by flask that allows the user to view the live feed remotely.
.png)
Gyroscope: Provides angular velocity (deg/s) data
Accelerometer: Measures proper acceleration
Challenge: The accelerometer output is noisy, hence we are unable to obtain reliable angular position from accelerometer alone due to frequent deviations in readings.
Solution: Implement a Complimentary Filter to effectively combine the two sensors by creating a bias towards the gyroscope in order to reduce the excessive noise from the accelerometer.
Since the complimentary filter is a fusion algorithm, it combines both gyro and accelerometer data. It uses gyroscope for short-term accuracy and accelerometer for long-term stability. The result is less noise and drift and a reliable angular position from IMU sensors.
.png)
The roll is used to control the radius of the turn.
The pitch is used to control how fast the vehicle moves. The closer the angle is to 0 degrees, the faster it is in the forwards direction.
SOFTWARE PROGRAMMING
The data is sent between processors on the vehicle and hand using a UDP socket. However, only one packet can be sent at a time. Therefore, to embed multiple variables into the packet, a '_' character is used to separate variables. Then on the receiving end, a separate script on the vehicle's raspberry pi decodes the byte into a string and then splits into its respective three variables in an array. Once this is done, they can be stored locally as variables. The following instructions are to simply set the motor speed for each motor according to their respective speeds.
Flask is a micro web frame work for python that enables simple web applications to be developed without the need for external libraries. Setting up the flask server with the video stream was a process of using OpenCV to create a capture object that could constantly return the real time frame of the camera to the web server. The web server is run directly from the raspberry pi 3 on the vehicle.
I used a multiprocessing library that supports spawning processes using an API similar to the threading module. Using the library, I spawned the process that controls the motors from the process that ran the flask server. In order to get the real time from another the vehicle control process, the flask process utilizes a queue to pass the motor speed and direction to the parent process. Then the motor speed and direction values are displayed on the live video feed using cv2.putText.
The Yolo models employ convolutional neural networks (CNN) to detect objects in real-time. However, a major obstacle that had to be put into consideration while coding this section of the project was the processing capabilities of the Raspberry Pi 3. This is mainly due to the lack of a GPU that can reduce the processing load for the CPU. In the early testing stages, using static images worked consistently at a very slow rate. However once switching the software to detecting real time from the camera, the frame rate of the camera dropped drastically, processing at below 1 frame per second, which is unsuitable for a moving vehicle. Therefore, in order to optimize object detection for the raspberry pi's low processing speed, an alternative approach is required. As an alternative I tried using Tiny-COCO which is an object detection, segmentation, and captioning dataset.