Welcome to the CYBATHLON Vision Assistance Race repository! Here you will find the code for a race that helps blind people identify and sort different colors according to their brightness.
Course of Designing Mechatronic System Coordinated by Fabien Verité
Table of Contents
The objective is for pilots to correctly sort a given set of colors by both base color and brightness. The colors are initially presented on a clothesline on the left, and the pilots must then correctly sort and hang them on a clothesline on the right. Our proposed solution is a standalone vision system, comprising of a Raspberry Pi 3B as the central processing unit, a power bank for power supply, a vision camera for image capture, and a Bluetooth speaker or headphones for audio output. The system is housed in a 3D-printed case with a belt for waist-worn portability.
- Raspberry PI 3
- Python 3.9
- OpenCV
- GTTS
- Solidworks
Before installing the requirements for LINUX users this tasks has to be done: Install mpg123
sudo apt-get update
sudo apt-get install mpg123 # or mpg321
pip3 install mpyg321In the file requirements.txt are inserted the packages needed in the Raspberry PI to run the program.
- intallation command
pip install -r requirements.txt
- You may want to Install just the requirements of this package
pip install -r req.txt
Set up SSH Linux Users
- Download this package:
sudo apt install openssh-server- Then log in via terminal with the command:
ssh "username of raspberry" ex:name@namepi
password: "password"Windows Users
-
Download PuTTY
-
Then log in via PuTTY: insert the full username: name@namepi
In order to run the code the file main.py has to be launched. Once you are in the same file folder you can run it using
python main.pyThe code runs both on Windows and Rasbperry PI system. However, to increase the performances and reduce the computational power, comment the cv2.imshow() command in the main.py script, at line 167. Note that The completion of this task is mandatory when operating on a Raspberry device, missing to do so will result in the script not functioning properly.
Acceleration of the system can be achieved by modifying certain parameters in the primary Python file, though this may result in a decreased robustness of the system as a whole. The tunable parameters can be found in constatants.py include:
UP_LIM(Deafault = 30): percentage of colored pixel present in the analysed frameLOW_LIM(Deafault = 10): percentage of colored pixel present in the analysed frame
if c1_perc > cs.UP_LIM and c2_perc < cs.LOW_LIM:
c1_num += 1
name = c1 + str(c1_num)
elif c2_perc > cs.UP_LIM and c1_perc < cs.LOW_LIM:
c2_num += 1
name = c2 + str(c2_num)
else:
name = 'None'The parameters UP_LIM and LOW_LIM serve as thresholds in the function utilities.recognize_color. The code snippet determines the color of a t-shirt in an analyzed frame based on the number of pixels of color 1 and color 2. If the number of pixels of color 1 exceeds UP_LIM and the number of pixels of color 2 is less than LOW_LIM, the t-shirt is assigned color 1. Conversely, if the number of pixels of color 2 exceeds LOW_LIM and the number of pixels of color 1 is less than UP_LIM, the t-shirt is assigned color 2. Otherwise the color assigned is None.
N_FRAME(Deafault = 5): number of consequtive frames that have to be acquired to assure stabilityFRAME_TOLL_UP(Deafault = 12) andFRAME_TOLL_LOW(Deafault = 2): are implemented to ensure that the t-shirt being analyzed is in a stationary position directly in front of the camera, rather than just temporarily passing by.WHITE_THRESHOLD(Deafault = 20): percentage of pixel that have to become white (meaning that the t-shirt is in front of the camera) before the analysis is performed.
if white_pixel_percentage > cs.WHITE_THRESHOLD:
frame_list.append(white_pixel_percentage) # Add the white pixel percentage to the frame list
# N_FRAME (minimum to assure stability, so that the tshirt is in fron of us and not just passing by)
if len(frame_list) >= cs.N_FRAME:
# Check if the difference between the maximum and minimum values in the frame list is within the
# specified tolerance range (minimum to assure stability, so that the tshirt is in fron of us and
# not just passing by)
if abs(max(frame_list) - min(frame_list)) < cs.FRAME_TOLL_UP and abs(
max(frame_list) - min(frame_list)) > cs.FRAME_TOLL_LOW:Moreover the colour mask for the segmentation can be modified in the same file:
color_dict_HSV = {'black': [[180, 255, 30], [0, 0, 0]],
'white': [[180, 18, 255], [0, 0, 231]],
'red1': [[180, 255, 255], [159, 50, 70]],
'red2': [[9, 255, 255], [0, 50, 70]],
'green': [[89, 255, 255], [36, 50, 70]],
'blue': [[130, 255, 255], [70, 10, 2]],
'yellow': [[35, 255, 255], [25, 50, 70]],
'purple': [[158, 255, 255], [129, 50, 70]],
'orange': [[50, 255, 255], [5, 50, 70]],
'gray': [[180, 18, 230], [0, 0, 40]]}Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE.txt for more information.
- Marco Milanesi - [email protected]
- Abdelghani Msaad - [email protected]
- Joseph Dittrick - [email protected]
