Object Flaw Detector

Details
Target OS:	Ubuntu* 16.04 LTS
Programming Language	C++
Time to complete:	30 min

Introduction

This object flaw detector application detects the anomalies present in the objects moving on a conveyor belt. The application identifies the number of objects moving on the conveyor belt and checks for the presence of defects in the color and orientation of the object. The objects are checked for the presence of cracks. The ones containing cracks are marked as defective.

Requirements

• Ubuntu 16.04
• OpenVINO™ toolkit
• Intel® System Studio 2018

How it works

• This application takes the input from a video camera or a video file for processing.

• Orientation defect detection: Get the frame and change the color space to HSV format. Threshold the image based on the color of the object using inRange function to create a mask. Perform morphological opening and closing on the mask and find the contours using findContours function. Filter the contours based on the area. Perform PCA on the contours to get the orientation of the object.

• Color anomaly detection: Threshold the image based on the defective color of the object using inRange function. Use the mask obtained from the inRange function to find the defective area.

• Crack detection: Transform the image from BGR to Grayscale format using cvtColor function. Blur the image using blur function to remove the noises. Use the contours found on the blurred image to detect the cracks.

• Save the images of defective objects in their respective folders. For example, objects with color defect are saved in color folder, objects containing cracks are saved in crack folder and objects with no defect are stored in no_defect folder.

Setting the build environment

On the system, open Intel® System Studio 2018 and choose your workspace.

1. Click File -> New -> Project -> Application Development.
2. Select C/C++ project to build and run on this Linux Operating System.

3. Select Intel Complier -> Matrix Multiplication -ICC example and change the name of the project to object_flaw_detector.

4. Click Next. Select Complier for the project as GNU Compiler Collection (GCC). Click Finish.

5. Delete the file named compiler_sample.cpp(example code) from the Project Explorer.
6. Click File -> New -> File. Select the parent folder and name the new file as product-flaw-detector.cpp.
7. Copy the code from product-flaw-detector.cpp of this repository to the newly created file.

Add Include Path

1. Select Project -> Properties -> C/C++ General -> Paths and Symbols
2. Select Includes -> GNU C++ and Click on Add...
3. Add /opt/intel/computer_vision_sdk_<version>/opencv/include to include the path of OpenVINO™ toolkit.

Add Libraries

1. Select Project -> Properties -> C/C++ Build -> Settings -> GCC C++ Linker -> Libraries
2. Add /opt/intel/computer_vision_sdk_<version>/opencv/lib to Library Search Path (-L)
3. Add opencv_core, opencv_highgui, opencv_imgproc, opencv_imgcodecs, opencv_videoio one at a time to the Libraries (-l)

Select Project -> Properties -> C/C++ Build -> Settings -> GCC C++ Compiler -> Dialect. Select the Language standard as ISO C++ 11(-std=c++0x) and click OK. If a dialog with the "Changes made will not be reflected in the index until it is rebuilt. Do you wish to rebuild it now?" message is shown, Click Yes.

Build the project

1. Select Project -> Build Project and click OK.

Run the project

1. Select Run -> Run Configuration.. -> C/C++ Application ->. Choose the project object_flaw_detector.
2. Click on Arguments and specify the path of the video under Program Arguments.
3. Click Run.

Run object_flaw_detector as remote application on UP²* then display the results on your laptop

1. Preparation:

   a. To run a graphic application on remote device and display on your host, here we use X11 Forwarding with SSH, open another Terminal on your laptop, type below command and keep this terminal open

    ssh [email protected] -X
   

   b. To get the read and write authority of the camera on the remote device, we need to type:

    sudo chmod 666 /dev/video0
   

   Note: upsquared@upsquared-UP-APL01:~$ sudo chmod 666 /dev/video0

   c. Download the object-flaw-detector.mp4 file from the data folder in this repository to the home directory of the Up2 board :

   wget https://github.com/SSG-DRD-IOT/object-flaw-detector-up2-lab/raw/master/data/object-flaw-detector.mp4
   

2. Right click object_flaw_detector from the Binaries folder, select Run As -> Run Configurations..., then doulbe click C/C++ Remote Application, it will generate a configuration named interactive_face_detection_sample, rename it to object_flaw_detector_remote, click Apply

3. Click New button after Connection:
   

   

   
   

4. In the droplist, choose SSH
   

   

   
   

5. Then type IP address of your UP² board(10.42.0.xxx), username: upsquared, choose password based authentication, then type upsquared as password, then click Finish
   

   

   
   

6. When asked for an eclipse secure password enter "upsquared" - the root password for the Up2 Board

7. In Remote Absolute File Path for C/C++ Application, type:

    /home/upsquared/object_flaw_detector
   

8. In Commands to execute before application, type:

   Note: Remember to open a Terminal on your laptop, type ssh [email protected] -X and keep this terminal open, for running a GUI application remotely and display it locally

    export DISPLAY=localhost:10.0
    source /opt/intel/computer_vision_sdk/bin/setupvars.sh
   

   
   

   

   
   

9. Click Arguments tag, add below arguments in Program arguments: then click Apply and OK

/home/upsquared/object-flaw-detector.mp4

To run the application from the UP2 terminal

• Open an SSH terminal to the Upsquared Board:

ssh [email protected] -X

• Updates to several environment variables are required to compile and run OpenVINO™ toolkit applications. Run the following script on the terminal to temporarily set the environment variables.

  source /opt/intel/computer_vision_sdk/bin/setupvars.sh

• Clone this repo onto the Up2 Board:

git clone https://github.com/SSG-DRD-IOT/object-flaw-detector-up2-lab.git

• Go to object flaw detector directory.

  cd <path-to-object-flaw-detector>

• Create a directory build.

  mkdir build

• Build the project.

  cd build 	
  cmake ..
  make
  

• Run the application.

  ./product-flaw-detector <VideoFile>

  Note: To get the input video from the camera, use the following command: ./product-flaw-detector cam