diff --git a/Acknowledgements/contributors.html b/Acknowledgements/contributors.html index 4289e30..bb70331 100644 --- a/Acknowledgements/contributors.html +++ b/Acknowledgements/contributors.html @@ -49,11 +49,11 @@
@@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • - + diff --git a/Examples/adb.html b/Examples/adb.html index 6979df0..eb46f57 100644 --- a/Examples/adb.html +++ b/Examples/adb.html @@ -5,7 +5,7 @@ - Using ADB and IP | Building Mobile Game Solvers + Using Image Processing | Building Mobile Game Solvers @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • - + diff --git a/Examples/circuit_ip.html b/Examples/circuit_ip.html index ad3aabc..4ed7451 100644 --- a/Examples/circuit_ip.html +++ b/Examples/circuit_ip.html @@ -5,7 +5,7 @@ - Using Electronic Circuits & IP | Building Mobile Game Solvers + Using Image Processing and Electronics | Building Mobile Game Solvers @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • - + - + diff --git a/Examples/move.html b/Examples/move.html index f2aebd1..15d56a0 100644 --- a/Examples/move.html +++ b/Examples/move.html @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • diff --git a/Examples/piano_tiles.html b/Examples/piano_tiles.html index 067bfde..193604b 100644 --- a/Examples/piano_tiles.html +++ b/Examples/piano_tiles.html @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • diff --git a/Examples/playing_the_song.html b/Examples/playing_the_song.html index 3a9e9e7..4378349 100644 --- a/Examples/playing_the_song.html +++ b/Examples/playing_the_song.html @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • - + diff --git a/Examples/stick_hero.html b/Examples/stick_hero.html index a216016..0314b52 100644 --- a/Examples/stick_hero.html +++ b/Examples/stick_hero.html @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • - + diff --git a/Examples/tic_tac_toe.html b/Examples/tic_tac_toe.html index 415a2ee..b8af9a3 100644 --- a/Examples/tic_tac_toe.html +++ b/Examples/tic_tac_toe.html @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • - + diff --git a/ImageProcessing/Matlab/debugging.html b/ImageProcessing/Matlab/debugging.html index b6c01af..83306b7 100644 --- a/ImageProcessing/Matlab/debugging.html +++ b/ImageProcessing/Matlab/debugging.html @@ -37,7 +37,7 @@ - + @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • diff --git a/ImageProcessing/Matlab/thresholding.html b/ImageProcessing/Matlab/thresholding.html index 0b41014..0915de3 100644 --- a/ImageProcessing/Matlab/thresholding.html +++ b/ImageProcessing/Matlab/thresholding.html @@ -49,11 +49,11 @@
    @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -
  • - + - + diff --git a/Introduction/0-adb.html b/Introduction/0-adb.html new file mode 100644 index 0000000..70b9430 --- /dev/null +++ b/Introduction/0-adb.html @@ -0,0 +1,763 @@ + + + + + + + + Setting up ADB tool | Building Mobile Game Solvers + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + + +
    + +
    + +
    +
    + + +
    +
    + + +
    + +

    Setting up adb tool

    +

    Android Debug Bridge (adb) is a library that allows the computer to communicate with attached electronic devices. We will be using adb tool to mostly capture screenshots of the mobile screen, simulate virtual touches and virtual swipes.

    +

    These are the steps that you have to follow to setup adb tool.

    +

    Step 1: Installing Java JDK or Android SDK

    +

    We will need Java JDK or Android SDK in order to be able to use the adb tool. In general, most people have Java JDK already installed, so you might not have to do anything here. But if you don't have it installed, install it.

    +

    Step 2: Enable USB Debugging

    +

    To use adb with your Android device, you have to enable USB debugging first. You will find this in the developer options menu inside settings. If the developer options menu is not available, go into about phone menu and tap the build number 10 times to enable developer options menu.

    +

    Step 3: Download adb tool files

    +

    Download the files in the provided link. You should have these files in the directory that you are working, or you can add adb.exe to your path in environment variables.

    +

    Step 4: Testing

    +

    To test whether ADB is working properly, connect your Android device to your computer using a USB cable and run the following command in the command prompt. This should list the

    +
    adb devices
    +

    Also run the following command on MATLAB

    +
    system('adb devices')
    +
    +

    The output of the command should look similar to this if your Android device is connected properly and detected.

    +
    List of devices attached  
    +T038509BHC    device
    +
    +

    If your device is connected but nothing appears in the list, you'll need to install the appropriate drivers for your phone. You can find the drivers from Universal drivers (http://adbdriver.com/downloads/) or Android developers website (http://developer.android.com/tools/extras/oem-usb.html#InstallingDriver).

    +

    Open the Device Manager (click Start, type Device Manager, and press Enter), locate your device, right-click it and select Properties. You may see a yellow exclamation mark next to the device if its driver isn't installed properly. You might also have to force windows to use the installed drivers for your phone.

    +
    Basic ADB commands
    +

    Here are some commands that can be used with adb tool.

    +

    To pull the image into working directory.

    +
    adb pull
    +

    To take a screenshot of the device connected.

    +
    adb shell screencap
    +

    To tap on the screen at coordinates (x,y).

    +
    adb shell input tap x y
    +

    To swipe the screen from coordinates (x1, y1) to coordinates (x2,y2) with a delay of w milliseconds.

    +
    adb shell input swipe x1 y1 x2 y2 w
    +

    To remove screenshot that has been stored.

    +
    adb shell rm
    +

    That's all you need to know about adb tool. Next, let us cover each of the approaches in detail.

    + + +
    + + +
    +
    +
    + + + + + + + +
    +
    + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Introduction/1-image.html b/Introduction/1-image.html new file mode 100644 index 0000000..990ad38 --- /dev/null +++ b/Introduction/1-image.html @@ -0,0 +1,752 @@ + + + + + + + + Using Image Processing | Building Mobile Game Solvers + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + + +
    + +
    + +
    +
    + + +
    +
    + + +
    + +

    Using Image Processing

    +

    In this approach, we use a combination of image processing and adb tool library to automate the games.

    +

    Approach

    +

    image1

    +

    Fig: The image depicts the block diagram of the approach using adb tool and image processing works for solving the games

    +

    We use adb tool library to take a screenshot of the phone screen and save it on the sdcard, and then use the next command to send over the image to the computer. Next, we use a set of image processing techniques to extract relevant features in the image that are required. Depending on the features, we can simuate the toch or swipe virtually again using adb tool.

    +

    We can run the above steps in a loop to automate a game that needs repition or has multiple levels.

    +
    Advantages of this approach
    +
      +
    • We can get direct screenshots of our android phone because of which we get the accurate pixels values of the images.
    • +
    • Since we access the locations on the screen through their coordinates, we get accurate touch and swipe.
    • +
    • Circuits are not required.
    • +
    • Complex algorithms can be implemented efficiently in MATLAB.
    • +
    • Everything happening on the screen is visible to us unlike as in the case of electronic circuit which covers the screen.
    • +
    +
    Disadvantages of this approach
    +
      +
    • Since ADB is taking the screenshots and sending it to laptop ,it requires some amount of time to perform that operation.Thus, its processing is slow of about some seconds.
    • +
    • ADB tool takes time to set up.
    • +
    • ADB tool cannot perform curved swipe.
    • +
    +

    Games which can be solved using ADB tool

    +

    An Android game Stick Hero and Find the difference can be solved using this ADB tool because the game is not time bounded and we just need to simulate virtual touch to play these game.

    + + +
    + + +
    +
    +
    + + + + + + + +
    +
    + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Introduction/2-electronic.html b/Introduction/2-electronic.html new file mode 100644 index 0000000..a259c7a --- /dev/null +++ b/Introduction/2-electronic.html @@ -0,0 +1,751 @@ + + + + + + + + Using Electronics | Building Mobile Game Solvers + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + + +
    + +
    + +
    +
    + + +
    +
    + + +
    + +

    Using Electronics

    +

    This is a way to automate the mobile game without using any internal functions of Android library by using clever electronic circuitry placed on top of your phone.

    +

    Approach

    +

    image9

    +

    Fig. The image depicts the block diagram for a typical electronic circuit that can solve a game on the mobile

    +

    The microcontroller senses the inputs from a sensor that is used to detect what's on the screen, analyses the data using a logic that has been programmed into it and sends appropriate commands to the touch circuitry.

    +

    Refer to the electronics chapter in the book to learn in detail about who the sensing and touching circuits work.

    +

    Advantages

    +
      +
    • Since ADB tool processing is slow as discussed earlier, so it become difficult to solve time bounded game. This problem can be overcome by using electronics circuits.
    • +
    +

    Disadvantages

    +
      +
    • External condition affects the working of the circuits.
    • +
    • Setting up the touch part of the circuit takes some time.
    • +
    • It becomes difficult to debug the circuit.
    • +
    • It's difficult to write complicated algorithms using Arduino.
    • +
    • Not many games can be solved because we can not implement Machine Learning to Arduino.
    • +
    • Circuit required to simulate swipe becomes a bit difficult.
    • +
    +

    Games which can be solved using this method

    +

    Piano Tiles and Ready Steady Bang are some Android game which can be solved using this method because in these we just need to detect a particular word or a small part using sensors and then simulating touch using electronic circuits.

    + + +
    + + +
    +
    +
    + + + + + + + +
    +
    + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Introduction/3-both.html b/Introduction/3-both.html new file mode 100644 index 0000000..3f0c5ba --- /dev/null +++ b/Introduction/3-both.html @@ -0,0 +1,750 @@ + + + + + + + + Using Image Processing and Electronics | Building Mobile Game Solvers + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + + +
    + +
    + +
    +
    + + +
    +
    + + +
    + +

    Using Electronics and Image Processing

    +

    In this method we will be using the concepts of electronics and image processing together to automate the game.The Basic concepts of image processing which we are going to use is described in detail in the next chapter of this book.

    +

    Approach

    +

    image11

    +

    Fig.The image depicts the connection between the computer,microcontroller and the phone

    +

    We use a webcam which takes the pictures of the mobile screen and sends it to MATLAB so that it can be used for the next processing step.The MATLAB algorithm processes the image according to our need and generates the duration of touch and sends it to the microcontroller(Arduino here),which in turn is used to simulate the touch on the phone screen using certain circuits. +For Example, In the game named piano tiles,our webcam will capture the image of the screen and send it to the laptop where we can separate the white and black tiles and accordingly we can send the information to the Arduino to simulate a touch on the particular area of the screen where the black tiles are present.

    +

    Advantages

    +
      +
    • This method is mostly used to solve more complex games for example- an Android game Flappy Bird.
    • +
    • Faster than ADB tool .
    • +
    • Debugging becomes easier.
    • +
    +

    Disadvantages

    +
      +
    • In some Android games we notice that background changes after completion of a level, so image processing for such games becomes difficult.
    • +
    • Complex circuits are required.
    • +
    • Since the amount of light affect the image taken by webcam, thus external conditions affects the working of the circuits.
    • +
    • Setting the touch part of circuit becomes difficult.
    • +
    +

    We can enhance the efficiency of this method by using OpenCV instead of MATLAB.

    + + +
    + + +
    +
    +
    + + + + + + + +
    +
    + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Introduction/index.html b/Introduction/index.html index fd93059..2906c00 100644 --- a/Introduction/index.html +++ b/Introduction/index.html @@ -37,7 +37,7 @@ - + @@ -53,7 +53,7 @@ data-chapter-title="Introduction" data-filepath="Introduction/README.md" data-basepath=".." - data-revision="Fri Feb 12 2016 13:26:45 GMT+0530 (IST)" + data-revision="Wed Feb 17 2016 12:29:41 GMT+0530 (IST)" data-innerlanguage=""> @@ -113,112 +113,61 @@ - - - -
  • - - - - - - - 3. - - Electronics - - - -