MapReduce

An implementation of MapReduce with dynamic node allocation

Table of Contents

1. About The Project
   • Built With
2. Getting Started
3. Contributing
4. Contact

About The Project

(back to top)

Built With

• 
• 
• gRPC

(back to top)

Getting Started

Follow the below installations to setup.

Installation

1. Fork the Project
2. In a seperate empty clone forked the repo

   git clone [email protected]:<YOUR-USERNAME>/MapReduce.git

3. Set the official repository as your upstream to synchronize with the latest update in the this repository. Add the original repository as upstream

   cd MapReduce
   git remote add upstream [email protected]:MaheshG11/MapReduce.git

4. Synchronize latest commit with this repository before coding

   git fetch upstream

5. Installation
   a. For Windows: If you use windows you will have to work with docker or WSL.

   docker build -t map_reduce .
   docker run -it map_reduce

   b. For Linux : Make sure cmake is not installed initially as we will use newer version of cmake installed through grpc

   sudo ./gRPCinstall.sh # installing grpc
   sudo ./initialize.sh # compiling code

   NOTE: Compiled executable will be found in project_dir/out/build/

6. Possible Errors
   a. For Linux: if during compilation you recieve cmake not found, try

   export PATH="$HOME/.local/bin:$PATH"

   and then run the compilation command.
7. Linking You working directory to project container

   docker run -it -v ./:/project map_reduce 
   # Assuming that you are in source directory 
   # i.e, where dockerfile is located

Tips

1. For Understanding Read Map Reduce Paper.
2. You can tune a few parameters for this read parameters.md

(back to top)

Contact

Mahesh Ghumare LinkedIn

(back to top)