kMM Classifier: Settlement & Electricity Detection

Detection of Settlements via Random Forest Classifier & Electricity via Max Projection VIIRS Algorithm

Team Members

• Katelyn Wang (@katelyn-wang)
• Michael Pien (@ThatMegamind)
• Maithy Le (@maithyy)

Links

[Tech Memo] [Presentation Slides] [Presentation Recording] [Project Poster]

Our Task

1. Use binary segmentation to identify settlements and non-settlements with a random forest classifier

2. Classify settlements as having electricity or no electricity using VIIRS nighttime data

Overview of Project Architecture

The architecture of our project can be split into 3 distinct phases:

1. Preprocessing
2. Settlement Detection
3. Electricity Detection

In our preprocessing step, we run a series of functions which extract, cleanup, and prepare the original satellite images to be fed into our machine learning models.

In the settlement detection phase, we can run one of four models: Random Forest Classifier (primary model), Segmentation CNN, Transfer Resnet101, or U-Net. These models produce a classification of pixels as either settlement or non-settlement.

Finally, in the electricity detection phase, we run an algorithm that takes the predictions created by the machine learning model and uses the max VIIRS projection data to determine whether each predicted settlement has electricity or not, producing a final output that identifies settlements with no electricity.

Pipeline

Segmentation Sample/Result

Installation + Getting Started

To begin, start by cloning this repository by running

git clone https://github.com/maithyy/electricity-and-settlement-detection-with-ai.git

Then, navigate to the project folder and run the following command to install the necessary libraries and packages

pip install -r requirements.txt

This will successfuly set up the repository for use.

Next, download the necessary data from the 2021 IEEE GRSS Dataset, and place the Train data into a directory titled data/raw/Train.

To prepare the data for the binary segmentation task, make sure to run scripts/extra_utils.py before anything else.

Now, you can train any of the models provided by setting flags in the command to run the program. Notably, you can set the model of Random Forest Classifier by setting the flag --model_type: RandomForestClassifier. With the model_type flag plus any additional desired parameters, run python3 scripts/train.py. This should produce a .ckpt file under the models directory.

You can then run python3 scripts/predict_electricity.py to produce a final prediction based on the output of the trained model, indicating what areas are likely to have settlements without electricity.

How We Assembled Our Architecture

Data Preprocessing

• Created new ground truth images, where 1 color is for settlements with electricity + settlements w/o electricity, and another color is the other two classes

Random Forest Classifier

• Created a new file called random_forest_module.py inside models/supervised, which contains our PyTorch implementation of random forest model
  • Used for classifying settlements vs. non-settlements
• Modified satellite_module.py to account for new type of model
• Modified utilities.py to account for new hyperparameters

Electricity Classification

• Wrote an algorithm to take the ouput of the random forest classifier to classify electricity vs. no electricity
  • Used VIIRS nighttime dataset and maxprojection_VIIRS
• Calculated thresholds for what qualifies as having electricity

Combining the Model and Algorithm

• Created another set of ground truth images, where 1 color is for settlements with no electricity and the other is for everything else
• Inputted the settlement detection output into the electricity detection algorithm to get final prediction of settlements with no electricity

Citing kMM Classifier

If you use kMM Classifier's work in your research, please use the following BibTeX entry.

@article{wang_le_pien_2024_settlement_detection,
  title={kMM Classifier Settlement and Electricity Detection},
  author={Wang, Katelyn and Le, Maithy and Pien, Michael},
  year={2024}
}

Reference Papers

• Mapping New Informal Settlements Using Machine Learning and Time Series Satellite Images
• Detection of rural electrification in Africa using DMSP-OLS night lights imagery