CurbPHX
| Index | Description |
|---|---|
| Overview | See the motivation behind this project. |
| Description | Learn more about the problem, implemented solution and challenges faced. |
| Deployment Guide | How to install CurbPHX architecture. |
| VPN Connection | Accessing through VPN. |
| AWS Rekognition | How to use AWS Rekognition. |
| How to Use | Instructions to use CurbPHX. |
| Future Enhancements | Limitations and next steps which can be taken. |
| Credits | Meet the team behind this. |
| License | License details. |
The City of Phoenix - Street Transportation Department and the Arizona State University Smart City Cloud Innovation Center Power by AWS(ASU CIC) are coming together to develop a tool that can build a complete sidewalk data inventory. A sidewalk inventory is a data bank with a collection of attributes about the sidewalk. In this project, we tackle the problem of identifying the different types of sidewalk which are attached sidewalks, detached sidewalks and places where there are no sidewalks. The City of Phoenix - Street Transportation department will use this data to better understand pedestrian mobility, safety and to improve sidewalk management.
As one of the the fastest growing cities in the United States, Phoenix has a constant need to update its sidewalk inventory to include new developments. The City has the opportunity to expand the data sets used to understand the location and condition of sidewalks. Access to sidewalk data makes it possible to identify and prioritize funding for infrastructure investments. A sidewalk data layer would help identify areas of need, measure the impact of projects on City's sidewalk network and prioritize funding for infrastructure investments.
Working together, the City of Phoenix and the CIC envisioned a new solution: CurbPHX. CurbPHX is a comprehensive digital inventory of all sidewalks in the city.
An initial data set composed of images of sidewalk will be constructed with the help of aerial drone imagery provided by third party vendors such as Eagleview. These images are then processed with AWS Rekognition using our cloud infrastructure to create the sidewalk inventory which can be viewed on our Google Maps Overlay and ArcGIS. The ASU CIC envisions a machine learning and geographic information system (GIS) framework that can process the high amounts of data. With the ability to collect, analyze, and store data on Phoenix’s sidewalk network, the City of Phoenix will be able to effectively identify areas that need repair and improve pedestrian safety and mobility.
High-level overview of the application
Detailed Architectural diagram
Given a set of aerial imagery, which is uploaded through an Amazon S3 bucket, a image recognition model is trained using a subset of high quality feature rich images to detect types of sidewalks such as attached, detached and no sidewalk regions.
Every dataset contains the possibility of duplicate features, which must be removed in order to reduce processing costs, time complexity, and improve efficiency. As a result, our initial step after receiving the imageset is to remove any photographs that cover the same area of the city. This is accomplished by constructing a grid of equal-sized rectangles based on the dataset limits. For each cell, we choose an image that has the maximum overlap with it, allowing us to pick images that cover the entire city while having few duplication among them.
Because an aerial photograph of a city area has a high resolution, it must be broken down into smaller parts so that it may be processed faster and more efficiently. We divided an image into 9 chunks before interpolating the spatial coordinates for all four corners of each piece. The algorithm consumes these chunks one at a time and uses them to generate a sidewalk inventory that includes detached, attached, and no sidewalk regions.
In this way, we generate an inventory for sidewalks for the given city by filtering, parsing and extracting features out of the imageset using Amazon Web Services. The final output is rendered on google maps overlay and as a shapefile to use on ArcGIS pro.
Using Python as our base language, we used the following technology stack to implement our system.
- Amazon Web Services ->
- GIS ->
- Exploratory Data Analysis using Python Notebooks
Except that fact that this system was designed, built and tested on Amazon Web Services, following are the requirements:
- User should have access to at least 500 images with 250 images (High resolution preferred such as 4096x2048) per label to train an AWS Rekognition model, these images can be obtained from Google Streetview or Aerial imagery providers (Eagleview, NearMap, etc.)
- Images are uploaded on to a public S3 bucket
- A metadata file, in XML format needs to be included, if your Image provider was Eagleview, they will provide you this out of the box but if you're using a different provider, you might want to generate your own version.
- Image metadata XML tree should have the following structure: root -> "Block" -> "Photo" (one for each image included in the bucket) -> "Id", "ImagePath" (it is the filename), "ProjectedCorners" (latitude and longitude coordinates in ECEF system )(EPSG:4978), "Pose" (Rotation metrics, not mandatory).
Refer to following documents for each deployment steps:
Check this doc on how to configure your own vpn connection and connecting to it for privacy of your service.
Check this doc for instructions on how to use Rekognition for training and evaluation of the model.
- Home page
- Allows user to select between Google Streetview feature or Eagleview which allows user to upload new images and run it across AWS Rekognition
- After Eagleview is select, the user has to specify the S3 upload links for images and a xml which contains metadata
- Once the images and xml file have been uploaded the images will be processed. If the links are not valid or if there is any issues
If the links are valid and all the process are successfull the download button will appear
- There are three options to view the data, either by using the Google Street View overlay, .shp files that can be used with ArcGIS or .kml file
The data in Google Street View:
The user can download the files by clicking on the respective download buttons
- Make the system scalable, more efficient and cost friendly. Right now, the bottleneck of the application is AWS Lambda, as it is designed to work for smaller processes. We need to use a service which can do time-consuming tasks faster and for less cost such as AWS Batch, AWS Simple Work Flows or AWS Fargate
- Implement image segmentation instead of bounding boxes for better edge detection
- Improve the dashboard to incorporate more control over the backend which provides the ability to pause/resume/abort the processing loop, display the database information, ability to move/delete items from the backend database, etc
- The crux of our system is AWS Lambda, which is designed for short-term jobs and hence there is a limit of 15 minutes. To work around this, we upload up to 10 images everytime the process is run, which then splits up into 9 smaller parts per image which are then consumed by the image recognition service
- Moving to an Auto-scalable infrastructure will increase performance and reduce time complexity of the application
- Drawing the sidewalks detected on the image to a map accurately and smoothly is a challenge, keeping in mind to remove the false positives, an expertise in GIS can solve this problem
"CurbPHX" is an open source software. The following people have contributed to this project.
Developers:
Soham Sahare
Krishna Teja Kalaparty
Yug Gulati
UI Designer: Nilo Exar Sarah Vue
Sr. Program Manager, AWS: Jubleen Vilku
Digital Innovation Lead, AWS: Jason Whittet
General Manager, ASU: Ryan Hendrix
This project is designed and developed with guidance and support from the ASU Cloud Innovation Center and the City of Phoenix, Arizona teams.
This project is distributed under the Apache License, Version 2.0.