-
Notifications
You must be signed in to change notification settings - Fork 10
/
README
150 lines (107 loc) · 5.13 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
Betashapes
==========
created by Melissa Santos and Schuyler Erle
(c) 2011 SimpleGeo, Inc.
What is this?
-------------
It's the code used by SimpleGeo to generate its international neighborhood
dataset.
See the blog post for an explanation:
http://blog.simplegeo.com/2011/08/05/its-a-beautiful-day-in-the-neighborhood/
(The blog post content is now in the BLOG_POST.mkd file of this repo)
Why's it here?
--------------
We had fun writing it. We like giving stuff away. Maybe you'll find it useful.
Maybe you'll improve it and send us a pull request! We provide no warranty, and
no support. If it breaks, you get to keep the pieces.
How's it work?
--------------
Well, it helps if you download Yahoo's GeoPlanet dump, and load both it and all
or some subset of Planet.osm into PostGIS.
You'll need to create a data/ directory, and dump a mapping of WoE ID -> Name
into a file called `data/names.txt`, and another mapping of Parent ID, Name,
Type -> WoE ID into another file called `data/suburbs.txt`. This is stupid and
could be done a lot more cleanly.
Here is a sample of the names.txt we're using:
29372661 San Francisco Javier
772864 San Francisco de Paula
108040 Villa de San Francisco
142610 San Francisco Culhuacán
349422 San Francisco de Limache
12521721 San Francisco International Airport
Here's a sample of the suburbs.txt:
44418 Streatham Common Suburb 20089509
44418 Upper Walthamstow Suburb 20089365
44418 Castelnau Suburb 20089570
44418 Harold Hill Suburb 22483
44418 Blackfriars Road Suburb 20094299
44418 Lampton Suburb 44314
44418 Lower Place Suburb 20089447
44418 Furzedown Suburb 20089510
44418 Crofton Suburb 20089334
44418 Collier's Wood Suburb 20089517
Running build_neighborhood.sh takes over from there.
What's in it?
-------------
build_neighborhood.sh <city> <woeid>
This shell script makes the magic happen. Depends on PostgreSQL and GRASS,
in addition to all the other stuff in here.
blockr.py <names.txt> <blocks.json> <points.txt>
The main neighborhood generation script. Takes a name file
(tab-separated, mapping WoE ID to name), a GeoJSON FeatureCollection
containing the block polygons to be assigned, and a points file (as
generated by geocrawlr.py).
Requires Shapely.
outliers.py <points.txt>
A module for reading points.txt files and discarding outlying points based
on median absolute distance. If run as a script, prints the bounding box of
the points after outliers are discarded.
geocrawlr.py <woe_id> [<woe_id> ...]
A script that crawls the Flickr API looking for geotagged photo records
associated with the given woe_ids. Writes line-by-line, tab-separated
values to stdout consisting of: Photo ID, WoE ID, Longitude, Latitude.
Uses Flickr.API. You must have your FLICKR_KEY and FLICKR_SECRET set in the
environment.
geoplanet.py
A utility script to query Y! GeoPlanet. Takes names, one per line, on stdin,
queries GeoPlanet, and outputs the first WoE ID and name returned on stdout.
Set YAHOO_APPID in your environment.
mapnik_render.py
A Mapnik script to visualize the neighborhood.json and blocks.json data
together.
leaves_from_woeid.py
Walks a table of GeoPlanet data in PostgreSQL and fetches all the leaves
descending from a given WoE ID.
What's a "betashape"?
---------------------
See:
http://code.flickr.com/blog/2008/10/30/the-shape-of-alpha/
also see:
http://code.flickr.com/blog/2009/01/12/living-in-the-donut-hole/
and for good measure:
http://code.flickr.com/blog/2011/01/08/flickr-shapefiles-public-dataset-2-0/
Propers to Aaron Straup Cope for his ideas and encouragement.
License
-------
Copyright (c) 2011, SimpleGeo, Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the SimpleGeo, Inc. nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL SIMPLEGEO, INC. BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.