Skip to content

Commit

Permalink
Merge pull request #436 from GIScience/filter-geometries
Browse files Browse the repository at this point in the history
[oshdb-filter] add additional geometry based filters
  • Loading branch information
tyrasd authored Nov 3, 2022
2 parents 945efb5 + 28cb88b commit abe3c9e
Show file tree
Hide file tree
Showing 14 changed files with 1,033 additions and 13 deletions.
6 changes: 4 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ Changelog

### new features

* Allow to flexibly combine (automatic) aggregation methods (like `aggregateByGeometry(…)` or `aggregateByTimestamp()`) with each other and with `filter` or `map`/`flatMap`, regardless of the order of the applied operations ([#451])
* allow to flexibly combine (automatic) aggregation methods (like `aggregateByGeometry(…)` or `aggregateByTimestamp()`) with each other and with `filter` or `map`/`flatMap`, regardless of the order of the applied operations ([#451])
* add new OSHDB filters: `perimeter`, `geometry.vertices`, `geometry.outers`, `geometry.inners`, `geometry.roundness` and `geometry.squareness` ([#436])

### bugfixes

Expand All @@ -36,6 +37,7 @@ Changelog
[#419]: https://github.com/GIScience/oshdb/pull/419
[#424]: https://github.com/GIScience/oshdb/pull/424
[#433]: https://github.com/GIScience/oshdb/issues/433
[#436]: https://github.com/GIScience/oshdb/pull/436
[#438]: https://github.com/GIScience/oshdb/pull/438
[#441]: https://github.com/GIScience/oshdb/pull/441
[#443]: https://github.com/GIScience/oshdb/pull/443
Expand All @@ -45,6 +47,7 @@ Changelog
[#453]: https://github.com/GIScience/oshdb/pull/453
[#454]: https://github.com/GIScience/oshdb/pull/454
[#459]: https://github.com/GIScience/oshdb/pull/459
[#467]: https://github.com/GIScience/oshdb/pull/467



Expand All @@ -58,7 +61,6 @@ Changelog
[#426]: https://github.com/GIScience/oshdb/pull/426
[#428]: https://github.com/GIScience/oshdb/pull/428


## 0.7.1

* fix a bug where contribution-based filters are not applied when used in an and/or operation. ([#409])
Expand Down
6 changes: 6 additions & 0 deletions oshdb-filter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,12 @@ Filters are defined in textual form. A filter expression can be composed out of
| `geometry:geom-type` | matches anything which has a geometry of the given type (_point_, _line_, _polygon_, or _other_) | `geometry:polygon` |
| `area:(from..to-range)` | matches all features with an area that falls into the given range/interval given as two numbers in decimal or scientific notation separated by `..`. The values are interpreted as square meters (``). The lower or upper limit of the interval may be omitted to select features having an area up to or starting from the given value, respectively. | `area:(123.4..1E6)` |
| `length:(from..to-range)` | matches all features with a length that falls into the given range/interval given as two numbers in decimal or scientific notation separated by `..`. The values are interpreted as meters (`m`). The lower or upper limit of the interval may be omitted to select features having an area up to or starting from the given value, respectively. | `length:(100..)` |
| `perimeter:(from..to-range)` | matches all features with a perimeter that falls into the given range/interval given as two numbers in decimal or scientific notation separated by `..`. The values are interpreted as meters (`m`). The lower or upper limit of the interval may be omitted to select features having an area up to or starting from the given value, respectively. | `perimeter:(100..)` |
| `geometry.vertices:(from..to-range)` | matches features by the number of points they consist of (the range is given as two integers separated by `..`). | `geometry.vertices:(1..10)` |
| `geometry.outers:number` or `geometry.outers:(from..to-range)` | matches features by the number of outer rings they consist of (the range is given as two integers separated by `..`) | `geometry.outers:1` or `geometry.outers:(2..)` |
| `geometry.inners:number` or `geometry.inners:(from..to-range)` | matches features by the number of holes (inner rings) they have (the range is given as two integers separated by `..`) | `geometry.inners:0` or `geometry.inners:(1..)` |
| `geometry.roundness:(from..to-range)` | matches polygons which have a _roundness_ (or _compactness_) in the given range of values (given as two numbers in decimal or scientific notation separated by `..`). This is using the ["Polsby–Popper test" score](https://en.wikipedia.org/wiki/Polsby%E2%80%93Popper_test) where all values fall in the interval 0 to 1 and 1 represents a perfect circle. | `geometry.roundness:(0.8..)` |
| `geometry.squareness:(from..to-range)` | matches features which have a _squareness_ in the given range of values (given as two numbers in decimal or scientific notation separated by `..`). This is using the [rectilinearity measurement by Žunić and Rosin](https://www.researchgate.net/publication/221304067_A_Rectilinearity_Measurement_for_Polygons) where all values fall in the interval 0 to 1 and 1 represents a perfectly rectilinear geometry. | `geometry.squareness:(0.8..)` |
| `changeset:id` | matches OSM contributions performed in the given OSM changeset. Can only be used in queries using the `OSMContributionView`. | `changeset:42` |
| `changeset:(list,of,ids)` | matches OSM contributions performed in one of the given OSM changeset ids. Can only be used in queries using the `OSMContributionView`. | `changeset:(1,42,100)` |
| `changeset:(from..to-range)` | matches OSM contributions performed in an OSM changeset falling into the given range of changeset ids. Can only be used in queries using the `OSMContributionView`. | `changeset:(100..200)` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ public FilterParser(TagTranslator tt) {
* @param allowContributorFilters if true enables filtering by contributor/user id.
*/
public FilterParser(TagTranslator tt, boolean allowContributorFilters) {
// todo: refactor this method into smaller chunks to make it more easily testable
final Parser<Void> whitespace = Scanners.WHITESPACES.skipMany();

final Parser<String> keystr = Patterns.regex("[a-zA-Z_0-9:-]+")
Expand Down Expand Up @@ -96,6 +97,15 @@ public FilterParser(TagTranslator tt, boolean allowContributorFilters) {
.map(ignored -> "*");
final Parser<Void> area = Patterns.string("area").toScanner("area");
final Parser<Void> length = Patterns.string("length").toScanner("length");
final Parser<Void> perimeter = Patterns.string("perimeter").toScanner("perimeter");
final Parser<Void> vertices = Patterns.string("geometry.vertices")
.toScanner("geometry.vertices");
final Parser<Void> outers = Patterns.string("geometry.outers").toScanner("geometry.outers");
final Parser<Void> inners = Patterns.string("geometry.inners").toScanner("geometry.inners");
final Parser<Void> roundness = Patterns.string("geometry.roundness")
.toScanner("geometry.roundness");
final Parser<Void> squareness = Patterns.string("geometry.squareness")
.toScanner("geometry.squareness");
final Parser<Void> changeset = Patterns.string("changeset").toScanner("changeset");
final Parser<Void> contributor = Patterns.string("contributor").toScanner("contributor");

Expand Down Expand Up @@ -194,7 +204,7 @@ public FilterParser(TagTranslator tt, boolean allowContributorFilters) {
Parsers.or(point, line, polygon, other))
.map(geometryType -> new GeometryTypeFilter(geometryType, tt));

final Parser<ValueRange> floatingRange = Parsers.between(
final Parser<ValueRange> positiveFloatingRange = Parsers.between(
Scanners.isChar('('),
Parsers.or(
Parsers.sequence(floatingNumber, dotdot, floatingNumber,
Expand All @@ -206,15 +216,53 @@ public FilterParser(TagTranslator tt, boolean allowContributorFilters) {
),
Scanners.isChar(')')
);
final Parser<ValueRange> positiveIntegerRange = Parsers.between(
Scanners.isChar('('),
Parsers.or(
Parsers.sequence(number, dotdot, number,
(min, ignored, max) -> new ValueRange(min, max)),
number.followedBy(dotdot).map(
min -> new ValueRange(min, Double.POSITIVE_INFINITY)),
Parsers.sequence(dotdot, number).map(
max -> new ValueRange(0, max))
),
Scanners.isChar(')')
);

// geometry filter
final Parser<GeometryFilter> geometryFilterArea = Parsers.sequence(
area, colon, floatingRange
area, colon, positiveFloatingRange
).map(GeometryFilterArea::new);
final Parser<GeometryFilter> geometryFilterLength = Parsers.sequence(
length, colon, floatingRange
length, colon, positiveFloatingRange
).map(GeometryFilterLength::new);
final Parser<GeometryFilter> geometryFilterPerimeter = Parsers.sequence(
perimeter, colon, positiveFloatingRange
).map(GeometryFilterPerimeter::new);
final Parser<GeometryFilter> geometryFilterVertices = Parsers.sequence(
vertices, colon, positiveIntegerRange
).map(GeometryFilterVertices::new);
final Parser<GeometryFilter> geometryFilterOuters = Parsers.sequence(
outers, colon, Parsers.or(positiveIntegerRange, number.map(n -> new ValueRange(n, n)))
).map(GeometryFilterOuterRings::new);
final Parser<GeometryFilter> geometryFilterInners = Parsers.sequence(
inners, colon, Parsers.or(positiveIntegerRange, number.map(n -> new ValueRange(n, n)))
).map(GeometryFilterInnerRings::new);
final Parser<GeometryFilter> geometryFilterRoundness = Parsers.sequence(
roundness, colon, positiveFloatingRange
).map(GeometryFilterRoundness::new);
final Parser<GeometryFilter> geometryFilterSquareness = Parsers.sequence(
squareness, colon, positiveFloatingRange
).map(GeometryFilterSquareness::new);
final Parser<GeometryFilter> geometryFilter = Parsers.or(
geometryFilterArea,
geometryFilterLength);
geometryFilterLength,
geometryFilterPerimeter,
geometryFilterVertices,
geometryFilterOuters,
geometryFilterInners,
geometryFilterRoundness,
geometryFilterSquareness);

// changeset id filters
final Parser<ChangesetIdFilterEquals> changesetIdFilter = Parsers.sequence(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package org.heigit.ohsome.oshdb.filter;

import javax.annotation.Nonnull;
import org.locationtech.jts.geom.Geometry;
import org.locationtech.jts.geom.MultiPolygon;
import org.locationtech.jts.geom.Polygon;

/**
* A filter which checks the number of inner rings of a multipolygon relation.
*/
public class GeometryFilterInnerRings extends GeometryFilter {
/**
* Creates a new inner rings filter object.
*
* @param range the allowed range (inclusive) of values to pass the filter
*/
public GeometryFilterInnerRings(@Nonnull ValueRange range) {
super(range, GeometryMetricEvaluator.fromLambda(GeometryFilterInnerRings::countInnerRings,
"geometry.inners"));
}

private static int countInnerRings(Geometry geometry) {
if (geometry instanceof Polygon) {
return ((Polygon) geometry).getNumInteriorRing();
} else if (geometry instanceof MultiPolygon) {
var counter = 0;
for (var i = 0; i < geometry.getNumGeometries(); i++) {
counter += ((Polygon) geometry.getGeometryN(i)).getNumInteriorRing();
}
return counter;
} else {
return -1;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package org.heigit.ohsome.oshdb.filter;

import javax.annotation.Nonnull;
import org.locationtech.jts.geom.Polygonal;

/**
* A filter which checks the number of outer rings of a multipolygon relation.
*/
public class GeometryFilterOuterRings extends GeometryFilter {
/**
* Creates a new outer rings filter object.
*
* @param range the allowed range (inclusive) of values to pass the filter
*/
public GeometryFilterOuterRings(@Nonnull ValueRange range) {
super(range, GeometryMetricEvaluator.fromLambda(geometry ->
geometry instanceof Polygonal ? geometry.getNumGeometries() : -1,
"geometry.outers"));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package org.heigit.ohsome.oshdb.filter;

import javax.annotation.Nonnull;
import org.heigit.ohsome.oshdb.util.geometry.Geo;
import org.locationtech.jts.geom.Polygonal;

/**
* A filter which checks the perimeter of polygonal OSM feature geometries.
*/
public class GeometryFilterPerimeter extends GeometryFilter {
/**
* Creates a new perimeter filter object.
*
* @param range the allowed range (inclusive) of values to pass the filter
*/
public GeometryFilterPerimeter(@Nonnull ValueRange range) {
super(range, GeometryMetricEvaluator.fromLambda(geometry -> {
if (!(geometry instanceof Polygonal)) {
return 0;
}
var boundary = geometry.getBoundary();
return Geo.lengthOf(boundary);
}, "perimeter"));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package org.heigit.ohsome.oshdb.filter;

import javax.annotation.Nonnull;
import org.heigit.ohsome.oshdb.util.geometry.Geo;

/**
* A filter which checks the roundness of polygonal OSM feature geometries.
*
* <p>Uses the Polsby-Popper test score, see
* <a href="https://en.wikipedia.org/wiki/Polsby%E2%80%93Popper_test">wikipedia</a> for details.</p>
*/
public class GeometryFilterRoundness extends GeometryFilter {
/**
* Creates a new "roundness" filter object.
*
* @param range the allowed range (inclusive) of values to pass the filter
*/
public GeometryFilterRoundness(@Nonnull ValueRange range) {
super(range, GeometryMetricEvaluator.fromLambda(Geo::roundness, "geometry.roundness"));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package org.heigit.ohsome.oshdb.filter;

import javax.annotation.Nonnull;
import org.heigit.ohsome.oshdb.util.geometry.Geo;

/**
* A filter which checks the squareness of OSM feature geometries.
*
* <p>For the measure for the rectilinearity (or squareness) of a geometry, a methods adapted from the
* paper "A Rectilinearity Measurement for Polygons" by Joviša Žunić and Paul L. Rosin
* (DOI:10.1007/3-540-47967-8_50, https://link.springer.com/chapter/10.1007%2F3-540-47967-8_50,
* https://www.researchgate.net/publication/221304067_A_Rectilinearity_Measurement_for_Polygons) is used.</p>
*/
public class GeometryFilterSquareness extends GeometryFilter {
/**
* Creates a new squareness filter object.
*
* @param range the allowed range (inclusive) of values to pass the filter
*/
public GeometryFilterSquareness(@Nonnull ValueRange range) {
super(range, GeometryMetricEvaluator.fromLambda(Geo::squareness, "squareness"));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package org.heigit.ohsome.oshdb.filter;

import javax.annotation.Nonnull;
import org.locationtech.jts.geom.Geometry;

/**
* A filter which checks the number of vertices of OSM feature geometries.
*/
public class GeometryFilterVertices extends GeometryFilter {
public GeometryFilterVertices(@Nonnull ValueRange range) {
super(range, GeometryMetricEvaluator.fromLambda(Geometry::getNumPoints, "geometry.vertices"));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ public FilterExpression negate() {
throw new IllegalStateException("Invalid call of inner negate() on a negatable filter");
}

/** Inverse of {@link FilterExpression#applyOSH(OSHEntity)} */
/** Inverse of {@link FilterExpression#applyOSH(OSHEntity)}. */
@Contract(pure = true)
boolean applyOSHNegated(OSHEntity entity) {
return true;
Expand All @@ -32,7 +32,7 @@ public boolean applyOSM(OSMEntity entity) {
return true;
}

/** Inverse of {@link FilterExpression#applyOSM(OSMEntity)} */
/** Inverse of {@link FilterExpression#applyOSM(OSMEntity)}. */
@Contract(pure = true)
boolean applyOSMNegated(OSMEntity entity) {
return true;
Expand Down
Loading

0 comments on commit abe3c9e

Please sign in to comment.