Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

facility link snapping #276

Merged
merged 7 commits into from
Aug 8, 2024
Merged

facility link snapping #276

merged 7 commits into from
Aug 8, 2024

Conversation

Theodore-Chatziioannou
Copy link
Contributor

@Theodore-Chatziioannou Theodore-Chatziioannou commented May 14, 2024

Adds CLI method for snapping activity facilities to a network geometry.

It can be invoked as: pam snap-facilities <path_population_in> <path_population_out> <path_network_geometry> -f <link_id_field>

pam snap-facilities --help

Usage: pam snap-facilities [OPTIONS] PATH_POPULATION_IN PATH_POPULATION_OUT
                           PATH_NETWORK_GEOMETRY

  Snap facilities to a network geometry.

Options:
  -f, --link_id_field TEXT  The link ID field to use in the network shapefile.
                            Defaults to 'id'.
  --help                    Show this message and exit.

For each activity, the method will identify the closest link in the shapefile, and add it as a link=.. attribute to the activity element. Finally, the updated plans file will be saved to a new xml file.

Checklist

Any checks which are not relevant to the PR can be pre-checked by the PR creator.
All others should be checked by the reviewer(s).
You can add extra checklist items here if required by the PR.

  • CHANGELOG updated
  • Tests added to cover contribution
  • Documentation updated

Copy link
Contributor

@brynpickering brynpickering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice and clean, just a few suggested minor changes.

src/pam/operations/snap.py Outdated Show resolved Hide resolved
src/pam/operations/snap.py Outdated Show resolved Hide resolved
tests/test_29_snap.py Outdated Show resolved Hide resolved
tests/test_29_snap.py Outdated Show resolved Hide resolved
tests/test_29_snap.py Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
@Theodore-Chatziioannou
Copy link
Contributor Author

thanks @brynpickering - I have made some updates in response to your comments. Before merging, @gac55 let me know if it works for your use case.

@gac55
Copy link

gac55 commented May 23, 2024

Thanks @Theodore-Chatziioannou , picking this up now

@syhwawa
Copy link
Contributor

syhwawa commented Jul 10, 2024

Hi @Theodore-Chatziioannou,

I've tested the new pam snap-facilities function with 100 agents' plan for Londinium, and it worked quite well!

When I tried it on TE pop, I noticed it perform quite slow when scaling up to larger populations. It took about 30 minutes of runtime for 0.01% of the population for TE pop.

I feel the primary slowdown seems to come from the way distances are calculated for each activity to all network links. Maybe some vectorized operations or something similar would help to potential speed-ups to process?

Copy link
Contributor

@syhwawa syhwawa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, the snap_facilities_to_network performed slowly when it needed to work with big population inputs.
Now it utilizes cKDTree for efficient nearest neighbour search and batch processing of activity locations. This update improves performance in snapping facilities to the network geometry now. @Theodore-Chatziioannou

for act in person.activities:
assert act.location.link is None

snap_facilities_to_network(population=population_heh, network=network)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is a def missing here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it just calls the functionsnap_facilities_to_network here.

@Theodore-Chatziioannou
Copy link
Contributor Author

Theodore-Chatziioannou commented Aug 5, 2024

thanks @syhwawa

  • your change has broken the CI - can you fix the linting? I believe this due to using single quotes
  • the code can be simpler. There is no need to two separate loops, one collecting the data and another applying it, saving the intermediate data in activity_points and activities_info. The updated locations can be queried and assigned directly in the first loop.

Comment on lines 37 to 38
if not hasattr(point, 'x') or not hasattr(point, 'y'):
point = point.centroid
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this needed? Loc should always be a point with x and y coordinates (or None).
See here: https://github.com/arup-group/pam/blob/3f817a662317d44fca2126295f07d89317db707c/src/pam/location.py#L7-L18.
If that is not the case in the input data, then the inputs should be corrected rather than fixing them silently here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed this.

@syhwawa
Copy link
Contributor

syhwawa commented Aug 6, 2024

Thanks for the comments @Theodore-Chatziioannou

The function has been updated to remove the separate loops for collecting and applying data. Now, the updated locations are queried and assigned directly within the first loop. And the lint error has been fixed.

@Theodore-Chatziioannou Theodore-Chatziioannou merged commit c8bff76 into main Aug 8, 2024
15 checks passed
@Theodore-Chatziioannou Theodore-Chatziioannou deleted the snap-facilities branch August 8, 2024 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants