Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance optimization of STL loading #1251

Closed
wants to merge 2 commits into from

Conversation

gonzalocasas
Copy link
Member

I was extremely unhappy about the loading speed of compas_robots which loads remote meshes (stl usually), and started digging into the causes. I found and fixed two issues:

  1. the ascii/binary detection was causing a double download (now this is not the case if the file is binary),
  2. downloading the file using urllib was pretty slow in itself

So, I switched the implementation to rely on .NET class library for the downloading :)

What type of change is this?

  • Bug fix in a backwards-compatible manner.
  • New feature in a backwards-compatible manner.
  • Breaking change: bug fix or new feature that involve incompatible API changes.
  • Other (e.g. doc update, configuration, etc)

I ran some tests, downloading 4 files in a loop, 5 times each, and the averaging the times, the results of the changes are as follows.

This is the baseline running on cpython:

(compas2-dev-311) C:\Users\gcasas\eth\Projects\compas-dev\compas>python src\compas\_iotools.py
Running on urllib.request implementation
  File 1, took 0.639 secs,   File 1, took 0.691 secs,   File 1, took 0.678 secs,   File 1, took 0.692 secs,   File 1, took 0.664 secs,
  File 2, took 0.817 secs,   File 2, took 0.855 secs,   File 2, took 0.868 secs,   File 2, took 0.839 secs,   File 2, took 0.771 secs,
  File 3, took 0.291 secs,   File 3, took 0.256 secs,   File 3, took 0.264 secs,   File 3, took 0.268 secs,   File 3, took 0.281 secs,
  File 4, took 0.519 secs,   File 4, took 0.532 secs,   File 4, took 0.545 secs,   File 4, took 0.505 secs,   File 4, took 0.510 secs,

Average File 1 (cpython): 0.673 secs
Average File 2 (cpython): 0.830 secs
Average File 3 (cpython): 0.272 secs
Average File 4 (cpython): 0.522 secs

And the same stuff running on IronPython using the previous (urllib2) implementation and the new implementation based on .net:

(compas2-dev-311) C:\Users\gcasas\eth\Projects\compas-dev\compas>ipy src\compas\_iotools.py
Running on urllib2 implementation
  File 1, took 1.266 secs,   File 1, took 1.307 secs,   File 1, took 1.121 secs,   File 1, took 1.082 secs,   File 1, took 1.326 secs,
  File 2, took 1.348 secs,   File 2, took 1.567 secs,   File 2, took 1.271 secs,   File 2, took 1.553 secs,   File 2, took 1.686 secs,
  File 3, took 0.338 secs,   File 3, took 0.365 secs,   File 3, took 0.362 secs,   File 3, took 0.423 secs,   File 3, took 0.432 secs,
  File 4, took 0.872 secs,   File 4, took 0.824 secs,   File 4, took 0.963 secs,   File 4, took 0.880 secs,   File 4, took 1.009 secs,

Average File 1 (urllib2): 1.220 secs
Average File 2 (urllib2): 1.485 secs
Average File 3 (urllib2): 0.384 secs
Average File 4 (urllib2): 0.909 secs

Switch to .NET implementation
  File 1, took 0.903 secs,   File 1, took 0.954 secs,   File 1, took 0.982 secs,   File 1, took 0.886 secs,   File 1, took 0.975 secs,
  File 2, took 1.271 secs,   File 2, took 1.298 secs,   File 2, took 1.325 secs,   File 2, took 1.291 secs,   File 2, took 1.285 secs,
  File 3, took 0.188 secs,   File 3, took 0.321 secs,   File 3, took 0.186 secs,   File 3, took 0.189 secs,   File 3, took 0.173 secs,
  File 4, took 0.663 secs,   File 4, took 0.641 secs,   File 4, took 0.725 secs,   File 4, took 0.602 secs,   File 4, took 0.742 secs,

Average File 1 (.net): 0.940 secs
Average File 2 (.net): 1.294 secs
Average File 3 (.net): 0.211 secs
Average File 4 (.net): 0.675 secs

Bottom line: although it's still not as fast as cpython, it's gotten at around 20% faster.

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I added a line to the CHANGELOG.md file in the Unreleased section under the most fitting heading (e.g. Added, Changed, Removed).
  • I ran all tests on my computer and it's all green (i.e. invoke test).
  • I ran lint on my computer and there are no errors (i.e. invoke lint).
  • I added new functions/classes and made them available on a second-level import, e.g. compas.datastructures.Mesh.
  • I have added tests that prove my fix is effective or that my feature works.
  • I have added necessary documentation (if appropriate)

@gonzalocasas gonzalocasas marked this pull request as draft January 6, 2024 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant