Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable testing for resource estimates #828

Merged
merged 11 commits into from
Aug 4, 2023

Conversation

fdmalone
Copy link
Collaborator

@fdmalone fdmalone commented Aug 1, 2023

Previously the resource estimation code was not being tested as part of the CI (#825) . This PR enables it, I've marked a few tests as slow and will open an issue to track the fact that one dependency (btas/pybtas) doesn't have any tests. I think we should add a weekly workflow which will run the slow tests and run a yet to be written pybtas unit test.

I'm only testing the resource estimate code for linux and mac as I don't think pyscf works on windows.

.github/workflows/ci.yml Outdated Show resolved Hide resolved
.github/workflows/ci.yml Outdated Show resolved Hide resolved
pytest.skip('Need pyscf for resource estimates', allow_module_level=True)
HAVE_DEPS_FOR_RESOURCE_ESTIMATES = True
except ModuleNotFoundError:
HAVE_DEPS_FOR_RESOURCE_ESTIMATES = False
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we re-raise the exception with a helpful error message? presumably if these imports fail something else will give a weird error message when you actually go and try to use any of the functionality

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, wouldn't this cause an import error (pytest will error rather than skip the tests)

check/pytest Outdated Show resolved Hide resolved
@fdmalone
Copy link
Collaborator Author

fdmalone commented Aug 2, 2023

@mpharrigan PTAL

@@ -19,7 +19,7 @@ cd "$(git rev-parse --show-toplevel)"

PYTEST_ARGS=()
ACTUALLY_QUIET=""
for arg in $@; do
for arg in "$@"; do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this changed? are you sure?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$@ splits quoted args on whitespace so the option -m "not slow" gets converted to '-m' 'not' 'slow', rather than '-m' 'not slow'. At least I couldn't figure out how to pass in quoted args otherwise. Happy to be schooled on bash though

@mpharrigan
Copy link
Collaborator

the output from the ci job looks like the desired tests aren't being run (??)

Run check/pytest -m 'not slow' src/openfermion/resource_estimates
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-[7](https://github.com/quantumlib/OpenFermion/actions/runs/5735347442/job/15542949189?pr=828#step:5:8).3.1, pluggy-1.0.0
rootdir: /home/runner/work/OpenFermion/OpenFermion/dev_tools/conf
configfile: pytest.ini
plugins: cov-4.0.0, asyncio-0.21.0, xdist-3.3.1
asyncio: mode=strict
collected 25 items / 7 deselected / 1[8](https://github.com/quantumlib/OpenFermion/actions/runs/5735347442/job/15542949189?pr=828#step:5:9) selected

dev_tools/conf/utils_test.py .....                                       [ 27%]
dev_tools/conf/df/compute_cost_df_test.py ..                             [ 38%]
dev_tools/conf/df/compute_lambda_df_test.py .                            [ 44%]
dev_tools/conf/sf/compute_cost_sf_test.py ..                             [ 55%]
dev_tools/conf/sf/compute_lambda_sf_test.py .                            [ 61%]
dev_tools/conf/sparse/costing_sparse_test.py ..                          [ 72%]
dev_tools/conf/thc/compute_cost_thc_test.py ..                           [ 83%]
dev_tools/conf/thc/compute_lambda_thc_test.py .                          [ 88%]
dev_tools/conf/thc/spacetime_test.py ..                                  [[10](https://github.com/quantumlib/OpenFermion/actions/runs/5735347442/job/15542949189?pr=828#step:5:11)0%]

=============================== warnings summary ===============================

@mpharrigan
Copy link
Collaborator

or is that a weird artifact of the runner? those test files aren't in dev_tools/conf and are indeed the correct tests

mpharrigan
mpharrigan previously approved these changes Aug 2, 2023
@fdmalone
Copy link
Collaborator Author

fdmalone commented Aug 2, 2023

very odd, I have no idea why it's giving those paths...

@fdmalone
Copy link
Collaborator Author

fdmalone commented Aug 2, 2023

Maybe to do with pytest saying the root_dir is dev_tools/conf, maybe I used the wrong command to specify the pytest.ini file.

@mpharrigan
Copy link
Collaborator

maybe that grep expression is doing something weird

@fdmalone
Copy link
Collaborator Author

fdmalone commented Aug 2, 2023

Nah the grep is quieting the output I think. I needed to pass --rootdir through to pytest, somehow specifying the pytest.ini file sets the rootdir to where the ini file is (dev_tools/conf). Should be fixed now.

Copy link
Collaborator

@mpharrigan mpharrigan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still lgtm. Do reviews get disqualified if you post another commit? We should make that not happen

@fdmalone fdmalone merged commit 945bcaf into master Aug 4, 2023
11 checks passed
@fdmalone
Copy link
Collaborator Author

fdmalone commented Aug 4, 2023

Seems like it needed a re-review. Thanks!

@fdmalone fdmalone deleted the enable_testing_for_resource_estimates branch August 4, 2023 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants