Failing Tests #55

gonsie · 2020-09-23T23:21:19Z

This issue is related to openjournals/joss-reviews#2685

I’m running Python 3.8 on a Mac. When running python -m pytest tests I am getting 2 warnings and 6 errors. Here is the summary output:

============================================ warnings summary ============================================
de_sim/simulation_object.py:473
/Users/gonsiorowski1/Projects/JOSS/de_sim/de_sim/simulation_object.py:473: UserWarning: SimulationObject 'Example' definition does not inherit or provide a non-empty 'messages_sent'.
      warnings.warn("SimulationObject '{}' definition does not inherit or provide a "

de_sim/simulation_object.py:473
/Users/gonsiorowski1/Projects/JOSS/de_sim/de_sim/simulation_object.py:473: UserWarning: SimulationObject 'SOwithDefaultClassPriority' definition does not inherit or provide a non-empty 'messages_sent'.
warnings.warn("SimulationObject '{}' definition does not inherit or provide a "

-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================================== short test summary info =========================================
            ERROR tests/test_simulator.py
            ERROR tests/test_utilities.py
            ERROR tests/examples/test_jupyter_examples.py
            ERROR tests/examples/test_phold.py
            ERROR tests/examples/test_random_walk.py
            ERROR tests/examples/test_sirs.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 6 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================== 2 warnings, 6 errors in 3.65s ======================================

It looks like most of errors come from:

ModuleNotFoundError: No module named 'capturer'

The text was updated successfully, but these errors were encountered:

jonrkarr · 2020-09-23T23:38:32Z

Hi, the tests require additional dependencies beyond the package itself. These dependencies aren't installed by default because they aren't necessary for most users. These dependencies can be installed by running pip install de_sim[tests] or pip install -r tests/requirements.txt (if you have cloned the repository). Our CI system is setup to install these dependencies.

jonrkarr · 2020-09-23T23:43:04Z

I added these instructions to the Guide to contributors.

gonsie · 2020-09-24T21:17:22Z

I’m not able to run the tests successfully. I’m getting 1 failure and 1 hang.

The failure:

AssertionError: '[email protected]:karrlab/de_sim' not found in ['https://github.com/karrlab/de_sim.git', '[email protected]:karrlab/de_sim.git', 'ssh://[email protected]/karrlab/de_sim.git']

tests/test_simulation_metadata.py:64: AssertionError

And the 11th or 12th test of test_simulator.py is hanging.

jonrkarr · 2020-09-24T21:28:00Z

I corrected the failing metadata test. The code is fine; the assertion is overly strict and was incorrectly failing in your environment. This is pushed to GitHub and PyPI.

@artgoldberg can you help debug the hanging test? It works fine in our environments (our machines, CircleCI, Mac, Linux) which use Python 3.7. I think its worth creating a Docker image for Python 3.8 to see if this is specific to Python 3.8.

artgoldberg · 2020-09-25T20:34:43Z

Hi @gonsie

I tested de_sim on the lastest micro-version of Python (3.8.5) in Ubuntu. I wasn't able to reproduce a hanging test, but the tests needed another module, ipykernel, which has been added to tests/requirements.txt.

In addition, the warning UserWarning: SimulationObject ... does not inherit or provide a non-empty 'messages_sent' has been removed.

I've pushed these updates to GitHub and PyPI. The new version of de_sim is 0.1.4.

Thanks
Arthur

artgoldberg · 2020-09-28T15:42:30Z

Hi @gonsie

Could you please try the unit tests again? I was not able to reproduce the hanging test you report.
If you can reproduce it, could you please report the specific MacOS and Python versions?

Thanks
Arthur

artgoldberg · 2020-09-28T23:08:17Z

Hi @gonsie

Alternatively, to confirm that DE Sim works under Python 3.8, you could run its unit tests in a Docker container running Python 3.8:

Docker commands (requires Docker):

docker pull python:3.8.6
# run an interactive shell in the Docker container
docker run --interactive --tty python:3.8.6  bash

Shell commands to execute in the running Docker container:

git clone https://github.com/KarrLab/de_sim.git
cd de_sim
# obtain all of DE Sim's dependencies from PyPI:
pip install -r requirements.txt -r tests/requirements.txt
pip install pytest
# setup the bash environment for subprocesses
export PYTHONPATH="$(pwd)"
pytest tests/

You should get this output:

tests/test_api.py .                                         [  1%]
tests/test_checkpoint.py ....                               [  5%]
tests/test_errors.py .                                      [  6%]
tests/test_event.py ...                                     [  9%]
tests/test_event_message.py ....                            [ 13%]
tests/test_init.py .                                        [ 14%]
tests/test_mock_simulation_object.py .                      [ 15%]
tests/test_simulation_checkpoint_object.py ...              [ 18%]
tests/test_simulation_config.py ........                    [ 26%]
tests/test_simulation_metadata.py ........                  [ 34%]
tests/test_simulation_object.py .............               [ 47%]
tests/test_simulator.py ..................s..........       [ 76%]
tests/test_template_sim_objs.py ...                         [ 79%]
tests/test_utilities.py .......                             [ 86%]
tests/test_visualize.py ..                                  [ 88%]
tests/config/test_core.py ..                                [ 90%]
tests/examples/test_jupyter_examples.py .                   [ 91%]
tests/examples/test_minimal_simulation.py ..                [ 93%]
tests/examples/test_phold.py ...                            [ 96%]
tests/examples/test_random_walk.py .                        [ 97%]
tests/examples/test_sirs.py ..                              [ 99%]
tests/joss_paper/test_gen_phold_space_time_plot.py .        [100%]

============ 99 passed, 1 skipped in 192.93s (0:03:12) ===========

The tests take under 5 min. to run on my laptop, which has a 2.9 GHz Intel Core i5 with 16 GB RAM. They take a while because test_jupyter_examples.py runs the DE Sim performance tests.

Arthur

gonsie · 2020-09-28T23:51:59Z

Here is a dump of the failing test. I’m including the full output because I’m not exactly sure which parts may be useful.

============================== FAILURES ===================================
____________________ TestSimulator.test_mem_use_measurement ____________________

self = <tests.test_simulator.TestSimulator testMethod=test_mem_use_measurement>

    def test_mem_use_measurement(self):
        self.make_one_object_simulation()
        max_time = 20
        config_dict = dict(max_time=max_time, output_dir=self.out_dir, object_memory_change_interval=10)
        self.simulator.simulate(config_dict=config_dict)
        expected_text = ['Memory use changes by SummaryTracker', '# objects', 'float']
        measurements = ''.join(open(self.measurements_pathname, 'r').readlines())
        for text in expected_text:
            self.assertIn(text, measurements)
    
        self.make_one_object_simulation()
>       with CaptureOutput(relay=False) as capturer:

tests/test_simulator.py:692: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../.virtualenvs/desim/lib/python3.8/site-packages/capturer/__init__.py:276: in __enter__
    self.start_capture()
../../../.virtualenvs/desim/lib/python3.8/site-packages/capturer/__init__.py:328: in start_capture
    pseudo_terminal.start_capture()
../../../.virtualenvs/desim/lib/python3.8/site-packages/capturer/__init__.py:487: in start_capture
    self.start_child(self.capture_loop)
../../../.virtualenvs/desim/lib/python3.8/site-packages/capturer/__init__.py:153: in start_child
    child_process.start()
../../../.virtualenvs/desim/lib/python3.8/multiprocessing/process.py:121: in start
    self._popen = self._Popen(self)
../../../homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py:224: in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
../../../homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py:284: in _Popen
    return Popen(process_obj)
../../../homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py:32: in __init__
    super().__init__(process_obj)
../../../homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py:19: in __init__
    self._launch(process_obj)
../../../homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py:47: in _launch
    reduction.dump(process_obj, fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <Process name='Process-1' parent=78795 initial daemon>
file = <_io.BytesIO object at 0x121207a90>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       TypeError: cannot pickle '_io.BufferedReader' object

../../../homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py:60: TypeError
=============================== warnings summary ===============================
/Users/gonsiorowski1/homebrew/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/codecs.py:905: 1 warning
tests/test_event.py: 3 warnings
tests/test_mock_simulation_object.py: 1 warning
tests/test_simulation_checkpoint_object.py: 3 warnings
tests/test_simulation_object.py: 8 warnings
tests/test_simulator.py: 14 warnings
  /Users/gonsiorowski1/homebrew/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/codecs.py:905: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
    file = builtins.open(filename, mode, buffering)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_simulator.py::TestSimulator::test_mem_use_measurement - Typ...
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
============ 1 failed, 59 passed, 30 warnings in 341.07s (0:05:41) =============

artgoldberg · 2020-09-29T15:27:03Z

Thanks, I'll investigate.

artgoldberg · 2020-09-29T18:52:43Z

Thanks @gonsie. capturer.CaptureOutput() is failing to initialize here. If you share your computing environment, such as the versions of OSX, virtualenv, and Python you're using I can report this problem as an issue for capturer.

With respect to DE Sim, to confirm that it passes all tests under Python 3.8, you could run its unit tests in a Docker container, as I suggested yesterday.

Regards
Arthur

…put, because CaptureOutput fails in test run by JOSS paper reviewer; see #55; bump version

artgoldberg · 2020-09-30T16:04:26Z

Hi @gonsie

I have replaced capturer.CaptureOutput() with another method in the code that failed for you. A new version (0.1.7) of DE Sim that contains this change is available in the DE Sim repo on GitHub and in the de-sim package on PyPI.

I don't know whether this change will fix the failure, as I have not reproduced the problem. Could you please try it?

In addition, I've filed an issue with capturer regarding the failure you experienced. If you wish to provide more details regarding your computing environment I will add them to the issue.

Thanks
Arthur

artgoldberg · 2020-10-24T21:09:59Z

Closing given no feedback in 24 days.

jonrkarr added the question Further information is requested label Sep 23, 2020

artgoldberg mentioned this issue Sep 29, 2020

[REVIEW]: DE-Sim: an object-oriented, discrete-event simulation tool for data-intensive modeling of complex systems in Python openjournals/joss-reviews#2685

Closed

60 tasks

artgoldberg added a commit that referenced this issue Sep 30, 2020

capture stdout using contextlib.redirect_stdout instead of CaptureOut…

bb266a9

…put, because CaptureOutput fails in test run by JOSS paper reviewer; see #55; bump version

artgoldberg closed this as completed Oct 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing Tests #55

Failing Tests #55

gonsie commented Sep 23, 2020 •

edited

Loading

jonrkarr commented Sep 23, 2020

jonrkarr commented Sep 23, 2020

gonsie commented Sep 24, 2020

jonrkarr commented Sep 24, 2020

artgoldberg commented Sep 25, 2020

artgoldberg commented Sep 28, 2020

artgoldberg commented Sep 28, 2020

gonsie commented Sep 28, 2020

artgoldberg commented Sep 29, 2020

artgoldberg commented Sep 29, 2020 •

edited

Loading

artgoldberg commented Sep 30, 2020

artgoldberg commented Oct 24, 2020

Failing Tests #55

Failing Tests #55

Comments

gonsie commented Sep 23, 2020 • edited Loading

jonrkarr commented Sep 23, 2020

jonrkarr commented Sep 23, 2020

gonsie commented Sep 24, 2020

jonrkarr commented Sep 24, 2020

artgoldberg commented Sep 25, 2020

artgoldberg commented Sep 28, 2020

artgoldberg commented Sep 28, 2020

gonsie commented Sep 28, 2020

artgoldberg commented Sep 29, 2020

artgoldberg commented Sep 29, 2020 • edited Loading

artgoldberg commented Sep 30, 2020

artgoldberg commented Oct 24, 2020

gonsie commented Sep 23, 2020 •

edited

Loading

artgoldberg commented Sep 29, 2020 •

edited

Loading