Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose progress bar class control #1110

Merged
merged 15 commits into from
May 20, 2024
Merged

Conversation

CodyCBakerPhD
Copy link
Contributor

@CodyCBakerPhD CodyCBakerPhD commented May 6, 2024

Motivation

For @garretmflynn and the GUIDE project

How to test the behavior?

I will add a minimal test with an in-place extension of tqdm

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

@@ -179,6 +179,12 @@ class GenericDataChunkIterator(AbstractDataChunkIterator):
doc="Display a progress bar with iteration rate and estimated completion time.",
default=False,
),
dict(
name="progress_bar_class",
type=callable,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea if docval is OK with this?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah it's giving an error.

[2024-05-13 14:06:09,926] ERROR in app: Exception on /startup/preload-imports [GET]
Traceback (most recent call last):
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/utils.py", line 605, in dec
    a['type'] = __resolve_type(a['type'])
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/utils.py", line 527, in __resolve_type
    raise ValueError(msg)
ValueError: argtype must be a type, a str, a list, a tuple, or None - got <class 'builtin_function_or_method'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/garrettflynn/Documents/GitHub/nwb-guide/pyflask/apis/startup.py", line 40, in get
    import neuroconv
  File "/Users/garrettflynn/Documents/GitHub/neuroconv/src/neuroconv/__init__.py", line 1, in <module>
    from .basedatainterface import BaseDataInterface
  File "/Users/garrettflynn/Documents/GitHub/neuroconv/src/neuroconv/basedatainterface.py", line 9, in <module>
    from pynwb import NWBFile
  File "/Users/garrettflynn/miniconda3/envs/nwb-guide/lib/python3.9/site-packages/pynwb/__init__.py", line 9, in <module>
    from hdmf.spec import NamespaceCatalog
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/__init__.py", line 2, in <module>
    from .backends.hdf5.h5_utils import H5Dataset, H5RegionSlicer
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/backends/__init__.py", line 1, in <module>
    from . import hdf5
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/backends/hdf5/__init__.py", line 1, in <module>
    from . import h5_utils, h5tools
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/backends/hdf5/h5_utils.py", line 20, in <module>
    from ...data_utils import DataIO, AbstractDataChunkIterator
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/data_utils.py", line 140, in <module>
    class GenericDataChunkIterator(AbstractDataChunkIterator):
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/data_utils.py", line 197, in GenericDataChunkIterator
    def __init__(self, **kwargs):
  File "/Users/garrettflynn/Documents/GitHub/hdmf/src/hdmf/utils.py", line 608, in dec
    raise Exception(msg)
Exception: docval for progress_bar_class: error parsing argument type: argtype must be a type, a str, a list, a tuple, or None - got <class 'builtin_function_or_method'>

Is there a simple way to fix this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rly @oruebel I want to specify a callable in a function/class wrapped by docval

Is there any way to do that or otherwise circumvent it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that callable is not a type in Python but rather a built-in function. If you want to restrict to progress bar classes from tqdm then you could set the type to tqdm.std, which I believe is the common base class for progress bars in tqdm. Or you could make a list of all the progress bar types you want to allow. I believe docval allows this to be specified as a string tqdm.std so that you should not need to import tqdm in order to specify this for docval. Alternatively, if you want any class to pass validation, then I believe you could set type to None.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tqdm.std is the module, not a parent class; tqdm.tqdm is the thing they recommend using as a base class when extending, but in general you could use any generic iterable wrapper if you wanted to

I'll just use the override for now

src/hdmf/data_utils.py Outdated Show resolved Hide resolved
Copy link

codecov bot commented May 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.72%. Comparing base (8a2658f) to head (0753f8a).
Report is 32 commits behind head on dev.

Additional details and impacted files
@@           Coverage Diff           @@
##              dev    #1110   +/-   ##
=======================================
  Coverage   88.72%   88.72%           
=======================================
  Files          45       45           
  Lines        9765     9766    +1     
  Branches     2773     2773           
=======================================
+ Hits         8664     8665    +1     
  Misses        779      779           
  Partials      322      322           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@CodyCBakerPhD
Copy link
Contributor Author

@garrettmflynn Should be good to try again

src/hdmf/data_utils.py Outdated Show resolved Hide resolved
@CodyCBakerPhD CodyCBakerPhD requested review from rly and oruebel May 17, 2024 20:43
@CodyCBakerPhD CodyCBakerPhD marked this pull request as ready for review May 17, 2024 20:43
@CodyCBakerPhD CodyCBakerPhD enabled auto-merge (squash) May 17, 2024 20:49
@CodyCBakerPhD
Copy link
Contributor Author

@oruebel @rly This is ready for review

@CodyCBakerPhD CodyCBakerPhD merged commit e6e6c5b into dev May 20, 2024
25 of 28 checks passed
@CodyCBakerPhD CodyCBakerPhD deleted the progress_bar_class_control branch May 20, 2024 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants