Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Small patch to html repr in #1100 #1201

Draft
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

h-mayorquin
Copy link
Contributor

Motivation

There are some edge cases that #1100 did not take into account. I am submitting this patch for discussion, happy to close if a better solution comes.

The problem is that the code here assumes that things that have an IO are hdf5 datasets:

# get info from hdf5 dataset
compressed_size = dataset.id.get_storage_size()
if hasattr(dataset, "nbytes"): # TODO: Remove this after h5py minimal version is larger than 3.0
uncompressed_size = dataset.nbytes
else:
uncompressed_size = dataset.size * dataset.dtype.itemsize
compression_ratio = uncompressed_size / compressed_size if compressed_size != 0 else "undefined"
hdf5_info_dict = {"Chunk shape": dataset.chunks,
"Compression": dataset.compression,
"Compression opts": dataset.compression_opts,
"Compression ratio": compression_ratio}

hdmf/src/hdmf/container.py

Lines 754 to 763 in be602e5

"""Generates HTML for array data"""
read_io = self.get_read_io() # if the Container was read from file, get IO object
if read_io is not None:
repr_html = read_io.generate_dataset_html(array)
else:
array_info_dict = get_basic_array_info(array)
repr_html = generate_array_html_repr(array_info_dict, array, "NumPy array")
return f'<div style="margin-left: {level * 20}px;" class="container-fields">{repr_html}</div>'

But sometimes, this assumption is false. For example, the starting frames of an ImageSeries are a numpy object even after they are written. Maybe there are more such cases?

How to test the behavior?

The following code generates an error when using dev.

from pynwb.testing.mock.file import mock_NWBFile
from pynwb.image import ImageSeries

nwbfile = mock_NWBFile()

series = ImageSeries(name="ImageSeries", description="", external_file=["test"], rate=0.1)
nwbfile.add_acquisition(series)


from pynwb import NWBHDF5IO

nwbfile_path = "./test_nwb.nwb"
with NWBHDF5IO(nwbfile_path, 'w') as io:
    io.write(nwbfile)
    
io = NWBHDF5IO(nwbfile_path, 'r')
nwbfile_read = io.read()
nwbfile_read

AttributeError: 'numpy.ndarray' object has no attribute 'id'

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

Copy link

codecov bot commented Nov 6, 2024

Codecov Report

Attention: Patch coverage is 66.66667% with 3 lines in your changes missing coverage. Please review.

Project coverage is 89.12%. Comparing base (be602e5) to head (1fc5879).

Files with missing lines Patch % Lines
src/hdmf/backends/hdf5/h5tools.py 66.66% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1201      +/-   ##
==========================================
- Coverage   89.12%   89.12%   -0.01%     
==========================================
  Files          45       45              
  Lines        9944     9945       +1     
  Branches     2825     2826       +1     
==========================================
  Hits         8863     8863              
  Misses        762      762              
- Partials      319      320       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@stephprince
Copy link
Contributor

@rly this was the case we discussed in person yesterday, do you know if there any other edge cases we should consider when a Container would have a read_io object but the array would not be an hdf5 dataset?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants