Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Dimension Labels and Expandable Datasets Edge Cases #1163

Open
mavaylon1 opened this issue Aug 7, 2024 · 1 comment · May be fixed by #1180
Open

[Bug]: Dimension Labels and Expandable Datasets Edge Cases #1163

mavaylon1 opened this issue Aug 7, 2024 · 1 comment · May be fixed by #1180
Assignees
Labels
category: bug errors in the code or code behavior category: enhancement improvements of code or code behavior

Comments

@mavaylon1
Copy link
Contributor

What happened?

  1. Within __get_dimension_labels_from_spec (or __get_spec_info depending on if VectorData Refactor Expandable #1158 has been merged) uses get_data_shape to get the shape. But this does not work when we have dtype defined:
data = [(1, 2, 3.)]
dtype=[('x', '<u4'), ('y', '<u4'), ('weight', '<f4')])

get_data_shape will return (1,3)
However, when writing the data we use this for the shape:

elif isinstance(dtype, np.dtype):
            data_shape = (len(data),)

which is (1,)

  1. Both VectorData Refactor Expandable #1158 and the current implementation of dimension_labels do not support the case of "untyped, named dataset":
elif spec.data_type_def is None and spec.data_type_inc is None:  # untyped, named dataset
                if spec.name in builder.datasets:
                    sub_builder = builder.datasets[spec.name]
                    self.logger.debug("        Retrieving existing DatasetBuilder '%s' for spec name %s and adding "
                                      "attributes" % (sub_builder.name, repr(spec.name)))
                else:
                    self.logger.debug("        Converting untyped dataset for spec name %s to spec dtype %s"
                                      % (repr(spec.name), repr(spec.dtype)))
                    try:
                        data, dtype = self.convert_dtype(spec, attr_value)
                    except Exception as ex:
                        msg = 'could not convert \'%s\' for %s \'%s\''
                        msg = msg % (spec.name, type(container).__name__, container.name)
                        raise BuildError(builder, msg) from ex
                    self.logger.debug("        Adding untyped dataset for spec name %s and adding attributes"
                                      % repr(spec.name))
                    sub_builder = DatasetBuilder(spec.name, data, parent=builder, source=source, dtype=dtype)
                    builder.set_dataset(sub_builder)

Steps to Reproduce

Above.

Traceback

No response

Operating System

macOS

Python Executable

Conda

Python Version

3.12

Package Versions

No response

@mavaylon1 mavaylon1 added category: bug errors in the code or code behavior category: enhancement improvements of code or code behavior labels Aug 7, 2024
@mavaylon1
Copy link
Contributor Author

@rly We talked about 1), but I did want to point out 2).

@mavaylon1 mavaylon1 self-assigned this Aug 21, 2024
@mavaylon1 mavaylon1 linked a pull request Aug 28, 2024 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: bug errors in the code or code behavior category: enhancement improvements of code or code behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant