Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rc/2.0.0 #1365

Merged
merged 8 commits into from
Aug 10, 2021
Merged

rc/2.0.0 #1365

merged 8 commits into from
Aug 10, 2021

Conversation

rly
Copy link
Contributor

@rly rly commented Jun 7, 2021

This branch/PR tracks breaking changes to be released in PyNWB 2.0.0.

Guidelines:

  • Merge PRs with breaking changes into this branch.
  • Periodically merge updates from dev into this branch and make fixes as necessary.
  • Otherwise avoid modifying this branch directly.
  • This PR may involve changes to tens of files, and because each commit has been individually approved, this PR need not be reviewed with a close eye.
  • When ready to release these changes:
    • merge dev into this branch
    • make fixes as necessary
    • rebase this branch
    • merge this branch into dev
    • delete this branch
  • After all CI tests in dev pass, and before any minor/patch releases are cut, cut a major release from dev.

This workflow modifies the simple GitHub Flow workflow to handle the bundling of breaking changes into a major release without disrupting minor/patch releases out of the main development branch.

@codecov
Copy link

codecov bot commented Jun 9, 2021

Codecov Report

Merging #1365 (0ef954f) into dev (441c64f) will increase coverage by 3.25%.
The diff coverage is 86.92%.

Impacted file tree graph

@@            Coverage Diff             @@
##              dev    #1365      +/-   ##
==========================================
+ Coverage   73.53%   76.78%   +3.25%     
==========================================
  Files          37       37              
  Lines        2331     2731     +400     
  Branches      357      455      +98     
==========================================
+ Hits         1714     2097     +383     
+ Misses        556      552       -4     
- Partials       61       82      +21     
Impacted Files Coverage Δ
src/pynwb/io/image.py 66.66% <0.00%> (ø)
src/pynwb/validate.py 0.00% <0.00%> (ø)
src/pynwb/io/base.py 55.81% <36.66%> (+25.30%) ⬆️
src/pynwb/icephys.py 89.52% <88.05%> (-0.56%) ⬇️
src/pynwb/file.py 86.36% <96.59%> (+3.60%) ⬆️
src/pynwb/base.py 94.68% <98.55%> (+1.82%) ⬆️
src/pynwb/behavior.py 100.00% <100.00%> (ø)
src/pynwb/image.py 95.06% <100.00%> (+0.61%) ⬆️
src/pynwb/io/file.py 82.27% <100.00%> (+1.88%) ⬆️
src/pynwb/io/icephys.py 100.00% <100.00%> (+33.33%) ⬆️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 441c64f...0ef954f. Read the comment docs.

@rly
Copy link
Contributor Author

rly commented Jul 21, 2021

Here are the remaining issues to address / PRs to merge for PyNWB 2.0.0 and nwb-schema 2.4.0. Happy to discuss removing/adding items.

PyNWB changes left to do (2.0.0):

Schema changes (NWB 2.4.0):

@oruebel
Copy link
Contributor

oruebel commented Jul 21, 2021

@rly just an update on the icephys PRs:

oruebel and others added 6 commits August 10, 2021 11:16
Declare SweepTable as deprecated on init

Add TimeSeriesReferenceVectorData Container class

Add simple test for TimeSeriesReferenceVectorData

Update nwb-schema branch

Add icephys metadata container classes

Update NWBFile to support icephys metadata tables

Add unit tests for icephys metadata tables

Update imports and fix SweepTable integration tests

Fix test available namespaces testcase

Update SweepTable tutorial and declare as deprecated

Add new icephys tutorial using the new hierarchical metadata tables

Add IntracellularRecordingsTable illustration

Fix flake8 issues

Remove use of HierarchiclaDynamicTableMixin use for now

Add source PowerPoint file for icephys docs

add doc for TimeIntervals (#1352)

* add doc for TimeIntervals

* add allensdk back

Fix make clean command for docs to also clean sphinx-gallery files

Updated CHANGELOG to describe changes from this PR

Add thumbnails for domain-specfic tutorials

Add thumbnaisl for general tutorials

Update changelog

Remove ic_filtering/icephys_filtering duplicate and fix test

Fix description of ExperimentalConditionsTable

Use HDMF 2.5.0 (#1354)

DOC: make template less puzzling in how to fill out ;-) (#1336)

Co-authored-by: Ryan Ly <[email protected]>

Add ReadTheDocs badge to README (#1311)

Co-authored-by: Ryan Ly <[email protected]>

Update to use upcoming HDMF 2.5.1 (#1355)

Revert breaking changes to scratch API (#1356)

Use HDMF 2.5.2 (#1360)

Add `DfOverF` to docs (#1351)

Co-authored-by: Ryan Ly <[email protected]>

docs/gallery/general/file.py: Fix argument count in documentation

add strain optional arg to Subject (#1241)

* remove validate_core_schema.

You can check the core schema with e.g.

* Update Subject unit test

* Fix Subject unit test, add check for date_of_birth

Co-authored-by: Ryan Ly <[email protected]>

add source channels to DecompSeries

add option to include source_channels for DecompositionSeries (#1258)

* add option to include source_channels for DecompositionSeries and include round trip test
* Use latest schema
* Add unit test

Add optional link to Device in ImageSeries (#1265)

* Add optional link to Device in ImageSeries

* Add support for device in ImageSeries subclasses

* Update ImageMaskSeries docstring

* Update schema

add continuity optional arg to TimeSeries.__init__ (#1226)

* add continuity optional arg to TimeSeries.__init__

* use new enum feature of hdmf to check against controlled vocabulary

* Fix flake8

* Add integration test

* Fix flake8

* Add correct mapping for data.continuity

Co-authored-by: Ryan Ly <[email protected]>

Add support for ElectricalSeries.filtering dataset (#1270)

Set fixed value for IZeroClampSeries stimulus description

Fix test

Update changelog

Add note about allensdk py3.8 incompatibility

Add workaround for validation against cached hdmf-experimental

Update CI to avoid click8 incompatibility

Use HDMF 2.5.5

Address comments, improve doc

Fix docs, add citation info, update repo infra (#1362)

Use latest HDMF, raise min pandas, dissociate req-min and setup reqs (#1363)

Use nwb schema branch for icephys meta

Ensure start_index and count are set to -1 if TimeSeries is None

Add test for conversion of start_index and index_count vals

Fix flake8 in __compute_index docstring

Fix icephys metadata roundtrip NWBFile test base on update DynamicTableRegion behavior

Update schema (updated version to 1.4.0-alpha)

improve extensions documentation (#1350)

* improve extensions documentation

* adjustments to extensions.rst

* add tips to extension docs

* create multi-part extensions tutorial

* move API building docs to chapter 5 of extensions guide

* restructure and fix up some spots

* Update docs/source/extensions_tutorial/2_create_extension_spec_walkthrough.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/2_create_extension_spec_walkthrough.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/2_create_extension_spec_walkthrough.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/6_documenting_extension.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/4_auto_api.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Oliver Ruebel <[email protected]>

* remove detailed GroupSpec API instructions from tutorial

* remove instructions for generated the directory structure, and instead focus on generating the HTML and filling in the missing pieces.

* add link to ndx catalog

* Update docs/source/extensions_tutorial/2_create_extension_spec_walkthrough.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/2_create_extension_spec_walkthrough.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/2_create_extension_spec_walkthrough.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/4_auto_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/5_custom_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/4_auto_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/4_auto_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/4_auto_api.rst

* Update docs/source/extensions_tutorial/3_spec_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/5_custom_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update docs/source/extensions_tutorial/5_custom_api.rst

Co-authored-by: Ryan Ly <[email protected]>

* Update extensions_tutorial_home.rst

* Minor text fixes

* Minor text fixes

* Minor text fixes

Co-authored-by: Oliver Ruebel <[email protected]>
Co-authored-by: Ryan Ly <[email protected]>

Set language for code block to fix readthedocs

Add RRID for PyNWB

Add RRID to README

Update changelog

make MotionCorrection test (#1374)

* attempt to make MotionCorrection test. I don't understand why it's failing

* Fix CorrectedImageStack nwbfields

* Fix tests

* Fix test

* Fix flake8

* Fix test

Co-authored-by: Ryan Ly <[email protected]>

Add unique electrode id check (#1344)

Co-authored-by: Ryan Ly <[email protected]>

Fix integration tests with invalid test data that will be caught by future hdmf validator version (#1366)

Co-authored-by: Ryan Ly <[email protected]>

Improve documentation and test for CorrectedImageStack (#1306)

Fix test for validator update (#1376)

Update changelog

Allow h5py 3 so that this can work with hdmf rc/3.0.0

Move SweepTable DeprectationWarning to the constructor of SweepTable to clean up code

Suppress SweepTable Deprecation warning in icephys integration test

Ignore and assert deprecation warning for SweepTable during SweepTable roundtrip tests

Merge branch 'rc/2.0.0' into add/icephys_meta

Fix falke8 on icephys tutorial galleries
Drop py36 support, update CI, reqs, tests for rc/2.0.0 (#1377)
Use  extension in docs to simplify linking to common targets

Move advanced data I/O tutorials to their own section and move parallel I/O tutorial

Update Changelog for tutorials

Updated deprecation warning in SweepTable tutorial

Fix minor spelling error in make_test_files

Several corrections and enhancments for the icephys tutorial

Moved functions to create icephys test file from tests to pynwb

Moved functions to create icephys test file from tests to pynwb

Start for new tutorial to show conversion of icephys tables to pandas dataframes

Test to_hierarchical_dataframe and to_denormalized_dataframe functions

Updated tutorial to convert icephys tabels to pandas

Fix minor spelling error in NWBFile docstring

Add TimeseriesReferenceVectorData.get method to mask missing values on load

Update IntracellularRecordingsTable. Update add_recording defaults for startindex and count to None and update to_dataframe method to handle missing values in TimeSeriesReferenceVectorData

Update create_icephys_testfile to create testdata with missing stimuli

Update existing tests to match the new behavior

Update icephys tutorial for match new behavior

Update icephys pandas tutorial to demonstrate new behavior

Remove debug print statements

Add table augmentation and queries to the icephys_pandas tutorial

Fix flake8

Update to use keyword args

Clarify text in icephys tutorial

Fix flake8 on tutorial

Fix flake8

Update to use dev branch of nwb-schema

Set default name and default description for TimeSeriesReferenceVectorData

added icephys_testutils to pynwb.testing.__init__

Added test for IntracellularRecordingsTable.to_dataframe with options

Always use MaskedArrya in TimeSeriesReferenceVectorData instead of MaskedConstant

Add tests for TimeSeriesReferenceVectorData.get

Updated change log

Fix minor docstring issue in TimeSeriesReferenceData.get

Clarify docstring for TimeSeriesReferenceData.get

Fix build warnings in docs (#1380)

* Remove unused and outdated convert.rst from the docs  Fix #1378
* Fix extra numbered footnote reference in ophys tutorial
* Change code block highlighting from c to bash to avoid build warning
* Fix duplicate target here warning in 3_spec_api.rst
* Fix missing section label for crossreferencing between the extension tutorial and gallery
* Updated Changelog

Update CHANGELOG.md

Update icephys.py

Update docs/gallery/domain/plot_icephys.py

Update plot_icephys.py

Minor text fixes

Update plot_icephys_pandas.py

Minor text edits

Update icephys_testutils.py

Minor docstring edits

Update file.py

Fix typos

Update docs/gallery/domain/plot_icephys.py Replace master with main

Mention use of create_icephys_testfile function

Co-authored-by: Ryan Ly <[email protected]>

Remove old comment in TimeSeriesReferenceVectorData

Co-authored-by: Ryan Ly <[email protected]>

Fix typo in changelog

Fix broken link target

Fix flake8 in docs/gallery

Minor text fixes

Minor text and formatting fixes

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Use namedtuple instead of numpy masked structed array to represent values of TimeSeriesReferenceVectorData

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Update tests/unit/test_icephys_metadata_tables.py

Co-authored-by: Ryan Ly <[email protected]>

Fix bad indent in test

Update base.py - minor text edits

Use hdmf 3.1.1

Enhance introspection, slicing, and data addition for TimeSeriesReference and TimeSeriesReferenceVectorData

Updated icephys query tutorial to use latest get_linked_tables behavior and improve rendering

Fix flake8 in gallery

Fix spelling in error message

Update src/pynwb/base.py

Co-authored-by: Ryan Ly <[email protected]>

Update src/pynwb/base.py

Co-authored-by: Ryan Ly <[email protected]>

Update src/pynwb/base.py

Co-authored-by: Ryan Ly <[email protected]>

Update src/pynwb/base.py

Co-authored-by: Ryan Ly <[email protected]>

Update src/pynwb/base.py

Co-authored-by: Ryan Ly <[email protected]>

Update src/pynwb/base.py

Co-authored-by: Ryan Ly <[email protected]>

Fix bug in TimeSeriesReference.timestamps and add comments

Fix gallery tests to handle allensdk pinning pynwb/hdmf

Fix rebase
@rly rly marked this pull request as ready for review August 10, 2021 18:23
@rly
Copy link
Contributor Author

rly commented Aug 10, 2021

I have painstakingly rebased this PR from 120+ commits down to 8 because I would like to merge this without squashing the individual commits. I double checked that the changes after rebase are the same as before the rebase.

This is now ready to go. The individual changes have previously been reviewed and need not be carefully inspected.

@rly rly requested review from oruebel and bendichter August 10, 2021 18:26
@oruebel
Copy link
Contributor

oruebel commented Aug 10, 2021

Great job @rly Thanks for all the hard work on this!

@rly rly merged commit c0ad3e7 into dev Aug 10, 2021
@rly rly deleted the rc/2.0.0 branch August 10, 2021 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants