CHANGES.txt

pypet 0.6.1

* Support for NumPy >= 1.20
* Removed travis-ci and appveyor dependency

pypet 0.6.0

* Fixed broken support for scipy sparse matrices
* Sparse matrices are now serialized using scipy save_npz function and hex conversion
* BACKWARDS INCOMPATIBLE CHANGE: Can no longer load sparse matrices stored with older versions!

pypet 0.5.2

* Fixed import from collections.abc for Python 3.10

pypet 0.5.1

* Updated package description to automatically convert md to rst for pypi
* Updated pngpath for Sphinx

pypet 0.5.0

* Fix to work with pandas 1.0
* Fix to work with brian2 2.3
* Fix to work with Python 3.7 and 3.8
* Removal `expectedrows` and `filters` option for HDF5Storage.put as this is no longer supported by pandas

pypet 0.4.3

*   DEPR: Removed pandas Panel and Panel4D (see also https://github.com/pandas-dev/pandas/pull/13776)

*   Removed support for Python 3.3 and 3.4

pypet 0.4.2

*   BUG FIX: Brian2Parameters can now be merged [Issue #50]

pypet 0.4.1

*   BUG FIX: Fixed slow `f_get_from_all_runs` with run groups

*   More intuitive behaviro for `f_to_dict` to only start at initiating group node.

*   BUG FIX: brian 2 removed get unit fast, this has been replaced with a pypet implementation


pypet 0.4.0

*   BUG FIX: Correct progressbar printing in Python3 using reprint

*   ENH: New ``nested`` option for ``f_to_dict``

*   Python 3 Support ONLY!!!


pypet 0.3.0

*   Support for BRIAN2

*   New Brian2Parameter

*   New Brian2Result and Brian2MontitorResult

*   New BRIAN2 Network framework

*   BUG FIX: Auto loading parameters will set the parameter to the current run.


pypet 0.2.0

*   No longer beta


pypet 0.2b1

*   ENH: Pypet can now apply fast merging if trajectories reside in different files.
    If one still wants to apply slow merging, one can set `slow_merge=True`.

*   ENH: New `merge_all_in_folder` function

*   The range is now a list that is by default copied upon request. To avoid
    quadratic complexity with multiple calls to `f_expand`.

*   ENH: Meta data of single runs is no longer stored during the
    single run to avoid overhead and guarantee faster runs

*   For more stability continuing is no longer supported for the
    ``'QUEUE'`` and ``'PIPE'`` wrap modes.

*   Queue retries are now logged in debug mode.

*   One can now choose what `f_iter_runs` should return.

*   New `f_run_map` and `f_pipeline_map` functionality for iterable arguments.

*   New wrap mode ``'LOCAL'`` to ensure data is only stored by the main program

*   Immediate post-processing is now supported by all wrap modes.

*   PIPE wrapping is no longer supported under Windows

*   The `add_time` parameter of the trajectory is by default now `False`

*   New `niceness` feature to prioritize child processes

*   New `scoop` feature to allow multiprocessing with the scoop package.

*   Passing `log_multiproc=False` to an environment disables multiprocess logging.

*   Parameters and Results can nest data in HDF5 themselves using dot notation.

*   Removed BrianDurationParameter

*   Parameters now support lists and no longer make numpy arrays unwritable

*   Results are now always sorted according to the run indices
    (except for immediate post-processing)

*   Process ID and hostname are also added to the logfiles' naming scheme

*   `freeze_pool_input` has been renamed to `freeze_input` because it can be used
    with SCOOP as well.

*   New ``'NETLOCK'`` wrap mode, to allow locking over a network and usage
    of servers with a shared home directory.

*   Prefixes are no longer mandatory if using `func` and `vars`:
    `traj.f_iter_runs()` is equivalent to `traj.func.iter_runs()` or
    `myresult.v_name` is equivalent to `myresult.vars.name`

*   BUG FIX: If a trajectory is loaded `as new` the explored parameters are
    now correctly loaded

*   New ``graceful_exit`` mode to allow to stop your programs via `CTRL+C` while finishing
    all your active runs first and storing all your remaining data.

*   New `v_no_clobber` property that allows to ignore additions to the trajectory that
    are already there.

*   New `rts_X` abbreviation that translates a given run index into the correspondin run set,
    e.g. ``rts_4007`` gives ``run_set_00004```.

*   `log_stdout` is `False` by default.

*   *pypet* can now be combined with DEAP optimization.

*   All deprecated functions have been removed!

*   Lazy adding is no longer supported!


pypet 0.2b.0

*   Erroneous Release due to PyPI fault :/


pypet 0.1b.12

*   Renaming of the MultiprocContext's `start` function to `f_start`

*   BUG FIX: Correct lock acquiring for multiprocessing with StorageContextManager

*   BUG FIX: `v_full_copy` is now `False` by default

*   BUG FIX: `v_full_copy` is no longer automatically set to `True` when using
    `freeze_pool_input`.

*   New `consecutive_merge` parameter for `f_merge` to allow faster merging of several
    trajectories.

*   New `f_merge_many` function to merge many trajectories at once.

*   New experimental ``'PIPE'`` wrapping which is even faster (but more unreliable)
    than ``'QUEUE'``.


pypet 0.1b.11

*   If one wants the old logging method, `log_config` should not be specified, i.e.
    setting it to `None` is no longer sufficient`

*   BUG FIX: Connection loss between the queue manager and the pool/processes has been resolved.
    This caused a minor slowing down of multiprocessing using a queue.

*   New ``freeze_pool_input`` functionality for faster starting of single runs if using
    a pool.


pypet 0.1b.10

*   New `v_crun_` property simply returning ``'run_ALL'`` if ``v_crun`` is ``None``.

*   BUG FIX: Removed recursive evaluation due to usage of `itertools.chain` during
    recursive node traversal

*   max_depth is now also supported by `f_store`, `f_store_child`, `f_load`, `f_load_child`

*   Loading and Storing internally are no longer truly recursive but iteratively handled.

*   New `v_auto_run_prepend` property of the trajectory to switch off auto run name prepending if
    desired.

*   The trajectory no longer relies on evil `eval` to construct a class. Instead
    it relies on the global scope.

*   Better counting of loading and storing nodes to display rate in nodes/s

*   BUG FIX: Minor bug in the progressbar has been fixed to detect automatic resets.

*   Now support for non-nested empty containers: Empty dictionary, empty list, empty tuple and
    empty numpy array. All of them supported by the ArrayParameter and normal Results.

*   Support for Sparse matrices containing *NO* data (i.e. only zeros).

*   Performance optimization for storage and loading

*   Improved test handling and parsing in `pypet.tests`

*   Environment now supports `git_fail` option to fail if there are not committed changes
    instead of triggering a new commit.

*   Users can now define their own functions to produce run-names

*   Likewise users can define their onw wildcards

*   The lazy version of adding elements (`traj.par.x = 42, 'A comment') now needs to
    be turned on by the user via (`traj.v_lazy_adding = True`) before it can be used.

*   HDF5_STRCOL_MAX_ARRAY_LENGTH has been renamed to HDF5_STRCOL_MAX_RANGE_LENGTH

*   The summary tables have been shortened. Now there's no distinction anymore between
    the actual runs and everything else.

*   Moreover, data added to summary tables is no longer deleted. There also exists a
    maximum length for these tables (1000).

*   The overview about the explored parameters in each run has been removed (due to size)

*   Summary tables are now only based on the comments not the names!

*   One can pass an estimate for memory that each run needs to better protect the memory
    cap.

*   All tree nodes except the trajectory now use __slots__ for faster and more compact
    creation.

*   You can now request to load a trajectory without `run_information` to save time for huge
    trajectories

*   Trajectories use ordered dictionaries to remember what was added during a single run.
    Accordingly, now every data added during a single run regardless if they were added below
    a group `run_XXXXXXXX` is stored.

*   BUG FIX: The `'QUEUE'` wrapping no longer waits for chunks of data, but starts
    storing immediately. Thus, if you have fast simulations, the storage service no longer
    waits until the end of you simulation before it starts storing data.
    In order to avoid overhead, the hdf5 is simply kept open until
    the queue is closed.

*   BUG FIX: If `log_stdout=True`, the original stream is restored instead of
    `sys.__stdout__`. Thus, using another Python interpreter with a redirection of
    `stdout` and calling `f_disable_logging` no longer disables `print` completely.

*   Refactored 'QUEUE' wrapping. The user can now decide the maximum size of the Storage Queue.

*   CAP values are now in `%`, so choose values between 0.0 and 100.0

*   BUG FIX: Links removed during single runs are now no longer stored

*   BUG FIX: `pypet.` is no longer prepended to unpickled logger names. Accordingly,
    pypet logger names are now fully qualified names like `pypet.trajectory.Trajectory`.


pypet 0.1b.9

*   BUG FIX: Fixed backwards compatibility

*   BUG FIX: Metadata is loaded only once

*   Results no longer support the ``v_no_data_string`` property

*   Data of Results is no longer sorted in case of calling `f_val_to_string`

*   In accordance with the python default to call `__repr__` for displaying contained
    objects, `f_val_to_str` calls `repr` on the contained data in parameters and results.

*   Added informative __repr__ for the most commonly used classes

*   The (annoyingly long) keyword `dynamically_imported_classes` is changed to
    `dynamic_imports`. For backwards compatibility, the old keyword can still
    be used.

*   New `f_get_default` method, now one can specify a default value that should be
    returned if the requested data is not found in the trajectory

*   `progressbar` displays the run and remaining time.

*   New LINK features allowing group nodes to refer to other parts of the tree

*   The SingleRun has been removed and all interactions are with real Trajectory objects,
    but the API remained the same.

*   All *pypet* relevant imported packages will be stored by the trajectory

*   Internally the queue no longer needs to be re-passed to the QueueSender, allowing for
    easier custom multiprocessing

*   New MultiprocessWrapper (aka a light-weight environment for multiprocessing) for
    custom multiprocessing

*   StorageServices provide an ``multiproc_safe`` attribute to enable the user to check
    if they work in a multi-process safe environment

*   Environments can be used as context managers to disable the logging to files after the
    experiment.

*   Environments provide the ``v_log_path`` property to get the current log path

*   BUG FIX: Trajectories with only a single explored parameter can now be merged
    several times

*   Backwards search no longer supported!

*   `f_get_all` now supports shortcuts and abbreviations like `crun` or `par`

*   `$` always translates to the run the trajectory is set to, also for
    adding new items to the tree

*   If the current run is not set, ``traj.v_crun`` is set to ``None``

*   Moreover, ``f_iter_nodes`` and ``f_iter_leaves`` is no longer affected by the setting
    of a current run and always return all nodes and leaves

*   The iteration functions from above now allow for a predicate function to filter potential nodes

*   Storing a leaf or a group via ``traj.f_store_item(item, overwrite=True)`` now also
    replaces all annotations and comments

*   Passing `overwrite_file=True` to an environment will overwrite the hdf5 file.

*   `remove_empty_groups` is no longer supported

*   All messages logged by *pypet* are now no longer using the root logger but one called 'pypet'.

*   Better customization of logging. The user can now pass a list of ``logger_names`` and
    corresponding ``log_levels`` which are logged to files in the ``log_path``.

*   The environment no longer adds config information about hdf5 to the trajectory directly.
    This is now done by the service itself.

*   The keyword arguments passed to the environment regarding the storage service are
    no longer handled by the environment but are directly passed to the service.

*   BUG FIX: Fixed merging of result summaries that are only found in one trajectory

*   BUG FIX: Log files are now closed when the handlers are removed

*   BUG FIX: ``max_depth`` is now really always in relation to the start node and not
    in relation to intermediate results

*   API change for `f_migrate` to match new concept of storage service

*   Short function names for item additions like `f_apar` besides `f_add_parameter`.

*   Abbreviations like `par` and `dpar` can now also be used for item creation and
    are always translated

*   To streamline the API you can now no longer specify the name of backup files for merging

*   Locked parameters can no longer be loaded and must be unlocked before.

*   Parameters are no longer required to implement ``__len__`` because it can be
    ambiguous, instead they must implement ``f_get_range_length`` function.

*   BUG FIX: ``crun`` is now also accepted for adding of data and not only for requests

*   Setting `ncores=0` lets *pypet* determine the number of CPUs automatically (requires psutil).


pypet 0.1b.8

*   Support for python 3.3 and 3.4!

*   Proper handling of unicode strings (well, see above^^)

*   Checking if names of leaf and group nodes only contain alphanumeric characters

*   PickleParameter and PickleResult now explicitly store the pickle protocol
    because retrieval from the pickle dump is not always possible in python 3.

*   Children of groups are no longer listed via __dir__ in case of debugging to
    prevent unwanted locking.

*   Better support for PyTables 2 and 3 with same code base.

*   pypet and pypet.brian now provide the __all__ list.


pypet 0.1b.7

*   StreamToLogger has moved to the pypetlogging.py module. The mplogging module was
    deleted.

*   The Network Manager now accepts custom network constructors

*   A SingleRun can now provide a `v_filename` and `v_as_run` property.
    Both cannot be changed and the latter simply returns the name of the run.

*   Better testing on travis

*   Better support for pandas 0.14.1

*   Now you can import most of the objects directly from the *pypet* package, e.g.
    ``from pypet import Trajectory`` instead of ``from pypet.trajectory import Trajectory``.


pypet 0.1b.6

*   The storage service prints status updates for loading and storing trees

*   v_as_run is not longer `None` for a trajectory but `run_ALL`

*   The Queue storage writer now stores batches of single runs to avoid
    re-opening of files as much as possible

*   Faster Loading of data

*   Support for PyTables 3.1.1

*   *pypet* stores the name of the main script as a config parameter

*   Data of Parameters can be accessed via `.data` or `param['data']`.
    Same holds for results that only contain a single data item.

*   Parameters provide the function `f_get_default` to return the default value
    if the parameter is not empty.

*   Large dictionaries and Object Tables are now split into chunks of 512 items

*   In case an object table has more than 32 columns, another table is created to
    store the data item types (which is faster than storing all of the types as
    hdf5 node attributes)


pypet 0.1b.5

*   New auto load feature

*   BUG FIX: When parameters are emptied, the default value is set to None (and no longer kept)

*   Now items are only saved once, if the node already exist on disk, storage is refused
    (Previously new data was added if it was not within the leaf before, but this can lead to
    strange inconsistencies).

*   BUG FIX: `f_has_children` of a group node, now returns the correct result

*   Refactored continuing of trajectories. Now this is based on `dill` and works also
    with data that cannot be pickled. `f_continue_run` is renamed `f_continue` to emphasize
    this change in API

*   Picking the search strategy and using `v_check_uniqueness` is no longer supported.
    Sorry for the inconvenience. So forward search will always check if it finds 2 nodes
    with the same name within the same depth, and skip search if the next tree depth is entered.

*   `f_contains` of group nodes has per default `shortcuts=False`

*   There exists now the abstract class `HasLogger` in pypetlogging.py that establishes
    a unified logging framework

*   Likewise the loggers of all network components are now private objects `_logger` and
    so is the function `_set_logger`.

*   BUG FIX: `f_get_run_information` now works without passing any arguments

*   Trajectories no longer accept a `file_tile` on initialisation

*   One can now decide if trajectories should be automatically stored and if
    data should be cleaned up after every run

*   BUG FIX: Storage of individual items during a single run do no longer require
    a full storage of the single run container.

*   If automatic storage is enabled, trajectories are now stored at the end of the experiment,
    no longer before the starting of the single runs

*   You can use the `$` character to decide where the HDF5 file tree should branch out for the
    individual runs

*   `v_creator_name` is now called `v_run_branch` (since single runs can also create
    items that are not part of a run branch, so this is no longer misleading`.

*   Results and parameters now issue a warning when they have been stored and you
    change their data

*   Parameters now have a property `v_explored` which is True for explored parameters
    even if the range has been removed

*   By default `backwards_search` is turned off!

*   Brian parameters no longer store the `storage_mode` explicitly

*   BUG FIX: Wildcard character now always defaults to `run_ALL` if trajectory is not set
    to particular run

*   BUG FIX: Now names with `XXXrun_` are again allowed only `run_` are forbidden.


pypet 0.1b.4

*   Annotations and Results now support __setitem__, which is analogue to `f_get` and `f_set`

*   Group Nodes can now contain comments as well

*   Comments are only stored to HDF5 if they are not the empty string

*   Large Overview Tables are off by default

*   BrianDurationParameter was removed and the annotations are used instead.
    Use a normal BrianParameter and instead of `v_order` use `v_annotations.order`

*   The user is advised to use `environment.f_run(manager.run_network)`, instead of
    `environment.f_run(run_network, manager)`

*   Now there is the distinction between `small`, `large`, and `summary` tables

*   BrianMonitorResult: Mean and variance values of State and MultiState Monitors are only
    extracted if they were recorded (which for newer BRIAN versions is only the case if you
    do NOT record traces)

*   Log Level can be passed to environment

*   BUG FIX: Scalars are no longer autoconverted by the storage service to zero-length numpy arrays

*   Names of loggers have been shortened

*   The trajectory now contains the functions `f_delete_item` and `f_delete_items` to erase stuff
    from disk. `f_remove_items` and `f_remove_item` no longer delete data from disk.

*   Loading and deletions of items can now be made with SingleRuns as well.

*   f_iter_nodes now iterates by default recursively all nodes

*   A group node now supports __iter__ which simply calls f_iter_nodes NON recursively

*   The structure of the trees are slightly changed. Results and derived parameters added
    with the trajectory are no longer assigned the prefix `trajectory`.
    Results and derived parameters added during single runs are now sorted into
    `runs.run_XXXXXXXXX`.

*   Useless shortcuts have been removed.

*   New *Backwards* search functionality

*   New `f_get_all` functionality to find all items in a tree matching a particular
    search string

*   Pandas Series and Panels are now supported, too!

*   Now Pypet allows compression of HDF5 files, this can yield a massive reduction in
    memory space.

*   `tr`, `cr`, `current_run`, `param`, `dparam` are no longer supported as a shortcuts

*   __getitem__ is equivalent to __getattr__

*   Now one can specify a maximum depth for accessing items.

*   Now one can specify if shortcuts, i.e. hopping over parts of the tree, are allowed.

*   New trajectroy attributes `v_backwards_search`, `v_max_depth`, `v_shortcuts` and
    `v_iter_recursive`. `v_iter_recursive` specifies the behavior of `__iter__`.

*   `__contains__` now greps is arguments from the attributes `v_max_depth` and `v_shortcuts`.

*   `log_stdout` parameter of the Environment determines if STDOUT and STDERROR will be logged
    to files. Can be disabled to allow better usage of pypet with interactive consoles.

*   git commits only happen on changes and not all the time

*   Now one can specify CPU, RAM and Swap cap levels if no pool is used.
    If these cap levels are crossed *pypet* won't start new processes.

*   `f_load` now has an argument `load_all` to quickly load all subtrees with the same setting.
    Also `f_load` now accepts a filename as well

*   New post-processing and pipeline modes!


pypet 0.1b.3

*   BUG FIX: Now `f_run` and `f_continue_run` of an environment return the results
    produced by `runfunc`

*   You can enforce a type convert for exploration

*   Added lazy_debug option for the environment

*   If you don't specify a filename, the environment defaults to a file with the name of
    the trajectory

*   New multiprocessing mode (`use_pool=False` for environment) to spawn individual processes
    for each run. Useful if data cannot be pickled.

*   New Brian framework with NetworkManager, NetworkComponent, NetworkAnalyser,
    NetworkRunner and DurationParameter

*   Components, Analysers and the network runner of the manager are now publicly available

*   Every component now provides the function `set_logger` to enable logging and instantiate a
    logger for `self.logger`



pypet 0.1b.2

*   DefaultReplacementError is now called PresettingError

*   Now the runtime of single runs is measured and stored.

*   __getitem__ of the trajectory always returns the instance and fast access is not applied

*   PickleResult and PickleParameter support the choice of protocol

*   Explored Sparse matrices are stored under a slightly different name to disk.

*   BUG FIX: BFS works properly

*   BUG FIX: f_iter_runs is now affected if f_as_run is chosen

*   Annotations support __iter__

*   Annotations support __getitem__

*   BrianMonitorResult, the property and columns 'times' for the Spike and StateSpikeMonitor
    has been renamed 'spiketimes'.

*   Results support __iter__

*   BrianMonitorResult, the name of state variables in array mode
    is changed to varname+'_%Xd', instead of varname+'_idx%08d', and 'spiketimes_%08d' to
    'spiketimes_%Xd' and X is chosen in accordance with the number of neurons

*   BUG FIX: nested_equal now supports Object Tables containing numpy arrays

*   Better categorizations of the utility functions

*   Comments are no longer limited in size

*   New Brian Result

*   Storage Service - in case of purging - now sets the comment to the result with the lowest index,
    in case of multiprocessing.

*   Old API names are kept, but emit a depricated warning.

*   The exploration array is now termed range. Accordingly, the function `f_is_array` is
    renamed `f_has_range` and `f_get_array` renamed to `f_get_range`.

*   v_leaf renamed to v_is_leaf

*   f_is_root renamed to v_is_root and changed to property

*   v_fast_accessible renamed to f_supports_fast_access and changed to function

*   v_parameter changed to v_is_parameter



pypet 0.1b.1

*   Support for `long` types

*   Documentation for the `f_find_idx` function

*   The parameters `trajectory_name` and `trajectory_index` in f_load have been renamed to
    `name` and `index`


pypet 0.1b.0

*   Group nodes support __getitem__

*   SparseResult

*   If you merge a trajectory, all environment settings of both are kept.

*   More information about the environment is added to the trajectory

*   BUG FIX:

    Recall of trajectory comments from disks yielded numpy strings instead of python strings
    This could cause trouble if the comment is empty!

*   Git Integration, you can now make autocommits for every experiment

*   New Sparse Parameter, for scipy sparse matrices

*   BUG FIX: Loading of Trajectory metadata, now v_time is loaded correctly

*   BUG FIX: Expand no longer repeats already run experiments

*   More fine grain overview tables

*   Comments for runs are only added once and not every run

*   The overview tables are now found in the group `overview`

*   Test are operating in a temp directory

*   Now you can have fast access with results if they contain only a single entry with the name of
    the result

*   New trajectory function `f_as_run` that makes the trajectory belief it is a particular
    single run and results and derived parameters of other runs are blinded out.

*   Every group node now knows how to store and load itself via `f_load` and `f_store`.

*   Storage of data is now analogous to loading with constants in (1,2,3).
    1 Storing data only of items not been stored before, 2 for storing data as
    previously known. 3 For overwriting data. For instance, ``traj.f_load(store_data=3)``
    overwrites all data on disk.

*   f_update_skeleton is now f_load_skeleton to be more in line with naming scheme.

*   setattr no longer supports shortcuts, i.e. ``traj.x = 4`` only works if ``x`` is directly
    below the trajectory root.

*   Using setattr with a tuple of exactly length 2 whereas the second element is a string,
    sets the value as well as a comment


pypet 0.1a.6

*   BUG FIX: (HDF5StorageService): storing a trajectory several times increased run and info table


pypet 0.1a.5

*   Removed unnecessary imports

*   Better documentation


pypet 0.1a.4

*   Adding positional results will add them with the result name


pypet 0.1a.3

*   Better handling of filenames, now relative paths are considered


pypet 0.1a.2

*   Added automatic version grapping in setup.py


pypet 0.1a.1

*   BaseParameter supports now __getitem__ if it is an array