NWB and HDMF Versioning and Compatibility

Overview

The purpose of this document is to define the standard practices for versioning and maintaining compatibility between different versions of the NWB and HDMF software and schema.

Definitions

API Versioning and Compatibility

PyNWB and HDMF (Python APIs)

PyNWB and HDMF use a loose variant of semantic versioning to govern deprecations, API compatibility, and version numbering.

A release version number is made up of MAJOR.MINOR.PATCH.

API breaking changes SHOULD only occur in major releases. These changes will be documented, with clear guidance on what is changing, why it's changing, and how to migrate existing code to the new behavior.

Whenever possible, a deprecation path SHOULD be provided rather than an outright breaking change.

Deprecations MAY be introduced in minor releases. These deprecations MUST preserve the existing behavior while emitting a warning that provide guidance on:

New deprecations MUST NOT be introduced in patch releases.

Deprecations MUST be enforced only in major releases. For example, if a behavior is deprecated in HDMF 3.1.0, it MUST continue to work, with a warning, for all releases in the 3.x series. The behavior MAY change and the deprecation notice MAY be removed in the next major release (4.0.0).

NOTE: Bug fixes MAY change the behavior of the API as part of minor or patch releases. Whether or not a change is a bug fix or an API-breaking change is a judgment call. We will do our best, and we invite you to participate in development discussions on the issue tracker.

These policies do not apply to features marked as experimental in the documentation. These experimental features include all functionality in the “hdmf-experimental” package and in the “hdmf-experimental” namespace. The behavior of experimental features MAY change at any time.

Installed releases of the PyNWB and HDMF packages contain the __version__ attribute:

# complete version string, including suffix ".post.devX" for dev versions

# e.g., "3.0.1" or "3.0.1.post.dev4"

hdmf.__version__

pynwb.__version__

This policy is adapted heavily from the pandas version policy and the scipy version policy.

Python Support

Support for specific Python versions (e.g., 3.6.x, 3.7.x) in PyNWB and HDMF will be dropped only in major or minor releases.

API File Reading Compatibility

Older file compatibility: The PyNWB and HDMF APIs SHOULD be able to read files generated by earlier releases in the same major release series, For example, HDMF API version 3.2.0 should be able to read files generated by HDMF API versions 3.0.0 to 3.2.0.

NOTE: If PyNWB or HDMF reads a file containing an older, cached release of the NWB or HDMF schema and then modifies the file, changes to the file will be written with the newer version of the schema. As a precaution, when writing to a file with a different version, the file should be validated against the newer version of the schema to ensure compliance. As a best practice, it is recommended to avoid using different versions to write to the same file.

NOTE: These policies do not apply to features marked as experimental in the documentation. These experimental features include all functionality in the “hdmf-experimental” package and in the “hdmf-experimental” namespace. The behavior of experimental features MAY change at any time.

Data written using types in the "hdmf-experimental" namespace MAY not be readable in future releases of HDMF.

Newer file compatibility: An older release of the PyNWB or HDMF API may not be able to read some data generated by later releases of the software, particularly if the data uses data types that are not in the older release.

Schema Versioning and Compatibility

In this section, "HDMF schema" refers to the schema defined by the “hdmf-common” namespace and does not include the schema defined by the “hdmf-experimental” namespace.

The NWB and HDMF schema follow the NWB Versioning Guidelines, which are based on semantic versioning.

A release version number is made up of MAJOR.MINOR.PATCH.

Forward compatibility: An NWB or HDMF file compliant with NWB or HDMF schema x.y SHOULD be generally compliant with later minor releases of NWB or HDMF schema x.(y+1).

Later minor and patch releases of the schema MAY:

Later minor and patch releases of the schema SHOULD NOT:

Rare exceptions to this rule, such as bug fixes, MAY be made and will be noted in the schema release notes, and the official NWB APIs will implement appropriate measures to ensure that files created with earlier versions of the schema are still compatible with the APIs. Whether or not a change merits a major release is a judgment call. We'll do our best, and we invite you to participate in development discussions on the issue tracker.

For example, a data type may be added in HDMF schema 2.2.0 and, shortly after, it was found that the shape was specified incorrectly and the type cannot be used as intended. This may be fixed in HDMF schema 2.2.1. In this case, an HDMF schema 2.2.0 file that includes the dataset with the wrong shape will not be compliant with HDMF schema 2.2.1 or later releases.

Backward compatibility: An NWB or HDMF file compliant with NWB or HDMF schema x.y is not necessarily compliant with earlier releases of the NWB or HDMF schema.

For example, a field that is required in HDMF schema 2.0 may become optional in HDMF schema 2.1. So, a file compliant with HDMF schema 2.1 may be missing that field, which would make it not compliant with HDMF schema 2.0.

Validation: The NWB and HDMF validator is continually being improved and patched. Files that succeed validation against a particular schema in an older version of the API may not succeed validation in a newer version of the API against the same schema.