The NWB-N format was designed to be easily extendable. Here we will demonstrate how to extend NWB using the PyNWB API.
Extensions should be defined separately from the code that uses the extensions. This design decision is based on the assumption that extension will be written once, and read or used multiple times. Here, we provide an example of how to create an extension for subsequent use. (For more information on the available tools for creating extensions, see Extending NWB).
The following block of code demonstrates how to create a new namespace, and then add a new neurodata_type to this namespace. Finally,
export to save the extensions to disk for downstream use.
from pynwb import NWBNamespaceBuilder, NWBGroupSpec, NWBAttributeSpec ns_path = "mylab.namespace.yaml" ext_source = "mylab.extensions.yaml" ns_builder = NWBNamespaceBuilder('Extension for use in my Lab', "mylab") ext = NWBGroupSpec('A custom ElectricalSeries for my lab', attributes=[NWBAttributeSpec('trode_id', 'int', 'the tetrode id')], neurodata_type_inc='ElectricalSeries', neurodata_type_def='TetrodeSeries') ns_builder.add_spec(ext_source, ext) ns_builder.export(ns_path)
Running this block will produce two YAML files.
The first file contains the specification of the namespace.
# mylab.namespace.yaml namespaces: - doc: Extension for use in my Lab name: mylab schema: - namespace: core - source: fake_extension.yaml
The second file contains the details on newly defined types.
# mylab.extensions.yaml groups: - attributes: - doc: the tetrode id dtype: int name: trode_id doc: A custom ElectricalSeries for my lab neurodata_type_def: TetrodeSeries neurodata_type_inc: ElectricalSeries
Detailed documentation of all components and neurodata_types that are part of the core schema of NWB:N are available in the schema docs at http://nwb-schema.readthedocs.io . Before creating a new type from scratch, please have a look at the schema docs to see if using or extending an existing type may solve your problem. Also, the schema docs are helpful when extending an existing type to better understand the design and structure of the neurodata_type you are using.
After an extension has been created, it can be used by downstream codes for reading and writing data.
There are two main mechanisms for reading and writing extension data with PyNWB.
The first involves defining new
NWBContainer classes that are then mapped to the neurodata types in the extension.
from pynwb import register_class, load_namespaces from pynwb.ecephys import ElectricalSeries ns_path = "mylab.namespace.yaml" load_namespaces(ns_path) @register_class('mylab', 'TetrodeSeries') class TetrodeSeries(ElectricalSeries): __nwbfields__ = ('tetrode_id',) def __init__(self, ...): ...
Although it is not used here, it is encouraged to use the
docval decorator for documenting constructors, methods, and functions.
If you do not want to write additional code to read your extensions, PyNWB is able to dynamically create an
NWBContainer subclass for use within the PyNWB API.
Dynamically created classes can be inspected using the built-in
help or the
from pynwb import get_class, load_namespaces ns_path = "mylab.namespace.yaml" load_namespaces(ns_path) TetrodeSeries = get_class('TetrodeSeries', 'mylab')
When defining your own
NWBContainer, the subclass name does not need to be the same as the extension type name. However,
it is encouraged to keep class and extension names the same for the purposes of readibility.
Converting data to NWB¶
The following are example Jupyter notebooks for converting custom lab data to NWB:
crcns-ret-1: Meister lab retina data¶
Example: This example shows:
- Use of
- Creation and use of custom namespace and extension to extend
ImageSeriesto add custom metadata attributes
- Create external link for
- Read of crcns-ret-1 dataset
- Convert of the data to NWB
- Comparison of H5Gate and PyNWB
- Use of
Data: Convert single-unit neural responses recorded from isolated retina from lab mice (Mus Musculus) using a 61-electrode array in response to various visual stimuli. Recordings were done by Yifeng Zhang in Markus Meister’s lab at Harvard University in 2008. Further description of the data are available here: https://crcns.org/data-sets/retina/ret-1/about-ret-1
Comparison to NWB 1.0.x`:
- Notebook: https://github.com/NeurodataWithoutBorders/pynwb/blob/dev/docs/notebooks/convert-crcns-ret-1-meisterlab-compare-nwb-1.0.6.ipynb
- Description: This notebook shows the convert of the same data using the orginal NWB 1.0.x API to allow for comparison of the NWB 1.0.x and NWB 2.x file.