Behavior Data

This tutorial will demonstrate the usage of the pynwb.behavior module for adding behavioral data to an NWBFile.

See also

You can learn more about the NWBFile format in the NWB File Basics tutorial.

The examples below follow this general workflow for adding behavior data to an NWBFile:

The following examples will reference variables that may not be defined within the block they are used in. For clarity, we define them here:

from datetime import datetime

import numpy as np
from dateutil import tz
from pynwb.misc import IntervalSeries

from pynwb.epoch import TimeIntervals

from pynwb import NWBFile, TimeSeries, NWBHDF5IO
from pynwb.behavior import (
    SpatialSeries,
    BehavioralTimeSeries,
    Position,
    BehavioralEvents,
    CompassDirection,
    BehavioralEpochs,
    PupilTracking,
    EyeTracking,
)

Create an NWB File

Create an NWBFile object with the required fields (session_description, identifier, session_start_time) and additional metadata.

session_start_time = datetime(2018, 4, 25, 2, 30, 3, tzinfo=tz.gettz("US/Pacific"))

nwbfile = NWBFile(
    session_description="Mouse exploring an open field",  # required
    identifier="Mouse5_Day3",  # required
    session_start_time=session_start_time,  # required
    session_id="session_1234",  # optional
    experimenter="My Name",  # optional
    lab="My Lab Name",  # optional
    institution="University of My Institution",  # optional
    related_publications="DOI:10.1016/j.neuron.2016.12.011",  # optional
)

nwbfile
root pynwb.file.NWBFile at 0x139763063920960
Fields:
  experimenter: ['My Name']
  file_create_date: [datetime.datetime(2022, 10, 20, 0, 56, 28, 774022, tzinfo=tzlocal())]
  identifier: Mouse5_Day3
  institution: University of My Institution
  lab: My Lab Name
  related_publications: ['DOI:10.1016/j.neuron.2016.12.011']
  session_description: Mouse exploring an open field
  session_id: session_1234
  session_start_time: 2018-04-25 02:30:03-07:00
  timestamps_reference_time: 2018-04-25 02:30:03-07:00

SpatialSeries: Storing continuous spatial data

SpatialSeries is a subclass of TimeSeries that represents data in space, such as the spatial direction, e.g., of gaze or travel, or position of an animal over time.

Create data that corresponds to x, y position over time.

(50, 2)

In SpatialSeries data, the first dimension is always time (in seconds), the second dimension represents the x, y position (in meters).

Note

SpatialSeries data should be stored as one continuous stream, as it is acquired, not by trial as is often reshaped for analysis. Data can be trial-aligned on-the-fly using the trials table. See the Trials tutorial for further information.

For position data reference_frame indicates the zero-position, e.g. the 0,0 point might be the bottom-left corner of an enclosure, as viewed from the tracking camera.

timestamps = np.linspace(0, 50) / 200

position_spatial_series = SpatialSeries(
    name="SpatialSeries",
    description="Position (x, y) in an open field.",
    data=position_data,
    timestamps=timestamps,
    reference_frame="(0,0) is bottom left corner",
)

position_spatial_series
SpatialSeries pynwb.behavior.SpatialSeries at 0x139763066553008
Fields:
  comments: no comments
  conversion: 1.0
  data: [[ 0.          0.        ]
 [ 0.20408163  0.16326531]
 [ 0.40816327  0.32653061]
 [ 0.6122449   0.48979592]
 [ 0.81632653  0.65306122]
 [ 1.02040816  0.81632653]
 [ 1.2244898   0.97959184]
 [ 1.42857143  1.14285714]
 [ 1.63265306  1.30612245]
 [ 1.83673469  1.46938776]
 [ 2.04081633  1.63265306]
 [ 2.24489796  1.79591837]
 [ 2.44897959  1.95918367]
 [ 2.65306122  2.12244898]
 [ 2.85714286  2.28571429]
 [ 3.06122449  2.44897959]
 [ 3.26530612  2.6122449 ]
 [ 3.46938776  2.7755102 ]
 [ 3.67346939  2.93877551]
 [ 3.87755102  3.10204082]
 [ 4.08163265  3.26530612]
 [ 4.28571429  3.42857143]
 [ 4.48979592  3.59183673]
 [ 4.69387755  3.75510204]
 [ 4.89795918  3.91836735]
 [ 5.10204082  4.08163265]
 [ 5.30612245  4.24489796]
 [ 5.51020408  4.40816327]
 [ 5.71428571  4.57142857]
 [ 5.91836735  4.73469388]
 [ 6.12244898  4.89795918]
 [ 6.32653061  5.06122449]
 [ 6.53061224  5.2244898 ]
 [ 6.73469388  5.3877551 ]
 [ 6.93877551  5.55102041]
 [ 7.14285714  5.71428571]
 [ 7.34693878  5.87755102]
 [ 7.55102041  6.04081633]
 [ 7.75510204  6.20408163]
 [ 7.95918367  6.36734694]
 [ 8.16326531  6.53061224]
 [ 8.36734694  6.69387755]
 [ 8.57142857  6.85714286]
 [ 8.7755102   7.02040816]
 [ 8.97959184  7.18367347]
 [ 9.18367347  7.34693878]
 [ 9.3877551   7.51020408]
 [ 9.59183673  7.67346939]
 [ 9.79591837  7.83673469]
 [10.          8.        ]]
  description: Position (x, y) in an open field.
  interval: 1
  offset: 0.0
  reference_frame: (0,0) is bottom left corner
  resolution: -1.0
  timestamps: [0.         0.00510204 0.01020408 0.01530612 0.02040816 0.0255102
 0.03061224 0.03571429 0.04081633 0.04591837 0.05102041 0.05612245
 0.06122449 0.06632653 0.07142857 0.07653061 0.08163265 0.08673469
 0.09183673 0.09693878 0.10204082 0.10714286 0.1122449  0.11734694
 0.12244898 0.12755102 0.13265306 0.1377551  0.14285714 0.14795918
 0.15306122 0.15816327 0.16326531 0.16836735 0.17346939 0.17857143
 0.18367347 0.18877551 0.19387755 0.19897959 0.20408163 0.20918367
 0.21428571 0.21938776 0.2244898  0.22959184 0.23469388 0.23979592
 0.24489796 0.25      ]
  timestamps_unit: seconds
  unit: meters

Position: Storing position measured over time

To help data analysis and visualization tools know that this SpatialSeries object represents the position of the subject, store the SpatialSeries object inside a Position object, which can hold one or more SpatialSeries objects.

position = Position(spatial_series=position_spatial_series)

See also

You can learn more about the SpatialSeries data type and Position interface in the Spatial Series and Position tutorial.

See also

You can learn more about best practices that can be applied to SpatialSeries at NWB Best Practices.

Create a Behavior Processing Module

Create a processing module called "behavior" for storing behavioral data in the NWBFile using the create_processing_module method, and then add the Position object to the processing module.

behavior_module = nwbfile.create_processing_module(
    name="behavior", description="Processed behavioral data"
)

behavior_module.add(position)
Position pynwb.behavior.Position at 0x139763066552384
Fields:
  spatial_series: {
    SpatialSeries <class 'pynwb.behavior.SpatialSeries'>
  }

See also

You can read more about the basic concepts of processing modules in the Processing Modules tutorial.

CompassDirection: Storing view angle measured over time

Analogous to how position can be stored, we can create a SpatialSeries object for representing the view angle of the subject.

For direction data reference_frame indicates the zero-axes, for instance “straight-ahead” might be a specific pixel on the monitor, or some other point in space. The unit of measurement for the SpatialSeries object should be radians or degrees.

view_angle_data = np.linspace(0, 4, 50)

direction_spatial_series = SpatialSeries(
    name="SpatialSeries",
    description="View angle of the subject measured in radians.",
    data=view_angle_data,
    timestamps=timestamps,
    reference_frame="straight ahead",
    unit="radians",
)

direction = CompassDirection(
    spatial_series=direction_spatial_series, name="CompassDirection"
)

We can add a CompassDirection object to the behavior processing module the same way as we have added the position data:

behavior_module.add(direction)
CompassDirection pynwb.behavior.CompassDirection at 0x139763168512320
Fields:
  spatial_series: {
    SpatialSeries <class 'pynwb.behavior.SpatialSeries'>
  }

BehavioralTimeSeries: Storing continuous behavior data

BehavioralTimeSeries is an interface for storing continuous behavior data, such as the speed of a subject.

speed_data = np.linspace(0, 0.4, 50)

speed_time_series = TimeSeries(
    name="speed",
    data=speed_data,
    timestamps=timestamps,
    description="The speed of the subject measured over time.",
    unit="m/s",
)

behavioral_time_series = BehavioralTimeSeries(
    time_series=speed_time_series,
    name="BehavioralTimeSeries",
)

behavior_module.add(behavioral_time_series)
BehavioralTimeSeries pynwb.behavior.BehavioralTimeSeries at 0x139763006379536
Fields:
  time_series: {
    speed <class 'pynwb.base.TimeSeries'>
  }

BehavioralEvents: Storing behavioral events

BehavioralEvents is an interface for storing behavioral events. We can use it for storing the timing and amount of rewards (e.g. water amount) or lever press times.

reward_amount = [1., 1.5, 1., 1.5]
events_timestamps = [1., 2., 5., 6.]

time_series = TimeSeries(
    name="lever_presses",
    data=reward_amount,
    timestamps=events_timestamps,
    description="The water amount the subject received as a reward.",
    unit="ml",
)

behavioral_events = BehavioralEvents(time_series=time_series, name="BehavioralEvents")

behavior_module.add(behavioral_events)
BehavioralEvents pynwb.behavior.BehavioralEvents at 0x139763006380640
Fields:
  time_series: {
    lever_presses <class 'pynwb.base.TimeSeries'>
  }

Storing only the timestamps of the events is possible with the ndx-events NWB extension. You can also add labels associated with the events with this extension. You can find information about installation and example usage here.

See also

You can learn more about using extensions in the Extending NWB tutorial.

BehavioralEpochs: Storing intervals of behavior data

BehavioralEpochs is for storing intervals of behavior data. BehavioralEpochs uses IntervalSeries to represent behavioral epochs.

Create IntervalSeries object that represents the time intervals when the animal was running. IntervalSeries uses 1 to indicate the beginning of an interval and -1 to indicate the end.

run_intervals = IntervalSeries(
    name="running",
    description="Intervals when the animal was running.",
    data=[1, -1, 1, -1, 1, -1],
    timestamps=[0.5, 1.5, 3.5, 4.0, 7.0, 7.3],
)

behavioral_epochs = BehavioralEpochs(name="BehavioralEpochs")

behavioral_epochs.add_interval_series(run_intervals)
running pynwb.misc.IntervalSeries at 0x139763006380544
Fields:
  comments: no comments
  conversion: 1.0
  data: [ 1 -1  1 -1  1 -1]
  description: Intervals when the animal was running.
  interval: 1
  offset: 0.0
  resolution: -1.0
  timestamps: [0.5 1.5 3.5 4.  7.  7.3]
  timestamps_unit: seconds
  unit: n/a

you can add more than one IntervalSeries to a BehavioralEpochs

sleep_intervals = IntervalSeries(
    name="sleeping",
    description="Intervals when the animal was sleeping.",
    data=[1, -1, 1, -1],
    timestamps=[15.0, 30.0, 60.0, 95.0],
)
behavioral_epochs.add_interval_series(sleep_intervals)

behavior_module.add(behavioral_epochs)
BehavioralEpochs pynwb.behavior.BehavioralEpochs at 0x139763006381696
Fields:
  interval_series: {
    running <class 'pynwb.misc.IntervalSeries'>,
    sleeping <class 'pynwb.misc.IntervalSeries'>
  }

Using TimeIntervals representing time intervals is often preferred over BehavioralEpochs and IntervalSeries. TimeIntervals is a subclass of DynamicTable which offers flexibility for tabular data by allowing the addition of optional columns which are not defined in the standard.

Create a TimeIntervals object that represents the time intervals when the animal was sleeping.

sleep_intervals = TimeIntervals(
    name="sleep_intervals",
    description="Intervals when the animal was sleeping.",
)

sleep_intervals.add_column(name="stage", description="The stage of sleep.")

sleep_intervals.add_row(start_time=0.3, stop_time=0.35, stage=1)
sleep_intervals.add_row(start_time=0.7, stop_time=0.9, stage=2)
sleep_intervals.add_row(start_time=1.3, stop_time=3.0, stage=3)

nwbfile.add_time_intervals(sleep_intervals)
sleep_intervals pynwb.epoch.TimeIntervals at 0x139763007145056
Fields:
  colnames: ['start_time' 'stop_time' 'stage']
  columns: (
    start_time <class 'hdmf.common.table.VectorData'>,
    stop_time <class 'hdmf.common.table.VectorData'>,
    stage <class 'hdmf.common.table.VectorData'>
  )
  description: Intervals when the animal was sleeping.
  id: id <class 'hdmf.common.table.ElementIdentifiers'>

PupilTracking: Storing continuous eye-tracking data of pupil size

PupilTracking is for storing eye-tracking data which represents pupil size. PupilTracking holds one or more TimeSeries objects that can represent different features such as the dilation of the pupil measured over time by a pupil tracking algorithm.

pupil_diameter = TimeSeries(
    name="pupil_diameter",
    description="Pupil diameter extracted from the video of the right eye.",
    data=np.linspace(0.001, 0.002, 50),
    timestamps=timestamps,
    unit="meters",
)

pupil_tracking = PupilTracking(time_series=pupil_diameter, name="PupilTracking")

behavior_module.add(pupil_tracking)
PupilTracking pynwb.behavior.PupilTracking at 0x139763007145296
Fields:
  time_series: {
    pupil_diameter <class 'pynwb.base.TimeSeries'>
  }

EyeTracking: Storing continuous eye-tracking data of gaze direction

EyeTracking is for storing eye-tracking data which represents direction of gaze as measured by an eye tracking algorithm. An EyeTracking object holds one or more SpatialSeries objects that represents the vertical and horizontal gaze positions extracted from a video.

right_eye_position = np.linspace(-20, 30, 50)

right_eye_positions = SpatialSeries(
    name="right_eye_position",
    description="The position of the right eye measured in degrees.",
    data=right_eye_position,
    timestamps=timestamps,
    reference_frame="bottom left",
    unit="degrees",
)

eye_tracking = EyeTracking(name="EyeTracking", spatial_series=right_eye_positions)

We can add another SpatialSeries representing the position of the left eye in degrees.

left_eye_position = np.linspace(-2, 20, 50)

left_eye_positions = SpatialSeries(
    name="left_eye_position",
    description="The position of the left eye measured in degrees.",
    data=left_eye_position,
    timestamps=timestamps,
    reference_frame="bottom left",
    unit="degrees",
)

eye_tracking.add_spatial_series(spatial_series=left_eye_positions)

behavior_module.add(eye_tracking)
EyeTracking pynwb.behavior.EyeTracking at 0x139763013662224
Fields:
  spatial_series: {
    left_eye_position <class 'pynwb.behavior.SpatialSeries'>,
    right_eye_position <class 'pynwb.behavior.SpatialSeries'>
  }

Writing the behavior data to an NWB file

As demonstrated in the Writing an NWB file tutorial, we will use NWBHDF5IO to write the file.

with NWBHDF5IO("behavioral_tutorial.nwb", "w") as io:
    io.write(nwbfile)

Reading and accessing the behavior data

To read the NWB file we just wrote, use another NWBHDF5IO object, and use the read method to retrieve an NWBFile object.

We can access the behavior processing module by indexing nwbfile.processing with the name of the processing module "behavior". We can also inspect the objects hierarchy within this processing module with the .children attribute.

with NWBHDF5IO("behavioral_tutorial.nwb", "r") as io:
    read_nwbfile = io.read()
    print(read_nwbfile.processing["behavior"].children)
(BehavioralEpochs pynwb.behavior.BehavioralEpochs at 0x139763016895264
Fields:
  interval_series: {
    running <class 'pynwb.misc.IntervalSeries'>,
    sleeping <class 'pynwb.misc.IntervalSeries'>
  }
, BehavioralEvents pynwb.behavior.BehavioralEvents at 0x139763016896224
Fields:
  time_series: {
    lever_presses <class 'pynwb.base.TimeSeries'>
  }
, BehavioralTimeSeries pynwb.behavior.BehavioralTimeSeries at 0x139763016828528
Fields:
  time_series: {
    speed <class 'pynwb.base.TimeSeries'>
  }
, CompassDirection pynwb.behavior.CompassDirection at 0x139763006378672
Fields:
  spatial_series: {
    SpatialSeries <class 'pynwb.behavior.SpatialSeries'>
  }
, EyeTracking pynwb.behavior.EyeTracking at 0x139763063380144
Fields:
  spatial_series: {
    left_eye_position <class 'pynwb.behavior.SpatialSeries'>,
    right_eye_position <class 'pynwb.behavior.SpatialSeries'>
  }
, Position pynwb.behavior.Position at 0x139763007257520
Fields:
  spatial_series: {
    SpatialSeries <class 'pynwb.behavior.SpatialSeries'>
  }
, PupilTracking pynwb.behavior.PupilTracking at 0x139763016341296
Fields:
  time_series: {
    pupil_diameter <class 'pynwb.base.TimeSeries'>
  }
)

For instance, we can access the SpatialSeries data by referencing the names of the objects in the hierarchy that contain it. We can access the Position object inside the behavior processing module by indexing it with the name of the Position object, "Position". Then, we can access the SpatialSeries object inside the Position object by indexing it with the name of the SpatialSeries object, "SpatialSeries".

with NWBHDF5IO("behavioral_tutorial.nwb", "r") as io:
    read_nwbfile = io.read()
    print(read_nwbfile.processing["behavior"]["Position"]["SpatialSeries"])
SpatialSeries pynwb.behavior.SpatialSeries at 0x139763015710464
Fields:
  comments: no comments
  conversion: 1.0
  data: <HDF5 dataset "data": shape (50, 2), type "<f8">
  description: Position (x, y) in an open field.
  interval: 1
  offset: 0.0
  reference_frame: (0,0) is bottom left corner
  resolution: -1.0
  timestamps: <HDF5 dataset "timestamps": shape (50,), type "<f8">
  timestamps_unit: seconds
  unit: meters

Data arrays are read passively from the file. Accessing the data attribute of the SpatialSeries object does not read the data values, but presents an HDF5 object that can be indexed to read data. You can use the [:] operator to read the entire data array into memory.

with NWBHDF5IO("behavioral_tutorial.nwb", "r") as io:
    read_nwbfile = io.read()
    print(read_nwbfile.processing["behavior"]["Position"]["SpatialSeries"].data[:])
[[ 0.          0.        ]
 [ 0.20408163  0.16326531]
 [ 0.40816327  0.32653061]
 [ 0.6122449   0.48979592]
 [ 0.81632653  0.65306122]
 [ 1.02040816  0.81632653]
 [ 1.2244898   0.97959184]
 [ 1.42857143  1.14285714]
 [ 1.63265306  1.30612245]
 [ 1.83673469  1.46938776]
 [ 2.04081633  1.63265306]
 [ 2.24489796  1.79591837]
 [ 2.44897959  1.95918367]
 [ 2.65306122  2.12244898]
 [ 2.85714286  2.28571429]
 [ 3.06122449  2.44897959]
 [ 3.26530612  2.6122449 ]
 [ 3.46938776  2.7755102 ]
 [ 3.67346939  2.93877551]
 [ 3.87755102  3.10204082]
 [ 4.08163265  3.26530612]
 [ 4.28571429  3.42857143]
 [ 4.48979592  3.59183673]
 [ 4.69387755  3.75510204]
 [ 4.89795918  3.91836735]
 [ 5.10204082  4.08163265]
 [ 5.30612245  4.24489796]
 [ 5.51020408  4.40816327]
 [ 5.71428571  4.57142857]
 [ 5.91836735  4.73469388]
 [ 6.12244898  4.89795918]
 [ 6.32653061  5.06122449]
 [ 6.53061224  5.2244898 ]
 [ 6.73469388  5.3877551 ]
 [ 6.93877551  5.55102041]
 [ 7.14285714  5.71428571]
 [ 7.34693878  5.87755102]
 [ 7.55102041  6.04081633]
 [ 7.75510204  6.20408163]
 [ 7.95918367  6.36734694]
 [ 8.16326531  6.53061224]
 [ 8.36734694  6.69387755]
 [ 8.57142857  6.85714286]
 [ 8.7755102   7.02040816]
 [ 8.97959184  7.18367347]
 [ 9.18367347  7.34693878]
 [ 9.3877551   7.51020408]
 [ 9.59183673  7.67346939]
 [ 9.79591837  7.83673469]
 [10.          8.        ]]

Alternatively, you can read only a portion of the data by indexing or slicing into the data attribute just like if you were indexing or slicing a numpy array.

with NWBHDF5IO("behavioral_tutorial.nwb", "r") as io:
    read_nwbfile = io.read()
    print(read_nwbfile.processing["behavior"]["Position"]["SpatialSeries"].data[:2])
[[0.         0.        ]
 [0.20408163 0.16326531]]

Gallery generated by Sphinx-Gallery