Note
Go to the end to download the full example code
Behavior Data¶
This tutorial will demonstrate the usage of the pynwb.behavior module for adding
behavioral data to an NWBFile.
See also
You can learn more about the NWBFile format in the NWB File Basics tutorial.
The examples below follow this general workflow for adding behavior data to an NWBFile:
create an object:
TimeSeriesfor continuous time series data,SpatialSeriesfor continuous spatial data (e.g. position, direction relative to some reference frame),IntervalSeriesorTimeIntervalsfor time intervals
store that object inside a behavior interface object:
Positionfor position measured over timeCompassDirectionfor view angle measured over timeBehavioralTimeSeriesfor continuous time series dataBehavioralEventsfor behavioral events (e.g. reward amount)BehavioralEpochsfor behavioral intervals (e.g. sleep intervals)PupilTrackingfor eye-tracking data of pupil sizeEyeTrackingfor eye-tracking data of gaze direction
create a behavior processing module for the
NWBFileand add the interface object(s) to it
The following examples will reference variables that may not be defined within the block they are used in. For clarity, we define them here:
from datetime import datetime
from uuid import uuid4
import numpy as np
from dateutil.tz import tzlocal
from pynwb import NWBHDF5IO, NWBFile, TimeSeries
from pynwb.behavior import (
BehavioralEpochs,
BehavioralEvents,
BehavioralTimeSeries,
CompassDirection,
EyeTracking,
Position,
PupilTracking,
SpatialSeries,
)
from pynwb.epoch import TimeIntervals
from pynwb.misc import IntervalSeries
Create an NWB File¶
Create an NWBFile object with the required fields
(session_description, identifier, session_start_time) and additional metadata.
nwbfile = NWBFile(
session_description="my first synthetic recording",
identifier=str(uuid4()),
session_start_time=datetime.now(tzlocal()),
experimenter=[
"Baggins, Bilbo",
],
lab="Bag End Laboratory",
institution="University of Middle Earth at the Shire",
experiment_description="I went on an adventure to reclaim vast treasures.",
session_id="LONELYMTN001",
)
nwbfile
SpatialSeries: Storing continuous spatial data¶
SpatialSeries is a subclass of TimeSeries
that represents data in space, such as the spatial direction, e.g., of gaze or travel,
or position of an animal over time.
Create data that corresponds to x, y position over time.
position_data = np.array([np.linspace(0, 10, 50), np.linspace(0, 8, 50)]).T
position_data.shape
(50, 2)
In SpatialSeries data, the first dimension is always time (in seconds),
the second dimension represents the x, y position (in meters).
Note
SpatialSeries data should be stored as one continuous stream,
as it is acquired, not by trial as is often reshaped for analysis.
Data can be trial-aligned on-the-fly using the trials table. See the Annotating Time Intervals tutorial
for further information.
For position data reference_frame indicates the zero-position, e.g.
the 0,0 point might be the bottom-left corner of an enclosure, as viewed from the tracking camera.
timestamps = np.linspace(0, 50) / 200
position_spatial_series = SpatialSeries(
name="SpatialSeries",
description="Position (x, y) in an open field.",
data=position_data,
timestamps=timestamps,
reference_frame="(0,0) is bottom left corner",
)
position_spatial_series
Position: Storing position measured over time¶
To help data analysis and visualization tools know that this SpatialSeries object
represents the position of the subject, store the SpatialSeries object inside
a Position object, which can hold one or more SpatialSeries
objects.
position = Position(spatial_series=position_spatial_series)
See also
You can learn more about the SpatialSeries data type and
Position interface in the Spatial Series and Position tutorial.
See also
You can learn more about best practices that can be applied to
SpatialSeries at NWB Best Practices.
Create a Behavior Processing Module¶
Create a processing module called "behavior" for storing behavioral data in the NWBFile
using the create_processing_module method, and then add the
Position object to the processing module.
behavior_module = nwbfile.create_processing_module(
name="behavior", description="Processed behavioral data"
)
behavior_module.add(position)
See also
You can read more about the basic concepts of processing modules in the Processing Modules tutorial.
CompassDirection: Storing view angle measured over time¶
Analogous to how position can be stored, we can create a SpatialSeries
object for representing the view angle of the subject.
For direction data reference_frame indicates the zero-axes, for instance
“straight-ahead” might be a specific pixel on the monitor, or some other point in space.
The unit of measurement for the SpatialSeries object should be radians or degrees.
view_angle_data = np.linspace(0, 4, 50)
direction_spatial_series = SpatialSeries(
name="SpatialSeries",
description="View angle of the subject measured in radians.",
data=view_angle_data,
timestamps=timestamps,
reference_frame="straight ahead",
unit="radians",
)
direction = CompassDirection(
spatial_series=direction_spatial_series, name="CompassDirection"
)
We can add a CompassDirection object to the behavior processing module
the same way as we have added the position data:
behavior_module.add(direction)
BehavioralTimeSeries: Storing continuous behavior data¶
BehavioralTimeSeries is an interface for storing continuous behavior data, such as the
speed of a subject.
speed_data = np.linspace(0, 0.4, 50)
speed_time_series = TimeSeries(
name="speed",
data=speed_data,
timestamps=timestamps,
description="The speed of the subject measured over time.",
unit="m/s",
)
behavioral_time_series = BehavioralTimeSeries(
time_series=speed_time_series,
name="BehavioralTimeSeries",
)
behavior_module.add(behavioral_time_series)
BehavioralEvents: Storing behavioral events¶
BehavioralEvents is an interface for storing behavioral events.
We can use it for storing the timing and amount of rewards (e.g. water amount) or lever press times.
reward_amount = [1.0, 1.5, 1.0, 1.5]
events_timestamps = [1.0, 2.0, 5.0, 6.0]
time_series = TimeSeries(
name="lever_presses",
data=reward_amount,
timestamps=events_timestamps,
description="The water amount the subject received as a reward.",
unit="ml",
)
behavioral_events = BehavioralEvents(time_series=time_series, name="BehavioralEvents")
behavior_module.add(behavioral_events)
Storing only the timestamps of the events is possible with the ndx-events NWB extension. You can also add labels associated with the events with this extension. You can find information about installation and example usage here.
See also
You can learn more about using extensions in the Extending NWB tutorial.
BehavioralEpochs: Storing intervals of behavior data¶
BehavioralEpochs is for storing intervals of behavior data.
BehavioralEpochs uses IntervalSeries
to represent behavioral epochs.
Create IntervalSeries object that represents the time intervals
when the animal was running. IntervalSeries uses 1 to indicate
the beginning of an interval and -1 to indicate the end.
run_intervals = IntervalSeries(
name="running",
description="Intervals when the animal was running.",
data=[1, -1, 1, -1, 1, -1],
timestamps=[0.5, 1.5, 3.5, 4.0, 7.0, 7.3],
)
behavioral_epochs = BehavioralEpochs(name="BehavioralEpochs")
behavioral_epochs.add_interval_series(run_intervals)
you can add more than one IntervalSeries to a
BehavioralEpochs
sleep_intervals = IntervalSeries(
name="sleeping",
description="Intervals when the animal was sleeping.",
data=[1, -1, 1, -1],
timestamps=[15.0, 30.0, 60.0, 95.0],
)
behavioral_epochs.add_interval_series(sleep_intervals)
behavior_module.add(behavioral_epochs)
Using TimeIntervals representing time intervals
is often preferred over BehavioralEpochs and IntervalSeries.
TimeIntervals is a subclass of DynamicTable which offers
flexibility for tabular data by allowing the addition of optional columns which are not defined in the standard.
Create a TimeIntervals object that represents the time
intervals when the animal was sleeping.
sleep_intervals = TimeIntervals(
name="sleep_intervals",
description="Intervals when the animal was sleeping.",
)
sleep_intervals.add_column(name="stage", description="The stage of sleep.")
sleep_intervals.add_row(start_time=0.3, stop_time=0.35, stage=1)
sleep_intervals.add_row(start_time=0.7, stop_time=0.9, stage=2)
sleep_intervals.add_row(start_time=1.3, stop_time=3.0, stage=3)
nwbfile.add_time_intervals(sleep_intervals)
PupilTracking: Storing continuous eye-tracking data of pupil size¶
PupilTracking is for storing eye-tracking data which
represents pupil size. PupilTracking holds one or more
TimeSeries objects that can represent different features
such as the dilation of the pupil measured over time by a pupil tracking algorithm.
pupil_diameter = TimeSeries(
name="pupil_diameter",
description="Pupil diameter extracted from the video of the right eye.",
data=np.linspace(0.001, 0.002, 50),
timestamps=timestamps,
unit="meters",
)
pupil_tracking = PupilTracking(time_series=pupil_diameter, name="PupilTracking")
behavior_module.add(pupil_tracking)
EyeTracking: Storing continuous eye-tracking data of gaze direction¶
EyeTracking is for storing eye-tracking data which
represents direction of gaze as measured by an eye tracking algorithm.
An EyeTracking object holds one or more
SpatialSeries objects that represents the vertical and
horizontal gaze positions extracted from a video.
right_eye_position = np.linspace(-20, 30, 50)
right_eye_positions = SpatialSeries(
name="right_eye_position",
description="The position of the right eye measured in degrees.",
data=right_eye_position,
timestamps=timestamps,
reference_frame="bottom left",
unit="degrees",
)
eye_tracking = EyeTracking(name="EyeTracking", spatial_series=right_eye_positions)
We can add another SpatialSeries representing the position
of the left eye in degrees.
left_eye_position = np.linspace(-2, 20, 50)
left_eye_positions = SpatialSeries(
name="left_eye_position",
description="The position of the left eye measured in degrees.",
data=left_eye_position,
timestamps=timestamps,
reference_frame="bottom left",
unit="degrees",
)
eye_tracking.add_spatial_series(spatial_series=left_eye_positions)
behavior_module.add(eye_tracking)
Writing the behavior data to an NWB file¶
As demonstrated in the Writing an NWB file tutorial, we will use NWBHDF5IO to write the file.
Reading and accessing the behavior data¶
To read the NWB file we just wrote, use another NWBHDF5IO object,
and use the read method to retrieve an
NWBFile object.
We can access the behavior processing module by indexing nwbfile.processing
with the name of the processing module "behavior". We can also inspect the objects
hierarchy within this processing module with the .children attribute.
(BehavioralEpochs pynwb.behavior.BehavioralEpochs at 0x140203625346064
Fields:
interval_series: {
running <class 'pynwb.misc.IntervalSeries'>,
sleeping <class 'pynwb.misc.IntervalSeries'>
}
, BehavioralEvents pynwb.behavior.BehavioralEvents at 0x140203624936912
Fields:
time_series: {
lever_presses <class 'pynwb.base.TimeSeries'>
}
, BehavioralTimeSeries pynwb.behavior.BehavioralTimeSeries at 0x140203624151312
Fields:
time_series: {
speed <class 'pynwb.base.TimeSeries'>
}
, CompassDirection pynwb.behavior.CompassDirection at 0x140203624464912
Fields:
spatial_series: {
SpatialSeries <class 'pynwb.behavior.SpatialSeries'>
}
, EyeTracking pynwb.behavior.EyeTracking at 0x140203625337296
Fields:
spatial_series: {
left_eye_position <class 'pynwb.behavior.SpatialSeries'>,
right_eye_position <class 'pynwb.behavior.SpatialSeries'>
}
, Position pynwb.behavior.Position at 0x140203625644432
Fields:
spatial_series: {
SpatialSeries <class 'pynwb.behavior.SpatialSeries'>
}
, PupilTracking pynwb.behavior.PupilTracking at 0x140203624883344
Fields:
time_series: {
pupil_diameter <class 'pynwb.base.TimeSeries'>
}
)
For instance, we can access the SpatialSeries data
by referencing the names of the objects in the hierarchy that contain it.
We can access the Position object inside the behavior
processing module by indexing it with the name of the Position object,
"Position". Then, we can access the SpatialSeries object inside the
Position object by indexing it with the name of the
SpatialSeries object, "SpatialSeries".
SpatialSeries pynwb.behavior.SpatialSeries at 0x140203625863184
Fields:
comments: no comments
conversion: 1.0
data: <HDF5 dataset "data": shape (50, 2), type "<f8">
description: Position (x, y) in an open field.
interval: 1
offset: 0.0
reference_frame: (0,0) is bottom left corner
resolution: -1.0
timestamps: <HDF5 dataset "timestamps": shape (50,), type "<f8">
timestamps_unit: seconds
unit: meters
Data arrays are read passively from the file.
Accessing the data attribute of the SpatialSeries object
does not read the data values, but presents an HDF5 object that can be indexed to read data.
You can use the [:] operator to read the entire data array into memory.
[[ 0. 0. ]
[ 0.20408163 0.16326531]
[ 0.40816327 0.32653061]
[ 0.6122449 0.48979592]
[ 0.81632653 0.65306122]
[ 1.02040816 0.81632653]
[ 1.2244898 0.97959184]
[ 1.42857143 1.14285714]
[ 1.63265306 1.30612245]
[ 1.83673469 1.46938776]
[ 2.04081633 1.63265306]
[ 2.24489796 1.79591837]
[ 2.44897959 1.95918367]
[ 2.65306122 2.12244898]
[ 2.85714286 2.28571429]
[ 3.06122449 2.44897959]
[ 3.26530612 2.6122449 ]
[ 3.46938776 2.7755102 ]
[ 3.67346939 2.93877551]
[ 3.87755102 3.10204082]
[ 4.08163265 3.26530612]
[ 4.28571429 3.42857143]
[ 4.48979592 3.59183673]
[ 4.69387755 3.75510204]
[ 4.89795918 3.91836735]
[ 5.10204082 4.08163265]
[ 5.30612245 4.24489796]
[ 5.51020408 4.40816327]
[ 5.71428571 4.57142857]
[ 5.91836735 4.73469388]
[ 6.12244898 4.89795918]
[ 6.32653061 5.06122449]
[ 6.53061224 5.2244898 ]
[ 6.73469388 5.3877551 ]
[ 6.93877551 5.55102041]
[ 7.14285714 5.71428571]
[ 7.34693878 5.87755102]
[ 7.55102041 6.04081633]
[ 7.75510204 6.20408163]
[ 7.95918367 6.36734694]
[ 8.16326531 6.53061224]
[ 8.36734694 6.69387755]
[ 8.57142857 6.85714286]
[ 8.7755102 7.02040816]
[ 8.97959184 7.18367347]
[ 9.18367347 7.34693878]
[ 9.3877551 7.51020408]
[ 9.59183673 7.67346939]
[ 9.79591837 7.83673469]
[10. 8. ]]
Alternatively, you can read only a portion of the data by indexing or slicing into
the data attribute just like if you were indexing or slicing a numpy array.
[[0. 0. ]
[0.20408163 0.16326531]]