Annotating Time Intervals

Annotating events in time is a common need in neuroscience, e.g. to describes epochs, trials, and invalid times during an experimental session. NWB supports annotation of time intervals via the TimeIntervals type. The TimeIntervals type is a DynamicTable with the following columns:

1. start_time and stop_time describe the start and stop times of intervals as floating point offsets in seconds relative to the timestamps_reference_time of the file. In addition,

2. tags is an optional, indexed column used to associate user-defined string tags with intervals (0 or more tags per time interval)

3. timeseries is an optional, indexed TimeSeriesReferenceVectorData column to map intervals directly to ranges in select, relevant TimeSeries (0 or more per time interval)

4. as a DynamicTable user may add additional columns to TimeIntervals via add_column

Hint

TimeIntervals is intended for storing general annotations of time ranges. Depending on the application (e.g., when intervals are generated by data acquisition or automatic data processing), it can be useful to describe intervals (or instantaneous events) in time as TimeSeries. NWB provides several types for this purposes, e.g., IntervalSeries, BehavioralEpochs, BehavioralEvents, EventDetection, or SpikeEventSeries.

Setup: Creating an example NWB file for the tutorial

from datetime import datetime
from uuid import uuid4

import numpy as np
from dateutil.tz import tzlocal

from pynwb import NWBFile, TimeSeries

# create the NWBFile
nwbfile = NWBFile(
    session_description="my first synthetic recording",  # required
    identifier=str(uuid4()),  # required
    session_start_time=datetime(2017, 4, 3, 11, tzinfo=tzlocal()),  # required
    experimenter="Baggins, Bilbo",  # optional
    lab="Bag End Laboratory",  # optional
    institution="University of Middle Earth at the Shire",  # optional
    experiment_description="I went on an adventure with thirteen dwarves to reclaim vast treasures.",  # optional
    session_id="LONELYMTN",  # optional
)
# create some example TimeSeries
test_ts = TimeSeries(
    name="series1",
    data=np.arange(1000),
    unit="m",
    timestamps=np.linspace(0.5, 601, 1000),
)
rate_ts = TimeSeries(
    name="series2", data=np.arange(600), unit="V", starting_time=0.0, rate=1.0
)
# Add the TimeSeries to the file
nwbfile.add_acquisition(test_ts)
nwbfile.add_acquisition(rate_ts)

Adding Time Intervals to a NWBFile

NWB provides a set of pre-defined TimeIntervals tables for epochs, trials, and invalid_times.

Trials

Trials can be added to an NWB file using the methods add_trial By default, NWBFile only requires trial start_time and stop_time. The tags and timeseries are optional. For timeseries we only need to supply the TimeSeries. PyNWB automatically calculates the corresponding index range (described by idx_start and count) for the supplied TimeSeries based on the given start_time and stop_time and the timestamps (or starting_time and rate) of the given TimeSeries.

Additional columns can be added using add_trial_column. This method takes a name for the column and a description of what the column stores. You do not need to supply data type, as this will inferred. Once all columns have been added, trial data can be populated using add_trial. Note that if you add a custom column, you must add at least one row to write the table to a file.

Lets add an additional column and some trial data with tags and timeseries references.

nwbfile.add_trial_column(name="stim", description="the visual stimuli during the trial")

nwbfile.add_trial(
    start_time=0.0,
    stop_time=2.0,
    stim="dog",
    tags=["animal"],
    timeseries=[test_ts, rate_ts],
)
nwbfile.add_trial(
    start_time=3.0,
    stop_time=5.0,
    stim="mountain",
    tags=["landscape"],
    timeseries=[test_ts, rate_ts],
)
nwbfile.add_trial(
    start_time=6.0,
    stop_time=8.0,
    stim="desert",
    tags=["landscape"],
    timeseries=[test_ts, rate_ts],
)
nwbfile.add_trial(
    start_time=9.0,
    stop_time=11.0,
    stim="tree",
    tags=["landscape", "plant"],
    timeseries=[test_ts, rate_ts],
)
nwbfile.add_trial(
    start_time=12.0,
    stop_time=14.0,
    stim="bird",
    tags=["animal"],
    timeseries=[test_ts, rate_ts],
)
nwbfile.add_trial(
    start_time=15.0,
    stop_time=17.0,
    stim="flower",
    tags=["animal"],
    timeseries=[test_ts, rate_ts],
)

Epochs

Similarly, epochs can be added to an NWB file using the method add_epoch and add_epoch_column.

nwbfile.add_epoch(
    2.0,
    4.0,
    ["first", "example"],
    [
        test_ts,
    ],
)
nwbfile.add_epoch(
    6.0,
    8.0,
    ["second", "example"],
    [
        test_ts,
    ],
)

Invalid Times

Similarly, invalid times can be added using the method add_invalid_time_interval and add_invalid_times_column.

nwbfile.add_epoch(
    2.0,
    4.0,
    ["first", "example"],
    [
        test_ts,
    ],
)
nwbfile.add_epoch(
    6.0,
    8.0,
    ["second", "example"],
    [
        test_ts,
    ],
)

Custom Time Intervals

To define custom, experiment-specific TimeIntervals we can add them either: 1) when creating the NWBFile by defining the intervals constructor argument or 2) via the add_time_intervals or create_time_intervals after the NWBFile has been created.

from pynwb.epoch import TimeIntervals

sleep_stages = TimeIntervals(
    name="sleep_stages",
    description="intervals for each sleep stage as determined by EEG",
)

sleep_stages.add_column(name="stage", description="stage of sleep")
sleep_stages.add_column(name="confidence", description="confidence in stage (0-1)")

sleep_stages.add_row(start_time=0.3, stop_time=0.5, stage=1, confidence=0.5)
sleep_stages.add_row(start_time=0.7, stop_time=0.9, stage=2, confidence=0.99)
sleep_stages.add_row(start_time=1.3, stop_time=3.0, stage=3, confidence=0.7)

_ = nwbfile.add_time_intervals(sleep_stages)

Accessing Time Intervals

We can access the predefined TimeIntervals tables via the corresponding epochs, trials, and invalid_times properties and for custom TimeIntervals via the get_time_intervals method. E.g.:

_ = nwbfile.intervals
_ = nwbfile.get_time_intervals("sleep_stages")

Like any DynamicTable, we can conveniently convert any TimeIntervals table to a pandas.DataFrame via to_dataframe, such as:

nwbfile.trials.to_dataframe()

	start_time	stop_time	stim	tags	timeseries
id
0	0.0	2.0	dog	[animal]	[(0, 3, series1 pynwb.base.TimeSeries at 0x140...
1	3.0	5.0	mountain	[landscape]	[(5, 3, series1 pynwb.base.TimeSeries at 0x140...
2	6.0	8.0	desert	[landscape]	[(10, 3, series1 pynwb.base.TimeSeries at 0x14...
3	9.0	11.0	tree	[landscape, plant]	[(15, 3, series1 pynwb.base.TimeSeries at 0x14...
4	12.0	14.0	bird	[animal]	[(20, 3, series1 pynwb.base.TimeSeries at 0x14...
5	15.0	17.0	flower	[animal]	[(25, 3, series1 pynwb.base.TimeSeries at 0x14...

This approach makes it easy to query the data to, e.g., locate all time intervals within a certain time range

trials_df = nwbfile.trials.to_dataframe()
trials_df.query("(start_time > 2.0) & (stop_time < 9.0)")

	start_time	stop_time	stim	tags	timeseries
id
1	3.0	5.0	mountain	[landscape]	[(5, 3, series1 pynwb.base.TimeSeries at 0x140...
2	6.0	8.0	desert	[landscape]	[(10, 3, series1 pynwb.base.TimeSeries at 0x14...

Accessing referenced TimeSeries

As mentioned earlier, the timeseries column is defined by a TimeSeriesReferenceVectorData which stores references to the corresponding ranges in TimeSeries. Individual references to TimeSeries are described via TimeSeriesReference tuples with the idx_start, count, and timeseries. Using TimeSeriesReference we can easily access the relevant data and timestamps for the corresponding time range from the TimeSeries.

# Get a single example TimeSeriesReference from the trials table
example_tsr = nwbfile.trials["timeseries"][0][0]

# Get the data values from the timeseries. This is a shorthand for:
# _ = example_tsr.timeseries.data[example_tsr.idx_start: (example_tsr.idx_start + example_tsr.count)]
_ = example_tsr.data

# Get the timestamps. Timestamps are either loaded from the TimeSeries or
# computed from the starting_time and rate
example_tsr.timestamps

array([0.5      , 1.1011011, 1.7022022])

Using isvalid we can further check if the reference is valid. A TimeSeriesReference is defined as invalid if both idx_start, count are set to -1. isvalid further also checks that the indicated index range and types are valid, raising IndexError and TypeError respectively, if bad idx_start, count or timeseries are found.

example_tsr.isvalid()

True

Adding TimeSeries references to other tables

Since TimeSeriesReferenceVectorData is a regular VectorData type, we can use it to add references to intervals in TimeSeries to any DynamicTable. In the IntracellularRecordingsTable, e.g., it is used to reference the recording of the stimulus and response associated with a particular intracellular electrophysiology recording.

Reading/Writing TimeIntervals to file

Reading and writing the data is as usual:

from pynwb import NWBHDF5IO

# write the file
with NWBHDF5IO("example_timeintervals_file.nwb", "w") as io:
    io.write(nwbfile)
# read the file
with NWBHDF5IO("example_timeintervals_file.nwb", "r") as io:
    nwbfile_in = io.read()

    # plot the sleep stages TimeIntervals table
    nwbfile_in.get_time_intervals("sleep_stages").to_dataframe()