File in/out: How to import data files from Spike2 (and Matlab) into Python

Spike2 is a type of signal acquisition software commonly used in physiological studies (eg. electromyography, kinematics, cardiovascular research, sleep, etc). It is possible to perform signal analysis in Spike2 by using the drop-down menus, or by programming scripts to automate analysis. However, the programming language for Spike2 is difficult to read, which discourages many scientists from learning to program scripts and automate analysis in Spike2. Also, Spike2 is proprietary software designed to acquire signals using CED (Cambridge Electronic Design) hardware. So even if scientists learned to automate analysis using Spike2 scripts, sharing these scripts with others is of little benefit if other users don’t own or use Spike2 and CED hardware.

For scientists who work in labs that use Spike2 and CED hardware, what kind of research practices could we implement to encourage scientific reproducibility but avoid being constrained to only one type of data acquisition system? For example, we could use Spike2 and CED to sample raw data, but import the raw Spike2 data into Python and analyse the signals for all subjects with a Python script. This automates the analysis such that it is consistent across all subjects, and importantly, the analysis is transparent and efficient.

Importing data from Spike2 files into Python is a little fiddly but managable when we understand how the data are formatted. Spike2 stores data in .smr files. To begin, use Spike2 to export data from the .smr file to a Matlab .mat file, then import the Matlab file into Python.

Data structures. When Spike2 data are exported to Matlab, the data are stored as a Matlab structure where each channel’s data values and times are nested within the channel. A Matlab structure is the equivalent of a Python dictionary, where data are stored as keys and values, and data values can be referenced by knowing their keys. See this post for a quick revision on dictionaries.

The following code demonstrates how the Matlab file is imported into Python. I use an example Spike file, exported as Matlab format, that contains data for a calibration routine. The data files are available for download here.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import numpy as np
import scipy.io as sio
import matplotlib.pyplot as plt
# Include Ipython magic command if working in a Jupyter notebook
%matplotlib inline

data = sio.loadmat('001.mat')
print('Type and size of structure:', type(data), np.size(data))
print(data.keys())

emgE = data['emgE']['values']
print('Type and size of structure:', type(emgE), np.size(emgE))

print('EMG values: \n', emgE)
print('EMG values index [0]: \n', emgE[0])
print('EMG values index [0][0]: \n', emgE[0][0])

emgE = emgE[0][0].flatten() # emgE = emgE[0,0].flatten()
print('Type and size of structure:', type(emgE), np.size(emgE))
print('Values of array:', emgE)

plt.plot(emgE)
plt.xlabel('Samples')
plt.ylabel('EMG (a.u.)')
plt.savefig('fig1.png')

Line 1-5. Import the usual numpy and matplotlib libraries. We also import a special input and output function from the Scipy library (scipy.io) and give it an alias (sio)

Line 7-12. Read in the datafile sub01.mat using the loadmat function.

Python imports data from the .mat file into a dictionary. We can confirm this by printing the type and size of the data structure, and printing the dictionary keys:

1
2
Type and size of structure: <class 'dict'> 1
dict_keys(['__globals__', 'emgE', 'emgF', 'l_angle', 'loadcell', '__header__', '__version__', 'r_angle', 'Keyboard', 'target'])

To grab the emgE data for example, we have to index the values by the keys, which as the print output shows, are stored as a Numpy array:

1
Type and size of structure: <class 'numpy.ndarray'> 1

Line 14-16. It turns out that data in the emgE array are stored as a nested list of lists of lists, so we have to index twice to get to the lowest level of lists. We can see this from the following output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
EMG values: 
 [[ array([[ 0.00762939],
       [ 0.00717163],
       [ 0.00610352],
       ..., 
       [ 0.0050354 ],
       [ 0.00518799],
       [ 0.00595093]])]]
EMG values index [0]: 
 [ array([[ 0.00762939],
       [ 0.00717163],
       [ 0.00610352],
       ..., 
       [ 0.0050354 ],
       [ 0.00518799],
       [ 0.00595093]])]
EMG values index [0][0]: 
 [[ 0.00762939]
 [ 0.00717163]
 [ 0.00610352]
 ..., 
 [ 0.0050354 ]
 [ 0.00518799]
 [ 0.00595093]]

Line 18-25. Since the lowest level data are still stored as an array of lists, we flatten the array using the flatten() function, which stores the EMG values as numeric data and allows us to plot them. This is confirmed by checking the data type and the values of the array.

1
2
3
Type and size of structure: <class 'numpy.ndarray'> 10302
Values of array: [ 0.00762939  0.00717163  0.00610352 ...,  0.0050354   0.00518799
  0.00595093]


Figure 1:

loadmat

Summary

To import data from Spike2 to Python, we first export the Spike2 data as a Matlab file, then import the data using the scipy.io function. Data from each channel are indexed using the dictionary key, and flattened for conversion into a numeric array for further analysis.

 

6 comments

  • Hi,

    Just wondering if you would know how to continuously stream data from Spike2 into MATLAB? I.E: as raw data is coming into Spike2, I want to be able to analyse the new data in MATLAB as well (as opposed to constantly saving Spike2 as a MATLAB format.

    Cheers,

    Aaron

    Like

    • Hi Aaron, interesting question. I have not used it myself but the MATLAB/SON library may be what you are after.

      See this MATLAB thread:
      https://au.mathworks.com/matlabcentral/newsreader/view_thread/88453

      And CED links:
      http://ced.co.uk/upgrades/spike2tools
      http://ced.co.uk/upgrades/spike2matson

      Cheers

      Like

      • Hi Joanna,

        Thanks for the quick reply. I’ve been trying to use the MATSON library but have encountered problems with it (mainly re-reading the same data multiple times since MATLAB processes the information multiple times before the buffer is emptied and replaced). I was wondering if you would have any knowledge into reading external buffers into MATLAB or know of an easy way of trying to interface MATLAB with external programs? The only solution I could think of atm is to constantly save Script2 data and using MATLAB to read it (but it seems inefficient).

        Cheers,

        Aaron

        Like

      • Thanks Aaron. I think an answer is beyond me. MATLAB has some functionality to run external commands (eg. here) but this might not be what you’re after. That MATLAB community threads may offer other alternatives?

        For the kind of data I collect (randomised conditions, repeated measures, etc. and process offline) I began to find Spike a little clunky to operate and am transitioning across to LabView. LabView is very customisable and could be a more suitable alternative for you in the long term, if existing systems can’t meet your needs.. Just a small suggestion.

        All the best,
        Jo

        Like

  • Hey,do you use Python to analysis the surface EMG signals? Is it better than matlab?please give me some advice, because I want change the programme platform(python) thank you so much!

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s