File in/out: How to import data files from Spike2 (and Matlab) into Python
Spike2 is a type of signal acquisition software commonly used in physiological studies (eg. electromyography, kinematics, cardiovascular research, sleep, etc). It is possible to perform signal analysis in Spike2 by using the drop-down menus, or by programming scripts to automate analysis. However, the programming language for Spike2 is difficult to read, which discourages many scientists from learning to program scripts and automate analysis in Spike2. Also, Spike2 is proprietary software designed to acquire signals using CED (Cambridge Electronic Design) hardware. So even if scientists learned to automate analysis using Spike2 scripts, sharing these scripts with others is of little benefit if other users don’t own or use Spike2 and CED hardware.
For scientists who work in labs that use Spike2 and CED hardware, what kind of research practices could we implement to encourage scientific reproducibility but avoid being constrained to only one type of data acquisition system? For example, we could use Spike2 and CED to sample raw data, but import the raw Spike2 data into Python and analyse the signals for all subjects with a Python script. This automates the analysis such that it is consistent across all subjects, and importantly, the analysis is transparent and efficient.
Importing data from Spike2 files into Python is a little fiddly but managable when we understand how the data are formatted. Spike2 stores data in .smr files. To begin, use Spike2 to export data from the .smr file to a Matlab .mat file, then import the Matlab file into Python.
Data structures. When Spike2 data are exported to Matlab, the data are stored as a Matlab structure where each channel’s data values and times are nested within the channel. A Matlab structure is the equivalent of a Python dictionary, where data are stored as keys and values, and data values can be referenced by knowing their keys. See this post for a quick revision on dictionaries.
The following code demonstrates how the Matlab file is imported into Python. I use an example Spike file, exported as Matlab format, that contains data for a calibration routine. The data files are available for download here.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
import numpy as np
import scipy.io as sio
import matplotlib.pyplot as plt
# Include Ipython magic command if working in a Jupyter notebook
%matplotlib inline
data = sio.loadmat('001.mat')
print('Type and size of structure:', type(data), np.size(data))
print(data.keys())
emgE = data['emgE']['values']
print('Type and size of structure:', type(emgE), np.size(emgE))
print('EMG values: \n', emgE)
print('EMG values index [0]: \n', emgE[0])
print('EMG values index [0][0]: \n', emgE[0][0])
emgE = emgE[0][0].flatten() # emgE = emgE[0,0].flatten()
print('Type and size of structure:', type(emgE), np.size(emgE))
print('Values of array:', emgE)
plt.plot(emgE)
plt.xlabel('Samples')
plt.ylabel('EMG (a.u.)')
plt.savefig('fig1.png')
|
Line 1-5. Import the usual numpy and matplotlib libraries. We also import a special input and output function from the Scipy library (scipy.io) and give it an alias (sio)
Line 7-12. Read in the datafile sub01.mat using the loadmat function.
Python imports data from the .mat file into a dictionary. We can confirm this by printing the type and size of the data structure, and printing the dictionary keys:
1 2 |
Type and size of structure: <class 'dict'> 1
dict_keys(['__globals__', 'emgE', 'emgF', 'l_angle', 'loadcell', '__header__', '__version__', 'r_angle', 'Keyboard', 'target'])
|
To grab the emgE data for example, we have to index the values by the keys, which as the print output shows, are stored as a Numpy array:
1 |
Type and size of structure: <class 'numpy.ndarray'> 1
|
Line 14-16. It turns out that data in the emgE array are stored as a nested list of lists of lists, so we have to index twice to get to the lowest level of lists. We can see this from the following output:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
EMG values:
[[ array([[ 0.00762939],
[ 0.00717163],
[ 0.00610352],
...,
[ 0.0050354 ],
[ 0.00518799],
[ 0.00595093]])]]
EMG values index [0]:
[ array([[ 0.00762939],
[ 0.00717163],
[ 0.00610352],
...,
[ 0.0050354 ],
[ 0.00518799],
[ 0.00595093]])]
EMG values index [0][0]:
[[ 0.00762939]
[ 0.00717163]
[ 0.00610352]
...,
[ 0.0050354 ]
[ 0.00518799]
[ 0.00595093]]
|
Line 18-25. Since the lowest level data are still stored as an array of lists, we flatten the array using the flatten() function, which stores the EMG values as numeric data and allows us to plot them. This is confirmed by checking the data type and the values of the array.
1 2 3 |
Type and size of structure: <class 'numpy.ndarray'> 10302
Values of array: [ 0.00762939 0.00717163 0.00610352 ..., 0.0050354 0.00518799
0.00595093]
|
Figure 1:
Summary
To import data from Spike2 to Python, we first export the Spike2 data as a Matlab file, then import the data using the scipy.io function. Data from each channel are indexed using the dictionary key, and flattened for conversion into a numeric array for further analysis.
Scientifically Sound 
Hello, I have a problem exporting data from spike to matlab, I’ve made every step there is to make, and I still get an error at the end saying “file operations or writing data into file failed”. I don’t know what’s happening and neither do my coworkers.
I’ve already tried reinstaling spike and even a different version, I’m on version 7.11 right now.
Please help.
LikeLike
Hi Fatima,
If you have exhausted all avenues, the best approach would be to contact CED for support as they are the developers of Spike2 and the hardware: http://ced.co.uk/support/introduction
It really should be a simple File -> Export to Matlab so I think they are best placed to help out.
LikeLike
Hi Fatima,
Please contact me directly Simong@ced.co.uk and we can arrange to have a data file from you. I am not aware of any problems with the Matlab export but please update your copy to the latest v7.20 free of charge on our website. We can also discuss getting you a Library of files for direct export to Python which is to be released very soon.
Simon
LikeLike
How can I export .smr files from spike 2.0 to .mat file . I have spike2.0 version 5.06 ? Thank you
LikeLike
Hi Juan,
Open the .smr file in Spike, then click on the tab File -> Export data. This should provide options to export the .smr file into .mat and other formats.
Alternatively, you could write a Spike2 script to batch export many .smr files as .mat files.
Cheers,
Joanna
LikeLike
Hi,
Just thought I would add that there are Spike2 (and Signal) script commands MatlabOpen(), …Put() and …Get() which allow access to Matlab working space. A copy of Matlab running on the same machine as Spike2 could be passed recorded data as it is acquired.This method could also be used off-line as an alternative to saving an smr or smrX file as .mat.
Hope this helps.
Simon
LikeLike
Hi,
Just wondering if you would know how to continuously stream data from Spike2 into MATLAB? I.E: as raw data is coming into Spike2, I want to be able to analyse the new data in MATLAB as well (as opposed to constantly saving Spike2 as a MATLAB format.
Cheers,
Aaron
LikeLike
Hi Aaron, interesting question. I have not used it myself but the MATLAB/SON library may be what you are after.
See this MATLAB thread:
https://au.mathworks.com/matlabcentral/newsreader/view_thread/88453
And CED links:
http://ced.co.uk/upgrades/spike2tools
http://ced.co.uk/upgrades/spike2matson
Cheers
LikeLike
Hi Joanna,
Thanks for the quick reply. I’ve been trying to use the MATSON library but have encountered problems with it (mainly re-reading the same data multiple times since MATLAB processes the information multiple times before the buffer is emptied and replaced). I was wondering if you would have any knowledge into reading external buffers into MATLAB or know of an easy way of trying to interface MATLAB with external programs? The only solution I could think of atm is to constantly save Script2 data and using MATLAB to read it (but it seems inefficient).
Cheers,
Aaron
LikeLike
Thanks Aaron. I think an answer is beyond me. MATLAB has some functionality to run external commands (eg. here) but this might not be what you’re after. That MATLAB community threads may offer other alternatives?
For the kind of data I collect (randomised conditions, repeated measures, etc. and process offline) I began to find Spike a little clunky to operate and am transitioning across to LabView. LabView is very customisable and could be a more suitable alternative for you in the long term, if existing systems can’t meet your needs.. Just a small suggestion.
All the best,
Jo
LikeLike
Hey,do you use Python to analysis the surface EMG signals? Is it better than matlab?please give me some advice, because I want change the programme platform(python) thank you so much!
LikeLiked by 1 person
Hi Kevin, thanks for your comment. Yes we do use Python to analyse EMG signals. I personally find Python a little more readable than Matlab but both programs are good. We wrote a short tutorial series on analysing EMG signals using Python (eg. see https://scientificallysound.org/2016/08/18/python-analysing-emg-signals-part-3/) You will find it easy to transition from Matlab to Python.
Cheers, Jo
LikeLiked by 1 person