Error reading maxwell data (version 20160704)

[michaelhadler]

Hi everyone,

I want to use SpikeInterface on my MaxOne data (recorded using the "Network" recording assay in MaxLab, version 20160704). For starters, I want to create a bin and probe file to give it a go in Kilosort 4. However, apparently the data can't be loaded when calling it with the maxwell extractor. My impression is that there is a discrepancy between the actual .h5 file structure and the expectations in init. The file is unchanged from the initial recording, so I'm not quite sure what's amiss. Any ideas?

#Performed on Windows 10, python 3.9.21, neo 0.14.0, numpy 1.26.4.
from spikeinterface.extractors import read_maxwell

recording = read_maxwell(file_path) #file_path refers to a previously selected .h5 file recorded in MaxLab "Network" assay

Here's the error traceback:
Error loading data: could not assign tuple of length 2 to structure with 3 fields.

(kilosort) D:\HD-MEA\250116_Mouse-Ca12\Slice_6\Recording\002_Baseline2>python run_kilosort.py
Error loading data: could not assign tuple of length 2 to structure with 3 fields.
Traceback (most recent call last):
  File "D:\HD-MEA\250116_Mouse-Ca12\Slice_6\Recording\002_Baseline2\run_kilosort.py", line 24, in <module>
    recording = read_maxwell(file_path)
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\spikeinterface\extractors\neoextractors\maxwell.py", line 62, in __init__
    NeoBaseRecordingExtractor.__init__(
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\spikeinterface\extractors\neoextractors\neobaseextractor.py", line 188, in __init__
    _NeoBaseExtractor.__init__(self, block_index, **neo_kwargs)
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\spikeinterface\extractors\neoextractors\neobaseextractor.py", line 27, in __init__
    self.neo_reader = self.get_neo_io_reader(self.NeoRawIOClass, **neo_kwargs)
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\spikeinterface\extractors\neoextractors\neobaseextractor.py", line 66, in get_neo_io_reader
    neo_reader.parse_header()
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\neo\rawio\baserawio.py", line 211, in parse_header
    self._parse_header()
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\neo\rawio\maxwellrawio.py", line 113, in _parse_header
    signal_streams = np.array(signal_streams, dtype=_signal_stream_dtype)
ValueError: could not assign tuple of length 2 to structure with 3 fields.

As the exemplary files are all > 1GB, I'm happy to share one privately if anyone can help me out :)

Thanks, and best,

Michael

[alejoe91]

Hi @michaelhadler

I pushed a fix here: #1659

The failure was due to a change in the _signal_stream_dtype, which was not (yet) propagated to this IO.

Do you know how to install and test from a PR?

[michaelhadler]

Nope, pointers are appreciated ;)

[alejoe91]

OK, in the environment where you have neo/spikeinterface installed, you can do:

pip uninstall -y neo
pip install git+https://github.com/alejoe91/python-neo.git@fix-maxwell-io

This will install NEO from my fork (see alejoe91) and the branch with the fix: fix-maxwell-io

[michaelhadler]

Perfect, thanks! For now, read.maxwell() seems to have worked :) should I close or do you want to give me an extra day to see whether the follow-up's (probe creation, etc) check out, too?

[alejoe91]

We'll close once the PR is merged! If you find other problems, feel free to open another issue (likely on the spikeinterface repo) :)

[michaelhadler]

Ok, I got a follow-up error when converting to binary with spikeinterface_to_binary (kilosort):

# data directory is a path to a separate folder, filename the original h5 file

from spikeinterface.extractors import read_maxwell
from kilosort import io

recording = read_maxwell(filename)

binary_filename, N, c, s, fs, probe_filename = io.spikeinterface_to_binary(
    recording, data_directory, data_name='data.bin', dtype=np.int16,
    chunksize=60000, export_probe=True, probe_name='probe.prb'
)

Traceback (most recent call last):
  File "D:\HD-MEA\250116_Mouse-Ca12\Slice_6\Recording\002_Baseline2\run_kilosort.py", line 31, in <module>
    binary_filename, N, c, s, fs, probe_filename = io.spikeinterface_to_binary(
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\kilosort\io.py", line 1161, in spikeinterface_to_binary
    N = recording.get_total_samples()
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\spikeinterface\core\baserecording.py", line 211, in get_total_samples
    return sum(samples_per_segment)
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\spikeinterface\core\baserecording.py", line 209, in <genexpr>
    samples_per_segment = (self.get_num_samples(segment_index) for segment_index in range(num_segments))
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\spikeinterface\core\baserecording.py", line 195, in get_num_samples
    return int(self._recording_segments[segment_index].get_num_samples())
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\spikeinterface\extractors\neoextractors\neobaseextractor.py", line 339, in get_num_samples
    num_samples = self.neo_reader.get_signal_size(
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\neo\rawio\baserawio.py", line 679, in get_signal_size
    return self._get_signal_size(block_index, seg_index, stream_index)
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\neo\rawio\baserawio.py", line 1401, in _get_signal_size
    buffer_desc = self.get_analogsignal_buffer_description(block_index, seg_index, buffer_id)
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\neo\rawio\baserawio.py", line 1371, in get_analogsignal_buffer_description
    descr = self._get_analogsignal_buffer_description(block_index, seg_index, buffer_id)
  File "C:\Users\analyzer\anaconda3\envs\kilosort\lib\site-packages\neo\rawio\maxwellrawio.py", line 228, in _get_analogsignal_buffer_description
    return self._buffer_descriptions[block_index][seg_index][buffer_id]
KeyError: ''

Is this still Neo or a kilosort problem?

Best,

Michael

[alejoe91]

Still NEO :)

[alejoe91]

I pushed a fix. Can you reinstall as in here?

[zm711]

PR looks good to me. As soon as we have confirmation it works I'll merge :)

[michaelhadler]

Hi again, code ran smoothly and both the bin-file and the probe were generated. Not entirely sure the result is correct, but I would open a new issue for that once I get a better feeling for KS4. Thanks so much for helping out!