BlueRecording

BlueRecording is used to produce an input file (also refered to as an electrodes file or a weights file) for the calculation of extracellular signals in neurodamus.

This branch provides code that produces an electrodes file compatible with the SONATA format. For scripts to produce an electrode file compatible with the old BlueConfig format, see the non-sonata branch of this repo.

---

Installation

Dependencies

BlueRecording requires mpi4py, h5py, and hdf5 built with MPI support. The provided setup.sh script handles all of this automatically, including building NEURON, neurodamus, and libsonatareport from source.

setup.sh works for both macOS (with brew) and Linux (with apt) systems:

source setup.sh

If you want the developer version with tests and editable source files:

source setup.sh --dev

If you want to skip the system packages installation append --no-system to the previous lines.

Use --quick to skip building NEURON, libsonatareport, neurodamus, and neurodamus-models from source (useful for CI or when you only need the Python package):

source setup.sh --dev --quick

Use --no-cache to wipe the virtual environment and all build artifacts before reinstalling from scratch (downloaded data is preserved):

source setup.sh --dev --no-cache

Finally, if you want to run the full testing suite you need --data. See the Testing section for details.

This is required only once to set up your python virtual environment. In future sessions you still need to run once:

source setup.sh

But the execution is much faster.

Input data

The initial input data of BlueRecording includes a compartment report. This can be generated by neurodamus, the BBP Simulation Control application for Neuron, neurodamus-models, which contains mod files for neural mechanisms. Since BlueRecording is simulator-agnostic, other simulators can be used as long as they can produce the compartment report.

Neurodamus

setup.sh automatically installs neurodamus, neurodamus-models (neocortex mechanisms), and libsonatareport from source. No separate spack environment is needed.

On the first run, the script will:

1. Create a Python virtual environment with MPI-enabled h5py and mpi4py
2. Clone and build libsonatareport
3. Clone and build NEURON from source (with libsonatareport support)
4. Install neurodamus from source
5. Clone and build neurodamus-models (neocortex)
6. Install the bluerecording package

In subsequent sessions, running source setup.sh again will simply activate the existing environment.

---

Testing

First, make sure you have set up the development environment with test data:

source setup.sh --dev --data

This only needs to be done once. It will download a few hundreds of Mb of data and run a few short simulations.

After that, the simplest way to run the full test suite is:

./run_tests.sh

You can also run subsets:

./run_tests.sh unit   # only unit tests
./run_tests.sh mpi    # only MPI tests

If you need to re-run setup before testing:

./run_tests.sh --setup

Alternatively, you can run the tests manually:

python -m pytest tests/unit/ -v --forked
mpirun -n 2 python -m pytest tests/unit-mpi/test_write_weights.py --with-mpi -v
mpirun -n 2 python -m pytest tests/unit-mpi/test_h5py_MPI.py --with-mpi -v
mpirun -n 2 python -m pytest tests/unit-mpi/test_get_positions.py --with-mpi -v

If you want to run only the base tests (without downloading data), after source setup.sh --dev:

mpirun -n 2 pytest -v tests/unit-mpi --with-mpi
pytest -v -m "not skip_in_ci" tests/unit

This is also what runs in CI, where we avoid downloading the full test data to keep pipelines fast and lightweight.

Note: If you installed with the --quick flag (i.e. source setup.sh --dev --quick), NEURON, neurodamus, libsonatareport, and neurodamus-models are not available. In that case only the non-MPI unit tests can run:

pytest -v -m "not skip_in_ci" tests/unit

The MPI tests and any test that requires neurodamus will be skipped or fail.

To run only the slow, data-intensive tests (e.g., single cell and 100-cell integration tests):

pytest -v -m "slow" tests/unit --forked
mpirun -n 2 pytest -v -m "slow" tests/unit-mpi --with-mpi

Steps to produce electrode files

1. Create a csv file containing information about the electrodes. Each row of the file contains information about one electrode contact. The format of the csv file is defined as follows:

   • The header is name,x,y,z,layer,region,type
   • The first column is the name of the electrode contact. It is either a string or an integer
   • The second through fourth columns are the x, y, and z coordinates of the contact in Cartesian space. They are floats.
   • The fifth column is the cortical layer in which the electrode is located. It is a string in the format LN, where N is an integer.
     • If the electrode is outside of the brain, the value in the column is the string Outside
     • If the electrode is in a region without laminar oraginzation, the value in the column is the string NA
   • The sixth column is the brain region in which the electrode is located. It is a string.
     • If the electrode is outside the brain, the value in the column is the string Outside
   • The seventh column is the calculation method used to determine the compartment weights. Supported values are PointSource, LineSource, Reciprocity, DipoleReciprocity, and various versions of ObjectiveCSD. The PointSource and LineSource methods assume that the neurons are in an infinite homogeneous medium. They should be used only for recordings made inside the brain tissue. If they are used, the tissue conductivity should be provided via the --sigma flag (CLI) or the sigma argument (Python API). Reciprocity and DipoleReciprocity assign the compartment weights based on a lead-field calculated in step 2. These should be used for EEG or ECoG recordings. In general, we recommend using the Reciprocity method. The different ObjectiveCSD variants assign a coefficient of 1 to each compartment that is within a specified region around the electrode and a 0 to all other compartments. More details about this option are available here

   The folder examples/makeCsvFiles contains an example python script that will generate a csv file for a Neuropixels probe.

2. If the Reciprocity or DipoleReciprocity methods are used, you must calculate a lead-field. The lead field is the potential field (for the reciprocity method) or the E-field (for the dipole reciprocity method) produced in the neural tissue by running a current of 1 nA between the recording electrode and the reference electrode. BlueRecording assumes that this field is calculated using the Sim4Life finite element solver and exported as an H5 file. Other calculation methods are possible, assuming the field is exported in the same format.

3. Compute segment positions and write the electrode weights file. The simplest way is via the CLI:

   bluerecording write_weights <path_to_simconfig> <electrode_csv> <output_path> \
       [--sigma <conductivity>] \
       [--path-to-fields <field1.h5> <field2.h5> ...] \
       [--no-replace-axons]

   This single command initializes the circuit, computes segment positions, creates the H5 electrode file, and populates it with the correct coefficients. It must be run with MPI (e.g. mpirun -n 4 bluerecording write_weights ...).

   If you only need the segment positions (e.g. for inspection or reuse), you can compute and save them separately:

   bluerecording write_positions <path_to_simconfig> <path_to_positions_folder> \
       [--no-replace-axons]

   Python API. The same steps can be performed from Python:

   from bluerecording.circuit import init_circuit
   from bluerecording.positions import get_positions, save_positions
   from bluerecording.weights import initializeH5File, writeH5File
   
   # Initialize the circuit (loads neurodamus, extracts discretization info)
   node_manager, ids, cols, population, population_name = init_circuit(path_to_simconfig)
   
   # Compute segment boundary positions for all cells on this MPI rank
   positions_df, cols = get_positions(
       node_manager, ids, cols, population,
       path_to_simconfig=path_to_simconfig,
   )
   
   # Optionally save positions to disk
   save_positions(positions_df, path_to_positions_folder)
   
   # Create the H5 electrode file (run on all ranks; only rank 0 writes)
   initializeH5File(cols, population_name, outputfile, electrode_csv)
   
   # Populate the electrode file with the correct coefficients (MPI-parallel)
   writeH5File(positions_df, cols, population_name, outputfile,
               sigma=[0.277], path_to_fields=None)

   Here sigma is the extracellular conductivity in S/m (default 0.277), used by the PointSource and LineSource methods. path_to_fields is a list of paths to finite element H5 files, required when using Reciprocity or DipoleReciprocity electrodes. Details about the objective_csd_array_indices argument are available here.

The two-step procedure (initialize then write) is used because creating the H5 file requires writing variable-length strings, which is not supported by HDF5 parallel I/O. The initialization runs on rank 0 only, after which all ranks write coefficients in parallel.

Running an extracellular recording simulation

Once the electrode file has been generated, it can be used in a Neurodamus simulation that includes extracellular recording. Instructions for this step are found here

---

Examples

See here

A good starting point is the single cell L5 TPC example, which walks through computing electrode weights, running a simulation, and visualizing extracellular signals from a single neuron.

The analyze_weights notebook is a diagnostic tool for inspecting weights files. It can validate the structure and show weight distributions for a single file, or compare two files for regression testing. When comparing multiple files, they must share the same discretization (same circuit, same compartment report settings), and at least one of the files must have been built with the --with-neurite-type flag so that per-section-type breakdowns are available.

---

Contribution Guidelines

Here

---

Citation

If you use this software, we kindly ask you to cite the following publication: Tharayil et al. BlueRecording: A Pipeline for efficient calculation of extracellular recordings in large-scale neural circuit models. bioRxiv, (2024)

---

Acknowledgment

The development of this software was supported by funding to the Blue Brain Project, a research center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government's ETH Board of the Swiss Federal Institutes of Technology.

Copyright (c) 2005-2024 Blue Brain Project/EPFL

Copyright (c) 2025-2026 Open Brain Institute