Teaching signal synchronization in deep neural networks with prospective neurons

This repository contains code to reproduce the results from our paper: Link to paper

Overview

This project demonstrates that neurons enhanced with an adaptive current can compensate for delays by responding to external stimuli prospectively – effectively predicting future inputs to synchronize with them. First, we show that such prospective neurons enable teaching signal synchronization across a range of learning algorithms that propagate error signals through hierarchical networks. Second, we demonstrate that this successfully guides learning in slowly integrating neurons, enabling the formation and retrieval of memories over extended timescales. We support our findings with a mathematical analysis of the prospective coding mechanism and learning experiments on motor control tasks. Together, our results reveal how neural adaptation could solve a critical timing problem and enable efficient learning in dynamic environments.

Installation

Prerequisites

• Python 3.11+
• pip

Setup

1. Clone the repository:

git clone <repository-url>
cd ProspectiveNeurons

2. Install dependencies:

pip install -r requirements.txt

Project Structure

prospective_neurons/          # (Figure 3, 5) Experiments
├── train.py                  # Training loop and utilities
├── model.py                  # Network architectures (prospective layers, recurrent layers)
├── data.py                   # Dataset loading and management
├── process.py                # Data processing utilities
├── RNN_online_learning.py    # Online learning algorithms
├── datasets/                 # Task definitions
│   ├── neurogym.py           # Delayed reach 2D task
│   └── no_memory_teacher_student.py  # Teacher-student task
cartpole/                     # CartPole A2C experiment (Figure 4)
├── EqProp_continuous_a2c_cartpole.py  # A2C training with continuous EqProp
├── EqPropNet.py              # Network architectures (FFNet, ModelState)
├── scan_a2c_BP_control_dt_1ms_10_runs.yml  # WandB sweep config for BP baseline
├── scan_a2c_prospective_control_dt_1ms_10_runs.yml  # WandB sweep config for ProspectiveBP
hep_comparison/               # (Table 1)
├── learning_bp.py            # 
├── scan_hEP.yml              # 
├── scan_online_FF.yml        # 
├── scan_RBP.yml              # 
figures/                      # Jupyter notebooks for figure generation
├── figure_1.ipynb            # 
├── figure_2.ipynb            # 
├── figure_3.ipynb            # 
├── figure_4.ipynb            # 
└── utils.py                  # Plotting utilities

Reproducing Paper Results

Figure 1, 2, and 3: Properties of Prospective Neurons

To reproduce the experiments for each figure, see the corresponding jupyter notebook file in /figures

Table 1: Teacher-student learning task

For feedforward network experiments

cd hep_comparison
python learning_bp.py --method <method> --lr 0.001

<method> is one of ProspEqBP, ProspEqFA, ProspEqDFA, OnlineBP, EqBP. See scan_online_FF.yml

For recurrent network experiments with RBP

cd hep_comparison
python learning_bp.py --method LeakyRBP --lr 0.00316
python learning_bp.py --method ProspRBP --lr 0.001

See scan_RBP.yml

For recurrent network experiments with hEP

cd hep_comparison
python learning_bp.py --method hEP --lr 0.0316 --omega_beta 597 --tau_ema 1.0
python learning_bp.py --method ProsphEP --lr 0.316 --omega_beta 597 --tau_ema 1.0

See scan_hEP.yml

Figure 4: Cartpole

For BP results

cd cartpole
wandb sweep scan_a2c_BP_control_dt_1ms_10_runs.yml

For ProspectiveBP results

cd cartpole
wandb sweep scan_a2c_prospective_control_dt_1ms_10_runs.yml

Figure 5: Delayed reach task

python run_train.py \
  --dataset delay_reach_2d \
  --network prospective_rnn \
  --lr 1e-2 \
  --epochs 1601 \
  --lr_schedule cosine \
  --n_recurrent 25 \
  --n_features 25 \
  --batch_size 1 

--network is one of prospective_rnn or prospective_ff for model comparison

Citation

If you use this code, please cite our paper:

@article{zucchet2025teaching,
  title={Teaching signal synchronization in deep neural networks with prospective neurons},
  author={Zucchet, Nicoas and Feng, Qianqian and Laborieux, Axel and Zenke, Friedemann and Senn, Walter and Sacramento, Jo{\~a}o},
  journal={arXiv preprint arXiv:2511.14917},
  year={2025}
}