Skip to content

Contrastive Self-Supervised Learning - InfoNCE + CSDP

Self-supervised learning for SNNs without labeled data. Two complementary approaches are exposed: global InfoNCE loss for batch training and a local CSDP rule for biologically plausible on-chip learning experiments.

SpikeContrastiveLoss - InfoNCE for Spikes

Adapted InfoNCE contrastive loss for spike-rate representations. Given two augmented views of the same batch, positive pairs = same input (different augmentation), negative pairs = different inputs. The loss encourages representations of the same input to be similar, and different inputs to be dissimilar.

loss = -mean(log(exp(sim(a_i, b_i)/τ) / Σ_j exp(sim(a_i, b_j)/τ)))

Parameter Default Meaning
temperature 0.5 Contrastive temperature scaling

temperature must be finite and positive. compute() accepts finite 2-D arrays with identical shape (batch, n_features) and returns 0.0 for batch size < 2 because no in-batch negatives are available.

CSDPRule - Contrastive Signal-Dependent Plasticity

Biologically plausible local learning rule. Generalizes the Forward-Forward algorithm to spiking circuits:

  • Positive phase: Present real data → Hebbian update: dW = lr * (post ⊗ pre) - decay * W
  • Negative phase: Present corrupted data → anti-Hebbian update: dW = -lr * (post ⊗ pre)
  • Goodness: g = Σ(activations²) — positive data should have high goodness, negative data low
Parameter Default Meaning
lr 0.01 Learning rate
decay 0.001 Weight decay

lr and decay must be finite and non-negative. Update inputs are validated as weights.shape == (len(post_spikes), len(pre_spikes)); all weights and spike vectors must be finite. Updates return new arrays and do not mutate the input matrix.

Usage

Python
import numpy as np
from sc_neurocore.contrastive import CSDPRule, SpikeContrastiveLoss

# InfoNCE training
loss_fn = SpikeContrastiveLoss(temperature=0.5)
view_a = np.random.randn(32, 128)  # batch of 32, 128 features
view_b = np.random.randn(32, 128)  # augmented version
loss = loss_fn.compute(view_a, view_b)

# CSDP local learning
csdp = CSDPRule(lr=0.01)
W = np.random.randn(64, 32) * 0.1
real_spikes = np.random.rand(32)
real_activations = np.random.rand(64)
noise_spikes = np.random.rand(32)
noise_activations = np.random.rand(64)
W = csdp.contrastive_step(
    W,
    pos_pre=real_spikes, pos_post=real_activations,
    neg_pre=noise_spikes, neg_post=noise_activations,
)

Validation and Benchmark Evidence

The public Python API is covered by the module-specific tests/test_contrastive.py surface. The maintained polyglot mirrors are:

Surface Scope Local check
Python Public InfoNCE and CSDP contracts, validation guards, deterministic algebra pytest tests/test_contrastive.py
Rust Safety mirror for row-wise InfoNCE and CSDP matrix updates rustc --edition=2021 --test src/sc_neurocore/accel/rust/safety/ssl.rs
Julia Row-wise InfoNCE and CSDP validation mirror julia --startup-file=no --history-file=no ... validate_ssl()
Mojo Standalone InfoNCE/CSDP validation kernel mojo src/sc_neurocore/accel/mojo/kernels/ssl.mojo

benchmarks/results/bench_contrastive_ssl.json records the latest local, non-isolated evidence. The Python public API measured 1000 deterministic calls at 10117.413 InfoNCE calls/s and 26799.118 CSDP contrastive steps/s on the current workstation. Rust, Julia, and Mojo validation checks all passed in that same run. The artifact is a local regression record, not an isolated production benchmark claim.

Reference: Ororbia 2024, Science Advances.

See Tutorial 80: Contrastive SSL.

sc_neurocore.contrastive.ssl

Contrastive self-supervised learning for SNNs.

SpikeContrastiveLoss: InfoNCE-style loss for spike representations. CSDPRule: Contrastive Signal-Dependent Plasticity — biologically plausible local learning rule (Science Advances 2024).

No SNN library ships self-supervised learning utilities.

SpikeContrastiveLoss

InfoNCE contrastive loss adapted for spike representations.

Computes similarity between spike-rate vectors from two augmented views of the same input. Positive pairs = same input, different augmentation. Negative pairs = different inputs.

Parameters

temperature : float Contrastive temperature scaling.

Source code in src/sc_neurocore/contrastive/ssl.py
Python
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
class SpikeContrastiveLoss:
    """InfoNCE contrastive loss adapted for spike representations.

    Computes similarity between spike-rate vectors from two augmented
    views of the same input. Positive pairs = same input, different
    augmentation. Negative pairs = different inputs.

    Parameters
    ----------
    temperature : float
        Contrastive temperature scaling.
    """

    def __init__(self, temperature: float = 0.5) -> None:
        self.temperature = _finite_scalar(temperature, "temperature", positive=True)

    def compute(
        self,
        view_a: np.ndarray[Any, Any],
        view_b: np.ndarray[Any, Any],
    ) -> float:
        """Compute contrastive loss for a batch of spike-rate pairs.

        Parameters
        ----------
        view_a : ndarray of shape (batch, n_features)
            Spike rates from augmentation A.
        view_b : ndarray of shape (batch, n_features)
            Spike rates from augmentation B.

        Returns
        -------
        float
            Mean InfoNCE loss. Single-item batches return ``0.0`` because no
            in-batch negatives are available.

        Raises
        ------
        ValueError
            If either view is not a finite 2-D array with the same shape.
        """
        view_a_matrix = _as_float_matrix(view_a, "view_a")
        view_b_matrix = _as_float_matrix(view_b, "view_b")
        if view_a_matrix.shape != view_b_matrix.shape:
            raise ValueError("view_a and view_b must have the same shape")

        batch = view_a_matrix.shape[0]
        if batch < 2:
            return 0.0

        a_norm = view_a_matrix / np.clip(
            np.linalg.norm(view_a_matrix, axis=1, keepdims=True),
            1e-8,
            None,
        )
        b_norm = view_b_matrix / np.clip(
            np.linalg.norm(view_b_matrix, axis=1, keepdims=True),
            1e-8,
            None,
        )

        sim = a_norm @ b_norm.T / self.temperature
        exp_sim = np.exp(sim - sim.max(axis=1, keepdims=True))
        log_prob = np.log(
            np.clip(
                np.diag(exp_sim) / exp_sim.sum(axis=1),
                1e-10,
                None,
            )
        )
        return -float(log_prob.mean())

compute(view_a, view_b)

Compute contrastive loss for a batch of spike-rate pairs.

Parameters

view_a : ndarray of shape (batch, n_features) Spike rates from augmentation A. view_b : ndarray of shape (batch, n_features) Spike rates from augmentation B.

Returns

float Mean InfoNCE loss. Single-item batches return 0.0 because no in-batch negatives are available.

Raises

ValueError If either view is not a finite 2-D array with the same shape.

Source code in src/sc_neurocore/contrastive/ssl.py
Python
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
def compute(
    self,
    view_a: np.ndarray[Any, Any],
    view_b: np.ndarray[Any, Any],
) -> float:
    """Compute contrastive loss for a batch of spike-rate pairs.

    Parameters
    ----------
    view_a : ndarray of shape (batch, n_features)
        Spike rates from augmentation A.
    view_b : ndarray of shape (batch, n_features)
        Spike rates from augmentation B.

    Returns
    -------
    float
        Mean InfoNCE loss. Single-item batches return ``0.0`` because no
        in-batch negatives are available.

    Raises
    ------
    ValueError
        If either view is not a finite 2-D array with the same shape.
    """
    view_a_matrix = _as_float_matrix(view_a, "view_a")
    view_b_matrix = _as_float_matrix(view_b, "view_b")
    if view_a_matrix.shape != view_b_matrix.shape:
        raise ValueError("view_a and view_b must have the same shape")

    batch = view_a_matrix.shape[0]
    if batch < 2:
        return 0.0

    a_norm = view_a_matrix / np.clip(
        np.linalg.norm(view_a_matrix, axis=1, keepdims=True),
        1e-8,
        None,
    )
    b_norm = view_b_matrix / np.clip(
        np.linalg.norm(view_b_matrix, axis=1, keepdims=True),
        1e-8,
        None,
    )

    sim = a_norm @ b_norm.T / self.temperature
    exp_sim = np.exp(sim - sim.max(axis=1, keepdims=True))
    log_prob = np.log(
        np.clip(
            np.diag(exp_sim) / exp_sim.sum(axis=1),
            1e-10,
            None,
        )
    )
    return -float(log_prob.mean())

CSDPRule dataclass

Contrastive Signal-Dependent Plasticity.

Local learning rule: weight update depends on (pre, post, contrastive_signal). Positive phase: present real data → Hebbian update. Negative phase: present corrupted data → anti-Hebbian update.

Generalizes Forward-Forward to spiking circuits.

Reference: Ororbia 2024, Science Advances

Parameters

lr : float Learning rate. decay : float Weight decay for regularization.

Source code in src/sc_neurocore/contrastive/ssl.py
Python
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
@dataclass
class CSDPRule:
    """Contrastive Signal-Dependent Plasticity.

    Local learning rule: weight update depends on (pre, post, contrastive_signal).
    Positive phase: present real data → Hebbian update.
    Negative phase: present corrupted data → anti-Hebbian update.

    Generalizes Forward-Forward to spiking circuits.

    Reference: Ororbia 2024, Science Advances

    Parameters
    ----------
    lr : float
        Learning rate.
    decay : float
        Weight decay for regularization.
    """

    lr: float = 0.01
    decay: float = 0.001

    def __post_init__(self) -> None:
        """Validate the scalar learning-rule parameters."""
        self.lr = _finite_scalar(self.lr, "lr")
        self.decay = _finite_scalar(self.decay, "decay")

    def positive_update(
        self,
        weights: np.ndarray[Any, Any],
        pre_spikes: np.ndarray[Any, Any],
        post_spikes: np.ndarray[Any, Any],
    ) -> np.ndarray[Any, Any]:
        """Hebbian update from positive (real) data.

        dW = lr * (post @ pre^T) - decay * W
        """
        weights_matrix, pre_vector, post_vector = self._validate_update_inputs(
            weights,
            pre_spikes,
            post_spikes,
        )
        dW = self.lr * np.outer(post_vector, pre_vector) - self.decay * weights_matrix
        return weights_matrix + dW

    def negative_update(
        self,
        weights: np.ndarray[Any, Any],
        pre_spikes: np.ndarray[Any, Any],
        post_spikes: np.ndarray[Any, Any],
    ) -> np.ndarray[Any, Any]:
        """Anti-Hebbian update from negative (corrupted) data.

        dW = -lr * (post @ pre^T)
        """
        weights_matrix, pre_vector, post_vector = self._validate_update_inputs(
            weights,
            pre_spikes,
            post_spikes,
        )
        dW = -self.lr * np.outer(post_vector, pre_vector)
        return weights_matrix + dW

    def contrastive_step(
        self,
        weights: np.ndarray[Any, Any],
        pos_pre: np.ndarray[Any, Any],
        pos_post: np.ndarray[Any, Any],
        neg_pre: np.ndarray[Any, Any],
        neg_post: np.ndarray[Any, Any],
    ) -> np.ndarray[Any, Any]:
        """Apply one positive phase followed by one negative phase.

        Parameters
        ----------
        weights : ndarray of shape (n_post, n_pre)
            Current synaptic weight matrix.
        pos_pre, neg_pre : ndarray of shape (n_pre,)
            Presynaptic spike-rate vectors for real and corrupted samples.
        pos_post, neg_post : ndarray of shape (n_post,)
            Postsynaptic activation vectors for real and corrupted samples.

        Returns
        -------
        ndarray of shape (n_post, n_pre)
            Updated weights after Hebbian and anti-Hebbian phases.
        """
        w = self.positive_update(weights, pos_pre, pos_post)
        w = self.negative_update(w, neg_pre, neg_post)
        return w

    def goodness(self, activations: np.ndarray[Any, Any]) -> float:
        """Compute 'goodness' score (sum of squared activations).

        Positive data should have high goodness, negative data low.
        """
        values = np.asarray(activations, dtype=np.float64)
        if not np.all(np.isfinite(values)):
            raise ValueError("activations must contain only finite values")
        return float(np.sum(values**2))

    @staticmethod
    def _validate_update_inputs(
        weights: np.ndarray[Any, Any],
        pre_spikes: np.ndarray[Any, Any],
        post_spikes: np.ndarray[Any, Any],
    ) -> tuple[FloatArray, FloatArray, FloatArray]:
        weights_matrix = _as_weight_matrix(weights)
        pre_vector = _as_float_vector(pre_spikes, "pre_spikes")
        post_vector = _as_float_vector(post_spikes, "post_spikes")
        expected_shape = (post_vector.shape[0], pre_vector.shape[0])
        if weights_matrix.shape != expected_shape:
            raise ValueError("weights must have shape (len(post_spikes), len(pre_spikes))")
        return weights_matrix, pre_vector, post_vector

__post_init__()

Validate the scalar learning-rule parameters.

Source code in src/sc_neurocore/contrastive/ssl.py
Python
165
166
167
168
def __post_init__(self) -> None:
    """Validate the scalar learning-rule parameters."""
    self.lr = _finite_scalar(self.lr, "lr")
    self.decay = _finite_scalar(self.decay, "decay")

positive_update(weights, pre_spikes, post_spikes)

Hebbian update from positive (real) data.

dW = lr * (post @ pre^T) - decay * W

Source code in src/sc_neurocore/contrastive/ssl.py
Python
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
def positive_update(
    self,
    weights: np.ndarray[Any, Any],
    pre_spikes: np.ndarray[Any, Any],
    post_spikes: np.ndarray[Any, Any],
) -> np.ndarray[Any, Any]:
    """Hebbian update from positive (real) data.

    dW = lr * (post @ pre^T) - decay * W
    """
    weights_matrix, pre_vector, post_vector = self._validate_update_inputs(
        weights,
        pre_spikes,
        post_spikes,
    )
    dW = self.lr * np.outer(post_vector, pre_vector) - self.decay * weights_matrix
    return weights_matrix + dW

negative_update(weights, pre_spikes, post_spikes)

Anti-Hebbian update from negative (corrupted) data.

dW = -lr * (post @ pre^T)

Source code in src/sc_neurocore/contrastive/ssl.py
Python
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
def negative_update(
    self,
    weights: np.ndarray[Any, Any],
    pre_spikes: np.ndarray[Any, Any],
    post_spikes: np.ndarray[Any, Any],
) -> np.ndarray[Any, Any]:
    """Anti-Hebbian update from negative (corrupted) data.

    dW = -lr * (post @ pre^T)
    """
    weights_matrix, pre_vector, post_vector = self._validate_update_inputs(
        weights,
        pre_spikes,
        post_spikes,
    )
    dW = -self.lr * np.outer(post_vector, pre_vector)
    return weights_matrix + dW

contrastive_step(weights, pos_pre, pos_post, neg_pre, neg_post)

Apply one positive phase followed by one negative phase.

Parameters

weights : ndarray of shape (n_post, n_pre) Current synaptic weight matrix. pos_pre, neg_pre : ndarray of shape (n_pre,) Presynaptic spike-rate vectors for real and corrupted samples. pos_post, neg_post : ndarray of shape (n_post,) Postsynaptic activation vectors for real and corrupted samples.

Returns

ndarray of shape (n_post, n_pre) Updated weights after Hebbian and anti-Hebbian phases.

Source code in src/sc_neurocore/contrastive/ssl.py
Python
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
def contrastive_step(
    self,
    weights: np.ndarray[Any, Any],
    pos_pre: np.ndarray[Any, Any],
    pos_post: np.ndarray[Any, Any],
    neg_pre: np.ndarray[Any, Any],
    neg_post: np.ndarray[Any, Any],
) -> np.ndarray[Any, Any]:
    """Apply one positive phase followed by one negative phase.

    Parameters
    ----------
    weights : ndarray of shape (n_post, n_pre)
        Current synaptic weight matrix.
    pos_pre, neg_pre : ndarray of shape (n_pre,)
        Presynaptic spike-rate vectors for real and corrupted samples.
    pos_post, neg_post : ndarray of shape (n_post,)
        Postsynaptic activation vectors for real and corrupted samples.

    Returns
    -------
    ndarray of shape (n_post, n_pre)
        Updated weights after Hebbian and anti-Hebbian phases.
    """
    w = self.positive_update(weights, pos_pre, pos_post)
    w = self.negative_update(w, neg_pre, neg_post)
    return w

goodness(activations)

Compute 'goodness' score (sum of squared activations).

Positive data should have high goodness, negative data low.

Source code in src/sc_neurocore/contrastive/ssl.py
Python
234
235
236
237
238
239
240
241
242
def goodness(self, activations: np.ndarray[Any, Any]) -> float:
    """Compute 'goodness' score (sum of squared activations).

    Positive data should have high goodness, negative data low.
    """
    values = np.asarray(activations, dtype=np.float64)
    if not np.all(np.isfinite(values)):
        raise ValueError("activations must contain only finite values")
    return float(np.sum(values**2))