Struct HybridLinearAttentionNeuron 

pub struct HybridLinearAttentionNeuron {
    pub v: f64,
    state_kv: Vec<f64>,
    pub dim: usize,
    pub lambda: f64,
    pub window_size: usize,
    window_buf: Vec<f64>,
    window_idx: usize,
    pub dt: f64,
}

Hybrid linear attention neuron for spiking environments.

Combines local windowed attention with linear (kernel-based) global attention, achieving near-linear training complexity O(L) instead of O(L²). Inspired by SpikingBrain’s hybrid attention architecture.

The neuron accumulates spike-weighted keys and values via a recurrent state S, avoiding the quadratic attention matrix:

S(t+1) = λ S(t) + φ(k_t) ⊗ v_t output = φ(q_t)ᵀ S(t)

where φ is an elu+1 feature map.

Fields

v: f64

state_kv: Vec<f64>

dim: usize

lambda: f64

window_size: usize

window_buf: Vec<f64>

window_idx: usize

dt: f64

Implementations

impl HybridLinearAttentionNeuron

pub fn new(dim: usize) -> Self

pub fn step_qkv(&mut self, query: f64, key: f64, value: f64) -> f64

Step with query, key, value (each scalar projections).

pub fn step(&mut self, current: f64) -> i32

Simple step (input treated as combined qkv).

pub fn reset(&mut self)

Trait Implementations

impl Clone for HybridLinearAttentionNeuron

fn clone(&self) -> HybridLinearAttentionNeuron

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HybridLinearAttentionNeuron

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for HybridLinearAttentionNeuron

fn default() -> Self

Returns the “default value” for a type.