Struct StochasticAttention 

pub struct StochasticAttention {
    pub dim_k: usize,
    pub temperature: f64,
}

Fields

dim_k: usize

temperature: f64

Softmax temperature. Default: sqrt(dim_k) (Vaswani et al. 2017).

Implementations

impl StochasticAttention

pub fn new(dim_k: usize) -> Self

pub fn with_temperature(dim_k: usize, temperature: f64) -> Self

pub fn forward_softmax( &self, q: &[f64], q_rows: usize, q_cols: usize, k: &[f64], k_rows: usize, k_cols: usize, v: &[f64], v_rows: usize, v_cols: usize, ) -> Result<Vec<f64>, String>

Softmax attention: Q·K^T / temperature → softmax → · V.

Numerically stable: subtract row max before exp().

pub fn forward_multihead_softmax( &self, q: &[f64], q_rows: usize, q_total_cols: usize, k: &[f64], k_rows: usize, k_total_cols: usize, v: &[f64], v_rows: usize, v_total_cols: usize, n_heads: usize, ) -> Result<Vec<f64>, String>

Multi-head softmax attention.

pub fn forward( &self, q: &[f64], q_rows: usize, q_cols: usize, k: &[f64], k_rows: usize, k_cols: usize, v: &[f64], v_rows: usize, v_cols: usize, ) -> Result<Vec<f64>, String>

Rate-mode attention forward pass.

Shapes:

• q: (q_rows, q_cols)
• k: (k_rows, k_cols)
• v: (v_rows, v_cols)

pub fn forward_sc( &self, q: &[f64], q_rows: usize, q_cols: usize, k: &[f64], k_rows: usize, k_cols: usize, v: &[f64], v_rows: usize, v_cols: usize, length: usize, seed: u64, ) -> Result<Vec<f64>, String>

SC-mode attention forward pass with Bernoulli bitstream encoding.

pub fn forward_multihead( &self, q: &[f64], q_rows: usize, q_total_cols: usize, k: &[f64], k_rows: usize, k_total_cols: usize, v: &[f64], v_rows: usize, v_total_cols: usize, n_heads: usize, ) -> Result<Vec<f64>, String>

Multi-head linear attention (backwards compat).