Fork me on GitHub
Arraymancer Technical reference Tutorial Spellbook (How-To's) Under the hood

Module linear

Types

LinearGate[TT] = ref object of Gate[TT]
  input, weight, bias: Variable[TT]
TODO: use fused AddMatMul gate: C <- alpha AB + beta C   Source Edit

Procs

proc linear[TT](input, weight: Variable[TT]; bias: Variable[TT] = nil): Variable[TT]
Input:
  • A x Variable of shape [batch_size, in_features]
  • A weight Variable of shape [out_features, in_features]
  • Optionally a bias Variable of shape [1, out_features]
Return:
  • Weight * x + bias
Future TODO:
In the future the linear layer will allow different input layout so that x can also be of shape [batch_size, in_features]
Warning ⚠:
  • Experimental, there is no tests yet for this layer
  Source Edit

Methods

method forward[TT](self: LinearGate[TT]; input: Variable[TT]): Variable[TT] {.
inline, locks: 0
.}
  Source Edit
method backward[TT](self: LinearGate[TT]; gradOutput: TT): SmallDiffs[TT] {.
noInit, inline, locks: 0
.}
  Source Edit