Fork me on GitHub
Arraymancer Technical reference Tutorial Spellbook (How-To's) Under the hood

Module ag_data_structure

Types

Context[TT] = ref ContextObj[TT]
An autograd context is a record of operations or layers.
It holds the following fields:
  • nodes: This records the list of operations(Node) applied in the context
  • no_grad: This disable tracing the list of operations altogether. This is useful to save memory when you don't need the gradient (for validation or prediction for example)

A context is also called a tape or a Wengert list.

Note: backpropagation empties the list of operations.

  Source Edit
Variable[TT] = ref VariableObj[TT]
A variable is a wrapper for Tensors that tracks operations applied to it.
It consists of:
  • A weak reference to a record of operations context
  • The tensor being tracked value
  • The gradient of the tensor grad

Warning ⚠: Make sure the Context outlives the Variable. In the future grad will be optional: Option[TT] or opt[TT]

  Source Edit
Gate[TT] = ref object {.
inheritable
.} nb_grads*: int
Base operator or layer. You can describe your custom operations or layers by inheriting from Gate and add a forward and optionally a backward method. Each operations should set the number of gradients produced during backpropagation. Additional fields specific to the operations like weights or inputs cache should be added too.   Source Edit
Node[TT] = ref object
  gate*: Gate[TT]
  parents*: Parents[TT]
  payload*: Variable[TT]
A node consist of:
  • The description of the operator or layer (gate)
  • A weak reference to the parents VariableObj
  • The actual value of the node (payload)
  Source Edit
SmallDiffs[TT] = array[MAX_NB_GRADS, TT]
  Source Edit

Procs

proc newContext(TT: typedesc): Context[TT] {.
noSideEffect
.}
Initialize a context   Source Edit
proc variable[TT](ctx: Context[TT]; value: TT; requires_grad = false): Variable[TT] {.
noSideEffect
.}
Wrap a variable to the context T is a Tensor[T, CudaTensor[T] or scalar T   Source Edit
proc weakRef[TT](v: Variable[TT]): VariablePtr[TT] {.
inline
.}
Get a weak/untraced reference to a Variable This is intended for library writers and Neural Network graphs to avoid strong cyclic references.   Source Edit
proc weakRef[TT](ctx: Context[TT]): ContextPtr[TT] {.
inline
.}
Get a weak/untraced reference to a Variable This is intended for library writers and Neural Network graphs to avoid strong cyclic references.   Source Edit
proc push[TT](ctx: ContextPtr[TT]; node: Node[TT]) {.
noSideEffect, inline
.}
Append a new operation to the context   Source Edit
proc is_grad_needed(v: Variable): bool {.
noSideEffect, inline
.}
Depending on the input variable and its context no_grad_mode, returns true if gradient computation is needed and false otherwise   Source Edit
proc check_ctx(a, b: Variable) {.
noSideEffect, inline
.}
  Source Edit
proc backprop[TT](v: Variable[TT])
Differentiate the chain of operations w.r.t to this variable. Context will be reset   Source Edit

Methods

method backward[TT](self: Gate[TT]; gradient: TT): SmallDiffs[TT] {.
noInit, base, inline
.}
  Source Edit
method forward[TT](self: Gate[TT]; a, b: Variable[TT]): Variable[TT] {.
base, inline
.}
  Source Edit
method forward[TT](self: Gate[TT]; a: Variable[TT]): Variable[TT] {.
base, inline
.}
  Source Edit

Templates

template no_grad_mode(ctx: Context; body: untyped): untyped

Within this block, the context will not track the operations applied to each Variable.

This should be used for validation or prediction to optimize memory.

  Source Edit