Procs
proc numerical_gradient[T: not Tensor](input: T; f: (proc (x: T): T); h = T(0.00001)): T {.inline.}
- Compute numerical gradient for any function w.r.t. to an input value, useful for gradient checking, recommend using float64 types to assure numerical precision. The gradient is calculated as: (f(x + h) - f(x - h)) / (2*h) where h is a small number, typically 1e-5. Source Edit
proc numerical_gradient[T](input: Tensor[T]; f: (proc (x: Tensor[T]): T); h = T(0.00001)): Tensor[T] {.noinit.}
- Compute numerical gradient for any function w.r.t. to an input Tensor, useful for gradient checking, recommend using float64 types to assure numerical precision. The gradient is calculated as: (f(x + h) - f(x - h)) / (2*h) where h is a small number, typically 1e-5 f(x) will be called for each input elements with +h and -h pertubation. Source Edit