Overview¶

This document lists the routines of the n-dimensional array package:

The NDArray API, defined in the ndarray (or simply nd) package, provides imperative tensor operations on CPU/GPU. An NDArray represents a multi-dimensional, fixed-size homogenous array.

>>> x = mx.nd.array([[1, 2, 3], [4, 5, 6]])
>>> type(x)

>>> x.shape
(2, 3)
>>> y = x + mx.nd.ones(x.shape)*3
>>> print(y.asnumpy())
[[ 4.  5.  6.]
 [ 7.  8.  9.]]
>>> z = y.as_in_context(mx.gpu(0))
>>> print(z)

A detailed tutorial is available at NDArray - Imperative tensor operations on CPU/GPU.

In the rest of this document, we first overview the methods provided by the ndarray.NDArray class, and then list other routines provided by the ndarray package.

The ndarray package provides several classes:

The NDArray class¶

Basic neural network functions¶

Array creation routines¶

Array manipulation routines¶

Mathematical functions¶

Neural network¶

API Reference¶

class mxnet.ndarray.NDArray(handle, writable=True)[source]¶

An array object representing a multidimensional, homogeneous array of fixed-size items.

__repr__()[source]¶

Returns a string representation of the array.

__add__(other)[source]¶

x.__add__(y) <=> x+y <=> mx.nd.add(x, y)

__iadd__(other)[source]¶

x.__iadd__(y) <=> x+=y

__sub__(other)[source]¶

x.__sub__(y) <=> x-y <=> mx.nd.subtract(x, y)

__isub__(other)[source]¶

x.__isub__(y) <=> x-=y

__rsub__(other)[source]¶

x.__rsub__(y) <=> y-x <=> mx.nd.subtract(y, x)

__mul__(other)[source]¶

x.__mul__(y) <=> x*y <=> mx.nd.multiply(x, y)

__neg__()[source]¶

x.__neg__(y) <=> -x

__imul__(other)[source]¶

x.__imul__(y) <=> x*=y

__div__(other)[source]¶

x.__div__(y) <=> x/y <=> mx.nd.divide(x, y)

__rdiv__(other)[source]¶

x.__rdiv__(y) <=> y/x <=> mx.nd.divide(y, x)

__idiv__(other)[source]¶

x.__rdiv__(y) <=> x/=y

__mod__(other)[source]¶

x.__mod__(y) <=> x%y <=> mx.nd.modulo(x, y)

__rmod__(other)[source]¶

x.__rmod__(y) <=> y%x <=> mx.nd.modulo(y, x)

__imod__(other)[source]¶

x.__rmod__(y) <=> x%=y

__pow__(other)[source]¶

x.__pow__(y) <=> x**y <=> mx.nd.power(x,y)

__rpow__(other)[source]¶

x.__pow__(y) <=> y**x <=> mx.nd.power(y,x)

__eq__(other)[source]¶

x.__eq__(y) <=> x==y <=> mx.nd.equal(x, y)

__ne__(other)[source]¶

x.__ne__(y) <=> x!=y <=> mx.nd.not_equal(x, y)

__gt__(other)[source]¶

x.__gt__(y) <=> x>y <=> mx.nd.greater(x, y)

__ge__(other)[source]¶

x.__ge__(y) <=> x>=y <=> mx.nd.greater_equal(x, y)

__lt__(other)[source]¶

x.__lt__(y) <=> x mx.nd.lesser(x, y)

__le__(other)[source]¶

x.__le__(y) <=> x<=y <=> mx.nd.less_equal(x, y)

__len__()[source]¶

Number of element along the first axis.

__setitem__(key, value)[source]¶

x.__setitem__(i, y) <=> x[i]=y

Set self[key] to value.

Parameters:	key (int, slice or tuple) – The indexing key. value (scalar, NDArray or numpy.ndarray) – The value to set.

Examples

>>> x = mx.nd.zeros((2,3))
>>> x[:] = 1
>>> x.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> x.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> x[:,1:2] = 2
>>> x.asnumpy()
array([[ 1.,  2.,  1.],
       [ 1.,  2.,  1.]], dtype=float32)
>>> x[1:2,1:] = 3
>>> x.asnumpy()
array([[ 1.,  2.,  1.],
       [ 1.,  3.,  3.]], dtype=float32)
>>> x[1:,0:2] = mx.nd.zeros((1,2))
>>> x.asnumpy()
array([[ 1.,  2.,  1.],
       [ 0.,  0.,  3.]], dtype=float32)
>>> x[1,2] = 4
>>> x.asnumpy()
array([[ 1.,  2.,  1.],
       [ 0.,  0.,  4.]], dtype=float32)

__getitem__(key)[source]¶

x.__getitem__(i) <=> x[i]

Returns a sliced view of this array.

Parameters:	key (int or slice, or array like) – Indexing key.

Examples

>>> x = mx.nd.arange(0,6).reshape((2,3))
>>> x.asnumpy()
array([[ 0.,  1.,  2.],
       [ 3.,  4.,  5.]], dtype=float32)
>>> x[1].asnumpy()
array([ 3.,  4.,  5.], dtype=float32)
>>> y = x[0:1]
>>> y[:] = 2
>>> x.asnumpy()
array([[ 2.,  2.,  2.],
       [ 3.,  4.,  5.]], dtype=float32)

reshape(shape)[source]¶

Returns a view of this array with a new shape without altering any data.

Parameters:	shape (tuple of int) – The new shape should not change the array size, namely np.prod(new_shape) should be equal to np.prod(self.shape). One dimension can be -1. In this case, the value is inferred from the length of the array and remaining dimensions. 0 Dimensions in shape will be copied from original shape, i.e. if x.shape == (3, 4, 5), x.reshape((0, 20)).shape will be (3, 20).
Returns:	An array with desired shape that shares data with this array.
Return type:	NDArray

Examples

>>> x = mx.nd.arange(0,6).reshape((2,3))
>>> x.asnumpy()
array([[ 0.,  1.,  2.],
       [ 3.,  4.,  5.]], dtype=float32)
>>> y = x.reshape((3,2))
>>> y.asnumpy()
array([[ 0.,  1.],
       [ 2.,  3.],
       [ 4.,  5.]], dtype=float32)
>>> y = x.reshape((3,-1))
>>> y.asnumpy()
array([[ 0.,  1.],
       [ 2.,  3.],
       [ 4.,  5.]], dtype=float32)
>>> y[:] = -1
>>> x.asnumpy()
array([[-1., -1., -1.],
       [-1., -1., -1.]], dtype=float32)

reshape_like(*args, **kwargs)[source]¶

Convenience fluent method for reshape_like().

The arguments are the same as for reshape_like(), with this array as data.

zeros_like(*args, **kwargs)[source]¶

Convenience fluent method for zeros_like().

The arguments are the same as for zeros_like(), with this array as data.

ones_like(*args, **kwargs)[source]¶

Convenience fluent method for ones_like().

The arguments are the same as for ones_like(), with this array as data.

broadcast_axes(*args, **kwargs)[source]¶

Convenience fluent method for broadcast_axes().

The arguments are the same as for broadcast_axes(), with this array as data.

repeat(*args, **kwargs)[source]¶

Convenience fluent method for repeat().

The arguments are the same as for repeat(), with this array as data.

pad(*args, **kwargs)[source]¶

Convenience fluent method for pad().

The arguments are the same as for pad(), with this array as data.

swapaxes(*args, **kwargs)[source]¶

Convenience fluent method for swapaxes().

The arguments are the same as for swapaxes(), with this array as data.

split(*args, **kwargs)[source]¶

Convenience fluent method for split().

The arguments are the same as for split(), with this array as data.

slice(*args, **kwargs)[source]¶

Convenience fluent method for slice().

The arguments are the same as for slice(), with this array as data.

slice_axis(*args, **kwargs)[source]¶

Convenience fluent method for slice_axis().

The arguments are the same as for slice_axis(), with this array as data.

take(*args, **kwargs)[source]¶

Convenience fluent method for take().

The arguments are the same as for take(), with this array as data.

one_hot(*args, **kwargs)[source]¶

Convenience fluent method for one_hot().

The arguments are the same as for one_hot(), with this array as data.

pick(*args, **kwargs)[source]¶

Convenience fluent method for pick().

The arguments are the same as for pick(), with this array as data.

sort(*args, **kwargs)[source]¶

Convenience fluent method for sort().

The arguments are the same as for sort(), with this array as data.

topk(*args, **kwargs)[source]¶

Convenience fluent method for topk().

The arguments are the same as for topk(), with this array as data.

argsort(*args, **kwargs)[source]¶

Convenience fluent method for argsort().

The arguments are the same as for argsort(), with this array as data.

argmax(*args, **kwargs)[source]¶

Convenience fluent method for argmax().

The arguments are the same as for argmax(), with this array as data.

argmax_channel(*args, **kwargs)[source]¶

Convenience fluent method for argmax_channel().

The arguments are the same as for argmax_channel(), with this array as data.

argmin(*args, **kwargs)[source]¶

Convenience fluent method for argmin().

The arguments are the same as for argmin(), with this array as data.

clip(*args, **kwargs)[source]¶

Convenience fluent method for clip().

The arguments are the same as for clip(), with this array as data.

abs(*args, **kwargs)[source]¶

Convenience fluent method for abs().

The arguments are the same as for abs(), with this array as data.

sign(*args, **kwargs)[source]¶

Convenience fluent method for sign().

The arguments are the same as for sign(), with this array as data.

flatten(*args, **kwargs)[source]¶

Convenience fluent method for flatten().

The arguments are the same as for flatten(), with this array as data.

expand_dims(*args, **kwargs)[source]¶

Convenience fluent method for expand_dims().

The arguments are the same as for expand_dims(), with this array as data.

tile(*args, **kwargs)[source]¶

Convenience fluent method for tile().

The arguments are the same as for tile(), with this array as data.

transpose(*args, **kwargs)[source]¶

Convenience fluent method for transpose().

The arguments are the same as for transpose(), with this array as data.

flip(*args, **kwargs)[source]¶

Convenience fluent method for flip().

The arguments are the same as for flip(), with this array as data.

sum(*args, **kwargs)[source]¶

Convenience fluent method for sum().

The arguments are the same as for sum(), with this array as data.

nansum(*args, **kwargs)[source]¶

Convenience fluent method for nansum().

The arguments are the same as for nansum(), with this array as data.

prod(*args, **kwargs)[source]¶

Convenience fluent method for prod().

The arguments are the same as for prod(), with this array as data.

nanprod(*args, **kwargs)[source]¶

Convenience fluent method for nanprod().

The arguments are the same as for nanprod(), with this array as data.

mean(*args, **kwargs)[source]¶

Convenience fluent method for mean().

The arguments are the same as for mean(), with this array as data.

max(*args, **kwargs)[source]¶

Convenience fluent method for max().

The arguments are the same as for max(), with this array as data.

min(*args, **kwargs)[source]¶

Convenience fluent method for min().

The arguments are the same as for min(), with this array as data.

norm(*args, **kwargs)[source]¶

Convenience fluent method for norm().

The arguments are the same as for norm(), with this array as data.

round(*args, **kwargs)[source]¶

Convenience fluent method for round().

The arguments are the same as for round(), with this array as data.

rint(*args, **kwargs)[source]¶

Convenience fluent method for rint().

The arguments are the same as for rint(), with this array as data.

fix(*args, **kwargs)[source]¶

Convenience fluent method for fix().

The arguments are the same as for fix(), with this array as data.

floor(*args, **kwargs)[source]¶

Convenience fluent method for floor().

The arguments are the same as for floor(), with this array as data.

ceil(*args, **kwargs)[source]¶

Convenience fluent method for ceil().

The arguments are the same as for ceil(), with this array as data.

trunc(*args, **kwargs)[source]¶

Convenience fluent method for trunc().

The arguments are the same as for trunc(), with this array as data.

sin(*args, **kwargs)[source]¶

Convenience fluent method for sin().

The arguments are the same as for sin(), with this array as data.

cos(*args, **kwargs)[source]¶

Convenience fluent method for cos().

The arguments are the same as for cos(), with this array as data.

tan(*args, **kwargs)[source]¶

Convenience fluent method for tan().

The arguments are the same as for tan(), with this array as data.

arcsin(*args, **kwargs)[source]¶

Convenience fluent method for arcsin().

The arguments are the same as for arcsin(), with this array as data.

arccos(*args, **kwargs)[source]¶

Convenience fluent method for arccos().

The arguments are the same as for arccos(), with this array as data.

arctan(*args, **kwargs)[source]¶

Convenience fluent method for arctan().

The arguments are the same as for arctan(), with this array as data.

degrees(*args, **kwargs)[source]¶

Convenience fluent method for degrees().

The arguments are the same as for degrees(), with this array as data.

radians(*args, **kwargs)[source]¶

Convenience fluent method for radians().

The arguments are the same as for radians(), with this array as data.

sinh(*args, **kwargs)[source]¶

Convenience fluent method for sinh().

The arguments are the same as for sinh(), with this array as data.

cosh(*args, **kwargs)[source]¶

Convenience fluent method for cosh().

The arguments are the same as for cosh(), with this array as data.

tanh(*args, **kwargs)[source]¶

Convenience fluent method for tanh().

The arguments are the same as for tanh(), with this array as data.

arcsinh(*args, **kwargs)[source]¶

Convenience fluent method for arcsinh().

The arguments are the same as for arcsinh(), with this array as data.

arccosh(*args, **kwargs)[source]¶

Convenience fluent method for arccosh().

The arguments are the same as for arccosh(), with this array as data.

arctanh(*args, **kwargs)[source]¶

Convenience fluent method for arctanh().

The arguments are the same as for arctanh(), with this array as data.

exp(*args, **kwargs)[source]¶

Convenience fluent method for exp().

The arguments are the same as for exp(), with this array as data.

expm1(*args, **kwargs)[source]¶

Convenience fluent method for expm1().

The arguments are the same as for expm1(), with this array as data.

log(*args, **kwargs)[source]¶

Convenience fluent method for log().

The arguments are the same as for log(), with this array as data.

log10(*args, **kwargs)[source]¶

Convenience fluent method for log10().

The arguments are the same as for log10(), with this array as data.

log2(*args, **kwargs)[source]¶

Convenience fluent method for log2().

The arguments are the same as for log2(), with this array as data.

log1p(*args, **kwargs)[source]¶

Convenience fluent method for log1p().

The arguments are the same as for log1p(), with this array as data.

sqrt(*args, **kwargs)[source]¶

Convenience fluent method for sqrt().

The arguments are the same as for sqrt(), with this array as data.

rsqrt(*args, **kwargs)[source]¶

Convenience fluent method for rsqrt().

The arguments are the same as for rsqrt(), with this array as data.

cbrt(*args, **kwargs)[source]¶

Convenience fluent method for cbrt().

The arguments are the same as for cbrt(), with this array as data.

rcbrt(*args, **kwargs)[source]¶

Convenience fluent method for rcbrt().

The arguments are the same as for rcbrt(), with this array as data.

square(*args, **kwargs)[source]¶

Convenience fluent method for square().

The arguments are the same as for square(), with this array as data.

reciprocal(*args, **kwargs)[source]¶

Convenience fluent method for reciprocal().

The arguments are the same as for reciprocal(), with this array as data.

relu(*args, **kwargs)[source]¶

Convenience fluent method for relu().

The arguments are the same as for relu(), with this array as data.

sigmoid(*args, **kwargs)[source]¶

Convenience fluent method for sigmoid().

The arguments are the same as for sigmoid(), with this array as data.

softmax(*args, **kwargs)[source]¶

Convenience fluent method for softmax().

The arguments are the same as for softmax(), with this array as data.

log_softmax(*args, **kwargs)[source]¶

Convenience fluent method for log_softmax().

The arguments are the same as for log_softmax(), with this array as data.

broadcast_to(shape)[source]¶

Broadcasts the input array to a new shape.

Broadcasting is only allowed on axes with size 1. The new shape cannot change the number of dimensions. For example, you could broadcast from shape (2, 1) to (2, 3), but not from shape (2, 3) to (2, 3, 3).

Parameters:	shape (tuple of int) – The shape of the desired array.
Returns:	A NDArray with the desired shape that is not sharing data with this array, even if the new shape is the same as self.shape.
Return type:	NDArray

Examples

>>> x = mx.nd.arange(0,3).reshape((1,3,1))
>>> x.asnumpy()
array([[[ 0.],
        [ 1.],
        [ 2.]]], dtype=float32)
>>> y = x.broadcast_to((2,3,3))
>>> y.asnumpy()
array([[[ 0.,  0.,  0.],
        [ 1.,  1.,  1.],
        [ 2.,  2.,  2.]],

       [[ 0.,  0.,  0.],
        [ 1.,  1.,  1.],
        [ 2.,  2.,  2.]]], dtype=float32)

wait_to_read()[source]¶

Waits until all previous write operations on the current array are finished.

This method guarantees that all previous write operations that pushed into the backend engine for execution are actually finished.

Examples

>>> import time
>>> tic = time.time()
>>> a = mx.nd.ones((1000,1000))
>>> b = mx.nd.dot(a, a)
>>> print(time.time() - tic) 
0.003854036331176758
>>> b.wait_to_read()
>>> print(time.time() - tic) 
0.0893700122833252

ndim¶

Returns the number of dimensions of this array

Examples

>>> x = mx.nd.array([1, 2, 3, 4])
>>> x.ndim
1
>>> x = mx.nd.array([[1, 2], [3, 4]])
>>> x.ndim
2

shape¶

Tuple of array dimensions.

Examples

>>> x = mx.nd.array([1, 2, 3, 4])
>>> x.shape
(4L,)
>>> y = mx.nd.zeros((2, 3, 4))
>>> y.shape
(2L, 3L, 4L)

size¶

Number of elements in the array.

Equivalent to the product of the array’s dimensions.

Examples

>>> import numpy as np
>>> x = mx.nd.zeros((3, 5, 2))
>>> x.size
30
>>> np.prod(x.shape)
30

context¶

Device context of the array.

Examples

>>> x = mx.nd.array([1, 2, 3, 4])
>>> x.context
cpu(0)
>>> type(x.context)

>>> y = mx.nd.zeros((2,3), mx.gpu(0))
>>> y.context
gpu(0)

dtype¶

Data-type of the array’s elements.

Returns:	This NDArray’s data type.
Return type:	numpy.dtype

Examples

>>> x = mx.nd.zeros((2,3))
>>> x.dtype

>>> y = mx.nd.zeros((2,3), dtype='int32')
>>> y.dtype

stype¶

Storage-type of the array.

T¶

Returns a copy of the array with axes transposed.

Equivalent to mx.nd.transpose(self) except that self is returned if self.ndim < 2.

Unlike numpy.ndarray.T, this function returns a copy rather than a view of the array unless self.ndim < 2.

Examples

>>> x = mx.nd.arange(0,6).reshape((2,3))
>>> x.asnumpy()
array([[ 0.,  1.,  2.],
       [ 3.,  4.,  5.]], dtype=float32)
>>> x.T.asnumpy()
array([[ 0.,  3.],
       [ 1.,  4.],
       [ 2.,  5.]], dtype=float32)

asnumpy()[source]¶

Returns a numpy.ndarray object with value copied from this array.

Examples

>>> x = mx.nd.ones((2,3))
>>> y = x.asnumpy()
>>> type(y)

>>> y
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> z = mx.nd.ones((2,3), dtype='int32')
>>> z.asnumpy()
array([[1, 1, 1],
       [1, 1, 1]], dtype=int32)

asscalar()[source]¶

Returns a scalar whose value is copied from this array.

This function is equivalent to self.asnumpy()[0]. This NDArray must have shape (1,).

Examples

>>> x = mx.nd.ones((1,), dtype='int32')
>>> x.asscalar()
1
>>> type(x.asscalar())

astype(dtype)[source]¶

Returns a copy of the array after casting to a specified type.

Parameters:	dtype (numpy.dtype or str) – The type of the returned array.
Returns:	The copied array after casting to the specified type.
Return type:	NDArray, CSRNDArray or RowSparseNDArray

Examples

>>> x = mx.nd.zeros((2,3), dtype='float32')
>>> y = x.astype('int32')
>>> y.dtype

copyto(other)[source]¶

Copies the value of this array to another array.

If other is a NDArray object, then other.shape and self.shape should be the same. This function copies the value from self to other.

If other is a context, a new NDArray will be first created on the target context, and the value of self is copied.

Parameters:	other (NDArray or Context) – The destination array or context.
Returns:	The copied array. If other is an NDArray, then the return value and other will point to the same NDArray.
Return type:	NDArray, CSRNDArray or RowSparseNDArray

Examples

>>> x = mx.nd.ones((2,3))
>>> y = mx.nd.zeros((2,3), mx.gpu(0))
>>> z = x.copyto(y)
>>> z is y
True
>>> y.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> y.copyto(mx.gpu(0))

copy()[source]¶

Makes a copy of this NDArray, keeping the same context.

Returns:	The copied array
Return type:	NDArray, CSRNDArray or RowSparseNDArray

Examples

>>> x = mx.nd.ones((2,3))
>>> y = x.copy()
>>> y.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)

as_in_context(context)[source]¶

Returns an array on the target device with the same value as this array.

If the target context is the same as self.context, then self is returned. Otherwise, a copy is made.

Parameters:	context (Context) – The target context.
Returns:	The target array.
Return type:	NDArray, CSRNDArray or RowSparseNDArray

Examples

>>> x = mx.nd.ones((2,3))
>>> y = x.as_in_context(mx.cpu())
>>> y is x
True
>>> z = x.as_in_context(mx.gpu(0))
>>> z is x
False

attach_grad(grad_req='write', stype=None)[source]¶

Attach a gradient buffer to this NDArray, so that backward can compute gradient with respect to it.

Parameters:	grad_req ({'write', 'add', 'null'}) – How gradient will be accumulated. - ‘write’: gradient will be overwritten on every backward. - ‘add’: gradient will be added to existing value on every backward. - ‘null’: do not compute gradient for this NDArray. stype (str, optional) – The storage type of the gradient array. Defaults to the same stype of this NDArray.

grad¶

Returns gradient buffer attached to this NDArray.

detach()[source]¶

Returns a new NDArray, detached from the current graph.

backward(out_grad=None, retain_graph=False, train_mode=True)[source]¶

Compute the gradients of this NDArray w.r.t variables.

Parameters:	out_grad (NDArray, optional) – Gradient with respect to head. retain_graph (bool, optional) – Whether to retain the computaion graph for another backward pass on the same graph. By default the computaion history is cleared. train_mode (bool, optional) – Whether to compute gradient for training or inference.

tostype(stype)[source]¶

Return a copy of the array with chosen storage type.

See also

mxnet.ndarray.cast_storage()

Returns:	A copy of the array with the chosen storage stype
Return type:	NDArray, CSRNDArray or RowSparseNDArray

NDArray API of MXNet.

mxnet.ndarray.Activation(data=None, act_type=_Null, out=None, name=None, **kwargs)¶

Applies an activation function element-wise to the input.

The following activation functions are supported:

• relu: Rectified Linear Unit, \(y = max(x, 0)\)
• sigmoid: \(y = \frac{1}{1 + exp(-x)}\)
• tanh: Hyperbolic tangent, \(y = \frac{exp(x) - exp(-x)}{exp(x) + exp(-x)}\)
• softrelu: Soft ReLU, or SoftPlus, \(y = log(1 + exp(x))\)

Defined in src/operator/activation.cc:L91

Parameters:	data (NDArray) – Input array to activation function. act_type ({'relu', 'sigmoid', 'softrelu', 'tanh'}, required) – Activation function to be applied. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.BatchNorm(data=None, gamma=None, beta=None, moving_mean=None, moving_var=None, eps=_Null, momentum=_Null, fix_gamma=_Null, use_global_stats=_Null, output_mean_var=_Null, axis=_Null, cudnn_off=_Null, out=None, name=None, **kwargs)¶

Batch normalization.

Normalizes a data batch by mean and variance, and applies a scale gamma as well as offset beta.

Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:

\[\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}\]

Then compute the normalized output, which has the same shape as input, as following:

\[out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]\]

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,). If output_mean_var is set to be true, then outputs both data_mean and data_var as well, which are needed for the backward pass.

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum)
moving_var = moving_var * momentum + data_var * (1 - momentum)

If use_global_stats is set to be true, then moving_mean and moving_var are used instead of data_mean and data_var to compute the output. It is often used during inference.

The parameter axis specifies which axis of the input shape denotes the ‘channel’ (separately normalized groups). The default is 1. Specifying -1 sets the channel axis to be the last item in the input shape.

Both gamma and beta are learnable parameters. But if fix_gamma is true, then set gamma to 1 and its gradient to 0.

Defined in src/operator/batch_norm.cc:L399

Parameters:	data (NDArray) – Input data to batch normalization gamma (NDArray) – gamma array beta (NDArray) – beta array moving_mean (NDArray) – running mean of input moving_var (NDArray) – running variance of input eps (double, optional, default=0.001) – Epsilon to prevent div 0. Must be no less than CUDNN_BN_MIN_EPSILON defined in cudnn.h when using cudnn (usually 1e-5) momentum (float, optional, default=0.9) – Momentum for moving average fix_gamma (boolean, optional, default=1) – Fix gamma while training use_global_stats (boolean, optional, default=0) – Whether use global moving statistics instead of local batch-norm. This will force change batch-norm into a scale shift operator. output_mean_var (boolean, optional, default=0) – Output All,normal mean and var axis (int, optional, default='1') – Specify which shape axis the channel is specified cudnn_off (boolean, optional, default=0) – Do not select CUDNN operator, if available out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.BatchNorm_v1(data=None, gamma=None, beta=None, eps=_Null, momentum=_Null, fix_gamma=_Null, use_global_stats=_Null, output_mean_var=_Null, out=None, name=None, **kwargs)¶

Batch normalization.

Normalizes a data batch by mean and variance, and applies a scale gamma as well as offset beta.

Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:

\[\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}\]

Then compute the normalized output, which has the same shape as input, as following:

\[out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]\]

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,). If output_mean_var is set to be true, then outputs both data_mean and data_var as well, which are needed for the backward pass.

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum)
moving_var = moving_var * momentum + data_var * (1 - momentum)

If use_global_stats is set to be true, then moving_mean and moving_var are used instead of data_mean and data_var to compute the output. It is often used during inference.

Both gamma and beta are learnable parameters. But if fix_gamma is true, then set gamma to 1 and its gradient to 0.

Defined in src/operator/batch_norm_v1.cc:L89

Parameters:	data (NDArray) – Input data to batch normalization gamma (NDArray) – gamma array beta (NDArray) – beta array eps (float, optional, default=0.001) – Epsilon to prevent div 0 momentum (float, optional, default=0.9) – Momentum for moving average fix_gamma (boolean, optional, default=1) – Fix gamma while training use_global_stats (boolean, optional, default=0) – Whether use global moving statistics instead of local batch-norm. This will force change batch-norm into a scale shift operator. output_mean_var (boolean, optional, default=0) – Output All,normal mean and var out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.BilinearSampler(data=None, grid=None, out=None, name=None, **kwargs)¶

Applies bilinear sampling to input feature map.

Bilinear Sampling is the key of [NIPS2015] “Spatial Transformer Networks”. The usage of the operator is very similar to remap function in OpenCV, except that the operator has the backward pass.

Given \(data\) and \(grid\), then the output is computed by

\[\begin{split}x_{src} = grid[batch, 0, y_{dst}, x_{dst}] \\ y_{src} = grid[batch, 1, y_{dst}, x_{dst}] \\ output[batch, channel, y_{dst}, x_{dst}] = G(data[batch, channel, y_{src}, x_{src})\end{split}\]

\(x_{dst}\), \(y_{dst}\) enumerate all spatial locations in \(output\), and \(G()\) denotes the bilinear interpolation kernel. The out-boundary points will be padded with zeros.The shape of the output will be (data.shape[0], data.shape[1], grid.shape[2], grid.shape[3]).

The operator assumes that \(data\) has ‘NCHW’ layout and \(grid\) has been normalized to [-1, 1].

BilinearSampler often cooperates with GridGenerator which generates sampling grids for BilinearSampler. GridGenerator supports two kinds of transformation: affine and warp. If users want to design a CustomOp to manipulate \(grid\), please firstly refer to the code of GridGenerator.

Example 1:

## Zoom out data two times
data = array([[[[1, 4, 3, 6],
                [1, 8, 8, 9],
                [0, 4, 1, 5],
                [1, 0, 1, 3]]]])

affine_matrix = array([[2, 0, 0],
                       [0, 2, 0]])

affine_matrix = reshape(affine_matrix, shape=(1, 6))

grid = GridGenerator(data=affine_matrix, transform_type='affine', target_shape=(4, 4))

out = BilinearSampler(data, grid)

out
[[[[ 0,   0,     0,   0],
   [ 0,   3.5,   6.5, 0],
   [ 0,   1.25,  2.5, 0],
   [ 0,   0,     0,   0]]]

Example 2:

## shift data horizontally by -1 pixel

data = array([[[[1, 4, 3, 6],
                [1, 8, 8, 9],
                [0, 4, 1, 5],
                [1, 0, 1, 3]]]])

warp_maxtrix = array([[[[1, 1, 1, 1],
                        [1, 1, 1, 1],
                        [1, 1, 1, 1],
                        [1, 1, 1, 1]],
                       [[0, 0, 0, 0],
                        [0, 0, 0, 0],
                        [0, 0, 0, 0],
                        [0, 0, 0, 0]]]])

grid = GridGenerator(data=warp_matrix, transform_type='warp')
out = BilinearSampler(data, grid)

out
[[[[ 4,  3,  6,  0],
   [ 8,  8,  9,  0],
   [ 4,  1,  5,  0],
   [ 0,  1,  3,  0]]]

Defined in src/operator/bilinear_sampler.cc:L244

Parameters:	data (NDArray) – Input data to the BilinearsamplerOp. grid (NDArray) – Input grid to the BilinearsamplerOp.grid has two channels: x_src, y_src out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.BlockGrad(data=None, out=None, name=None, **kwargs)¶

Stops gradient computation.

Stops the accumulated gradient of the inputs from flowing through this operator in the backward direction. In other words, this operator prevents the contribution of its inputs to be taken into account for computing gradients.

Example:

v1 = [1, 2]
v2 = [0, 1]
a = Variable('a')
b = Variable('b')
b_stop_grad = stop_gradient(3 * b)
loss = MakeLoss(b_stop_grad + a)

executor = loss.simple_bind(ctx=cpu(), a=(1,2), b=(1,2))
executor.forward(is_train=True, a=v1, b=v2)
executor.outputs
[ 1.  5.]

executor.backward()
executor.grad_arrays
[ 0.  0.]
[ 1.  1.]

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L168

Parameters:	data (NDArray) – The input array. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.Cast(data=None, dtype=_Null, out=None, name=None, **kwargs)¶

Casts all elements of the input to a new type.

Note

Cast is deprecated. Use cast instead.

Example:

cast([0.9, 1.3], dtype='int32') = [0, 1]
cast([1e20, 11.1], dtype='float16') = [inf, 11.09375]
cast([300, 11.1, 10.9, -1, -3], dtype='uint8') = [44, 11, 10, 255, 253]

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L312

Parameters:	data (NDArray) – The input. dtype ({'float16', 'float32', 'float64', 'int32', 'uint8'}, required) – Output data type. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.Concat(*data, **kwargs)¶

Joins input arrays along a given axis.

Note

Concat is deprecated. Use concat instead.

The dimensions of the input arrays should be the same except the axis along which they will be concatenated. The dimension of the output array along the concatenated axis will be equal to the sum of the corresponding dimensions of the input arrays.

Example:

x = [[1,1],[2,2]]
y = [[3,3],[4,4],[5,5]]
z = [[6,6], [7,7],[8,8]]

concat(x,y,z,dim=0) = [[ 1.,  1.],
                       [ 2.,  2.],
                       [ 3.,  3.],
                       [ 4.,  4.],
                       [ 5.,  5.],
                       [ 6.,  6.],
                       [ 7.,  7.],
                       [ 8.,  8.]]

Note that you cannot concat x,y,z along dimension 1 since dimension
0 is not the same for all the input arrays.

concat(y,z,dim=1) = [[ 3.,  3.,  6.,  6.],
                      [ 4.,  4.,  7.,  7.],
                      [ 5.,  5.,  8.,  8.]]

Defined in src/operator/concat.cc:L103

Parameters:	data (NDArray[]) – List of arrays to concatenate dim (int, optional, default='1') – the dimension to be concated. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.Convolution(data=None, weight=None, bias=None, kernel=_Null, stride=_Null, dilate=_Null, pad=_Null, num_filter=_Null, num_group=_Null, workspace=_Null, no_bias=_Null, cudnn_tune=_Null, cudnn_off=_Null, layout=_Null, out=None, name=None, **kwargs)¶

Compute N-D convolution on (N+2)-D input.

In the 2-D convolution, given input data with shape (batch_size, channel, height, width), the output is computed by

\[out[n,i,:,:] = bias[i] + \sum_{j=0}^{channel} data[n,j,:,:] \star weight[i,j,:,:]\]

where \(\star\) is the 2-D cross-correlation operator.

For general 2-D convolution, the shapes are

• data: (batch_size, channel, height, width)
• weight: (num_filter, channel, kernel[0], kernel[1])
• bias: (num_filter,)
• out: (batch_size, num_filter, out_height, out_width).

Define:

f(x,k,p,s,d) = floor((x+2*p-d*(k-1)-1)/s)+1

then we have:

out_height=f(height, kernel[0], pad[0], stride[0], dilate[0])
out_width=f(width, kernel[1], pad[1], stride[1], dilate[1])

If no_bias is set to be true, then the bias term is ignored.

The default data layout is NCHW, namely (batch_size, channel, height, width). We can choose other layouts such as NHWC.

If num_group is larger than 1, denoted by g, then split the input data evenly into g parts along the channel axis, and also evenly split weight along the first dimension. Next compute the convolution on the i-th part of the data with the i-th weight part. The output is obtained by concatenating all the g results.

1-D convolution does not have height dimension but only width in space.

• data: (batch_size, channel, width)
• weight: (num_filter, channel, kernel[0])
• bias: (num_filter,)
• out: (batch_size, num_filter, out_width).

3-D convolution adds an additional depth dimension besides height and width. The shapes are

• data: (batch_size, channel, depth, height, width)
• weight: (num_filter, channel, kernel[0], kernel[1], kernel[2])
• bias: (num_filter,)
• out: (batch_size, num_filter, out_depth, out_height, out_width).

Both weight and bias are learnable parameters.

There are other options to tune the performance.

• cudnn_tune: enable this option leads to higher startup time but may give faster speed. Options are
  • off: no tuning
  • limited_workspace:run test and pick the fastest algorithm that doesn’t exceed workspace limit.
  • fastest: pick the fastest algorithm and ignore workspace limit.
  • None (default): the behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT. 0 for off, 1 for limited workspace (default), 2 for fastest.
• workspace: A large number leads to more (GPU) memory usage but may improve the performance.

Defined in src/operator/convolution.cc:L169

Parameters:	data (NDArray) – Input data to the ConvolutionOp. weight (NDArray) – Weight matrix. bias (NDArray) – Bias parameter. kernel (Shape(tuple), required) – convolution kernel size: (h, w) or (d, h, w) stride (Shape(tuple), optional, default=()) – convolution stride: (h, w) or (d, h, w) dilate (Shape(tuple), optional, default=()) – convolution dilate: (h, w) or (d, h, w) pad (Shape(tuple), optional, default=()) – pad for convolution: (h, w) or (d, h, w) num_filter (int (non-negative), required) – convolution filter(channel) number num_group (int (non-negative), optional, default=1) – Number of group partitions. workspace (long (non-negative), optional, default=1024) – Maximum temporary workspace allowed for convolution (MB). no_bias (boolean, optional, default=0) – Whether to disable bias parameter. cudnn_tune ({None, 'fastest', 'limited_workspace', 'off'},optional, default='None') – Whether to pick convolution algo by running performance test. cudnn_off (boolean, optional, default=0) – Turn off cudnn for this layer. layout ({None, 'NCDHW', 'NCHW', 'NCW', 'NDHWC', 'NHWC'},optional, default='None') – Set layout for input, output and weight. Empty for default layout: NCW for 1d, NCHW for 2d and NCDHW for 3d. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.Convolution_v1(data=None, weight=None, bias=None, kernel=_Null, stride=_Null, dilate=_Null, pad=_Null, num_filter=_Null, num_group=_Null, workspace=_Null, no_bias=_Null, cudnn_tune=_Null, cudnn_off=_Null, layout=_Null, out=None, name=None, **kwargs)¶

This operator is DEPRECATED. Apply convolution to input then add a bias.

Parameters:	data (NDArray) – Input data to the ConvolutionV1Op. weight (NDArray) – Weight matrix. bias (NDArray) – Bias parameter. kernel (Shape(tuple), required) – convolution kernel size: (h, w) or (d, h, w) stride (Shape(tuple), optional, default=()) – convolution stride: (h, w) or (d, h, w) dilate (Shape(tuple), optional, default=()) – convolution dilate: (h, w) or (d, h, w) pad (Shape(tuple), optional, default=()) – pad for convolution: (h, w) or (d, h, w) num_filter (int (non-negative), required) – convolution filter(channel) number num_group (int (non-negative), optional, default=1) – Number of group partitions. Equivalent to slicing input into num_group partitions, apply convolution on each, then concatenate the results workspace (long (non-negative), optional, default=1024) – Maximum tmp workspace allowed for convolution (MB). no_bias (boolean, optional, default=0) – Whether to disable bias parameter. cudnn_tune ({None, 'fastest', 'limited_workspace', 'off'},optional, default='None') – Whether to pick convolution algo by running performance test. Leads to higher startup time but may give faster speed. Options are: ‘off’: no tuning ‘limited_workspace’: run test and pick the fastest algorithm that doesn’t exceed workspace limit. ‘fastest’: pick the fastest algorithm and ignore workspace limit. If set to None (default), behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT: 0 for off, 1 for limited workspace (default), 2 for fastest. cudnn_off (boolean, optional, default=0) – Turn off cudnn for this layer. layout ({None, 'NCDHW', 'NCHW', 'NDHWC', 'NHWC'},optional, default='None') – Set layout for input, output and weight. Empty for default layout: NCHW for 2d and NCDHW for 3d. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.Correlation(data1=None, data2=None, kernel_size=_Null, max_displacement=_Null, stride1=_Null, stride2=_Null, pad_size=_Null, is_multiply=_Null, out=None, name=None, **kwargs)¶

Applies correlation to inputs.

The correlation layer performs multiplicative patch comparisons between two feature maps.

Given two multi-channel feature maps \(f_{1}, f_{2}\), with \(w\), \(h\), and \(c\) being their width, height, and number of channels, the correlation layer lets the network compare each patch from \(f_{1}\) with each patch from \(f_{2}\).

For now we consider only a single comparison of two patches. The ‘correlation’ of two patches centered at \(x_{1}\) in the first map and \(x_{2}\) in the second map is then defined as:

\[c(x_{1}, x_{2}) = \sum_{o \in [-k,k] \times [-k,k]} \]

for a square patch of size \(K:=2k+1\).

Note that the equation above is identical to one step of a convolution in neural networks, but instead of convolving data with a filter, it convolves data with other data. For this reason, it has no training weights.

Computing \(c(x_{1}, x_{2})\) involves \(c * K^{2}\) multiplications. Comparing all patch combinations involves \(w^{2}*h^{2}\) such computations.

Given a maximum displacement \(d\), for each location \(x_{1}\) it computes correlations \(c(x_{1}, x_{2})\) only in a neighborhood of size \(D:=2d+1\), by limiting the range of \(x_{2}\). We use strides \(s_{1}, s_{2}\), to quantize \(x_{1}\) globally and to quantize \(x_{2}\) within the neighborhood centered around \(x_{1}\).

The final output is defined by the following expression:

\[out[n, q, i, j] = c(x_{i, j}, x_{q})\]

where \(i\) and \(j\) enumerate spatial locations in \(f_{1}\), and \(q\) denotes the \(q^{th}\) neighborhood of \(x_{i,j}\).

Defined in src/operator/correlation.cc:L191

Parameters:	data1 (NDArray) – Input data1 to the correlation. data2 (NDArray) – Input data2 to the correlation. kernel_size (int (non-negative), optional, default=1) – kernel size for Correlation must be an odd number max_displacement (int (non-negative), optional, default=1) – Max displacement of Correlation stride1 (int (non-negative), optional, default=1) – stride1 quantize data1 globally stride2 (int (non-negative), optional, default=1) – stride2 quantize data2 within the neighborhood centered around data1 pad_size (int (non-negative), optional, default=0) – pad for Correlation is_multiply (boolean, optional, default=1) – operation type is either multiplication or subduction out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.Crop(*data, **kwargs)¶

Note

Crop is deprecated. Use slice instead.

Crop the 2nd and 3rd dim of input data, with the corresponding size of h_w or with width and height of the second input symbol, i.e., with one input, we need h_w to specify the crop height and width, otherwise the second input symbol’s size will be used

Defined in src/operator/crop.cc:L49

Parameters:	data (Symbol or Symbol[]) – Tensor or List of Tensors, the second input will be used as crop_like shape reference offset (Shape(tuple), optional, default=(0,0)) – crop offset coordinate: (y, x) h_w (Shape(tuple), optional, default=(0,0)) – crop height and width: (h, w) center_crop (boolean, optional, default=0) – If set to true, then it will use be the center_crop,or it will crop using the shape of crop_like out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.Custom(*data, **kwargs)¶

Apply a custom operator implemented in a frontend language (like Python).

Custom operators should override required methods like forward and backward. The custom operator must be registered before it can be used. Please check the tutorial here: /versions/0.12.1/how_to/new_op.html.

Defined in src/operator/custom/custom.cc:L369

Parameters:	data (NDArray[]) – Input data for the custom operator. op_type (string) – Name of the custom operator. This is the name that is passed to mx.operator.register to register the operator. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

Example

Applies a custom operator named my_custom_operator to input.

>>> output = mx.symbol.Custom(op_type='my_custom_operator', data=input)

mxnet.ndarray.Deconvolution(data=None, weight=None, bias=None, kernel=_Null, stride=_Null, dilate=_Null, pad=_Null, adj=_Null, target_shape=_Null, num_filter=_Null, num_group=_Null, workspace=_Null, no_bias=_Null, cudnn_tune=_Null, cudnn_off=_Null, layout=_Null, out=None, name=None, **kwargs)¶

Computes 2D transposed convolution (aka fractionally strided convolution) of the input tensor. This operation can be seen as the gradient of Convolution operation with respect to its input. Convolution usually reduces the size of the input. Transposed convolution works the other way, going from a smaller input to a larger output while preserving the connectivity pattern.

Parameters:	data (NDArray) – Input tensor to the deconvolution operation. weight (NDArray) – Weights representing the kernel. bias (NDArray) – Bias added to the result after the deconvolution operation. kernel (Shape(tuple), required) – Deconvolution kernel size: (h, w) or (d, h, w). This is same as the kernel size used for the corresponding convolution stride (Shape(tuple), optional, default=()) – The stride used for the corresponding convolution: (h, w) or (d, h, w). dilate (Shape(tuple), optional, default=()) – Dilation factor for each dimension of the input: (h, w) or (d, h, w). pad (Shape(tuple), optional, default=()) – The amount of implicit zero padding added during convolution for each dimension of the input: (h, w) or (d, h, w). (kernel-1)/2 is usually a good choice. If target_shape is set, pad will be ignored and a padding that will generate the target shape will be used. adj (Shape(tuple), optional, default=()) – Adjustment for output shape: (h, w) or (d, h, w). If target_shape is set, adj will be ignored and computed accordingly. target_shape (Shape(tuple), optional, default=()) – Shape of the output tensor: (h, w) or (d, h, w). num_filter (int (non-negative), required) – Number of output filters. num_group (int (non-negative), optional, default=1) – Number of groups partition. workspace (long (non-negative), optional, default=512) – Maximum temporal workspace allowed for deconvolution (MB). no_bias (boolean, optional, default=1) – Whether to disable bias parameter. cudnn_tune ({None, 'fastest', 'limited_workspace', 'off'},optional, default='None') – Whether to pick convolution algorithm by running performance test. cudnn_off (boolean, optional, default=0) – Turn off cudnn for this layer. layout ({None, 'NCDHW', 'NCHW', 'NCW', 'NDHWC', 'NHWC'},optional, default='None') – Set layout for input, output and weight. Empty for default layout, NCW for 1d, NCHW for 2d and NCDHW for 3d. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.Dropout(data=None, p=_Null, mode=_Null, out=None, name=None, **kwargs)¶

Applies dropout operation to input array.

• During training, each element of the input is set to zero with probability p. The whole array is rescaled by \(1/(1-p)\) to keep the expected sum of the input unchanged.
• During testing, this operator does not change the input if mode is ‘training’. If mode is ‘always’, the same computaion as during training will be applied.

Example:

random.seed(998)
input_array = array([[3., 0.5,  -0.5,  2., 7.],
                    [2., -0.4,   7.,  3., 0.2]])
a = symbol.Variable('a')
dropout = symbol.Dropout(a, p = 0.2)
executor = dropout.simple_bind(a = input_array.shape)

## If training
executor.forward(is_train = True, a = input_array)
executor.outputs
[[ 3.75   0.625 -0.     2.5    8.75 ]
 [ 2.5   -0.5    8.75   3.75   0.   ]]

## If testing
executor.forward(is_train = False, a = input_array)
executor.outputs
[[ 3.     0.5   -0.5    2.     7.   ]
 [ 2.    -0.4    7.     3.     0.2  ]]

Defined in src/operator/dropout.cc:L77

Parameters:	data (NDArray) – Input array to which dropout will be applied. p (float, optional, default=0.5) – Fraction of the input that gets dropped out during training time. mode ({'always', 'training'},optional, default='training') – Whether to only turn on dropout during training or to also turn on for inference. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays