Overview¶

This document lists the routines of the n-dimensional array package:

The NDArray API, defined in the ndarray (or simply nd) package, provides imperative tensor operations on CPU/GPU. An NDArray represents a multi-dimensional, fixed-size homogenous array.

>>> x = mx.nd.array([[1, 2, 3], [4, 5, 6]])
>>> type(x)

>>> x.shape
(2, 3)
>>> y = x + mx.nd.ones(x.shape)*3
>>> print(y.asnumpy())
[[ 4.  5.  6.]
 [ 7.  8.  9.]]
>>> z = y.as_in_context(mx.gpu(0))
>>> print(z)

A detailed tutorial is available at NDArray - Imperative tensor operations on CPU/GPU.

In the rest of this document, we first overview the methods provided by the ndarray.NDArray class, and then list other routines provided by the ndarray package.

The ndarray package provides several classes:

The NDArray class¶

Basic neural network functions¶

Array creation routines¶

Array manipulation routines¶

Mathematical functions¶

Neural network¶

API Reference¶

class mxnet.ndarray.NDArray(handle, writable=True)[source]¶

An array object representing a multidimensional, homogeneous array of fixed-size items.

__repr__()[source]¶

Returns a string representation of the array.

__add__(other)[source]¶

x.__add__(y) <=> x+y <=> mx.nd.add(x, y)

__iadd__(other)[source]¶

x.__iadd__(y) <=> x+=y

__sub__(other)[source]¶

x.__sub__(y) <=> x-y <=> mx.nd.subtract(x, y)

__isub__(other)[source]¶

x.__isub__(y) <=> x-=y

__rsub__(other)[source]¶

x.__rsub__(y) <=> y-x <=> mx.nd.subtract(y, x)

__mul__(other)[source]¶

x.__mul__(y) <=> x*y <=> mx.nd.multiply(x, y)

__neg__()[source]¶

x.__neg__(y) <=> -x

__imul__(other)[source]¶

x.__imul__(y) <=> x*=y

__div__(other)[source]¶

x.__div__(y) <=> x/y <=> mx.nd.divide(x, y)

__rdiv__(other)[source]¶

x.__rdiv__(y) <=> y/x <=> mx.nd.divide(y, x)

__idiv__(other)[source]¶

x.__rdiv__(y) <=> x/=y

__mod__(other)[source]¶

x.__mod__(y) <=> x%y <=> mx.nd.modulo(x, y)

__rmod__(other)[source]¶

x.__rmod__(y) <=> y%x <=> mx.nd.modulo(y, x)

__imod__(other)[source]¶

x.__rmod__(y) <=> x%=y

__pow__(other)[source]¶

x.__pow__(y) <=> x**y <=> mx.nd.power(x,y)

__rpow__(other)[source]¶

x.__pow__(y) <=> y**x <=> mx.nd.power(y,x)

__eq__(other)[source]¶

x.__eq__(y) <=> x==y <=> mx.nd.equal(x, y)

__hash__()[source]¶

Default hash function.

__ne__(other)[source]¶

x.__ne__(y) <=> x!=y <=> mx.nd.not_equal(x, y)

__gt__(other)[source]¶

x.__gt__(y) <=> x>y <=> mx.nd.greater(x, y)

__ge__(other)[source]¶

x.__ge__(y) <=> x>=y <=> mx.nd.greater_equal(x, y)

__lt__(other)[source]¶

x.__lt__(y) <=> x mx.nd.lesser(x, y)

__le__(other)[source]¶

x.__le__(y) <=> x<=y <=> mx.nd.less_equal(x, y)

__len__()[source]¶

Number of element along the first axis.

__setitem__(key, value)[source]¶

x.__setitem__(i, y) <=> x[i]=y

Sets value to self[key]. This functions supports advanced indexing defined in the following reference with some restrictions.

https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.indexing.html#combining-advanced-and-basic-indexing

• If key is a list type, only a list of integers is supported, e.g. key=[1, 2] is supported, while not for key=[[1, 2]].
• Ellipsis (...) and np.newaxis are not supported.
• Boolean array indexing is not supported.

Parameters:	key (int, mxnet.ndarray.slice, list, np.ndarray, NDArray, or tuple of all previous types) – The indexing key. value (scalar or array-like object that can be broadcast to the shape of self[key]) – The value to set.

Examples

>>> x = mx.nd.zeros((2,3))
>>> x[:] = 1
>>> x.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> x.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> x[:,1:2] = 2
>>> x.asnumpy()
array([[ 1.,  2.,  1.],
       [ 1.,  2.,  1.]], dtype=float32)
>>> x[1:2,1:] = 3
>>> x.asnumpy()
array([[ 1.,  2.,  1.],
       [ 1.,  3.,  3.]], dtype=float32)
>>> x[1:,0:2] = mx.nd.zeros((1,2))
>>> x.asnumpy()
array([[ 1.,  2.,  1.],
       [ 0.,  0.,  3.]], dtype=float32)
>>> x[1,2] = 4
>>> x.asnumpy()
array([[ 1.,  2.,  1.],
       [ 0.,  0.,  4.]], dtype=float32)
>>> x[[0], [1, 2]] = 5
>>> x.asnumpy()
array([[ 1.,  5.,  5.],
       [ 0.,  0.,  4.]], dtype=float32)
>>> x[::-1, 0:2:2] = [6]
>>> x.asnumpy()
array([[ 6.,  5.,  5.],
       [ 6.,  0.,  4.]], dtype=float32)

__getitem__(key)[source]¶

x.__getitem__(i) <=> x[i]

Returns a sliced view of this array if the elements fetched are contiguous in memory; otherwise, returns a newly created NDArray. This functions supports advanced indexing defined in the following reference with some restrictions.

https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.indexing.html#combining-advanced-and-basic-indexing

• If key is a list type, only a list of integers is supported, e.g. key=[1, 2] is supported, while not for key=[[1, 2]].
• Ellipsis (...) and np.newaxis are not supported.
• Boolean array indexing is not supported.

Parameters:	key (int, mxnet.ndarray.slice, list, np.ndarray, NDArray, or tuple of all previous types) – Indexing key.

Examples

>>> x = mx.nd.arange(0,6).reshape((2,3))
>>> x.asnumpy()
array([[ 0.,  1.,  2.],
       [ 3.,  4.,  5.]], dtype=float32)
>>> x[1].asnumpy()
array([ 3.,  4.,  5.], dtype=float32)
>>> y = x[0:1]
>>> y[:] = 2
>>> x.asnumpy()
array([[ 2.,  2.,  2.],
       [ 3.,  4.,  5.]], dtype=float32)
>>> x = mx.nd.arange(0, 8, dtype='int32').reshape((2, 2, 2))
>>> x[[0, 1]]
[[[0 1]
  [2 3]]
 [[4 5]
  [6 7]]]
>>> x[1:, [0, 1]]
[[[4 5]
  [6 7]]]
>>> y = np.array([0, 1], dtype='int32')
>>> x[1:, y]
[[[4 5]
  [6 7]]]
>>> y = mx.nd.array([0, 1], dtype='int32')
>>> x[1:, y]
[[[4 5]
  [6 7]]]

reshape(*shape, **kwargs)[source]¶

Returns a view of this array with a new shape without altering any data.

Parameters:	shape (tuple of int, or n ints) – The new shape should not change the array size, namely np.prod(new_shape) should be equal to np.prod(self.shape). Some dimensions of the shape can take special values from the set {0, -1, -2, -3, -4}. The significance of each is explained below: 0 copy this dimension from the input to the output shape. Example: - input shape = (2,3,4), shape = (4,0,2), output shape = (4,3,2) - input shape = (2,3,4), shape = (2,0,0), output shape = (2,3,4) -1 infers the dimension of the output shape by using the remainder of the input dimensions keeping the size of the new array same as that of the input array. At most one dimension of shape can be -1. Example: - input shape = (2,3,4), shape = (6,1,-1), output shape = (6,1,4) - input shape = (2,3,4), shape = (3,-1,8), output shape = (3,1,8) - input shape = (2,3,4), shape=(-1,), output shape = (24,) -2 copy all/remainder of the input dimensions to the output shape. Example: - input shape = (2,3,4), shape = (-2,), output shape = (2,3,4) - input shape = (2,3,4), shape = (2,-2), output shape = (2,3,4) - input shape = (2,3,4), shape = (-2,1,1), output shape = (2,3,4,1,1) -3 use the product of two consecutive dimensions of the input shape as the output dimension. Example: - input shape = (2,3,4), shape = (-3,4), output shape = (6,4) - input shape = (2,3,4,5), shape = (-3,-3), output shape = (6,20) - input shape = (2,3,4), shape = (0,-3), output shape = (2,12) - input shape = (2,3,4), shape = (-3,-2), output shape = (6,4) -4 split one dimension of the input into two dimensions passed subsequent to -4 in shape (can contain -1). Example: - input shape = (2,3,4), shape = (-4,1,2,-2), output shape =(1,2,3,4) - input shape = (2,3,4), shape = (2,-4,-1,3,-2), output shape = (2,1,3,4) If the argument reverse is set to 1, then the special values are inferred from right to left. Example: - without reverse=1, for input shape = (10,5,4), shape = (-1,0), output shape would be (40,5). - with reverse=1, output shape will be (50,4). reverse (bool, default False) – If true then the special values are inferred from right to left. Only supported as keyword argument.
Returns:	An array with desired shape that shares data with this array.
Return type:	NDArray

Examples

>>> x = mx.nd.arange(0,6).reshape(2,3)
>>> x.asnumpy()
array([[ 0.,  1.,  2.],
       [ 3.,  4.,  5.]], dtype=float32)
>>> y = x.reshape(3,2)
>>> y.asnumpy()
array([[ 0.,  1.],
       [ 2.,  3.],
       [ 4.,  5.]], dtype=float32)
>>> y = x.reshape(3,-1)
>>> y.asnumpy()
array([[ 0.,  1.],
       [ 2.,  3.],
       [ 4.,  5.]], dtype=float32)
>>> y = x.reshape(3,2)
>>> y.asnumpy()
array([[ 0.,  1.],
       [ 2.,  3.],
       [ 4.,  5.]], dtype=float32)
>>> y = x.reshape(-3)
>>> y.asnumpy()
array([ 0.  1.  2.  3.  4.  5.], dtype=float32)
>>> y[:] = -1
>>> x.asnumpy()
array([[-1., -1., -1.],
       [-1., -1., -1.]], dtype=float32)

reshape_like(*args, **kwargs)[source]¶

Convenience fluent method for reshape_like().

The arguments are the same as for reshape_like(), with this array as data.

zeros_like(*args, **kwargs)[source]¶

Convenience fluent method for zeros_like().

The arguments are the same as for zeros_like(), with this array as data.

ones_like(*args, **kwargs)[source]¶

Convenience fluent method for ones_like().

The arguments are the same as for ones_like(), with this array as data.

broadcast_axes(*args, **kwargs)[source]¶

Convenience fluent method for broadcast_axes().

The arguments are the same as for broadcast_axes(), with this array as data.

repeat(*args, **kwargs)[source]¶

Convenience fluent method for repeat().

The arguments are the same as for repeat(), with this array as data.

pad(*args, **kwargs)[source]¶

Convenience fluent method for pad().

The arguments are the same as for pad(), with this array as data.

swapaxes(*args, **kwargs)[source]¶

Convenience fluent method for swapaxes().

The arguments are the same as for swapaxes(), with this array as data.

split(*args, **kwargs)[source]¶

Convenience fluent method for split().

The arguments are the same as for split(), with this array as data.

slice(*args, **kwargs)[source]¶

Convenience fluent method for slice().

The arguments are the same as for slice(), with this array as data.

slice_axis(*args, **kwargs)[source]¶

Convenience fluent method for slice_axis().

The arguments are the same as for slice_axis(), with this array as data.

slice_like(*args, **kwargs)[source]¶

Convenience fluent method for slice_like().

The arguments are the same as for slice_like(), with this array as data.

take(*args, **kwargs)[source]¶

Convenience fluent method for take().

The arguments are the same as for take(), with this array as data.

one_hot(*args, **kwargs)[source]¶

Convenience fluent method for one_hot().

The arguments are the same as for one_hot(), with this array as data.

pick(*args, **kwargs)[source]¶

Convenience fluent method for pick().

The arguments are the same as for pick(), with this array as data.

sort(*args, **kwargs)[source]¶

Convenience fluent method for sort().

The arguments are the same as for sort(), with this array as data.

topk(*args, **kwargs)[source]¶

Convenience fluent method for topk().

The arguments are the same as for topk(), with this array as data.

argsort(*args, **kwargs)[source]¶

Convenience fluent method for argsort().

The arguments are the same as for argsort(), with this array as data.

argmax(*args, **kwargs)[source]¶

Convenience fluent method for argmax().

The arguments are the same as for argmax(), with this array as data.

argmax_channel(*args, **kwargs)[source]¶

Convenience fluent method for argmax_channel().

The arguments are the same as for argmax_channel(), with this array as data.

argmin(*args, **kwargs)[source]¶

Convenience fluent method for argmin().

The arguments are the same as for argmin(), with this array as data.

clip(*args, **kwargs)[source]¶

Convenience fluent method for clip().

The arguments are the same as for clip(), with this array as data.

abs(*args, **kwargs)[source]¶

Convenience fluent method for abs().

The arguments are the same as for abs(), with this array as data.

sign(*args, **kwargs)[source]¶

Convenience fluent method for sign().

The arguments are the same as for sign(), with this array as data.

flatten(*args, **kwargs)[source]¶

Convenience fluent method for flatten().

The arguments are the same as for flatten(), with this array as data.

shape_array(*args, **kwargs)[source]¶

Convenience fluent method for shape_array().

The arguments are the same as for shape_array(), with this array as data.

size_array(*args, **kwargs)[source]¶

Convenience fluent method for size_array().

The arguments are the same as for size_array(), with this array as data.

expand_dims(*args, **kwargs)[source]¶

Convenience fluent method for expand_dims().

The arguments are the same as for expand_dims(), with this array as data.

tile(*args, **kwargs)[source]¶

Convenience fluent method for tile().

The arguments are the same as for tile(), with this array as data.

transpose(*args, **kwargs)[source]¶

Convenience fluent method for transpose().

The arguments are the same as for transpose(), with this array as data.

flip(*args, **kwargs)[source]¶

Convenience fluent method for flip().

The arguments are the same as for flip(), with this array as data.

depth_to_space(*args, **kwargs)[source]¶

Convenience fluent method for depth_to_space().

The arguments are the same as for depth_to_space(), with this array as data.

space_to_depth(*args, **kwargs)[source]¶

Convenience fluent method for space_to_depth().

The arguments are the same as for space_to_depth(), with this array as data.

diag(k=0, **kwargs)[source]¶

Convenience fluent method for diag().

The arguments are the same as for diag(), with this array as data.

sum(*args, **kwargs)[source]¶

Convenience fluent method for sum().

The arguments are the same as for sum(), with this array as data.

nansum(*args, **kwargs)[source]¶

Convenience fluent method for nansum().

The arguments are the same as for nansum(), with this array as data.

prod(*args, **kwargs)[source]¶

Convenience fluent method for prod().

The arguments are the same as for prod(), with this array as data.

nanprod(*args, **kwargs)[source]¶

Convenience fluent method for nanprod().

The arguments are the same as for nanprod(), with this array as data.

mean(*args, **kwargs)[source]¶

Convenience fluent method for mean().

The arguments are the same as for mean(), with this array as data.

max(*args, **kwargs)[source]¶

Convenience fluent method for max().

The arguments are the same as for max(), with this array as data.

min(*args, **kwargs)[source]¶

Convenience fluent method for min().

The arguments are the same as for min(), with this array as data.

norm(*args, **kwargs)[source]¶

Convenience fluent method for norm().

The arguments are the same as for norm(), with this array as data.

round(*args, **kwargs)[source]¶

Convenience fluent method for round().

The arguments are the same as for round(), with this array as data.

rint(*args, **kwargs)[source]¶

Convenience fluent method for rint().

The arguments are the same as for rint(), with this array as data.

fix(*args, **kwargs)[source]¶

Convenience fluent method for fix().

The arguments are the same as for fix(), with this array as data.

floor(*args, **kwargs)[source]¶

Convenience fluent method for floor().

The arguments are the same as for floor(), with this array as data.

ceil(*args, **kwargs)[source]¶

Convenience fluent method for ceil().

The arguments are the same as for ceil(), with this array as data.

trunc(*args, **kwargs)[source]¶

Convenience fluent method for trunc().

The arguments are the same as for trunc(), with this array as data.

sin(*args, **kwargs)[source]¶

Convenience fluent method for sin().

The arguments are the same as for sin(), with this array as data.

cos(*args, **kwargs)[source]¶

Convenience fluent method for cos().

The arguments are the same as for cos(), with this array as data.

tan(*args, **kwargs)[source]¶

Convenience fluent method for tan().

The arguments are the same as for tan(), with this array as data.

arcsin(*args, **kwargs)[source]¶

Convenience fluent method for arcsin().

The arguments are the same as for arcsin(), with this array as data.

arccos(*args, **kwargs)[source]¶

Convenience fluent method for arccos().

The arguments are the same as for arccos(), with this array as data.

arctan(*args, **kwargs)[source]¶

Convenience fluent method for arctan().

The arguments are the same as for arctan(), with this array as data.

degrees(*args, **kwargs)[source]¶

Convenience fluent method for degrees().

The arguments are the same as for degrees(), with this array as data.

radians(*args, **kwargs)[source]¶

Convenience fluent method for radians().

The arguments are the same as for radians(), with this array as data.

sinh(*args, **kwargs)[source]¶

Convenience fluent method for sinh().

The arguments are the same as for sinh(), with this array as data.

cosh(*args, **kwargs)[source]¶

Convenience fluent method for cosh().

The arguments are the same as for cosh(), with this array as data.

tanh(*args, **kwargs)[source]¶

Convenience fluent method for tanh().

The arguments are the same as for tanh(), with this array as data.

arcsinh(*args, **kwargs)[source]¶

Convenience fluent method for arcsinh().

The arguments are the same as for arcsinh(), with this array as data.

arccosh(*args, **kwargs)[source]¶

Convenience fluent method for arccosh().

The arguments are the same as for arccosh(), with this array as data.

arctanh(*args, **kwargs)[source]¶

Convenience fluent method for arctanh().

The arguments are the same as for arctanh(), with this array as data.

exp(*args, **kwargs)[source]¶

Convenience fluent method for exp().

The arguments are the same as for exp(), with this array as data.

expm1(*args, **kwargs)[source]¶

Convenience fluent method for expm1().

The arguments are the same as for expm1(), with this array as data.

log(*args, **kwargs)[source]¶

Convenience fluent method for log().

The arguments are the same as for log(), with this array as data.

log10(*args, **kwargs)[source]¶

Convenience fluent method for log10().

The arguments are the same as for log10(), with this array as data.

log2(*args, **kwargs)[source]¶

Convenience fluent method for log2().

The arguments are the same as for log2(), with this array as data.

log1p(*args, **kwargs)[source]¶

Convenience fluent method for log1p().

The arguments are the same as for log1p(), with this array as data.

sqrt(*args, **kwargs)[source]¶

Convenience fluent method for sqrt().

The arguments are the same as for sqrt(), with this array as data.

rsqrt(*args, **kwargs)[source]¶

Convenience fluent method for rsqrt().

The arguments are the same as for rsqrt(), with this array as data.

cbrt(*args, **kwargs)[source]¶

Convenience fluent method for cbrt().

The arguments are the same as for cbrt(), with this array as data.

rcbrt(*args, **kwargs)[source]¶

Convenience fluent method for rcbrt().

The arguments are the same as for rcbrt(), with this array as data.

square(*args, **kwargs)[source]¶

Convenience fluent method for square().

The arguments are the same as for square(), with this array as data.

reciprocal(*args, **kwargs)[source]¶

Convenience fluent method for reciprocal().

The arguments are the same as for reciprocal(), with this array as data.

relu(*args, **kwargs)[source]¶

Convenience fluent method for relu().

The arguments are the same as for relu(), with this array as data.

sigmoid(*args, **kwargs)[source]¶

Convenience fluent method for sigmoid().

The arguments are the same as for sigmoid(), with this array as data.

softmax(*args, **kwargs)[source]¶

Convenience fluent method for softmax().

The arguments are the same as for softmax(), with this array as data.

log_softmax(*args, **kwargs)[source]¶

Convenience fluent method for log_softmax().

The arguments are the same as for log_softmax(), with this array as data.

softmin(*args, **kwargs)[source]¶

Convenience fluent method for softmin().

The arguments are the same as for softmin(), with this array as data.

squeeze(*args, **kwargs)[source]¶

Convenience fluent method for squeeze().

The arguments are the same as for squeeze(), with this array as data.

broadcast_to(shape)[source]¶

Broadcasts the input array to a new shape.

Broadcasting is only allowed on axes with size 1. The new shape cannot change the number of dimensions. For example, you could broadcast from shape (2, 1) to (2, 3), but not from shape (2, 3) to (2, 3, 3).

Parameters:	shape (tuple of int) – The shape of the desired array.
Returns:	A NDArray with the desired shape that is not sharing data with this array, even if the new shape is the same as self.shape.
Return type:	NDArray

Examples

>>> x = mx.nd.arange(0,3).reshape((1,3,1))
>>> x.asnumpy()
array([[[ 0.],
        [ 1.],
        [ 2.]]], dtype=float32)
>>> y = x.broadcast_to((2,3,3))
>>> y.asnumpy()
array([[[ 0.,  0.,  0.],
        [ 1.,  1.,  1.],
        [ 2.,  2.,  2.]],

       [[ 0.,  0.,  0.],
        [ 1.,  1.,  1.],
        [ 2.,  2.,  2.]]], dtype=float32)

broadcast_like(other)[source]¶

Broadcasts the input array to the shape of other.

Broadcasting is only allowed on axes with size 1. The new shape cannot change the number of dimensions. For example, you could broadcast from shape (2, 1) to (2, 3), but not from shape (2, 3) to (2, 3, 3).

Parameters:	other (NDArray) – Array with shape of the desired array.
Returns:	A NDArray with the desired shape that is not sharing data with this array, even if the new shape is the same as self.shape.
Return type:	NDArray

Examples

>>> x = mx.nd.arange(0,3).reshape((1,3,1))
>>> x.asnumpy()
array([[[ 0.],
        [ 1.],
        [ 2.]]], dtype=float32)
>>> y = x.broadcast_like(mx.nd.ones((2,3,3)))
>>> y.asnumpy()
array([[[ 0.,  0.,  0.],
        [ 1.,  1.,  1.],
        [ 2.,  2.,  2.]],

       [[ 0.,  0.,  0.],
        [ 1.,  1.,  1.],
        [ 2.,  2.,  2.]]], dtype=float32)

wait_to_read()[source]¶

Waits until all previous write operations on the current array are finished.

This method guarantees that all previous write operations that pushed into the backend engine for execution are actually finished.

Examples

>>> import time
>>> tic = time.time()
>>> a = mx.nd.ones((1000,1000))
>>> b = mx.nd.dot(a, a)
>>> print(time.time() - tic) 
0.003854036331176758
>>> b.wait_to_read()
>>> print(time.time() - tic) 
0.0893700122833252

ndim¶

Returns the number of dimensions of this array

Examples

>>> x = mx.nd.array([1, 2, 3, 4])
>>> x.ndim
1
>>> x = mx.nd.array([[1, 2], [3, 4]])
>>> x.ndim
2

shape¶

Tuple of array dimensions.

Examples

>>> x = mx.nd.array([1, 2, 3, 4])
>>> x.shape
(4L,)
>>> y = mx.nd.zeros((2, 3, 4))
>>> y.shape
(2L, 3L, 4L)

size¶

Number of elements in the array.

Equivalent to the product of the array’s dimensions.

Examples

>>> import numpy as np
>>> x = mx.nd.zeros((3, 5, 2))
>>> x.size
30
>>> np.prod(x.shape)
30

context¶

Device context of the array.

Examples

>>> x = mx.nd.array([1, 2, 3, 4])
>>> x.context
cpu(0)
>>> type(x.context)

>>> y = mx.nd.zeros((2,3), mx.gpu(0))
>>> y.context
gpu(0)

dtype¶

Data-type of the array’s elements.

Returns:	This NDArray’s data type.
Return type:	numpy.dtype

Examples

>>> x = mx.nd.zeros((2,3))
>>> x.dtype

>>> y = mx.nd.zeros((2,3), dtype='int32')
>>> y.dtype

stype¶

Storage-type of the array.

T¶

Returns a copy of the array with axes transposed.

Equivalent to mx.nd.transpose(self) except that self is returned if self.ndim < 2.

Unlike numpy.ndarray.T, this function returns a copy rather than a view of the array unless self.ndim < 2.

Examples

>>> x = mx.nd.arange(0,6).reshape((2,3))
>>> x.asnumpy()
array([[ 0.,  1.,  2.],
       [ 3.,  4.,  5.]], dtype=float32)
>>> x.T.asnumpy()
array([[ 0.,  3.],
       [ 1.,  4.],
       [ 2.,  5.]], dtype=float32)

asnumpy()[source]¶

Returns a numpy.ndarray object with value copied from this array.

Examples

>>> x = mx.nd.ones((2,3))
>>> y = x.asnumpy()
>>> type(y)

>>> y
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> z = mx.nd.ones((2,3), dtype='int32')
>>> z.asnumpy()
array([[1, 1, 1],
       [1, 1, 1]], dtype=int32)

asscalar()[source]¶

Returns a scalar whose value is copied from this array.

This function is equivalent to self.asnumpy()[0]. This NDArray must have shape (1,).

Examples

>>> x = mx.nd.ones((1,), dtype='int32')
>>> x.asscalar()
1
>>> type(x.asscalar())

astype(dtype, copy=True)[source]¶

Returns a copy of the array after casting to a specified type.

Parameters:	dtype (numpy.dtype or str) – The type of the returned array. copy (bool) – Default True. By default, astype always returns a newly allocated ndarray on the same context. If this is set to False, and the dtype requested is the same as the ndarray’s dtype, the ndarray is returned instead of a copy.
Returns:	The copied array after casting to the specified type, or the same array if copy=False and dtype is the same as the input array.
Return type:	NDArray, CSRNDArray or RowSparseNDArray

Examples

>>> x = mx.nd.zeros((2,3), dtype='float32')
>>> y = x.astype('int32')
>>> y.dtype

copyto(other)[source]¶

Copies the value of this array to another array.

If other is a NDArray object, then other.shape and self.shape should be the same. This function copies the value from self to other.

If other is a context, a new NDArray will be first created on the target context, and the value of self is copied.

Parameters:	other (NDArray or Context) – The destination array or context.
Returns:	The copied array. If other is an NDArray, then the return value and other will point to the same NDArray.
Return type:	NDArray, CSRNDArray or RowSparseNDArray

Examples

>>> x = mx.nd.ones((2,3))
>>> y = mx.nd.zeros((2,3), mx.gpu(0))
>>> z = x.copyto(y)
>>> z is y
True
>>> y.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> y.copyto(mx.gpu(0))

copy()[source]¶

Makes a copy of this NDArray, keeping the same context.

Returns:	The copied array
Return type:	NDArray, CSRNDArray or RowSparseNDArray

Examples

>>> x = mx.nd.ones((2,3))
>>> y = x.copy()
>>> y.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)

as_in_context(context)[source]¶

Returns an array on the target device with the same value as this array.

If the target context is the same as self.context, then self is returned. Otherwise, a copy is made.

Parameters:	context (Context) – The target context.
Returns:	The target array.
Return type:	NDArray, CSRNDArray or RowSparseNDArray

Examples

>>> x = mx.nd.ones((2,3))
>>> y = x.as_in_context(mx.cpu())
>>> y is x
True
>>> z = x.as_in_context(mx.gpu(0))
>>> z is x
False

attach_grad(grad_req='write', stype=None)[source]¶

Attach a gradient buffer to this NDArray, so that backward can compute gradient with respect to it.

Parameters:	grad_req ({'write', 'add', 'null'}) – How gradient will be accumulated. - ‘write’: gradient will be overwritten on every backward. - ‘add’: gradient will be added to existing value on every backward. - ‘null’: do not compute gradient for this NDArray. stype (str, optional) – The storage type of the gradient array. Defaults to the same stype of this NDArray.

grad¶

Returns gradient buffer attached to this NDArray.

detach()[source]¶

Returns a new NDArray, detached from the current graph.

backward(out_grad=None, retain_graph=False, train_mode=True)[source]¶

Compute the gradients of this NDArray w.r.t variables.

Parameters:	out_grad (NDArray, optional) – Gradient with respect to head. retain_graph (bool, optional) – Whether to retain the computaion graph for another backward pass on the same graph. By default the computaion history is cleared. train_mode (bool, optional) – Whether to compute gradient for training or inference.

tostype(stype)[source]¶

Return a copy of the array with chosen storage type.

See also

mxnet.ndarray.cast_storage()

Returns:	A copy of the array with the chosen storage stype
Return type:	NDArray, CSRNDArray or RowSparseNDArray

to_dlpack_for_read()[source]¶

Returns a reference view of NDArray that represents as DLManagedTensor until all previous write operations on the current array are finished.

Returns:	a reference view of NDArray that represents as DLManagedTensor.
Return type:	PyCapsule (the pointer of DLManagedTensor)

Examples

>>> x = mx.nd.ones((2,3))
>>> y = mx.nd.to_dlpack_for_read(x)
>>> type(y)

>>> z = mx.nd.from_dlpack(y)
>>> z
[[1. 1. 1.]
 [1. 1. 1.]]

to_dlpack_for_write()[source]¶

Returns a reference view of NDArray that represents as DLManagedTensor until all previous read/write operations on the current array are finished.

Returns:	a reference view of NDArray that represents as DLManagedTensor.
Return type:	PyCapsule (the pointer of DLManagedTensor)

Examples

>>> x = mx.nd.ones((2,3))
>>> w = mx.nd.to_dlpack_for_write(x)
>>> type(w)

>>> u = mx.nd.from_dlpack(w)
>>> u += 1
>>> x
[[2. 2. 2.]
 [2. 2. 2.]]

NDArray API of MXNet.

mxnet.ndarray.Activation(data=None, act_type=_Null, out=None, name=None, **kwargs)¶

Applies an activation function element-wise to the input.

The following activation functions are supported:

• relu: Rectified Linear Unit, \(y = max(x, 0)\)
• sigmoid: \(y = \frac{1}{1 + exp(-x)}\)
• tanh: Hyperbolic tangent, \(y = \frac{exp(x) - exp(-x)}{exp(x) + exp(-x)}\)
• softrelu: Soft ReLU, or SoftPlus, \(y = log(1 + exp(x))\)
• softsign: \(y = \frac{x}{1 + abs(x)}\)

Defined in src/operator/nn/activation.cc:L168

Parameters:	data (NDArray) – The input array. act_type ({'relu', 'sigmoid', 'softrelu', 'softsign', 'tanh'}, required) – Activation function to be applied. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.BatchNorm(data=None, gamma=None, beta=None, moving_mean=None, moving_var=None, eps=_Null, momentum=_Null, fix_gamma=_Null, use_global_stats=_Null, output_mean_var=_Null, axis=_Null, cudnn_off=_Null, out=None, name=None, **kwargs)¶

Batch normalization.

Normalizes a data batch by mean and variance, and applies a scale gamma as well as offset beta.

Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:

\[\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}\]

Then compute the normalized output, which has the same shape as input, as following:

\[out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]\]

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,). If output_mean_var is set to be true, then outputs both data_mean and the inverse of data_var, which are needed for the backward pass. Note that gradient of these two outputs are blocked.

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum)
moving_var = moving_var * momentum + data_var * (1 - momentum)

If use_global_stats is set to be true, then moving_mean and moving_var are used instead of data_mean and data_var to compute the output. It is often used during inference.

The parameter axis specifies which axis of the input shape denotes the ‘channel’ (separately normalized groups). The default is 1. Specifying -1 sets the channel axis to be the last item in the input shape.

Both gamma and beta are learnable parameters. But if fix_gamma is true, then set gamma to 1 and its gradient to 0.

Note

When fix_gamma is set to True, no sparse support is provided. If fix_gamma is set to False, the sparse tensors will fallback.

Defined in src/operator/nn/batch_norm.cc:L574

Parameters:	data (NDArray) – Input data to batch normalization gamma (NDArray) – gamma array beta (NDArray) – beta array moving_mean (NDArray) – running mean of input moving_var (NDArray) – running variance of input eps (double, optional, default=0.001) – Epsilon to prevent div 0. Must be no less than CUDNN_BN_MIN_EPSILON defined in cudnn.h when using cudnn (usually 1e-5) momentum (float, optional, default=0.9) – Momentum for moving average fix_gamma (boolean, optional, default=1) – Fix gamma while training use_global_stats (boolean, optional, default=0) – Whether use global moving statistics instead of local batch-norm. This will force change batch-norm into a scale shift operator. output_mean_var (boolean, optional, default=0) – Output the mean and inverse std axis (int, optional, default='1') – Specify which shape axis the channel is specified cudnn_off (boolean, optional, default=0) – Do not select CUDNN operator, if available out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.BatchNorm_v1(data=None, gamma=None, beta=None, eps=_Null, momentum=_Null, fix_gamma=_Null, use_global_stats=_Null, output_mean_var=_Null, out=None, name=None, **kwargs)¶

Batch normalization.

This operator is DEPRECATED. Perform BatchNorm on the input.

Normalizes a data batch by mean and variance, and applies a scale gamma as well as offset beta.

Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:

\[\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}\]

Then compute the normalized output, which has the same shape as input, as following:

\[out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]\]

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,). If output_mean_var is set to be true, then outputs both data_mean and data_var as well, which are needed for the backward pass.

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum)
moving_var = moving_var * momentum + data_var * (1 - momentum)

If use_global_stats is set to be true, then moving_mean and moving_var are used instead of data_mean and data_var to compute the output. It is often used during inference.

Both gamma and beta are learnable parameters. But if fix_gamma is true, then set gamma to 1 and its gradient to 0.

There’s no sparse support for this operator, and it will exhibit problematic behavior if used with sparse tensors.

Defined in src/operator/batch_norm_v1.cc:L95

Parameters:	data (NDArray) – Input data to batch normalization gamma (NDArray) – gamma array beta (NDArray) – beta array eps (float, optional, default=0.001) – Epsilon to prevent div 0 momentum (float, optional, default=0.9) – Momentum for moving average fix_gamma (boolean, optional, default=1) – Fix gamma while training use_global_stats (boolean, optional, default=0) – Whether use global moving statistics instead of local batch-norm. This will force change batch-norm into a scale shift operator. output_mean_var (boolean, optional, default=0) – Output All,normal mean and var out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.BilinearSampler(data=None, grid=None, cudnn_off=_Null, out=None, name=None, **kwargs)¶

Applies bilinear sampling to input feature map.

Bilinear Sampling is the key of [NIPS2015] “Spatial Transformer Networks”. The usage of the operator is very similar to remap function in OpenCV, except that the operator has the backward pass.

Given \(data\) and \(grid\), then the output is computed by

\[\begin{split}x_{src} = grid[batch, 0, y_{dst}, x_{dst}] \\ y_{src} = grid[batch, 1, y_{dst}, x_{dst}] \\ output[batch, channel, y_{dst}, x_{dst}] = G(data[batch, channel, y_{src}, x_{src})\end{split}\]

\(x_{dst}\), \(y_{dst}\) enumerate all spatial locations in \(output\), and \(G()\) denotes the bilinear interpolation kernel. The out-boundary points will be padded with zeros.The shape of the output will be (data.shape[0], data.shape[1], grid.shape[2], grid.shape[3]).

The operator assumes that \(data\) has ‘NCHW’ layout and \(grid\) has been normalized to [-1, 1].

BilinearSampler often cooperates with GridGenerator which generates sampling grids for BilinearSampler. GridGenerator supports two kinds of transformation: affine and warp. If users want to design a CustomOp to manipulate \(grid\), please firstly refer to the code of GridGenerator.

Example 1:

## Zoom out data two times
data = array([[[[1, 4, 3, 6],
                [1, 8, 8, 9],
                [0, 4, 1, 5],
                [1, 0, 1, 3]]]])

affine_matrix = array([[2, 0, 0],
                       [0, 2, 0]])

affine_matrix = reshape(affine_matrix, shape=(1, 6))

grid = GridGenerator(data=affine_matrix, transform_type='affine', target_shape=(4, 4))

out = BilinearSampler(data, grid)

out
[[[[ 0,   0,     0,   0],
   [ 0,   3.5,   6.5, 0],
   [ 0,   1.25,  2.5, 0],
   [ 0,   0,     0,   0]]]

Example 2:

## shift data horizontally by -1 pixel

data = array([[[[1, 4, 3, 6],
                [1, 8, 8, 9],
                [0, 4, 1, 5],
                [1, 0, 1, 3]]]])

warp_maxtrix = array([[[[1, 1, 1, 1],
                        [1, 1, 1, 1],
                        [1, 1, 1, 1],
                        [1, 1, 1, 1]],
                       [[0, 0, 0, 0],
                        [0, 0, 0, 0],
                        [0, 0, 0, 0],
                        [0, 0, 0, 0]]]])

grid = GridGenerator(data=warp_matrix, transform_type='warp')
out = BilinearSampler(data, grid)

out
[[[[ 4,  3,  6,  0],
   [ 8,  8,  9,  0],
   [ 4,  1,  5,  0],
   [ 0,  1,  3,  0]]]

Defined in src/operator/bilinear_sampler.cc:L256

Parameters:	data (NDArray) – Input data to the BilinearsamplerOp. grid (NDArray) – Input grid to the BilinearsamplerOp.grid has two channels: x_src, y_src cudnn_off (boolean or None, optional, default=None) – whether to turn cudnn off out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.BlockGrad(data=None, out=None, name=None, **kwargs)¶

Stops gradient computation.

Stops the accumulated gradient of the inputs from flowing through this operator in the backward direction. In other words, this operator prevents the contribution of its inputs to be taken into account for computing gradients.

Example:

v1 = [1, 2]
v2 = [0, 1]
a = Variable('a')
b = Variable('b')
b_stop_grad = stop_gradient(3 * b)
loss = MakeLoss(b_stop_grad + a)

executor = loss.simple_bind(ctx=cpu(), a=(1,2), b=(1,2))
executor.forward(is_train=True, a=v1, b=v2)
executor.outputs
[ 1.  5.]

executor.backward()
executor.grad_arrays
[ 0.  0.]
[ 1.  1.]

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L267

Parameters:	data (NDArray) – The input array. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays

mxnet.ndarray.CTCLoss(data=None, label=None, data_lengths=None, label_lengths=None, use_data_lengths=_Null, use_label_lengths=_Null, blank_label=_Null, out=None, name=None, **kwargs)¶

Connectionist Temporal Classification Loss.

Note

The existing alias contrib_CTCLoss is deprecated.

The shapes of the inputs and outputs:

• data: (sequence_length, batch_size, alphabet_size)
• label: (batch_size, label_sequence_length)
• out: (batch_size)

The data tensor consists of sequences of activation vectors (without applying softmax), with i-th channel in the last dimension corresponding to i-th label for i between 0 and alphabet_size-1 (i.e always 0-indexed). Alphabet size should include one additional value reserved for blank label. When blank_label is "first", the 0-th channel is be reserved for activation of blank label, or otherwise if it is “last”, (alphabet_size-1)-th channel should be reserved for blank label.

label is an index matrix of integers. When blank_label is "first", the value 0 is then reserved for blank label, and should not be passed in this matrix. Otherwise, when blank_label is "last", the value (alphabet_size-1) is reserved for blank label.

If a sequence of labels is shorter than label_sequence_length, use the special padding value at the end of the sequence to conform it to the correct length. The padding value is 0 when blank_label is "first", and -1 otherwise.

For example, suppose the vocabulary is [a, b, c], and in one batch we have three sequences ‘ba’, ‘cbb’, and ‘abac’. When blank_label is "first", we can index the labels as {‘a’: 1, ‘b’: 2, ‘c’: 3}, and we reserve the 0-th channel for blank label in data tensor. The resulting label tensor should be padded to be:

[[2, 1, 0, 0], [3, 2, 2, 0], [1, 2, 1, 3]]

When blank_label is "last", we can index the labels as {‘a’: 0, ‘b’: 1, ‘c’: 2}, and we reserve the channel index 3 for blank label in data tensor. The resulting label tensor should be padded to be:

[[1, 0, -1, -1], [2, 1, 1, -1], [0, 1, 0, 2]]

out is a list of CTC loss values, one per example in the batch.

See Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks, A. Graves et al. for more information on the definition and the algorithm.

Defined in src/operator/nn/ctc_loss.cc:L100

Parameters:	data (NDArray) – Input ndarray label (NDArray) – Ground-truth labels for the loss. data_lengths (NDArray) – Lengths of data for each of the samples. Only required when use_data_lengths is true. label_lengths (NDArray) – Lengths of labels for each of the samples. Only required when use_label_lengths is true. use_data_lengths (boolean, optional, default=0) – Whether the data lenghts are decided by data_lengths. If false, the lengths are equal to the max sequence length. use_label_lengths (boolean, optional, default=0) – Whether the label lenghts are decided by label_lengths, or derived from padding_mask. If false, the lengths are derived from the first occurrence of the value of padding_mask. The value of padding_mask is 0 when first CTC label is reserved for blank, and -1 when last label is reserved for blank. See blank_label. blank_label ({'first', 'last'},optional, default='first') – Set the label that is reserved for blank label.If “first”, 0-th label is reserved, and label values for tokens in the vocabulary are between 1 and alphabet_size-1, and the padding mask is -1. If “last”, last label value alphabet_size-1 is reserved for blank label instead, and label values for tokens in the vocabulary are between 0 and alphabet_size-2, and the padding mask is 0. out (NDArray, optional) – The output NDArray to hold the result.
Returns:	out – The output of this function.
Return type:	NDArray or list of NDArrays